Export reviews, findings, finding outcomes, validations and spend with proofline export, one page at a time, and follow the cursor to get a complete window.
Export one page#
Set PROOFLINE_TOKEN to a token with analytics:read and PROOFLINE_ORGANIZATION_ID to your organization's ID, then name a dataset and a window:
proofline export reviews --from 2026-10-01T00:00:00Z --to 2026-10-25T00:00:00Z --format csv
| Option | Value |
|---|---|
DATASET | One of the datasets below |
--from TIME | The inclusive RFC3339 start of the window |
--to TIME | The exclusive RFC3339 end of the window, at most 31 days after --from |
--format | ndjson, the default, or csv |
--group-by | The grouping of finding_outcomes, and of no other dataset |
--cursor | The continuation the previous page printed |
The command writes one page to standard output. NDJSON holds one JSON object per line. CSV repeats its column header on every page. A page holds at most 8 MiB and 5,000 rows, or 1,000 groups for finding_outcomes.
Read every page#
One invocation reads one page. The command prints the settled horizon and, when another page follows, the next cursor to standard error. Run the command again with --cursor and the same dataset, window, format and grouping, until a page prints no cursor:
proofline export reviews --from 2026-10-01T00:00:00Z --to 2026-10-25T00:00:00Z --format csv --cursor "$CURSOR"
Treat the cursor as opaque. It belongs to one export, and reusing it with a different window or grouping is refused.
A failed page exits 1. Output collected from several invocations is complete only when the last page printed no cursor; partial output is never a complete export.
The first page fixes the settled horizon five minutes before it reads, and later pages exclude records newer than it. Pages are not a snapshot: a status or completion time can change between pages. Start another export to include later changes.
Datasets#
| Dataset | Version | Window applies to |
|---|---|---|
reviews | 1 | Review creation time |
findings | 2 | Finding creation time |
finding_validations | 2 | Validation creation time |
finding_outcomes | 1 | First publication time |
spend | 1 | Debit time |
Each row names its dataset's version, and so does the export's file name.
finding_outcomes groups by repository, the default, environment, risk_domain, severity or subject_kind.
spend holds daily debit totals in microcredits. It needs your current administrator membership in the organization on GitHub, and a token restricted to repositories cannot read it.
A token restricted to repositories exports only the records of those repositories. Exporting findings also needs read access on GitHub to each repository the selected findings come from.
HTTP request#
The command sends GET /api/v1/organizations/{org_id}/exports/{dataset} with from, to and the optional format, group_by and cursor as query parameters. Export reviews in the API reference describes it.
A page is an export page only when the status is 200 and the response carries X-Proofline-Export-Version. Any other status returns a problem document, such as a 403 for a token that lacks access or a 429 with Retry-After: 60, and that body is not CSV. X-Proofline-Export-Settled-Until holds the settled horizon. X-Proofline-Export-Next-Cursor holds the cursor for the next page; read it from the headers of the same 200 response, and send it with the same dataset, window, format and grouping. The last page omits it.
A page can hold no rows. In CSV it is the column header alone, still a 200 with its version and horizon. The row count never ends an export: it is complete when a 200 page carries no cursor, and an empty first page that carries none is a complete export with no records.
This script reads every page of one window into reviews.csv. It keeps each page in a temporary directory beside the output, joins the pages without their repeated headers, and replaces reviews.csv only after the page without a cursor arrives. A refused page, a dropped connection or a truncated body stops it with a nonzero status, prints the problem document to standard error and leaves any earlier reviews.csv as it was:
#!/bin/sh
set -eu
dataset=reviews
from=2026-10-01T00:00:00Z
to=2026-10-25T00:00:00Z
out="$dataset.csv"
url="${PROOFLINE_BASE_URL:-https://proofline.sh}/api/v1/organizations/$PROOFLINE_ORGANIZATION_ID/exports/$dataset"
work=$(mktemp -d "./.$dataset-export.XXXXXX")
trap 'rm -rf "$work"' EXIT
header() {
awk -v name="$1" '{ sub(/\r$/, ""); i = index($0, ":") }
i && tolower(substr($0, 1, i - 1)) == tolower(name) { v = substr($0, i + 1); sub(/^[ \t]+/, "", v) }
END { print v }' "$work/headers"
}
page=1
cursor=
while :; do
set -- --data-urlencode "from=$from" --data-urlencode "to=$to" --data-urlencode format=csv
if [ -n "$cursor" ]; then set -- "$@" --data-urlencode "cursor=$cursor"; fi
status=$(curl -sS -G "$url" "$@" -H "Authorization: Bearer $PROOFLINE_TOKEN" \
-D "$work/headers" -o "$work/page" -w '%{http_code}') || {
echo "page $page: the transfer failed" >&2
exit 1
}
if [ "$status" != 200 ] || [ -z "$(header X-Proofline-Export-Version)" ]; then
echo "page $page: HTTP $status" >&2
cat "$work/page" >&2
echo >&2
exit 1
fi
if [ "$page" = 1 ]; then
echo "settled until $(header X-Proofline-Export-Settled-Until)" >&2
cp "$work/page" "$work/export"
else
tail -n +2 "$work/page" >> "$work/export"
fi
cursor=$(header X-Proofline-Export-Next-Cursor)
[ -n "$cursor" ] || break
page=$((page + 1))
done
mv -f "$work/export" "$out"
echo "$out: complete, $page page(s)" >&2
Change dataset, from and to for another export, and add --data-urlencode group_by=environment to the first set -- line to group finding_outcomes. The cursor belongs to that one export, so running the script again starts a new export with a new horizon. Set PROOFLINE_BASE_URL for a console other than proofline.sh.
SDKs#
The SDKs read the same pages. TypeScript's exportReviews and Python's export_reviews read one page and return it with these fields:
| Field | Value |
|---|---|
rows | The page's records as objects. A CSV page's header becomes their keys and is not a row, and CSV values stay strings. |
text | The page as the server sent it. Every CSV page starts with the column header. |
version | The dataset version from X-Proofline-Export-Version. |
settledUntil, settled_until in Python | The settled horizon, which the first page fixes for the export. |
nextCursor, next_cursor in Python | The cursor for the next page, or null on the last page. |
Join rows across pages, or keep only the first page's header when you join CSV text. A page can hold no rows and still carry a cursor, so keep reading until nextCursor or next_cursor is null.
These examples read every page of the window that the script above reads, and write reviews.ndjson only after the last page arrives. Each page uses the same dataset, window, format and grouping, and only the cursor changes.
TypeScript reads one page per call, so the loop follows the cursor itself. It stops at 100 pages and refuses a cursor that repeats:
import { writeFileSync } from "node:fs";
import { ProoflineClient } from "@proofline/sdk";
const client = new ProoflineClient({
baseUrl: "https://proofline.sh",
token: process.env.PROOFLINE_TOKEN!,
});
const path = { org_id: process.env.PROOFLINE_ORGANIZATION_ID!, dataset: "reviews" as const };
const filters = { from: "2026-10-01T00:00:00Z", to: "2026-10-25T00:00:00Z", format: "ndjson" as const };
const rows: Record<string, unknown>[] = [];
const seen = new Set([] as string[]);
let cursor: string | undefined;
let complete = false;
for (let page = 1; page <= 100 && !complete; page += 1) {
const result = await client.exportReviews({ path, query: { ...filters, cursor } });
if (page === 1) console.error(`version ${result.version}, settled until ${result.settledUntil}`);
rows.push(...result.rows);
const next = result.nextCursor;
if (next === null) complete = true;
else if (!next || seen.has(next)) throw new Error(`page ${page} repeats a cursor`);
else {
seen.add(next);
cursor = next;
}
}
if (!complete) throw new Error("the export has more than 100 pages");
writeFileSync("reviews.ndjson", rows.map((row) => `${JSON.stringify(row)}\n`).join(""));
Python's iter_export_pages follows the cursors and yields each page. It reads at most max_pages pages, 100 unless you set another bound up to 1,000. It raises ProoflineError after yielding earlier pages when a page is refused, a cursor repeats, or the bound runs out while a cursor remains. So collect the rows first and write the file after the loop ends:
import json
import os
import sys
from uuid import UUID
from proofline import ProoflineClient
rows = []
with ProoflineClient(os.environ["PROOFLINE_TOKEN"], base_url="https://proofline.sh") as client:
pages = client.iter_export_pages(
UUID(os.environ["PROOFLINE_ORGANIZATION_ID"]),
"reviews",
from_time="2026-10-01T00:00:00Z",
to_time="2026-10-25T00:00:00Z",
format="ndjson",
max_pages=100,
)
for number, page in enumerate(pages, start=1):
if number == 1:
print(f"version {page.version}, settled until {page.settled_until}", file=sys.stderr)
rows.extend(page.rows)
with open("reviews.ndjson", "w", encoding="utf-8") as out:
out.writelines(json.dumps(row) + "\n" for row in rows)
Both SDKs raise ProoflineError for a refused page or a failed transfer, with the problem's status, type and detail. Python's iterator also raises it for a repeated cursor, with the type urn:proofline:sdk:invalid-pagination, and when its bound runs out, with urn:proofline:sdk:pagination-limit; the TypeScript loop throws its own errors for those two. In every case the example stops before it writes the file, and the rows it read are not a complete export. Running it again starts a new export with a new horizon.