skip to content

With the GitHub CLI, what must a query contain for gh api graphql --paginate to fetch every page, and what can it not page through?

level: seniorimportance: nice to knowfreq 9%

answer

  1. a cursor gh fills in
  2. nullable on the first request
  3. hasNextPage and endCursor
  4. the first pageInfo wins
  5. one connection per query

basics

~20 s

The query must declare a nullable $endCursor: String variable, pass it as after: on the connection, and select pageInfo { hasNextPage endCursor }. gh pages only through the connection whose pageInfo it meets first; nested connections need their own queries.

solid answer

~50 s

`gh api graphql --paginate` works only if the query "accepts an `$endCursor: String` variable" and fetches `pageInfo { hasNextPage endCursor }`. gh sends the first request without the cursor, so the variable must be nullable; it then reads `endCursor` from the response, sets the `endCursor` variable and repeats while `hasNextPage` is true. It takes the cursor from the **first** `pageInfo` it finds, so only one connection is paged: if each repository also returns a nested connection with its own `pageInfo`, ask for that connection without `pageInfo` (or in a separate query), or gh may follow the wrong cursor. Other variables go in as fields (`-F org=acme`), every page is a separate JSON object (`--slurp` wraps them), `--jq` runs per page, and a response carrying an `errors` array makes gh exit non-zero even with HTTP 200.

code

bash · 12 lines
bash
gh api graphql --paginate -F org=acme -f query='
  query($org: String!, $endCursor: String) {
    organization(login: $org) {
      repositories(first: 100, after: $endCursor) {
        pageInfo { hasNextPage endCursor }
        nodes { nameWithOwner isArchived rulesets(first: 1) { totalCount } }
      }
    }
  }' \
  --jq '.data.organization.repositories.nodes[]
        | select(.isArchived | not)
        | select(.rulesets.totalCount == 0) | .nameWithOwner'

go deeper

for a junior

Recall the three required pieces: the $endCursor variable, after: on the connection, and pageInfo with hasNextPage and endCursor.

for a middle

Explain how gh loops: first request without a cursor, endCursor copied from the response, repeat while hasNextPage is true.

for a senior

Show the production traps: the first pageInfo wins, nested connections are not paged, pages print separately, and an errors array fails the command even with HTTP 200.

for a principal

Weigh GraphQL's fewer, richer requests against REST's simpler paging for a nightly organisation-wide job, and decide who maintains the query.

## Why GraphQL pagination is different REST pagination in gh follows the `next` link in the `Link` response header. GraphQL has no such header: each **connection** (a paged list field such as `repositories`) takes `first:` and `after:` arguments and returns a `pageInfo` object with `hasNextPage` and `endCursor`. To page, a client must feed the `endCursor` of one response into the `after:` argument of the next request. `gh api graphql --paginate` automates exactly that loop, but only under a contract stated in its help. ## The contract 1. **Declare `$endCursor: String`** in the operation. gh's first request does not send this variable, so it must be the nullable `String`; `String!` would make the first request invalid. 2. **Pass it to the connection** as `after: $endCursor`, with a page size such as `first: 100`. 3. **Select `pageInfo { hasNextPage endCursor }`** on that connection. After each response gh scans for `pageInfo`, and if `hasNextPage` is true it sets `endCursor` in the variables and sends the same query again. When `hasNextPage` is false, or no cursor is found, the loop ends. ## The loop, request by request For an organisation with 250 repositories and `first: 100`: 1. **Request 1** is sent with `org` set and no `endCursor`, so `after:` is null and the connection starts at the beginning. The response holds 100 nodes and `pageInfo { hasNextPage: true, endCursor: "Y3Vyc29yOjEwMA==" }`. 2. **Request 2** is the same query with `endCursor` set to that opaque string; GitHub returns nodes 101 to 200 and a new cursor. 3. **Request 3** returns the last 50 nodes with `hasNextPage: false`, and gh stops. Three pages are printed as three JSON objects, one after another. The cursor is opaque: scripts should never build or decode it, only pass it back, which is exactly what gh does for you. ## The limit: one cursor, taken from the first `pageInfo` gh does not know which connection your `$endCursor` belongs to. It walks the response JSON in order and uses the **first** `pageInfo` object it meets. Consequences: - Only **one connection per query** is paged automatically. - If an inner connection (say, each repository's issues) also selects `pageInfo` and appears earlier in the response, gh can take that cursor or stop early. - Nested lists that may exceed one page need a second query per parent, or a page size large enough, which is a design decision for the script. The safe pattern is to select `pageInfo` only on the connection you want gh to page, and to place it before `nodes` so it is met first. ## Variables, output and errors | Concern | Behaviour | |---|---| | Other variables | Every field except `query` and `operationName` becomes a GraphQL variable: `-F org=acme`, `-F first=50` (an integer), `-F owner='{owner}'` (placeholder filled) | | Output shape | Each page is printed as a separate JSON object; `--slurp` wraps them in one array | | Filtering | `--jq` runs once per page, so it should select, not count | | Errors | A response with an `errors` array is treated as a failure: gh prints the messages and exits non-zero, even when the HTTP status is 200 | The error rule matters in scripts: a typo in a field name or a permission gap produces partial data plus `errors`, and gh stops rather than letting the script use the partial page. ## Why use GraphQL here at all For the nightly "repositories without a ruleset" report, one GraphQL page can return 100 repositories **and** a count of their rulesets, replacing 100 follow-up REST calls. Whether GraphQL is the right API for a given job, and how its cost is counted against rate limits, belongs to the material on the GitHub API; the CLI's part is the pagination contract above. ## Checklist before running it nightly - `$endCursor: String`, nullable, used as `after:` on exactly one connection. - `pageInfo { hasNextPage endCursor }` on that connection only. - Inner connections asked for a count or a bounded `first:` without `pageInfo`. - `--slurp` plus an external jq when the whole result must be seen at once.

  • Why does declaring $endCursor: String! break gh api graphql --paginate?
    gh sends the first request without an `endCursor` value, because there is no cursor yet. A non-null `String!` variable must be provided, so the server rejects that first request. The nullable `String` starts the connection from the beginning.
  • How do you collect every page of a GraphQL result into one document for a total?
    Add `--slurp` to `--paginate`: gh writes each page's response object into one outer array. Because `--slurp` refuses `--jq`, pipe into an external jq, for example `[.[].data.organization.repositories.nodes[]] | length`.

It is like reading a long book with a single bookmark: gh puts the bookmark wherever the first pageInfo says and turns to it next time, so it can follow exactly one book, not the footnotes inside it.

  • gh pages every connection in the query automatically
  • Declaring $endCursor: String! is the correct variable type
  • gh paginates GraphQL by following the Link header
  • A GraphQL error with HTTP 200 still exits 0 in gh
  • pageInfo is optional because gh counts the returned nodes