skip to content

With Git LFS, how does the client find its LFS server and use the Batch API to move an object?

level: seniorimportance: nice to knowfreq 12%

answer

  1. remote URL plus a fixed suffix
  2. config that overrides the guess
  3. one POST per batch of objects
  4. the server hands back URLs

basics

~10 s

Git LFS appends .git/info/lfs to the remote URL unless lfs.url or remote.<name>.lfsurl overrides it, then POSTs to <endpoint>/objects/batch naming the operation and objects; the response gives each object a URL for the actual transfer.

solid answer

~40 s

Without configuration, Git LFS guesses the endpoint: `https://git.example.com/acme/game.git/info/lfs` for both the HTTPS and SSH forms of the remote. `lfs.url` (any remote) or `remote.<name>.lfsurl` (one remote) override it, and a committed `.lfsconfig` can carry them, with local Git config still winning. For SSH remotes on the hybrid scheme (SSH for authentication, HTTPS for data) the client runs `git-lfs-authenticate` to get headers or a different `href`; since Git LFS 3.8, if the server lacks that command, it falls back to the guessed HTTPS endpoint and Git credentials. The transfer is a `POST` to `<endpoint>/objects/batch` with `operation` (`download` or `upload`), `transfers` (normally `basic`), `objects` (`oid`, `size`) and optional `hash_algo`. The response returns per-object `actions` with an `href`, headers and expiry, and the client GETs or PUTs the bytes there. `git lfs env` shows the resolved endpoint.

code

json · 9 lines
json
{
  "operation": "upload",
  "transfers": ["basic"],
  "ref": {"name": "refs/heads/main"},
  "objects": [
    {"oid": "<64 hex characters>", "size": 524288000}
  ],
  "hash_algo": "sha256"
}

go deeper

for a junior

Recall that LFS content is stored on an LFS server separate from Git's own objects, and that git lfs env shows which endpoint a clone uses.

for a middle

Explain how the endpoint is guessed from the remote URL and how lfs.url, remote.<name>.lfsurl and .lfsconfig override it.

for a senior

Walk through a Batch API exchange, read its error codes, and diagnose SSH-only setups, including the 3.8 fallback when git-lfs-authenticate is missing.

for a principal

Decide when to run LFS storage on a separate server or storage service, and what that means for authentication, client configuration and failure modes.

## Where the objects actually live A Git LFS repository talks to two services. The **Git remote** stores commits, trees and the small **pointer files**. The **LFS server** stores the large objects themselves and hands out transfer URLs for them. The two are often on the same host, but they need not be, and the LFS server frequently redirects the actual bytes to a separate storage service. ## Finding the endpoint The client works out the LFS endpoint in this order: 1. **Guess from the remote.** With no configuration, Git LFS appends `.git/info/lfs` to the remote URL (without doubling `.git`). An SSH remote is turned into the matching HTTPS URL. 2. **Explicit configuration.** `lfs.url` sets one endpoint regardless of remote; `remote.<name>.lfsurl` sets it for one remote; `lfs.pushurl` and `remote.<name>.lfspushurl` override it for uploads. 3. **A committed `.lfsconfig`.** The same keys can be committed so every clone uses a custom server. Settings in Git config files override `.lfsconfig`, so a developer can still point a local clone elsewhere. 4. **SSH authentication.** For an SSH remote, Git LFS by default first tries the pure SSH transfer protocol (controlled by `lfs.sshtransfer`) and otherwise uses the older hybrid scheme: it runs `ssh git@host git-lfs-authenticate <path> <operation>`, and the JSON reply can add headers (a short-lived token) and even a different `href` for the server, while the data itself goes over HTTPS. | Git remote | Guessed LFS endpoint | |---|---| | `https://git.example.com/acme/game` | `https://git.example.com/acme/game.git/info/lfs` | | `https://git.example.com/acme/game.git` | `https://git.example.com/acme/game.git/info/lfs` | | `[email protected]:acme/game.git` | `https://git.example.com/acme/game.git/info/lfs` | **Git LFS 3.8** changed the SSH case: if `git-lfs-authenticate` is not available on the server (the remote command exits with status 127 or reports that the command was not found), the client now falls back to the guessed HTTPS endpoint and authenticates with the normal Git credential helper. Any other non-zero exit is still fatal. `git lfs env` prints the endpoint each remote resolves to. ## The Batch API Every transfer starts with a **Batch API** call: a `POST` to `<endpoint>/objects/batch` with the `application/vnd.git-lfs+json` media type in `Accept` and `Content-Type`. The body names: - `operation`: `download` or `upload`; - `transfers`: the transfer adapters the client supports, normally `basic` (assumed if omitted); - `objects`: each object's `oid` (its SHA-256) and `size`; - optional `ref` (used for ref-aware authorisation) and `hash_algo` (defaults to `sha256`). The response lists each object with `actions`. A download gets a `download` action; an upload gets `upload` and possibly `verify`; an object the server already has comes back with no actions, so the client skips it. Each action carries an `href`, optional `header` values and an expiry (`expires_in` or `expires_at`). With the **basic** adapter, the client then performs plain HTTP GETs or PUTs against those URLs. The batch call never carries file contents itself. ```json { "operation": "download", "transfers": ["basic"], "objects": [{"oid": "<64 hex characters>", "size": 1048576}], "hash_algo": "sha256" } ``` ## Errors the API defines | Status | Meaning | |---|---| | 401 | credentials needed; the client looks them up and retries | | 403 | read but not write access (upload only) | | 404 | repository not found for this user | | 413 | too many objects in one batch | | 429 | rate limited; the client honours `Retry-After` | | 507 / 509 | optional: storage capacity / bandwidth limit exceeded | Per-object errors (404 missing, 410 removed by the owner, 422 invalid) come back inside a 200 response. ## Diagnosing a failing transfer - **Check the endpoint first.** `git lfs env` shows the endpoint for each remote; a stale `lfs.url` in `.lfsconfig` or local config can send requests to the wrong place after a repository moves. - **401 against a custom server.** When `lfs.url` points at a different host from the Git remote, the credential helper needs credentials for that host too. - **403 on upload only.** The account can read but not write; downloads keep working. - **413 Request Entity Too Large.** The server rejected the batch size; `lfs.transfer.batchSize` (default 100 objects per request) can be lowered, with care. - **Expired transfer URLs.** Each action carries an expiry (`expires_in` or `expires_at`), after which its `href` is no longer valid. - **Retries.** `lfs.transfer.maxretries` (default 8) sets how many attempts an object gets before the transfer is marked failed. ## What 3.8 added to the transfer layer - `lfs.transfer.maxretrydelay` caps the exponential backoff between retries (default 10 seconds; `0` disables delays the server did not request). A server's `Retry-After` still wins for that attempt. - `lfs.transfer.httpDownloadEncoding` selects `gzip` (default) or `zstd` for downloads, and can be scoped to one URL. - The SSH-to-HTTPS fallback described above.

  • An older client fails on an SSH remote because the server has no git-lfs-authenticate. What are the options?
    Upgrade to Git LFS 3.8, which treats a missing command (exit status 127 or a not-found message) as unsupported and falls back to the guessed HTTPS endpoint with the Git credential helper. Or set `lfs.url` to the HTTPS endpoint explicitly. Either way the client now needs HTTPS credentials for the LFS server.
  • What do 429 and 509 from the Batch API tell you, and how does the client react?
    429 means rate limiting; the client retries and honours a `Retry-After` header, which overrides its own backoff even beyond `lfs.transfer.maxretrydelay`. 509 is an optional code meaning a bandwidth limit for the user or repository is exhausted; retrying will not help, so look at the host's quota.
  • How do you ask for zstd-compressed downloads from one LFS host only?
    From Git LFS 3.8, set `lfs.transfer.httpDownloadEncoding` to `zstd` scoped to that URL, for example `git config lfs.transfer.https://lfs.example.com/.httpDownloadEncoding zstd`. Valid values are `gzip`, the default, and `zstd`; URL scoping follows Git's `http.<url>.*` matching rules.

  • LFS objects are uploaded inside the Git packfile during a normal push.
  • The LFS endpoint always has to be configured by hand with lfs.url.
  • A committed .lfsconfig overrides a developer's local Git config.
  • The Batch API request carries the file contents in its POST body.
  • Over SSH, git-lfs-authenticate is the command that transfers the objects.

Assumes Git LFS 3.8, which falls back to the guessed HTTPS endpoint when git-lfs-authenticate is unavailable over SSH and adds lfs.transfer.maxretrydelay and zstd downloads; the Batch API contract is unchanged.