skip to content

What does a Python package index serve through the simple repository API, and how do its HTML and JSON forms differ?

level: middleimportance: should knowfreq 38%

answer

  1. one page per project
  2. normalized names, trailing slash
  3. hash in the link fragment
  4. data- attributes vs JSON keys
  5. Accept header picks the form

basics

~20 s

A Python package index serves a project list and one page per normalized project name listing each file's URL and hash; the HTML form (PEP 503) uses links with data- attributes, the JSON form (PEP 691) the same facts as keys.

solid answer

~40 s

The API is rooted at a base URL; PyPI's is `https://pypi.org/simple/`. The root lists projects, and `/<project>/` (the normalized name: lowercased, runs of `.`, `-` and `_` collapsed to `-`, with a trailing slash) lists one entry per distribution file, wheels and sdists alike. In HTML each file is an anchor whose `href` should carry a `#sha256=...` fragment, with optional `data-requires-python`, `data-yanked` and `data-core-metadata` attributes. The JSON form lives at the same URLs and is chosen through the `Accept` header with `application/vnd.pypi.simple.v1+json`; it carries `hashes`, `requires-python`, `yanked` and `core-metadata` as keys, plus a mandatory `size`, an optional `upload-time`, a `versions` list and `meta.api-version`. The HTML form is now declared frozen, so new features land in JSON. An installer reads this page, filters files by name, tags and `requires-python`, and downloads from each file's URL.

code

json · 23 lines
json
{
  "meta": {"api-version": "1.4"},
  "name": "holygrail",
  "files": [
    {
      "filename": "holygrail-1.0.tar.gz",
      "url": "https://example.com/files/holygrail-1.0.tar.gz",
      "hashes": {"sha256": "..."},
      "requires-python": ">=3.7",
      "yanked": "Had a vulnerability",
      "size": 123456
    },
    {
      "filename": "holygrail-1.0-py3-none-any.whl",
      "url": "https://example.com/files/holygrail-1.0-py3-none-any.whl",
      "hashes": {"sha256": "..."},
      "requires-python": ">=3.7",
      "core-metadata": true,
      "size": 1337
    }
  ],
  "versions": ["1.0"]
}

go deeper

for a junior

Recall that a Python package index serves one page per project listing every downloadable file, and that names are normalized in the URL.

for a middle

Explain the per-file facts, hash, requires-python, yanked and core metadata, and how the same URL serves HTML or JSON through the Accept header.

for a senior

Diagnose installs that pick the wrong file by reading the project page the client received, checking for missing hashes, stripped requires-python or yanked flags, or a proxy that ignores the Accept header.

for a principal

When standing up an internal index or proxy, require the JSON form and faithful upstream metadata, since installers' choices depend on fields the HTML form no longer gains.

## Base URL and the two pages The **simple repository API** is how a Python package index tells installers which files exist. A repository is defined by its **base URL**; PyPI's is `https://pypi.org/simple/`, which is where the name "simple" comes from. Every other URL is relative to it, and there are only two kinds of page: - **The project list** at `/`, one entry per project. - **The project detail page** at `/<project>/`, one entry per distribution file of that project: every wheel and every sdist of every version. The `<project>` segment is the **normalized name**: lowercased, with every run of `.`, `-` and `_` replaced by a single `-`. A project called `Silly_Walk` lives at `/silly-walk/`. Pages must end with `/`; a repository should redirect the slashless form and may redirect unnormalized names, but clients must request the normalized URL themselves rather than rely on that redirect. ## What the project page says about each file Both forms carry the same facts; they differ in how they spell them. | Fact | HTML form (PEP 503 and later) | JSON form (PEP 691 and later) | |---|---|---| | download location | anchor `href`, text equal to the file name | `url` and `filename` | | file hash | URL fragment `#sha256=<hex>` (should be present) | `hashes` dictionary (must be present, may be empty) | | interpreter constraint | `data-requires-python`, with `<` and `>` HTML-escaped | `requires-python` | | yanked | `data-yanked`, optional reason as value | `yanked`: boolean or reason string | | separate core metadata file | `data-core-metadata` | `core-metadata` | | file size, upload time | not available | `size` (mandatory), `upload-time` (optional) | | version list | not available | `versions` | | API version | `<meta name="pypi:repository-version">` | `meta.api-version` | The spec now declares the HTML representation **frozen** and tells producers and consumers to prefer JSON, and the fields that API version 1.1 added, `size`, `upload-time` and `versions`, exist only in the JSON form. ## Choosing a form: content negotiation JSON does not live at a different path. The same URLs answer in either form, selected by HTTP content negotiation: 1. The client sends an `Accept` header listing the types it understands, such as `application/vnd.pypi.simple.v1+json`, `application/vnd.pypi.simple.v1+html` and, for legacy servers, `text/html` at a low quality value. 2. The server picks one and answers with that `Content-Type`. `text/html` is an alias for the v1 HTML type. 3. If it supports none of them, the server may answer in a default type of its choosing or return `406 Not Acceptable` (a `300 Multiple Choices` is allowed but discouraged). A repository manager or proxy in front of an index has to honour this, or JSON-capable clients silently fall back to HTML and lose `size` and `versions`. ## Yanked is not deleted **Yanking** marks a file without removing it: - The file stays listed and downloadable. - An installer must ignore yanked files whenever a non-yanked file satisfies the request, and may refuse a yanked file even when nothing else does. - The suggested exceptions are an exact `==` or `===` pin, or a lock file, that matches only the yanked file; the installer should warn when it uses one. - The flag is not permanent: a file can be unyanked and yanked again. ## File names the page lists The page lists files by name, and installers parse those names. A wheel is named `{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl`; an sdist is `{name}-{version}.tar.gz`. A repository that renames files on upload breaks that selection. ## Debugging with the project page When an installer picks the wrong file, or none, read what it read: 1. Request `/<normalized-name>/` from the index the installer uses, sending the same `Accept` header, and note the `Content-Type` that comes back. 2. Find the file you expected; if it is absent, the upload or the proxy's cached page is the problem, not the installer. 3. Check its `requires-python`, its yanked flag and its hash against what you know about the file. 4. Check `meta.api-version` (or the HTML meta tag) to see which features the index claims to support. ## What a repository manager has to get right - Serve normalized project URLs with trailing slashes, and redirect the other spellings. - Keep the file hashes, so installers can verify downloads and hash-pinned installs keep working. - Preserve the yanked flag and `requires-python` from the upstream, or older interpreters start picking files they cannot run. - Rewrite file URLs when proxying if needed; the spec places no constraint on where files are hosted relative to the index.

  • Why would an installer on an old interpreter pick an older release when a newer file is listed on the project page?
    The newer files carry `requires-python` (or `data-requires-python` in HTML) that the old interpreter does not satisfy, and installers should ignore such downloads. The installer falls back to the newest file whose constraint matches. If a proxy strips that attribute, the reverse can happen: the old interpreter ends up downloading a file it cannot run.
  • What is the .metadata file next to a wheel on a Python package index for?
    When a file entry carries `core-metadata` (`data-core-metadata` in HTML), the index serves the distribution's core metadata at the file URL plus `.metadata`. An installer can read dependencies from that small file instead of downloading the whole distribution. The key was first named `dist-info-metadata` and renamed by PEP 714; clients may read the old name as a fallback.

  • A yanked file is deleted from the index and returns 404.
  • The JSON form lives at a separate path from the HTML pages.
  • Clients may rely on the index redirecting /HolyGrail/ to /holygrail/.
  • An index must host the distribution files on its own host.
  • The HTML form is where new simple API features are added.