What does a Python package index serve through the simple repository API, and how do its HTML and JSON forms differ?
answer
- one page per project
- normalized names, trailing slash
- hash in the link fragment
- data- attributes vs JSON keys
- Accept header picks the form
basics
~20 sA Python package index serves a project list and one page per normalized project name listing each file's URL and hash; the HTML form (PEP 503) uses links with data- attributes, the JSON form (PEP 691) the same facts as keys.
solid answer
~40 sThe API is rooted at a base URL; PyPI's is `https://pypi.org/simple/`. The root lists projects, and `/<project>/` (the normalized name: lowercased, runs of `.`, `-` and `_` collapsed to `-`, with a trailing slash) lists one entry per distribution file, wheels and sdists alike. In HTML each file is an anchor whose `href` should carry a `#sha256=...` fragment, with optional `data-requires-python`, `data-yanked` and `data-core-metadata` attributes. The JSON form lives at the same URLs and is chosen through the `Accept` header with `application/vnd.pypi.simple.v1+json`; it carries `hashes`, `requires-python`, `yanked` and `core-metadata` as keys, plus a mandatory `size`, an optional `upload-time`, a `versions` list and `meta.api-version`. The HTML form is now declared frozen, so new features land in JSON. An installer reads this page, filters files by name, tags and `requires-python`, and downloads from each file's URL.
code
json · 23 lines{
"meta": {"api-version": "1.4"},
"name": "holygrail",
"files": [
{
"filename": "holygrail-1.0.tar.gz",
"url": "https://example.com/files/holygrail-1.0.tar.gz",
"hashes": {"sha256": "..."},
"requires-python": ">=3.7",
"yanked": "Had a vulnerability",
"size": 123456
},
{
"filename": "holygrail-1.0-py3-none-any.whl",
"url": "https://example.com/files/holygrail-1.0-py3-none-any.whl",
"hashes": {"sha256": "..."},
"requires-python": ">=3.7",
"core-metadata": true,
"size": 1337
}
],
"versions": ["1.0"]
}go deeper
Recall that a Python package index serves one page per project listing every downloadable file, and that names are normalized in the URL.
Explain the per-file facts, hash, requires-python, yanked and core metadata, and how the same URL serves HTML or JSON through the Accept header.
Diagnose installs that pick the wrong file by reading the project page the client received, checking for missing hashes, stripped requires-python or yanked flags, or a proxy that ignores the Accept header.
When standing up an internal index or proxy, require the JSON form and faithful upstream metadata, since installers' choices depend on fields the HTML form no longer gains.
## Base URL and the two pages The **simple repository API** is how a Python package index tells installers which files exist. A repository is defined by its **base URL**; PyPI's is `https://pypi.org/simple/`, which is where the name "simple" comes from. Every other URL is relative to it, and there are only two kinds of page: - **The project list** at `/`, one entry per project. - **The project detail page** at `/<project>/`, one entry per distribution file of that project: every wheel and every sdist of every version. The `<project>` segment is the **normalized name**: lowercased, with every run of `.`, `-` and `_` replaced by a single `-`. A project called `Silly_Walk` lives at `/silly-walk/`. Pages must end with `/`; a repository should redirect the slashless form and may redirect unnormalized names, but clients must request the normalized URL themselves rather than rely on that redirect. ## What the project page says about each file Both forms carry the same facts; they differ in how they spell them. | Fact | HTML form (PEP 503 and later) | JSON form (PEP 691 and later) | |---|---|---| | download location | anchor `href`, text equal to the file name | `url` and `filename` | | file hash | URL fragment `#sha256=<hex>` (should be present) | `hashes` dictionary (must be present, may be empty) | | interpreter constraint | `data-requires-python`, with `<` and `>` HTML-escaped | `requires-python` | | yanked | `data-yanked`, optional reason as value | `yanked`: boolean or reason string | | separate core metadata file | `data-core-metadata` | `core-metadata` | | file size, upload time | not available | `size` (mandatory), `upload-time` (optional) | | version list | not available | `versions` | | API version | `<meta name="pypi:repository-version">` | `meta.api-version` | The spec now declares the HTML representation **frozen** and tells producers and consumers to prefer JSON, and the fields that API version 1.1 added, `size`, `upload-time` and `versions`, exist only in the JSON form. ## Choosing a form: content negotiation JSON does not live at a different path. The same URLs answer in either form, selected by HTTP content negotiation: 1. The client sends an `Accept` header listing the types it understands, such as `application/vnd.pypi.simple.v1+json`, `application/vnd.pypi.simple.v1+html` and, for legacy servers, `text/html` at a low quality value. 2. The server picks one and answers with that `Content-Type`. `text/html` is an alias for the v1 HTML type. 3. If it supports none of them, the server may answer in a default type of its choosing or return `406 Not Acceptable` (a `300 Multiple Choices` is allowed but discouraged). A repository manager or proxy in front of an index has to honour this, or JSON-capable clients silently fall back to HTML and lose `size` and `versions`. ## Yanked is not deleted **Yanking** marks a file without removing it: - The file stays listed and downloadable. - An installer must ignore yanked files whenever a non-yanked file satisfies the request, and may refuse a yanked file even when nothing else does. - The suggested exceptions are an exact `==` or `===` pin, or a lock file, that matches only the yanked file; the installer should warn when it uses one. - The flag is not permanent: a file can be unyanked and yanked again. ## File names the page lists The page lists files by name, and installers parse those names. A wheel is named `{distribution}-{version}(-{build tag})?-{python tag}-{abi tag}-{platform tag}.whl`; an sdist is `{name}-{version}.tar.gz`. A repository that renames files on upload breaks that selection. ## Debugging with the project page When an installer picks the wrong file, or none, read what it read: 1. Request `/<normalized-name>/` from the index the installer uses, sending the same `Accept` header, and note the `Content-Type` that comes back. 2. Find the file you expected; if it is absent, the upload or the proxy's cached page is the problem, not the installer. 3. Check its `requires-python`, its yanked flag and its hash against what you know about the file. 4. Check `meta.api-version` (or the HTML meta tag) to see which features the index claims to support. ## What a repository manager has to get right - Serve normalized project URLs with trailing slashes, and redirect the other spellings. - Keep the file hashes, so installers can verify downloads and hash-pinned installs keep working. - Preserve the yanked flag and `requires-python` from the upstream, or older interpreters start picking files they cannot run. - Rewrite file URLs when proxying if needed; the spec places no constraint on where files are hosted relative to the index.
- Why would an installer on an old interpreter pick an older release when a newer file is listed on the project page?The newer files carry `requires-python` (or `data-requires-python` in HTML) that the old interpreter does not satisfy, and installers should ignore such downloads. The installer falls back to the newest file whose constraint matches. If a proxy strips that attribute, the reverse can happen: the old interpreter ends up downloading a file it cannot run.
- What is the .metadata file next to a wheel on a Python package index for?When a file entry carries `core-metadata` (`data-core-metadata` in HTML), the index serves the distribution's core metadata at the file URL plus `.metadata`. An installer can read dependencies from that small file instead of downloading the whole distribution. The key was first named `dist-info-metadata` and renamed by PEP 714; clients may read the old name as a fallback.
- A yanked file is deleted from the index and returns 404.
- The JSON form lives at a separate path from the HTML pages.
- Clients may rely on the index redirecting /HolyGrail/ to /holygrail/.
- An index must host the distribution files on its own host.
- The HTML form is where new simple API features are added.