skip to content

What does tarfile's 'data' extraction filter block, and why is it the 3.14 default?

level: middleimportance: must knowfreq 35%

answer

  1. A PEP changed a dangerous default
  2. Three named policies, weakest to strictest
  3. The strict one is named for content
  4. 3.14 stopped trusting the archive by default
  5. OutsideDestinationError, AbsoluteLinkError, SpecialFileError

basics

~20 s

The data filter refuses tar members that would land outside the destination directory, refuses links whose target is absolute or escapes, and rejects device and FIFO members, while clearing ownership and risky permission bits. Python 3.14 makes it the default for extraction.

solid answer

~40 s

PEP 706 gave `TarFile.extract()` and `TarFile.extractall()` a keyword-only `filter` argument in Python 3.12, with three built-in policies: `fully_trusted` (the old honour-everything behaviour), `tar` (strip leading slashes, refuse anything resolving outside the destination, clear setuid/setgid/sticky and group-and-other write bits), and `data` (all of that, plus refuse device and FIFO members, refuse links whose target is absolute or resolves outside the destination, and drop uid/gid/uname/gname). In 3.12 and 3.13 omitting `filter` meant `fully_trusted` with a DeprecationWarning; **Python 3.14 makes `'data'` the default**, so the naive `extractall(dest)` call is now the safe one. A rejected member raises a `tarfile.FilterError` subclass, which aborts the extraction, so extract into a scratch directory and promote it only on success.

code

python · 20 lines
python
import io
import os
import tarfile
import tempfile

buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tar:
    payload = b"pwned"
    info = tarfile.TarInfo("../escape.txt")
    info.size = len(payload)
    tar.addfile(info, io.BytesIO(payload))

with tempfile.TemporaryDirectory() as dest:
    buf.seek(0)
    with tarfile.open(fileobj=buf) as tar:
        try:
            tar.extractall(dest)  # on 3.14 filter defaults to "data"
        except tarfile.OutsideDestinationError as exc:
            print("refused:", type(exc).__name__)
    print("destination contents:", os.listdir(dest))

go deeper

for a junior

Know that extracting an archive from a user is dangerous, and that tarfile has named safety policies you pass as filter. Be able to say that 'data' is the strict one and that Python 3.14 uses it by default.

for a middle

Be ready to list what 'data' enforces versus 'tar' versus 'fully_trusted': containment of names, refusal of absolute or escaping link targets, refusal of device and FIFO members, and the dropping of ownership and risky mode bits. Name the 3.14 default change.

for a senior

Show the operational side: a rejected member aborts extraction mid-way, so unpack into a scratch directory and promote on success. Explain what breaks when a backup-restore path is upgraded to 3.14 and why making the trust level explicit is the improvement.

for a principal

Own the policy across services: one reviewed extraction helper rather than a filter argument decided at each call site, an explicit rule for which pipelines may use 'tar' or 'fully_trusted', and a plan for the interpreters in your fleet that predate the flipped default.

## Why tar needed a filter at all A tar archive is not a bag of file contents; it is a stream of member headers, each carrying a name, a *type* (regular file, directory, symlink, hard link, character or block device, FIFO), a permission mode, and ownership metadata (`uid`, `gid`, `uname`, `gname`). `TarFile.extractall()` historically reproduced all of it faithfully — which is exactly what a backup format should do, and exactly the wrong thing to do with an archive a stranger uploaded. Three attacks follow directly from faithful reproduction: * **Escape by name.** A member named `../../etc/cron.d/job` is written above the destination. * **Escape by link.** A symlink member `cfg -> /etc/passwd`, followed by a regular member also named `cfg`, makes the second write land wherever the link points. * **Dangerous metadata.** A member arrives setuid-root, or as a device node, or owned by another user when extraction runs privileged. The escape has been reported against CPython since 2007. Rather than silently changing a documented behaviour that real backup tooling depends on, PEP 706 made the *trust decision explicit*. ## The three built-in filters `TarFile.extract()` and `TarFile.extractall()` take a keyword-only `filter` argument, either a name or a callable. Weakest to strictest: * **`'fully_trusted'` / `tarfile.fully_trusted_filter`** — the historical behaviour: honour every name, type, mode and owner in the archive. Correct only for archives you produced yourself. * **`'tar'` / `tarfile.tar_filter`** — honour most metadata, but keep the extraction inside the destination. It strips a leading separator from the member name, raises `tarfile.AbsolutePathError` if the name is *still* absolute afterwards (a Windows drive path, for instance), raises `tarfile.OutsideDestinationError` when the resolved target is not under the destination, and masks each mode down so that setuid, setgid, sticky and group/other write bits are cleared. * **`'data'` / `tarfile.data_filter`** — everything `tar` does, and then treats the archive as *data* rather than as a filesystem image. ## What `data` enforces, precisely On CPython 3.14 the `data` filter adds, on top of the containment checks: * **Special files are refused.** Anything that is not a regular file, hard link, symlink or directory raises `tarfile.SpecialFileError` — device nodes and FIFOs never land. * **Ownership is discarded.** `uid`, `gid`, `uname` and `gname` are cleared, so extraction never tries to chown anything. * **Modes are normalised.** Regular files and hard links keep an executable bit only if the owner had one, and are forced to be owner-readable and writable; directories and symlinks have their mode dropped entirely so the platform default applies. * **Link targets are checked.** An absolute `linkname` raises `tarfile.AbsoluteLinkError`. A relative one is normalised, resolved (for a symlink, relative to the member's own directory) and must land under the destination, or `tarfile.LinkOutsideDestinationError` is raised. All of these derive from `tarfile.FilterError`, so one `except tarfile.FilterError` catches the whole policy. ## The 3.14 default In 3.12 and 3.13, omitting `filter` still meant `fully_trusted`, with a `DeprecationWarning` telling you to choose. **Python 3.14 flipped the default to `'data'`.** Two consequences matter in review: 1. On 3.14 the naive call is the safe call, and code that passes `filter="data"` explicitly is now documenting intent rather than adding protection. 2. Code that legitimately needed the old behaviour — restoring a backup with real ownership, or unpacking an image containing device nodes — **breaks on upgrade** and must now say `filter="tar"` or `filter="fully_trusted"` out loud. That visibility is the point of the PEP. ## What happens when a member is rejected The filter raises a `FilterError` subclass, and with `TarFile.errorlevel` at its default of `1` that exception propagates out of `extractall()` and stops the run — members already written stay on disk. So the production shape is: extract into a fresh temporary directory, and only move the result into place if extraction returned normally; on any exception, delete the directory whole. ## The limits worth naming in an interview The filter is about **paths, types and metadata — never volume**. It will happily extract a member that declares two hundred gigabytes; capping decompressed size is a separate job. It also does not apply to `zipfile`, which has no filter API at all. And `shutil.unpack_archive()` grew its own `filter` parameter in 3.12, which it forwards to `tarfile`. ## Older interpreters Filters were added in 3.12 and backported into maintenance releases of the 3.8–3.11 branches, so `filter="data"` works on a patched 3.11 but raises `TypeError` on an unpatched build. If you must run across both, test for the feature (`hasattr(tarfile, "data_filter")`) rather than the version, or pin the policy process-wide at start-up with `tarfile.TarFile.extraction_filter = staticmethod(tarfile.data_filter)` — the `staticmethod` wrapper matters, because a plain function assigned to a class attribute would be bound as a method.

  • How do you get the same protection on Python 3.12 or 3.13, where the default is still fully trusted?
    Pass filter="data" at every extract call, or set tarfile.TarFile.extraction_filter = staticmethod(tarfile.data_filter) once during start-up so every TarFile inherits it. Filters were backported to maintenance releases of 3.8-3.11 as well, so on mixed fleets feature-test with hasattr(tarfile, "data_filter") rather than checking a version number, because an unpatched build raises TypeError on the unknown keyword.
  • When would 'tar' or 'fully_trusted' still be the right choice?
    When you are restoring an archive you produced and control, and the metadata is the payload: a system backup that must come back with its real ownership, permissions and device nodes. Use 'tar' when you want that metadata but still refuse escapes, and 'fully_trusted' only inside a pipeline where the archive is your own build artefact. Anything arriving over the network gets 'data'.
  • A member trips the filter halfway through extractall. What state is the destination in?
    Partially populated. The filter raises a FilterError subclass and, with TarFile.errorlevel at its default of 1, that propagates straight out of extractall, leaving already-written members on disk. Treat extraction as non-atomic: unpack into a fresh temporary directory, and rename or copy into the real location only after the call returns normally, deleting the scratch directory on any failure.

It is the difference between restoring a backup and opening someone's parcel: a restore is supposed to recreate owners, links and device nodes exactly, while a parcel from a stranger should yield nothing but plain files, inside the room you opened it in.

saying these in an interview costs you the question

  • Claims extractall always refused paths escaping the destination
  • Thinks the data filter also caps decompressed size
  • Believes the filter checks member names but not link targets
  • Assumes filter=None means the strictest policy
  • Says fully_trusted is fine because the uploader was authenticated
  • Thinks a rejected member is skipped and extraction continues

context

resolving links…