Skip to content

fix: apply tar "data" extraction filter whenever available (path traversal in Untar) - #544

Open
gaoflow wants to merge 1 commit into
fatiando:mainfrom
gaoflow:fix-543-untar-data-filter-feature-detect
Open

fix: apply tar "data" extraction filter whenever available (path traversal in Untar)#544
gaoflow wants to merge 1 commit into
fatiando:mainfrom
gaoflow:fix-543-untar-data-filter-feature-detect

Conversation

@gaoflow

@gaoflow gaoflow commented Jun 11, 2026

Copy link
Copy Markdown

Problem

Untar enables the tarfile data extraction filter only on Python ≥ 3.12:

filter_kwarg = {} if sys.version_info < (3, 12) else {"filter": "data"}

But the data filter (PEP 706) was backported to 3.9.17, 3.10.12 and 3.11.4. On those interpreters Pooch skips the filter even though it is available, so tarfile.extractall() runs with no filter and no per-member validation.

A malicious archive served from a caller-influenceable URL can then escape the destination: a symlink member pointing outside .untar, followed by a regular file written through that symlink, writes the file outside the extraction directory when an application calls pooch.retrieve(url, known_hash=None, processor=Untar()). Reported in #543.

I verified this on Python 3.11.11 (which does have tarfile.data_filter): the two-entry PoC writes outside the destination under the current < (3, 12) gate, and is blocked (AbsoluteLinkError) once the filter is applied.

Fix

Detect the filter by feature instead of minor version:

filter_kwarg = {"filter": "data"} if hasattr(tarfile, "data_filter") else {}

The protection now applies on every interpreter that ships the filter, while interpreters without it keep the previous behavior (no regression). Scope is limited to Untar; Unzip relies on ZipFile.extractall, which already sanitizes .. / absolute member names.

Test

Adds test_untar_blocks_symlink_path_traversal: it builds the symlink-escape tar in memory, then asserts extraction raises tarfile.FilterError and that no file escapes the destination. The test is skipif-guarded on the rare interpreter without data_filter. It fails on the old < (3, 12) gate (the escape succeeds) and passes with the fix.

Closes #543

Untar gated the "data" tarfile extraction filter on Python >= 3.12, so on
3.9-3.11 it extracted with no filter and no member validation. Since the
filter was backported to 3.9.17, 3.10.12 and 3.11.4, a malicious archive (a
symlink escaping the destination followed by a file written through it) could
write outside the .untar directory on those interpreters. Detect the filter by
feature (hasattr(tarfile, "data_filter")) instead of minor version so the
protection applies on every interpreter that supports it.

Closes fatiando#543
@gaoflow

gaoflow commented Jun 11, 2026

Copy link
Copy Markdown
Author

Heads-up: the red CI here isn't from this change — every job fails in the "Collect requirements" step with Could not find a version that satisfies the requirement dependente==0.3.0 (from versions: none), before any of the package or tests run. dependente 0.3.0 is in fact the current release on PyPI, so this looks like a transient index hiccup during that run; a re-run should clear it (I don't have rerun rights on the repo).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pooch Untar processor skips tarfile data filter on Python 3.9–3.11, allowing symlink path traversal outside the extraction directory

1 participant