#5303 [Other] packaging: swap the Terminal bundle's stale pip-vendored copies and re-align its SBOM (the last CVE carriers)

closed · @didlawowo · 0 comentarios · En GitHub

Setup & installAMD / HIPNVIDIA / CUDAModels & quantsSecurityWindowsLinux

Descripción

Problem and result: The published `strata-cuda13` container images carry the Terminal archive, whose bundled pip vendors its own dependency copies inside `pip/_vendor/`. Those vendored copies are not upgraded when pip upgrades itself, so the bundle shipped `pip/_vendor/[email protected]` (GHSA-6q48 CRITICAL, revoked emSign roots — reachable by pip during first-start downloads), `pip/_vendor/[email protected]`, `pip/_vendor/[email protected]` and `pip/_vendor/[email protected]`. Scanner reports therefore stay red even on a bundle whose requirements are already fixed. Result: at archive build time the vendored copies are replaced with the fixed versions of the same interpreter, pip's `vendor.txt` manifest and its embedded CycloneDX BOM are re-aligned with the files actually shipped, and a Trivy-grade scan of the image reports the remaining carriers only.

Related discussion: #5284 (our Trivy scan of the published `0.2.9-cuda13` image, digest `83e37db8`: 37 HIGH/CRITICAL findings). #4220 already fixed Pillow; the current locks pin `certifi 2026.10.19`, `urllib3 2.8.0`, `pillow 13.0.0`. In a rebuild of the 0.2.9 image with the requirements fixed but nothing else, Trivy still reported 8 findings and all but two of them attributed to the `pip/_vendor` copies listed above — this change is the remaining piece.

Approach: `packaging/package_terminal.py`, after `install_locked()` and before the probe:

- install the two fixed versions the runtime locks deliberately do not provide (`msgpack>=1.2.3,<2`, `setuptools>=78.1.1,<79`) into the bundled interpreter — packaging internals, not runtime deps of the bundle;
- replace `pip/_vendor/{certifi,msgpack,urllib3,pkg_resources}` with the site-packages copies of the same interpreter (unmodified upstream versions, ABI-compatible);
- re-align `pip/_vendor/vendor.txt` with the swapped versions (`pkg_resources` swap also updates the `setuptools` line) and drop the redundant vendored source trees pip ships under its own `dist-info/licenses/src/pip/_vendor/`;
- refresh the embedded `pip/_vendor/bom.cdx.json` so component versions and purls match the shipped files;
- run `pip --version` as a check that pip still works with the swapped copies.

pip itself is deliberately not upgraded (the interpreter's pip stays, as documented in the module docstring). The manifest records the swap under `pip_vendored_swapped`.

Validation:
  Passed: end-to-end terminal packaging on darwin-arm64 (`python3 packaging/package_terminal.py --out ... --cache ...`, commit `e4329584` of this branch): exit 0, archive built, `pip_vendored_swapped` = `[certifi, msgpack, pkg_resources, urllib3]`; unpacking the produced archive shows `vendor.txt` = `certifi==2026.10.19, msgpack==1.2.4, urllib3==2.8.0, setuptools==78.1.1` and the BOM components showing the same versions.
  Passed: `unittest tests/terminal/test_packaging_pins.py` (9/9), `test_bundle_imports.py`, `test_terminal_commit.py`, `test_bundle_tools_paths.py`, `test_tuiqa_version.py` — all OK on this branch.
  Failed: `test_archive_gaps.py` has 1 failure and `test_docker_files.py` 1 error — reproduced identically on a pristine checkout of `main` (commit `c3b3f503`), so both are pre-existing and unrelated to this change.
  Not run: a full rebuild of the published docker image from `main` plus a Trivy re-scan (the evidence for the swap chain comes from our 0.2.9 rebuilds, where the same steps took the scan from 37 findings to 2); the `windows-x86_64` and `linux-aarch64` targets (the swap code is target-independent, but only the darwin-arm64 path was exercised end to end).

Limits: pip's vendored `setuptools` ships `jaraco.context` and `wheel` internally that scanners still flag (build-time only, never executed at first start); the two installed "fix" versions are outside the hash lock by design (they are packaging internals; the lock-match check in `install_locked()` runs before this step, so the bundle's runtime packages stay exactly the lock's); if a future lock drops `certifi` or `urllib3` from site-packages, the corresponding vendored copy is left untouched (the swap skips missing sources).

En el sitio

Enlaces a install, modelos, releases.