A TorchServe Commit Pointed pip at Two Extra Package Indexes

TorchServe, the model server in pytorch/serve, keeps its Python requirements in a requirements/ folder, one file per platform and accelerator. In June 2023 commit b260776b5, “Enable opt-6.7b benchmark on inf2”, added support for AWS Inferentia2 machines. It added two requirements files, and each begins by pointing pip at another package index:

requirements/neuronx.txt              --extra-index-url https://pip.repos.neuron.amazonaws.com
requirements/torch_neuronx_linux.txt  --extra-index-url https://download.pytorch.org/whl/cpu

Both indexes belong to the people who make the packages: AWS publishes its Neuron compiler and libraries on the first, and the PyTorch project publishes its CPU builds of torch on the second. That’s the normal way to get those packages. We ran Remixware 0.3.1’s dependency gate on the commit against its parent, offline, to see how it handles a change like this. The commands, hashes and output are in the Remixware repository.

What the Gate Printed

remixware deps . --base-ref 2551a99e495d973fc916322bed352b3a13bf9d5f

The repository had 39 requirements files at that commit, the examples’ included, and the gate read them all on both sides. It reported 10 changes and failed with exit code 1. Four of them fail. Two are the new index lines:

NOT CHECKED          requirements/neuronx.txt changed
                     This change tells pip to look for packages somewhere new (--extra-index-url
                     https://pip.repos.neuron.amazonaws.com), and a package of the same name there
                     can take the place of the one you expect. Check the index by hand, or take
                     not-checked out of deps.fail_on.

The other two are packages no requirements file on the base side named: neuronx-cc, the Neuron compiler, and torchdata 0.5.1. Both are not-allowed. torch-neuronx and transformers-neuronx are in the new file too, but an example in examples/large_models/inferentia2 already listed them, so they weren’t new to the repository.

The remaining six are updated: packages that now appear at versions the base didn’t have, such as torch 1.13.1+cpu and protobuf 3.20.3, which the Neuron file pins.

Why an Index Line Is Worth a Look

With --extra-index-url, pip looks for each package on PyPI and on the extra index, and installs the best version it finds on either. If a name exists in both places, the higher version wins. That’s how dependency confusion works: someone publishes a package on PyPI under a name a private or vendor index uses, with a bigger version number. Pinning a version, or hashes, narrows it. neuronx-cc, torch-neuronx and transformers-neuronx aren’t pinned in this file.

None of that means anything went wrong here. It means the change moved part of the trust from PyPI to two more places, and the gate can’t judge another index from a requirements file. So it says so, names the index, and fails until someone decides. That’s what not-checked is for.

Something We Noticed in the Output

neuronx-cc has no version in the file, and 0.3.1 printed it as pypi/neuronx-cc , added, with an empty version before the comma, and "to": [""] in the JSON. The verdict was right, but the display was untidy. Remixware 0.3.2 fixes it: the same run now prints pypi/neuronx-cc unpinned, added, and the JSON has "to": ["unpinned"]. Nothing else in the output changed.

What It Didn’t Check

Requirements files don’t record install scripts the way an npm lockfile does, so the install script check listed 8 versions as needing --online, and the publisher check the same. We left the lookups off: a requirement without a pinned version can’t be looked up anyway, and a 2023 release is outside the 90-day window the publisher check uses by default.

The dependency gate post lists everything not-checked covers, and scripts/deps_case_studies.sh in the repository runs this comparison again.