releases

Remixware Now Fails Pull Requests That Bring In Risky Dependencies

A pull request that adds one line to package.json can add forty packages to the lockfile. Reviewers read the line. Hardly anyone reads the lockfile, and that’s where a package nobody chose, or one that runs a script on every developer’s machine at install time, gets in.

Remixware 0.3 adds a second check for that moment. remixware deps compares the packages your lockfiles name with the ones on your main branch. A new package that isn’t on your allow list fails the change, whether you asked for it or another package pulled it in. So does an added or updated package that runs code when it’s installed, and, if you let it look online, a recent release by someone who didn’t publish or maintain the version before it.

The licence check asks what copied code expects of you. This one asks what a dependency change lets into your build.

A Short Example

In a pull request, point it at main:

remixware deps . --base-ref origin/main

Here is part of what it printed for the test change in Remixware’s own repository, which adds a Go module, two npm packages and a Python package, and moves esbuild to a newer version:

INSTALL SCRIPT       npm/esbuild 0.19.2 -> 0.21.5 (direct)
                     in web/package-lock.json
                     Runs an install script (preinstall, install or postinstall, or a node-gyp
                     build) when npm installs it. Accept it with a deps allow rule that has
                     install_scripts: true and a reason.

NOT ALLOWED          npm/is-thirteen 2.0.0, added (transitive)
                     in web/package-lock.json
                     New, and not on the allow list. Add a deps allow rule with a reason to the
                     policy file, or drop it.

10 changes: 1 install-script, 2 not-allowed, 3 allowed, 4 updated.
FAIL: 3 changes break the policy (fail on: not-allowed, install-script, publisher-changed, not-checked).

The allow list lives in .remixware.json, next to the licence policy. Each rule names a package and says why it’s there:

{
  "deps": {
    "allow": [
      {"package": "golang/golang.org/x/*", "reason": "modules of the Go project"},
      {"package": "npm/@shop/*", "reason": "our own packages"},
      {"package": "npm/esbuild", "install_scripts": true, "reason": "downloads its platform binary"}
    ]
  }
}

A rule allows the package. It doesn’t wave through its install scripts unless it says so. Go and npm names are compared exactly, so github.com/Sirupsen/logrus doesn’t pass on a rule for github.com/sirupsen/logrus.

Offline First

By default deps reads lockfiles and nothing else, so the same files always give the same answer. It understands go.mod with go.sum, package-lock.json (all three lockfile versions), pip requirements files and poetry.lock. The allow list runs offline. So does the install script check for npm lockfiles of version 2 or 3, which record which packages have scripts, and for poetry.lock, where a package with no wheel means pip will build it from source.

Add --online and it also asks the npm registry and PyPI about each added or updated version: whether it runs install scripts, and who published it. A release counts as a publisher change when its publisher neither published nor maintained the version it’s compared with, and it came out within the last 90 days. npm records the account that published each version. PyPI only has the release’s own author and maintainer fields, which are weaker evidence, and the output treats them as such. Answers are kept in a cache file, since what a registry says about a published version never changes.

Go modules don’t run code when they’re downloaded, and the Go module proxy keeps no record of who published what. For Go, both registry checks say “not applicable” and leave it there.

It Says What It Checked

A pass is only worth something if you know what it covered. Every run ends with a block that says, for each check, how many packages it looked at, how, and why it skipped the rest:

Checks:
  allow list         ran, 5 added packages: 5 checked from the lockfiles
  install scripts    partly, 10 added or updated versions: 4 checked from the
                     lockfiles, 3 not checked (needs --online), 3 not applicable (Go
                     modules run no install scripts)
  publisher changes  not run, 10 added or updated versions: 7 not checked (needs
                     --online), 3 not applicable (the Go module proxy records no
                     publisher)

Without --online it never reports an online check, and the JSON output carries the same counts.

Some things can’t be judged from a lockfile at all, and those fail too unless you decide otherwise. A yarn.lock or pnpm-lock.yaml that changed is reported as not checked, since Remixware doesn’t read those yet. So is a requirements file that starts sending pip to another package index, and an update that moves a package from the public registry to a URL or a git repository. A broken lockfile on either side stops the command with exit code 2. A registry lookup that fails does the same. Neither ever ends in a pass.

One more catch. The policy file comes from the pull request, so a pull request can add an allow rule for its own new package. deps lists every allow rule the change adds, and --base-policy reads the policy from main instead, so only rules that are already merged count.

How It Was Tested

The tests read every supported lockfile format, good and broken, and run the online checks against a fake registry. Then there’s a test aimed at the promise itself. It generates changes across all five lockfile formats and checks each outcome:

  • 400 changes added a package that wasn’t on the allow list, and all 400 failed.
  • 800 changes kept the set the same in a shuffled order, or added an allowed package, and all 800 passed.
  • 80 updates started running an install script, and all 80 failed.
  • 400 lockfiles were cut short or had a stray line put in, and all 400 stopped with an error.

The release workflow also runs the gate on the test change before it publishes anything. The test output is in the repository under test/results/deps.

On Real Projects

Three case studies run the gate on real commits from public projects, offline, and show what it printed. TypeScript’s move to esbuild adds one line to package.json and 23 packages to the lockfile, one of them with an install script. Hugo’s deploy command brings in 18 new Go modules, the AWS, Azure and Google Cloud libraries among them. And a TorchServe commit points pip at two extra package indexes, which the gate reports as not checked. Five tested recipes cover the GitHub Actions job, allow lists, --base-policy, the JSON output and monorepos.

Getting It

Get 0.3 from the download page. The reference lists the new command, its options and the deps policy keys, and the quick start shows where it fits in a pull request. Indexes, policy files and licence reports work as they did in 0.2.