I scanned the 600 most-downloaded npm CLI packages for Windows bugs

Why do this at all

I maintain a small static checker called winbreak. It looks for the handful of patterns that make Node code work on macOS and fail on Windows: spawning a .cmd without a shell, spawning an extensionless node_modules/.bin shim, shelling out to a command Windows does not have, hardcoding /tmp.

A checker with no calibration is worthless. Anyone can write a regex that reports something in every file, and the resulting number tells you nothing except that the regex is loose. So the question I wanted answered was not "how much npm code is broken." It was "is my threshold sane?"

The method, so you can re-run it

I searched the npm registry for packages tagged cli, devtools, build-tool, scaffold, generator and process, which gave 1,417 unique names. I ranked those by real weekly download counts from the npm downloads API — not by the registry's own popularity score, which returned an identical value for every package and is useless for ranking — and took the top 600.

Each one was fetched as its published tarball, extracted, scanned including dist/, and deleted before the next batch. Two fetches failed, leaving 598.

The headline result

That 91.6% is the number I actually wanted. A tool that lit up on most of the ecosystem would have a broken threshold, and a tool that found nothing anywhere would be doing nothing. Mature, widely-used packages are mostly fine, which is what you would hope and is worth saying out loud.

Then I read all forty-eight

This is where a survey usually stops and publishes the percentage. It is also where it stops being honest, because a count of findings is not a count of bugs.

3 real   38 deliberately Linux-only   7 my checker being wrong

The correction, because it is the interesting part

My first pass through these findings put eleven in the “real” column. I published that. Then I went back to write each one up properly, which meant opening the actual source rather than the single line my own tool had shown me. Four of the eleven evaporated:

• create-storybook — the ps call is there, but one function up getProcessAncestry dispatches on os.platform() === "win32" and only reaches it on Unix.
• pi-subagents — the caller sets posixGroupOwned = process.platform !== "win32" and returns "unsupported-platform" before the call.
• metaharness — an if (platform === 'win32') { …powershell…; return } sits directly above the ps call.
• disk — the /tmp path is set as an environment variable for a remote process. It is not a path on this machine at all.

Three of those four hide the guard in a different function. That is cross-function analysis, and a tool that reads text one file at a time is not going to do it. I fixed the fourth — the block-form early return — and left the rest as a stated limitation rather than pretending otherwise.

The three that survived

agent-cli-detector, at 4.5M downloads a week, accounts for two. It calls execFileSync("ps", …) inside a try/catch that returns an empty string, and there is no mention of platform, win32, darwin or linux anywhere in the 326-line file. On Windows the call throws, the catch swallows it, and the entire process-tree detection strategy quietly finds nothing.

The third is pmx, part of the pm2 family, and it is the whole article in five lines:

childProcess.execFile(/^win/.test(process.platform) ? 'npm.cmd' : 'npm',
  ['ls', '--json', '--production'],
  { windowsHide: true, maxBuffer: 1024 * 1024 },
  function (error, stdout, stderr) {
    // if we can't spawn the npm binary, stop here
    if (error) return

Someone thought carefully about Windows here — there is a platform check, and windowsHide is a Windows-only option. But since Node 18.20.2 / 20.12.2 shipped the fix for CVE-2024-27980, handing a .cmd to execFile without shell: true throws EINVAL. The callback catches it and returns. So on Windows this quietly collects no dependency data, forever, and nothing anywhere logs a thing.

The thirty-eight that are not bugs

The largest group by far, and the most interesting one. This code is POSIX-only on purpose, and my checker has no way to know that.

There is no clever fix for this. execSync('rm -rf ...') in a script that only ever runs on a Linux build agent is indistinguishable, as text, from the same call in a cross-platform CLI. One is fine and one is a bug, and the difference is intent. A checker that reads text cannot see intent, and pretending otherwise is how you get a tool nobody trusts.

The seven that were my fault

The four from the correction above, plus firebase-tools, flagged for "/home/firebase/app/…" — a path inside a Docker image, not on the machine running the code — and generify, flagged for a /tmp path in its example.js, which is documentation.

That is a 15% false-positive rate on findings, and I would rather print it than have you discover it. Every checker has one. Most do not tell you what theirs is.

Seven bugs in my own checker

Getting from the first run to the numbers above meant fixing the checker seven times. This was the survey's actual yield.

1. whoami was on my list of commands Windows does not have. It ships with Windows, and has since Vista. Two packages were reported for perfectly good code. The general trap is worth naming: "this is a Unix command" is not the same as "Windows does not have it." Windows also has find, sort, more, where, tasklist and taskkill.
2. Guard detection did not understand a platform name. It looked for process.platform and isWindows. agent-browser stores the platform in a local variable and writes if (os === 'linux'), with an os === 'win32' branch immediately below. Correct code, reported as a bug.
3. And when I fixed that, it still failed. To stop braces inside strings from confusing the brace counter, the code stripped string literals before recording a block header — and then recorded the stripped version. So the guard arrived as if (os === '') with the platform name deleted. It now counts on the stripped line and keeps the original.
4. An early return is a guard. if (win32) return; followed by the POSIX call does not enclose anything, so walking the enclosing blocks never found it.
5. Handing a .cmd to cmd.exe was reported as a bug. That is the recommended fix. projen does exactly the right thing and got told off for it.
6. A platform branch that returns is a guard too, even spread over a block. The single-line if (win32) return; was handled; the same thing written as if (win32) { …; return } above the call was not, because that block does not enclose the call, it only leaves before it. metaharness was reported for exactly this.
7. rm -rf matched anywhere in 2,000 characters of extracted call text. sails-generate was flagged because a console.log("rm -rf node_modules && npm install") — advice printed for a human to read — fell inside a nearby call's window.

There was an eighth, of a different kind: findings on minified lines. Next.js ships cross-spawn compiled to a single line, and my checker reported it as a .cmd spawning bug. cross-spawn is the library that exists to fix .cmd spawning. A finding on line 1 of a 200KB line is useless even when it is correct, so those are suppressed now.

What I would take away from this

Most of what a text-based checker flags in mature code is deliberate. Four fifths of my findings were people knowingly writing Linux-only code in a Linux-only path. If you build something like this, the default assumption for a finding in a well-used package should be "this is intentional and I do not understand the context yet," not "I found a bug."

The survey audited the tool, not the ecosystem. I set out to measure npm and measured myself. Every one of those six bugs was invisible until real code walked into it, and none would have been caught by more unit tests, because I would have written the tests with the same wrong assumptions.

Calibrate against code you believe is correct. It is tempting to test a checker on things you suspect are broken, because finding something feels like success. The useful signal is the opposite: point it at code you are confident about and see what it says. Every improvement here came from a finding I did not want.