npm lockfiles can contain package aliases where the lockfile key differs
from the actual package name (e.g., "string-width-cjs" aliasing
"string-width"). Previously we always used the lockfile key, causing us
to fetch packuments for the wrong package.
Use the package's own "name" field when present, falling back to the
lockfile key. This ensures we fetch the correct packument for aliased
packages, fixing non-deterministic builds where the wrong packument
fetch could succeed or fail depending on network timing.
The current approach parses tarball URLs to extract package names for
packument fetching. This is fragile as it only handles npmjs.org URLs
and requires special-casing other registries.
Use lockfile keys directly instead. The lockfile already contains the
canonical package names in the form "node_modules/@scope/name", so we
can simply strip the prefix rather than parsing URLs.
This handles all registries uniformly and eliminates the URL parsing
code along with its tests.
Add a cacheVersion parameter to fetchNpmDeps and npmDepsCacheVersion to
buildNpmPackage. When set to 2, prefetch-npm-deps will also fetch and
cache packuments (package metadata) in addition to tarballs.
npm can request packuments with two different Accept headers:
- corgiDoc: abbreviated metadata (default)
- fullDoc: full metadata (used for workspaces)
npm's cache policy requires headers to match, so we cache both versions.
This is opt-in via cacheVersion to avoid breaking existing hashes.
Set npmDepsCacheVersion = 2 for projects using npm workspaces.
Also fix cacache index format to properly separate multiple entries
with newlines, and update map_cache() to parse multi-line index files.
Previously, users had to add hacks to add additional nativeBuildInputs
entries to the fetcher. This way we allow users to append
nativeBuildInputs, impureEnvVars and also merge potential passthru
attributes.
This might cause rebuilds.
Signed-off-by: Sefa Eyeoglu <contact@scrumplex.net>
This moves all packages to the new top-level attributes introduced
previously.
This doesn't cause any rebuilds.
Signed-off-by: Sefa Eyeoglu <contact@scrumplex.net>
move pnpm.{fetchDeps,configHook} to fetchPnpmDeps and pnpmConfigHook
respectively. Also adds aliases to the former package-level attributes.
Additionally, pnpmConfigHook does not propagate pnpm anymore, to make
changing pnpm versions easier.
This brings pnpm in line with the other Node.js/JavaScript tooling in
nixpkgs.
This doesn't cause any rebuilds.
Signed-off-by: Sefa Eyeoglu <contact@scrumplex.net>
This commit was created by a combination of scripts and tools:
- an ast-grep script to prefix things in meta with `lib.`,
- a modified nixf-diagnose / nixf combination to remove unused `with
lib;`, and
- regular nixfmt.
Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
Adds an `npmRegistryOverrides` argument & variable to fetchNpmDeps
& prefetch-npm-deps. This is similar in usage to `url.<url>.insteadOf`
in `.gitconfig`, allowing one to redirect requests to a registry to
another URL.
It's currently possible to run into a race condition when entering the same development environment concurrently:
```
❯ git fetch; jj rebase -b 'all:mutable() & mine()' -d main@origin --skip-emptied
Rebased 4 commits onto destination
Abandoned 1 newly emptied commits
Working copy (@) now at: tnyknvqt 93e36def (empty) (no description set)
Parent commit (@-) : wnqxqyyl e0983a05 main@origin | Increase limit of max number of words
Added 0 files, modified 8 files, removed 0 files
direnv: loading ~/dev/REDACTED/.envrc
direnv: loading https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.4/direnvrc (sha256-DzlYZ33mWF/Gs8DDeyjr8mnVmQGx7ASYqA5WlxwvBG4=)
Executing linkNodeModulesHook
node:internal/fs/promises:782
return await PromisePrototypeThen(
^
Error: ENOENT: no such file or directory, rename 'node_modules/.bin-nix-hook-temp' -> 'node_modules/.bin'
at async Object.rename (node:internal/fs/promises:782:10)
at async /nix/store/ps9ivjjxzi0fks67j6vd4gbw5dcnhp0w-link-node-modules.js:84:7
at async Promise.all (index 0)
at async main (/nix/store/ps9ivjjxzi0fks67j6vd4gbw5dcnhp0w-link-node-modules.js:58:3) {
errno: -2,
code: 'ENOENT',
syscall: 'rename',
path: 'node_modules/.bin-nix-hook-temp',
dest: 'node_modules/.bin'
}
Node.js v20.17.0
Finished executing linkNodeModulesShellHook
```
This change removes the intermediate file creation and simply tries to create a symlink directly.
If the target `node_modules/foo` already exists we unlink it and try to create the symlink again.
these options affect npm generally, and aren't specific to our
`buildNpmPackage`. pushing these options into the `npmConfigHook` (which
is used by `buildNpmPackage`, among others) makes them available to more
packages, especially to enable cross compilation.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
