fetchPnpmDeps: fix reproducibility of pnpm v11 store index

The pnpm v11 store uses a SQLite database (index.db) whose binary format
is non-deterministic across platforms (version-valid-for number, etc).
This caused hash mismatches when building the same pnpmDeps on different
machines despite identical logical content.

Fix by dumping the SQLite database to a text SQL file during the fetch
phase and reconstructing it during the build phase. This ensures the
stored representation is fully deterministic.

Assisted-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Co-authored-by: Gergő Gutyina <gutyina.gergo.2@gmail.com>
This commit is contained in:
Maciej Krüger
2026-05-19 20:24:30 +02:00
committed by Sefa Eyeoglu
parent f63696258a
commit f2ba699f37
3 changed files with 23 additions and 3 deletions

View File

@@ -342,7 +342,7 @@ stdenv.mkDerivation (finalAttrs: {
pnpmDeps = fetchPnpmDeps {
inherit (finalAttrs) pname version src;
inherit pnpm;
fetcherVersion = 3;
fetcherVersion = 4;
hash = "...";
};
})
@@ -384,7 +384,7 @@ It is highly recommended to use a pinned version of pnpm (i.e., `pnpm_9` or `pnp
pnpmDeps = fetchPnpmDeps {
inherit (finalAttrs) pname version src;
+ pnpm = pnpm_10;
fetcherVersion = 3;
fetcherVersion = 4;
hash = "...";
};
})
@@ -498,7 +498,7 @@ This is the version of the output of `fetchPnpmDeps`. New packages should use `3
# ...
pnpmDeps = fetchPnpmDeps {
# ...
fetcherVersion = 3;
fetcherVersion = 4;
hash = "..."; # clear this hash and generate a new one
};
}
@@ -516,6 +516,7 @@ Version 3 is the recommended value for new packages. Versions 1 and 2 are deprec
- 1: Initial version, nothing special.
- 2: [Ensure consistent permissions](https://github.com/NixOS/nixpkgs/pull/422975)
- 3: [Build a reproducible tarball](https://github.com/NixOS/nixpkgs/pull/469950)
- 4: [Dump SQLite database to an SQL file](https://github.com/NixOS/nixpkgs/pull/522703)
### Yarn {#javascript-yarn}

View File

@@ -8,6 +8,7 @@
makeSetupHook,
pnpm,
pnpm-fixup-state-db,
sqlite,
writableTmpDirAsHomeHook,
yq,
zstd,
@@ -19,6 +20,7 @@ let
1 # First version. Here to preserve backwards compatibility
2 # Ensure consistent permissions. See https://github.com/NixOS/nixpkgs/pull/422975
3 # Build a reproducible tarball. See https://github.com/NixOS/nixpkgs/pull/469950
4 # Dump SQLite database to an SQL file. See https://github.com/NixOS/nixpkgs/pull/522703
];
in
{
@@ -87,6 +89,7 @@ in
moreutils
pnpm # from args
pnpm-fixup-state-db'
sqlite
writableTmpDirAsHomeHook
yq
zstd
@@ -172,6 +175,13 @@ in
if [ -f "$storePath/v11/index.db" ]; then
pnpm-fixup-state-db "$storePath/v11";
# Dump the SQLite database to a SQL text file for reproducibility.
# SQLite's binary format is non-deterministic (version-valid-for number, etc),
# so we store the logical contents as SQL statements and reconstruct during build.
if [[ ${toString fetcherVersion} -ge 4 ]]; then
sqlite3 "$storePath/v11/index.db" .dump > "$storePath/v11/index.db.sql"
rm "$storePath/v11/index.db"
fi
fi
# This folder contains symlinks to /build/source which we don't need
@@ -231,6 +241,7 @@ in
pnpmConfigHook = makeSetupHook {
name = "pnpm-config-hook";
propagatedBuildInputs = [
sqlite
writableTmpDirAsHomeHook
zstd
];

View File

@@ -55,6 +55,14 @@ pnpmConfigHook() {
chmod -R +w "$STORE_PATH"
# Reconstruct the SQLite database from the SQL dump if needed.
# The fetch phase stores a text SQL dump instead of the binary db
# to ensure reproducibility across platforms.
if [ -f "$STORE_PATH/v11/index.db.sql" ]; then
sqlite3 "$STORE_PATH/v11/index.db" < "$STORE_PATH/v11/index.db.sql"
rm "$STORE_PATH/v11/index.db.sql"
fi
pnpm config set store-dir "$STORE_PATH"
# Prevent hard linking on file systems without clone support.