nathan/nixpkgs
1
0
Fork 0
mirror of https://github.com/NixOS/nixpkgs.git synced 2026-06-05 21:03:40 +00:00
Code Issues Packages Projects Releases Wiki Activity
1,011,408 Commits 355 Branches 81 Tags
master
Commit Graph

51 Commits

Author SHA1 Message Date
zimbatm
363ae292bf splice: defer per-output spliceReal recursion behind genAttrs 
`//` forces its rhs to compute the merged keyset, so the original
`// spliceReal (...)` recursed eagerly on every spliced derivation's
outputs even when no specific output (.dev, .doc) was selected.

genAttrs over the known output names with the splice in the
value-thunk defers it: pkg.dev forces, pkg alone doesn't.

Addresses #338231.

Repro:

  EXPR='let pkgs = import ./. { localSystem = "x86_64-linux"; crossSystem = "aarch64-linux"; }; in (pkgs.python3.withPackages (ps: [ ps.requests ])).drvPath'
  NIX_SHOW_STATS=1 nix-instantiate --eval -E "$EXPR"

drvPath byte-exact (mb82vdkqafg9i8x0makrgxw8xcc4y8lz both ways).

Before:

  cpuTime: 0.649
  nrFunctionCalls: 651343
  nrOpUpdates: 153257
  nrOpUpdateValuesCopied: 7908622
  nrThunks: 2018744
  values: 5061740
  sets (bytes): 205342176
  gc totalBytes: 320504688

After:

  cpuTime: 0.554
  nrFunctionCalls: 586266
  nrOpUpdates: 125579
  nrOpUpdateValuesCopied: 5482176
  nrThunks: 1494021
  values: 3498806
  sets (bytes): 141905680
  gc totalBytes: 229067888
 2026-04-08 13:10:56 +02:00
Robert Hensing
3ae1aa9968 pkgs/splice: drop reference to past code 
References to previous states of the code generally only belong in
commit messages.
A reminder or warning to future readers is not needed here.
 2026-04-06 12:27:06 +02:00
Robert Hensing
d7dff39146 pkgs/splice: bring mapAttrs into scope 
... and take it from lib. Inconsequential, but it's a good convention
because it has occasional improvements to builtins' intentionally
highly unchanging behavior.
 2026-04-06 12:25:46 +02:00
zimbatm
6ae8efaf5f splice: replace listToAttrs+map with mapAttrs 
listToAttrs forces every list element to extract `.name`, calling merge
eagerly for the entire union of attribute names (~60k). mapAttrs already
knows the keys; the lambda is only invoked when a key is selected,
letting the 99.7% unforced ones stay as untouched thunks.

NIX_SHOW_STATS delta on pkgsCross.aarch64-multiplatform.hello.drvPath:

  nrFunctionCalls:  567,512 -> 507,404   (-10.6%)
  nrThunks:         613,138 -> 552,356   (-9.9%)
  sets.number:      158,787 -> 98,339    (-38.1%)
  list.elements:    295,085 -> 174,189   (-41.0%)
  gc.totalBytes:      117.5M -> 111.7M   (-4.9%)

Repro:

  NIX_SHOW_STATS=1 nix-instantiate --eval \
    -E '(import ./. {}).pkgsCross.aarch64-multiplatform.hello.drvPath' \
    2>stats.json >/dev/null
  jq . stats.json
 2026-04-04 14:18:29 +02:00
quantenzitrone
99f20e75ab pkgs.xorg: deprecate package set 2026-02-02 00:20:15 +01:00
jopejoe1
91d2625d20 top-level/splice.nix: remove special treatment for xorg 
This removes the special treatment of Xorg in `splice.nix` by either inheriting the still-used
aliases in `all-packages.nix` or adding the unused ones to `aliases.nix`.
 2026-01-21 23:04:23 +01:00
Philip Taron
21623b7ce7 Reapply "splice.nix: make pkgs splicedPackages" (#456138) 2025-11-01 13:49:33 +00:00
Robert Hensing
acb6191c6e lib: Add splicing utilities (#426889) 2025-10-30 21:22:54 +00:00
Artturin
2494e3664f splice.nix: make pkgs splicedPackages when required 
This will make `pkgs` used in `callPackage`, and `pkgsCross.X.pkgs` have
packages with `__spliced`.

3029741718/pkgs/development/interpreters/python/passthrufun.nix (L37)

d2bd9a39de/pkgs/top-level/python-packages.nix (L10720)

```
nix-repl>
pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
error:
       … while evaluating the attribute
'aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced'
         at
/home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/development/python-modules/protobuf/4.nix:119:13:
          118|   passthru = {
          119|     inherit protobuf;
             |             ^
          120|   };

       error: attribute '__spliced' missing
       at «string»:1:1:
            1|
pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
             | ^
```

to

```
nix-repl>
pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
{
  buildBuild = «derivation
/nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildHost = «derivation
/nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildTarget = «repeated»;
  hostHost = «derivation
/nix/store/mszvybzs4zxh43awyrjnybsfcb265n9r-protobuf-aarch64-unknown-linux-gnu-25.4.drv»;
  hostTarget = «repeated»;
}
```

Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
 2025-10-28 21:54:10 +01:00
NAHO
c8d4dabc43 pkgs: remove optional builtins prefixes from prelude functions 
Remove optional builtins prefixes from prelude functions by running:

    builtins=(
      abort
      baseNameOf
      break
      derivation
      derivationStrict
      dirOf
      false
      fetchGit
      fetchMercurial
      fetchTarball
      fetchTree
      fromTOML
      import
      isNull
      map
      null
      placeholder
      removeAttrs
      scopedImport
      throw
      toString
      true
    )

    fd \
      --type file \
      . \
      pkgs \
      --exec-batch sed --in-place --regexp-extended "
        s/\<builtins\.($(
          printf '%s\n' "${builtins[@]}" |
            paste --delimiter '|' --serial -
        ))\>/\1/g
      "

    nix fmt
 2025-10-04 19:02:37 +02:00
Robert Hensing
f8566d22c1 pkgs/splice.nix: Refactor, use cross index helpers 2025-09-09 04:42:37 +02:00
Silvan Mosberger
374e6bcc40 treewide: Format all Nix files 
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:

  nix-build ci -A fmt.check

This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).

This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).

Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).

If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
 2025-04-01 20:10:43 +02:00
Artturin
9d9f4b50b4 Revert "splice.nix: make pkgs splicedPackages" 2024-12-06 18:58:53 +02:00
Artturin
41960fcdac splice.nix: make pkgs splicedPackages when required 
This will make `pkgs` used in `callPackage`, and `pkgsCross.X.pkgs` have
packages with `__spliced`.

3029741718/pkgs/development/interpreters/python/passthrufun.nix (L37)

d2bd9a39de/pkgs/top-level/python-packages.nix (L10720)

```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
error:
       … while evaluating the attribute 'aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced'
         at /home/artturin/nixgits/my-nixpkgs/.worktree/1/pkgs/development/python-modules/protobuf/4.nix:119:13:
          118|   passthru = {
          119|     inherit protobuf;
             |             ^
          120|   };

       error: attribute '__spliced' missing
       at «string»:1:1:
            1| pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
             | ^
```

to

```
nix-repl> pkgsCross.aarch64-multiplatform.python3Packages.protobuf4.protobuf.__spliced
{
  buildBuild = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildHost = «derivation /nix/store/s7da5mfvx4h1n86j78knaj9cprglxqz6-protobuf-25.4.drv»;
  buildTarget = «repeated»;
  hostHost = «derivation /nix/store/mszvybzs4zxh43awyrjnybsfcb265n9r-protobuf-aarch64-unknown-linux-gnu-25.4.drv»;
  hostTarget = «repeated»;
}
```
 2024-11-11 00:08:20 +02:00
Artturin
6977b4cdab splice.nix: improve performance with early cut-off (#350295) 2024-10-31 18:42:15 +02:00
Artturin
ea7c2e29be splice.nix: improve performance with early cut-off 
All these 062c4f5974/pkgs/top-level/splice.nix (L116-L121)
are already in pkgs, so they don't have to be added on top again when there's no splicing

|stat                  | before         | after          | Δ              | Δ%    |
|----------------------|----------------|----------------|----------------|-------|
|cpuTime               | 767.33         | 757.25         | ↘ 10.08        | -1.31%|
|envs-bytes            | 23,767,294,088 | 23,767,243,520 | ↘ 50,568       | -0.00%|
|envs-elements         | 1,260,156,605  | 1,260,151,924  | ↘ 4,681        | -0.00%|
|envs-number           | 855,377,578    | 855,376,758    | ↘ 820          | -0.00%|
|gc-heapSize           | 66,525,835,264 | 64,428,679,168 | ↘ 2,097,156,096| -3.15%|
|gc-totalBytes         | 146,231,201,712| 145,437,495,488| ↘ 793,706,224  | -0.54%|
|list-bytes            | 3,319,349,160  | 3,319,349,160  | 0              |       |
|list-concats          | 88,567,304     | 88,567,304     | 0              |       |
|list-elements         | 414,918,645    | 414,918,645    | 0              |       |
|nrAvoided             | 989,429,178    | 989,427,108    | ↘ 2,070        | -0.00%|
|nrFunctionCalls       | 784,049,358    | 784,048,538    | ↘ 820          | -0.00%|
|nrLookups             | 412,401,378    | 412,399,016    | ↘ 2,362        | -0.00%|
|nrOpUpdateValuesCopied| 2,132,453,124  | 2,082,900,933  | ↘ 49,552,191   | -2.32%|
|nrOpUpdates           | 99,050,775     | 99,048,489     | ↘ 2,286        | -0.00%|
|nrPrimOpCalls         | 411,581,753    | 411,581,753    | 0              |       |
|nrThunks              | 1,248,612,305  | 1,248,600,848  | ↘ 11,457       | -0.00%|
|sets-bytes            | 48,379,815,792 | 47,586,523,600 | ↘ 793,292,192  | -1.64%|
|sets-elements         | 2,840,072,218  | 2,790,496,848  | ↘ 49,575,370   | -1.75%|
|sets-number           | 183,666,269    | 183,660,877    | ↘ 5,392        | -0.00%|
|sizes-Attr            | 16             | 16             | 0              |       |
|sizes-Bindings        | 16             | 16             | 0              |       |
|sizes-Env             | 16             | 16             | 0              |       |
|sizes-Value           | 24             | 24             | 0              |       |
|symbols-bytes         | 3,454,822      | 3,454,838      | ↗ 16           | 0.00% |
|symbols-number        | 188,756        | 188,757        | ↗ 1            | 0.00% |
|values-bytes          | 38,132,032,032 | 38,131,756,776 | ↘ 275,256      | -0.00%|
|values-number         | 1,588,834,668  | 1,588,823,199  | ↘ 11,469       | -0.00%|

The table was formatted manually from stats copied from ofborg(I forgot how to do it properly), so it may
look a bit wonky
 2024-10-25 18:12:41 +03:00
Artturin
b14916e83c splice.nix: remove toplevel {build,host,target}Platform in __splicedPackages 
These should have been removed in ecab3ede but were not discovered.

Since these were only in `__splicedPackages` they were only available in
`callPackage`, now that they are removed the entries in aliases.nix will take over.

e056730f13/pkgs/top-level/aliases.nix (L1817)
 2024-10-21 21:13:21 +03:00
John Ericson
944b1056a6 generateSplicesForMkScope: Support dot in attribute path 
Only the list form supports the full domain of possible attribute names.
 2024-05-01 10:10:26 -04:00
Adam Joseph
a1fdbae706 lib.customisation: add uncurried form of makeScopeWithSplicing 
Deeply-curried functions are pretty error-prone in untyped languages
like Nix.  This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.

Let's add a version that uses attrset-passing form, to provide some
minimal level of sanity-checking.

This also provides defaults for keep and extra (these are often
unneeded by the user).
 2023-08-14 02:50:32 +03:00
Jan Malakhovski
85d4248a4f treewide: fix some comments (#247365) 
* pkgs/top-level/splice.nix: fix broken comment

* stdenv: use a value that actually works in the comment there
 2023-08-05 23:01:26 +02:00
Silvan Mosberger
a0b8caf3bc Revert "lib.customisation: uncurry makeScopeWithSplicing" 2023-07-28 23:04:09 +02:00
Adam Joseph
cb13669b00 lib.customisation: uncurry makeScopeWithSplicing 
Deeply-curried functions are pretty error-prone in untyped languages
like Nix.  This is a particularly bad case because
`top-level/splice.nix` *also* declares a makeScopeWithSplicing, but
it takes *two fewer arguments*.

Let's switch to attrset-passing form, to provide some minimal level
of sanity-checking.
 2023-07-27 21:31:59 -07:00
Artturin
ec7d5c518c splice.nix: finish nativeDrv,crossDrv removal 2023-06-22 23:11:08 +03:00
Artturin
665e15ee40 splice.nix: add convenience functions 2022-12-25 14:05:02 +02:00
Artturin
0a3d351240 splice.nix: run nixpkgs-fmt 
the file was a bit hard to read
 2022-12-24 15:55:22 +02:00
Artturin
341e6fd558 splice.nix: start deprecating nativeDrv and crossDrv 2022-11-19 00:04:54 +02:00
Ryan Burns
c47f991435 top-level: add depsHostHost splicing 
This was originally made to throw because pkgsHostHost was unimplemented.
Now that we have the full range of pkgs*, we can add this normally.
 2021-12-02 17:22:54 -08:00
John Ericson
70d71bbbe4 top-level: Create pkgs{Build,Host,Target}{Build,Host,Target} 
This is needed to avoid confusing and repeated boilerplate for
`fooForTarget`.  The vast majority of use-cases can still use
`buildPackages or `targetPackages`, which are now defined in terms of
these.
 2019-03-24 22:12:15 -04:00
Andrew Childs
47bf3f3d3f xorg: don't splice scope attributes into main package set 2018-10-11 15:03:07 -04:00
John Ericson
dc214ed99b __splicedPackages: Don't include xorg 
For historical reasons, callPackage has the xorg packages in scope. This
unstable building block is more versatile without the xorg hack baked in.
 2018-10-11 13:27:31 -04:00
John Ericson
0828e2d8c3 treewide: Remove usage of remaining redundant platform compatability stuff 
Want to get this out of here for 18.09, so it can be deprecated
thereafter.
 2018-08-30 17:20:32 -04:00
John Ericson
740cfffcb2 haskell infra: Fix built-time overrides 
These should all come from `buildHaskellPackages`
 2018-07-09 18:09:17 -04:00
John Ericson
f27f491784 Haskell infra: Do callPackage splicing 2018-07-09 16:39:37 -04:00
John Ericson
0eb142cd46 top-level: Expose splicePackages for newScope users 
Otherwise extra packages in scope can't be made to work for cross. As
much as I think splicing is an evil trick, I think it's best to do this
and at least have it work consistently for now.

It would seems simpler to expose a `newScopeWithSplicing`, but there's a
few attrs (like `buildPackages` or `buildHaskellPackages`) that
shouldn't be spliced. Users should instead splice, override the splicing
on those packages, and apply `newScope` to that.
 2018-07-09 12:33:43 -04:00
Will Dietz
218d4dc154 make-derivation: Don't add host-suffix to fixed-output derivations names 
Not only does the suffix unnecessarily reduce sharing, but it also breaks
unpacker setup hooks (e.g. that of `unzip`) which identify interesting tarballs
using the file extension.

This also means we can get rid of the splicing hacks for fetchers.
 2018-07-09 11:07:10 -04:00
John Ericson
bb18a3b573 top-level: Splice in more package sets for new types of deps 
This is done in preparation for the next commit where, among other
changes, I add support for the new `dep*` attributes.
 2017-12-30 22:04:20 -05:00
John Ericson
6c36d3c9e0 top-level: Fix splicing, again 2017-12-10 17:01:23 -05:00
John Ericson
965cc5da57 top-level: Fix splicing in the presence of evaluation errors 
- No more `or {}`, this was misleading at best since those values
   wouldn't be used unless the attr they are defined from was present
   anyways.

 - `tryEval` for get outputs. This ensures that if some derivations fail, they
   won't take out the others. This benefited from the `or {}` before, but that
   was never good enough. `tryEval` is strictly better.
 2017-12-09 20:02:45 -05:00
John Ericson
5ae8f18f4d Rename __targetPackages to targetPackages 2017-11-05 17:10:53 -05:00
Shea Levy
1a1ad1a17d Revert "Merge branch 'improved-make-overridable' of git://github.com/ElvishJerricco/nixpkgs" 
This reverts commit c3af1210b4, reversing
changes made to 49f175cd0c.
 2017-09-29 09:11:26 -04:00
Shea Levy
d96da5af87 Revert "Avoid polluting lib namespace unncessarily" 
Reverting #27319

This reverts commit 01a3f0b8aa.
 2017-09-29 09:09:53 -04:00
Shea Levy
01a3f0b8aa Avoid polluting lib namespace unncessarily 2017-09-28 18:51:20 -04:00
Shea Levy
c3af1210b4 Merge branch 'improved-make-overridable' of git://github.com/ElvishJerricco/nixpkgs 2017-09-28 18:10:50 -04:00
John Ericson
64aebc7679 stdenv: Also splice unlisted default "out" output 
Otherwise, some cross builds will use the wrong type of dep.

This won't affect native builds, and perhaps no extant cross buids
either.
 2017-09-19 14:48:23 -04:00
Will Fancher
05f9db601a Added self views of the interface in makeExtensibleWithInterface 
Fixing the `overrideScope` in `haskellpackages`.
 2017-07-14 06:43:33 -04:00
Will Fancher
af479c182f Added callPackageWithSelfWith and callPackageWithSelf 2017-07-12 18:35:23 -04:00
John Ericson
863d79b364 top-level: Introduce targetPackages and a "double link fold" 
Each bootstrapping stage ought to just depend on the previous stage, but
poorly-written compilers break this elegence. This provides an easy-enough
way to depend on the next stage: targetPackages. PLEASE DO NOT USE IT
UNLESS YOU MUST!

I'm hoping someday in a pleasant future I can revert this commit :)
 2017-04-23 14:01:12 -04:00
John Ericson
af8f87a3a1 top-level: Do not splice pkgs, buildPackages or *Platform 
- `pkgs` is self-similar, and thus already spliced
 - `buildPackages` is an ingredient of splicing and should be kept as is
 - The platforms are not packages or package sets and couldn't be spliced

There's probably other things that shouldn't be spliced too. The best long-
term solution is simply to stop splicing altogether.
 2017-04-23 13:54:42 -04:00
John Ericson
c869fe022e top-level: no more need to expose splicedPackages 
This was just done temporarily on the last cross-overhauling PR for
testing purposes.
 2017-01-25 09:24:55 -05:00
John Ericson
bfb147b6a8 top-level: Only splice as needed for performance 2017-01-24 11:37:56 -05:00