Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
In #296645, envFunc (and thus shellFor) was changed from
`mkDerivation {phases=[..]}` to `runCommand ..`. So since
then, the derivation of shellFor contains a `buildCommand`
("echo $nativeBuildInputs $buildInputs > $out"),
and as a result the custom phases defined in shellFor's
test were ignored.
Having unbuildable tests in the attrset makes it difficult to ask Nix to
build all the working ones at once. See discussion in #380683. So
instead of flagging them, let's just remove them entirely.
Some platforms implement permissions for symlinks, while others -
including Linux - ignore them. As a result, testing this hook's handling
of unreadable symlinks requires careful attention to which kind of
platform we're on. See the comments by `meta.badPlatforms` for details.
This protects the rest of the standard setup hooks from unreadable and
other broken symlinks. (Otherwise, they can choke on the `readlink` step
and fail silently before `no-broken-symlinks.sh`` is even reached.)
GitHub apparently immediately terminates the connection for our previous
bogus curl user agent. Let's be more polite about this (default curl
user agent would work, too).
This also changes stdenv's substitute --replace-fail to error out when
the replacement is the same as the search pattern, but can't be found.
This should not cause any problems in existing code, from what I can
tell from grepping nixpkgs.
The exception for pattern==replacement was previously introduced all the
way back in 5ff872aa24, but this was
apparently only used to make the check for the warning "simpler".
This fixes a long standing issue where rust hooks behave differently when used inside buildRustPackage vs inside mkDerivation, which lead to surprising behavior, like for example the package being built for the wrong paltform or the linker not being found especially in cross compilation scenarios.
The reason for this inconsitency was, that buildRustPackage consumed the hooks in a non-spliced form, via [this inherit statement](4506ece030/pkgs/development/compilers/rust/make-rust-platform.nix (L60)), and therefore the usual platform shift on the hooks introduced by putting them in `nativeBuildInputs` was not applied here.
Thoug whenever the hook was used inside other builders like `mkDerivation` the platform shift did apply correctly as the hook was consumed via the spliced package set, introducing the inconsitecy.
Because of the wrong (non-spliced) use in buildRustPackage, most rust hooks have been designed with the wrong build/host/target shift in mind which is fixed by this change.
Due to the inconsitent behavior between different builders, workarounds like `rust.envVars`, which were previously introduced, likely become obsolete by this change.
This likely fixes a bunch of cross compilation issues for rust packages that are not based on `buildRustPackage` but instead consume the hooks directly.
Done:
- ensure that `buildRustPackage` consumes spliced hooks by using makeScopeWithSplicing' in make-rust-platform.nix.
- refactor hooks to make them refer to correct build/host/target packages.
- remove `rust.envVars` workaround from all rust hooks
- implement tests for most rust hooks in /pkgs/test/rut-hooks
The newly added tests can be executed for native as well as cross compilation via:
```
nix-build -A tests.rust-hooks -A pkgsCross.riscv64.tests.rust-hooks
```
When using pkgsCross with a system that ends up the same as the
localSystem, then modifications for package sets like pksgMusl need to
be done for **both** localSystem and crossSystem. Consider the following
on x86_64-linux:
pkgsCross.gnu64.pkgsMusl
Before this change, this would result in a musl buildPlatform, but a gnu
hostPlatform. This breaks the promise of "stacking" package sets on top
of each other.
After this change, it results in a musl buildPlatform and a musl
hostPlatform. This works better.
One could expect this to result in the same as pkgsCross.musl64, i.e. a
gnu buildPlatform and a musl hostPlatform, however I couldn't get this
to work without increasing memory usage for ci/eval by many, many GB.
This is caused by usage of pkgsi686Linux inside the main package set,
which follows the same hybrid pattern.
The various pkgsXYZ top-level package sets did not pass localSystem /
crossSystem to lower levels, so far. This change propagates original
arguments to lower levels, which include the overrides made by an upper
package sets.
There is an extensive test-suite to test various combinations of package
sets in pkgs/test/top-level. There are a few basic promises made:
- Package sets must be idempotent. pkgsMusl.pkgsMusl === pkgsMusl.
- Once pkgsCross is used any subsequent package sets should affect the
**host platform** and not the build platform. Examples:
- pkgsMusl.pkgsCross.aarch64-multiplatform is a cross compilation from
musl to glibc/aarch64
- pkgsCross.aarch64-multiplatform.pkgsMusl is a cross compilation to
musl/aarch64
- Modifications from an earlier layer should not be lost, unless
explicitly overwritten. Examples:
- pkgsStatic.pkgsMusl should still be static.
- pkgsStatic.pkgsCross.gnu64 should be static, but with glibc instead
of musl.
Exceptions / TODOs:
- pkgsExtraHardening is currently not idempotent, because it applies the
same flags over and over again.
Supersedes #136549Resolves#114510Resolves#212494Resolves#281596
This adds some basic tests to compose package sets. The cases that are
currently broken
are commented out, they include things like:
- pkgsStatic.pkgsMusl losing the isStatic flag
- pkgsCross.ppc64-musl.pkgsMusl losing the gcc.abi setting
- pkgsCross.mingwW64.pkgsStatic losing the config string
- pkgsLLVM.pkgsMusl losing the useLLVM flag
- pkgsLLVM.pkgsStatic losing the useLLVM flag
- pkgsLLVM.pkgsi686Linux losing the useLLVM flag
And probably more.
It's not the longest-open PR (May 28, 2023 -> Jan 22, 2025) but it took a while. This PR introduces a unified approach to implementing build helpers that support fixed-point arguments and bring such support to existing build helpers.
Split tests in the let-in block into
tests-stdenv, tests-go, and tests-python.
Place the dependent variables into the let-in block of each category.
This chore commit causes no rebuilds.
