As is retrospectively obvious from the fact that it was looking at
pkgsTargetTarget, this should have been in the section for the target,
not the host. This fixes e.g. pkgsStatic.xq for 64-bit RISC-V.
(For platforms other than 64-bit RISC-V this being wrong was masked by
the musl Rust targets being static by default.)
This fixes a long standing issue where rust hooks behave differently when used inside buildRustPackage vs inside mkDerivation, which lead to surprising behavior, like for example the package being built for the wrong paltform or the linker not being found especially in cross compilation scenarios.
The reason for this inconsitency was, that buildRustPackage consumed the hooks in a non-spliced form, via [this inherit statement](4506ece030/pkgs/development/compilers/rust/make-rust-platform.nix (L60)), and therefore the usual platform shift on the hooks introduced by putting them in `nativeBuildInputs` was not applied here.
Thoug whenever the hook was used inside other builders like `mkDerivation` the platform shift did apply correctly as the hook was consumed via the spliced package set, introducing the inconsitecy.
Because of the wrong (non-spliced) use in buildRustPackage, most rust hooks have been designed with the wrong build/host/target shift in mind which is fixed by this change.
Due to the inconsitent behavior between different builders, workarounds like `rust.envVars`, which were previously introduced, likely become obsolete by this change.
This likely fixes a bunch of cross compilation issues for rust packages that are not based on `buildRustPackage` but instead consume the hooks directly.
Done:
- ensure that `buildRustPackage` consumes spliced hooks by using makeScopeWithSplicing' in make-rust-platform.nix.
- refactor hooks to make them refer to correct build/host/target packages.
- remove `rust.envVars` workaround from all rust hooks
- implement tests for most rust hooks in /pkgs/test/rut-hooks
The newly added tests can be executed for native as well as cross compilation via:
```
nix-build -A tests.rust-hooks -A pkgsCross.riscv64.tests.rust-hooks
```
We need this stuff to be available in lib so make-derivation.nix can
access it to construct the Meson cross file.
This has a couple of other advantages:
- It makes Rust less special. Now figuring out what Rust calls a
platform is the same as figuring out what Linux or QEMU call it.
- We can unify the schema used to define Rust targets, and the schema
used to access those values later. Just like you can set "config"
or "system" in a platform definition, and then access those same
keys on the elaborated platform, you can now set "rustcTarget" in
your crossSystem, and then access "stdenv.hostPlatform.rustcTarget"
in your code.
"rustcTarget", "rustcTargetSpec", "cargoShortTarget", and
"cargoEnvVarTarget" have the "rustc" and "cargo" prefixes because
these are not exposed to code by the compiler, and are not
standardized. The arch/os/etc. variables are all named to match the
forms in the Rust target spec JSON.
The new rust.target-family only takes a list, since we don't need to
worry about backwards compatibility when that name is used.
The old APIs are all still functional with no warning for now, so that
it's possible for external code to use a single API on both 23.05 and
23.11. We can introduce the warnings once 23.05 is EOL, and make them
hard errors when 23.11 is EOL.
Cargo will never need to link for the target platform — that'd be for
the package being built to do at runtime. Cargo should know about the
build and host linkers.
This fixes e.g. pkgsCross.musl64.fd from x86_64-linux.
Fixes: 67a4f828b4 ("rust: hooks: fix cross compilation")
> If using a target spec JSON file, the <triple> value is the filename
> stem. For example --target foo/bar.json would match [target.bar].
- https://doc.rust-lang.org/cargo/reference/config.html#target
I've also exposed toRustTargetSpecShort as a public function, because
it's useful to be able to know what the target subdirectory will be.
Currently there is a state of severe confusion in
pkgs/build-support/rust/hooks/ regarding host vs target; right now
there is only "host" defined, but whether it means "host" or
"target" seems to fluctuate.
This commit corrects that, ensuring that all variables come in all
three flavors (build, host, target) and are used consistently with
the nixpkgs convention.
This also fixes the cross-compilation of packages which use
`maturinBuildHook` -- hooks go in `nativeBuildInputs` and are
phase-shifted backwards by one platform, so they need to be careful
about distinguishing between build and host.
Closes#247441
If RUSTFLAGS is set in the environment, Cargo will ignore rustflags
settings in its TOML configuration. So setting RUSTFLAGS=-g (like
separateDebugInfo does) to generate debug info breaks
dynamically-linked Rust packages on musl. This breakage is visible
for any packages that call into C dynamic libraries. If the binary is
linked directly to a C dynamic library, it will fail to build, and if
it depends on a Rust library which links a C dynamic library, it will
segfault at runtime when it tries to call a function from the C
library. I noticed this because pkgsMusl.crosvm is broken for this
reason, since it sets separateDebugInfo = true.
It shouldn't be possible to end up with broken binaries just by using
RUSTFLAGS to do something innocuous like enable debug info, so I think
that, even though we liked the approach of modiyfing .cargo/config
better at the time, it's become clear that it's too brittle, and we
should bite the bullet and patch the compiler instead when targetting
musl. It does not appear to be necessary to modify the compiler at
all when cross-compiling /from/ dynamically-linked Musl to another
target, so I'm only checking whether the target system is
dynamically-linked Musl when deciding whether to make the modification
to the compiler.
This reverts commit c2eaaae50d
("cargoSetupHook: pass host config flags"), and implements the
compiler patching approach instead.
This change switches to using GCC 11 by default on aarch64-linux, as well as passing `-lgcc` to the linker, per #201485.
See #201254 and #208412 for wider context on the issue.
Tell rust if we want our binaries linked statically or dynamically.
Otherwise the compiler will always produce statically linked binaries for musl
targets, as this is the default.
API change:
`cargoParallelTestThreads` suggests that this attribute sets the
number of threads used during tests, while it is actually a boolean
option (use 1 thread or NIX_BUILD_CORES threads). In the hook, this
is replaced by a more canonical name `dontUseCargoParallelTests`.
- API change: remove the `target` argument of `buildRustPackage`, the
target should always be in sync with the C/C++ compiler that is used.
- Gathering of binaries has moved from `buildPhase` to `installPhase`,
this simplifies the hook and orders this functionality logically
with the installation logic.
