Previously the user-supplied package list was passed directly through
`lib.closePropagation` and an `isHaskellLibrary`-attribute filter, so
mistakes such as `[ dontCheck hp.foo ]` (forgetting parentheses around
the override) silently produced a GHC environment with the affected
package missing instead of failing.
Validate each top-level entry up front. We deliberately accept any
derivation (not just Haskell libraries), because shellFor passes
mixed inputs like `libraryFrameworkDepends` through here on Darwin;
those are still silently dropped by the existing closure-side filter.
The user-error case the issue is about -- non-derivation values like
functions or strings -- is now rejected with a helpful message that
points at the canonical fix.
Fixes#30304
Assisted-by: claude-code with claude-opus-4-8[1m]-high
LLVM 12–17 have been dropped for Nixpkgs 25.11. As discussed recently
on Matrix, this backports upstream changes to allow the use of
LLVM 20 for all GHC versions from 9.4.8 onward.
I looked over GHC commits mentioning LLVM since the release of 9.4.8,
and read the discussions and issues around the relevant bumps, and
attempted to be quite thorough, but I obviously cannot guarantee that
this is wholly comprehensive. It seems like upstream generally bumps
the upper bound on the basis of “it builds successfully for me”,
with specific adaptations for new versions being fairly uncommon and
only coming for obvious build blockers or reactively in response to
bug reports. I have backported both kinds of changes here.
For some commits, trivial conflict resolutions and adaptations
were required. It would be possible to pass the affected files to
`fetchpatch` as `excludes` and keep smaller fix‐up patches in tree in
some cases, but I opted to keep it simple and vendor complete backport
patches instead. I did not attempt to backport every single change to
the LLVM backend, only those that seemed directly relevant to support
for newer versions; if you’d get the same issue with the older LLVM,
that’s just a GHC bug.
These changes should actually make it easier to cross‐compile for
new architectures, as more recent LLVMs will have better support for
newer platforms, and it will be easier to backport GHC changes to
enable new platforms with less drift in the backend.
These patches do result in two breaking changes. Firstly, the minimum
LLVM version is bumped to 13 across the board. This is irrelevant for
Nixpkgs as we pin a specific LLVM version anyway, and versions below
LLVM 18 will be removed imminently. Secondly, support for the hidden
`-fno-llvm-tbaa` flag is dropped. This can be replaced with custom
`-optlo` flags to control the passes more directly, but the main
use of this undocumented flag appears to have been to [work around]
the lack of support for newer LLVM versions, anyway.
[work around]: <https://gitlab.haskell.org/ghc/ghc/-/issues/22220>
I successfully built the following on `aarch64-linux`:
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc948`
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc967`
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc984`
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc9102`
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc9121`
* `pkgsCross.armv7l-hf-multiplatform.buildPackages.haskell.compiler.ghc9122`
* `pkgsCross.riscv64.haskell.compiler.ghc948`
The GHC 9.4.8 with an ARMv7 host platform segfaults when I try to run
GHC, though e.g. `ghc-pkg --help` runs successfully. The GHC 9.10.3
build targeting ARMv7 crashed inside `llc(1)`, so I tried RISC‐V,
which has some platform mismatch issue relating to `libffi`, so I
tried z/Architecture, which failed with an invalid floating point
constant in the LLVM IR, so I tried 64‐bit MIPS, which failed with
a different `libffi` issue, so I tried 32‐bit MIPS, which failed
to compile `compiler-rt`, so I gave up. I confirmed that both of the
ARMv7 issues reproduce with 944e8fd4f4,
the revision before they were bumped from their old versions of LLVM,
so these are not regressions.
I built a test program with the ARMv7 cross‐compilers and
confirmed that they run on the AArch64 builder. I also confirmed
that the cross‐compiled RISC‐V GHC successfully runs under
`qemu-riscv64(1)`. It will only try to build programs via the C
backend, though, as that is the only option for unregisterised™
targets, so it’s not clear to me how useful LLVM support in 9.4.8
really is for bootstrapping new platforms; I guess even RISC‐V
would require more backporting work to produce a cross‐compiled
GHC that will use LLVM to compile its own input. I didn’t bother
setting up all the binfmt machinery to get it through compiling and
running a test program, but it at least makes the attempt.
Bump the LLVM version used by our GHC packages in preparation for
the LLVM drops. This breaks the LLVM backend; it will be fixed soon
by backporting patches, but that will require a mass rebuild.
- The LLVM backend always needs an LLVM specific assembler, i.e. clang
that ideally matches the version of LLVM actually used for codegen.
- The LLVM backend on Darwin requires some version of clang to be
available. In light of LLVMAS, using a matching version seems to be
best though this does risk messing with clang used for compiling C
in the derivation.
This mess can be avoided by compiling GHC with useLLVM = true which
sets absolute paths in GHC's settings file. Due to closure size
constraints, we can't really do that if NCG is available.
There’s no sensible notion of a Haskell package set’s LLVM, only a
GHC’s LLVM. `haskellPackages.ghc.llvmPackages` is still accessible,
although you probably don’t actually want to use it very often.
Co-authored-by: sternenseemann <sternenseemann@systemli.org>
This affects haskelPackages.mkDerivation, ghcWithPackages and
hoogleWithPackages which means that it is not possible to re-introduce
a ghcjs derivation downstream and create a ghcjs package set with an up
to date Nixpkgs.
Previously, `ghcWithPackages` would fail to build on macOS because
package `.conf` files now contain a literal `${pkgroot}` (which of
course does not exist):
```
$ grep -Poz "dynamic-library-dirs:\s*\K .+\n" /nix/store/s5xrmb3583fd5qjrqsg73fblaq46gqpd-ghc-9.10.1-with-packages/lib/ghc-9.10.1/lib/package.conf.d/*
...
/nix/store/s5xrmb3583fd5qjrqsg73fblaq46gqpd-ghc-9.10.1-with-packages/lib/ghc-9.10.1/lib/package.conf.d/unix-2.8.5.1-ce1b.conf.copy: ${pkgroot}/../lib/aarch64-osx-ghc-9.10.1
...
```
Therefore, we use `ghc-pkg --simple-output list` to get a list of
`.conf` files and `ghc-pkg --simple-output field "$pkg"
dynamic-library-dirs` to get the value of the field while expanding
`${pkgroot}` prefixes.
Package sets built with `haskellPackages.ghcWithHoogle` provide a Hoogle
executable which points to a Hoogle database, but there's no way to
programmatically access the derivation containing the Hoogle database
from the package set.
This adds a `passthru.hoogle` attribute to package sets built with
`haskellPackages.ghcWithHoogle`, and a `passthru.database` helper to
derivations built with `haskellPackages.hoogleWithPackages`.
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.
Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.
A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.
This commit was automatically created and can be verified using
nix-build a08b3a4d19.tar.gz \
--argstr baseRev b32a094368
result/bin/apply-formatting $NIXPKGS_PATH
When making ghcWithPackages, haskell package db configuration
is modified by sed, but newlines are not handled properly, so
resulting in
```
dynamic-library-dirs: ${pkgroot}/../lib/aarch64-osx-ghc-9.6.1
data-dir:
${pkgroot}/../share/aarch64-osx-ghc-9.6.1/filepath-1.4.100.1
```
is replaced like (data-dir: is swallowed)
```
dynamic-library-dirs:
/nix/store/zxmwvns8zg4ff47vw9hqgazjkiqa26kv-ghc-9.6.1-with-packages/lib/links
${pkgroot}/../share/aarch64-osx-ghc-9.6.1/filepath-1.4.100.1
```
This fixes the bug.
From GHC 9.6 on, the boot libraries (dependencies of the ghc library) are
present in a separate directory after the installation, and thus the
wrapper environment provided by ghcWithPackages needs to handle the links
to the boot dynamic libraries separately than other ordinary ones.
* Will make it so that GHC.Paths's docdir NIX_GHC_DOCDIR points to an
actual directory.
* Documentation of all packages in the environment is available in
`$out/share/doc`.
This has previously been attempted in #76842 and reverted in #77442,
since documentation can collide when the libraries wouldn't (thanks to
the hash in the lib filename). `symlinkJoin` allows collision, so this
solution should be akin to #77523 (minus `buildEnv`, one step at a
time). `installDocumentation = false` restores the old behavior.
Collision in the documentation only happen if the dependency closure of
the given packages has more than one different derivation for the same
library of the very same version. I'm personally inclined not to claim
that our infrastructure does anything sensible in this case.
Additionally, the documentation is likely largely the same in such
cases (unless it is heavily patched).
Resolves#150666.
Resolves#76837.
Closes#150968.
Closes#77523.
This is done for consistency with generic-builder.nix and because it's
easier for downstream users to replicate which will inevitably use our
code as inspiration.
Adding a fake override function via passthru will at least give users of
the old override interface a more helpful error message. Additionally we
also document the changes in the changelog.
This is achieved by passing the entire package set to the respective
wrappers and passing the select function as a second attribute. Together
with the new support for callPackage-ing functions this allows for
things like `ghcWithPackages.override { useLLVM = true; } (p: [ … ])`.
To make this possible for `ghcWithHoogle` as well, we need to make the
wrapper a bit more bespoke and inline the hoogle feature as well. The
hoogle wrapper, however, can remain separate and is exposed as
`hoogleWithPackages` additionally, as it can also serve standalone use.
`hoogleLocal` is kept for backwards compatibility (including the old,
suboptimal API), but will inform users about the better alternative via
a warning.
Since GHC now will have LLVM available when needed, we don't need to add
it in the wrapper anymore. It can still be added if NCG is available,
but -fllvm should be used (e. g. to work around an NCG bug).
useLLVM is what we are using in the GHC derivations already -- for
better or for worse -- so we should rename the argument here for
consistency which we are free to do as this is purely internal at the
moment (with overriding being impossible).
This check was wrong and caused by a bit of confusion on my part.
GHC >= 8.10.5 && < 9 supports aarch64-darwin via LLVM and GHC >= 9.2.1
introduces the NCG backend for aarch64-darwin.
Based on https://gitlab.haskell.org/ghc/ghc/-/wikis/platforms, although
it sadly doesn't list when the backends were introduced.
* PowerPC, x86 (and x86_64) and Sparc have been supported for longer.
* aarch64-darwin is new in 9.2.1 and backported to 8.10.5, 8.10.6 and
8.10.7 (check is dumb here since we'll grep for 8.10.7 anyways when
upgrading)
Fixes ghcWithPackages failing to evaluate on aarch64-darwin because of
missing support for the platform in LLVM 9's compiler-rt.
The availability of native codegen (which allows us to disable the LLVM
backend by default) hinges on the target platform of the compiler (that
is GHC), not on the platform it runs on (the host platform).
Now that `buildEnv` is ready, always put `makeWrapper` in
`nativeBuildInputs`, rather than `buildInputs` or (worse) mucking around
with setup hooks by hand.
(C.f. #112276, which didn't catch these because the manual setup hook
sourcing is such a hack to being with!)
Fixes#114687
While we are at it I:
* Disable tests, because I can‘t keep up with the speed hls introduces
more tests that cause trouble in nixpkgs.
* Fix builds of fourmolu and retrie
* Remove the wrapper for hls which is obsolete because of improved
package detection in hie-bios. And added a note that this can be removed
for ghcide soon, too.
`with-packages-wrapper.nix` has a hack to workaround the linker limit
in MacOS Sierra. However that is now broken with GHC 8.8, because of
slight change in the format of the package config.
In short, the package config produced by GHC 8.8 has a new line between
the key and list of values, while earlier versions have them separated
by a single space.
This PR fixes the linker hack by modifying the `grep` and `sed` commands
to pattern match on either space or new line, so that the hack can work
on all versions of GHC.
This closes#79441.
ghcWithPackages is using `ghc-pkg recache` to build its package
database. By doing so, it overrides the `package.cache[.lock]` files.
Details are unclear, but GHC 8.10 changed a bit the behavior.
Previously, it was unconditionally replacing the files by new ones. Now
it tries to open (for modification) the files. These files are symlinks
to another nix derivation, which is hence read-only.
This commit removes the files before running `ghc-pkg recache`, hence it
will just write the new files.
Tested with `haskellPackages.ghcWithPackages` (i.e. GHC 8.8) and
`haskell.packages.ghc8101.ghcWithPackages` (i.e GHC 8.10) with the
following nix file, at the root of the nixpkgs repository:
```
with import ./. {
overlays = [
(
self: super: {
haskellPackages = super.haskell.packages.ghc8101.override {
overrides = selfh: superh: {
th-lift-instances = super.haskell.lib.doJailbreak superh.th-lift-instances;
th-expand-syns = super.haskell.lib.doJailbreak superh.th-expand-syns;
th-reify-many = super.haskell.lib.doJailbreak superh.th-reify-many;
th-orphans = super.haskell.lib.doJailbreak superh.th-orphans;
haskell-src-meta = super.haskell.lib.doJailbreak superh.haskell-src-meta;
};
};
}
)
];
};
haskellPackages.ghcWithPackages(p:[p.PyF])
```
This will test with GHC 8.10. Comment out the `overlays` to test with
GHC 8.8.
