MLIRConfig.cmake unconditionally overwrites MLIR_TABLEGEN_EXE and does
not create an imported target, breaking standalone builds that provide
their own mlir-tblgen binary (e.g. Nix sandboxed builds).
The patch adds guards to respect caller-set MLIR_TABLEGEN_EXE,
MLIR_PDLL_TABLEGEN_EXE, and MLIR_SRC_SHARDER_TABLEGEN_EXE values, and
auto-creates an imported mlir-tblgen target for downstream consumers.
This replaces the previous dummy-target workaround in flang's
CMakeLists.txt.
Upstream issue: https://github.com/llvm/llvm-project/issues/150986
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
- add patch from 2 upstream commits that were not backported to llvm_18
and llvm_19:
41eb186fbb
Fixes build failure with gcc15:
```
In file included from /build/mlir-src-18.1.8/mlir/lib/Dialect/Affine/IR/ValueBoundsOpInterfaceImpl.cpp:9:
/build/mlir-src-18.1.8/mlir/include/mlir/Dialect/Affine/IR/ValueBoundsOpInterfaceImpl.h:31:11:
error: 'int64_t' was not declared in this scope
31 | FailureOr<int64_t> fullyComposeAndComputeConstantDelta(Value value1,
| ^~~~~~~
/build/mlir-src-18.1.8/mlir/include/mlir/Dialect/Affine/IR/ValueBoundsOpInterfaceImpl.h:13:1:
note: 'int64_t' is defined in header '<cstdint>'; this is probably
fixable by adding '#include <cstdint>'
12 | #include "mlir/Support/LogicalResult.h"
+++ |+#include <cstdint>
13 |
```
101109fc54
Fixes failure with gcc15:
```
In file included from /build/mlir-src-19.1.7/mlir/lib/Target/SPIRV/Deserialization/Deserialization.cpp:9:
/build/mlir-src-19.1.7/mlir/include/mlir/Target/SPIRV/Deserialization.h:29:51:
error: 'uint32_t' was not declared in this scope
29 | OwningOpRef<spirv::ModuleOp> deserialize(ArrayRef<uint32_t> binary,
| ^~~~~~~~
/build/mlir-src-19.1.7/mlir/include/mlir/Target/SPIRV/Deserialization.h:18:1:
note: 'uint32_t' is defined in header '<cstdint>'; this is probably
fixable by adding '#include <cstdint>'
17 | #include "mlir/Support/LLVM.h"
+++ |+#include <cstdint>
18 |
```
First patch can be applied with `fetchpatch`, but only on llvm_19, because
of changed include line right above.
So use 2 versions of combined patches with 2 commits.
Mark these packages as big-parallel to match llvmPackages.llvm.
They are chosen because they are relatively expensive to build, as
opposed to others where I currently believe they are less of an issue
from a CPU time/memory standpoint.
I suspect most people don't care about big-parallel or it would have
been noticed earlier. I have a use case where I am building many
variants of clang all at once and I want to avoid OOM by constraining
the number of parallel builds of items marked big-parallel.
Signed-off-by: Peter Waller <p@pwaller.net>
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
mlir piggybacks onto LLVM_BUILD_LLVM_DYLIB for determining if a
shared libMLIR.so is emitted. This also fixes build failures
in MLIR 20+ where tablegen'd files aren't generated.
In file included from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/UseDefLists.h:16,
from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/Value.h:17,
from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/BlockSupport.h:16,
from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/OperationSupport.h:18,
from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/Dialect.h:17,
from /build/mlir-src-21.0.0-unstable-2025-03-09/mlir/tools/mlir-lsp-server/mlir-lsp-server.cpp:9:
/build/mlir-src-21.0.0-unstable-2025-03-09/mlir/include/mlir/IR/Location.h:145:10: fatal error: mlir/IR/BuiltinLocationAttributes.h.inc: No such file or directory
145 | #include "mlir/IR/BuiltinLocationAttributes.h.inc"
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
Background:
LLVM has some tools that run at build time. In native builds, these are
built as a part of the usual build, but in cross builds they need to
come from buildPackages.
In many scenarios this is a small problem because LLVM from
buildPackages is already available as a build; but if cross building a
version of LLVM which is not available (e.g. a new git commit of LLVM)
this results in two builds of LLVM and clang, one native and one for the
cross.
Full builds of LLVM are expensive; and unnecessary in this scenario. We
don't need a native LLVM, only a native copy of the tools which run at
build time. This is only tablegen and related tooling, which are cheap
to build.
Implementation-wise, we introduce a derivation llvmPackages.tblgen,
which specifies the tablegen targets which need to be built and has a
custom installPhase to copy them to the output.
A previous attempt in https://github.com/NixOS/nixpkgs/pull/359967
dropped the use of LLVM_TABLEGEN_EXE and friends on the grounds that
llvm can already cross build these things, but that is false since it's
necessary in that case to wire in a cross compiler. This PR avoids that
problem by allowing access to buildPackages.tblgen.
Signed-off-by: Peter Waller <p@pwaller.net>
Also exposes owner, repo, and other attributes available on the original
monorepoSrc. These attributes were previously hidden behind a
runCommand that consumes monorepoSrc and hides this information away.
```
$ nix eval --raw .#llvmPackages_18.llvm.src.rev
llvmorg-18.1.8
$ nix eval --raw .#llvmPackages_git.llvm.src.rev
0e8555d4dbfdfeddc01dc2ecf9a9b6e804f7b645
```
Signed-off-by: Peter Waller <p@pwaller.net>
This change implements a leftover task from #307211, namely passing
monorepoSrc to the different llvmPackages_13 package expressions. Before
this change, all packages llvmPackages_13 would be built from a
subdirectory of the full LLVM monorepo tree. After this change only the
relevant directories are made available at build time. This
- reduces the size of the source that needs to be made available to the
builder.
- prevents LLVM from sidestepping our instructions and including extra
sources from other directories it shouldn't.
Since LLVM 12 and 13 don't have the `cmake` directory at the top level,
the runCommand expressions filtering the source need to be adjusted, but
this causes no rebuild for any other LLVM version (ofborg should confirm
this).
The only problem encountered was in lld:
- We need to make the patch to the inclusion of libunwind headers
unconditional now. lld needs this on non-darwin as well. In the
full monorepo, LLVM_MAIN_SRC_DIR would be set correctly, so the
patch wasn't necessary.
- The substitute mechanism for LLVM 12 and 13 can't be unified yet since
LLVM 12 still uses a non monorepo build, so we come up with a
different LLVM_MAIN_SRC_DIR.
Change was tested by building the following expression on x86_64-linux.
with import ./. {};
builtins.removeAttrs llvmPackages_13 [ "lldb" "lldbPlugins" ]'
lld was also tested on aarch64-darwin.
In preparation for the deprecation of `stdenv.isX`.
These shorthands are not conducive to cross-compilation because they
hide the platforms.
Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way
One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059
There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.
```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
cmake flags have a 'last flag wins' logic, so by appending to the end of
the flags it is possible to override any cmake flag.
It also ignores (and warns) if a flag is unused, so passing flags across
all packages should be safe if you want to target one package.
In combination with #320261, this PR allows consistently overriding all
packages within LLVM with additional cmake arguments. Consistency here
means for example 'if you override LLVM, then all dependencies on it are
also see the overridden LLVM in their input'. Consistency is hard to
achieve with the other obvious way of implementing this as a user: if
you use overrideAttrs then you have to write a big mess of override code
in order to override all dependents, and this can be very difficult in a
cross-compilation scenario using crossSystem and useLLVM, for example.
With this PR it is possible to write an overlay which overlays
`llvmPackages` with `llvmPackage.override { devExtraCmakeFlags = [ ... ]; }`,
and then the toolchain used with useLLVM in effect should respect
these flags.
This is useful in development for experimenting with the effect of
various flags, hence the chosen name `devCmakeFlags`.
This won't work out of the box without #341855 applied, which fixes
override passthrough.
See-Also: #320261, #341855
Signed-off-by: Peter Waller <p@pwaller.net>
