These features are internal-only, have special characters that bash
doesn't support in variable names, and aren't normally given
environment variables by cargo as far as I can tell.
(cherry picked from commit ede639a8d6)
(cherry picked from commit a94a643de9)
We are replicating one mechanism behind `-Z build-std`.
There isn't yet crate2nix support for this, but one can (and I do) add
the missing stdlib deps (for this feature to pick up) with overrides.
(cherry picked from commit cc29693a09)
(cherry picked from commit 5aef865cef)
(cherry picked from commit c19d19615c)
It is possible to both be bare metal and have a libc (newlib).
This libc doesn't provide very much --- not enough for CMake to think
the C toolchain works. We therefore adjust our logic so we hit the "bare
metal" case with or without libc.
The "use LLVM" bootstrap is intentionally not affected.
(cherry picked from commit bf39e32272)
(cherry picked from commit c820cd8cee)
(cherry picked from commit 0a284ae9b9)
autoPatchelfHook actually doesn't depend on stdenv and only needs
bintools (with its wrapper). This change uses $NIX_BINTOOLS instead of
$NIX_CC and makes the dependency on bintools explicit.
(cherry picked from commit a7f5e8321e)
Conflicts:
pkgs/top-level/all-packages.nix
Fully enabling crossSystem support for autoPatchelfHook came with some
perhaps unintended consequences of being a bit more aggressive about
patching ELF files from architectures/ABIs that differ from the target
(previously, those files would be ignored because ldd usually couldn't
handle them).
This change adds architecture and rough OS ABI detection to the script
so that it doesn't try to blindly replace the interpreter of files that
can't possibly use that interpreter, and also makes sure it doesn't
accidentally use libraries of other architectures/ABIs.
(cherry picked from commit 4765a3e153)
In #84415, autoPatchelfHook was taught to use the correct path to the
readelf binary when a crossSystem is specified. Unfortunately, the
remainder of the functionality in the script depended on ldd, which only
reads ELF files of its own architecture. It has the further unfortunate
quality of not reporting any useful error, but rather that the file is
not a dynamic executable.
This change uses patchelf to directly analyze the DT_NEEDED tags in the
target files instead, which correctly works across architectures. It
also updates the use of objdump to be prefix-aware $OBJDUMP (which would
have been required in the PR mentioned above, but we never made it that
far into the script execution).
(cherry picked from commit b79483d2b7)
IBM publishes their IBM Spectrum Protect client
for Linux in two flavors:
* "Linux x86_64 client"
* "Linux x86_64 Ubuntu client"
Up to this commit, nixpkgs used the Ubuntu
flavor to build its `tsm-client` derivation.
However, the history of published archive files in
* https://public.dhe.ibm.com/storage/tivoli-storage-management/maintenance/client/v8r1/Linux/
* https://public.dhe.ibm.com/storage/tivoli-storage-management/patches/client/v8r1/Linux/
suggests that updates in the fourth level of
the version numbers (e.g. 8.1.13.0 -> 8.1.13.1)
do not get published as Ubuntu flavor.
It order to be able to always use the latest release,
this commit switches to the non-Ubuntu flavor.
The non-Ubuntu archive contains rpm files,
so this commit switches from `ar` to `rpmextract`.
Instead of unpacking all deb files,
the build recipe now unpacks all _but one_ rpm file:
The file `TIVsm-WEBGUI.x86_64.rpm` apparently
contains a plugin that is not included
in the Ubuntu version (see note below).
Comparing the old and the new derivation's output indicates
that this choice minimizes the difference between the results:
The output of the old (Ubuntu flavor) derivation contains:
* `commons-codec-1.6.jar`
* `share/` with changelog and copyright information
for the packages `gskssl64` and `gskcrypt64`
The output of the new (non-Ubuntu flavor) derivation contains:
* `lib64`, symlink to `lib`
* `commons-codec-1.14.jar`
* `opt/tivoli/tsm/license/{api,baclient}/sm/`
with license agreement files in many languages
Besides these differences, the outputs' file names are equal.
Note: I don't know what functionality
`TIVsm-WEBGUI.x86_64.rpm` actually provides.
Unpacking it with the other rpm files makes patchelf complain
about missing X11 libraries, so in order to include it here,
one would likely need to add those to `buildInputs`.
However, as the old (Ubuntu flavor) `tsm-client` package
did not contain this functionality and as I cannot test
or use it in any way, I opted to not include it now.
If we want to include this with a later commit,
we should add another package build option (like `enableGui`)
so that the default `tsm-client` package does not pull in
X11 libraries and its closure size therefore stays small.
cherry-picked/adapted from https://github.com/NixOS/nixpkgs/pull/138386
IBM has changed the URL structures of their support web pages.
The commit at hand updates most URLs and
in particular the package update instructions
so they follow the new structure.
It also calculates the source download URL from the
version number, so package updates no longer have to
update the URL in addition to the version string.
cherry-picked from https://github.com/NixOS/nixpkgs/pull/138386
The build was failing due to mismatched libstdc++ between
stdenv.cc and cudatoolkit.cc. This can be fixed by bumping
the build-time cudatoolkit to 11, which is able to use the
same cc as stdenv.
(cherry picked from commit e263bdd82b)
by default a ps/2 keyboard input is used which seems to cause issues
on aarch64-linux when the machine is used high load, causing the keymap
qwertz test to always fail and azerty to sometimes fail
See https://github.com/NixOS/nixpkgs/issues/147294
(cherry picked from commit 39c5525cb1)
Snapserver expects the arguments `--tcp.bind_to_address` and
`--http.bind_to_address` instead of the `--tcp.address` (and http
equivalent) versions.
This caused the process to listen on `0.0.0.0` (for TCP and HTTP
sockets) regardless of the configuration value. It also never listend on
the IPv6 address `::` as our module system made the user believe.
This commit fixes the above issue and ensures that (at least for the TCP
socket) that our default `::` does indeed allow connections via IPv6
(to localhost aka ::1).
(cherry picked from commit c9c93b0add)
This seems to break
`boot.kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages`
causing it to use `linuxPackages`.
(cherry picked from commit e6ca3fc976)
The config script does that automatically for a few architectures [1],
but on 64-bit platforms that are not listed (like riscv64) the
freebl build fails. Debian always adds the USE_64=1 flag when
compiling on 64-bit architectures (they use legacy make instead of gyp),
and we should do that as well to fix the general problem at the cost of
a mass rebuild.
[1] 0ef2306a62/coreconf/config.gypi (l212)
[2] c446c61808/debian/rules (L66)
(cherry picked from commit 345e777888)
This fixes the qemu-kvm wrapper we add for convenience
silently not using KVM, when the system would support it
by at least leaving an indication in the log that the build ran
slower because it ran without KVM.
(cherry picked from commit 5718276542)
KVM should only be considered abailable if /dev/kvm exists and
is read-writable by the user that is trying to launch it.
The previous check for existance only had the consequence that
on some Linux distributions running VMs with Nix's QEMU only worked
if KVM was NOT installed.
fixes#124371
(cherry picked from commit 046392279c)
This does not update to the latest version from the 2.5.x range,
since the build fails with that. It appears to be trying to access
an url, which is not allowed in the sandbox.
Since this is just a drive-by, I think this is better than keeping
all those packages broken on hydra, since the build is broken with 2.5.1
After linux 5.14.11 FB_SIMPLE conflicts with DRM_SIMPLEDRM, which
will fail configuration, when DRM_SIMPLEDRM is configured as a module
and FB_SIMPLE gets requested as builtin.
Do not enable DRM_SIMPLEDRM as a temporary workaround, until good
enough migration path is found.
(cherry picked from commit 2ef28fb77d)
Some upstream changes broke the automatic fallback to SwiftShader.
Without this workaround the GPU initialization fails (apparently due to requiring Vulkan):
machine # libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
machine # [1001:1001:1101/104304.000625:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # libva error: vaGetDriverNameByIndex() failed with unknown libva error, driver_name = (null)
machine # [1052:1052:1101/104305.060496:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # [1084:1084:1101/104305.165898:ERROR:angle_platform_impl.cc(44)] Display.cpp:894 (initialize): ANGLE Display::initialize error 0: Internal Vulkan error (-3): Initialization of an object could not be completed for implementation-specific reasons, in ../../third_party/angle/src/libANGLE/renderer/vulkan/RendererVk.cpp, initialize:1048.
machine # [1084:1084:1101/104305.171191:ERROR:gl_surface_egl.cc(782)] EGL Driver message (Critical) eglInitialize: Internal Vulkan error (-3): Initialization of an object could not be completed for implementation-specific reasons, in ../../third_party/angle/src/libANGLE/renderer/vulkan/RendererVk.cpp, initialize:1048.
machine # [1084:1084:1101/104305.178707:ERROR:gl_surface_egl.cc(1382)] eglInitialize SwANGLE failed with error EGL_NOT_INITIALIZED
machine # [1084:1084:1101/104305.180111:ERROR:gl_ozone_egl.cc(20)] GLSurfaceEGL::InitializeOneOff failed.
machine # [1084:1084:1101/104305.189760:ERROR:viz_main_impl.cc(161)] Exiting GPU process due to errors during initialization
machine # [1092:1092:1101/104305.233470:ERROR:gpu_init.cc(457)] Passthrough is not supported, GL is disabled, ANGLE is
This workaround restores the previous result:
machine # [1004:1004:1101/104551.131190:ERROR:gpu_init.cc(457)] Passthrough is not supported, GL is swiftshader, ANGLE is
The workaround is only required for Chromium, not Google Chrome.
(cherry picked from commit a188a74486)
Make Qt applications work on macOS Big Sur even if they're built with
an older version of the macOS SDK (<10.14 - we're currently using
10.12). This issue is fixed in 5.12.11, but it requires macOS SDK
10.13 to build. See https://bugreports.qt.io/browse/QTBUG-87014 for
more info.
(cherry picked from commit 39ce18a7ec)
This is important since legacy bios mode is still the default for Intel
and AMD based instances on AWS. That is, even if your image is setup to
use UEFI on the OS level, the AMI will still use BIOS unless the boot
mode is explicitly set during registration.
(cherry picked from commit ed4170733c)
firefox nightly now requires libXtst.so.6
without this change firefox nightly from the mozilla overlay fails to
start
added libXrandr too since libxul.so requires it
libXrandr.so.2 => not found
libXtst.so.6 => not found
(cherry picked from commit d9c66e55a0)
- `toRustTarget` and friends pulled out from rust tools into rust
library. Since they don't depend on any packages they can be more
widely useable.
- `build-rust-package` gets its own directory
- `fetch-cargo-tarball` gets its own directory
(cherry picked from commit 18ed048c7b)
This is a browser, the 78 ESR series is end of life, so we can expect
this browser to be a security vulnerability any day.
Recommend everyone to move to ESR 91.
Until now we merged kernel updates even if no hardened versions were
available yet. On one hand we don't want to delay patch-level updates,
on the other hand users of hardened kernels have frequent breakage now[1].
This change aims to provide a solution this issue:
* The hardened patchset now references the kernel version it's released
for (including a sha256 hash for the fixed-output path of the source
tarball).
* The `hardenedKernelFor`-function doesn't just append hardened patches
now, but also overrides version & src to match the kernel version the
patch was built & tested for.
Refs #140281
[1] https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.linuxPackages_hardened.kernel.x86_64-linux/all
(cherry picked from commit bb5aa0109b)
Now there are a few more folks who should get pinged on kernel changes:
$ nix-instantiate -E 'with import ./. {}; (map (x: x.github) linux.meta.maintainers)' --eval --strict
[ "TredwellGit" "mweinelt" "ma27" "nequissimus" "alyssais" "thoughtpolice" ]
Refs #140281
(cherry picked from commit 65930caffe)
Previously the code took the kernelPatches of the final derivation, which
might or might not be what was passed to the derivation in the original call.
The previous behaviour caused various hacks to become neccessary to avoid duplicates in kernelPatches.
(cherry picked from commit 436f61c878)
Commit 5395397f removed the assertions from the buildMachines to
ensure that either system or systems is set for each buildmachine.
This patch re-implements those assertions.
The symptom is that if both system and systems are omitted, then the
/etc/machines file has the wrong number of columns and any attempt to
run a `nix` operation that has to perform a build will fail with a
`strtoull` exception.
(cherry picked from commit 58921a4904)
Change the path to the public directory reported by Discourse
to its real path instead of the symlink in the store, since
the store path won't be matched by any nginx rules.
Fixes#142528.
(cherry picked from commit 1fb77e822b)
The latest version of Subsonic (6.1.6) does not suport Java SE 9 or later
because it depends on the JAXB APIs. Those are considered to be Java EE
APIs are no longer contained on the default classpath in Java SE 9 and
are completely removed in Java SE 11..
(cherry picked from commit 14c5fe8c1b)
youtube-dl development seems to have stalled and yt-dlp is the de-facto upstream
nowadays. This provides an easy way to use yt-dlp as a drop-in replacement:
youtube-dl = yt-dlp.override { withAlias = true; };
* teamviewer: fix issue #96633
Fix teamviewer's breakage post 15.5.3 -> 15.15.5.
Teamviewer client was no longer able to connect to its backing
server as it now uses dbus to do so. Following changes were
required:
- add missing dbus and polkit service/policy files to package.
- add missing dbus lib to `LD_LIBRARY_PATH`.
Changes to the nixos module as a separate changeset.
(cherry picked from commit 506966d156)
* nixos/teamviewer: fix issue #96633
Add teamviewer package as a dbus package now that the
client / server communication depends on dbus.
(cherry picked from commit 200e959995)
* nixos/teamviewer: fix issue #44307
Move to a forefront launch of the daemon. Doing so allowed us
to move the service from forking to simple to avoid the
missing pid error log.
Also:
- Make the dbus dependency explicit.
(cherry picked from commit 953bbc0d73)
* teamviewer: 15.15.5 -> 15.18.5
Upgrading to the last version still using qt5.14. Later version
will be using qt5.15 which is not in 21.05 stable branch.
This fixes us the crash observed in 15.15.5 when stopping
the service.
(cherry picked from commit db889eb913)
* teamviewer: refactor executable wrapping
This centralizes `PATH` and `LD_LIBRARY_PATH`, avoid multiple
layers of wrappers.
Refactor as suggested by @Artturin in PR provided patch:
<https://github.com/NixOS/nixpkgs/pull/140076#issuecomment-934770391>.
(cherry picked from commit c55bc5bfd3)
* teamviewer: fix 97148 (busybox installed issue)
Simply add `coreutils` as a runtime dependency which will
prevent teamviewer from using incomplete busybox implementation
of expected gnu binaries.
As suggested by @Artturin in PR comment:
<https://github.com/NixOS/nixpkgs/pull/140076#issuecomment-934770391>.
(cherry picked from commit 4fb188e1d1)
* teamviewer: 15.18.5 -> 15.22.3
Required move from libsForQt514 -> libsForQt515.
Note that this changset won't be backportable to 21.05.
(cherry picked from commit 975ab7f3a0)
Co-authored-by: Raymond Gauthier <jraygauthier@gmail.com>
This reverts commit f182b8d23b.
This doesn't need to be on 21.05, because 21.05 doesn't have MariaDB
10.6.
It shouldn't be on 21.05 because the package version warning mentions
21.11, which is confusing and misleading.
The `$(</path/to/file)`-expansion appears verbatim in the cmdline of
`nextcloud-occ` which means that an unprivileged user could find
sensitive values (i.e. admin password & database password) by monitoring
`/proc/<pid>/cmdline`.
Now, these values don't appear in a command line anymore, but will be
passed as environment variables to `nextcloud-occ`.
(cherry picked from commit 9f37d6aee0)
calibre-web no longer starts without proper calibre DB path configured,
so the default testcase (completely unconfigured) is removed.
(cherry picked from commit 80f7656229)
Due to the old root certificate used by letsencrypt expiring, arcanist
could not connect anymore to servers using letsencrypt (like
reviews.llvm.org).
Fix it by using the default nix certificates.
(cherry picked from commit 93eb7786c9)
The specific reason to update is to pull in logging fix:
c6718a9d950214 "Don't reset the logger in a vfork"
Otherwise 'nix build' does not report build progress correctly.
(cherry picked from commit 5323bc0f14)
The Mozilla Trademark Policy is not a license, and applies equally to
our own source builds of Firefox, so it doesn't make sense to mark the
package as unfree because of that.
Quoting <about:license>:
> Binaries of this product have been made available to you by the
> Mozilla Project under the Mozilla Public License 2.0 (MPL).
Since all this does is download a large binary and wrap it, there's
still no point in it ending up in cache.nixos.org, so disable it on
Hydra now that Hydra would otherwise try to build it.
(cherry picked from commit 6a71c7eb1a)
With this change I can do
metrics:
enabled: true
listen_port: 8080
and retrieve metrics from `localhost:8080` for the telegram bridge.
(cherry picked from commit 9f9e32238b)
GHC 8.10.7 is only in 21.05 for bootstrapping GHCJS, so it shouldn't be
included in the hls wrapper. This change was a result of an overeager
backport and we'll revert it here.
Also rollback some unnecessary change to release-haskell.nix, so it can
be used for testing evaluation still.
This ensures CI tests the nano variant. It also makes it easier to use
with `libc = "newlib-nano";` in a platform description.
(cherry picked from commit e3ff6189f6)
Discourse normally overrides the default notification email setting,
which makes the `notificationEmailAddress` setting ineffective. Add a
patch to remove this override.
Fixes#140114.
(cherry picked from commit 917a0cfe47)
The version update has been cherrypicked from master where bundle has
been updated to 2.2.20. It was then failing to run with:
./result/bin/foreman
Warning: the running version of Bundler (2.1.4) is older than the version that created the lockfile (2.2.20). We suggest you to upgrade to the version that created the lockfile by running `gem install bundler:2.2.20`.
Traceback (most recent call last):
2: from ./result/bin/foreman:20:in `<main>'
1: from /nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:413:in `block in replace_bin_path'
/nix/store/ggqacj06n6qfm1iww0bih9ph0j89wcna-bundler-2.1.4/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/rubygems_integration.rb:374:in `block in replace_bin_path': can't find executable foreman for gem foreman. foreman is not currently included in the bundle, perhaps you meant to add it to your Gemfile? (Gem::Exception)
In $out/share it is NOT picked up into /etc/systemd/... when psi-notify is
added to configuration.systemd.packages .
(Originally put in $out/share b/c a hook copies from $out/lib into there
... so why not just put there? This is why.)
(cherry picked from commit a774af3605)
The MariaDB version 10.6 doesn't seem supported with current Nextcloud
versions and the test fails with the following error[1]:
nextcloud # [ 14.950034] nextcloud-setup-start[1001]: Error while trying to initialise the database: An exception occurred while executing a query: SQLSTATE[HY000]: General error: 4047 InnoDB refuses to write tables with ROW_FORMAT=COMPRESSED or KEY_BLOCK_SIZE.
According to a support-thread in upstream's Discourse[2] this is because
of a missing support so far.
Considering that we haven't received any bugreports so far - even though
the issue already exists on master - and the workaround[3] appears to
work fine, an evaluation warning for administrators should be
sufficient.
[1] https://hydra.nixos.org/build/155015223
[2] https://help.nextcloud.com/t/update-to-next-cloud-21-0-2-has-get-an-error/117028/15
[3] setting `innodb_read_only_compressed=0`
(cherry picked from commit 675e262f5a)
appimage-exec.sh parses its arguments with getopts, so we need to
delimit arguments intended for the wrapped executable with ‘--’, in
case some of them begin with ‘-’.
Without this fix, a wrapped application like Zulip Desktop can’t be
opened the normal way using the .desktop file, which includes
‘Exec=zulip --no-sandbox %U’ (as per the electron-builder default):
$ gtk-launch zulip.desktop
/usr/bin/appimage-exec.sh: illegal option -- -
Usage: appimage-run [appimage-run options] <AppImage> [AppImage options]
[…]
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
(cherry picked from commit b5160bba86)
2.30 can's paste to `dpaste` (default driver). 2.32 has it fixed
by using v2 endpoint. While at it switch homepage to github project.
(cherry picked from commit 17927dee09)
* chromedriver: add dbus to libraries
It is apparently required since version 94.
Fixes issue #139547
(cherry picked from commit 433b2bc444)
* chromedriver: remove extraneous LD_LIBRARY_PATH in wrapProgram
wrapProgram already prepends passed value to the specified environment
variable; no need to specify it explicitly.
(cherry picked from commit 38ddfe7f3f)
* chromedriver: add a package test checking the reported version
(cherry picked from commit 1106f4bafc)
* Update pkgs/development/tools/selenium/chromedriver/default.nix
(cherry picked from commit 852612eac9)
Co-authored-by: Ivan Timokhin <nixpkgs@ivan.timokhin.name>
Co-authored-by: Sandro <sandro.jaeckel@gmail.com>
`pkgs.fetchgit` uses `fetchSubmodules = true;` by default, however
`nix-prefetch-git` doesn't. This means that hashes for a Git repository
with fetched submodules will be wrong in `yarn.nix`.
Considering that this got unnoticed before, it seems as if this case is
an exception to a certain degree.
An exemplary problem is the last `hedgedoc` update[1] where
`js-sequence-diagrams` - a Git repo with submodules - from upstream's
package.json caused a hash mismatch. This got unnoticed because
`nix-build --check` doesn't seem to reveal these issues for fixed-output
derivations.
[1] https://github.com/NixOS/nixpkgs/pull/139238
(cherry picked from commit 46c98ae132)
- Use 1 line per input argument.
- Don't use let ... in if not needed.
- Use ${version} in url string.
- Run hooks in explicit installPhase.
(cherry picked from commit e1d6051d81)
When restoring a backup, discourse decompresses the backup archive in
the /share/discourse/tmp dir. Before this change, it is linked to /run
which is typically backed by memory, so the backup will fail to
restore if you do not have enough memory on your system to contain the
backup. This has already happened to me on two small forums.
This moves tmp to the StateDirectory /var/lib/discourse/tmp which is
typically backed by disk.
(cherry picked from commit f933c68374)
Need to patch out the contextvars dependency (which is included in
python 3.7+).
The same patch is discussed in arch:
https://bugs.archlinux.org/task/71344
(cherry picked from commit c0b46c6b59)
ChangeLog: https://github.com/hedgedoc/hedgedoc/releases/tag/1.9.0
As documented in the Nix expression, I unfortunately had to patch
`yarn.lock` manually (the `yarn.nix` result isn't affected by this). By
adding a `git+https`-prefix to
`midi "https://github.com/paulrosen/MIDI.js.git#abcjs"` in the lock-file
I ensured that `yarn` actually uses the `MIDI.js` from the offline-cache
from `yarn2nix` rather than trying to download a tarball from GitHub.
Also, this release contains a fix for CVE-2021-39175 which doesn't seem
to be backported to 1.8. To quote NVD[1]:
> In versions prior to 1.9.0, an unauthenticated attacker can inject
> arbitrary JavaScript into the speaker-notes of the slide-mode feature
> by embedding an iframe hosting the malicious code into the slides or by
> embedding the HedgeDoc instance into another page.
Even though it "only" has a medium rating by NVD (6.1), this seems
rather problematic to me (also, GitHub rates this as "High"), so it's
actually a candidate for a backport.
[1] https://nvd.nist.gov/vuln/detail/CVE-2021-39175
(cherry picked from commit 0a10c17c8d)
The src points to the obsidiansystems repo as it has the ghcjs ported from
8.10.5 to 8.10.7, and a bunch of other fixes (#812, #811, #809)
(cherry picked from commit ba25b274f4)
Modified the stm_2_5_0_1 -> stm_2_5_0_0
Backport which adds, rather than updates, the GHC release.
----
The only big change is required for darwin since GHC 8.10.5 now
runs xattr in the install phase on darwin:
* 11e1dcde0d
* ec451cac39
Unfortunately, it uses the host /usr/bin/xattr by default which is
present in the build due to a lack of sandboxing on darwin. That xattr
version however still requires Python 2.7 whereas Python 3.8 is in PATH
in our build. We solve this by setting the XATTR environment variable.
We can't use python3Packages.xattr since GHC expects Apple's fork of
xattr which provides some extra flags to utilize.
Co-authored-by: Cheng Shao <cheng.shao@tweag.io>
(Adapted from cb330ce4f0)
The `services.mastodon` module currently hardcodes sidekiq's concurrency
to 25, but doesn't set a DB pool size, which defaults to 5 or the number
of configured web threads.
(This behaviour is very strange, and arguably a mastodon bug.)
This also makes sidekiq's concurrency configurable, because 25 is a tad
high for the hardware I'm running it on.
(cherry picked from commit e8fd7792d1)
I'm not sure this is the best way to get these patches, but it's better than `master` (at commit `e9617f553284b170a8b520d051ac1fc1b83cff30` on `nginx` these patches moved into a `nginx` subdirectory, breaking the build unless the patches are cached).
(cherry picked from commit c5d8765113)
This drops the python3Packages.ansible-collections attribute in favor of
a local callPackage that overwrites the collections package per ansible
version.
(cherry picked from commit ca618d6401)
ChangeLog: https://lists.zx2c4.com/pipermail/wireguard/2021-September/007049.html
This doesn't seem to have any implications for NixOS users, but appears
to have a few fixes that seem relevant to Darwin users including:
wg-quick: darwin: account for "link#XX" gateways
On macOS, under specific configurations, the `netstat -nr -f inet` and
`netstat -nr -f inet6` outputs break gateway collection.
(cherry picked from commit d37ab4d0ee)
As of yet, a patch caused the runner to discard update messages.
Unfortunately, GitHub keeps sending update messages to outdated runners
causing them to no longer pick up jobs.
This commit causes the runner to send a high version to GitHub which
should be more recent for quite a time. That way, GitHub does not send
update message and keeps scheduling jobs even for outdated runners.
Naturally, an oudated runner can still break at any time as GitHub's
current approach assumes that all runners are always up-to-date. We
should still strive for quick nixpkgs updates but this patch should give
us some time.
(cherry picked from commit e8bbcc79fd)
The runner only references `GitInfoCommitHash`/`CommitHash` to print
informational log entries. To allow for just referencing the tag of a
version instead of the commit hash, this commit sets the value of the
`GitInfoCommitHash` to a static dummy value.
(cherry picked from commit cc5c902fdf)
https://chromereleases.googleblog.com/2021/09/stable-channel-update-for-desktop.html
This update includes 11 security fixes. Google is aware that exploits
for CVE-2021-30632 and CVE-2021-30633 exist in the wild.
CVEs:
CVE-2021-30625 CVE-2021-30626 CVE-2021-30627 CVE-2021-30628
CVE-2021-30629 CVE-2021-30630 CVE-2021-30631 CVE-2021-30632
CVE-2021-30633
(cherry picked from commit 61e54424ba)
See upstream PR#261[1] for further reference. Previously, the
`enable-features`-setting was entirely discarded due to an earlier
regression resulting in an attempt to start `element-desktop` in
Wayland-mode without all necessary components.
Closes#137377
[1] https://github.com/vector-im/element-desktop/pull/261
(cherry picked from commit 5a0d0ec1cf)
Having a disks object with a dictionary of all the disks and their
properties makes it easier to process multi-disk images.
Note the rename of `label` to `system_label` is because `$label`i
is something of a special token to jq.
(cherry picked from commit 71b3d18181)
Introduce an AWS EC2 AMI which supports aarch64 and x86_64 with a ZFS
root.
This uses `make-zfs-image` which implies two EBS volumes are needed
inside EC2, one for boot, one for root. It should not matter which
is identified `xvda` and which is `xvdb`, though I have always
uploaded `boot` as `xvda`.
(cherry picked from commit bd38b059ea)
This is a private interface for internal NixOS use. It is similar
to `make-disk-image` except it is much more opinionated about what
kind of disk image it'll make.
Specifically, it will always create *two* disks:
1. a `boot` disk formatted with FAT in a hybrid GPT mode.
2. a `root` disk which is completely owned by a single zpool.
The partitioning and FAT decisions should make the resulting images
bootable under EFI or BIOS, with systemd-boot or grub.
The root disk's zpools options are highly customizable, including
fully customizable datasets and their options.
Because the boot disk and partition are highly opinionated, it is
expected that the `boot` disk will be mounted at `/boot`. It is
always labeled ESP even on BIOS boot systems.
In order for the datasets to be mounted properly, the `datasets`
passed in to `make-zfs-image` are turned in to NixOS configuration
stored at /etc/nixos/configuration.nix inside the VM.
NOTE: The function accepts a system configuration in the `config`
argument. The *caller* must manually configure the system
in `config` to have each specified `dataset` be represented
by a corresponding `fileSystems` entry.
One way to test the resulting images is with qemu:
```sh
boot=$(find ./result/ -name '*.boot.*');
root=$(find ./result/ -name '*.root.*');
echo '`Ctrl-a h` to get help on the monitor';
echo '`Ctrl-a x` to exit';
qemu-kvm \
-nographic \
-cpu max \
-m 16G \
-drive file=$boot,snapshot=on,index=0,media=disk \
-drive file=$root,snapshot=on,index=1,media=disk \
-boot c \
-net user \
-net nic \
-msg timestamp=on
```
(cherry picked from commit 076f6e2d94)
Because of `ProtectSystem=strict`, Promtail cannot write to the positions
file if it's not in its `CacheDirectory` (the default value).
(cherry picked from commit b6ad701a2c)
To avoid unnecessary builds but this needs to be fixed ASAP. Chromium
already depends on it and a lot of additional packages, including Mesa,
will depend on it after the stable release.
(cherry picked from commit 5661f7dbee)
Those functions can be required anywhere in the Nix expressions for
Chromium and therefore they should be defined in default.nix and
inherited where necessary.
This fixes the chromiumBeta build which failed because I forgot to
update the channel conditional when the beta channel advanced to M94.
This is exactly why the version based conditionals should be used
everywhere.
(cherry picked from commit 186315def7)
This should make it a little bit easier to read. The (native) build
inputs and options now have a consistent order, defaultDependencies and
yasm aren't required anymore, and this adds some brief comments.
Note: The third_party/node/linux/node-linux-x64/bin/node change causes
rebuilds.
(cherry picked from commit 4a5b367259)
Note: I've only tested this with llvmPackages_git but it should work
with llvmPackages_13 as well.
fieldtrial_testing_like_official_build was renamed to
disable_fieldtrial_testing_config:
486e9d58c0
(cherry picked from commit 1e372f4004)
e7f8fe4f67
changed the binary name to always end with its module version. This
makes sure gitaly's internal version references are set to the package
version and renames the binary, postfixing it with the package
version.
(cherry picked from commit d14e9188d1)
This doesn't cause any rebuilds because we use a set but should make it
a bit easier to understand what's going on. This also moves two flags
that where incorrectly in "optionalAttrs pulseSupport" (enabled by
default) out of there.
The native client deprecation is stated here:
https://developer.chrome.com/docs/native-client/
(cherry picked from commit 555c3afaf2)
Our system FFmpeg version is too outdated and Snappy causes a linking
failure (I didn't have time to investigate yet). Hopefully we can revert
this before the stable release of M93.
(cherry picked from commit bd22d2425c)
This wasn't done in 97570d30c7 to allow reusing chromiumBeta builds
(without having to perform any changes) in the meantime.
(cherry picked from commit 9ac3188552)
I'm having trouble building bluez on a CentOS 7.9 builder.
Turns out that the test-mesh-crypto test depends on the following Linux
kernel configuration items:
CONFIG_CRYPTO_[USER|USER_API|USER_API_AEAD|USER_API_HASH|AES|CCM|AEAD|CMAC]
and will fail if run wih an incompatible kernel with the following error
message:
...
IVindex = 12345678
NetworkNonce = 00800000011201000012345678
00800000011201000012345678 => PASS
PrivacyRandom = 000000000012345678b5e5bfdacbaf6c
000000000012345678fffd034b50057e => FAIL
FAIL unit/test-mesh-crypto (exit status: 1)
I found the same bug reported on Gentoo https://bugs.gentoo.org/704190.
Their fix is to only run the test if the kernel options are available,
which we don't want to do for build reproducibility.
Instead we just want to skip the test unconditionally. As it's simpler
to completely override the test instead of patching the build system, we
opt for doing just that.
(cherry picked from commit 05c517698d)
This error occurs if `nextcloud-occ maintenance:install` fails and the
`upgrade` command is attempted to be executed afterwards.
Due to the nature of the installer we can't do much about it, so I guess
it makes sense to add some notes about it. The other notes in the
`Pitfalls`-section are semantically a list of different topics, so I
changed that accordingly now.
Closes#111175
(cherry picked from commit 561418f996)
dyld: lazy symbol binding failed: Symbol not found: _dsyevd_
Referenced from: /nix/store/lr8grz1knmh6vc7j830gni0ka68qf1lk-xfitter-2.0.1/bin/xfitter
Expected in: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
(cherry picked from commit de4c61a515)
This commit exposes in addition to the tuxedo-keyboard module:
- clevo_acpi
- clevo_wmi
- tuxedo_io
These modules are required to run the Tuxedo Control Center.
(cherry picked from commit 2e6d563067)
This is needed because `mautrix-telegram --generate-registration`
appears to need write-access to `config.json` as well.
Closes#135884
(cherry picked from commit bae65a3c06)
This doesn't work anymore and thus breaks the installation leaving a
broken `/var/lib/nextcloud`.
It isn't a big deal since we set this value in the override config
before, so the correct table-prefix is still used. In order to confirm
that, I decided to add a custom prefix to the basic test.
(cherry picked from commit eaeb4fe04e)
tcl-8_6 was updated in 4fb92ae60d from
8.6.9 -> 8.6.11 but tk's hash wasn't updated at the same time,
which means the current hash was still from tk 8.6.9 rather than 8.6.11.
(cherry picked from commit d9bf02324a)
This update is required because the autoupdater prevents the app from
running without being updated (which is not possible because it is a
store path).
So this updates the app and patches out the updater (essentially it
never thinks it needs to update, even though it prints on the command
line that it should).
Also fix the desktop item.
(cherry picked from commit 9aa05d18b7)
The previous backport of `mautrix-telegram`[1] appears to be a mistake
of mine (sorry for that!), unfortunately v0.9 isn't forward-compatible
due to a database migration, so this change can't be reversed.
The primary issue was that `telethon` was too old, so I decided to
upgrade it to 1.21 which is what I had on my 21.05-based tracking
branch while using `mautrix-telegram` anyways, so that's proven to work.
Closes#135583
[1] e1042038ac / cd84b8ad8b
Elpa only serves the latest version of a given package uncompressed.
Once that release is no longer the latest & greatest it gets archived and compressed
meaning that both the URL and the hash changes.
To work around this issue we fall back to the URL with the .lz suffix and if that's the
one we downloaded we uncompress the file to ensure the hash matches regardless of compression.
Fixes nmcli being passed an empty string before the interface name,
which would stop captive-browser from starting up.
(cherry picked from commit 910f233fb7)
twisted is used in matrix-synapse for smtp handling.
Mostly this is used for password resets, but also notifications
are delivered that way.
older versions of twisted require the e-mail server to
have TLS1.0 enabled.
Obviously, quite a lot of servers have this disabled which means
synapse won't be able to deliver mails using such servers.
matrix-synapse issue:
https://github.com/matrix-org/synapse/issues/6211
Without this dependency JetBrains editors, on NixOS, will throw errors
about a missing object file:
java.lang.UnsatisfiedLinkError: libe2p.so
(cherry picked from commit 70f3763e14)
This is needed since 0.10.1 of mautrix-telegram, but it'd be a breaking
change of mautrix, which can be used for any kind of matrix client, so I
don't really want to break it.
The udev rules were updated upstream without an explanation as you can
see in [this comment][commit].
[commit]: 0bcf928ada
(cherry picked from commit eab36fabf6)
Some plugin repos already have a `gems` directory. This lets the
packager choose whether it should be kept and the nix packaged ruby
gems should be copied into it or if it should be removed in favor of
our ruby gems.
(cherry picked from commit 04e6b03fa9)
Let the update.py script handle the initial, repetitive task of
packaging new plugins. With this in place, the plugin only needs to be
added to the list in `update-plugins` and most of the work will be
done automatically when the script is run. Metadata still needs to be
filled in manually and some packages may of course require additional
work/patching.
(cherry picked from commit f8096460bd)
Change the path to the auto generated plugin assets, which defaults to
the plugin's directory and isn't writable at the time of asset
generation.
(cherry picked from commit 443b318ee9)
Perform the tests on the package that the `tests` attribute is a child
of, i.e. if `discourseAllPlugins.tests` is built, the tests will run
with the `discourseAllPlugins` package, not the `discourse` package as
previously.
(cherry picked from commit 6fd5a40cca)
With Claws Mail's latest double release of 3.18.0 and 4.0.0, the package
will refer to the more "modern" GTK+ 3 release, major version four. The
GTK+ 2 release, major version 3, is now available in the
`claws-mail-gtk2` package.
In other words, this commit bumps the GTK+ 2 version from 3.17.8 to
3.18.0, the previously unstable GTK+ 3 version 3.99.0 to 4.0.0 and
changes the default to GTK+ 3.
(cherry picked from commit 26f52bf6b5)
There is no new release yet (see mailing list post on oss-security), so
I'm picking the commit that fixes the CVE.
There is another security flaw (without a CVE number) that is also
mentioned in the oss-security announcement but it is not explained which
commit patches the problem.
(cherry picked from commit 46b7a5be1c)
At some point the clj script stopped just calling clojure, but instead
$bin_dir/clojure. As a result we now need to additionally substitute
BINDIR in the clj script.
cgcc command has shebang of /usr/bin/perl, which obviously does not
work for nixos, adding perl to buildInputs seems to make all the magic
happen, as per usual.
(cherry picked from commit e7836bc5a5)
I expect it suffices that the channel only blocks on one firefox ESR
test - the one for the default ESR. I didn't want to have the
information about the default in two places, so either of the tests will
be evaluated twice (but to the same *.drv I hope).
(cherry picked from commit c0097aa84a)
Previously these ended up without u+w permissions which meant they could
not be regenerated, which was hugely annoying when these venvs were for
example created and recreated in a nix-shell.
(cherry picked from commit 4fa69858d9)
nvidia 470.57.02 changed the path of `nvidia-sleep.sh` and systemd
scripts, making `builder.sh` miss them and suspend-to-ram on systems
where `hardware.nvidia.powerManagement.enable = true` is set fail.
(cherry picked from commit 3f4bb8ff5a)
This fixes:
```
systemd[1]: Started VictoriaMetrics time series database.
victoria-metrics[379550]: 2021-08-04T19:33:39.833Z panic VictoriaMetrics/lib/storage/partition.go:954 FATAL: unrecoverable error when merging small parts in the partition "/var/lib/victoriametrics/data/small/2021_08": cannot open source part for merging: cannot open metaindex file in stream mode: cannot open file "/var/lib/victoriametrics/data/small/2021_08/1228_1228_20210804184120.712_20210804184121.899_16982E83CD7A763A/metaindex.bin": open /var/lib/victoriametrics/data/small/2021_08/1228_1228_20210804184120.712_20210804184121.899_16982E83CD7A763A/metaindex.bin: too many open files
victoria-metrics[379550]: panic: FATAL: unrecoverable error when merging small parts in the partition "/var/lib/victoriametrics/data/small/2021_08": cannot open source part for merging: cannot open metaindex file in stream mode: cannot open file "/var/lib/victoriametrics/data/small/2021_08/1228_1228_20210804184120.712_20210804184121.899_16982E83CD7A763A/metaindex.bin": open /var/lib/victoriametrics/data/small/2021_08/1228_1228_20210804184120.712_20210804184121.899_16982E83CD7A763A/metaindex.bin: too many open files
victoria-metrics[379550]: goroutine 629 [running]:
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logMessage(0xbb3ea1, 0x5, 0xc001113800, 0x1e7, 0x4)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:270 +0xc69
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logLevelSkipframes(0x1, 0xbb3ea1, 0x5, 0xbe3f8b, 0x4b, 0xc000bb3f88, 0x2, 0x2)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:138 +0xd1
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.logLevel(...)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:130
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger.Panicf(...)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/logger/logger.go:126
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.(*partition).smallPartsMerger(0xc0014d7980)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/partition.go:954 +0x145
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.(*partition).startMergeWorkers.func1(0xc0014d7980)
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/partition.go:933 +0x2b
victoria-metrics[379550]: created by github.com/VictoriaMetrics/VictoriaMetrics/lib/storage.(*partition).startMergeWorkers
victoria-metrics[379550]: github.com/VictoriaMetrics/VictoriaMetrics/lib/storage/partition.go:932 +0x6c
systemd[1]: victoriametrics.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
systemd[1]: victoriametrics.service: Failed with result 'exit-code'.
systemd[1]: victoriametrics.service: Consumed 587ms CPU time, received 6.5K IP traffic, sent 1.7K IP traffic.
systemd[1]: victoriametrics.service: Scheduled restart job, restart counter is at 2064.
systemd[1]: Stopped VictoriaMetrics time series database.
systemd[1]: victoriametrics.service: Consumed 587ms CPU time, received 6.5K IP traffic, sent 1.7K IP traffic.
systemd[1]: Starting VictoriaMetrics time series database...
```
(cherry picked from commit fb6fbcb85c)
In April, 2021, Jetbrains announced the end of support for 32-bit OS
IntelliJ-based IDEs[1]:
> The final major version that will be guaranteed to run on a 32-bit OS will be
> v2021.1 for all IntelliJ-based IDEs, including AppCode, Clion, DataGrip, GoLand,
> IntelliJ IDEA, PhpStorm, PyCharm, Rider, RubyMine, and WebStorm.
[1]: https://blog.jetbrains.com/idea/2021/04/end-of-support-for-32-bit-operating-systems-in-intellij-based-ides/
(cherry picked from commit 147d36b474)
Jetbrains seems to have deviated from the usual URL pattern of
`/<product>/<version>/<name>-<version>.<ext>` by only using the major.minor part
of the version number in the first URL segment for MPS.
(cherry picked from commit 597f94bd8a)
* terraria-server: update source URL
The old URL redirects to the main page.
(cherry picked from commit 1e919bfeed)
* Apply suggestions from code review
Co-authored-by: Naïm Favier <n@monade.li>
Co-authored-by: Jörg Thalheim <Mic92@users.noreply.github.com>
https://github.com/NixOS/cabal2nix/blob/v2.18.0/CHANGELOG.md
2.18.0 mostly brings bug fixes, especially the one fixing the unclear
error message if a cabal file was missing (which was forgotten in the
changelog). It is not technically a non-breaking change since the
Haskell API changed.
Since there shouldn't be any breakage for users and all known reverse
dependencies of cabal2nix have been broken for a while, the benefits
of backporting outweigh the risks.
`functionArgs` should give valid results on
functions that have been identified with `lib.isFunction`
instead of erroring out.
(cherry picked from commit cf8e219b7e)
binfmt activation script creates /run/binfmt before mounting /run
when system activation.
To fix it I added dependency to specialfs to binfmt activation
script.
(cherry picked from commit bf22778585)
Upstream is aware but they don't have a fix yet.
I'm not aware of NixPkgs being used for another platform
affected by this (e.g. ppc64le).
(cherry picked from commit 895ce97eea)
Had to do this manually, because this `name`->`pname` "refactoring"
makes automated backports impossible.
Changes: 08450562e5...9c707dc32a
(cherry picked from commit bee0468d7b)
It seems as since Chromium 92, `chromium` crashes on startup if
`XDG_CONFIG_HOME` points to a read-only (store-)path.
(cherry picked from commit 8c35a69a6e)
* GCC 10 support.
* clang 10 support.
* SBAT support.
* LUKS2 support.
* Drop small MBR gap support.
* Xen Security Modules (XSM/FLASK) support.
* The lockdown mechanism similar to the Linux kernel one.
* Disable the os-prober by default.
* Many backports of GRUB distros specific patches.
* BootHole and BootHole2 fixes.
* XFS bigtime support.
* ...and tons of other fixes and cleanups...
(cherry picked from commit 963b0a1dbf)
Before this commit, the `flake` option was typed with `types.unspecified`.
This type get's merged via [`mergeDefaultOption`](ebb592a04c/lib/options.nix (L119-L128)), which has a line
```nix
else if all isFunction list then x: mergeDefaultOption loc (map (f: f x) list)
```
`lib.isFunction` detects an attrs in the shape of `{__functor = ...}` as
a function and hence this line substitutes such attrs with a function
(f: f x).
If now, a flake input has a `__functor` as it's output, this will
coerce the once attrs to a function. This breaks a lot of things later
in the stack, for example a later `lib.filterAttrs seive <LAMBDA>` will
fail for obious reasons.
According to @infinisil, `types.unspecified` is due to deprecation. In
the meantime this PR provides a specific fix for the specific problem
discovered.
(cherry picked from commit ecae25c3ef)
The current tg_owt version (since the update of tdesktop to version
2.8.0 in 0d509d366d) isn't compatible with kotatogram-desktop anymore.
(cherry picked from commit f287805faf)
LPAE was enabled to support native armv7l builders running in QEMU on aarch64,
but this option disables support for processors which don't support LPAE, which
are still relatively common. In particular, Beaglebones use the Cortex-A8, which
doesn't support LPAE.
Also, if you attempt to boot an LPAE kernel on a CPU that doesn't support it,
it fails before even earlycon is initialized. This makes the problem difficult
to debug without enabling CONFIG_DEBUG_LL or using a hardware debugger.
(cherry picked from commit 988c12faed)
In 9d03ff5222 I made the CPython builds
reproducible. This required not generating default unoptimized bytecode.
I was under the impression the optimized bytecode would be used then,
but you need to opt-in on that. Not having the default bytecode resulted
in a significant performance hit. Therefore, bytecode is generated again
in this commit, and thereby the builds are no longer reproducible.
https://bugs.python.org/issue29708
(cherry picked from commit 23e348bfe2)
At some point the name of the doc-tool script changed and we stopped
patching up the shabang line. The new one is just scripts/bpf_doc.py.
(cherry picked from commit 8997f8cc70)
installer media can be used on top of existing host configs. In such
scenarions, root fs types will already be defined.
Before this change, this will inevitably lead to the following error:
```console
error: The option `fileSystems./.fsType' has conflicting definition values:
- In `/nix/store/2nl5cl4mf6vnldpbxhrbzfh0n8rsv9fm-source/DevOS/os/hardware/common.nix': "ext4"
- In `/nix/store/jbch90yqx6gg1h3fq30jjj2b6h6jfjgs-source/nixos/modules/installer/cd-dvd/iso-image.nix': "tmpfs"
```
With this patch, the installers will override those values according to
their own local requirement.
Use `mkOverride 60` so that conscientious overriding specially targeted
at the installer, e.g. with `mkForce` is still straight forward.
(cherry picked from commit c219fdffad)
Without this, failure of nixBuild() and nixFlakeBuild() was ignored
(since bash doesn't inherit 'set -e' in subshells by default), so the
script would proceed with a bogus ./result link, e.g.
++ readlink -f /tmp/nixos-rebuild.NfHKxx/result
+ pathToConfig='/nix/store/m7dvk6an18cpr95qn5wnig2600qhv6w7-nix-2.4pre20210727_706777a/bin/nix
/tmp/nixos-rebuild.NfHKxx/result'
+ '[' test = switch -o test = boot ']'
+ copyToTarget '/nix/store/m7dvk6an18cpr95qn5wnig2600qhv6w7-nix-2.4pre20210727_706777a/bin/nix
/tmp/nixos-rebuild.NfHKxx/result'
+ '[' '' = '' ']'
+ '[' test = switch -o test = boot -o test = test -o test = dry-activate ']'
+ targetHostCmd /nix/store/m7dvk6an18cpr95qn5wnig2600qhv6w7-nix-2.4pre20210727_706777a/bin/nix /tmp/nixos-rebuild.NfHKxx/result/bin/switch-to-configuration test
+ '[' -z '' ']'
+ sudo -- /nix/store/m7dvk6an18cpr95qn5wnig2600qhv6w7-nix-2.4pre20210727_706777a/bin/nix /tmp/nixos-rebuild.NfHKxx/result/bin/switch-to-configuration test
error: '/tmp/nixos-rebuild.NfHKxx/result/bin/switch-to-configuration' is not a recognised command
Try '/nix/store/m7dvk6an18cpr95qn5wnig2600qhv6w7-nix-2.4pre20210727_706777a/bin/nix --help' for more information.
+ echo 'warning: error(s) occurred while switching to the new configuration'
warning: error(s) occurred while switching to the new configuration
(cherry picked from commit 0ad27c8653)
Turns VMware guest mouse support on in the kernel. This is needed for running Wayland and non-root X in a VMWare guest. In a pre-Wayland world the `xf86-input-vmmouse` userspace driver would have handled this for us. This allows the mouse to properly work in a vmware guest (for example it can now leave the vmware window).
See: https://github.com/vmware/open-vm-tools/issues/528
(cherry picked from commit 1207e7581f)
The conclusion is that Phabricator does not, in fact, generate stable
patch links. In any case, these have landed, so we can just use the
patches from Mercurial's hgweb instance instead, which should be more
stable.
(cherry picked from commit 67444f8a39)
I know that the en_EN.json vs. en-us.json is a nasty hack, but I don't
really understand where this `en-us.json` is supposed to be coming from.
(cherry picked from commit b856b19540)
The stable channel update to M92 (97570d30c7) broke the Wayland support:
$ chromium --enable-features=UseOzonePlatform --ozone-platform=wayland
[31712:31712:0721/114725.940557:ERROR:wayland_connection.cc(137)] Failed to load wayland client libraries.
[31712:31712:0721/114725.940641:FATAL:ozone_platform_wayland.cc(177)] Failed to initialize Wayland platform
[0721/114725.947566:ERROR:process_memory_range.cc(75)] read out of range
Trace/breakpoint trap (core dumped)
(cherry picked from commit bb651d27fd)
https://chromereleases.googleblog.com/2021/07/stable-channel-update-for-desktop_20.html
This update includes 35 security fixes.
CVEs:
CVE-2021-30565 CVE-2021-30566 CVE-2021-30567 CVE-2021-30568
CVE-2021-30569 CVE-2021-30571 CVE-2021-30572 CVE-2021-30573
CVE-2021-30574 CVE-2021-30575 CVE-2021-30576 CVE-2021-30577
CVE-2021-30578 CVE-2021-30579 CVE-2021-30580 CVE-2021-30581
CVE-2021-30582 CVE-2021-30583 CVE-2021-30584 CVE-2021-30585
CVE-2021-30586 CVE-2021-30587 CVE-2021-30588 CVE-2021-30589
Note: This won't be the smoothest update. Chromium seems to be fine but
requires gtk3 in $LD_LIBRARY_PATH to find libgtk-3.so.0 (otherwise it
crashes during startup) but Google Chrome fails to initialize
("GPU process exited unexpectedly: exit_code=132") and requires
"--use-gl=angle --use-angle=swiftshader" for hardware(?) acceleration
(which seems to work work fine and performant but SwiftShader should
actually use the CPU instead of the GPU).
(cherry picked from commit 97570d30c7)
This should catch regressions like #131074 in the future. In that case a
glibc update caused a regression that caused most of the text to become
invisible (just not the "Web Store" we've already been checking for).
(cherry picked from commit 11400dcd65)
This can be very useful when running the test headless or e.g. when
looking at Hydra logs. Especially the chrome://gpu content contains a
lot of interesting information.
I also decided to refactor the test_new_win() function to avoid
duplicate code and rely less on xdo.
(cherry picked from commit c33015a0c9)
Unfortunately there are some regressions in the GPU code that cause
Chromium and Google Chrome to crash, e.g.:
machine # [0709/084047.890436:ERROR:process_memory_range.cc(75)] read out of range[ 30.153484] show_signal: 20 callbacks suppressed
machine # [ 30.153490] traps: chrome[1036] trap invalid opcode ip:55af03357b29 sp:7ffeaa69ad10 error:0 in chrome[55aefe7a4000+81ec000]
machine #
machine # [0709/084047.955039:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)
machine # [0709/084047.955078:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2)
machine # [ 30.126905] systemd[1]: Created slice system-systemd\x2dcoredump.slice.
machine # [ 30.137012] systemd[1]: Started Process Core Dump (PID 1038/UID 0).
machine # [ 30.571987] systemd-coredump[1039]: Process 1036 (chrome) of user 1000 dumped core.
machine # [992:1021:0709/084048.501937:ERROR:gpu_process_host.cc(995)] GPU process exited unexpectedly: exit_code=132
machine # [ 30.594747] systemd[1]: systemd-coredump@0-1038-0.service: Succeeded.
Hopefully this'll be fixed upstream before the final release (there are
bug reports for it) but for the meantime we have to launch the beta and
dev versions with "--use-gl=angle --use-angle=swiftshader".
(cherry picked from commit f9645002a2)
This most notably fixes cross _evaluation_ of chromium which previously
would fail because makeWrapper relies on runtimeShell which is not
available in the HostTarget package set.
I tested that the native chromium build still works, but haven't tried
cross compiling it yet. There very well may be additional errors, but at
least they will be build errors, not hard to understand evaluation
errors.
(cherry picked from commit 524aa1c87c)
Updated firecracker to v0.24.4. This required updating the buildPhase to
point to new compilation result paths.
Formatting changes were performed by `nix-update --format`
(cherry picked from commit 6fc6e325e6)
This executable is required to fix a startup error.
TODO: Refactor the Nix expressions to allow chromiumVersionAtLeast, etc.
"everywhere" and investigate the VM test failure.
(cherry picked from commit ef7f020ec8)
Would need to temporarily remove "ffmpeg" from gnSystemLibraries and
disable use_thin_lto to fix the build (theoretically).
(cherry picked from commit 5cae434566)
The final linking still fails though, even with llvm-git.
We might have to diable use_thin_lto for now:
ld.lld: error: undefined symbol: snappy::Compress(char const*, unsigned long, std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >*)
>>> referenced by compression_module.cc
>>> thinlto-cache/Thin-ed5ed5.tmp.o:(reporting::CompressionModule::CompressRecord(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, base::OnceCallback<void (std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >, absl::optional<reporting::CompressionInformation>)>) const)
clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
(cherry picked from commit fcdcb81936)
See https://bugs.chromium.org/p/chromium/issues/detail?id=1215229.
Before this the build failed with this error:
[101/47617] ACTION //build/util:chromium_git_revision(//build/toolchain/linux/unbundle:default)oaded_data.pbchain/linux/unbundle:default)
FAILED: gen/build/util/chromium_git_revision.h
python3 ../../build/util/lastchange.py --header gen/build/util/chromium_git_revision.h --revision-id-only --revision-id-prefix @ -m\ CHROMIUM_GIT_REVISION
ERROR:root:Failed to get git top directory from '/build/chromium-93.0.4542.2/build/util': Git command 'git git rev-parse --show-toplevel' in /build/chromium-93.0.4542.2/build/util failed: [Errno 2] No such file or directory: 'git'
(cherry picked from commit 8af443906d)
If our Chrome derivation is Vulkan enabled, the Chrome GPU process
reliably crashes for me under M92 using the proprietary Nvidia drivers.
This is because the PCI-based GPU detection path fails, and we attempt
to use the Vulkan fallback instead, which then crashes(!!)
Including libpci allows us to use Angle's
src/gpu_info_util/SystemInfo_libpci.cpp path instead, which doesn't
crash, unlike src/gpu_info_util/SystemInfo_vulkan.cpp.
(cherry picked from commit 51d83077ff)
It's worth to think about setting -DDESKTOP_APP_USE_PACKAGED_FONTS=OFF
since it's impossible to install fonts as dependencies of packages with
Nix and tdesktop's widgets are developed only with Open Sans in mind (it
has a lot of hardcoded values and wide fonts like DejaVu may
even go out of widgets' bounds)
https://github.com/NixOS/nixpkgs/pull/130827#issuecomment-885212649
(cherry picked from commit 27585b9897)
Do not pollute top-level, but do provide allow the derivations to
be accessed for debugging and reuse if necessary.
(cherry picked from commit 2c2e1db91d)
As has been done in https://github.com/NixOS/nix/pull/4944
This introduces the boehmgc_nix and boehmgc_nixUnstable attributes
which are useful for external packages that link with Nix and its
boehmgc.
(cherry picked from commit 596ac242af)
This should fix
Jul 20 07:16:12 bastion mirror-tarballs-start[21663]: Use of uninitialized value $algo in concatenation (.) or string at ./maintainers/scripts/copy-tarballs.pl line 80.
Jul 20 07:16:12 bastion mirror-tarballs-start[21663]: Use of uninitialized value in subroutine entry at ./maintainers/scripts/copy-tarballs.pl line 185.
Jul 20 07:16:13 bastion mirror-tarballs-start[21663]: error: unknown hash algorithm '' at ./maintainers/scripts/copy-tarballs.pl line 185.
(cherry picked from commit c48be3c17d)
I've also updated the URL for the RISC-V patch in case the content of
the old URL will change (not sure if that's possible after a merge
request is merged but now that the patch is upstream it seems like a
good idea regardless; and the content has actually already changed so
the old hash wasn't correct anymore).
Note: This update likely causes some issues when running an application
that has a direct dependency on Mesa (e.g. Sway and XWayland) and was
compiled against a different Nixpkgs revision. See 7106fca0fe for more
details regarding that issue.
Note: The update to Mesa 21.0.2 was reverted (25ae1fd29f) because it
caused major issues with Sway (segfault on startup [0]).
This is still the case and might affect all packages that directly
depend on "mesa" (for libgbm or libglapi) but it only causes issues when
the package depends on a "mesa" version that differs from "mesa.drivers"
used for "/run/opengl-driver/". I've noticed this while testing Mesa
updates with the NixOS option "hardware.opengl.package" (as usual)
instead of rebuilding my whole system (which would work). Unfortunately
this can/will likely also cause issues when mixing different channels,
using Flakes/Overlays, etc.
The cause of this should be similar to [1] ("mesa" updates now cause the
same issues that "glibc" updates already do, maybe triggered by certain
Mesa changes) and some additional discussions is in [2],[3].
Note: Don't backport this to NixOS 21.05, at least not without careful
consideration.
[0]: https://github.com/NixOS/nixpkgs/pull/118753#issuecomment-818950977
[1]: https://github.com/NixOS/nixpkgs/issues/95808
[2]: https://github.com/NixOS/nixpkgs/pull/120325
[3]: https://github.com/NixOS/nixpkgs/pull/119558
ChangeLog: https://github.com/grafana/grafana/releases/tag/v7.5.10
On `master` (i.e. 21.11) we already have Grafana 8.0, but I don't really
want to backport this major release, so it's fair IMHO to update Grafana
here to make sure that 21.05-users still get bugfixes.
This derivation was introduced in commit
e1d60a05af with version 2.2.2 but the
sha256 hash in that commit actually corresponded to version 2.2.1,
as you can see below:
$ nix-prefetch-github --rev "v2.2.1" fitnr convertdate | jq '.sha256'
"1xgi7x9b9kxm0q51bqnmwdm5lp8vwhx5yk4d1b23r37spz9dbhw5"
(cherry picked from commit 3e84c7ea0d)
Maybe bundix doesn't handle them properly? They cause runtime issues
and don't seem necessary when the binary gems are built from scratch
anyway.
(cherry picked from commit a2dbc3af1d)
make gtk3Support non-optional, because it hasn't been for a long time
also make gtk2 conditional on firefox older than 90, because we can get
rid of it with firefox 90, but it's still needed by the current ESR
release
(cherry picked from commit b332794adf)
pandoc can embed fonts and stylesheets on export which apostrophe uses —
however it has hardcoded paths for the fonts which only work in the
flatpak version, so we need to substitute them.
(cherry picked from commit 4e1acab880)
This basically has two reasons:
* I forgot to remove v19 here and I don't want to break this now. Going
up to the most recent patch-level is the least we can do.
* With a changed version, the `permittedInsecurePackages`-setting in
people's deployments has to change, so people will actually notice the
EOL of v19.
I actually wanted to remove it before 21.05, but unfortunately I forgot
about this. This is a less drastic way of getting rid of v19 in contrast
to the actual breaking change in master[1].
[1] b922990a8e
Upstream changes:
* Fix ordering between systemd socket file descriptor names and rules.
* Fix usage of C library path as discovered by Meson.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 8b7f8e2e69)
- remove check for `connected .JID: focus@auth.server` because
- log format was changed in c1945ea6cb
- connection.getUser() in jicofo also appears to be broken, returning null instead of username
- testing for this log line shouldn't be necessary, as we also test for "Authenticated as focus@auth.server"
- remove check for `External component successfully authenticated` because
- [JVB no longer uses component](https://community.jitsi.org/t/jvb-not-connecting/91157/2)
- increase VM memory
(cherry picked from commit 85aa4bf92b)
This allows cacert to be used with Haskell-based fetchers like
you would with regular OpenSSL-based fetchers:
buildInputs = [ cacert ];
(cherry picked from commit 5d57104d84)
Previously, a failed backup would always overwrite ${db}.sql.gz,
because the bash `>` redirect truncates the file; even if the
backup was going to fail.
On the next run, the ${db}.prev.sql.gz backup would be
overwritten by the bad ${db}.sql.gz.
Now, if the backup fails, the ${db}.in-progress.sql.gz is in an
unknown state, but ${db}.sql.gz will not be written.
On the next run, ${db}.prev.sql.gz (our only good backup) will
not be overwritten because ${db}.sql.gz does not exist.
(cherry picked from commit 81c8189a84)
Voice messages don’t work without gstreamer “base” and “good” plugins.
This change adds a an override for GST_PLUGIN_SYSTEM_PATH_1_0
environment variable providing necessary dependencies.
(cherry picked from commit 20d4e8bd39)
As per #121293, I ensured the UMask is set correctly
and removed any unnecessary chmod/chown/chgrp commands.
The test suite already partially covered permissions
checking but I added an extra check for the selfsigned
cert permissions.
(cherry picked from commit 083aba4f83)
We use the sources from 2019.11.28 to build, but inherit the cert bundle
from the maintained Python3-only version of certifi.
Co-authored-by: adisbladis <adisbladis@gmail.com>
(cherry picked from commit ada27b88e2)
> libuv was vulnerable to out-of-bounds reads in the uv__idna_toascii()
> function which is used to convert strings to ASCII. This is called by
> the DNS resolution function and can lead to information disclosures or
> crashes.
b7466e31e4
Fixes: CVE-2021-22918
(cherry picked from commit 742b8f71f7)
30286ebcc1 updated glibc to 2.32.
This removed the deprecated <sys/sysctl.h> header, which caused a build
error on this package.
Since this header doesn't seem to be required it was removed with this
patch.
(cherry picked from commit c9f2aeec17)
mkEnableOption already adds "Whether to enable" and ends with a ".", so
remove that duplication from the help text.
Also reword it slightly while at it.
(cherry picked from commit 5d3dca497b)
Logically re-apply 64c70a8c4c ("doc: point out that nixos-21.05 has gnuradio
3.9"), because it was lost in the conversion from docbook to markdown, in
commit 32c2dd304d ("docs: nixos release notes to CommonMark (2105)").
(Apparently we have both .md and .xml release notes now, and CI fails
unless they have the same content (after .md processing), so update the
.xml file to match...)
(cherry picked from commit cfe8c3a75e)
Logically re-apply 7afaacf9a8 ("doc: fix link to kodi-19.0 announcement"),
because it was lost in the conversion from docbook to markdown, in commit
32c2dd304d ("docs: nixos release notes to CommonMark (2105)").
(Hm, apparently we have *both* docbook and markdown? CI failed before I
updated the .xml file.)
(cherry picked from commit c2a3ff28be)
This is breaking the tarball build, because #128502 depends on this test
existing. After this commit, nixpkgs.tarball once again evaluates.
(cherry picked from commit 0dccbe2729)
IPFS evolves quite fast. The network protocol is compatible, but we
don't want to force migrations on stable, so we add a new version
instead.
See #100676 for last time we did this.
(Adapted from from commit d96ccfaf16)
The current build works by linking against CoreSymbolication in
/System/Library/PrivateFrameworks. This is impure and doesn't work in
newer versions of macOS.
See https://github.com/NixOS/nixpkgs/issues/128576
(cherry picked from commit 55dcd99859)
Some discourse plugins have Ruby dependencies and require a
specialized builder. This introduces a generic builder that can be
used whether the plugin has Ruby dependencies or not. It also adds a
set of pre-packaged plugins available through `discourse.plugins` and
provides an easy way to add more.
(cherry picked from commit 7671b90919)
For plugins to work properly, their assets need to be precompiled
along with the rest of Discourse's assets. This means we need to build
new packages when the list of plugins change.
(cherry picked from commit 9af3672f4f)
Add patch which fixes builds with (at least) ocaml 4.12.0, and
remove build constraint for 4.09. Necessary because unison built
with 4.09 is not compatible with unison built with 4.12, e.g. on
recent Homebrew.
(cherry picked from commit 7282b4fc03)
The new release comes with breaking changes so we temporarily introduce
wlroots_0_13 for packages that don't yet support wlroots 0.14.
For the rest of the packages the required upstream patches for this new
wlroots release are fetched (if feasible).
(cherry picked from commit 203c8edcda)
systemd ships `units/serial-getty@.service.m4` with the `--keep-baud`
option.
We override that unit, and didn't add the `--keep-baud` option. (We have
it in our other getty options there).
Having `--keep-baud` in `serial-getty@` makes a lot of sense - the
console keeps working if it's initialized with a less standard baud
rate, such as the [Helios64](https://wiki.kobol.io/helios64/intro/).
(cherry picked from commit ba42d639f1)
Different boards using u-boot SPL require to write to different
locations. Sometimes, the 8MiB gap isn't sufficient - rk3399 boards
write to 0x16384 for example, which is at 8MiB, thus overriding the
fat32 partition with the SPL.
(cherry picked from commit 1db54a5522)
Apologies for the pushes straight to release without PR, I will follow
the new workflow moving forward. Unfortunately my last push broke the
package in a way that was invisible locally due to the source already
being downloaded... A perfect example of why we should let ofborg
build!
clion: 2020.2.1 -> 2021.1.2
datagrip: 2020.2.2 -> 2021.1.2
goland: 2020.2.2 -> 2021.1.2
idea-community: 2020.2.1 -> 2021.1.2
idea-ultimate: 2020.2.1 -> 2021.1.2
mps: 2020.1.4 -> 2021.1
phpstorm: 2020.2.1 -> 2021.1.3
pycharm-community: 2020.2.1 -> 2021.1.2
pycharm-professional: 2020.2.1 -> 2021.1.2
rider: 2020.2.1 -> 2021.1.3
ruby-mine: 2020.2.1 -> 2021.1.2
webstorm: 2020.2.1 -> 2021.1.2
These updates have all been generated with the newly fixed update.pl
script.
The script update.pl, which gets the latest versions from upstream
web server, has been failing for roughly half a year. It failed to
handle the new url pattern of MPS. The script and the url pattern
is updated to work with the latest version.
The clion specific postFixup hook had to be adapted, because names
of shared libraries changed.
(cherry picked from commit 8593496928ebacc8d0779ff5706b8d511dbc0177)
(cherry picked from commit a887b48e1d)
nixos tests are blended with other system configurations, hence
their settings must be either enforced or defaulted.
This particular setting is set via lib.nixosSystem as
`system.nixos.revision = final.mkIf (self ? rev) self.rev;` which would
mean that without this change no flake generated nixos could be blended
with nixos testing.
This setting was made previously constant in
169c6b4b14 in order to avoid pointless
rebuilds of the testing VMs, but was set without enforcing it.
(cherry picked from commit 8bbdff4581)
Reading the release notes I got the impression that the latest (and
default) was GR3.8, but it is in fact 3.9. Make that more obvioius.
(cherry picked from commit c789c53ce5)
Currently fails with:
$ nixos-rebuild test --target-host myhost --use-remote-sudo
building Nix...
sudo: /run/current-system/sw/bin/sudo must be owned by uid 0 and have the setuid bit set
It seems setting an explicit PATH in the ssh command overrides the
remote setuid wrappers.
(cherry picked from commit 35a8d78e44)
They should have gnome-shell-extension prefix like most other extension packages.
This is what other distros listed on Repology use so Repology will be able to unify them.
Exception is chrome-gnome-shell, which is estabilished under that name.
(cherry picked from commit 5ba789eeca)
The only in-tree use of this is the DBus NixOS module, which always
passes apparmor (but the services.dbus.apparmor option defaults to
"disabled"). Set the default in the function as well so other users
of the function can take advantage of it.
(cherry picked from commit e9f45cc69d)
Mostly bug fixes, the following changes were required in the package:
* set-correct-program-name-for-sleep.patch needed to be rebased,
b08781e2a6e183986eb1c24f51cdeff879b7a6af partially implemented
the changes done in this patch, so we don't need to touch
password-store.sh anymore.
* Remove wayland patch since it is part of the release now
* Reworked assert logic wrt to x11-/wayland- and dmenuSupport:
passmenu now supports wayland as well via dmenu-wayland. Sadly
the choice of menu is not changeable, pending
https://lists.zx2c4.com/pipermail/password-store/2021-January/004363.html
* Rebased no-darwin-getopt.patch
* Note that f.el is no longer required
Reviews would be appreciated, I might've missed something.
(cherry picked from commit 9b794eb21a)
Commit 3a2e0c36e7 has removed
`--reuse-key` from default renew options, yet the tests still expected
keys not to change. This assertion is now removed, as they are supposed
to change on each renew/change.
(cherry picked from commit b00bcf21ab)
Reload only works with a static configuration path as there is no way to
pass the dynamically generated config path to a running solanum
instance, therefore we symlink the configuration to
/etc/solanum/ircd.conf.
But that will prevent reloads of the ircd, because the systemd unit
wouldn't change when the configuration changes. That is why we add the
actual location of the config file to restartTriggers and enable
reloadIfChanged, so changes will not restart, but reload on changes.
(cherry picked from commit 60c62214f5)
Previously this defaulted to the default MOTD in the solanum source
tree, and I don't want my friends to laugh at me. Includes a patch to
the tests to ensure that the MOTD is actually set.
This replicates the fix done in #109705 (solanum is a fork of charybdis,
so they share fundamental logic for this).
Signed-off-by: Christine Dodrill <me@christine.website>
(cherry picked from commit b1fe9fab6f)
On `master` there are a few more bugfixes. Most notably, each message
sent by me was marked as "not sent" by WhatsApp for the last four days
until you hit a "Resend" in WhatsApp.
As the update to the latest `master` has solved the issue and being able
to correctly send messages is one of the core features of this package,
I decided to update the package in `nixpkgs` as well.
(cherry picked from commit 5c7156faf1)
GHC calls otool on darwin which is contained in the
stdenv.cc.bintools.bintools derivation and thus needs adding to the
runtime PATH of GHC. Since this is toolchain specific technically, we
check for cctools instead of darwin (although I don't know if GHC
or nixpkgs work on macOS without cctools).
This fixes usage of GHC in an environment where otool is not available
and more specifically in stdenvNoCC which is used by writers.writeHaskell.
Resolves#123228.
(cherry picked from commit 118b28a127)
Whatever change has necessitated
https://github.com/NixOS/nixpkgs/pull/122044, it also broke clangd --
<clang-wrapper>/resource-root/include is no longer automagically
searched for includes, which kills pretty much any indexing since that
directory contains vital stuff like stddef.h etc.
Fix by appending the directory to CPATH & CPLUS_INCLUDE_PATH in the
clangd wrapper.
(cherry picked from commit 8e06a39574)
server.py tries to launch a matrix_sso_helper binary when connecting to
a homeserver that uses some SSO mechanism instead of plain login and
password, but doesn't have $out/bin in $PATH.
Using substituteInPlace to patch server.py so that the helper process is
started by using its actual filesystem location instead of relying on
$PATH.
Fixes: https://github.com/NixOS/nixpkgs/issues/124186
(cherry picked from commit f7ccc5f35d)
Calling `nix run poetry` or another python package usually fails
because of the "pythonX" prefix in name.
Adjust mainProgram to ignore that prefix.
(cherry picked from commit b59875ef23)
fixes adding a new feed to ttrss.
the release only contains the fix, so it's low risk.
It's not a backport because the plugin has become unnecessary so we are
simply removing it from unstable.
Add support for folder jobs
(https://plugins.jenkins.io/cloudbees-folder/) by reworking the service
to support nested jobs.
This also fixes this deprecation warning (as a happy side effect):
WARNING:jenkins_jobs.cli.subcommand.test:(Deprecated) The default output behavior of `jenkins-jobs test` when given the --output flag will change in JJB 3.0. Instead of writing jobs to OUTPUT/jobname; they will be written to OUTPUT/jobname/config.xml. The new behavior can be enabled by the passing `--config-xml` parameter
(cherry picked from commit 4bcb22e17a)
```
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:617:7: error: use of undeclared
identifier 'TARGET_OS_IOS'
if (TARGET_OS_IOS || TARGET_OS_TV) return 6;
^
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:617:24: error: use of undeclared
identifier 'TARGET_OS_TV'
if (TARGET_OS_IOS || TARGET_OS_TV) return 6;
^
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:618:7: error: use of undeclared
identifier 'TARGET_OS_WATCH'
if (TARGET_OS_WATCH) return 13;
^
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:687:7: error: use of undeclared
identifier 'TARGET_OS_IOS'
if (TARGET_OS_IOS || TARGET_OS_TV)
^
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:687:24: error: use of undeclared
identifier 'TARGET_OS_TV'
if (TARGET_OS_IOS || TARGET_OS_TV)
^
/tmp/nix-build-compiler-rt-libc-12.0.0.drv-0/compiler-rt-12.0.0.src/lib/sanitizer_common/sanitizer_mac.cpp:689:12: error: use of undeclared
identifier 'TARGET_OS_WATCH'
else if (TARGET_OS_WATCH)
^
6 errors generated.
```
(cherry picked from commit cf4e1b9e62)
Firefox 81 introduced a new print dialog. Under NixOS, this dialog
offers only "Save as PDF" as the destination. To print to a real
printer, one has to click "Print using the system dialog" and print
from there. This is not only one unnecessary extra click, but the
system dialog also does not offer preview.
With this commit, Firefox starts offering real printers in its
printing dialog, removing the above mentioned deficiencies.
CUPS is needed because Firefox uses dlopen() to load libcups.so.2 at
runtime. See
https://searchfox.org/mozilla-central/rev/b52cf6bbe214bd9d93ed9333d0403f7d556ad7c8/widget/nsCUPSShim.cpp#28
(cherry picked from commit 5102a12471)
This executable is required to fix a startup error:
[990:990:0609/092114.482805:FATAL:double_fork_and_exec.cc(131)] execv /nix/store/k02xhxzn6sn2cihaal68wwsyk8cg9pkg-chromium-unwrapped-93.0.4535.3/libexec/chromium/crashpad_handler: No such file or directory (2)
Unfortunately Chromium M93 still segfaults in the VM test:
machine # [0610/100626.225850:ERROR:process_memory_range.cc(75)] read out of range
machine # [0610/100626.227312:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq: No such file or directory (2)
machine # [0610/100626.240410:ERROR:file_io_posix.cc(144)] open /sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq: No such file or directory (2)
machine # [ 19.810981] systemd-coredump[1015]: Process 987 (chromium) of user 1000 dumped core.
(cherry picked from commit 1d6a0d3cf2)
https://chromereleases.googleblog.com/2021/06/stable-channel-update-for-desktop.html
This update includes 14 security fixes. Google is aware that an exploit
for CVE-2021-30551 exists in the wild.
CVEs:
CVE-2021-30544 CVE-2021-30545 CVE-2021-30546 CVE-2021-30547
CVE-2021-30548 CVE-2021-30549 CVE-2021-30550 CVE-2021-30551
CVE-2021-30552 CVE-2021-30553
(cherry picked from commit 053f1dc490)
The build was failing with:
clang++: error: unknown argument: '-fsanitize-ignorelist=../../tools/cfi/ignores.txt'
(cherry picked from commit 950b321244)
this is a reintroduction of CVE-2018-1000518 which i had been calling
CVE-2018-1000518-redux before it got its own CVE assigned
(cherry picked from commit aba83e7f87)
(yes, a forward cherry-pick because i fully expected the websockets
9.1 to make it into 21.05)
Firefox has been decoupled from the system certificate store since the
nss p11-kit integration in combination with our cacert package does not
expose CKA_NSS_MOZILLA_CA_POLICY, which among other things is required
for addon updates.
(cherry picked from commit 2d4ed9bae6)
The test doesn't evaluate since #125469 because Linux 5.11 got removed
as it's EOL.
As this fixes the evaluation of the test and it only removes a
declaration that was apparently forgotten, I figured that a push to
unbreak the test is fine.
(cherry picked from commit 10eab5b6b3)
`nix-direnv` depends on `nixUnstable` for flakes support, but that
causes it to print a warning unless your user is trusted or the
`experimental-features` setting is enabled. Neither of these are
desirable (note that setting `experimental-features` in nix.conf causes
warnings on `nixStable`).
Since flakes are experimental, this commit makes `nixStable` the default
and optionally allows `nixUnstable` with a new flag `enableFlakes`,
prioritising a good experience for users not using experimental
features.
(cherry picked from commit 03310df843)
Fixes https://nvd.nist.gov/vuln/detail/CVE-2021-33896.
The current 9acb54df9254609f2fe4de83c9047d408412de28 patch landed in
dino as 4592b72dfa324d8a4b9f8c25b359110889b2206c. Removing it from the
patch list.
(cherry picked from commit 70173c1519)
In order to make the man pages accessible, the previous code used
nix-support/propagated-user-env-packages. However this file is also used to set
the PATH when the application is executed with `nix run`, thus including the
wrapped and the wrappee in the environment.
Having the wrappee enumerated first in the environment caused `firefox` to
default to the wrappee, and as such not being able to find a proper GTK. This
was a source of failures while opening a file-picker.
This change removes the code to propagate the wrappe in the environment, as the
man pages are already linked in the wrapper output.
(cherry picked from commit efef092ba5)
clang needs to find headers + libraries for compiling with libc++.
On Darwin we will include CXX headers when compiling C.
This closes#124396
(cherry picked from commit 99b351b4bd)
The upstream tarball changed. Specifically, files now extract onto a
box2d-2.3.1/ directory rather than Box2D-2.3.1/. The files themselves
(and their contents) seem to remain the same.
(cherry picked from commit a8b85f6d68)
The old URL doesn't seem to be available anymore (fails with 403
Forbidden error).
The tarball in the new URL contains a few changes from the old one, so
the hash has changed.
Specifically, only the following files have changed:
sundials-5.7.0/doc/arkode/ark_examples.pdf
sundials-5.7.0/doc/arkode/ark_guide.pdf
sundials-5.7.0/doc/cvode/cv_examples.pdf
sundials-5.7.0/doc/cvode/cv_guide.pdf
sundials-5.7.0/doc/cvodes/cvs_examples.pdf
sundials-5.7.0/doc/cvodes/cvs_guide.pdf
sundials-5.7.0/doc/ida/ida_examples.pdf
sundials-5.7.0/doc/ida/ida_guide.pdf
sundials-5.7.0/doc/idas/idas_examples.pdf
sundials-5.7.0/doc/idas/idas_guide.pdf
sundials-5.7.0/doc/kinsol/kin_examples.pdf
sundials-5.7.0/doc/kinsol/kin_guide.pdf
(cherry picked from commit 4ab604c4c4)
The old URL wasn't available anymore.
There is a minor version change but it's only an unimportant change to
the Makefile and some changes to the documentation.
(cherry picked from commit 908569cf06)
62733b37b4 broke evaluation in all
places `pkgs.mysql` was used. Fix this by changing all occurrences to
`pkgs.mariadb`.
(cherry picked from commit 59e0120aa5)
If run as root we were leaking mounts to the parent namespace,
which lead to an error when removing the temporary mountroot.
To fix this we remount the whole tree as private as soon as we created
the new mountenamespace.
(cherry picked from commit 43908f4c1d)
ChangeLog: 1886c8abed/CHANGELOG.md (560-beta6-2021-05-31)
Even though this isn't explicitly noted in the Changelog, this seems to
have fixed the Element integration for me.
Additionally, I added a (hacky) `xdg-open` wrapper which removes the
`GDK_BACKEND` variable to fix the XWayland integration[1]. The problem
is that if a Firefox is running with Wayland (`ferdi` is running under
X11) and `GDK_BACKEND=x11` is passed to the `xdg-open` (and thus
`firefox`) process, Firefox refuses to start since another instance of
it is running under Wayland (but attempts to start in X11 mode because of
`GDK_BACKEND=x11`).
[1] https://github.com/electron/electron/issues/28436
(cherry picked from commit cd4ad7d2fe)
Rambox hasn't had a stable release in a while and an increasing number
of issues which is why I don't intend to use this anymore.
While taking a closer look at the source I also realized that it uses
Electron 7.2.4[1]. This is not only EOLed[2], it also contains a few
security vulnerabilities which is why I decided to mark it as insecure.
A few (most likely not all) vulnerabilities can be found by looking at
the Electron 7 changelog[3]: after 7.2.4 there were a few more releases
with security backports - mostly from Chromium. Security issues that
were found later on (and are probably exploitable on the dependency
chain of rambox) aren't listed here. I only added two issues that seemed
applicable to `rambox`, but I haven't researched enough to check the
other ones.
[1] https://github.com/ramboxapp/community-edition/blob/0.7.7/package.json#L70
[2] https://www.electronjs.org/docs/tutorial/support#currently-supported-versions
[3] https://www.electronjs.org/releases/stable?version=7
(cherry picked from commit e2a15cd395)
In 0.3.0 of the json-exporter[1] it was switched to a different jsonpath
library which made some changes - especially for spaces in keys -
necessary. Also I decided to remove the pretty-printed JSON as this
would interfere with the bash quoting too much. If one needs
pretty-printed output, they can still pipe the output to `jq`.
[1] https://github.com/prometheus-community/json_exporter/releases/tag/v0.3.0
(cherry picked from commit 976d668e5c)
The samba package was marked as broken, when enableGlusterFS is true.
The samba build with glusterfs fails due to API breakage that I am
unable to debug:
[3562/4088] Compiling source3/modules/vfs_virusfilter.c
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_pread’:
../../source3/modules/vfs_glusterfs.c:856:8: error: too few arguments to function ‘glfs_pread’
856 | ret = glfs_pread(glfd, data, n, offset, 0);
| ^~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:713:1: note: declared here
713 | glfs_pread(glfs_fd_t *fd, void *buf, size_t count, off_t offset, int flags,
| ^~~~~~~~~~
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_pread_do’:
../../source3/modules/vfs_glusterfs.c:938:16: error: too few arguments to function ‘glfs_pread’
938 | state->ret = glfs_pread(state->fd, state->buf, state->count,
| ^~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:713:1: note: declared here
713 | glfs_pread(glfs_fd_t *fd, void *buf, size_t count, off_t offset, int flags,
| ^~~~~~~~~~
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_pwrite_do’:
../../source3/modules/vfs_glusterfs.c:1077:16: error: too few arguments to function ‘glfs_pwrite’
1077 | state->ret = glfs_pwrite(state->fd, state->buf, state->count,
| ^~~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:717:1: note: declared here
717 | glfs_pwrite(glfs_fd_t *fd, const void *buf, size_t count, off_t offset,
| ^~~~~~~~~~~
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_pwrite’:
../../source3/modules/vfs_glusterfs.c:1161:8: error: too few arguments to function ‘glfs_pwrite’
1161 | ret = glfs_pwrite(glfd, data, n, offset, 0);
| ^~~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:717:1: note: declared here
717 | glfs_pwrite(glfs_fd_t *fd, const void *buf, size_t count, off_t offset,
| ^~~~~~~~~~~
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_fsync_do’:
../../source3/modules/vfs_glusterfs.c:1287:16: error: too few arguments to function ‘glfs_fsync’
1287 | state->ret = glfs_fsync(state->fd);
| ^~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:790:1: note: declared here
790 | glfs_fsync(glfs_fd_t *fd, struct glfs_stat *prestat,
| ^~~~~~~~~~
../../source3/modules/vfs_glusterfs.c: In function ‘vfs_gluster_ftruncate’:
../../source3/modules/vfs_glusterfs.c:1621:8: error: too few arguments to function ‘glfs_ftruncate’
1621 | ret = glfs_ftruncate(glfd, offset);
| ^~~~~~~~~~~~~~
In file included from ../../source3/modules/vfs_glusterfs.c:41:
/nix/store/0gzaf6fqgfxfns19zlc07dyjqigj7ak7-glusterfs-9.0/include/glusterfs/api/glfs.h:768:1: note: declared here
768 | glfs_ftruncate(glfs_fd_t *fd, off_t length, struct glfs_stat *prestat,
| ^~~~~~~~~~~~~~
../../source3/modules/vfs_virusfilter.c: In function ‘quarantine_create_dir’:
../../source3/modules/vfs_virusfilter.c:132:13: warning: implicit declaration of function ‘strlcat’; did you mean ‘strncat’? [-Wimplicit-function-declaration]
132 | cat_len = strlcat(new_dir, "/", len + 1);
| ^~~~~~~
| strncat
Waf: Leaving directory `/build/samba-4.14.4/bin/default'
Build failed
-> task in 'vfs_glusterfs.objlist' failed with exit status 1 (run with -v to display more information)
(cherry picked from commit fac761a55a)
This should help in rare hardware-specific situations where the root is
not automatically detected properly.
We search using a marker file. This should help some weird UEFI setups
where the root is set to `(hd0,msdos2)` by default.
Defaulting to `(hd0)` by looking for the ESP **will break themeing**. It
is unclear why, but files in `(hd0,msdos2)` are not all present as they
should be.
This also fixes an issue introduced with cb5c4fcd3c
where rEFInd stopped booting in many cases. This is because it ended up
using (hd0) rather than using the `search` which was happening
beforehand, which in turn uses (hd0,msdos2), which is the ESP.
Putting back the `search` here fixes that.
(cherry picked from commit 20b023b5ea)
This technically changes nothing. In practice `$root` is always the
"CWD", whether searched for automatically or not.
But this serves to announce we are relying on `$root`... I guess...
(cherry picked from commit c9bb054dd6)
This commit has been generated by maintainers/scripts/haskell/regenerate-hackage-packages.sh
Main point here is to apply the new cabal2nix-unstable generation with
a libNixName entry for libXScrnSaver, so greenclip builds again.
Usually, on the stable channel, we have a nss_latest attribute that is
more up to date than the nss attribute (which is usually frozen during
branch-off and only receives security updates). Cacerts are a sensitive
matter and should be updated more frequently than the stable NSS package,
if required. By making the update script aware of the nss_latest
attribute we can prefer that when it exists.
By having this change in the unstable branch of Nixpgks we can carry it
from release to release without requiring more churn from those doing
the stable release maintenance.
(cherry picked from commit 4e318bcca1)
Adds includeStorePaths, allowing the omission of the store paths.
You generally want to leave it on, but tooling may disable this
to insert the store paths more efficiently via other means, such
as bind mounting the host store.
(cherry picked from commit 5259d66b74)
Currently the docker client is only available on non-linux platforms as `docker`,
this makes the client available on linux and other platforms as `docker-client`.
(cherry picked from commit 7233acd515)
Make this use of config.allowAliases consistent with every other use
in the tree.
Signed-off-by: Anders Kaseorg <andersk@mit.edu>
(cherry picked from commit 4f0cf23ea3)
I'm not sure how the build broke[1] or how it worked before, but
the problem is zip is being used in place of p7zip, which obviously
fail as the flags have different meanings.
[1]: https://hydra.nixos.org/build/143354937
(cherry picked from commit 302d6b1b8b)
libccd has the common bug of assuming CMAKE_INSTALL_*DIR is relative. I have
submitted the fix upstream, but don't have much hope of getting it merged
because there have been no updates since 2018.
(cherry picked from commit 3d2092ab58)
This is because we are in a huge "rec" that takes precedence for
symbols in its scope, despite the more nested "with".
(cherry picked from commit 7ee8945a12)
The root filesystem resizing step, `resize2fs -M', does not provide any
control over the amount of slack left in the result. It can produce an
arbitrarily tight fit, depending on how well the payload aligns with
ext4 data structures.
This is problematic, as NixOS must create a few files and directories
during its first boot, before the root is enlarged to match the size of
the containing SD card.
An overly tight fit can cause failures in the first stage:
mkdir: can't create directory '/mnt-root/proc': No space left on device
or in the second stage:
install: cannot create directory '/var': No space left on device
A previous version of `make-ext4-fs' (before PR #79368) was explicitly
"reserving" 16 MiB of free space in the final filesystem. Manually
calculating the size of an ext4 filesystem is a perilous endeavor,
however, and the method it employed was apparently unreliable.
Reverting is consequently not a good option.
A solution would be to create some sort of "balloon" occupying inodes
and blocks in the image prior to invoking `resize2fs -M', and to remove
these temporary files/directories before the compression step.
This changeset takes the simpler approach of simply dropping the
resizing step.
Note that this does *not* result in a larger image in general, as the
current procedure does not truncate the `.img' file anyway. In fact, it
has been observed to yield *smaller* compressed images---probably
because of some "noise" left after resizing. E.g., before-vs-after:
-r--r--r-- 2 root root 607M 1. Jan 1970 nixos-sd-image-21.11pre-git-x86_64-linux.img.zst
-r--r--r-- 2 root root 606M 1. Jan 1970 nixos-sd-image-21.11pre-git-x86_64-linux.img.zst
(cherry picked from commit 7c2adb1d5c)
Reusing the same private/public key on renewal has two issues:
- some providers don't accept to sign the same public key
again (Buypass Go SSL)
- keeping the same private key forever partly defeats the purpose of
renewing the certificate often
Therefore, let's remove this option. People wanting to keep the same
key can set extraLegoRenewFlags to `[ --reuse-key ]` to keep the
previous behavior. Alternatively, we could put this as an option whose
default value is true.
(cherry picked from commit 632c8e1d54)
iptables is currently defined in `all-packages.nix` to be
iptables-compat. That package does however not contain `ethertypes`.
Only `iptables-nftables-compat` contains this file so the symlink
dangles.
(cherry picked from commit 2eeecef3fc)
Some python packages (e.g. functools32) are conditionally disabled for
certain interpreter versions by having them return null instead of a
derivation. `getRecursivePropagatedBuildInputs` fails if such a package
is part of the dependency tree. This commit makes it only recurse into
derivations and ignore everything else.
This also deduplicates the final plugin list to improve startup time.
(cherry picked from commit 9b3a2e66b1)
The import-ssl-module.patch does not seem to be need and the tool seem
to be working just fine. Drop the patch.
Signed-off-by: Otavio Salvador <otavio@ossystems.com.br>
(cherry picked from commit 94d5fd438d)
A secret key generated by the nixos module was misspelled, which could
possibly impact the security of session cookies.
To recover from this situation we will wipe all security keys that were
previously generated by the NixOS module, when the misspelled one is
found. This will result in all session cookies being invalidated. This
is confirmed by the wordpress documentation:
> You can change these at any point in time to invalidate all existing
> cookies. This does mean that all users will have to login again.
https://wordpress.org/support/article/editing-wp-config-php/#security-keys
Meanwhile this issue shouldn't be too grave, since the salting function
of wordpress will rely on the concatenation of both the user-provided
and automatically generated values, that are stored in the database.
> Secret keys are located in two places: in the database and in the
> wp-config.php file. The secret key in the database is randomly
> generated and will be appended to the secret keys in wp-config.php.
https://developer.wordpress.org/reference/functions/wp_salt/
Fixes: 2adb03fdae ("nixos/wordpress:
generate secrets locally")
Reported-by: Moritz Hedtke <Moritz.Hedtke@t-online.de>
(cherry picked from commit 724ed08df0)
Assert that the PostgreSQL version being deployed is the one used
upstream. Allow the user to override this assertion, since it's not
always possible or preferable to use the recommended one.
(cherry picked from commit 544adbfcab)
As per `man systemd.path`:
> When a service unit triggered by a path unit terminates
> (regardless whether it exited successfully or failed),
> monitored paths are checked immediately again,
> **and the service accordingly restarted instantly**.
Thus the existence of the path unit made it impossible to stop the
wireguard service using e.g.
systemctl stop wireguard-wg0.service
Systemd path units are not intended for program inputs such
as private key files.
This commit simply removes this usage; the private key is still
generated by the `generateKeyServiceUnit`.
(cherry picked from commit d344dccf3d)
Note that it made into 2 entries, one about new options in the first section.
Another in the breaking compatibility section due to the openFirewall option
which changes the behavior.
Co-authored-by: schmittlauch <t.schmittlauch+nixos@orlives.de>
(cherry picked from commit 93a80a4390)
The tests timeout on AArch64 (e.g. [0] and [1]), likely because the QEMU
option "-vga virtio" isn't supported there (unfortunately I currently
lack access to an AArch64 system with NixOS to investigate).
This also affects the test for Cage but that one is already limited to
x86_64-linux.
[0]: https://hydra.nixos.org/build/144148809
[1]: https://hydra.nixos.org/build/144103034
(cherry picked from commit abb9ea73f7)
* The options tlsKey and tlsCert require being accessible by DynamicUser at runtime, which currently requires copying the files into the matrix service state directory. Fixing this might require breaking changes. Thus the module should not be included in a stable release.
Since 21.01, poppler supports PDF signing. As applications like okular
start to make use of that feature, nss support for poppler is enabled by
default to avoid unnecessary package duplication.
When building a `minimal` version of poppler, nss is disabled as well.
closes#120928
(cherry picked from commit 5f9862d524)
There's a bit of LaTeX math code in a comment in a test file:
```
.. math:: L = 10 \cdot \\log_{10}{\\left(\\frac{MS}{p_r^2} \\right)}
```
which of course should be `\\cdot`.
I could patch it, but I think skipping deprecation warnings in tests is
probably ok... (If nothing else it's easier.)
(cherry picked from commit dbc085cb44)
There was an override of autoconfHook to use automake 1.15.x; I'm not
sure what changed since this package was added but it explicitly needs
1.16.1 or greater for the submodule libltdl.
Anyhow, just remove the override.
(cherry picked from commit 3e4aeea693)
The rpath structure for the bootstrap tools was reworked to minimize
the amount of rewriting required on unpack, but the test was not
updated to match the different structure.
Additionally [1] builds that use the bootstrap version of libc++
cannot find libc++abi if the reference includes the "lib"
component (ie, libc++ refers to libc++abi with
@rpath/lib/libc++abi.dylib).
[1] https://logs.nix.samueldr.com/nix-darwin/2021-05-18#4993282
Test failure observed on Hydra: https://hydra.nixos.org/build/143130126
(cherry picked from commit 38207735f4)
The default backend is chosen based on the content of the $DISPLAY
variable *and* a successfull call to libX11, loaded via dlopen().
The test fails because dlopen looks in /usr/lib and /lib, so matplotlib
falls back to a headless backend.
To reproduce try running:
$ nix-shell -I nixpkgs=$PWD -p \
'python3.withPackages (p: [ p.matplotlib ])' --run python
>>> import matplotlib.pyplot as plt
>>> assert plt.get_backend() == "TkAgg"
(cherry picked from commit 7e2ec8f8a1)
Since some GitLab versions, we need to remove the mail-smtp-pool
dependency from the Gemfile, or do other modifications.
Let's add those changes, that will probably be needed in the next
version as well, to the update script.
(cherry picked from commit 0c8f5b7b44)
nixos/filesystems: condition mount-pstore.service on unmounted /sys/fs/pstore
(cherry picked from commit d7555732bc)
Reason: activation throws an error from failing to start the unit
When using GNU binutils, clang passes the LLVMgold.so plugin to the
linker for certain operations that require special support in the linker
like doing link time optimization (LTO). When passing the plugin to the
linker's command line, clang assumes that llvm and itself are installed
in the same prefix and thus `/path/to/clang/bin/../lib/LLVMgold.so` is
the plugin.
Since we install clang and llvm to separate store paths, this assumption
does not hold. When clang-unwrapped only had a single output, we worked
around this issue by symlinking `$out/lib/LLVMgold.so` to
`${llvm}/lib/LLVMgold.so`. However since we split all llvm packages into
multiple outputs clang's `$out` no longer has a lib directory and clang
can't discover clangs lib output on its own. As a result LTO was broken.
Instead of introducing yet another hack and having a symlink to
LLVMgold.so in `$out/lib` (despite having `$lib/lib` as well), we patch
clang to use a hard coded path to `${libllvm.lib}/lib` for discovering
`LLVMgold.so`.
Resolves#123361.
(cherry picked from commit 3530837417)
Previously, mangleVarList would be used which would concatenate the
variables using a space as a separator. Paths are however separated by
`:` in PKG_CONFIG_PATH leading to entries being broken.
This is fixed by introducing mangleVarListGeneric which allows us to
specify the desired separator.
Reproducer for the issue prior to this change:
$ nix-shell -A pkgsLLVM.wayland
[nix-shell] $ pkg-config --libs expat
Package expat was not found in the pkg-config search path.
Perhaps you should add the directory containing `expat.pc'
to the PKG_CONFIG_PATH environment variable
No package 'expat' found
$ printf 'Host: %s\nBuild: %s' $PKG_CONFIG_PATH $PKG_CONFIG_PATH_FOR_BUILD
Host: /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/z3y9ska2h4l1map25m195iq577g7g3gz-libxml2-x86_64-unknown-linux-gnu-2.9.12-dev/lib/pkgconfig:/nix/store/lbz5m1s0r7zn0cxvl21czfspli6ribzb-zlib-1.2.11-x86_64-unknown-linux-gnu-dev/lib/pkgconfig:/nix/store/rfhvp8r8n3ygpzh8j0l34lk8hwwi3z0h-libffi-3.3-x86_64-unknown-linux-gnu-dev/lib/pkgconfig
Build: /nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig
strace reveals the issue:
stat("/nix/store/dw11ywy7qwfz53qisz0dggbgix88jah2-wayland-1.19.0-bin/lib/pkgconfig /nix/store/5h308a4ab8w7prcp8iflh5pnl78mayi2-expat-2.2.10-x86_64-unknown-linux-gnu-dev/lib/pkgconfig/expat-uninstalled.pc", 0x7fff49829fa0) = -1 ENOENT (No such file or directory)
In the pkg-config wrapper $PKG_CONFIG_PATH_FOR_BUILD and
$PKG_CONFIG_PATH are concatenated with a space which leads to two paths
being messed up. This issue likely only affects native cross
compilation.
(cherry picked from commit b11d65c850)
The configure script checks whether iconv supports errno. Unfortunately, on PHP < 8, the test program includes $PHP_ICONV_H_PATH, which defaults to FHS path so it fails to build:
conftest.c:13:10: fatal error: /usr/include/iconv.h: No such file or directory
13 | #include </usr/include/iconv.h>
| ^~~~~~~~~~~~~~~~~~~~~~
That causes the feature check to report a false negative, leading PHP to use a degraded code that returns PHP_ICONV_ERR_UNKNOWN when error occurs, breaking granular error handling in applications.
To prevent this, let’s just include <iconv.h>.
PHP 8 just uses include path so the detection works there: 7bd1d70341
(cherry picked from commit 024243bac4)
this switches from GitHub to PyPi, because the build requires
a proper release tarball
also PyPi release does contain tests, so we don't strictly require
GitHub checkout
(cherry picked from commit 0f2201119b)
This reverts commit 45bd7b39a4.
The database for users on NixOS 20.09 is still encrypted so we don't
need this wrapper for users that upgrade from NixOS 20.09 to 21.05.
<!-- Please check what applies. Note that these are not hard requirements but merely serve as information for reviewers. -->
- [ ] Tested using sandboxing ([nix.useSandbox](https://nixos.org/nixos/manual/options.html#opt-nix.useSandbox) on NixOS, or option `sandbox` in [`nix.conf`](https://nixos.org/nix/manual/#sec-conf-file) on non-NixOS linux)
- Built on platform(s)
- [ ]x86_64-linux
- [ ]aarch64-linux
- [ ]x86_64-darwin
- [ ]aarch64-darwin
- [ ]For non-Linux: Is `sandbox = true` set in `nix.conf`? (See [Nix manual](https://nixos.org/manual/nix/stable/command-ref/conf-file.html))
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [ ]Tested via one or more NixOS test(s) if existing and applicable for the change (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- [ ]Tested compilation of all pkgs that depend on this change using`nix-shell -p nixpkgs-review --run "nixpkgs-review wip"`
- [ ] Tested execution of all binary files (usually in `./result/bin/`)
- [ ] Added a release notes entry if the change is major or breaking
- To remove the stale label, just leave a new comment.
- _How to find the right people to ping?_ → [`git blame`](https://git-scm.com/docs/git-blame) to the rescue! (or GitHub's history and blame buttons.)
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/), [our Matrix room](https://matrix.to/#/#nix:nixos.org), or on the [#nixos IRC channel](https://web.libera.chat/#nixos).
- You can always ask for help on [our Discourse Forum](https://discourse.nixos.org/) or on the [#nixos IRC channel](https://webchat.freenode.net/#nixos).
Bot-based backport to `${target_branch}`, triggered by a label in #${pull_number}.
* [ ] Before merging, ensure that this backport complies with the [Criteria for Backporting](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#criteria-for-backporting-changes).
* Even as a non-commiter, if you find that it does not comply, leave a comment.
Periodic merge from `${{ matrix.pairs.from }}` into `${{ matrix.pairs.into }}` has [failed](https://github.com/NixOS/nixpkgs/actions/runs/${{ github.run_id }}).
Periodic merge from `${{ matrix.pairs.from }}` into `${{ matrix.pairs.into }}` has [failed](https://github.com/NixOS/nixpkgs/actions/runs/${{ github.run_id }}).
Note: contributing implies licensing those contributions
under the terms of [COPYING](COPYING), which is an MIT-like license.
under the terms of [COPYING](../COPYING), which is an MIT-like license.
## Opening issues
@@ -59,27 +59,6 @@ Follow these steps to backport a change into a release branch in compliance with
5. Push to GitHub and open a backport pull request. Make sure to select the release branch (e.g. `release-20.09`) as the target branch of the pull request, and link to the pull request in which the original change was comitted to `master`. The pull request title should be the commit title with the release version as prefix, e.g. `[20.09]`.
6. When the backport pull request is merged and you have the necessary privileges you can also replace the label `9.needs: port to stable` with `8.has: port to stable` on the original pull request. This way maintainers can keep track of missing backports easier.
## Criteria for Backporting changes
Anything that does not cause user or downstream dependency regressions can be backported. This includes:
- New Packages / Modules
- Security / Patch updates
- Version updates which include new functionality (but no breaking changes)
- Services which require a client to be up-to-date regardless. (E.g. `spotify`, `steam`, or `discord`)
- Security critical applications (E.g. `firefox`)
## Generating 21.11 Release Notes
(This section also applies to backporting 21.05 release notes: substitute "rl-2111" for "rl-2105".)
Documentation in nixpkgs is transitioning to a markdown-centric workflow. Release notes now require a translation step to convert from markdown to a compatible docbook document.
Steps for updating 21.11 Release notes:
1. Edit `nixos/doc/manual/release-notes/rl-2111.section.md` with the desired changes
2. Run `./nixos/doc/manual/md-to-db.sh` to render `nixos/doc/manual/from_md/release-notes/rl-2111.section.xml`
3. Include changes to `rl-2111.section.md` and `rl-2111.section.xml` in the same commit.
## Reviewing contributions
See the nixpkgs manual for more details on how to [Review contributions](https://nixos.org/nixpkgs/manual/#chap-reviewing-contributions).
When using Nix, you will frequently need to download source code and other files from the internet. For this purpose, Nix provides the [_fixed output derivation_](https://nixos.org/manual/nix/stable/#fixed-output-drvs) feature and Nixpkgs provides various functions that implement the actual fetching from various protocols and services.
When using Nix, you will frequently need to download source code and other files from the internet. Nixpkgs comes with a few helper functions that allow you to fetch fixed-output derivations in a structured way.
## Caveats
Because fixed output derivations are _identified_ by their hash, a common mistake is to update a fetcher's URL or a version parameter, without updating the hash. **This will cause the old contents to be used.** So remember to always invalidate the hash argument.
For those who develop and maintain fetchers, a similar problem arises with changes to the implementation of a fetcher. These may cause a fixed output derivation to fail, but won't normally be caught by tests because the supposed output is already in the store or cache. For the purpose of testing, you can use a trick that is embodied by the [`invalidateFetcherByDrvHash`](#sec-pkgs-invalidateFetcherByDrvHash) function. It uses the derivation `name` to create a unique output path per fetcher implementation, defeating the caching precisely where it would be harmful.
## `fetchurl` and `fetchzip` {#fetchurl}
Two basic fetchers are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
The two fetcher primitives are `fetchurl` and `fetchzip`. Both of these have two required arguments, a URL and a hash. The hash is typically `sha256`, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use `sha256`. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
```nix
{stdenv,fetchurl}:
@@ -28,58 +20,59 @@ The main difference between `fetchurl` and `fetchzip` is in how they store the c
`fetchpatch` works very similarly to `fetchurl` with the same arguments expected. It expects patch files as a source and performs normalization on them before computing the checksum. For example it will remove comments or other unstable parts that are sometimes added by version control systems and can change over time.
Most other fetchers return a directory rather than a single file.
## `fetchsvn` {#fetchsvn}
Other fetcher functions allow you to add source code directly from a VCS such as subversion or git. These are mostly straightforward nambes based on the name of the command used with the VCS system. Because they give you a working repository, they act most like `fetchzip`.
## `fetchsvn`
Used with Subversion. Expects `url` to a Subversion directory, `rev`, and `sha256`.
## `fetchgit` {#fetchgit}
## `fetchgit`
Used with Git. Expects `url` to a Git repo, `rev`, and `sha256`. `rev` in this case can be full the git commit id (SHA1 hash) or a tag name like `refs/tags/v1.0`.
Additionally the following optional arguments can be given: `fetchSubmodules = true` makes `fetchgit` also fetch the submodules of a repository. If `deepClone` is set to true, the entire repository is cloned as opposing to just creating a shallow clone. `deepClone = true` also implies `leaveDotGit = true` which means that the `.git` directory of the clone won't be removed after checkout.
## `fetchfossil` {#fetchfossil}
## `fetchfossil`
Used with Fossil. Expects `url` to a Fossil archive, `rev`, and `sha256`.
## `fetchcvs` {#fetchcvs}
## `fetchcvs`
Used with CVS. Expects `cvsRoot`, `tag`, and `sha256`.
## `fetchhg` {#fetchhg}
## `fetchhg`
Used with Mercurial. Expects `url`, `rev`, and `sha256`.
A number of fetcher functions wrap part of `fetchurl` and `fetchzip`. They are mainly convenience functions intended for commonly used destinations of source code in Nixpkgs. These wrapper fetchers are listed below.
## `fetchFromGitHub` {#fetchfromgithub}
## `fetchFromGitHub`
`fetchFromGitHub` expects four arguments. `owner` is a string corresponding to the GitHub user or organization that controls this repository. `repo` corresponds to the name of the software repository. These are located at the top of every GitHub HTML page as `owner`/`repo`. `rev` corresponds to the Git commit hash or tag (e.g `v1.0`) that will be downloaded from Git. Finally, `sha256` corresponds to the hash of the extracted directory. Again, other hash algorithms are also available but `sha256` is currently preferred.
`fetchFromGitHub` uses `fetchzip` to download the source archive generated by GitHub for the specified revision. If `leaveDotGit`, `deepClone` or `fetchSubmodules` are set to `true`, `fetchFromGitHub` will use `fetchgit` instead. Refer to its section for documentation of these options.
## `fetchFromGitLab` {#fetchfromgitlab}
## `fetchFromGitLab`
This is used with GitLab repositories. The arguments expected are very similar to fetchFromGitHub above.
## `fetchFromGitiles` {#fetchfromgitiles}
## `fetchFromGitiles`
This is used with Gitiles repositories. The arguments expected are similar to fetchgit.
## `fetchFromBitbucket` {#fetchfrombitbucket}
## `fetchFromBitbucket`
This is used with BitBucket repositories. The arguments expected are very similar to fetchFromGitHub above.
## `fetchFromSavannah` {#fetchfromsavannah}
## `fetchFromSavannah`
This is used with Savannah repositories. The arguments expected are very similar to fetchFromGitHub above.
## `fetchFromRepoOrCz` {#fetchfromrepoorcz}
## `fetchFromRepoOrCz`
This is used with repo.or.cz repositories. The arguments expected are very similar to fetchFromGitHub above.
## `fetchFromSourcehut` {#fetchfromsourcehut}
## `fetchFromSourcehut`
This is used with sourcehut repositories. The arguments expected are very similar to fetchFromGitHub above. Don't forget the tilde (~) in front of the user name!
`pkgs.appimageTools` is a set of functions for extracting and wrapping [AppImage](https://appimage.org/) files. They are meant to be used if traditional packaging from source is infeasible, or it would take too long. To quickly run an AppImage file, `pkgs.appimage-run` can be used as well.
::: {.warning}
::: warning
The `appimageTools` API is unstable and may be subject to backwards-incompatible changes in the future.
`pkgs.dockerTools` is a set of functions for creating and manipulating Docker images according to the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120). Docker itself is not used to perform any of the operations done by these functions.
`pkgs.dockerTools` is a set of functions for creating and manipulating Docker images according to the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120). Docker itself is not used to perform any of the operations done by these functions.
## buildImage {#ssec-pkgs-dockerTools-buildImage}
@@ -52,7 +52,7 @@ The above example will build a Docker image `redis/latest` from the given base i
> **_NOTE:_** Using this parameter requires the `kvm` device to be available.
-`config` is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions).
-`config` is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions).
After the new layer has been created, its closure (to which `contents`, `config` and `runAsRoot` contribute) will be copied in the layer itself. Only new dependencies that are not already in the existing layers will be copied.
@@ -14,7 +14,7 @@ Currently, `makeSnap` does not support creating GUI stubs.
The following expression packages GNU Hello as a Snapcraft snap.
```{#ex-snapTools-buildSnap-hello .nix}
```{#ex-snapTools-buildSnap-hello .nix}
let
inherit (import <nixpkgs> { }) snapTools hello;
in snapTools.makeSnap {
@@ -35,7 +35,7 @@ in snapTools.makeSnap {
Graphical programs require many more integrations with the host. This example uses Firefox as an example, because it is one of the most complicated programs we could package.
Certain calls in glibc require access to runtime files found in /etc such as `/etc/protocols` or `/etc/services` -- [getprotobyname](https://linux.die.net/man/3/getprotobyname) is one such function.
On non-NixOS distributions these files are typically provided by packages (i.e. [netbase](https://packages.debian.org/sid/netbase)) if not already pre-installed in your distribution. This can cause non-reproducibility for code if they rely on these files being present.
If [iana-etc](https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.iana-etc.x86_64-linux) is part of your _buildInputs_ then it will set the environment varaibles `NIX_ETC_PROTOCOLS` and `NIX_ETC_SERVICES` to the corresponding files in the package through a _setup-hook_.
Nixpkg's version of [glibc](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/glibc/default.nix) has been patched to check for the existence of these environment variables. If the environment variable are *not set*, then it will attempt to find the files at the default location within _/etc_.
## Build wrapped Firefox with extensions and policies {#build-wrapped-firefox-with-extensions-and-policies}
## Build wrapped Firefox with extensions and policies
The `wrapFirefox` function allows to pass policies, preferences and extension that are available to Firefox. With the help of `fetchFirefoxAddon` this allows build a Firefox version that already comes with addons pre-installed:
The `wrapFirefox` function allows to pass policies, preferences and extension that are available to firefox. With the help of `fetchFirefoxAddon` this allows build a firefox version that already comes with addons pre-installed:
```nix
{
# Nix firefox addons only work with the firefox-esr package.
myFirefox=wrapFirefoxfirefox-esr-unwrapped{
myFirefox=wrapFirefoxfirefox-unwrapped{
nixExtensions=[
(fetchFirefoxAddon{
name="ublock";# Has to be unique!
@@ -46,7 +45,5 @@ or type into the Firefox url bar: `about:policies#documentation`.
Nix installed addons do not have a valid signature, which is why signature verification is disabled. This does not compromise security because downloaded addons are checksumed and manual addons can't be installed. Also make sure that the `name` field of fetchFirefoxAddon is unique. If you remove an addon from the nixExtensions array, rebuild and start Firefox the removed addon will be completly removed with all of its settings.
## Troubleshooting {#sec-firefox-troubleshooting}
If addons are marked as broken or the signature is invalid, make sure you have Firefox ESR installed. Normal Firefox does not provide the ability anymore to disable signature verification for addons thus nix addons get disabled by the normal Firefox binary.
If addons do not appear installed although they have been defined in your nix configuration file reset the local addon state of your Firefox profile by clicking `help -> restart with addons disabled -> restart -> refresh firefox`. This can happen if you switch from manual addon mode to nix addon mode and then back to manual mode and then again to nix addon mode.
@@ -16,7 +16,7 @@ How to add a new (major) version of the Linux kernel to Nixpkgs:
1. Copy the old Nix expression (e.g. `linux-2.6.21.nix`) to the new one (e.g. `linux-2.6.22.nix`) and update it.
2. Add the new kernel to the `kernels` attribute set in `linux-kernels.nix` (e.g., create an attribute `kernel_2_6_22`).
2. Add the new kernel to `all-packages.nix` (e.g., create an attribute `kernel_2_6_22`).
3. Now we’re going to update the kernel configuration. First unpack the kernel. Then for each supported platform (`i686`, `x86_64`, `uml`) do the following:
@@ -36,6 +36,6 @@ How to add a new (major) version of the Linux kernel to Nixpkgs:
5. Copy `.config` over the new config file (e.g. `config-2.6.22-i686-smp`).
4. Test building the kernel: `nix-build -A linuxKernel.kernels.kernel_2_6_22`. If it compiles, ship it! For extra credit, try booting NixOS with it.
4. Test building the kernel: `nix-build -A kernel_2_6_22`. If it compiles, ship it! For extra credit, try booting NixOS with it.
5. It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the `linuxPackagesFor` function in `linux-kernels.nix` (such as the NVIDIA drivers, AUFS, etc.). If the updated packages aren’t backwards compatible with older kernels, you may need to keep the older versions around.
5. It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the `linuxPackagesFor` function in `all-packages.nix` (such as the NVIDIA drivers, AUFS, etc.). If the updated packages aren’t backwards compatible with older kernels, you may need to keep the older versions around.
@@ -4,11 +4,11 @@ OpenGL support varies depending on which hardware is used and which drivers are
Broadly, we support both GL vendors: Mesa and NVIDIA.
## NixOS Desktop {#nixos-desktop}
## NixOS Desktop
The NixOS desktop or other non-headless configurations are the primary target for OpenGL libraries and applications. The current solution for discovering which drivers are available is based on [libglvnd](https://gitlab.freedesktop.org/glvnd/libglvnd). `libglvnd` performs "vendor-neutral dispatch", trying a variety of techniques to find the system's GL implementation. In practice, this will be either via standard GLX for X11 users or EGL for Wayland users, and supporting either NVIDIA or Mesa extensions.
## Nix on GNU/Linux {#nix-on-gnulinux}
## Nix on GNU/Linux
If you are using a non-NixOS GNU/Linux/X11 desktop with free software video drivers, consider launching OpenGL-dependent programs from Nixpkgs with Nixpkgs versions of `libglvnd` and `mesa.drivers` in `LD_LIBRARY_PATH`. For Mesa drivers, the Linux kernel version doesn't have to match nixpkgs.
@@ -20,7 +20,6 @@ Use `programs.steam.enable = true;` if you want to add steam to systemPackages a
## Troubleshooting {#sec-steam-troub}
- **Steam fails to start. What do I do?**
Try to run
```ShellSession
@@ -33,30 +32,37 @@ Use `programs.steam.enable = true;` if you want to add steam to systemPackages a
- The `newStdcpp` parameter was removed since NixOS 17.09 and should not be needed anymore.
- Steam ships statically linked with a version of libcrypto that conflics with the one dynamically loaded by radeonsi_dri.so. If you get the error
```
steam.sh: line 713: 7842 Segmentation fault (core dumped)
```
have a look at [this pull request](https://github.com/NixOS/nixpkgs/pull/20269).
- **Java**
1. There is no java in steam chrootenv by default. If you get a message like
```
/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found
```
```
/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found
```
you need to add
You need to add
```nix
steam.override { withJava = true; };
```
```nix
steam.override { withJava = true; };
```
## steam-run {#sec-steam-run}
The FHS-compatible chroot used for Steam can also be used to run other Linux games that expect a FHS environment. To use it, install the `steam-run-native` package and run the game with
The FHS-compatible chroot used for steam can also be used to run other linux games that expect a FHS environment. To do it, add
```nix
pkgs.steam.override ({
nativeOnly = true;
newStdcpp = true;
}).run
```
to your configuration, rebuild, and run the game with
The Nix expressions for the X.org packages reside in `pkgs/servers/x11/xorg/default.nix`. This file is automatically generated from lists of tarballs in an X.org release. As such it should not be modified directly; rather, you should modify the lists, the generator script or the file `pkgs/servers/x11/xorg/overrides.nix`, in which you can override or add to the derivations produced by the generator.
## Katamari Tarballs {#katamari-tarballs}
## Katamari Tarballs
X.org upstream releases used to include [katamari](https://en.wiktionary.org/wiki/%E3%81%8B%E3%81%9F%E3%81%BE%E3%82%8A) releases, which included a holistic recommended version for each tarball, up until 7.7. To create a list of tarballs in a katamari release:
The upstream release process for [X11R7.8](https://x.org/wiki/Releases/7.8/) does not include a planned katamari. Instead, each component of X.org is released as its own tarball. We maintain `pkgs/servers/x11/xorg/tarballs.list` as a list of tarballs for each individual package. This list includes X.org core libraries and protocol descriptions, extra newer X11 interface libraries, like `xorg.libxcb`, and classic utilities which are largely unused but still available if needed, like `xorg.imake`.
For each of the tarballs in the `.list` files, the script downloads it, unpacks it, and searches its `configure.ac` and `*.pc.in` files for dependencies. This information is used to generate `default.nix`. The generator caches downloaded tarballs between runs. Pay close attention to the `NOT FOUND: $NAME` messages at the end of the run, since they may indicate missing dependencies. (Some might be optional dependencies, however.)
## Overriding the Generator {#overriding-the-generator}
## Overriding the Generator
If the expression for a package requires derivation attributes that the generator cannot figure out automatically (say, `patches` or a `postInstall` hook), you should modify `pkgs/servers/x11/xorg/overrides.nix`.
@@ -37,7 +37,7 @@ This works just like `runCommand`. The only difference is that it also provides
Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network roundrip and can speed up a build.
::: {.note}
::: note
This sets [`allowSubstitutes` to `false`](https://nixos.org/nix/manual/#adv-attr-allowSubstitutes), so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g. just copying some files to a different location or adding symlinks), because there the `system` is usually the same as `builtins.currentSystem`.
:::
@@ -47,28 +47,6 @@ These functions write `text` to the Nix store. This is useful for creating scrip
Many more commands wrap `writeTextFile` including `writeText`, `writeTextDir`, `writeScript`, and `writeScriptBin`. These are convenience functions over `writeTextFile`.
This can be used to easily produce a shell script that has some dependencies (`runtimeInputs`). It automatically sets the `PATH` of the script to contain all of the listed inputs, sets some sanity shellopts (`errexit`, `nounset`, `pipefail`), and checks the resulting script with [`shellcheck`](https://github.com/koalaman/shellcheck).
For example, look at the following code:
```nix
writeShellApplication{
name="show-nixos-org";
runtimeInputs=[curlw3m];
text=''
curl-s'https://nixos.org'|w3m-dump-Ttext/html
'';
}
```
Unlike with normal `writeShellScriptBin`, there is no need to manually write out `${curl}/bin/curl`, setting the PATH
was handled by `writeShellApplication`. Moreover, the script is being checked with `shellcheck` for more strict
validation.
## `symlinkJoin` {#trivial-builder-symlinkJoin}
This can be used to put many derivations into the same directory structure. It works by creating a new derivation and adding symlinks to each of the paths listed. It expects two arguments, `name`, and `paths`. `name` is the name used in the Nix store path for the created derivation. `paths` is a list of paths that will be symlinked. These paths can be to Nix store derivations or any other subdirectory contained within.
- Do not use tab characters, i.e. configure your editor to use soft tabs. For instance, use `(setq-default indent-tabs-mode nil)` in Emacs. Everybody has different tab settings so it’s asking for trouble.
- Use `lowerCamelCase` for variable names, not `UpperCamelCase`. Note, this rule does not apply to package attribute names, which instead follow the rules in [](#sec-package-naming).
- Use `lowerCamelCase` for variable names, not `UpperCamelCase`. Note, this rule does not apply to package attribute names, which instead follow the rules in <xref linkend="sec-package-naming"/>.
- Function calls with attribute set arguments are written as
@@ -181,23 +181,11 @@
rev = "${version}";
```
- Building lists conditionally _should_ be done with `lib.optional(s)` instead of using `if cond then [ ... ] else null` or `if cond then [ ... ] else [ ]`.
```nix
buildInputs = lib.optional stdenv.isDarwin iconv;
```
instead of
```nix
buildInputs = if stdenv.isDarwin then [ iconv ] else null;
```
As an exception, an explicit conditional expression with null can be used when fixing a important bug without triggering a mass rebuild.
If this is done a follow up pull request _should_ be created to change the code to `lib.optional(s)`.
- Arguments should be listed in the order they are used, with the exception of `lib`, which always goes first.
- The top-level `lib` must be used in the master and 21.05 branch over its alias `stdenv.lib` as it now causes evaluation errors when aliases are disabled which is the case for ofborg.
`lib` is unrelated to `stdenv`, and so `stdenv.lib` should only be used as a convenience alias when developing locally to avoid having to modify the function inputs just to test something out.
## Package naming {#sec-package-naming}
The key words _must_, _must not_, _required_, _shall_, _shall not_, _should_, _should not_, _recommended_, _may_, and _optional_ in this section are to be interpreted as described in [RFC 2119](https://tools.ietf.org/html/rfc2119). Only _emphasized_ words are to be interpreted in this way.
@@ -224,7 +212,7 @@ There are a few naming guidelines:
- Dashes in the package name _should_ be preserved in new variable names, rather than converted to underscores or camel cased — e.g., `http-parser` instead of `http_parser` or `httpParser`. The hyphenated style is preferred in all three package names.
- If there are multiple versions of a package, this _should_ be reflected in the variable names in `all-packages.nix`, e.g. `json-c-0-9` and `json-c-0-11`. If there is an obvious “default” version, make an attribute like `json-c = json-c-0-9;`. See also [](#sec-versioning)
- If there are multiple versions of a package, this _should_ be reflected in the variable names in `all-packages.nix`, e.g. `json-c-0-9` and `json-c-0-11`. If there is an obvious “default” version, make an attribute like `json-c = json-c-0-9;`. See also <xref linkend="sec-versioning" />
## File naming and organisation {#sec-organisation}
@@ -477,9 +465,9 @@ Preferred source hash type is sha256. There are several ways to get it.
For package updates it is enough to change one symbol to make hash fake. For new packages, you can use `lib.fakeSha256`, `lib.fakeSha512` or any other fake hash.
This is last resort method when reconstructing source URL is non-trivial and `nix-prefetch-url -A` isn’t applicable (for example, [one of `kodi` dependencies](https://github.com/NixOS/nixpkgs/blob/d2ab091dd308b99e4912b805a5eb088dd536adb9/pkgs/applications/video/kodi/default.nix#L73)). The easiest way then would be replace hash with a fake one and rebuild. Nix build will fail and error message will contain desired hash.
This is last resort method when reconstructing source URL is non-trivial and `nix-prefetch-url -A` isn't applicable (for example, [one of `kodi` dependencies](https://github.com/NixOS/nixpkgs/blob/d2ab091dd308b99e4912b805a5eb088dd536adb9/pkgs/applications/video/kodi/default.nix#L73")). The easiest way then would be replace hash with a fake one and rebuild. Nix build will fail and error message will contain desired hash.
::: {.warning}
::: warning
This method has security problems. Check below for details.
:::
@@ -535,7 +523,7 @@ If you do need to do create this sort of patch file, one way to do so is with gi
4. Use git to create a diff, and pipe the output to a patch file:
```ShellSession
$ git diff -a > nixpkgs/pkgs/the/package/0001-changes.patch
If a patch is available online but does not cleanly apply, it can be modified in some fixed ways by using additional optional arguments for `fetchpatch`:
@@ -552,34 +540,9 @@ Note that because the checksum is computed after applying these effects, using o
Tests are important to ensure quality and make reviews and automatic updates easy.
The following types of tests exists:
Nix package tests are a lightweight alternative to [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests). They can be used to create simple integration tests for packages while the module tests are used to test services or programs with a graphical user interface on a NixOS VM. Unittests that are included in the source code of a package should be executed in the `checkPhase`.
* [NixOS **module tests**](https://nixos.org/manual/nixos/stable/#sec-nixos-tests), which spawn one or more NixOS VMs. They exercise both NixOS modules and the packaged programs used within them. For example, a NixOS module test can start a web server VM running the `nginx` module, and a client VM running `curl` or a graphical `firefox`, and test that they can talk to each other and display the correct content.
* Nix **package tests** are a lightweight alternative to NixOS module tests. They should be used to create simple integration tests for packages, but cannot test NixOS services, and some programs with graphical user interfaces may also be difficult to test with them.
* The **`checkPhase` of a package**, which should execute the unit tests that are included in the source code of a package.
Here in the nixpkgs manual we describe mostly _package tests_; for _module tests_ head over to the corresponding [section in the NixOS manual](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
- [Multiple tests for nanopb](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/nanopb/default.nix)
### Linking NixOS module tests to a package {#ssec-nixos-tests-linking}
Like [package tests](#ssec-package-tests-writing) as shown above, [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests) can also be linked to a package, so that the tests can be easily run when changing the related package.
For example, assuming we're packaging `nginx`, we can link its module test via `passthru.tests`:
# Contributing to this documentation {#chap-contributing}
The sources of the Nixpkgs manual are in the [doc](https://github.com/NixOS/nixpkgs/tree/master/doc) subdirectory of the Nixpkgs repository. The manual is still partially written in DocBook but it is progressively being converted to [Markdown](#sec-contributing-markup).
The DocBook sources of the Nixpkgs manual are in the [doc](https://github.com/NixOS/nixpkgs/tree/master/doc) subdirectory of the Nixpkgs repository.
You can quickly check your edits with `make`:
@@ -22,85 +22,3 @@ $ nix-shell
```
If the build succeeds, the manual will be in `./result/share/doc/nixpkgs/manual.html`.
## Syntax {#sec-contributing-markup}
As per [RFC 0072](https://github.com/NixOS/rfcs/pull/72), all new documentation content should be written in [CommonMark](https://commonmark.org/) Markdown dialect.
Additionally, the following syntax extensions are currently used:
- []{#ssec-contributing-markup-anchors}
Explicitly defined **anchors** on headings, to allow linking to sections. These should be always used, to ensure the anchors can be linked even when the heading text changes, and to prevent conflicts between [automatically assigned identifiers](https://github.com/jgm/commonmark-hs/blob/master/commonmark-extensions/test/auto_identifiers.md).
It uses the widely compatible [header attributes](https://github.com/jgm/commonmark-hs/blob/master/commonmark-extensions/test/attributes.md) syntax:
```markdown
## Syntax {#sec-contributing-markup}
```
- []{#ssec-contributing-markup-anchors-inline}
**Inline anchors**, which allow linking arbitrary place in the text (e.g. individual list items, sentences…).
They are defined using a hybrid of the link syntax with the attributes syntax known from headings, called [bracketed spans](https://github.com/jgm/commonmark-hs/blob/master/commonmark-extensions/test/bracketed_spans.md):
```markdown
- []{#ssec-gnome-hooks-glib} `glib` setup hook will populate `GSETTINGS_SCHEMAS_PATH` and then `wrapGAppsHook` will prepend it to `XDG_DATA_DIRS`.
```
- []{#ssec-contributing-markup-automatic-links}
If you **omit a link text** for a link pointing to a section, the text will be substituted automatically. For example, `[](#chap-contributing)` will result in [](#chap-contributing).
This syntax is taken from [MyST](https://myst-parser.readthedocs.io/en/latest/using/syntax.html#targets-and-cross-referencing).
- []{#ssec-contributing-markup-inline-roles}
If you want to link to a man page, you can use `` {manpage}`nix.conf(5)` ``, which will turn into {manpage}`nix.conf(5)`.
The references will turn into links when a mapping exists in {file}`doc/build-aux/pandoc-filters/unix-man-urls.lua`.
This syntax is taken from [MyST](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#roles-an-in-line-extension-point). Though, the feature originates from [reStructuredText](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-manpage) with slightly different syntax.
- []{#ssec-contributing-markup-admonitions}
**Admonitions**, set off from the text to bring attention to something.
It uses pandoc’s [fenced `div`s syntax](https://github.com/jgm/commonmark-hs/blob/master/commonmark-extensions/test/fenced_divs.md):
[**Definition lists**](https://github.com/jgm/commonmark-hs/blob/master/commonmark-extensions/test/definition_lists.md), for defining a group of terms:
```markdown
pear
: green or yellow bulbous fruit
watermelon
: green fruit with red flesh
```
which renders as
> pear
> : green or yellow bulbous fruit
>
> watermelon
> : green fruit with red flesh
For contributing to the legacy parts, please see [DocBook: The Definitive Guide](https://tdg.docbook.org/) or the [DocBook rocks! primer](https://web.archive.org/web/20200816233747/https://docbook.rocks/).
2. Find a good place in the Nixpkgs tree to add the Nix expression for your package. For instance, a library package typically goes into `pkgs/development/libraries/pkgname`, while a web browser goes into `pkgs/applications/networking/browsers/pkgname`. See [](#sec-organisation) for some hints on the tree organisation. Create a directory for your package, e.g.
2. Find a good place in the Nixpkgs tree to add the Nix expression for your package. For instance, a library package typically goes into `pkgs/development/libraries/pkgname`, while a web browser goes into `pkgs/applications/networking/browsers/pkgname`. See <xref linkend="sec-organisation" /> for some hints on the tree organisation. Create a directory for your package, e.g.
The following section is a draft, and the policy for reviewing is still being discussed in issues such as [#11166](https://github.com/NixOS/nixpkgs/issues/11166) and [#20836](https://github.com/NixOS/nixpkgs/issues/20836).
:::
@@ -35,18 +35,15 @@ Reviewing process:
- Building the package locally.
- pull requests are often targeted to the master or staging branch, and building the pull request locally when it is submitted can trigger many source builds.
- It is possible to rebase the changes on nixos-unstable or nixpkgs-unstable for easier review by running the following commands from a nixpkgs clone.
```ShellSession
$ git fetch origin nixos-unstable
$ git fetch origin pull/PRNUMBER/head
$ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD
```
- The first command fetches the nixos-unstable branch.
- The second command fetches the pull request changes, `PRNUMBER` is the number at the end of the pull request title and `BASEBRANCH` the base branch of the pull request.
- The third command rebases the pull request changes to the nixos-unstable branch.
- The [nixpkgs-review](https://github.com/Mic92/nixpkgs-review) tool can be used to review a pull request content in a single command. `PRNUMBER` should be replaced by the number at the end of the pull request title. You can also provide the full github pull request url.
@@ -71,7 +71,6 @@ Security fixes are submitted in the same way as other changes and thus the same
- If a new version fixing the vulnerability has been released, update the package;
- If the security fix comes in the form of a patch and a CVE is available, then add the patch to the Nixpkgs tree, and apply it to the package.
The name of the patch should be the CVE identifier, so e.g. `CVE-2019-13636.patch`; If a patch is fetched the name needs to be set as well, e.g.:
```nix
(fetchpatch {
name = "CVE-2019-11068.patch";
@@ -90,7 +89,7 @@ There is currently no policy when to remove a package.
Before removing a package, one should try to find a new maintainer or fix smaller issues first.
### Steps to remove a package from Nixpkgs {#steps-to-remove-a-package-from-nixpkgs}
### Steps to remove a package from Nixpkgs
We use jbidwatcher as an example for a discontinued project here.
@@ -101,7 +100,6 @@ We use jbidwatcher as an example for a discontinued project here.
1. Add an alias for the package name in `pkgs/top-level/aliases.nix` (There is also `pkgs/misc/vim-plugins/aliases.nix`. Package sets typically do not have aliases, so we can't add them there.)
For example in this case:
```
jbidwatcher = throw "jbidwatcher was discontinued in march 2021"; # added 2021-03-15
```
@@ -240,7 +238,7 @@ The `staging` branch is a development branch where mass-rebuilds go. It should o
The `staging-next` branch is for stabilizing mass-rebuilds submitted to the `staging` branch prior to merging them into `master`. Mass-rebuilds must go via the `staging` branch. It must only see non-breaking commits that are fixing issues blocking it from being merged into the `master ` branch.
The `staging-next` branch is for stabilizing mass-rebuilds submitted to the `staging` branch prior to merging them into `master`. Mass-rebuilds should go via the `staging` branch. It should only see non-breaking commits that are fixing issues blocking it from being merged into the `master ` branch.
If the branch is already in a broken state, please refrain from adding extra new breakages. Stabilize it for a few days and then merge into master.
@@ -250,8 +248,6 @@ For cherry-picking a commit to a stable release branch (“backporting”), use
Add a reason for the backport by using `git cherry-pick -xe <original commit>` instead when it is not obvious from the original commit message. It is not needed when it's a minor version update that includes security and bug fixes but don't add new features or when the commit fixes an otherwise broken package.
For backporting Pull Requests to stable branches, assign label `backport <branch>` to the original Pull Requests and automation should take care of the rest once the Pull Requests is merged.
Here is an example of a cherry-picked commit message with good reason description:
```
@@ -269,14 +265,3 @@ Other examples of reasons are:
- Previously the build would fail due to, e.g., `getaddrinfo` not being defined
- The previous download links were all broken
- Crash when starting on some X11 systems
#### Acceptable backport criteria
The stable branch does have some changes which cannot be backported. Most notable are breaking changes. The desire is to have stable users be uninterrupted when updating packages.
However, many changes are able to be backported, including:
- New Packages / Modules
- Security / Patch updates
- Version updates which include new functionality (but no breaking changes)
- Services which require a client to be up-to-date regardless. (E.g. `spotify`, `steam`, or `discord`)
Nix is a unityped, dynamic language, this means every value can potentially appear anywhere. Since it is also non-strict, evaluation order and what ultimately is evaluated might surprise you. Therefore it is important to be able to debug nix expressions.
In the `lib/debug.nix` file you will find a number of functions that help (pretty-)printing values while evaluation is running. You can even specify how deep these values should be printed recursively, and transform them on the fly. Please consult the docstrings in `lib/debug.nix` for usage information.
Nix is a unityped, dynamic language, this means every value can potentially appear anywhere. Since it is also non-strict, evaluation order and what ultimately is evaluated might surprise you. Therefore it is important to be able to debug nix expressions.
</para>
<para>
In the <literal>lib/debug.nix</literal> file you will find a number of functions that help (pretty-)printing values while evaluation is runnnig. You can even specify how deep these values should be printed recursively, and transform them on the fly. Please consult the docstrings in <literal>lib/debug.nix</literal> for usage information.
Generators are functions that create file formats from nix data structures, e.g. for configuration files. There are generators available for: `INI`, `JSON` and `YAML`
All generators follow a similar call interface: `generatorName configFunctions data`, where `configFunctions` is an attrset of user-defined functions that format nested parts of the content. They each have common defaults, so often they do not need to be set manually. An example is `mkSectionName ? (name: libStr.escape [ "[" "]" ] name)` from the `INI` generator. It receives the name of a section and sanitizes it. The default `mkSectionName` escapes `[` and `]` with a backslash.
Generators can be fine-tuned to produce exactly the file format required by your application/service. One example is an INI-file format which uses `: ` as separator, the strings `"yes"`/`"no"` as boolean values and requires all string values to be quoted:
```nix
withlib;
let
customToINI=generators.toINI{
# specifies how to format a key/value pair
mkKeyValue=generators.mkKeyValueDefault{
# specifies the generated string for a subset of nix values
mkValueString=v:
ifv==truethen''"yes"''
elseifv==falsethen''"no"''
elseifisStringvthen''"${v}"''
# and delegats all other values to the default generator
elsegenerators.mkValueStringDefault{}v;
}":";
};
# the INI file can now be given as plain old nix values
incustomToINI{
main={
pushinfo=true;
autopush=false;
host="localhost";
port=42;
};
mergetool={
merge="diff3";
};
}
```
This will produce the following INI file as nix string:
```INI
[main]
autopush:"no"
host:"localhost"
port:42
pushinfo:"yes"
str\:ange:"very::strange"
[mergetool]
merge:"diff3"
```
::: {.note}
Nix store paths can be converted to strings by enclosing a derivation attribute like so: `"${drv}"`.
:::
Detailed documentation for each generator can be found in `lib/generators.nix`.
Generators are functions that create file formats from nix data structures, e.g. for configuration files. There are generators available for: <literal>INI</literal>, <literal>JSON</literal> and <literal>YAML</literal>
</para>
<para>
All generators follow a similar call interface: <code>generatorName configFunctions data</code>, where <literal>configFunctions</literal> is an attrset of user-defined functions that format nested parts of the content. They each have common defaults, so often they do not need to be set manually. An example is <code>mkSectionName ? (name: libStr.escape [ "[" "]" ] name)</code> from the <literal>INI</literal> generator. It receives the name of a section and sanitizes it. The default <literal>mkSectionName</literal> escapes <literal>[</literal> and <literal>]</literal> with a backslash.
</para>
<para>
Generators can be fine-tuned to produce exactly the file format required by your application/service. One example is an INI-file format which uses <literal>: </literal> as separator, the strings <literal>"yes"</literal>/<literal>"no"</literal> as boolean values and requires all string values to be quoted:
</para>
<programlisting>
with lib;
let
customToINI = generators.toINI {
# specifies how to format a key/value pair
mkKeyValue = generators.mkKeyValueDefault {
# specifies the generated string for a subset of nix values
mkValueString = v:
if v == true then ''"yes"''
else if v == false then ''"no"''
else if isString v then ''"${v}"''
# and delegats all other values to the default generator
else generators.mkValueStringDefault {} v;
} ":";
};
# the INI file can now be given as plain old nix values
in customToINI {
main = {
pushinfo = true;
autopush = false;
host = "localhost";
port = 42;
};
mergetool = {
merge = "diff3";
};
}
</programlisting>
<para>
This will produce the following INI file as nix string:
</para>
<programlisting>
[main]
autopush:"no"
host:"localhost"
port:42
pushinfo:"yes"
str\:ange:"very::strange"
[mergetool]
merge:"diff3"
</programlisting>
<note>
<para>
Nix store paths can be converted to strings by enclosing a derivation attribute like so: <code>"${drv}"</code>.
</para>
</note>
<para>
Detailed documentation for each generator can be found in <literal>lib/generators.nix</literal>.
`pkgs.nix-gitignore` is a function that acts similarly to `builtins.filterSource` but also allows filtering with the help of the gitignore format.
## Usage {#sec-pkgs-nix-gitignore-usage}
`pkgs.nix-gitignore` exports a number of functions, but you\'ll most likely need either `gitignoreSource` or `gitignoreSourcePure`. As their first argument, they both accept either 1. a file with gitignore lines or 2. a string with gitignore lines, or 3. a list of either of the two. They will be concatenated into a single big string.
Those filter functions accept the same arguments the `builtins.filterSource` function would pass to its filters, thus `fn: gitignoreFilterSourcePure fn ""` should be extensionally equivalent to `filterSource`. The file is blacklisted if it\'s blacklisted by either your filter or the gitignoreFilter.
If you want to make your own filter from scratch, you may use
<function>pkgs.nix-gitignore</function> is a function that acts similarly to <literal>builtins.filterSource</literal> but also allows filtering with the help of the gitignore format.
</para>
<sectionxml:id="sec-pkgs-nix-gitignore-usage">
<title>Usage</title>
<para>
<literal>pkgs.nix-gitignore</literal> exports a number of functions, but you'll most likely need either <literal>gitignoreSource</literal> or <literal>gitignoreSourcePure</literal>. As their first argument, they both accept either 1. a file with gitignore lines or 2. a string with gitignore lines, or 3. a list of either of the two. They will be concatenated into a single big string.
Those filter functions accept the same arguments the <literal>builtins.filterSource</literal> function would pass to its filters, thus <literal>fn: gitignoreFilterSourcePure fn ""</literal> should be extensionally equivalent to <literal>filterSource</literal>. The file is blacklisted iff it's blacklisted by either your filter or the gitignoreFilter.
</para>
<para>
If you want to make your own filter from scratch, you may use
`prefer-remote-fetch` is an overlay that download sources on remote builder. This is useful when the evaluating machine has a slow upload while the builder can fetch faster directly from the source. To use it, put the following snippet as a new overlay:
```nix
self:super:
(super.prefer-remote-fetchselfsuper)
```
A full configuration example for that sets the overlay up for your own account, could look like this
<function>prefer-remote-fetch</function> is an overlay that download sources on remote builder. This is useful when the evaluating machine has a slow upload while the builder can fetch faster directly from the source. To use it, put the following snippet as a new overlay:
<programlisting>
self: super:
(super.prefer-remote-fetch self super)
</programlisting>
A full configuration example for that sets the overlay up for your own account, could look like this
Agda is available as the [agda](https://search.nixos.org/packages?channel=unstable&show=agda&from=0&size=30&sort=relevance&query=agda)
package.
@@ -43,7 +43,6 @@ agda.withPackages (p: [
```
You can also reference a GitHub repository
```nix
agda.withPackages(p:[
(p.standard-library.overrideAttrs(oldAttrs:{
@@ -60,7 +59,6 @@ agda.withPackages (p: [
If you want to use a library not added to Nixpkgs, you can add a
dependency to a local library by calling `agdaPackages.mkDerivation`.
```nix
agda.withPackages(p:[
(p.mkDerivation{
@@ -94,21 +92,20 @@ See [Building Agda Packages](#building-agda-packages) for more information on `m
Agda will not by default use these libraries. To tell Agda to use a library we have some options:
* Call `agda` with the library flag:
```ShellSession
$ agda -l standard-library -i . MyFile.agda
```
```ShellSession
$ agda -l standard-library -i . MyFile.agda
```
* Write a `my-library.agda-lib` file for the project you are working on which may look like:
```
name: my-library
include: .
depend: standard-library
```
```
name: my-library
include: .
depend: standard-library
```
* Create the file `~/.agda/defaults` and add any libraries you want to use by default.
More information can be found in the [official Agda documentation on library management](https://agda.readthedocs.io/en/v2.6.1/tools/package-system.html).
## Compiling Agda {#compiling-agda}
## Compiling Agda
Agda modules can be compiled using the GHC backend with the `--compile` flag. A version of `ghc` with `ieee754` is made available to the Agda program via the `--with-compiler` flag.
This can be overridden by a different version of `ghc` as follows:
@@ -119,8 +116,7 @@ agda.withPackages {
}
```
## Writing Agda packages {#writing-agda-packages}
## Writing Agda packages
To write a nix derivation for an Agda library, first check that the library has a `*.agda-lib` file.
A derivation can then be written using `agdaPackages.mkDerivation`. This has similar arguments to `stdenv.mkDerivation` with the following additions:
@@ -144,37 +140,19 @@ agdaPackages.mkDerivation {
}
```
### Building Agda packages {#building-agda-packages}
### Building Agda packages
The default build phase for `agdaPackages.mkDerivation` simply runs `agda` on the `Everything.agda` file.
If something else is needed to build the package (e.g. `make`) then the `buildPhase` should be overridden.
Additionally, a `preBuild` or `configurePhase` can be used if there are steps that need to be done prior to checking the `Everything.agda` file.
`agda` and the Agda libraries contained in `buildInputs` are made available during the build phase.
The default install phase copies Agda source files, Agda interface files (`*.agdai`) and `*.agda-lib` files to the output directory.
This can be overridden.
By default, Agda sources are files ending on `.agda`, or literate Agda files ending on `.lagda`, `.lagda.tex`, `.lagda.org`, `.lagda.md`, `.lagda.rst`. The list of recognised Agda source extensions can be extended by setting the `extraExtensions` config variable.
## Maintaining the Agda package set on Nixpkgs {#maintaining-the-agda-package-set-on-nixpkgs}
We are aiming at providing all common Agda libraries as packages on `nixpkgs`,
and keeping them up to date.
Contributions and maintenance help is always appreciated,
but the maintenance effort is typically low since the Agda ecosystem is quite small.
The `nixpkgs` Agda package set tries to take up a role similar to that of [Stackage](https://www.stackage.org/) in the Haskell world.
It is a curated set of libraries that:
1. Always work together.
2. Are as up-to-date as possible.
While the Haskell ecosystem is huge, and Stackage is highly automatised,
the Agda package set is small and can (still) be maintained by hand.
### Adding Agda packages to Nixpkgs {#adding-agda-packages-to-nixpkgs}
## Adding Agda packages to Nixpkgs
To add an Agda package to `nixpkgs`, the derivation should be written to `pkgs/development/libraries/agda/${library-name}/` and an entry should be added to `pkgs/top-level/agda-packages.nix`. Here it is called in a scope with access to all other Agda libraries, so the top line of the `default.nix` can look like:
@@ -204,53 +182,6 @@ mkDerivation {
'';
}
```
This library has a file called `.agda-lib`, and so we give an empty string to `libraryFile` as nothing precedes `.agda-lib` in the filename. This file contains `name: IAL-1.3`, and so we let `libraryName = "IAL-1.3"`. This library does not use an `Everything.agda` file and instead has a Makefile, so there is no need to set `everythingFile` and we set a custom `buildPhase`.
When writing an Agda package it is essential to make sure that no `.agda-lib` file gets added to the store as a single file (for example by using `writeText`). This causes Agda to think that the nix store is a Agda library and it will attempt to write to it whenever it typechecks something. See [https://github.com/agda/agda/issues/4613](https://github.com/agda/agda/issues/4613).
In the pull request adding this library,
you can test whether it builds correctly by writing in a comment:
```
@ofborg build agdaPackages.iowa-stdlib
```
### Maintaining Agda packages
As mentioned before, the aim is to have a compatible, and up-to-date package set.
These two conditions sometimes exclude each other:
For example, if we update `agdaPackages.standard-library` because there was an upstream release,
this will typically break many reverse dependencies,
i.e. downstream Agda libraries that depend on the standard library.
In `nixpkgs` we are typically among the first to notice this,
since we have build tests in place to check this.
In a pull request updating e.g. the standard library, you should write the following comment:
In this document and related Nix expressions, we use the term, _BEAM_, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction.
## Available versions and deprecations schedule {#available-versions-and-deprecations-schedule}
### Elixir {#elixir}
nixpkgs follows the [official elixir deprecation schedule](https://hexdocs.pm/elixir/compatibility-and-deprecations.html) and keeps the last 5 released versions of Elixir available.
## Structure {#beam-structure}
All BEAM-related expressions are available via the top-level `beam` attribute, which includes:
@@ -68,107 +62,27 @@ Erlang.mk functions similarly to Rebar3, except we use `buildErlangMk` instead o
`mixRelease` is used to make a release in the mix sense. Dependencies will need to be fetched with `fetchMixDeps` and passed to it.
#### mixRelease - Elixir Phoenix example {#mix-release-elixir-phoenix-example}
#### mixRelease - Elixir Phoenix example
there are 3 steps, frontend dependencies (javascript), backend dependencies (elixir) and the final derivation that puts both of those together
for phoenix projects, inside of nixpkgs you can either use yarn2nix (mkYarnModule) or node2nix. An example with yarn2nix can be found [here](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/web-apps/plausible/default.nix#L39). An example with node2nix will follow. To package something outside of nixpkgs, you have alternatives like [npmlock2nix](https://github.com/nix-community/npmlock2nix) or [nix-npm-buildpackage](https://github.com/serokell/nix-npm-buildpackage)
There are 2 ways to package backend dependencies. With mix2nix and with a fixed-output-derivation (FOD).
###### mix2nix {#mix2nix}
mix2nix is a cli tool available in nixpkgs. it will generate a nix expression from a mix.lock file. It is quite standard in the 2nix tool series.
Note that currently mix2nix can't handle git dependencies inside the mix.lock file. If you have git dependencies, you can either add them manually (see [example](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/pleroma/default.nix#L20)) or use the FOD method.
The advantage of using mix2nix is that nix will know your whole dependency graph. On a dependency update, this won't trigger a full rebuild and download of all the dependencies, where FOD will do so.
practical steps:
- run `mix2nix > mix_deps.nix` in the upstream repo.
- pass `mixNixDeps = with pkgs; import ./mix_deps.nix { inherit lib beamPackages; };` as an argument to mixRelease.
If there are git depencencies.
- You'll need to fix the version artificially in mix.exs and regenerate the mix.lock with fixed version (on upstream). This will enable you to run `mix2nix > mix_deps.nix`.
- From the mix_deps.nix file, remove the dependencies that had git versions and pass them as an override to the import function.
```nix
mixNixDeps=import./mix.nix{
inheritbeamPackageslib;
overrides=(final:prev:{
# mix2nix does not support git dependencies yet,
# so we need to add them manually
prometheus_ex=beamPackages.buildMixrec{
name="prometheus_ex";
version="3.0.5";
# Change the argument src with the git src that you actually need
# you can re-use the same beamDeps argument as generated
beamDeps=withfinal;[prometheus];
};
});
};
```
You will need to run the build process once to fix the sha256 to correspond to your new git src.
###### FOD {#fixed-output-derivation}
A fixed output derivation will download mix dependencies from the internet. To ensure reproducibility, a hash will be supplied. Note that mix is relatively reproducible. An FOD generating a different hash on each run hasn't been observed (as opposed to npm where the chances are relatively high). See [elixir_ls](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/beam-modules/elixir_ls.nix) for a usage example of FOD.
Practical steps
- start with the following argument to mixRelease
```nix
mixFodDeps=fetchMixDeps{
pname="mix-deps-${pname}";
inheritsrcversion;
sha256=lib.fakeSha256;
};
```
The first build will complain about the sha256 value, you can replace with the suggested value after that.
Note that if after you've replaced the value, nix suggests another sha256, then mix is not fetching the dependencies reproducibly. An FOD will not work in that case and you will have to use mix2nix.
##### mixRelease - example {#mix-release-example}
Here is how your `default.nix` file would look for a phoenix project.
Here is how your `default.nix` file would look.
```nix
withimport<nixpkgs>{};
let
# beam.interpreters.erlangR23 is available if you need a particular version
@@ -202,7 +142,7 @@ Setup will require the following steps:
- you can now `nix-build .`
- To run the release, set the `RELEASE_TMP` environment variable to a directory that your program has write access to. It will be used to store the BEAM settings.
#### Example of creating a service for an Elixir - Phoenix project {#example-of-creating-a-service-for-an-elixir---phoenix-project}
#### Example of creating a service for an Elixir - Phoenix project
In order to create a service with your release, you could add a `service.nix`
in your project with the following
@@ -219,8 +159,6 @@ in
systemd.services.${release_name}={
wantedBy=["multi-user.target"];
after=["network.target""postgresql.service"];
# note that if you are connecting to a postgres instance on a different host
# postgresql.service should not be included in the requires.
# in case you have migration scripts or you want to use a remote shell
environment.systemPackages=[release];
}
```
@@ -269,18 +206,23 @@ in
Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of Erlang and any other interpreters, and then use your normal build tools. As an example with Elixir:
The Coq derivation is overridable through the `coq.override overrides`, where overrides is an attribute set which contains the arguments to override. We recommend overriding either of the following
@@ -8,7 +8,7 @@ The Coq derivation is overridable through the `coq.override overrides`, where ov
*`customOCamlPackage` (optional, defaults to `null`, which lets Coq choose a version automatically), which can be set to any of the ocaml packages attribute of `ocaml-ng` (such as `ocaml-ng.ocamlPackages_4_10` which is the default for Coq 8.11 for example).
*`coq-version` (optional, defaults to the short version e.g. "8.10"), is a version number of the form "x.y" that indicates which Coq's version build behavior to mimic when using a source which is not a release. E.g. `coq.override { version = "d370a9d1328a4e1cdb9d02ee032f605a9d94ec7a"; coq-version = "8.10"; }`.
The recommended way of defining a derivation for a Coq library, is to use the `coqPackages.mkCoqDerivation` function, which is essentially a specialization of `mkDerivation` taking into account most of the specifics of Coq libraries. The following attributes are supported:
@@ -28,12 +28,11 @@ The recommended way of defining a derivation for a Coq library, is to use the `c
*`domain` (optional, defaults to `"github.com"`), domains including the strings `"github"` or `"gitlab"` in their names are automatically supported, otherwise, one must change the `fetcher` argument to support them (cf `pkgs/development/coq-modules/heq/default.nix` for an example),
*`releaseRev` (optional, defaults to `(v: v)`), provides a default mapping from release names to revision hashes/branch names/tags,
*`displayVersion` (optional), provides a way to alter the computation of `name` from `pname`, by explaining how to display version numbers,
*`namePrefix` (optional, defaults to `[ "coq" ]`), provides a way to alter the computation of `name` from `pname`, by explaining which dependencies must occur in `name`,
*`namePrefix` (optional), provides a way to alter the computation of `name` from `pname`, by explaining which dependencies must occur in `name`,
*`extraBuildInputs` (optional), by default `buildInputs` just contains `coq`, this allows to add more build inputs,
*`mlPlugin` (optional, defaults to `false`). Some extensions (plugins) might require OCaml and sometimes other OCaml packages. Standard dependencies can be added by setting the current option to `true`. For a finer grain control, the `coq.ocamlPackages` attribute can be used in `extraBuildInputs` to depend on the same package set Coq was built against.
*`useDune2ifVersion` (optional, default to `(x: false)` uses Dune2 to build the package if the provided predicate evaluates to true on the version, e.g. `useDune2if = versions.isGe "1.1"` will use dune if the version of the package is greater or equal to `"1.1"`,
*`useDune2` (optional, defaults to `false`) uses Dune2 to build the package if set to true, the presence of this attribute overrides the behavior of the previous one.
*`opam-name` (optional, defaults to concatenating with a dash separator the components of `namePrefix` and `pname`), name of the Dune package to build.
*`enableParallelBuilding` (optional, defaults to `true`), since it is activated by default, we provide a way to disable it.
*`extraInstallFlags` (optional), allows to extend `installFlags` which initializes the variable `COQMF_COQLIB` so as to install in the proper subdirectory. Indeed Coq libraries should be installed in `$(out)/lib/coq/${coq.coq-version}/user-contrib/`. Such directories are automatically added to the `$COQPATH` environment variable by the hook defined in the Coq derivation.
*`setCOQBIN` (optional, defaults to `true`), by default, the environment variable `$COQBIN` is set to the current Coq's binary, but one can disable this behavior by setting it to `false`,
## Building a Crystal package {#building-a-crystal-package}
## Building a Crystal package
This section uses [Mint](https://github.com/mint-lang/mint) as an example for how to build a Crystal package.
If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory. Executable projects should usually commit the `shard.lock` file, but sometimes that's not the case, which means you need to generate it yourself. With an existing `shard.lock` file,`crystal2nix`can be run.
If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory, then run`crystal2nix`in it
## Local Development Workflow {#local-development-workflow}
## Local Development Workflow
For local development, it's recommended to use nix-shell to create a dotnet environment:
@@ -16,7 +16,7 @@ mkShell {
}
```
### Using many sdks in a workflow {#using-many-sdks-in-a-workflow}
### Using many sdks in a workflow
It's very likely that more than one sdk will be needed on a given project. Dotnet provides several different frameworks (E.g dotnetcore, aspnetcore, etc.) as well as many versions for a given framework. Normally, dotnet is able to fetch a framework and install it relative to the executable. However, this would mean writing to the nix store in nixpkgs, which is read-only. To support the many-sdk use case, one can compose an environment using `dotnetCorePackages.combinePackages`:
@@ -28,7 +28,8 @@ mkShell {
packages=[
(withdotnetCorePackages;combinePackages[
sdk_3_1
sdk_5_0
sdk_3_0
sdk_2_1
])
];
}
@@ -36,7 +37,7 @@ mkShell {
This will produce a dotnet installation that has the dotnet 3.1, 3.0, and 2.1 sdk. The first sdk listed will have it's cli utility present in the resulting environment. Example info output:
## dotnet-sdk vs dotnetCorePackages.sdk {#dotnet-sdk-vs-dotnetcorepackages.sdk}
## dotnet-sdk vs dotnetCorePackages.sdk
The `dotnetCorePackages.sdk_X_Y` is preferred over the old dotnet-sdk as both major and minor version are very important for a dotnet environment. If a given minor version isn't present (or was changed), then this will likely break your ability to build a project.
## dotnetCorePackages.sdk vs dotnetCorePackages.runtime vs dotnetCorePackages.aspnetcore {#dotnetcorepackages.sdk-vs-dotnetcorepackages.runtime-vs-dotnetcorepackages.aspnetcore}
## dotnetCorePackages.sdk vs dotnetCorePackages.net vs dotnetCorePackages.netcore vsdotnetCorePackages.aspnetcore
The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `runtime` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications.
The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `net`, `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. For runtime versions >= .NET 5 `net` is used while `netcore` is used for older .NET Core runtime version.
## Packaging a Dotnet Application {#packaging-a-dotnet-application}
## Packaging a Dotnet Application
To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions:
Ideally, we would like to build against the sdk, then only have the dotnet runtime available in the runtime closure.
*`projectFile` has to be used for specifying the dotnet project file relative to the source root. These usually have `.sln` or `.csproj` file extensions. This can be an array of multiple projects as well.
*`nugetDeps` has to be used to specify the NuGet dependency file. Unfortunately, these cannot be deterministically fetched without a lockfile. This file should be generated using `nuget-to-nix` tool, which is available in nixpkgs.
*`executables` is used to specify which executables get wrapped to `$out/bin`, relative to `$out/lib/$pname`. If this is unset, all executables generated will get installed. If you do not want to install any, set this to `[]`.
*`runtimeDeps` is used to wrap libraries into `LD_LIBRARY_PATH`. This is how dotnet usually handles runtime dependencies.
*`buildType` is used to change the type of build. Possible values are `Release`, `Debug`, etc. By default, this is set to `Release`.
*`dotnet-sdk` is useful in cases where you need to change what dotnet SDK is being used.
*`dotnet-runtime` is useful in cases where you need to change what dotnet runtime is being used. This can be either a regular dotnet runtime, or an aspnetcore.
*`dotnet-test-sdk` is useful in cases where unit tests expect a different dotnet SDK. By default, this is set to the `dotnet-sdk` attribute.
*`testProjectFile` is useful in cases where the regular project file does not contain the unit tests. By default, this is set to the `projectFile` attribute.
*`disabledTests` is used to disable running specific unit tests. This gets passed as: `dotnet test --filter "FullyQualifiedName!={}"`, to ensure compatibility with all unit test frameworks.
*`dotnetRestoreFlags` can be used to pass flags to `dotnet restore`.
*`dotnetBuildFlags` can be used to pass flags to `dotnet build`.
*`dotnetTestFlags` can be used to pass flags to `dotnet test`.
*`dotnetInstallFlags` can be used to pass flags to `dotnet install`.
*`dotnetFlags` can be used to pass flags to all of the above phases.
Here is an example `default.nix`, using some of the previously discussed arguments:
This example uses `zlib` from nixpkgs but instead of compiling **C** to **ELF** it compiles **C** to **JS** since we were using `pkgs.zlib.override` and changed stdenv to `pkgs.emscriptenStdenv`. A few adaptions and hacks were set in place to make it working. One advantage is that when `pkgs.zlib` is updated, it will automatically update this package as well. However, this can also be the downside...
Use `nix-shell -I nixpkgs=/some/dir/nixpkgs -A emscriptenPackages.libz` and from there you can go trough the individual steps. This makes it easy to build a good `unit test` or list the files of the project.
@@ -175,7 +177,7 @@ Use `nix-shell -I nixpkgs=/some/dir/nixpkgs -A emscriptenPackages.libz` and from
6.`buildPhase`
7. ... happy hacking...
## Summary {#summary}
## Summary
Using this toolchain makes it easy to leverage `nix` from NixOS, MacOSX or even Windows (WSL+ubuntu+nix). This toolchain is reproducible, behaves like the rest of the packages from nixpkgs and contains a set of well working examples to learn and adapt from.
@@ -102,7 +102,7 @@ For convenience, it also adds `dconf.lib` for a GIO module implementing a GSetti
- []{#ssec-gnome-hooks-gobject-introspection} `gobject-introspection` setup hook populates `GI_TYPELIB_PATH` variable with `lib/girepository-1.0` directories of dependencies, which is then added to wrapper by `wrapGAppsHook`. It also adds `share` directories of dependencies to `XDG_DATA_DIRS`, which is intended to promote GIR files but it also [pollutes the closures](https://github.com/NixOS/nixpkgs/issues/32790) of packages using `wrapGAppsHook`.
::: {.warning}
::: warning
The setup hook [currently](https://github.com/NixOS/nixpkgs/issues/56943) does not work in expressions with `strictDeps` enabled, like Python packages. In those cases, you will need to disable it with `strictDeps = false;`.
@@ -13,7 +13,6 @@ In the following is an example expression using `buildGoModule`, the following a
-`vendorSha256`: is the hash of the output of the intermediate fetcher derivation. `vendorSha256` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorSha256 = null;`
-`runVend`: runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build.
-`proxyVendor`: Fetches (go mod download) and proxies the vendor directory. This is useful if any dependency has case-insensitive conflicts which will produce platform dependant `vendorSha256` checksums.
```nix
pet=buildGoModulerec{
@@ -45,7 +44,7 @@ pet = buildGoModule rec {
The function `buildGoPackage` builds legacy Go programs, not supporting Go modules.
### Example for `buildGoPackage` {#example-for-buildgopackage}
### Example for `buildGoPackage`
In the following is an example expression using buildGoPackage, the following arguments are of special significance to the function:
@@ -113,31 +112,23 @@ done
Both `buildGoModule` and `buildGoPackage` can be tweaked to behave slightly differently, if the following attributes are used:
### `ldflags` {#var-go-ldflags}
### `buildFlagsArray` and `buildFlags`: {#ex-goBuildFlags-noarray}
Arguments to pass to the Go linker tool via the `-ldflags` argument of `go build`. The most common use case for this argument is to make the resulting executable aware of its own version. For example:
These attributes set build flags supported by `go build`. We recommend using `buildFlagsArray`. The most common use case of these attributes is to make the resulting executable aware of its own version. For example:
```nix
ldflags=[
"-s""-w"
"-Xmain.Version=${version}"
"-Xmain.Commit=${version}"
];
```
### `tags` {#var-go-tags}
Arguments to pass to the Go via the `-tags` argument of `go build`. For example:
@@ -146,4 +137,4 @@ Removes the pre-existing vendor directory. This should only be used if the depen
### `subPackages` {#var-go-subPackages}
Limits the builder from building child packages that have not been listed. If `subPackages` is not specified, all child packages will be built.
Limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built.
### Installation without packages {#installation-without-packages}
You can install `hy` via nix-env or by adding it to `configuration.nix` by reffering to it as a `hy` attribute. This kind of installation adds `hy` to your environment and it succesfully works with `python3`.
::: {.caution}
Packages that are installed with your python derivation, are not accesible by `hy` this way.
:::
### Installation with packages {#installation-with-packages}
Creating `hy` derivation with custom `python` packages is really simple and similar to the way that python does it. Attribute `hy` provides function `withPackages` that creates custom `hy` derivation with specified packages.
For example if you want to create shell with `matplotlib` and `numpy`, you can do it like so:
This contains instructions on how to package javascript applications.
The various tools available will be listed in the [tools-overview](#javascript-tools-overview). Some general principles for packaging will follow. Finally some tool specific instructions will be given.
## Getting unstuck / finding code examples
If you find you are lacking inspiration for packing javascript applications, the links below might prove useful.
Searching online for prior art can be helpful if you are running into solved problems.
### Github
- Searching Nix files for `mkYarnPackage`: <https://github.com/search?q=mkYarnPackage+language%3ANix&type=code>
- Searching just `flake.nix` files for `mkYarnPackage`: <https://github.com/search?q=mkYarnPackage+filename%3Aflake.nix&type=code>
### Gitlab
- Searching Nix files for `mkYarnPackage`: <https://gitlab.com/search?scope=blobs&search=mkYarnPackage+extension%3Anix>
- Searching just `flake.nix` files for `mkYarnPackage`: <https://gitlab.com/search?scope=blobs&search=mkYarnPackage+filename%3Aflake.nix>
## Tools overview {#javascript-tools-overview}
## General principles {#javascript-general-principles}
The following principles are given in order of importance with potential exceptions.
### Try to use the same node version used upstream {#javascript-upstream-node-version}
It is often not documented which node version is used upstream, but if it is, try to use the same version when packaging.
This can be a problem if upstream is using the latest and greatest and you are trying to use an earlier version of node. Some cryptic errors regarding V8 may appear.
An exception to this:
### Try to respect the package manager originally used by upstream (and use the upstream lock file) {#javascript-upstream-package-manager}
A lock file (package-lock.json, yarn.lock...) is supposed to make reproducible installations of node_modules for each tool.
Guidelines of package managers, recommend to commit those lock files to the repos. If a particular lock file is present, it is a strong indication of which package manager is used upstream.
It's better to try to use a nix tool that understand the lock file. Using a different tool might give you hard to understand error because different packages have been installed. An example of problems that could arise can be found [here](https://github.com/NixOS/nixpkgs/pull/126629). Upstream uses npm, but this is an attempt to package it with yarn2nix (that uses yarn.lock)
Using a different tool forces to commit a lock file to the repository. Those files are fairly large, so when packaging for nixpkgs, this approach does not scale well.
Exceptions to this rule are:
- when you encounter one of the bugs from a nix tool. In each of the tool specific instructions, known problems will be detailed. If you have a problem with a particular tool, then it's best to try another tool, even if this means you will have to recreate a lock file and commit it to nixpkgs. In general yarn2nix has less known problems and so a simple search in nixpkgs will reveal many yarn.lock files committed
- Some lock files contain particular version of a package that has been pulled off npm for some reason. In that case, you can recreate upstream lock (by removing the original and `npm install`, `yarn`, ...) and commit this to nixpkgs.
- The only tool that supports workspaces (a feature of npm that helps manage sub-directories with different package.json from a single top level package.json) is yarn2nix. If upstream has workspaces you should try yarn2nix.
### Try to use upstream package.json {#javascript-upstream-package-json}
Exceptions to this rule are
- Sometimes the upstream repo assumes some dependencies be installed globally. In that case you can add them manually to the upstream package.json (`yarn add xxx` or `npm install xxx`, ...). Dependencies that are installed locally can be executed with `npx` for cli tools. (e.g. `npx postcss ...`, this is how you can call those dependencies in the phases).
- Sometimes there is a version conflict between some dependency requirements. In that case you can fix a version (by removing the `^`).
- Sometimes the script defined in the package.json does not work as is. Some scripts for example use cli tools that might not be available, or cd in directory with a different package.json (for workspaces notably). In that case, it's perfectly fine to look at what the particular script is doing and break this down in the phases. In the build script you can see `build:*` calling in turns several other build scripts like `build:ui` or `build:server`. If one of those fails, you can try to separate those into:
```Shell
yarn build:ui
yarn build:server
# OR
npm run build:ui
npm run build:server
```
when you need to override a package.json. It's nice to use the one from the upstream src and do some explicit override. Here is an example.
you will still need to commit the modified version of the lock files, but at least the overrides are explicit for everyone to see.
### Using node_modules directly {#javascript-using-node_modules}
each tool has an abstraction to just build the node_modules (dependencies) directory. you can always use the stdenv.mkDerivation with the node_modules to build the package (symlink the node_modules directory and then use the package build command). the node_modules abstraction can be also used to build some web framework frontends. For an example of this see how [plausible](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/web-apps/plausible/default.nix) is built. mkYarnModules to make the derivation containing node_modules. Then when building the frontend you can just symlink the node_modules directory
The `pkgs/development/node-packages` folder contains a generated collection of
[NPM packages](https://npmjs.com/) that can be installed with the Nix package
manager.
As a rule of thumb, the package set should only provide _end user_ software
packages, such as command-line utilities. Libraries should only be added to the
package set if there is a non-NPM package that requires it.
When it is desired to use NPM libraries in a development project, use the
`node2nix` generator directly on the `package.json` configuration file of the
project.
The package set provides support for the official stable Node.js versions.
The latest stable LTS release in `nodePackages`, as well as the latest stable
Current release in `nodePackages_latest`.
If your package uses native addons, you need to examine what kind of native
build system it uses. Here are some examples:
-`node-gyp`
-`node-gyp-builder`
-`node-pre-gyp`
After you have identified the correct system, you need to override your package
expression while adding in build system as a build input. For example, `dat`
requires `node-gyp-build`, so [we override](https://github.com/NixOS/nixpkgs/blob/32f5e5da4a1b3f0595527f5195ac3a91451e9b56/pkgs/development/node-packages/default.nix#L37-L40) its expression in [`default.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix):
you will need to generate a nix expression for the dependencies
- don't forget the `-l package-lock.json` if there is a lock file
- Most probably you will need the `--development` to include the `devDependencies`
so the command will most likely be
`node2nix --development -l package-lock.json`
[link to the doc in the repo](https://github.com/svanderburg/node2nix)
#### Pitfalls {#javascript-node2nix-pitfalls}
- if upstream package.json does not have a "version" attribute, node2nix will crash. You will need to add it like shown in [the package.json section](#javascript-upstream-package-json)
- node2nix has some [bugs](https://github.com/svanderburg/node2nix/issues/238). related to working with lock files from npm distributed with nodejs-16_x
- node2nix does not like missing packages from npm. If you see something like `Cannot resolve version: vue-loader-v16@undefined` then you might want to try another tool. The package might have been pulled off of npm.
this will by default try to generate a binary. For package only generating static assets (Svelte, Vue, React...), you will need to explicitly override the build step with your instructions. It's important to use the `--offline` flag. For example if you script is `"build": "something"` in package.json use
```nix
buildPhase=''
yarnbuild--offline
'';
```
The dist phase is also trying to build a binary, the only way to override it is with
```nix
distPhase="true";
```
the configure phase can sometimes fail because it tries to be too clever.
this will generate a derivation including the node_modules. If you have to build a derivation for an integrated web framework (rails, phoenix..), this is probably the easiest way. [Plausible](https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/web-apps/plausible/default.nix#L39) offers a good example of how to do this.
#### Overriding dependency behavior
In the `mkYarnPackage` record the property `pkgConfig` can be used to override packages when you encounter problems building.
For instance, say your package is throwing errors when trying to invoke node-sass: `ENOENT: no such file or directory, scandir '/build/source/node_modules/node-sass/vendor'`
To fix this we will specify different versions of build inputs to use, as well as some post install steps to get the software built the way we want:
```nix
mkYarnPackagerec{
pkgConfig={
node-sass={
buildInputs=withfinal;[pythonlibsasspkg-config];
postInstall=''
LIBSASS_EXT=autoyarn--offlinerunbuild
rmbuild/config.gypi
'';
};
};
}
```
#### Pitfalls {#javascript-yarn2nix-pitfalls}
- if version is missing from upstream package.json, yarn will silently install nothing. In that case, you will need to override package.json as shown in the [package.json section](#javascript-upstream-package-json)
- having trouble with node-gyp? Try adding these lines to the `yarnPreBuild` steps:
- The `echo 9` steps comes from this answer: <https://stackoverflow.com/a/49139496>
- Exporting the headers in `npm_config_nodedir` comes from this issue: <https://github.com/nodejs/node-gyp/issues/1191#issuecomment-301243919>
## Outside of nixpkgs {#javascript-outside-nixpkgs}
There are some other options available that can't be used inside nixpkgs. Those other options are written in nix. Importing them in nixpkgs will require moving the source code into nixpkgs. Using [Import From Derivation](https://nixos.wiki/wiki/Import_From_Derivation) is not allowed in hydra at present. If you are packaging something outside nixpkgs, those can be considered
### npmlock2nix {#javascript-npmlock2nix}
[npmlock2nix](https://github.com/nix-community/npmlock2nix) aims at building node_modules without code generation. It hasn't reached v1 yet, the api might be subject to change.
#### Pitfalls {#javascript-npmlock2nix-pitfalls}
- there are some [problems with npm v7](https://github.com/tweag/npmlock2nix/issues/45).
[nix-npm-buildpackage](https://github.com/serokell/nix-npm-buildpackage) aims at building node_modules without code generation. It hasn't reached v1 yet, the api might change. It supports both package-lock.json and yarn.lock.
Now you can use the Lua interpreter, as well as the extra packages (`busted`,
`luafilesystem`) that you added to the environment.
#### Lua environment defined in `~/.config/nixpkgs/config.nix` {#lua-environment-defined-in-.confignixpkgsconfig.nix}
#### Lua environment defined in `~/.config/nixpkgs/config.nix`
If you prefer to, you could also add the environment as a package override to the Nixpkgs set, e.g.
using `config.nix`,
```nix
{# ...
@@ -50,16 +46,14 @@ using `config.nix`,
};
}
```
and install it in your profile with
```shell
nix-env -iA nixpkgs.myLuaEnv
```
The environment is installed by referring to the attribute, and considering
the `nixpkgs` channel was used.
#### Lua environment defined in `/etc/nixos/configuration.nix` {#lua-environment-defined-in-etcnixosconfiguration.nix}
#### Lua environment defined in `/etc/nixos/configuration.nix`
For the sake of completeness, here's another example how to install the environment system-wide.
@@ -72,7 +66,7 @@ For the sake of completeness, here's another example how to install the environm
}
```
### How to override a Lua package using overlays? {#how-to-override-a-lua-package-using-overlays}
### How to override a Lua package using overlays?
Use the following overlay template:
@@ -93,22 +87,18 @@ final: prev:
}
```
### Temporary Lua environment with `nix-shell` {#temporary-lua-environment-with-nix-shell}
### Temporary Lua environment with `nix-shell`
There are two methods for loading a shell with Lua packages. The first and recommended method
is to create an environment with `lua.buildEnv` or `lua.withPackages` and load that. E.g.
```sh
$ nix-shell -p 'lua.withPackages(ps: with ps; [ busted luafilesystem ])'
```
opens a shell from which you can launch the interpreter
```sh
[nix-shell:~] lua
```
The other method, which is not recommended, does not create an environment and requires you to list the packages directly,
```sh
@@ -118,7 +108,7 @@ Again, it is possible to launch the interpreter from the shell.
The Lua interpreter has the attribute `pkgs` which contains all Lua libraries for that specific interpreter.
## Developing with Lua {#developing-with-lua}
## Developing with Lua
Now that you know how to get a working Lua environment with Nix, it is time
to go forward and start actually developing with Lua. There are two ways to
@@ -126,7 +116,7 @@ package lua software, either it is on luarocks and most of it can be taken care
of by the luarocks2nix converter or the packaging has to be done manually.
Let's present the luarocks way first and the manual one in a second time.
### Packaging a library on luarocks {#packaging-a-library-on-luarocks}
### Packaging a library on luarocks
[Luarocks.org](www.luarocks.org) is the main repository of lua packages.
The site proposes two types of packages, the rockspec and the src.rock
@@ -139,15 +129,16 @@ the whitelist maintainers/scripts/luarocks-packages.csv and updated by running m
[luarocks2nix](https://github.com/nix-community/luarocks) is a tool capable of generating nix derivations from both rockspec and src.rock (and favors the src.rock).
The automation only goes so far though and some packages need to be customized.
These customizations go in `pkgs/development/lua-modules/overrides.nix`.
For instance if the rockspec defines `external_dependencies`, these need to be manually added to the overrides.nix.
For instance if the rockspec defines `external_dependencies`, these need to be manually added in its rockspec file then it won't work.
You can try converting luarocks packages to nix packages with the command `nix-shell -p luarocks-nix` and then `luarocks nix PKG_NAME`.
Nix rely on luarocks to install lua packages, basically it runs:
`luarocks make --deps-mode=none --tree $out`
#### Packaging a library manually {#packaging-a-library-manually}
#### Packaging a library manually
You can develop your package as you usually would, just don't forget to wrap it
There is also the `buildLuaPackage` function that can be used when lua modules
are not packaged for luarocks. You can see a few examples at `pkgs/top-level/lua-packages.nix`.
## Lua Reference {#lua-reference}
## Lua Reference
### Lua interpreters {#lua-interpreters}
### Lua interpreters
Versions 5.1, 5.2, 5.3 and 5.4 of the lua interpreter are available as
respectively `lua5_1`, `lua5_2`, `lua5_3` and `lua5_4`. Luajit is available too.
Versions 5.1, 5.2 and 5.3 of the lua interpreter are available as
respectively `lua5_1`, `lua5_2` and `lua5_3`. Luajit is available too.
The Nix expressions for the interpreters can be found in `pkgs/development/interpreters/lua-5`.
#### Attributes on lua interpreters packages {#attributes-on-lua-interpreters-packages}
#### Attributes on lua interpreters packages
Each interpreter has the following attributes:
@@ -172,7 +164,8 @@ Each interpreter has the following attributes:
-`withPackages`. Simpler interface to `buildEnv`.
-`pkgs`. Set of Lua packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
#### `buildLuarocksPackage` function {#buildluarockspackage-function}
#### `buildLuarocksPackage` function
The `buildLuarocksPackage` function is implemented in `pkgs/development/interpreters/lua-5/build-lua-package.nix`
The following is an example:
@@ -212,17 +205,16 @@ install the package
By default `meta.platforms` is set to the same value as the interpreter unless overridden otherwise.
#### `buildLuaApplication` function {#buildluaapplication-function}
#### `buildLuaApplication` function
The `buildLuaApplication` function is practically the same as `buildLuaPackage`.
The difference is that `buildLuaPackage` by default prefixes the names of the packages with the version of the interpreter.
Because with an application we're not interested in multiple version the prefix is dropped.
#### lua.withPackages function {#lua.withpackages-function}
#### lua.withPackages function
The `lua.withPackages` takes a function as an argument that is passed the set of lua packages and returns the list of packages to be included in the environment.
Using the `withPackages` function, the previous example for the luafilesystem environment can be written like this:
`withPackages` passes the correct package set for the specific interpreter version as an argument to the function. In the above example, `ps` equals `luaPackages`.
You find this demo project at https://github.com/fzakaria/nixos-maven-example
## Solving for dependencies {#solving-for-dependencies}
## Solving for dependencies
### buildMaven with NixOS/mvn2nix-maven-plugin {#buildmaven-with-nixosmvn2nix-maven-plugin}
### buildMaven with NixOS/mvn2nix-maven-plugin
> ⚠️ Although `buildMaven` is the "blessed" way within nixpkgs, as of 2020, it hasn't seen much activity in quite a while.
@@ -82,7 +82,6 @@ This file is then given to the `buildMaven` function, and it returns 2 attribute
A simple derivation that runs through `mvn compile` & `mvn package` to build the JAR. You may use this as inspiration for more complicated derivations.
Here is an [example](https://github.com/fzakaria/nixos-maven-example/blob/main/build-maven-repository.nix) of building the Maven repository
```nix
{pkgs?import<nixpkgs>{}}:
withpkgs;
@@ -104,8 +103,7 @@ The benefit over the _double invocation_ as we will see below, is that the _/nix
> ⚠️ This pattern is the simplest but may cause unnecessary rebuilds due to the output hash changing.
@@ -165,7 +163,7 @@ The build will fail, and tell you the expected `outputHash` to place. When you'v
If your package uses _SNAPSHOT_ dependencies or _version ranges_; there is a strong likelihood that over-time your output hash will change since the resolved dependencies may change. Hence this method is less recommended then using `buildMaven`.
## Building a JAR {#building-a-jar}
## Building a JAR
Regardless of which strategy is chosen above, the step to build the derivation is the same.
@@ -203,7 +201,7 @@ in stdenv.mkDerivation rec {
2 directories, 1 file
```
## Runnable JAR {#runnable-jar}
## Runnable JAR
The previous example builds a `jar` file but that's not a file one can run.
@@ -215,7 +213,7 @@ We will use the same repository we built above (either _double invocation_ or _b
The following two methods are more suited to Nix then building an [UberJar](https://imagej.net/Uber-JAR) which may be the more traditional approach.
### CLASSPATH {#classpath}
### CLASSPATH
> This is ideal if you are providing a derivation for _nixpkgs_ and don't want to patch the project's `pom.xml`.
@@ -254,12 +252,11 @@ in stdenv.mkDerivation rec {
}
```
### MANIFEST file via Maven Plugin {#manifest-file-via-maven-plugin}
### MANIFEST file via Maven Plugin
> This is ideal if you are the project owner and want to change your `pom.xml` to set the CLASSPATH within it.
Augment the `pom.xml` to create a JAR with the following manifest:
The `pkgs/development/node-packages` folder contains a generated collection of
[NPM packages](https://npmjs.com/) that can be installed with the Nix package
manager.
As a rule of thumb, the package set should only provide *end user* software
packages, such as command-line utilities. Libraries should only be added to the
package set if there is a non-NPM package that requires it.
When it is desired to use NPM libraries in a development project, use the
`node2nix` generator directly on the `package.json` configuration file of the
project.
The package set provides support for the official stable Node.js versions.
The latest stable LTS release in `nodePackages`, as well as the latest stable
Current release in `nodePackages_latest`.
If your package uses native addons, you need to examine what kind of native
build system it uses. Here are some examples:
*`node-gyp`
*`node-gyp-builder`
*`node-pre-gyp`
After you have identified the correct system, you need to override your package
expression while adding in build system as a build input. For example, `dat`
requires `node-gyp-build`, so [we override](https://github.com/NixOS/nixpkgs/blob/32f5e5da4a1b3f0595527f5195ac3a91451e9b56/pkgs/development/node-packages/default.nix#L37-L40) its expression in [`default.nix`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/node-packages/default.nix):
OCaml libraries are available in attribute sets of the form `ocaml-ng.ocamlPackages_X_XX` where X is to be replaced with the desired compiler version. For example, ocamlgraph compiled with OCaml 4.12 can be found in `ocaml-ng.ocamlPackages_4_12.ocamlgraph`. The compiler itself is also located in this set, under the name `ocaml`.
If you don't care about the exact compiler version, `ocamlPackages` is a top-level alias pointing to a recent version of OCaml.
OCaml applications are usually available top-level, and not inside `ocamlPackages`. Notable exceptions are build tools that must be built with the same compiler version as the compiler you intend to use like `dune` or `ocaml-lsp`.
To open a shell able to build a typical OCaml project, put the dependencies in `buildInputs` and add `ocamlPackages.ocaml` and `ocamlPackages.findlib` to `nativeBuildInputs` at least.
OCaml libraries should be installed in `$(out)/lib/ocaml/${ocaml.version}/site-lib/`. Such directories are automatically added to the `$OCAMLPATH` environment variable when building another package that depends on them or when opening a `nix-shell`.
Given that most of the OCaml ecosystem is now built with dune, nixpkgs includes a convenience build support function called `buildDunePackage` that will build an OCaml package using dune, OCaml and findlib and any additional dependencies provided as `buildInputs` or `propagatedBuildInputs`.
Here is a simple package example.
- It defines an (optional) attribute `minimalOCamlVersion` that will be used to
throw a descriptive evaluation error if building with an older OCaml is
attempted.
- It uses the `fetchFromGitHub` fetcher to get its source.
-`useDune2 = true` ensures that the latest version of Dune is used for the
build (this may become the default value in a future release).
- It sets the optional `doCheck` attribute such that tests will be run with
`dune runtest -p angstrom` after the build (`dune build -p angstrom`) is
complete, but only if the Ocaml version is at at least `"4.05"`.
- It uses the package `ocaml-syntax-shims` as a build input, `alcotest` and
`ppx_let` as check inputs (because they are needed to run the tests), and
`bigstringaf` and `result` as propagated build inputs (thus they will also be
available to libraries depending on this library).
- The library will be installed using the `angstrom.install` file that dune
generates.
Here is a simple package example. It defines an (optional) attribute `minimumOCamlVersion` that will be used to throw a descriptive evaluation error if building with an older OCaml is attempted. It uses the `fetchFromGitHub` fetcher to get its source. It sets the `doCheck` (optional) attribute to `true` which means that tests will be run with `dune runtest -p angstrom` after the build (`dune build -p angstrom`) is complete. It uses `alcotest` as a build input (because it is needed to run the tests) and `bigstringaf` and `result` as propagated build inputs (thus they will also be available to libraries depending on this library). The library will be installed using the `angstrom.install` file that dune generates.
Here is a second example, this time using a source archive generated with `dune-release`. It is a good idea to use this archive when it is available as it will usually contain substituted variables such as a `%%VERSION%%` field. This library does not depend on any other OCaml library and no tests are run after building it.
Nixpkgs provides a function `buildOctavePackage`, a generic package builder function for any Octave package that complies with the Octave's current packaging format.
All Octave packages are defined in [pkgs/top-level/octave-packages.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/octave-packages.nix) rather than `pkgs/all-packages.nix`.
Each package is defined in their own file in the [pkgs/development/octave-modules](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/octave-modules) directory.
Octave packages are made available through `all-packages.nix` through both the attribute `octavePackages` and `octave.pkgs`.
You can test building an Octave package as follows:
```ShellSession
$ nix-build -A octavePackages.symbolic
```
When building Octave packages with `nix-build`, the `buildOctavePackage` function adds `octave-octaveVersion` to; the start of the package's name attribute.
This can be required when installing the package using `nix-env`:
```ShellSession
$ nix-env -i octave-6.2.0-symbolic
```
Although, you can also install it using the attribute name:
```ShellSession
$ nix-env -i -A octavePackages.symbolic
```
You can build Octave with packages by using the `withPackages` passed-through function.
```ShellSession
$ nix-shell -p 'octave.withPackages (ps: with ps; [ symbolic ])'
The `buildOctavePackage` does several things to make sure things work properly.
1. Sets the environment variable `OCTAVE_HISTFILE` to `/dev/null` during package compilation so that the commands run through the Octave interpreter directly are not logged.
2. Skips the configuration step, because the packages are stored as gzipped tarballs, which Octave itself handles directly.
3. Change the hierarchy of the tarball so that only a single directory is at the top-most level of the tarball.
4. Use Octave itself to run the `pkg build` command, which unzips the tarball, extracts the necessary files written in Octave, and compiles any code written in C++ or Fortran, and places the fully compiled artifact in `$out`.
`buildOctavePackage` is built on top of `stdenv` in a standard way, allowing most things to be customized.
In Octave packages, there are four sets of dependencies that can be specified:
`nativeBuildInputs`
: Just like other packages, `nativeBuildInputs` is intended for architecture-dependent build-time-only dependencies.
`buildInputs`
: Like other packages, `buildInputs` is intended for architecture-independent build-time-only dependencies.
`propagatedBuildInputs`
: Similar to other packages, `propagatedBuildInputs` is intended for packages that are required for both building and running of the package.
See [Symbolic](https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/octave-modules/symbolic/default.nix) for how this works and why it is needed.
`requiredOctavePackages`
: This is a special dependency that ensures the specified Octave packages are dependent on others, and are made available simultaneously when loading them in Octave.
By default, the `buildOctavePackage` function does _not_ install the requested package into Octave for use.
The function will only build the requested package.
This is due to Octave maintaining an text-based database about which packages are installed where.
To this end, when all the requested packages have been built, the Octave package and all its add-on packages are put together into an environment, similar to Python.
1. First, all the Octave binaries are wrapped with the environment variable `OCTAVE_SITE_INITFILE` set to a file in `$out`, which is required for Octave to be able to find the non-standard package database location.
2. Because of the way `buildEnv` works, all tarballs that are present (which should be all Octave packages to install) should be removed.
3. The path down to the default install location of Octave packages is recreated so that Nix-operated Octave can install the packages.
4. Install the packages into the `$out` environment while writing package entries to the database file.
This database file is unique for each different (according to Nix) environment invocation.
5. Rewrite the Octave-wide startup file to read from the list of packages installed in that particular environment.
6. Wrap any programs that are required by the Octave packages so that they work with all the paths defined within the environment.
It is highly encouraged to have testing as part of the package build. This
helps to avoid situations where the package was able to build and install,
@@ -560,11 +559,10 @@ thus can cause issues when a test suite asserts on that behavior.
as many tests should be enabled as possible. Failing tests can still be
a good indication that the package is not in a valid state.
#### Using pytest {#using-pytest}
#### Using pytest
Pytest is the most common test runner for python repositories. A trivial
test run would be:
```
checkInputs = [ pytest ];
checkPhase = "pytest";
@@ -574,7 +572,6 @@ However, many repositories' test suites do not translate well to nix's build
sandbox, and will generally need many tests to be disabled.
To filter tests using pytest, one can do the following:
```
checkInputs = [ pytest ];
# avoid tests which need additional data or touch network
@@ -590,20 +587,19 @@ easier than having to create a new package.
`-k` is used to define a predicate for test names. In this example, we are
filtering out tests which contain `download` or `update` in their test case name.
Only one `-k` argument is allowed, and thus a long predicate should be concatenated
with “\\” and wrapped to the next line.
Only one `-k` argument is allows, and thus a long predicate should be concatenated
with "\" and wrapped to the next line.
*NOTE:* In pytest==6.0.1, the use of “\\” to continue a line (e.g. `-k 'not download \'`) has
*NOTE:* In pytest==6.0.1, the use of "\" to continue a line (e.g. `-k 'not download \'`) has
been removed, in this case, it's recommended to use `pytestCheckHook`.
#### Using pytestCheckHook {#using-pytestcheckhook}
#### Using pytestCheckHook
`pytestCheckHook` is a convenient hook which will substitute the setuptools
`test` command for a checkPhase which runs `pytest`. This is also beneficial
when a package may need many items disabled to run the test suite.
Using the example above, the analagous pytestCheckHook usage would be:
```
checkInputs = [ pytestCheckHook ];
@@ -641,7 +637,7 @@ Trying to concatenate the related strings to disable tests in a regular checkPha
would be much harder to read. This also enables us to comment on why specific tests
are disabled.
#### Using pythonImportsCheck {#using-pythonimportscheck}
#### Using pythonImportsCheck
Although unit tests are highly prefered to validate correctness of a package, not
all packages have test suites that can be ran easily, and some have none at all.
@@ -663,7 +659,7 @@ However, this is done in it's own phase, and not dependent on whether `doCheck =
This can also be useful in verifying that the package doesn't assume commonly
present packages (e.g. `setuptools`)
### Develop local package {#develop-local-package}
### Develop local package
As a Python developer you're likely aware of [development mode](http://setuptools.readthedocs.io/en/latest/setuptools.html#development-mode)
(`python setup.py develop`); instead of installing the package this command
@@ -698,7 +694,7 @@ buildPythonPackage rec {
It is important to note that due to how development mode is implemented on Nix
it is not possible to have multiple packages simultaneously in development mode.
### Organising your packages {#organising-your-packages}
### Organising your packages
So far we discussed how you can use Python on Nix, and how you can develop with
it. We've looked at how you write expressions to package Python packages, and we
@@ -710,7 +706,7 @@ like to be able to use in different projects. In order to minimise unnecessary
duplication we now look at how you can maintain a repository with your
own packages. The important functions here are `import` and `callPackage`.
### Including a derivation using `callPackage` {#including-a-derivation-using-callpackage}
### Including a derivation using `callPackage`
Earlier we created a Python environment using `withPackages`, and included the
`toolz` package via a `let` expression.
@@ -760,12 +756,12 @@ don't explicitly define which `python` derivation should be used. In the above
example we use `buildPythonPackage` that is part of the set `python38Packages`,
and in this case the `python38` interpreter is automatically used.
## Reference {#reference}
## Reference
### Interpreters {#interpreters}
### Interpreters
Versions 2.7, 3.6, 3.7, 3.8 and 3.9 of the CPython interpreter are available as
respectively `python27`, `python37`, `python38` and `python39`. The
respectively `python27`, `python36`, `python37`, `python38` and `python39`. The
aliases `python2` and `python3` correspond to respectively `python27` and
`python39`. The attribute `python` maps to `python2`. The PyPy interpreters
compatible with Python 2.7 and 3 are available as `pypy27` and `pypy3`, with
@@ -777,11 +773,11 @@ All packages depending on any Python interpreter get appended
`out/{python.sitePackages}` to `$PYTHONPATH` if such directory
exists.
#### Missing `tkinter` module standard library {#missing-tkinter-module-standard-library}
#### Missing `tkinter` module standard library
To reduce closure size the `Tkinter`/`tkinter` is available as a separate package, `pythonPackages.tkinter`.
#### Attributes on interpreters packages {#attributes-on-interpreters-packages}
#### Attributes on interpreters packages
Each interpreter has the following attributes:
@@ -793,7 +789,7 @@ Each interpreter has the following attributes:
- `executable`. Name of the interpreter executable, e.g. `python3.8`.
- `pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
### Optimizations {#optimizations}
### Optimizations
The Python interpreters are by default not build with optimizations enabled, because
the builds are in that case not reproducible. To enable optimizations, override the
@@ -810,7 +806,7 @@ let
in mypython
```
### Building packages and applications {#building-packages-and-applications}
### Building packages and applications
Python libraries and applications that use `setuptools` or
`distutils` are typically built with respectively the `buildPythonPackage` and
@@ -830,19 +826,19 @@ attribute set is created for each available Python interpreter. The available
sets are
* `pkgs.python27Packages`
* `pkgs.python36Packages`
* `pkgs.python37Packages`
* `pkgs.python38Packages`
* `pkgs.python39Packages`
* `pkgs.python310Packages`
* `pkgs.pypyPackages`
and the aliases
* `pkgs.python2Packages` pointing to `pkgs.python27Packages`
* `pkgs.python3Packages` pointing to `pkgs.python39Packages`
* `pkgs.python3Packages` pointing to `pkgs.python38Packages`
* `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
#### `buildPythonPackage` function {#buildpythonpackage-function}
#### `buildPythonPackage` function
The `buildPythonPackage` function is implemented in
### Rationale of non-existent global site-packages {#rationale-of-non-existent-global-site-packages}
### Rationale of non-existent global site-packages
On most operating systems a global `site-packages` is maintained. This however
becomes problematic if you want to run multiple Python versions or have multiple
@@ -1391,7 +1384,7 @@ If you want to create a Python environment for development, then the recommended
method is to use `nix-shell`, either with or without the `python.buildEnv`
function.
### How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems? {#how-to-consume-python-modules-using-pip-in-a-virtual-environment-like-i-am-used-to-on-other-operating-systems}
### How to consume Python modules using pip in a virtual environment like I am used to on other Operating Systems?
While this approach is not very idiomatic from Nix perspective, it can still be
useful when dealing with pre-existing projects or in situations where it's not
@@ -1504,7 +1497,7 @@ is executed it will attempt to download the Python modules listed in
requirements.txt. However these will be cached locally within the `virtualenv`
folder and not downloaded again.
### How to override a Python package from `configuration.nix`? {#how-to-override-a-python-package-from-configuration.nix}
### How to override a Python package from `configuration.nix`?
If you need to change a package's attribute(s) from `configuration.nix` you could do:
@@ -1513,7 +1506,7 @@ If you need to change a package's attribute(s) from `configuration.nix` you coul
### What inputs do `setup_requires`, `install_requires` and `tests_require` map to? {#what-inputs-do-setup_requires-install_requires-and-tests_require-map-to}
### What inputs do `setup_requires`, `install_requires` and `tests_require` map to?
In a `setup.py` or `setup.cfg` it is common to declare dependencies:
@@ -1576,9 +1569,9 @@ In a `setup.py` or `setup.cfg` it is common to declare dependencies:
* `install_requires` corresponds to `propagatedBuildInputs`
Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. The attribute `ruby` refers to the default Ruby interpreter, which is currently MRI 2.6. It's also possible to refer to specific versions, e.g. `ruby_2_y`, `jruby`, or `mruby`.
@@ -8,11 +8,11 @@ In the Nixpkgs tree, Ruby packages can be found throughout, depending on what th
There are two main approaches for using Ruby with gems. One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. The other is to depend on the common gems, which we'll explain further down, and rely on them being updated regularly.
The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_7.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use.
The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_6.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use.
Since not all gems have executables like `nokogiri`, it's usually more convenient to use the `withPackages` function like this: `ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the Ruby in your environment will be able to find the gem and it can be used in your Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` as usual.
### Temporary Ruby environment with `nix-shell` {#temporary-ruby-environment-with-nix-shell}
### Temporary Ruby environment with `nix-shell`
Rather than having a single Ruby environment shared by all Ruby development projects on a system, Nix allows you to create separate environments per project. `nix-shell` gives you the possibility to temporarily load another environment akin to a combined `chruby` or `rvm` and `bundle exec`.
Again, it's possible to launch the interpreter from the shell. The Ruby interpreter has the attribute `gems` which contains all Ruby gems for that specific interpreter.
#### Load Ruby environment from `.nix` expression {#load-ruby-environment-from-.nix-expression}
#### Load Ruby environment from `.nix` expression
As explained in the Nix manual, `nix-shell` can also load an expression from a `.nix` file. Say we want to have Ruby 2.6, `nokogori`, and `pry`. Consider a `shell.nix` file with:
@@ -45,7 +45,7 @@ What's happening here?
2. Then we create a Ruby environment with the `withPackages` function.
3. The `withPackages` function expects us to provide a function as an argument that takes the set of all ruby gems and returns a list of packages to include in the environment. Here, we select the packages `nokogiri` and `pry` from the package set.
#### Execute command with `--run` {#execute-command-with---run}
#### Execute command with `--run`
A convenient flag for `nix-shell` is `--run`. It executes a command in the `nix-shell`. We can e.g. directly open a `pry` REPL:
@@ -65,7 +65,7 @@ Or run a script using this environment:
#### Using `nix-shell` as shebang {#using-nix-shell-as-shebang}
#### Using `nix-shell` as shebang
In fact, for the last case, there is a more convenient method. You can add a [shebang](<https://en.wikipedia.org/wiki/Shebang_(Unix)>) to your script specifying which dependencies `nix-shell` needs. With the following shebang, you can just execute `./example.rb`, and it will run with all dependencies.
@@ -80,9 +80,9 @@ body = RestClient.get('http://example.com').body
putsNokogiri::HTML(body).at('h1').text
```
## Developing with Ruby {#developing-with-ruby}
## Developing with Ruby
### Using an existing Gemfile {#using-an-existing-gemfile}
### Using an existing Gemfile
In most cases, you'll already have a `Gemfile.lock` listing all your dependencies. This can be used to generate a `gemset.nix` which is used to fetch the gems and combine them into a single environment. The reason why you need to have a separate file for this, is that Nix requires you to have a checksum for each input to your build. Since the `Gemfile.lock` that `bundler` generates doesn't provide us with checksums, we have to first download each gem, calculate its SHA256, and store it in this separate file.
@@ -120,7 +120,7 @@ One common issue that you might have is that you have Ruby 2.6, but also `bundle
Of course for this use-case one could also use overlays since the configuration for `pg` depends on the `postgresql` alias, but for demonstration purposes this has to suffice.
### Adding a gem to the default gemset {#adding-a-gem-to-the-default-gemset}
### Adding a gem to the default gemset
Now that you know how to get a working Ruby environment with Nix, it's time to go forward and start actually developing with Ruby. We will first have a look at how Ruby gems are packaged on Nix. Then, we will look at how you can use development mode with your code.
@@ -215,7 +215,7 @@ To test that it works, you can then try using the gem with:
NIX_PATH=nixpkgs=$PWD nix-shell -p "ruby.withPackages (ps: with ps; [ name-of-your-gem ])"
A common task is to add a ruby executable to nixpkgs, popular examples would be `chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp` function, that allows you to make a package that only exposes the listed executables, otherwise the package may cause conflicts through common paths like `bin/rake` or `bin/bundler` that aren't meant to be used.
@@ -243,7 +243,7 @@ bundlerApp {
All that's left to do is to generate the corresponding `Gemfile.lock` and `gemset.nix` as described above in the `Using an existing Gemfile` section.
#### Packaging executables that require wrapping {#packaging-executables-that-require-wrapping}
#### Packaging executables that require wrapping
Sometimes your app will depend on other executables at runtime, and tries to find it through the `PATH` environment variable.
@@ -1004,3 +806,32 @@ Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.
```
Note that this will fetch the latest overlay version when rebuilding your system.
### Rust overlay usage
The overlay contains attribute sets corresponding to different versions of the rust toolchain, such as:
* `latest.rustChannels.stable`
* `latest.rustChannels.nightly`
* a function `rustChannelOf`, called as `(rustChannelOf { date = "2018-04-11"; channel = "nightly"; })`, or...
* `(nixpkgs.rustChannelOf { rustToolchain = ./rust-toolchain; })` if you have a local `rust-toolchain` file (see https://github.com/mozilla/nixpkgs-mozilla#using-in-nix-expressions for an example)
Each of these contain packages such as `rust`, which contains your usual rust development tools with the respective toolchain chosen.
For example, you might want to add `latest.rustChannels.stable.rust` to the list of packages in your configuration.
Imperatively, the latest stable version can be installed with the following command:
- For basic usage just pull `texlive.combined.scheme-basic` for an environment with basic LaTeX support.
- It typically won't work to use separately installed packages together. Instead, you can build a custom set of packages like this:
```nix
@@ -15,7 +14,6 @@ Since release 15.09 there is a new TeX Live packaging that lives entirely under
```
- There are all the schemes, collections and a few thousand packages, as defined upstream (perhaps with tiny differences).
- By default you only get executables and files needed during runtime, and a little documentation for the core packages. To change that, you need to add `pkgFilter` function to `combine`.
@@ -12,7 +12,7 @@ At the moment we support three different methods for managing plugins:
- Pathogen
- vim-plug
## Custom configuration {#custom-configuration}
## Custom configuration
Adding custom .vimrc lines can be done using the following code:
@@ -56,7 +56,7 @@ neovim-qt.override {
}
```
## Managing plugins with Vim packages {#managing-plugins-with-vim-packages}
## Managing plugins with Vim packages
To store you plugins in Vim packages (the native Vim plugin manager, see `:help packages`) the following example can be used:
@@ -116,11 +116,11 @@ The resulting package can be added to `packageOverrides` in `~/.nixpkgs/config.n
After that you can install your special grafted `myVim` or `myNeovim` packages.
### What if your favourite Vim plugin isn’t already packaged? {#what-if-your-favourite-vim-plugin-isnt-already-packaged}
### What if your favourite Vim plugin isn't already packaged?
If one of your favourite plugins isn't packaged, you can package it yourself:
```nix
```
{ config, pkgs, ... }:
let
@@ -154,34 +154,7 @@ in
}
```
### Specificities for some plugins
#### Tree sitter
By default `nvim-treesitter` encourages you to download, compile and install
the required tree-sitter grammars at run time with `:TSInstall`. This works
poorly on NixOS. Instead, to install the `nvim-treesitter` plugins with a set
of precompiled grammars, you can use `nvim-treesitter.withPlugins` function:
```nix
(pkgs.neovim.override{
configure={
packages.myPlugins=withpkgs.vimPlugins;{
start=[
(nvim-treesitter.withPlugins(
plugins:withplugins;[
tree-sitter-nix
tree-sitter-python
]
))
];
};
};
})
```
To enable all grammars packaged in nixpkgs, use `(pkgs.vimPlugins.nvim-treesitter.withPlugins (plugins: pkgs.tree-sitter.allGrammars))`.
## Managing plugins with vim-plug {#managing-plugins-with-vim-plug}
## Managing plugins with vim-plug
To use [vim-plug](https://github.com/junegunn/vim-plug) to manage your Vim
plugins the following example can be used:
@@ -210,14 +183,14 @@ neovim.override {
}
```
## Managing plugins with VAM {#managing-plugins-with-vam}
## Managing plugins with VAM
### Handling dependencies of Vim plugins {#handling-dependencies-of-vim-plugins}
### Handling dependencies of Vim plugins
VAM introduced .json files supporting dependencies without versioning
assuming that "using latest version" is ok most of the time.
### Example {#example}
### Example
First create a vim-scripts file having one plugin name per line. Example:
@@ -307,9 +280,9 @@ Sample output2:
]
```
## Adding new plugins to nixpkgs {#adding-new-plugins-to-nixpkgs}
## Adding new plugins to nixpkgs
Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]@[gitref]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
@@ -325,7 +298,7 @@ To add a new plugin, run `./update.py --add "[owner]/[name]"`. **NOTE**: This sc
Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `update.py` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of Language Server Protocol integration with vim/neovim.
## Updating plugins in nixpkgs {#updating-plugins-in-nixpkgs}
## Updating plugins in nixpkgs
Run the update script with a GitHub API token that has at least `public_repo` access. Running the script without the token is likely to result in rate-limiting (429 errors). For steps on creating an API token, please refer to [GitHub's token documentation](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token).
@@ -339,7 +312,7 @@ Alternatively, set the number of processes to a lower count to avoid rate-limiti
./pkgs/misc/vim-plugins/update.py --proc 1
```
## Important repositories {#important-repositories}
## Important repositories
- [vim-pi](https://bitbucket.org/vimcommunity/vim-pi) is a plugin repository
from VAM plugin manager meant to be used by others as well used by
This chapter will be organized in three parts. First, it will describe the basics of how to package software in a way that supports cross-compilation. Second, it will describe how to use Nixpkgs when cross-compiling. Third, it will describe the internal infrastructure supporting cross-compilation.
## Packaging in a cross-friendly manner {#sec-cross-packaging}
@@ -64,8 +65,8 @@ The exact schema these fields follow is a bit ill-defined due to a long and conv
### Theory of dependency categorization {#ssec-cross-dependency-categorization}
::: {.note}
This is a rather philosophical description that isn't very Nixpkgs-specific. For an overview of all the relevant attributes given to `mkDerivation`, see [](#ssec-stdenv-dependencies). For a description of how everything is implemented, see [](#ssec-cross-dependency-implementation).
::: note
This is a rather philosophical description that isn't very Nixpkgs-specific. For an overview of all the relevant attributes given to `mkDerivation`, see <xref linkend="ssec-stdenv-dependencies"/>. For a description of how everything is implemented, see <xref linkend="ssec-cross-dependency-implementation"/>.
:::
In this section we explore the relationship between both runtime and build-time dependencies and the 3 Autoconf platforms.
@@ -80,10 +81,10 @@ Finally, if the depending package is a compiler or other machine-code-producing
Putting this all together, that means we have dependencies in the form "host → target", in at most the following six combinations:
#### Possible dependency types {#possible-dependency-types}
@@ -112,18 +113,15 @@ On less powerful machines, it can be inconvenient to cross-compile a package onl
$ nix-build '<nixpkgs>' -A pkgsCross.raspberryPi.hello
```
#### What if my package’s build system needs to build a C program to be run under the build environment? {#cross-qa-build-c-program-in-build-environment}
#### What if my package's build system needs to build a C program to be run under the build environment? {#cross-qa-build-c-program-in-build-environment}
Add the following to your `mkDerivation` invocation.
```nix
depsBuildBuild=[buildPackages.stdenv.cc];
```
#### My package’s testsuite needs to run host platform code. {#cross-testsuite-runs-host-code}
#### My package's testsuite needs to run host platform code. {#cross-testsuite-runs-host-code}
Add the following to your `mkDerivation` invocation.
@@ -136,7 +134,7 @@ Nixpkgs can be instantiated with `localSystem` alone, in which case there is no
$ nix-build '<nixpkgs>' --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever
```
::: {.note}
::: note
Eventually we would like to make these platform examples an unnecessary convenience so that
```ShellSession
@@ -148,7 +146,7 @@ works in the vast majority of cases. The problem today is dependencies on other
While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields. As discussed in the previous section, only one of `system`, `config`, and `parsed` is needed to infer the other two. Additionally, `libc` will be inferred from `parse`. Finally, `localSystem.system` is also _impurely_ inferred based on the platform evaluation occurs. This means it is often not necessary to pass `localSystem` at all, as in the command-line example in the previous paragraph.
::: {.note}
::: note
Many sources (manual, wiki, etc) probably mention passing `system`, `platform`, along with the optional `crossSystem` to Nixpkgs: `import <nixpkgs> { system = ..; platform = ..; crossSystem = ..; }`. Passing those two instead of `localSystem` is still supported for compatibility, but is discouraged. Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience.
:::
@@ -158,9 +156,9 @@ One would think that `localSystem` and `crossSystem` overlap horribly with the t
### Implementation of dependencies {#ssec-cross-dependency-implementation}
The categories of dependencies developed in [](#ssec-cross-dependency-categorization) are specified as lists of derivations given to `mkDerivation`, as documented in [](#ssec-stdenv-dependencies). In short, each list of dependencies for "host → target" is called `deps<host><target>` (where `host`, and `target` values are either `build`, `host`, or `target`), with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `deps<host><target>` is automatically taken from `pkgs<host><target>`. (These `pkgs<host><target>`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.gcc` should be used at run-time.
The categories of dependencies developed in <xref linkend="ssec-cross-dependency-categorization"/> are specified as lists of derivations given to `mkDerivation`, as documented in <xref linkend="ssec-stdenv-dependencies"/>. In short, each list of dependencies for "host → target" of "foo → bar" is called `depsFooBar`, with exceptions for backwards compatibility that `depsBuildHost` is instead called `nativeBuildInputs` and `depsHostTarget` is instead called `buildInputs`. Nixpkgs is now structured so that each `depsFooBar` is automatically taken from `pkgsFooBar`. (These `pkgsFooBar`s are quite new, so there is no special case for `nativeBuildInputs` and `buildInputs`.) For example, `pkgsBuildHost.gcc` should be used at build-time, while `pkgsHostTarget.gcc` should be used at run-time.
Now, for most of Nixpkgs's history, there were no `pkgs<host><target>` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
Now, for most of Nixpkgs's history, there were no `pkgsFooBar` attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just `buildPackages`, `pkgs`, and `targetPackages`. Those are now redefined as aliases to `pkgsBuildHost`, `pkgsHostTarget`, and `pkgsTargetTarget`. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
But before that, there was just `pkgs`, even though both `buildInputs` and `nativeBuildInputs` existed. \[Cross barely worked, and those were implemented with some hacks on `mkDerivation` to override dependencies.\] What this means is the vast majority of packages do not use any explicit package set to populate their dependencies, just using whatever `callPackage` gives them even if they do correctly sort their dependencies into the multiple lists described above. And indeed, asking that users both sort their dependencies, _and_ take them from the right attribute set, is both too onerous and redundant, so the recommended approach (for now) is to continue just categorizing by list and not using an explicit package set.
@@ -180,7 +178,7 @@ While there are many package sets, and thus many edges, the stages can also be a
In each stage, `pkgsBuildHost` refers to the previous stage, `pkgsBuildBuild` refers to the one before that, and `pkgsHostTarget` refers to the current one, and `pkgsTargetTarget` refers to the next one. When there is no previous or next stage, they instead refer to the current stage. Note how all the invariants regarding the mapping between dependency and depending packages' build host and target platforms are preserved. `pkgsBuildTarget` and `pkgsHostHost` are more complex in that the stage fitting the requirements isn't always a fixed chain of "prevs" and "nexts" away (modulo the "saturating" self-references at the ends). We just special case each instead. All the primary edges are implemented is in `pkgs/stdenv/booter.nix`, and secondarily aliases in `pkgs/top-level/stage.nix`.
::: {.note}
::: note
The native stages are bootstrapped in legacy ways that predate the current cross implementation. This is why the bootstrapping stages leading up to the final stages are ignored in the previous paragraph.
:::
@@ -188,7 +186,6 @@ If one looks at the 3 platform triples, one can see that they overlap such that
```
(native, native, native, foreign, foreign)
```
If one imagines the saturating self references at the end being replaced with infinite stages, and then overlays those platform triples, one ends up with the infinite tuple:
@@ -196,8 +193,8 @@ If one imagines the saturating self references at the end being replaced with in
One can then imagine any sequence of platforms such that there are bootstrap stages with their 3 platforms determined by "sliding a window" that is the 3 tuple through the sequence. This was the original model for bootstrapping. Without a target platform (assume a better world where all compilers are multi-target and all standard libraries are built in their own derivation), this is sufficient. Conversely if one wishes to cross compile "faster", with a "Canadian Cross" bootstrapping stage where `build != host != target`, more bootstrapping stages are needed since no sliding window provides the pesky `pkgsBuildTarget` package set since it skips the Canadian cross stage's "host".
::: {.note}
It is much better to refer to `buildPackages` than `targetPackages`, or more broadly package sets that do not mention “target”. There are three reasons for this.
::: note
It is much better to refer to `buildPackages` than `targetPackages`, or more broadly package sets that do not mention "target". There are three reasons for this.
First, it is because bootstrapping stages do not have a unique `targetPackages`. For example a `(x86-linux, x86-linux, arm-linux)` and `(x86-linux, x86-linux, x86-windows)` package set both have a `(x86-linux, x86-linux, x86-linux)` package set. Because there is no canonical `targetPackages` for such a native (`build == host == target`) package set, we set their `targetPackages`
@@ -206,6 +203,6 @@ Second, it is because this is a frequent source of hard-to-follow "infinite recu
Thirdly, it is because everything target-mentioning only exists to accommodate compilers with lousy build systems that insist on the compiler itself and standard library being built together. Of course that is bad because bigger derivations means longer rebuilds. It is also problematic because it tends to make the standard libraries less like other libraries than they could be, complicating code and build systems alike. Because of the other problems, and because of these innate disadvantages, compilers ought to be packaged another way where possible.
:::
::: {.note}
If one explores Nixpkgs, they will see derivations with names like `gccCross`. Such `*Cross` derivations is a holdover from before we properly distinguished between the host and target platforms—the derivation with “Cross” in the name covered the `build = host != target` case, while the other covered the `host = target`, with build platform the same or not based on whether one was using its `.nativeDrv` or `.crossDrv`. This ugliness will disappear soon.
::: note
If one explores Nixpkgs, they will see derivations with names like `gccCross`. Such `*Cross` derivations is a holdover from before we properly distinguished between the host and target platforms—the derivation with "Cross" in the name covered the `build = host != target` case, while the other covered the `host = target`, with build platform the same or not based on whether one was using its `.nativeDrv` or `.crossDrv`. This ugliness will disappear soon.
@@ -114,10 +114,6 @@ For details, see [Licenses](#sec-meta-license).
A list of the maintainers of this Nix expression. Maintainers are defined in [`nixpkgs/maintainers/maintainer-list.nix`](https://github.com/NixOS/nixpkgs/blob/master/maintainers/maintainer-list.nix). There is no restriction to becoming a maintainer, just add yourself to that list in a separate commit titled “maintainers: add alice”, and reference maintainers with `maintainers = with lib.maintainers; [ alice bob ]`.
### `mainProgram` {#var-meta-mainProgram}
The name of the main binary for the package. This effects the binary `nix run` executes and falls back to the name of the package. Example: `"rg"`
### `priority` {#var-meta-priority}
The *priority* of the package, used by `nix-env` to resolve file name conflicts between packages. See the Nix manual page for `nix-env` for details. Example: `"10"` (a low-priority package).
@@ -134,32 +130,11 @@ Attribute Set `lib.platforms` defines [various common lists](https://github.com/
### `tests` {#var-meta-tests}
::: {.warning}
::: warning
This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations.
:::
An attribute set with tests as values. A test is a derivation that builds when the test passes and fails to build otherwise.
You can run these tests with:
```ShellSession
$ cd path/to/nixpkgs
$ nix-build -A your-package.tests
```
#### Package tests
Tests that are part of the source package are often executed in the `installCheckPhase`.
Prefer `passthru.tests` for tests that are introduced in nixpkgs because:
*`passthru.tests` tests the 'real' package, independently from the environment in which it was built
* we can run `passthru.tests` independently
*`installCheckPhase` adds overhead to each build
For more on how to write and run package tests, see <xref linkend="sec-package-tests"/>.
#### NixOS tests
An attribute set with as values tests. A test is a derivation, which builds successfully when the test passes, and fails to build otherwise. A derivation that is a test needs to have `meta.timeout` defined.
The NixOS tests are available as `nixosTests` in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
@@ -173,8 +148,6 @@ The NixOS tests are available as `nixosTests` in parameters of derivations. For
}
```
NixOS tests run in a VM, so they are slower than regular package tests. For more information see [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
### `timeout` {#var-meta-timeout}
A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, it can fail due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in `nixpkgs`.
@@ -202,20 +175,20 @@ The `meta.license` attribute should preferably contain a value from `lib.license
Although it’s typically better to indicate the specific license, a few generic options are available:
Unfree package that can be redistributed in binary form. That is, it’s legal to redistribute the *output* of the derivation. This means that the package can be included in the Nixpkgs channel.
Sometimes proprietary software can only be redistributed unmodified. Make sure the builder doesn’t actually modify the original binaries; otherwise we’re breaking the license. For instance, the NVIDIA X11 drivers can be redistributed unmodified, but our builder applies `patchelf` to make them work. Thus, its license is `"unfree"` and it cannot be included in the Nixpkgs channel.
Unfree package that cannot be redistributed. You can build it yourself, but you cannot redistribute the output of the derivation. Thus it cannot be included in the Nixpkgs channel.
This package supplies unfree, redistributable firmware. This is a separate value from `unfree-redistributable` because not everybody cares whether firmware is free.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
