Build was failing because we were depending on tagged versions of
the deblobbing scripts. The tags are not updated and thus newer
changes required won't be reflected unless the tag is re-created, which
might not be reliably the case.
So bumping revision and switching to use the branches to access the
deblob scripts.
For context, in our case the missing change is:
--- /nix/store/sfc0rrhj5l44zpqgpsymq5750k5wzg8p-tags-r16790/4.19-gnu/deblob-4.19 1970-01-01 01:00:01.000000000 +0100
+++ ../deblob-4.19 2019-09-14 14:53:44.637404289 +0200
@@ -1879,7 +1879,11 @@
announce BRCMFMAC - "Broadcom IEEE802.11n embedded FullMAC WLAN driver"
reject_firmware drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.c
-reject_firmware drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c
+if grep -q firmware_request_nowarn drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c; then
+ reject_firmware_nowarn drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c
+else
+ reject_firmware drivers/net/wireless/broadcom/brcm80211/brcmfmac/common.c
+fi
clean_blob drivers/net/wireless/broadcom/brcm80211/brcmfmac/feature.c
clean_blob drivers/net/wireless/broadcom/brcm80211/brcmfmac/firmware.h
(cherry picked from commit 2a8f7d71ce)
This change is needed because the order of profiles correlate to the
order in PATH, therefore "/etc/profiles/per-user/$USER" always appeared
after the system packages directories.
(cherry picked from commit 33c834f2fb)
GDK_PIXBUF_MODULE_FILE is often set to a nix store path not available in a app
sandbox. This can cause some apps to fail launching, simply reset this env var
when running applications.
fixes https://github.com/NixOS/nixpkgs/issues/53441
(cherry picked from commit 752e176d67)
(cherry picked from commit be89fd6a03)
Only includes a small fix for the login into the Robot Web interface,
which is used to eg. provide access to admin accounts (which in turn is
used by the NixOps Hetzner backend).
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 56009d4a8d)
this should fix CVE-2018-3836, CVE-2018-7186, CVE-2018-7247 & CVE-2018-7440
a fix for CVE-2018-7441 & CVE-2018-7442 is *not* included as its patches
are very wide-ranging
also enable tests to give confidence that I'm not completely butchering
the code
(cherry-picked from commit 0861ad591a)
texlive attribute was accidentally added in attrset wrapped with
stdenv.lib.optionalAttrs (!stdenv.isDarwin)
Fixes: dbc2c1c4b8 ('texlive: add missing perl dependencies for latexindent')
(cherry picked from commit b6d0bdf513)
darwin-no-system-python.patch does not apply cleany on Boost 1.55's
sources. Fix this patch file for Boost 1.55, making it build
successfully on macOS.
(cherry picked from commit 153e7fca32)
trace: warning: config.services.gitea.database.password will be stored as plaintext
in the Nix store. Use database.passwordFile instead.
(Arguably, this shouldn't be a warning at all. But making it happy is
easier than having a debate on the value of this warning.)
(cherry picked from commit c0829a0859)
QtWebEngine's build system is setup to perform certain platform checks
(see mkspecs/features/platform.prf). But a failed check will not cause
configuration phase to fail, but instead it configures no build targets.
So in such case the build will successfully perform build and install
phases. An empty output directories will are produced and the build
succeeds.
This patches qtwebengine qmake files to properly fail during
configuration phase.
This doesn't touch qt56 as it doesn't have this mechanism.
(cherry picked from commit 3e3e3918f2)
This ports a patch that is already used in 5.11. Also one of the
substituteInPlace call fails because a file doesn't exist.
(cherry picked from commit 0624deb5df)
The previous code using substituteInPlace was broken for both 5.9 and
5.11 (latter was broken after a minor release bump). Using patches
should make things fail loudly and prevent modules/qtwebengine.nix from
being littered with version checks.
(cherry picked from commit 1809832958)
When I backported https://github.com/NixOS/nixpkgs/pull/58639 to 18.09,
I didn't realize that buildGoPackage still required the `name`
attribute. Thanks to @primeos for notifying me of this.
See https://github.com/browserpass/browserpass-native/issues/31
Additionally browserpass was removed from systemPackages, because it
doesn't need to be installed, browsers will get the path to the binary
from the native messaging host JSON.
(cherry picked from commit e98ee8d70c)
This is tagged as version 9.26a in the ghostpdl repo, but unfortunately
there are no tarballs released with that version number so far. We'll
continue calling this version 9.26 for now for simplicity's sake (and we
can switch to 9.26a and remove the patch when it's properly released).
Fixes#58262Fixes#58089
(cherry picked from commit 91c46d17d5)
No security problems have been published about 3.6.x so far,
but I'd certainly count the almost-transparent TLS 1.3 support
as a security improvement.
(cherry picked from commit 2ff530ec53)
This caused some reverse dependencies of adoptopenjdk to depend on
adoptopenjdk's libfreetype, rather than the NixOS
libfreetype. For example: https://github.com/NixOS/nixpkgs/issues/57733
Now the derivation does not contain libfreetype.so . The JRE links to
nixpkg's freetype:
$ ldd /nix/store/9iyxm1nkn35xhjgri041r980z4p5ls5g-adoptopenjdk-hotspot-bin-11.0.2/lib/libfontmanager.so | grep -i freetype
libfreetype.so => /nix/store/ycbkq39cngzx19j829qzgdnw3mx3z78g-freetype-2.9.1/lib/libfreetype.so (0x0000795e30702000)
(cherry picked from commit 6bae07337e)
seems that this got broken when the config option was made to use enums. "secure" got replaced with "enum", which isn't a valid option for the failure mode.
(cherry picked from commit 0c34b9fcf8)
This is a requirement for Firefox66. To retain compatibility with users
of rust-cbindgen I leave the others around since it usually breaks some
kind of compatibility.
It is already listed in aliases.nix, and I managed to get this from Nix:
error: Alias deadpixi-sam is still in all-packages.nix
Fixes: 218d81bc9d ("all-packages: move aliases to aliases.nix")
(cherry picked from commit 38ae8aed16)
nixpkgs.config.virtualbox.enableExtensionPack doesn't do anything, but
used to. Add a warning for the unsuspecting.
(cherry picked from commit 8b6a38ce7e)
From http://www.ntp.org/index.html:
> ntp-4.2.8p13 was released on 07 March 2019.
> It addresses 1 medium-severity security issue in ntpd, and provides 17
> other non-security fixes and 1 improvements over 4.2.8p12.
(cherry picked from commit d7c23cc65f)
On very large graphs (14k+ paths), we'd end up with a massive in
memory tree of mostly duplication.
We can safely cache trees and point back to them later, saving
memory.
(cherry picked from commit 09362bc3e8)
If our old Nix can’t evaluate the Nixpkgs channel, try the fallback
from the new channel /first/. That way we can upgrade Nix to a newer
version and support breaking changes to Nix (like seen in the upgrade
o Nix 2.0).
This change should be backported to older NixOS versions!
(cherry picked from commit 475c8aa018)
fetchFromGitHub and thus fetchzip hashes the contents of the archive and
not the archive itself. Unicode file names lead to different checksums
on HFS+ vs. other file systems because of Unicode normalisation
(cherry picked from commit f466c9f961)
buildbot was always broken on release-18.09 due to failing tests
One of the failures is:
[ERROR]
Traceback (most recent call last):
File "/build/buildbot-1.2.0/buildbot/process/properties.py", line 459, in getRenderingFor
rv = yield build.render(value[index])
File "/nix/store/sqr3s9cva7r3z12hqb6rxw3w8kiqzmhd-python2.7-Twisted-18.7.0/lib/python2.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
result = g.send(result)
File "/build/buildbot-1.2.0/buildbot/process/properties.py", line 495, in getRenderingFor
raise KeyError(error_message)
exceptions.KeyError: "secrets service not started, need to configure SecretManager in c['services'] to use 'secrets'in Interpolate"
buildbot.test.unit.test_master.StartupAndReconfig.test_reconfigService_db_url_changed
There is a mention of this at https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=907687
Because buildbot was never working before, we can just bump the version
to the one which passes the test.
(cherry picked from commit 71c42462ab)
The unfree variant of elasticsearch uses autoPatchelfHook and since we
removed the dependency on file for the hook itself in
58a97dfb49 we no longer have zlib
propagated.
So we need to explicitly state that dependency here.
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @apeschar, @basvandijk
(cherry picked from commit 8df68a93e6)
The autojump plugin in oh-my-zsh assumes autojump.zsh resides in
/run/current-system/sw/share/autojump/ but these links are not created
by default.
The new programs.autojump.enable option forces the creation of these
links.
(cherry picked from commit 0d749e58f7)
Signed-off-by: Domen Kožar <domen@dev.si>
Unfortunately we don’t have access to NSWindowStyleMask. These patches
should go away once we switch to a newer SDK.
(cherry picked from commit 81531046e8)
This reverts commit b40b39b1bc.
tarball job failed:
error: while evaluating anonymous function at /build/source/maintainers/scripts/find-tarballs.nix:6:1, called from undefined position:
while evaluating 'operator' at /build/source/maintainers/scripts/find-tarballs.nix:27:16, called from undefined position:
while evaluating 'immediateDependenciesOf' at /build/source/maintainers/scripts/find-tarballs.nix:39:29, called from /build/source/maintainers/scripts/find-tarballs.nix:27:44:
while evaluating anonymous function at /build/source/lib/attrsets.nix:225:10, called from undefined position:
while evaluating anonymous function at /build/source/maintainers/scripts/find-tarballs.nix:40:37, called from /build/source/lib/attrsets.nix:225:16:
while evaluating 'derivationsIn' at /build/source/maintainers/scripts/find-tarballs.nix:42:19, called from /build/source/maintainers/scripts/find-tarballs.nix:40:40:
while evaluating 'optional' at /build/source/lib/lists.nix:241:20, called from /build/source/maintainers/scripts/find-tarballs.nix:44:33:
while evaluating 'canEval' at /build/source/maintainers/scripts/find-tarballs.nix:48:13, called from /build/source/maintainers/scripts/find-tarballs.nix:44:43:
while evaluating the attribute 'drvPath' at /build/source/lib/customisation.nix:149:7:
while evaluating the derivation attribute 'name' at /build/source/pkgs/stdenv/generic/make-derivation.nix:177:11:
while evaluating 'nullSrc' at /build/source/pkgs/development/haskell-modules/make-package-set.nix:263:19, called from undefined position:
while evaluating 'overrideCabal' at /build/source/pkgs/development/haskell-modules/lib.nix:37:24, called from /build/source/pkgs/development/haskell-modules/make-package-set.nix:263:22:
attribute 'override' missing, at /build/source/pkgs/development/haskell-modules/lib.nix:37:28
Remove the last use of the "placeholder" primop on release-18.09 branch.
Fixes: a57d03fcc3 ("teeworlds: 0.6.4 -> 0.6.5"), which was backported
from master with the "placeholder" primop.
If the file in question is not a shared object file but an ELF, we
really want to skip the file, because we won't have anything to patch
there.
For example if the file is created via "gcc -c -o foo.o foo.c", we don't
get a segment header and so far autoPatchelf was trying to patch such a
file.
By checking for missing segment headers, we're now no longer going to
attempt patching such a file.
Signed-off-by: aszlig <aszlig@nix.build>
Reported-by: Sander van der Burg <svanderburg@gmail.com>
(cherry picked from commit 4a6e3e4185)
While declaring it as an array doesn't do any harm in our usage, it
might be a bit confusing when reading the code.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 9f23a63f79)
This function is useful if autoPatchelf is invoked during some of the
phases of a build and allows to add arbitrary shared objects to the
search path.
So far the same functionality was in autoPatchelf itself, but not
available as a separate function, so when adding shared objects to the
dependency cache one would have to do so manually.
The function also has the --no-recurse flag, which prevents recursing
into subdirectories.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 2faf905f98)
This is to be used with the autoPatchelf command and allows to only
patch a specific file or directory without recursing into
subdirectories.
Apart from being able to run the command in a standalone way, as
detailled in the previous commit this is also needed for the Android SDK
emulator, because according to @svanderburg there are subdirectories we
don't want to patch.
The reason why I didn't use GNU getopt is that it might not be available
on all operating systems and the getopts bash builtin doesn't support
long arguments. Apart from that, the implementation for recognizing the
flag is pretty trivial and it's also using bash builtins only, so if we
want to do something really fancy someday, we can still change it.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 3ca35ce0b2)
If you want to only run autoPatchelf on a specific path and leave
everything else alone, we now have a $dontAutoPatchelf environment
variable, which causes the postFixup hook to not run at all.
The name "dontAutoPatchelf" probably is a bit weird in conjunction with
putting "autoPatchelfHook" in nativeBuildInputs, but unless someone
comes up with a better name I keep it that way because it's consistent
with all the other dontStrip, dontPatchShebangs, dontPatchELF and
whatnot.
A specific example where this is needed is when building the Android SDK
emulator, which contains a few ARM binaries in subdirectories that
should not be patched. If we were to run autoPatchelf on all outputs
unconditionally we'd run into errors because some ARM libraries couldn't
be found.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit e4fbb244ee)
The autoPatchelf main function which is run against all of the outputs
was pretty much tailored towards this specific setup-hook and was
relying on $prefix to be set globally.
So if you wanted to run autoPatchelf manually - let's say during
buildPhase - you would have needed to run it like this:
prefix=/some/directory autoPatchelf
This is now more intuitive and all you need to do is run the following:
autoPatchelf /some/directory
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit d03e4ffdbf)
I originally thought it would just be enough to just check for an INTERP
section in isExecutable, however this would mean that we don't detect
statically linked ELF files, which would break our recent improvement to
gracefully handle those.
In theory, we are only interested in ELF files that have an INTERP
section, so checking for INTERP would be enough. Unfortunately the
isExecutable function is already used outside of autoPatchelfHook, so we
can't easily get rid of it now, so let's actually strive for more
correctness and make isExecutable actually match ELF files that are
executable.
So what we're doing instead now is to check whether either the ELF type
is EXEC *or* we have an INTERP section and if one of them is true we
should have an ELF executable, even if it's statically linked.
Along the way I also set LANG=C for the invocations of readelf, just to
be sure we don't get locale-dependent output.
Tested this with the following command (which contains almost[1] all the
packages using autoPatchelfHook), checking whether we run into any
library-related errors:
nix-build -E 'with import ./. { config.allowUnfree = true; };
runCommand "test-executables" {
drvs = [
anydesk cups-kyodialog3 elasticsearch franz gurobi
masterpdfeditor oracle-instantclient powershell reaper
sourcetrail teamviewer unixODBCDrivers.msodbcsql17 virtlyst
vk-messenger wavebox zoom-us
];
} ("for i in $drvs; do for b in $i/bin/*; do " +
"[ -x \"$b\" ] && timeout 10 \"$b\" || :; done; done")
'
Apart from testing against library-related errors I also compared the
resulting store paths against the ones prior to this commit. Only
anydesk and virtlyst had the same as they didn't have self-references,
everything else differed only because of self-references, except
elasticsearch, which had the following PIE binaries:
* modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autoconfig
* modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/autodetect
* modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/categorize
* modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/controller
* modules/x-pack/x-pack-ml/platform/linux-x86_64/bin/normalize
These binaries were now patched, which is what this commit is all about.
[1]: I didn't include the "maxx" package (MaXX Interactive Desktop)
because the upstream URLs are no longer existing and I couldn't
find them elsewhere on the web.
Signed-off-by: aszlig <aszlig@nix.build>
Fixes: https://github.com/NixOS/nixpkgs/issues/48330
Cc: @gnidorah (for MaXX Interactive Desktop)
(cherry picked from commit c64624b843)
The "maxx" package recursively runs isExecutable on a bunch of files and
since the change to use "readelf" instead of "file" a lot of errors like
this one are printed during build:
readelf: Error: Not an ELF file - it has the wrong magic bytes at the
start
While the isExecutable was never meant to be used outside of the
autoPatchelfHook, it's still a good idea to silence the errors because
whenever readelf fails, it clearly indicates that the file in question
is not a valid ELF file.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit b4526040a2)
If the ELF file is not an executable, we do not get a PT_INTERP section,
because after all, it's a *shared* library.
So instead of checking for PT_INTERP (to avoid statically linked
executables) for all ELF files, we add another check to see if it's an
executable and *only* skip it when it is and there's no PT_INTERP.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 9920215d00)
The database name needs to be quoted
in case it contains special characters
so the MySQL service does not fail to start.
(cherry picked from commit 429c0bf60c)
This adds the "missing" qtvirtualkeyboard module of qt56. I just add
this so I can apply (& test) the patches for a CVE in the next commit.
This might seem strange but in case anyone decided to add / use this in
the future we are on the safe(r) side.
(cherry picked from commit 295a210a23)
There is a bug when using git repo in that the package does not get a
version (shows up as `0+unknown`). Using pypi fixes this issue allows
allows for auto upgrades.
(cherry picked from commit e5cd420e73)
The github repository was downloaded instead of the pypi repository
for testing (needed `conftest.py`). Major work was done on the
underlying dependencies to make distributed work on python 2.7,
3+. Note that the test **do** take a significant amount of time (10-15
minutes).
- moved to `python-modules`
- compatible with 2.7, 3+
- all tests pass (previously tests were not run)
(cherry picked from commit 4f70170f8a)
A new dependency (at-spi2-atk) is required for libatk-bridge-2.0.so.
Also libstdc++.so.6 is now required from stdenv.cc.cc.lib (stdenv.cc.cc
doesn't offer it). The binary will only find libstdc++.so.6 via
$LD_LIBRARY_PATH (i.e. not via rpath, the run-time search path).
(cherry picked from commit e01209f2c7)
Since CVE-2018-14574 and CVE-2019-3498 affect 1.11, it is very likely
they also apply to 1.8. However, Django 1.8 has reached EOL in April
2018 and the patches were not backported.
(cherry picked from commit c6e08579c5)
There have been some more changes to the source tree which broke the
buildconfig patch. This commit adds another patch that can be used for
the future versions. Once all the flavors are based off a new(ish)
firefox release we can remove the old patch.
https://groups.google.com/forum/#!msg/golang-announce/mVeX35iXuSw/Flp8FX7QEAAJ
We have just released Go 1.11.5 and Go 1.10.8 to address a recently reported security issue. We recommend that all users update to one of these releases (if you’re not sure which, choose Go 1.11.5).
This DoS vulnerability in the crypto/elliptic implementations of the P-521 and P-384 elliptic curves may let an attacker craft inputs that consume excessive amounts of CPU.
These inputs might be delivered via TLS handshakes, X.509 certificates, JWT tokens, ECDH shares or ECDSA signatures. In some cases, if an ECDH private key is reused more than once, the attack can also lead to key recovery.
The issue is CVE-2019-6486 and Go issue golang.org/issue/29903. See the Go issue for more details.
(cherry picked from commit d803da845f)
This reverts commit 5574df3549.
I also can't reproduce the problem anymore; discussion: #41312.
Fixes#53569, fixes#53948. (Vulnerabilities in old curl.)
(cherry picked from commit 51ac3db79c)
I've noticed no complaints about this change on unstable/master.
Development releases of ncurses are uploaded but often also removed
again. This is causing a lot of trouble, so let's instead use a
GitHub mirror.
(cherry picked from commit 43e867a226)
Firefox >=65 will depend on icu >=63. All the older firefox versions
(and derived packages) seem to work fine with this change.
Also the system path environment patch will fail to apply since there
was a trivial whitespace change in the source file. By adding `-l` to
patch we can avoid having to track two patches that do basically the
same. Having patchFlags per file without resorting to pre-/postPatch
would be nicer but there doesn't seem to be a facility for that right
now.
(cherry picked from commit e37b765e7897c015153552d4e33a8571ab9c5457)
jemalloc with stripped prefix would cause segfaults in free:
https://github.com/NixOS/nixpkgs/pull/49557#issuecomment-436734677
This commit has been adjusted from the one on master to only carry the
relevant changes (the new optional `stripPrefix`) flag while still
keeping the other flags that were removed on master since they are no
loner supported by the version used there.
(cherry picked from commit 973eca740b)
This reverts commit d6e3db44cf.
See #53935 for explanations. In short, it may be causing issues with
tests on the build infrastructure.
(cherry picked from commit 3aab228d09)
discord now depends on gtk3 instead of gtk2, unfortunately
there is no public changelog, so other changes are unknown
(cherry picked from commit 4e4a9babd7)
packaging the `ktouch` touch typing tutor. Due to Qt impurities, it
needs to be installed in a profile to find it runtime dependencies.
(cherry picked from commit e93bd1d445)
This package contains several CMake files used for setting up its
provided tools for use in other projects build with CMake.
While packaging *ktouch* I found out that the ${_qt5Core_install_prefix}
variable doesn't expand at all, rendering the path to the `qmlcachegen`
binary useless. As a fix, the command itself is used instead of the path
to the binary.
(cherry picked from commit 4b76c4605e)
See #49441 for an earlier attempt, which was subsequently reverted. I am
assuming that doubling the time will be sufficient if the machine is
overloaded since so many of the tests already pass at 5 minutes, while
still not holding back failures for needlessly long.
(cherry picked from commit b28b37eb00)
https://lists.torproject.org/pipermail/tor-announce/2019-January/000171.html
FWIW, in the ChangeLog (in the source, sorry) it mentions:
As a reminder, the Tor 0.3.4 series will be supported until 10 June
2019. Some time between now and then, users should switch to the Tor
0.3.5 series, which will receive long-term support until at least 1
Feb 2022.
So we should consider moving to 0.3.5 "soon" :).
(cherry picked from commit 91859c0504)
Long live... eh, I hope the new KSK won't live as long as the old one.
Anyway, it doesn't really matter how fast people update this.
https://www.ietf.org/mail-archive/web/dnsop/current/msg24989.html
See RFC 5011 for details of the protocol.
I re-tested validation with both of these files, to be sure.
(cherry picked from commit c4a5565e7a)
electron 4 had many breaking changes in their API,
breaking rambox. Since the 3.x version is still maintained,
we can add an older variant.
(cherry picked from commit 66c1f82631)
The idea is that we only need this target during boot,
however there is no point on restarting it on every upgrade.
This hopefully fixes#21954
(cherry picked from commit 0a2c8cc1db)
Symlinking works for most plugins and themes, but Avada, for instance, fails to
understand the symlink, causing its file path stripping to fail. This results in
requests that look like:
https://example.com/wp-content//nix/store/...plugin/path/some-file.js
Since hard linking directories is not allowed, copying is the next best thing.
(cherry picked from commit 9d2c9157d7)
This adds a NixOS option for setting the CPU max and min frequencies
with `cpufreq`. The two options that have been added are:
- `powerManagement.cpufreq.max`
- `powerManagement.cpufreq.min`
(cherry picked from commit b0f10d2d53)
(cherry picked from commit 46ecec8239)
Dropbox version 55.4.171 is too old which results in an error when doing
the initial account linking.
(cherry picked from commit d5dbfad91a)
Reason: To have a usable dropbox package for the next release.
The bump includes all the patches we had and adds some more,
e.g. for CVE-2017-1000127. I don't think all CVEs get fixed by that,
but 0.27 would currently break too many packages; /cc #50496.
(cherry picked from commit bf7d1b538d)
Imports the `journaldriver` module into the top-level NixOS module
list to make it usable without extra work.
This went unnoticed in #42134 (mostly because my setup imports modules
explicitly from pinned versions).
Fixes#50390
(cherry picked from commit d5ea09758d)
Pdfjs was downloaded in two derivations, where one is sufficient.
Also there was allegedly a typo in the download URL.
(cherry picked from commit 0be7bfe1bf)
Currently there are two calls to curl in the reloadScript, neither which
check for errors. If something is misconfigured (like wrong authToken),
the only trace that something wrong happened is this log message:
Asking Jenkins to reload config
<h1>Bad Message 400</h1><pre>reason: Illegal character VCHAR='<'</pre>
The service isn't marked as failed, so it's easy to miss.
Fix it by passing --fail to curl.
While at it:
* Add $curl_opts and $jenkins_url variables to keep the curl command
lines DRY.
* Add --show-error to curl to show short error message explanation when
things go wrong (like HTTP 401 error).
* Lower-case the $CRUMB variable as upper case is for exported environment
variables.
The new behaviour, when having wrong accessToken:
Asking Jenkins to reload config
curl: (22) The requested URL returned error: 401
And the service is clearly marked as failed in `systemctl --failed`.
(cherry picked from commit 8ebfd5c45c)
Manual still refers to 'programs.ohMyZsh' although it should be 'programs.zsh.ohMyZsh'.
(cherry picked from commit 685c4f5608)
Signed-off-by: Maximilian Bosch <maximilian@mbosch.me>
Part of #49783. NextCloud tracks in its `config.php` the application's
state which makes it hard for the module to modify configurations during
upgrades.
It will take time until the issue is properly fixed, therefore we
decided to warn about this in the manual.
This PR addresses two things:
* Adding a basic example for nextcloud. I figured it to be helpful to
add some basic usage instructions when adding a new manual entry.
Advanced documentation may follow later.
For now this document actively links to the service options, so users
are guided to the remaining options that can be helpful in certain
cases.
* Add a warning about upgrades and manual changes in
`/var/lib/nextcloud`. This will be fixed in the future, but it's
definetely helpful to document the current issues in the manual (as
proposed in https://github.com/NixOS/nixpkgs/issues/49783#issuecomment-439691127).
(cherry picked from commit 216a954540)
Updates to the latest version of the desktop client available. Tested
the config migration from `nextcloud-client` 2.3.3 with a Nextcloud
14.0.3 instance (hosted using `services.nextcloud`).
Additionally the derivation required the following changes:
* Dropped `Qt5Sql` patch: this has been fixed upstream and isn't needed
anymore (furthermore their CMake structure has changed and the patch
wouldn't apply anymore on 2.5.0).
* Moved to a new upstream repository (nextcloud/desktop), kept
`fetchgit` to properly fetch submodules.
* Added OpenSSL 1.1 integration: `libsync` (the syncing provided by this
package) requires 1.1, furthermore the linking flags had to be fixed
manually by passing `NIX_LDFLAGS` to the derivation.
Furthermore I moved the support for a Gnome3 keyring into its own
wrapper to avoid a full rebuild of the package whenever you alter
`withGnomeKeyring` in an override expressions.
It's still possible to enable keyring (now without recompile) like this:
```
nextcloud-client.override { withGnomeKeyring = true; }
```
To override the derivation itself you now have to use
`nextcloud-client-unwrapped`:
```
nextcloud-client-unwrapped.overrideAttrs (old: {
src = yoursrc;
})
```
(cherry picked from commit e75f922e91)
after feedback from @veprbl in #51805 I added this as another package
which can be used for the 18.09 firefox builds.
(cherry picked from commit f1ee26bc4b)
Motivation: unstable-2018-02-24 (which reports to be
0.19.1, but probably doesn't correspond to any actual
geckodriver release) isn't compatible with the currently
shipped Firefox version 63.0
closesNixOS/nixpkgs#50380
They consistently fail since openjdk bump with some out-of-space errors.
That's not a problem by itself, but each test instance ties a build slot
for many hours and consequently they also delay channels as those wait
for all builds to finish.
Feel free to re-enable when fixed, of course.
(cherry picked from commit 3946d83a3c)
- use nix build instead of nix-build
- writes per-build log in the current working directory
- symlinks the builds in the current working directory
- detects & deduplicates build aliases
- markdown reports
- filter builds by regex
- generate nix expression files that can be build by the user
(cherry picked from commit 53f1ffa4d3)
This is to try and squeeze more lost space from the image, so that hydra
starts building it again.
The fsck previous to the resize2fs is required so resize2fs works.
The one afterwards is a sanity check.
Using `-M` from resize2fs will not give much saved space due to a known
(in the manual) issue.
```
[samueldr@aarch64:~/nixpkgs]$ ls -lh result-*/*/*.img
-r--r--r-- 1 root root 2.2G Jan 1 1970 result-original/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 2.1G Jan 1 1970 result-M/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
-r--r--r-- 1 root root 1.9G Jan 1 1970 result-slimmed/sd-image/nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img
```
```
[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-original
/nix/store/c8k9n78gylx293rjh762fr05a069kxp2-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img 3844125000
[samueldr@aarch64:~/nixpkgs]$ nix path-info -S ./result-slimmed
/nix/store/962238skj5mnzhrsmjy23dyzmxk77sp4-nixos-sd-image-18.09.git.a7fd431-aarch64-linux.img 3447473208
```
(cherry picked from commit 61bdaad9a9)
Notable changes:
- Morocco switched to permanent +01 on 2018-10-27
- Volgograd moved from +03 to +04 on 2018-10-28
- Fiji ends DST 2019-01-13, not 2019-01-20
- Most of Chile changes DST dates, effective 2019-04-06
tzdata: fetch over https
(cherry picked from commit 7c25174154)
Instead of setting User/Group only when DynamicUser is disabled, the
previous version of the code set it only when it was enabled. This
caused services with DynamicUser enabled to actually run as nobody, and
services without DynamicUser enabled to run as root.
Regression from fbb7e0c82f.
(cherry picked from commit 3873f43fc3)
GitLab 11.5.1 dropped the dependency to posix_spawn, which is broken on
32bit. (See https://gitlab.com/gitlab-org/gitlab-ce/issues/53525)
The only part missing is decreasing virtualisation.memorySize to
something that a 32 bit qemu still executes.
The maximum seems to be 2047, and tests passed with that value for me.
(cherry picked from commit 5c82aa8854)
This includes fixes for CVE-2018-10754.
While we're changing things, also set the `--with-manpage-format=normal`
configure flag, which prevents the `configure` script from looking in
/usr to determine whether to compress manpages. This was already the
format on NixOS (where these directories don't exist), but making this
explicit makes the build more reproducible on other distros.
(cherry picked from commit f485b2e71f)
The patch is not applying and no longer needed after upgrade in
045575e744. Now the same result is
achieved by the following lines in the driver package itself:
#if defined(NV_DRM_LEGACY_PCI_INIT_PRESENT)
#define nv_drm_pci_init drm_legacy_pci_init
#define nv_drm_pci_exit drm_legacy_pci_exit
#else
#define nv_drm_pci_init drm_pci_init
#define nv_drm_pci_exit drm_pci_exit
#endif
Commit df6d40feb6 renamed colemak/en-latin9 to
colemak/colemak, but the keymap test wasn't adjusted to refer to the new path.
(cherry picked from commit 74598ca8d3)
On x86_64-linux this looks good, only R fails tests on Hydra,
but I've been unable to reproduce that failure locally,
so let me not delay the CVE fixes.
Undefined symbols for architecture x86_64:
"_OBJC_CLASS_$_NSData", referenced from:
objc-class-ref in _x002.o
"_OBJC_CLASS_$_NSDictionary", referenced from:
objc-class-ref in _x002.o
"_OBJC_CLASS_$_NSURL", referenced from:
objc-class-ref in _x002.o
ld: symbol(s) not found for architecture x86_64
(cherry picked from commit be9fc0d669)
- Use `autoPatchelfHook`
- Don't explicitly set phases - Part of #28910
- Correct `version` by hoisting out the suffix
(cherry picked from commit 02e9d06e5e)
The new reuse behaviour is cool and really useful but it breaks one of
my use cases. When using kexec, I have a script which will unlock the
disks in my initrd. However, do_open_passphrase will fail if the disk is
already unlocked.
(cherry picked from commit 9cd4ce98bf)
Reason: The new behaviour breaks my existing setup and I'd like to have
it fixed before March.
Current build of nixos-18.09-small (nixos-18.09.1473.c1acb5ce6d8) fails
with
error: undefined variable 'postgresqlPackage' at […]/nixos/modules/services/databases/postgresql.nix:251:38
The right variable name should be cfg.package; fix.
ZFS's popularity is growing, and not including it by default is a
bit frustrating. On top of that, the base iso includes ZFS
_anyway_ due to other packages depending upon it.
I think we're in the clear to do this on the basis that Oracle
probably doesn't care, it is probably fine (the SFLC agrees) and
we're a small fish. If a copyright holder asks us to, we can
definitely revert it again.
This reverts commit 33d07c7ea9.
(cherry picked from commit 6db866cbd2)
```
b Batch mode. Optimized for huge recursive copies, but less secure if a crash happens during the copy.
```
It seems the "less secure if a crash happens" does not need a crash to
happen.
With batch mode:
```
/[...]/.
Start (0) does not point to parent (___)
```
For pretty much everything copied in.
Without batch mode, everything passes `fsck`.
See #51150
(cherry picked from commit 1b6a4d3979)
This results in a marginal closure size increase of ~137mb -> ~141mb.
This only affects the 'out' output, not the 'lib' output, so
libpq-dependent expressions should not bloat as a result of this.
The type of the systemd service is changed to 'notify' on 9.6+, which
will help identify when Postgres is starting/shutting down.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
(cherry picked from commit 1e83cf3174)
Signed-off-by: Domen Kožar <domen@dev.si>
This includes a bunch of security fixes (#49786), and mimics what Debian
has done in moving to a git snapshot instead of a released version +
backported security patches.
(cherry picked from commit 3137c609a4)
Instead of pinning Darwin to older versions, add small patches to
configure.in (7.1) / configure.ac (7.2) to fix the build of the intl
extension on recent PHP versions on Darwin.
fix-paths-php7.patch also required changes -- since we now run autoconf
at build time (through ./buildconf), it needs to patch the input .m4
files instead of ./configure directly.
(cherry picked from commit ea10173045)
The nixos test is a bit misleading, as the given nginx configuration
would always cause gitlab to redirect to localhost, which is clearly not
what you want in a production setup.
Instead we now enable services.nginx.recommendedProxySettings,
curl against http://gitlab, and assure we get redirected to that same
hostname, too.
(cherry picked from commit 6870eafe72)
Upstream bug: curl/curl#3218.
This causes nixos/tests/ipv6.nix to fix since the last staging merge.
(cherry picked from commit 90720d0139)
In the end I decided to pick it directly without staging,
so that the various security fixes get to stable* channels fast.
(cherry picked from commit 30008a1cda)
Fixes#47932. On 18.09 we still have libxcb 1.12, but I expect the
change will help with mixing versions, and it seems safe anyway.
With this commit, we *can* swap python2 for python3 to run synapse using python3
instead.
The reason for not making the switch is that a number of CLI tools provided with
synapse do not yet work under py3 despite synapse running fine.
So this doesn't actually do anything on its own except to prepare for the
upcoming py3 switch.
(cherry picked from commit dc74c44e67)
AMD license agreement (currently unavailable at the given URL, but
included in tarball) disallows reverse-engineering, modification,
redistribution etc;
BSL licenses limit commercial production use.
(cherry picked from commit 8fb1e5e166)
This ensures the proper version is reported in the server status
information; otherwise it has a '-PRERELEASE' suffix.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
(cherry picked from commit 47b3086bdf)
Otherwise netdata will not find python modules.
To make sure netdata still pick up our setuid version of apps.plugin
we rename the original executable.
FoundationDB uses Python at build time for some code generation.
However, it also has the official python bindings inside the source code
too, and the code for the Python bindings has some of it auto-generated
at compile time.
This made building python packages unattractive: we want to use the
source code generated from the FoundationDB build, but we don't want to
rebuild it. Previously we would override the 'python' input to the
FoundationDB module, but this meant we would do a complete rebuild, as
it was a necessary build time dependency, even though the resulting
generated code itself would not change. Furthermore, FoundationDB
versions < 6.0 don't properly support Python 3 *for the build system*,
though the bindings supported it, so that caused build failures. But the
first effect is the worst: it meant building separate python2 and
python3 packages implied two complete rebuilds of a single FoundationDB
version. This meant rather than 3 FDB builds, we'd do 3*N where N = the
number of major Python versions we support.
Finally, because we did not use pip to generate a wheel that we install
with metadata recorded for the installation, the FoundationDB python
package couldn't be used as an input to other setup.py-based packages:
there would be no recorded metadata in the dist-info folder which would
say this is the foundationdb package. This greatly limits its utility.
To fix all this, we do a few things:
- Apply some patches to fix the build system with Python 3.x for
older FoundationDB versions. (This is nice if end-users have
overridden the global Python version for some reason.)
- Move python directly into nativeBuildInputs, so it is only a
build time dependency.
- Take the python source code from the ./bindings directory and
tar it up use later after the build is done, so we get to keep
the generated code. This is the new 'pythonsrc' output from the
build. This code doesn't change based on whether or not the input
or resulting package is using Python 2 or 3, it's totally
deterministic.
- The build system also patches up the python source code a little,
so it can be installed directly with setup.py (it needs a little
stuff that it normally expects the build system to do.)
- Rework the python package to a separate file that uses
buildPythonPackage directly. Because the source code is already
prepared, it needs almost nothing else. Furthermore, this kills
the override itself for the foundationdb package, meaning rebuilds
are no longer needed.
- This package is very simple and just uses foundationdb.pythonsrc
as its source input. It also ensures a link to libfdb_c.so can
be found by ctypes (using substituteInPlace)
- python-packages.nix now just uses callPackage directly.
The net effect of this is, most importantly, that python packages do not
imply a full rebuild of the server source code: building python2 and
python3 packages from a version of FoundationDB now does not need to
override the foundationdb python input, reducing the number of needless
builds. They instead just run setup.py with the given version as input.
The second biggest effect is that wheel metadata is recorded correctly,
meaning dependent-python-packages that want to use the FoundationDB
bindings e.g. from PyPi should now work fine with buildPythonPackage.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
(cherry picked from commit 6054dabc11)
The Z Garbage Collector is a concurrent, scalable, low latency garbage
collector designed to meet extremely-low-pause-time requirements for
small-to-multi-TB heap sizes.
ZGC can be enabled with the magical incantation:
$ java -XX:+UnlockExperimentalVMOptions -XX:+UseZGC ...
Currently, ZGC is only available for x86_64-linux (though a port for
aarch64-linux may become available at a future time.) There are also a
number of other features that currently aren't present, such as JVMCI
integration (meaning compiler tools like Graal which require JVMCI will
not work with ZGC enabled.)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
(cherry picked from commit 1629147423)
./Write: error while loading shared libraries: libGL.so.1: cannot open shared object file: No such file or directory
(cherry picked from commit 04b234e995)
Update includes patches for sec issues:
- CVE-2017-14166
- CVE-2017-14501
- CVE-2017-14502
- Upstream includes patch for libressl version check
fixes#49583
(cherry picked from commit 95fa7b6370)
The `.desktop` referenced a non existent script to run `QGroundControl`. This commits adds a `sed` command, which replaces the non existent script by the name of the `QGroundControl` binary in the relevant `.desktop` file.
(cherry picked from commit c26b42be74)
This reverts commit 3b2944dbcb, reversing
changes made to 496a23896f.
I wasn't aware we had a Hydra job for this. Looking at it now, it seems we
have a lot more failures, so I am reverting this merge.
This bumps lark-parser to a released version that doesn't loop forever
and consumes all memory on certain grammars.
erezsh/lark redirects now to lark-parser/lark, so also update homepage.
(cherry picked from commit becf8879f4)
Also reworked dependencies:
* blist and ujson are marked as no longer needed
* pytz has no mention throughout `git log -p` on synapse's repository
* systemd and affinity are optional (but turned on by default)
(cherry picked from commit bb86056575)
There could be old *.rb files in the config directory that import
non-existant gems that would cause errors in the rake task.
(cherry picked from commit eadb998581)
SSRF in Kubernetes integration
The GitLab Kubernetes integration was vulnerable to a SSRF issue which could allow an attacker to make requests to access any internal URLs. The issue is now mitigated in the latest release and is assigned CVE-2018-18843.
(cherry picked from commit 9e491f732a)
On Hetzner machine with 1 core: 4h40m
On Hetzner machine with 8 cores: 1h20m
(cherry picked from commit 89ede978a0)
Signed-off-by: Domen Kožar <domen@dev.si>
As reported by @andir, the regular expressions that match the sandbox
output are no longer matching in the recent Chromium bump as of
bb03fbc2c8.
Instead of a boolean field that determines whether namespace sandboxes
are on, the namespace sandbox is now an enum within "Layer 1 Sandbox".
I've modified the regular expressions accordingly and also ran the test
for the stable branch, which now succeeds.
Signed-off-by: aszlig <aszlig@nix.build>
Issue: https://github.com/NixOS/nixpkgs/issues/49442
Cc: @bendlas, @andir
(cherry picked from commit 73cdd5a476)
Clean up the expression and add pkgconfig support
(cherry picked from commit 9ac64e5bb6)
Reason: Pre-Requisite to backport uriparser 0.9.0, which patches
security issues of the 0.8.x branch.
Semi-automatic update generated by
https://github.com/ryantm/nixpkgs-update tools. This update was made
based on information from
gtest
(cherry picked from commit 9cfbcc28d0)
Reason: Pre-Requisite to backport uriparser 0.9.0, which patches
security issues of the 0.8.x branch.
This release adds support for building with cmake!
So switch to that eagerly instead of fighting with bam.
(if nothing else cmake is the devil we know...)
Also:
* fixup 'DATA_DIR' so programs can find resources
(without need for wrappers)
* install readme+license as previously done ("docs")
* don't install tools since not built or installed by default
* esp since doesn't appear to have non-adhoc method for installation
* other distros don't seem to include
(cherry picked from commit 18258bae34)
Fixes CVE-2018-18541.
After a series of amendments the seccomp.patch made ntpd work properly
but only on 32-bit systems.
This commit replaces that patch with the one submitted upstream by
cleverca22 and that fixes the issue also on 64-bit systems.
Close#38627, #45885
(cherry picked from commit 6759b7900e)
Documents functions in `lib.options` for docs generation with nixdoc.
The formatting change in the `mkOption` arguments is due to the way
`nixdoc` parses documentation comments on pattern arguments. It's not
ideal, but it works.
Documents functions in `lib.debug` for docs generation with nixdoc.
Note that type signatures and clearer descriptions are still missing
on some of these functions, but this is good enough for a first run.
Updates documentation comments with extra information for nixdoc[1]
compatibility.
Some documentation strings have additionally been reworded for
clarity.
"Faux types" are added where applicable, but some functions do things
that are not trivially representable in the type notation used so they
were ignored for this purpose.
[1]: https://github.com/tazjin/nixdoc
Fix the mesa dependency for `xorg.xf86videovmware`. Add `llvm_6` because
`mesa_noglu` has a runtime dependency on clang (see TODO in
`development/libraries/mesa`).
Sage is commonly referred to as sagemath (probably to improve
searchability). Many other distors call their package sagemath.
(cherry picked from commit 80833d13fe)
This is a minimal change mainly to make sure repology recognizes this
package as sage. I plan a bit more refactoring later, but this is
intentionally kept minimally invasive so that it won't break anything
and can be backported to 18.09.
(cherry picked from commit 2513747356)
Create a many-layered Docker Image.
Implements much less than buildImage:
- Doesn't support specific uids/gids
- Doesn't support runninng commands after building
- Doesn't require qemu
- Doesn't create mutable copies of the files in the path
- Doesn't support parent images
If you want those feature, I recommend using buildLayeredImage as an
input to buildImage.
Notably, it does support:
- Caching low level, common paths based on a graph traversial
algorithm, see referencesByPopularity in
0a80233487993256e811f566b1c80a40394c03d6
- Configurable number of layers. If you're not using AUFS or not
extending the image, you can specify a larger number of layers at
build time:
pkgs.dockerTools.buildLayeredImage {
name = "hello";
maxLayers = 128;
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
- Parallelized creation of the layers, improving build speed.
- The contents of the image includes the closure of the configuration,
so you don't have to specify paths in contents and config.
With buildImage, paths referred to by the config were not included
automatically in the image. Thus, if you wanted to call Git, you
had to specify it twice:
pkgs.dockerTools.buildImage {
name = "hello";
contents = [ pkgs.gitFull ];
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
buildLayeredImage on the other hand includes the runtime closure of
the config when calculating the contents of the image:
pkgs.dockerTools.buildImage {
name = "hello";
config.Cmd = [ "${pkgs.gitFull}/bin/git" ];
};
Minor Problems
- If any of the store paths change, every layer will be rebuilt in
the nix-build. However, beacuse the layers are bit-for-bit
reproducable, when these images are loaded in to Docker they will
match existing layers and not be imported or uploaded twice.
Common Questions
- Aren't Docker layers ordered?
No. People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
- What happens if I have more than maxLayers of store paths?
The first (maxLayers-2) most "popular" paths will have their own
individual layers, then layer #(maxLayers-1) will contain all the
remaining "unpopular" paths, and finally layer #(maxLayers) will
contain the Image configuration.
(cherry picked from commit 4fe9006190)
(cherry picked from commit d1e46df24b)
Using a simple algorithm, convert the references to a path in to a
sorted list of dependent paths based on how often they're referenced
and how deep in the tree they live. Equally-"popular" paths are then
sorted by name.
The existing writeReferencesToFile prints the paths in a simple
ascii-based sorting of the paths.
Sorting the paths by graph improves the chances that the difference
between two builds appear near the end of the list, instead of near
the beginning. This makes a difference for Nix builds which export a
closure for another program to consume, if that program implements its
own level of binary diffing.
For an example, Docker Images. If each store path is a separate layer
then Docker Images can be very efficiently transfered between systems,
and we get very good cache reuse between images built with the same
version of Nixpkgs. However, since Docker only reliably supports a
small number of layers (42) it is important to pick the individual
layers carefully. By storing very popular store paths in the first 40
layers, we improve the chances that the next Docker image will share
many of those layers.*
Given the dependency tree:
A - B - C - D -\
\ \ \ \
\ \ \ \
\ \ - E ---- F
\- G
Nodes which have multiple references are duplicated:
A - B - C - D - F
\ \ \
\ \ \- E - F
\ \
\ \- E - F
\
\- G
Each leaf node is now replaced by a counter defaulted to 1:
A - B - C - D - (F:1)
\ \ \
\ \ \- E - (F:1)
\ \
\ \- E - (F:1)
\
\- (G:1)
Then each leaf counter is merged with its parent node, replacing the
parent node with a counter of 1, and each existing counter being
incremented by 1. That is to say `- D - (F:1)` becomes `- (D:1, F:2)`:
A - B - C - (D:1, F:2)
\ \ \
\ \ \- (E:1, F:2)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
Then each leaf counter is merged with its parent node again, merging
any counters, then incrementing each:
A - B - (C:1, D:2, E:2, F:5)
\ \
\ \- (E:1, F:2)
\
\- (G:1)
And again:
A - (B:1, C:2, D:3, E:4, F:8)
\
\- (G:1)
And again:
(A:1, B:2, C:3, D:4, E:5, F:9, G:2)
and then paths have the following "popularity":
A 1
B 2
C 3
D 4
E 5
F 9
G 2
and the popularity contest would result in the paths being printed as:
F
E
D
C
B
G
A
* Note: People who have used a Dockerfile before assume Docker's
Layers are inherently ordered. However, this is not true -- Docker
layers are content-addressable and are not explicitly layered until
they are composed in to an Image.
(cherry picked from commit fd045173ce)
This replaces standard checkPhase using setup.py that uses pytest-runner
with an explicit call to pytest. One reason to do that is because
setup.py triggers another rebuild when asked to do "test". Another
reason is that there seems to be a conflict between possible imports for
pyarrow: `pwd`/pyarrow vs $out/pyarrow. By some unknown reason this
triggers an import error on hydra and ofborg, but not on my machine. The
solution here is to remove `pwd`/pyarrow, keep the tests and use direct
call to pytest (setup.py needs `pwd`/pyarrow). The added benefit is that
we are now testing what is installed in $out.
(cherry picked from commit 6ae3cb4d36)
This backport to 18.09 fixes a serious of security related issues in
mosquitto when run as a server.
Use cmake on all platforms instead of only macos.
(cherry picked from commit a45fa35742)
The `name` arg of `vim_configurable.customize` does not only determine
the package name, but also the name of the command/ executable to be
called.
In my opinion this is not documented properly and finding that out took
me several hours.
the mapmap binary segfaults at the moment at master and nixos-18.09. I
will speak with upstream for a fix and mark this broken in the meantime.
(cherry picked from commit 5ab10923d6)
Usuage: Add breakpointHook to your `buildInputs` like this:
stdenv.mkDerivation rec {
# ...
buildInputs = [ breakpointHook ];
});
When the build fails as show in this example:
pkgs.hello.overrideAttrs (old: {
buildInputs = [ breakpointHook ];
postPatch = ''
false
'';
});
It will halt execution printing the following message:
build failed in patchPhase with exit code 1
To attach to this build run the following command as root:
cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10
Installing cntr and running the command will provide shell access to the
build sandbox of failed build:
sudo cntr attach -t command cntr-/nix/store/ynyb4n82x2r7sldd58pbb405jdqh5f00-hello-2.10
WARNING: bad ownership on /nix/var/nix/profiles/per-user/root, should be 1000
[nixbld@localhost:/var/lib/cntr]$
At /var/lib/cntr the sandbox filesystem is mounted. All commands and
files of the system are still accessible within the shell.
To execute commands from the sandbox use the `cntr exec` subcommand.
(cherry picked from commit f10b935f84)
Same fix as d301d5cb74.
A traditional backport with `git cherry-pick` is impossible as the
override approach for VIM plugins has significantly changed.
Startup notification doesn't work in recent versions of Kitty:
> Traceback (most recent call last):
> File "/nix/store/3a3b0xd952gp8jw70k5kh3a4zhgzf0p7-kitty-0.12.3/bin/../lib/kitty/kitty/utils.py", line 216, in init_startup_notification
> return init_startup_notification_x11(window_handle, startup_id)
> File "/nix/store/3a3b0xd952gp8jw70k5kh3a4zhgzf0p7-kitty-0.12.3/bin/../lib/kitty/kitty/utils.py", line 201, in init_startup_notification_x11
> return init_x11_startup_notification(display, window_handle, sid)
> OSError: Failed to load libstartup-notification-1.so with error: libstartup-notification-1.so: cannot open shared object file: No such file or directory
Apparently dispatching of startup notification has been moved to a C
binding in kitty 0.6.0 [1] so the substituion had to be modified to reflect
that. Without this fix Kitty still works except that window managers
which depend on startup notifications to be fired (e.g. Awesome)
cannot apply special placement rules.
The substitution mechanism is also changed to use a patch file to detect
this kind of mistakes in the future.
[1]: b08f4ab593
(cherry picked from commit 1842c4aaa4)
On AMD hardware with Mesa 18, compton renders some colours incorrectly
when using the glx backend. This patch sets an environmental variable
for compton so colours are rendered correctly.
Topical bug: <https://bugs.freedesktop.org/show_bug.cgi?id=104597>
This breaks with networking backends enabled and
also creates large delays on boot when some services depends
on the network target. It is also not really required
because tinc does create those interfaces itself.
fixes#27070
(cherry picked from commit 5a1f0f9aa3)
Initially, rst2html was called via the python interpreter which would
fail if the script was wrapped in a launcher as on NixOS.
(cherry picked from commit d4f88f1d0c)
when the build was refactored in ef5e212d, this was lost. a4ba8c65
got most of it back, but missed the addition of gst_plugins to
buildInputs.
fixes#39960
(cherry picked from commit 6fcee4e1dd)
Signed-off-by: Maximilian Bosch <maximilian@mbosch.me>
Previously you either had to set the setuid bit yourself or workaround
`isSystemUser = true` (for a loginable shell) to access the weechat
screen.
`programs.screen` shouldn't do this by default to avoid taking too much
assumptions about the setup, however `services.weechat` explicitly
requires tihs.
See #45728
(cherry picked from commit 018573b757)
* nat/bind/dhcp.service:
Remove. Those services have nothing to do with a link-level service.
* sys-subsystem-net-devices-${if}.device:
Add as BindsTo dependency as this will make hostapd stop when the
device is unplugged.
* network-link-${if}.service:
Add hostapd as dependency for this service via requiredBy clause,
so that the network link is only considered to be established
only after hostapd has started.
* network.target:
Remove this from wantedBy clause as this is already implied from
dependencies stacked above hostapd. And if it's not implied than
starting hostapd is not required for this particular network
configuration.
(cherry picked from commit 725fcdef3f)
Fixes CVE-2018-10933:
libssh versions 0.6 and above have an authentication bypass
vulnerability in the server code. By presenting the server an
SSH2_MSG_USERAUTH_SUCCESS message in place of the
SSH2_MSG_USERAUTH_REQUEST message which the server would expect to
initiate authentication, the attacker could successfully authentciate
without any credentials.
Source:
https://www.libssh.org/2018/10/16/libssh-0-8-4-and-0-7-6-security-and-bugfix-release/
(cherry picked from commit eca462813d)
Greybird is dual-licensed as GPLv2 or later and CC-BY-SA 3.0 or later.
The first is free, and the second is unfree in nixpkgs. Currently both
licenses are listed in the package derivation. And nix takes that it is
unfree. If one of the licenses in the list is unfree. nix consider
that the software is unfree. Remove the unfree one.
(cherry picked from commit 1944defa4c)
This adds configuration options which automate the configuration of NVIDIA Optimus using PRIME. This allows using the NVIDIA proprietary driver on Optimus laptops, in order to render using the NVIDIA GPU while outputting to displays connected only to the integrated Intel GPU. It also adds an option for enabling kernel modesetting for the NVIDIA driver (via a kernel command line flag); this is particularly useful together with Optimus/PRIME because it fixes tearing on PRIME-connected screens.
The user still needs to enable the Optimus/PRIME feature and specify the bus IDs of the Intel and NVIDIA GPUs, but this is still much easier for users and more reliable. The implementation handles both the X configuration file as well as getting display managers to run certain necessary `xrandr` commands just after X has started.
Configuration of commands run after X startup is done using a new configuration option `services.xserver.displayManager.setupCommands`. Support for this option is implemented for LightDM, GDM and SDDM; all of these have been tested with this feature including logging into a Plasma session.
Note: support of `setupCommands` for GDM is implemented by making GDM run the session executable via a wrapper; the wrapper will run the `setupCommands` before execing. This seemed like the simplest and most reliable approach, and solves running these commands both for GDM's X server and user X servers (GDM starts separate X servers for itself and user sessions). An alternative approach would be with autostart files but that seems harder to set up and less reliable.
Note that some simple features for X configuration file generation (in `xserver.nix`) are added which are used in the implementation:
- `services.xserver.extraConfig`: Allows adding arbitrary new sections. This is used to add the Device section for the Intel GPU.
- `deviceSection` and `screenSection` within `services.xserver.drivers`. This allows the nvidia configuration module to add additional contents into the `Device` and `Screen` sections of the "nvidia" driver, and not into such sections for other drivers that may be enabled.
It was reported that the 2nd solution wasn't working as expected because
it was ran in the wrong phase.
This commit creates a new phase, in between the installCheckPhase and distPhase.
(cherry picked from commit 20a4a4b593)
The previous description "string" is misleading in the full options
manual pages; they are actually concatenated strings, with a specific
character.
The empty string version ("types.string") has been special-cased to
provide a better message.
(cherry picked from commit 0808c7cd31)
The package is no longer maintained and includes potential security
vulnerabilities (a use-after-free and several crashes) that are known
and not fixed in the version present in nixpkgs. They don't look too bad
judging from the trigger vectors mentioned in the changelog, but with
the project going unmaintained one of them becoming a security
vulnerability would likely not be noticed.
(cherry picked from commit ad992cbde1)
When a bridge interface was reconfigured, running containers using
this bridge lost connectivity: restarting network-addresses-brN.service
triggered a restart of network-setup.service via a "partOf" relationship
introduced in 07e0c0e0a2.
This in turn restarted brN-netdev.service.
The bridge was thus destroyed and recreated with the same name but a new
interface id, causing attached veth interfaces to lose their connection.
This change removes the "partOf" relationship between
network-setup.service and network-addresses-brN.service for all bridges.
(cherry picked from commit 2742063677)
Hard-code kubernetes version as the upstream python script is broken/not-up-to-date
Signed-off-by: Vincent Demeester <vincent@sbr.pm>
(cherry picked from commit ba98a7aea1)
Backport of #47765
This fixes a regression introduced in commit
700e21d6da
nix needs ssh on path for the SSH substituter functionality,
not only the distributed builds functionality.
Signed-off-by: Niklas Hambüchen <mail@nh2.me>
(cherry picked from commit 31919bce6b)
Prior to this commit, the default values for `Restart`, `PrivateTmp` and
`WorkingDirectory` were falsely ignored.
I also added myself as maintainer.
(cherry picked from commit fbb7e0c82f)
nfs-utils had a dependency on gcc through
etc/systemd/system-generators/*-server-generator. It was not stripped
correctly because it’s not in an expected path. This adds it to the
strip list.
(cherry picked from commit 1427c50cea)
The previous tentative to the fix got the order mixed up a bit. This
new fix has been re-verified to get them in the good order as per the
instructions in the following chapters.
(cherry picked from commit 467bec34bb)
Included changes:
* upstream repository has moved, URLs changed accordingly
* journaldriver bumped to new upstream release
The new release includes an important workaround for an issue that
could cause log-forwarding to fail after service restarts due to
invalid journal cursors being persisted.
(cherry picked from commit 5ead27394d)
backported because of moved upstream repo
From upstream kbd, some keymap names are reused across very different
keyboard layouts. This is a a problem because loadkeys just picks the
first keymap it sees. The clashing names lead to e.g. "loadkeys no"
defaulting to a norwegian dvorak map instead of the much more common
qwerty one.
Used Arch Linux's list of keymaps that need renaming, with a small
deviation: the norwegian dvorak map becomes dvorak-no instead of
no-dvorak, to match the naming scheme for all the other dvorak maps.
fixes#47878
(cherry picked from commit f1987fb58f)
When logging into a container by using
nixos-container root-login
all nix-related commands in the container would fail, as they
tried to modify the nix db and nix store, which are mounted
read-only in the container. We want nixos-container to not
try to modify the nix store at all, but instead delegate
any build commands to the nix daemon of the host operating system.
This already works for non-root users inside a nixos-container,
as it doesn't 'own' the nix-store, and thus defaults
to talking to the daemon socket at /nix/var/nix/daemon-socket/,
which is bind-mounted to the host daemon-socket, causing all nix
commands to be delegated to the host.
However, when we are the root user inside the container, we have the
same uid as the nix store owner, eventhough it's not actually
the same root user (due to user namespaces). Nix gets confused,
and is convinced it's running in single-user mode, and tries
to modify the nix store directly instead.
By setting `NIX_REMOTE=daemon` in `/etc/profile`, we force nix
to operate in multi-user mode, so that it will talk to the host
daemon instead, which will modify the nix store for the container.
This fixes#40355
(cherry picked from commit 3624bb5362)
snapshot.debian.org actually keeps track of all of the updates as they
come in rather than doing arbitrary (?) snapshots.
(cherry picked from commit 9cc18fa7f9)
tor-browser-bundle-bin: 8.0.1 -> 8.0.2
Update to latest version due to security updates and 8.0.1 not being
available on the mirrors anymore.
Release notes: https://blog.torproject.org/new-release-tor-browser-802
(cherry picked from commit 248ed3575c)
reason: The binary dist-file of tor-browser-bundle-bin-8.0.2 is gone
from the mirrors, making the derivation for v8.0.1 fail.
This is just a minor version bump.
The ESR release is no longer supported by Mozilla. We should mark it as
insecure as we did on 18.03 and master.
Originally marked as insecure on master in 0a3f07077b.
Highlights in this release include:
This release fixes problems with argument handling, some unintended results of the security fixes to the SAFER file access restrictions (specifically accessing ICC profile files), and some additional security issues over the recent 9.24 release.
CVE-2018-16802
CVE-2018-17183
Note: The ps2epsi utility does not, and cannot call Ghostscript with the -dSAFER command line option. It should never be called with input from untrusted sources.
Security issues have been the primary focus of this release, including solving several (well publicised) real and potential exploits.
PLEASE NOTE: We strongly urge users to upgrade to this latest release to avoid these issues.
As well as Ghostscript itself, jbig2dec has had a significant amount of work improving its robustness in the face of out specification files.
IMPORTANT: We are in the process of forking LittleCMS. LCMS2 is not thread safe, and cannot be made thread safe without breaking the ABI. Our fork will be thread safe, and include performance enhancements (these changes have all be been offered and rejected upstream). We will maintain compatibility between Ghostscript and LCMS2 for a time, but not in perpetuity. Our fork will be available as its own package separately from Ghostscript (and MuPDF).
The usual round of bug fixes, compatibility changes, and incremental improvements.
(cherry picked from commit 5b77b0d2f1)
This attempts to improve stability of the test by using existing
services for miniupnpd and transmission.
It also uses explicit addresses for the network interfaces so that the
external IP addresses are valid internet addresses (thus fixing
validation problems from upnpc).
Also disable eth0 from being used to transfer torrents over without that
being the intention.
(cherry picked from commit 276ffc5656)
Hydra passes the full revision in to the input, which we pass through.
If we don't get this ,we try to get it from other sources, or default to
master which should have the definition in a close-ish location.
All published docs should have theURL resolve properly, only local
hackers will have the link break.
(cherry picked from commit 5daee73ce4)
Covers assert functions and about half of the attrsets functions.
Some internal consistency around IDs could be improved.
(cherry picked from commit f835f77e02)
(cherry picked from commit 18b468ed81)
Reason: Security update: "Depend on new version of gollum-lib that
relies on a patched version of sanitize, which solves a vulnerability
(CVE-2018-3740). See https://github.com/gollum/gollum-lib/pull/296."
The problem was that the non-fatal warning was not omitted
from the output when constructing a nix expression.
Now it seems OK for me. When return code is OK,
the warnings don't get passed anywhere, but I expect
that won't matter for this utility. Fatal errors are still shown.
(cherry picked from commit de93b32f90)
They are known to cause more issues than solving issues; futhermore
using `dd` should work everywhere without fail.
(cherry picked from commit 8467dc857b)
The tests in <nixos/tests/installer.nix> are using `parted`, so they are
bound to be better tested than `fdisk`.
This is brought on by a couple issues, plus reports on IRC that the
`fdisk` instructions didn't work as expected.
* #39354
* #46309
* #39942
* #45478
Care was taken so that the other documented steps did not need changes.
In all this kerfufle, a slight re-organization of the Chapter has been
made, allowing better deep linking.
(cherry picked from commit 6cfbf403ca)
While it seemingly brings more attention to the macOS notes with the
default docbook template, it better represents which parts of the
section are about macOS, and which parts are simply in the flow of the
text; otherwise the last paragraph may be lost into the details for
macOS.
(cherry picked from commit 8192fcd0fd)
Currently, a self-extracting archive created by makeself will fail to
properly execute on NixOS because the boilerplate Bash code it uses to
clean up the temporary directory it extracted its contents into
assumes that the `rm` command is installed at `/bin/rm`, which is not
the case on NixOS.
This commit, a backport of a pull request I made to the upstream
repository at megastep/makeself#142, fixes the issue by causing the
boilerplate code to call `rm` without specifying an absolute path,
which allows the version of `rm` from one's current Nix environment to
be used instead.
(cherry picked from commit eb84586cad)
The autoupgrade service defined in `system.autoUpgrade`
(`nixos/modules/installer/tools/auto-upgrade.nix`) doesn't have `su` in
its path and thus yields a warning during the `daemon-reload`.
Specifying the absolute path fixes the issue.
Fixes#47648
(cherry picked from commit 7297cc5501 and
resolved conflict)
This update bumps the package to the latest stable version containing a
few security fixes:
- CVE-2018-12386: Type confusion in JavaScript
A vulnerability in register allocation in JavaScript can lead to type
confusion, allowing for an arbitrary read and write. This leads to
remote code execution inside the sandboxed content process when
triggered.
- CVE-2018-12387
A vulnerability where the JavaScript JIT compiler inlines
Array.prototype.push with multiple arguments that results in the stack
pointer being off by 8 bytes after a bailout. This leaks a memory
address to the calling function which can be used as part of an
exploit inside the sandboxed content process.
Source: https://www.mozilla.org/en-US/security/advisories/mfsa2018-24/
(cherry picked from commit 246d2848ff)
This update bumps the package to the latest stable version containing a
few security fixes:
- CVE-2018-12386: Type confusion in JavaScript
A vulnerability in register allocation in JavaScript can lead to type
confusion, allowing for an arbitrary read and write. This leads to
remote code execution inside the sandboxed content process when
triggered.
- CVE-2018-12387
A vulnerability where the JavaScript JIT compiler inlines
Array.prototype.push with multiple arguments that results in the stack
pointer being off by 8 bytes after a bailout. This leaks a memory
address to the calling function which can be used as part of an
exploit inside the sandboxed content process.
Source: https://www.mozilla.org/en-US/security/advisories/mfsa2018-24/
(cherry picked from commit e7785f1148)
This update bumps the package to the latest stable version containing a
few security fixes:
- CVE-2018-12386: Type confusion in JavaScript
A vulnerability in register allocation in JavaScript can lead to type
confusion, allowing for an arbitrary read and write. This leads to
remote code execution inside the sandboxed content process when
triggered.
- CVE-2018-12387
A vulnerability where the JavaScript JIT compiler inlines
Array.prototype.push with multiple arguments that results in the stack
pointer being off by 8 bytes after a bailout. This leaks a memory
address to the calling function which can be used as part of an
exploit inside the sandboxed content process.
Source: https://www.mozilla.org/en-US/security/advisories/mfsa2018-24/
(cherry picked from commit 64d02660cb)
Semi-automatic update generated by
https://github.com/ryantm/nixpkgs-update tools. This update was made
based on information from
matomo
(cherry picked from commit c45dd04946)
backported for security updates
darwin.security_tool is currently broken in Mojave. See issue #45042
for more info. Our security_tool stuff comes from 10.9 so I suspect
that it needs an update.
Here I am putting in a hack to get things working again. This uses the
system provided security binary at /usr/bin/security to avoid the
issue in Haskell’s x509-system package. Unfortunately, this will break
with the sandbox. I am also working on a proper fix, but this requires
updating lots of Apple stuff (and also copumpkin’s new CF). You can
follow the progress on this branch:
https://github.com/matthewbauer/nixpkgs/tree/xcode-security
This commit should be backported to release-18.03 and release-18.09.
/cc @copumpkin @lnl7 @pikajude
We've recently seen a lot of kernel build timeouts on hydra,
so let's increase the timeout.
(cherry picked from commit 83fd9785f6 and
resolved conflict)
Based on popular demand I decided to upgraded alacritty from my own fork.
We currently also have a version in staging that works without but we
cannot easily backport this one because it relies on some changes in the rust
build infrastructure
(cherry picked from commit cb6ccb6a7e)
There were two things to fix:
- Boost started shipping libboost_python.so as libboost_python{Major}{Minor}
- Make sure that mapnik and boost link to the correct version of python.
(cherry picked from commit be6451e020)
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
[dezgeg: Also squash in the bugfix commit "nixos/kexec: Fix typo in meta.platforms" by aszlig]
(cherry picked from commit 599c4df46a)
(cherry picked from commit fd8bca45c9)
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
(cherry picked from commit f4dd2fed7f)
This reverts commit 79d8353b5e.
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
(cherry picked from commit 5e84926a66)
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
(cherry picked from commit 50d0360888)
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
(cherry picked from commit cdeafe7cdf)
Conflicts:
pkgs/tools/package-management/nix/default.nix
This sort of code breaks config.{allowBroken, allowUnsupportedSystem} =
true by making them do unpredictable things.
(cherry picked from commit fe7919f7a1)
Fixes#47498. Note that the releases have moved to a new URL; I'm not sure they still host the earlier versions.
(cherry picked from commit 9d727e0a91)
backported because previous source url returns 404.
Nix commands inside the container have been broken since 18.03,
and no fix is yet in sight. Lets remove from the documentation
that this is a usecase that we support, as it doesn't seem
likely that this will be fixed before 18.09 either.
See #40355
(cherry picked from commit f309440ee3)
Similar to some other builds, sdl-config gets stuck and times out.
checking for sdl-config... /nix/store/q2hm1ka0qxs2gv5bmrxj7j80lygk4b5z-SDL-1.2.15-dev/bin/sdl-config
building of '/nix/store/m2d7v0n5hd2498vfxp18i37p7r1lf76p-freedroidrpg-0.16.1' timed out after 7200 seconds of silence
/cc ZHF #45961
(cherry picked from commit 9b99819283)
Doesn't build on darwin and probably other platforms.
GsWidget.H:26:3: error: 'Atom' does not name a type; did you mean 'tm'?
Atom atoms[5];
^~~~
tm
GsWidget.cxx: In member function 'void GsWidget::setProps()':
GsWidget.cxx:47:2: error: 'atoms' was not declared in this scope
atoms[0] = XInternAtom(fl_display,"GHOSTVIEW" , false);
^~~~~
/cc ZHF #45961
(cherry picked from commit cd78d0cc3f)
Undefined symbols for architecture x86_64:
"_MPCreateSemaphore", referenced from:
FX::FXSemaphore::FXSemaphore(int) in FXThread.o
FX::FXSemaphore::FXSemaphore(int) in FXThread.o
"_MPDeleteSemaphore", referenced from:
FX::FXSemaphore::~FXSemaphore() in FXThread.o
FX::FXSemaphore::~FXSemaphore() in FXThread.o
"_MPSignalSemaphore", referenced from:
FX::FXSemaphore::post() in FXThread.o
"_MPWaitOnSemaphore", referenced from:
FX::FXSemaphore::wait() in FXThread.o
FX::FXSemaphore::trywait() in FXThread.o
ld: symbol(s) not found for architecture x86_64
/cc ZHF #45961
(cherry picked from commit 6390b8b63e)
The socket activation I added to the rspamd module doesn't actually work
and can't be made to work without changes to rspamd.
See: #47421
See: rspamd/rspamd#2035
(cherry picked from commit 783a58f363)
Likewise logind we should not try to restart this service after upgrade,
the user's current session depends on it.
(cherry picked from commit aa69bb5743)
3.5.7 doesn't build with kernel 4.14.71.
Needs most recent upstream revision to build, which supports
kernels up to 4.17.
(cherry picked from commit f0e4b31eb2)
Thanks to @dezgeg for prototype implementation, I've
cleaned it up and added documentation.
(cherry picked from commit d6e3db44cf)
Signed-off-by: Domen Kožar <domen@dev.si>
/cc @Ericson2314
PR was https://github.com/NixOS/nixpkgs/pull/46857
This line broke MacOS cross compilation. paxctl cannot be built on
macOS. Maybe it can be fixed, but no reason to break things
unnecessarily.
Regardless, you definitely need to be more careful about backporting.
I think it’s fine to move fast and break things on master but
with release-18.09 we should be more careful. Something like more
automated testing for cross compilation would also be
helpful (hopefully even making it block).
* podofo: fix library linkage on Darwin
Because the library is not yet installed when the tools are build, it does not contain its final store path as its install name. Linking the tools picks up this incorrect install name and needs to be fixed after installing.
(cherry picked from commit 4b1ffa0b00)
The patchPhase wasn't being applied at all.
This patch re-enables that and also re-thinks the setting that we want
to have. Turning off the auto-update is more accurate and doesn't lose
information like it did before.
(cherry picked from commit 6060940c24)
- build fails with qt511 and qt59
(it used to build with qt510 which is not available anymore)
- no nixpkgs maintainer
- upstream repo looks inactive, last commit a year ago
(cherry picked from commit 9beaa6dcad)
Update to the latest release.
Highlights for c-lightning users
--------------------------------
- Less stuck payments: Liveness ping test before locking up funds with peers.
- Better routing: now considers size of channels.
- Fewer spurious closes: fee estimate improvements, and new feerates command
- Several annoying bugs fixed.
Highlights for the network
--------------------------
- Gossipd now less spammy with channel_update.
- option_data_loss_protect to protect peers against being out-of-date.
- Payment errors now refer to the correct channel.
Internal Improvements
---------------------
- Simplified client flow; after init message exchange by connectd, each is
isolated in its own daemon.
- JSON parameter handling vastly simplfied.
- Python testing framework now uses proper fixtures, and split into separate
files.
- Many other cleanups and clarifications.
- We keepachangelog.com!
Signed-off-by: William Casarin <jb55@jb55.com>
(cherry picked from commit 2aa0b85653)
Backport of #47286
Fixes a critical issue with macOS
[NEWS](https://bitbucket.org/mituharu/emacs-mac/raw/master/NEWS-mac)
* emacs-26.1-mac-7.2 (2018-09-09)
** Fixed bugs
*** Buffer contents are not displayed on macOS 10.14.
This is mainly because now NSViews are backed by Core Animation Layer
(layer-backed) by default and non-deferred drawing into views no
longer works. Instead of switching to deferred drawing (i.e., draw
only inside -[NSView drawRect:]), we draw into our own backing bitmap
in a non-deferred way as before, and update the view contents with the
resulting image via -[NSView updateLayer]. This "application-side
double buffering" is also available on OS X 10.8 - macOS 10.13 if you
set the frame parameter `inhibit-double-buffering' to nil when
creating a frame. Just like on macOS 10.14, such a frame does not do
LCD smoothing.
*** Screenshot grabbed via Services is displayed in wrong size when we
have display mirroring between Retina and non-Retina displays.
*** Cursor movement just after frame resize sometimes leaves garbage.
*** Crash by the Fall_threads call from the GUI thread at the select
emulation when there are multiple Lisp threads.
*** Info title has ASCII underline unlike other window systems.
*** Vertical scroll bar is created as horizontal if frame font height
is short.
** Improvements
*** macOS 10.14 adds property :appearance to (mac-application-state).
The value may be "NSAppearanceNameAqua" or "NSAppearanceNameDarkAqua".
*** Add new color format "mac:COLOR-LIST-NAME:COLOR-NAME" and
"mac:COLOR-NAME" (shorthand for "mac:System:COLOR-NAME"). The actual
color may be different depending on the global appearance setting on
macOS 10.14. For example, "mac:textColor" is black on the Light Mode
but is white on the Dark Mode.
*** Default frame colors respect appearance setting on macOS 10.14.
Now the default frame foreground/background color is
"mac:textColor"/"mac:textBackgroundColor", respectively. Changes of
the system setting of the global appearance are dynamically reflected.
*** New function `mac-color-list-alist' to get the available
combinations of COLOR-LIST-NAMEs and COLOR-NAMEs. Note that this
value is dependent on user environment and OS version. Also, some
combinations may represent image patterns rather than colors. For the
former cases, `(color-values "mac:COLOR-LIST-NAME:COLOR-NAME")'
returns nil.
Most importantly, this sets PrivateTmp, ProtectHome, and ProtectSystem
so that Chrony flaws are mitigated, should they occur.
Moving to ProtectSystem=full however, requires moving the chrony key
files under /var/lib/chrony -- which should be fine, anyway.
This also ensures ConditionCapability=CAP_SYS_TIME is set, ensuring
that chronyd will only be launched in an environment where such a
capability can be granted.
Signed-off-by: Austin Seipp <aseipp@pobox.com>
(cherry picked from commit 0ce90d58cc)
by combining all parted commands into a single parted call.
This eliminates one cause of non-deterministic failure.
(cherry picked from commit a5183762c5)
The test failed on Hydra in one instance because a request to the
server was sent before indexing was finished.
Retry the request until it succeeds (or times out).
(cherry picked from commit c525111133)
The darwin build fails and other platforms will probably run into
similar issues.
Determining if the Q_WS_QWS exist failed with the following output:
Change Dir: /tmp/nix-build-eiskaltdcpp-2.2.10.drv-0/source/build/CMakeFiles/CMakeTmp
Run Build Command:"/nix/store/qfag24z6xsr6jkyi8gb4cv62rp945rbk-gnumake-4.2.1/bin/make" "cmTC_14e67/fast"
/nix/store/qfag24z6xsr6jkyi8gb4cv62rp945rbk-gnumake-4.2.1/bin/make -f CMakeFiles/cmTC_14e67.dir/build.make CMakeFiles/cmTC_14e67.dir/build
make[1]: Entering directory '/private/tmp/nix-build-eiskaltdcpp-2.2.10.drv-0/source/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_14e67.dir/CheckSymbolExists.cxx.o
/nix/store/2gdwhdzhy4iwkp7fh8v6gy6nxj1zi9pv-clang-wrapper-5.0.2/bin/clang++ -I/nix/store/8dzqilmdr0p3qmmrxh51xk7wli6grm0i-qt-4.8.7/include -F/nix/store/8dzqilmdr0p3qmmrxh51xk7wli6grm0i-qt-4.8.7/lib -std=c++0x -pipe -Wformat -Werror=format-security -arch x86_64 -o CMakeFiles/cmTC_14e67.dir/CheckSymbolExists.cxx.o -c /tmp/nix-build-eiskaltdcpp-2.2.10.drv-0/source/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx
/tmp/nix-build-eiskaltdcpp-2.2.10.drv-0/source/build/CMakeFiles/CMakeTmp/CheckSymbolExists.cxx:8:19: error: use of undeclared identifier 'Q_WS_QWS'
return ((int*)(&Q_WS_QWS))[argc];
^
/cc ZHF #45961
(cherry picked from commit 52ad963b50)
This reverts commit 67c8c49177.
'nix run nixos.firefox' is *not* supposed to work - the Nix 2.x
interface attempts to standardize on nixpkgs.*, to get rid of the
nixos/nixpkgs confusion that existed with the channels interface. So
let's not bring that confusion back.
(cherry picked from commit 9c53116d49)
environment.sessionVariables cannot refer to the values of env vars,
and as a result this has caused problems in a variety of scenarios.
One use for these is that they're injected into /etc/profile,
elewhere these are used to populate an 'envfile' for pam
(`pam 5 pam_env.conf`) which mentions use of HOME being
potentially problematic.
Anyway if the goal is to make things easier for users,
simply do the NIX_PATH modification as extraInit.
This fixes the annoying problems generated by the current approach
(#40165 and others) while hopefully serving the original goal.
One way to check if things are borked is to try:
$ sudo env | grep NIX_PATH
Which (before this change) prints NIX_PATH variable with
an unexpanded $HOME in the value.
-------
This does mean the following won't contain user channels for 'will':
$ sudo -u will nix-instantiate --eval -E builtins.nixPath
However AFAICT currently they won't be present either,
due to unescaped $HOME. Unsure if similar situation for other users
of sessionVariables (not sudo) work with current situation
(if they exist they will regress after this change AFAIK).
(cherry picked from commit fa67ca1543)
/run/rmilter is set by systemd, and have root:root ownership, which
prevent pid file to write.
This fix suggested to be promoted to 18.09 branch.
(Although rmilter itself is deprecated, and I plan to remove it, after
18.09 would be released)
(cherry picked from commit 08f266490b)
Without this the graphical installer has no way to open the manual.
You can fix it yourself by installing any HTML browser but this might
be unfamiliar to users new to NixOS and without any other way to open
the manual. The downside is it will also increase download sizes.
Fixes#46537
See https://hydra.nixos.org/build/80705583
Recent `boost` versions with `python` enabled have changed their naming
scheme for `boost_python` shared objects which causes issues with the
proper linking when building `pyftgl`.
Previously the library was named `boost_python3`, no it's named
`boost_python36` for current python (3.6.x).
The same issue applies for the `python2`.
Addresses #45960
(cherry picked from commit 5ad7aedf63)
The ZSH completion script of `tig` basically imports the bash completion
script and uses it as completion approach. Unfortunately the script
takes several assumptions about the directory structure using
`$funcsourcetrace`[1] that don't apply on NixOS.
The easiest workaround is t opatch the completion script and import the
bash completion script from `$out`.
[1] http://zsh.sourceforge.net/Doc/Release/Zsh-Modules.html#index-funcsourcetrace
For technical reasons, we cannot easily add a warning to top-level
definitions, so 2a6e4ae49a and
e51f736076 reverted the deprecation. But
we can still remove mention of the would-be deprecated definitions to
steer people towards using the preferred alternatives.
(cherry picked from commit e39a73cc55)
2a6e4ae49a and
e51f736076 reverted a bit too much, and I
initially missed this when reviewing. The release notes already still
mention this change, too.
(cherry picked from commit 359d00f8b0)
This reverts commit 713991132e.
libtensorflow is only used by the Haskell tensorflow packages and they
don't work with tensorflow-1.10 yet. So the easiest solution is to
just revert this commit and add it back when they do gain support.
(cherry picked from commit 6e14b9fc2d)
This reverts commit 93ed13f86b.
libtensorflow is only used by the Haskell tensorflow packages and they
don't work with tensorflow-1.10 yet. So the easiest solution is to
just revert this commit and add it back when they do gain support.
(cherry picked from commit 39d35e77ee)
(cherry picked from commit 0fa04d646d)
It is unfortunate to cherry-pick something with release notes to the
release staging branch so close to the release. But the change is to a
new service so no one comming from 18.03 would be surprised to a
last-minute change in behavior. Furthermore, the change is a month old
and it appears it only wasn't cherry-picked already by mistake.
PHP tries to discover the mysql default socket path during configure
phase by probing the file system:
cf3b852109/ext/mysqli/config.m4 (L4)
This obviously fails to discover /run/mysqld/mysqld.sock, which is being
used (hardcoded) across all MySQL flavours.
This leads to PHP having no mysql socket path set for the mysql[i]
extensions, and `/tmp/mysql.sock` set for pdo_mysql,
meaning one currently has to manually configure and set it in php.ini.
Luckily, PHP supports setting that path via
`--with-mysql-sock=/run/mysqld/mysqld.sock` during configure phase,
so let's do this as soon as one of the three modules is enabled.
(cherry picked from commit baa04e4204)
As per the following bug report, sometimes erts/start_scripts will fail
to compile because of a Makefile ordering issue. Apply the upstream
patches to fix this.
https://bugs.erlang.org/browse/ERL-241
(cherry picked from commit c987ed0896)
The test didn't catch an issue with luadbi because it used
internal storage only. Switch to sqlite to test sql access.
(cherry picked from commit 589b290b43)
Loading olm.lua as weechat script with `/script load olm.lua' causes
errors like this:
```
/nix/store/43jbh7yxh8j4gjfzbvpd9clncah5dip1-weechat-matrix-bridge-2018-05-29/lib/ffi.so: undefined symbol: lua_tointeger
```
As `olm.lua' is loaded by `matrix.lua' it doesn't need to be included
manually by the weechat configuration.
(cherry picked from commit 61a9463498)
In my previous PR I missed that ${sec.foobar} (syntax to retrieve
secrets in a weechat runtime) breaks the shell evaluation.
Furthermore `;` shall be used rather than `\n` to concat scripts and the
init config.
(cherry picked from commit e2b4644f9c)
This adds several improvements the previously introduced
`services.weechat` module:
* Dropped `services.weechat.init` as the initialization script can now
be done on package-level since 2af41719bc using the `configure`
function.
* Added `sessionName` option to explicitly configure a name for the
`screen` session (by default: weechat-screen).
* Added `binary` option to configure the binary name (e.g.
`weechat-headless`).
* Added docs regarding `screen` session and `weechat.service`.
(cherry picked from commit 18d419141d)
This aims to make the `weechat` package even more configurable. It
allows to specify scripts and commands using the `configure` function
inside a `weechat.override` expression.
The package can be configured like this:
```
with import <nixpkgs> { };
weechat.override {
plugins = { availablePlugins, ... }: {
plugins = builtins.attrValues availablePlugins;
init = ''
/set foo bar
/server add freenode chat.freenode.org
'';
scripts = [ "/path/to/script.py" ];
};
}
```
All commands are passed to `weechat --run-command "/set foo bar;/server ..."`.
The `plugins' attribute is not necessarily required anymore, if it's
sufficient to add `init' commands, the `plugins' will be
`builtins.attrValues availablePlugins' by default.
Additionally the result contains `weechat` and `weechat-headless`
(introduced in WeeChat 2.1) now.
(cherry picked from commit a8efe61412)
UEFI iso image for i686 didn't boot because the
default boot file was incorrectly named bootx32.efi.
The correct name is bootia32.efi.
(cherry picked from commit 845ae983f6)
This isn't necessary and causes build failures for certain versions.
CC libavfilter/libmpcodecs/vf_eq2.o
CC libavfilter/libmpcodecs/vf_fil.o
CC libavfilter/libmpcodecs/vf_fspp.o
libavfilter/libmpcodecs/vf_fspp.c:939:18: error: unknown token in expression
movq %mm2, 0*8+(%rsp)
/cc ZHF #45961
(cherry picked from commit 52a1179b6c)
Switch to maintained fork that is compatible with lua5.2/lua5.3.
This package was tested with prosody.
(cherry picked from commit a8413d816a)
Backported because prosody database access is broken without it.
This way we don't need to disable flags etc by platform and can still
backport new versions to stable for linux even if there's a bug or
something in the darwin build.
(cherry picked from commit 0b82fbc3af)
Unlike docker (cli only) this probably won't work on darwin.
github.com/docker/libnetwork/networkdb
can't load package: package github.com/docker/libnetwork/ns: build constraints exclude all Go files in /private/tmp/nix-build-docker-proxy-7b2b1feb1de4817d522cc372af149ff48d25028e.drv-0/go/src/github.com/docker/libnetwork/ns
/cc ZHF #45961
(cherry picked from commit 33f818198b)
Never built successfully.
Undefined symbols for architecture x86_64:
"_rpl_strerror", referenced from:
_default_print_diag in libgrecs.a(diag.o)
_grecs_symtab_strerror in libgrecs.a(symtab.o)
_parse_inet in libgrecs.a(sockaddr.o)
ld: symbol(s) not found for architecture x86_64
/cc ZHF #45961
(cherry picked from commit 709a828e71)
The default version isn't enabled on darwin either, however it did work
at some point.
/tmp/nix-build-dolphin-emu-2018-08-17.drv-0/source/Source/Core/Core/NetPlayServer.cpp:1180:26: error: unknown type name 'lzo_uint32_t'; did you mean 'lzo_uint32'?
std::vector<u8> wrkmem(LZO1X_1_MEM_COMPRESS);
/cc ZHF #45961
(cherry picked from commit f6bc086b21)
The test failed in one run on Hydra, logs look like
dhcpcd changed ipv6 routing at just the wrong time.
Disable dhcpcd. It's not needed, the test uses static IPs anyway.
(cherry picked from commit 5e7b7b805a)
The test failed non-deterministically when an at command was
issued before the atd daemon was running.
(cherry picked from commit 8ffd65b12e)
Backport to 18.09
Because dates are an impurity, by default buildImage will use a static
date of one second past the UNIX Epoch. This can be a bit frustrating
when listing docker images in the CLI:
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
hello latest 08c791c7846e 48 years ago 25.2MB
If you want to trade the purity for a better user experience, you can
set created to now.
pkgs.dockerTools.buildImage {
name = "hello";
tag = "latest";
created = "now";
contents = pkgs.hello;
config.Cmd = [ "/bin/hello" ];
}
and now the Docker CLI will display a reasonable date and sort the
images as expected:
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
hello latest de2bf4786de6 About a minute ago 25.2MB
(cherry picked from commit a32d7e0c74)
A lot of the functionality of the z3 library depends on it being able to
find the z3 executable on $PATH. Hard-coding it here means it will never
be unable to find it and z3 doesn't need to pollute $PATH.
(cherry picked from commit c8598daad4)
Signed-off-by: Austin Seipp <aseipp@pobox.com>
- add missing dependencies readline and libtool that used to be
propagated by guile but aren't anymore
- build with guile 2.0 instead of 1.8
(cherry picked from commit 2c5ae2ae61)
Test didn't run because it tried to create a VM with 4096M RAM
but qemu-system-i386 has a hard 2047M memory limit.
- reduce memory to 2047M on i686.
- increase timeout 300s -> 1800s because the tests are much slower
on i686 and timed out.
(cherry picked from commit e40d850fd1)
The test failed non-deterministically on Hydra because interfaces
sometimes weren't yet fully cleaned up when the result was checked.
(cherry picked from commit 05659962cd)
The `pkgs.yabar` package is relatively old (2016-04) and contains
several issues fixed on master. `yabar-unstable` containsa recent master
build with several fixes and a lot of new features (I use
`yabar-unstable` for some time now and had no issues with it).
In the upstream bugtracker some bugs could be fixed on ArchLinux by
simply installing `yabar-git` (an AUR package which builds a recent
master).
To stabilize the module, the option `programs.yabar.package` now
defaults to `pkgs.yabar-unstable` and yields a warning with several
linked issues that are known on `pkgs.yabar`.
The test has been refactored as well to ensure that `yabar` actually
starts (and avoid non-deterministic random success) and takes a
screenshot of a very minimalistic configuration on IceWM.
Fixes#46899
(cherry picked from commit 8b58a7187c)
A previous attempt to fix tests failing on some machines (#46850)
was unsuccessful, they still failed on Hydra sometimes while
succeeding locally. Revert #46850 and disable these tests instead.
(cherry picked from commit f3a0c8a3aa)
... that occurred mostly on i686. Improve timing.
With this, the failures are no longer reproducible on my machine.
(cherry picked from commit beb8ca2887)
Test failed on i686 in a sandbox because some packages required
to build the nixos manual for the container were missing. Add them.
(cherry picked from commit daf40ab165)
Build used python34Packages and failed because python34Packages.pytest
didn't build anymore. Use python3Packages instead.
(cherry picked from commit 012682222f)
These tests failed on some machines, probably due to lower arithmetic
presicion. Reduce required precision from 13 to 8 decimal digits.
(cherry picked from commit 970393c34a)
This is only a minor bugfix release and updates the fallback CA root
certificates. For NixOS this is usually not required as the probe paths
will match there, but for non-NixOS users it might be helpful.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 48d1c50f7a)
Reason: This might be relevant for NixOps users on Mac OS X and the
update won't break anything that wasn't broken before.
This test doesn't work in a sandbox and never succeeded on Hydra.
It simulates an EC2 instance reconfiguring itself at runtime,
which needs network access.
(cherry picked from commit a83d61b708)
The automake file was patched but `automake` not run.
Also since the texk/web2c folder is not in autoconfig's
SUBDIRS the autoreconfHook has to be run in there.
Completely fixes#46376
(cherry picked from commit ce6e72a11c)
If meta.outputsToInstall is set to include absent outputs, various
tools break including channel updates and nix-env.
grahamc@Morbo> nix-env -i -f . -A elf-header-real
installing 'elf-header'
error: this derivation has bad 'meta.outputsToInstall'
This patch verifies each value in meta.outputsToInstall is a valid
output. It validates this condition only if checkMeta is true.
grahamc@Morbo> nix-build . -A elf-header-real
error: Package ‘elf-header’ in /home/grahamc/projects/nixpkgs/pkgs/development/libraries/elf-header/default.nix:36 has invalid meta.outputsToInstall, refusing to evaluate.
The package elf-header has set meta.outputsToInstall to: bin
however elf-header only has the outputs: out
and is missing the following ouputs:
- bin
(use '--show-trace' to show detailed location information)
Note, now the nix-env experience is decidedly worse for users who have
checkMeta set to true:
grahamc@Morbo> nix-env -i -f . -A elf-header-real; echo $?
0
though since this is already an issue for unfree, broken, unsupported,
and insecure validity problems I'm not sure we should do something
different here.
(cherry picked from commit b80c9ce4a9)
Calling /run/current-sw/bin/sendmail fails under postfix because
setgid bits are not set. Switching the hardcoded path to an invocation
via execvp should cover both cases, when the sendmail binary is
setgid-wrapped and when it is not.
(cherry picked from commit 6d97096d8b)
Most dependencies are vendored in the upstream source anyway,
deps.nix just contains the few that are not.
Dependencies were updated manually because go2nix doesn't
work correctly for this package - it produces an empty deps.nix.
(cherry picked from commit f896867ae5)
Qt 5.11 was downgraded to 5.9 because of two issues:
- spawns errors like
```
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:20:9: QML BusyIndicator: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
qrc:/qml/SignInWaiting.qml:26:9: QML Text: Detected anchors on an item that is managed by a layout. This is undefined behavior; use Layout.alignment instead.
```
- Google login doesn't work. It just doesn't start embedded webbrowser
(cherry picked from commit 969a39bd09)
This allows the definition of a custom derivation of Exim,
which can be used to enable custom features such as LDAP and PAM support.
The default behaviour remains unchanged (defaulting to pkgs.exim).
(cherry picked from commit d73ed4264f)
Build failed after update to 4.4.0 because the tests are incorrectly
packages in the pypi version.
Switch to upstream github repo, fix tests, disable failing test cases
(cherry picked from commit a66c00d780)
This update was generated by hackage2nix v2.11-9-gb3613cb from Hackage revision
3869e8d541.
(cherry picked from commit 1b1cb6305c)
Signed-off-by: Domen Kožar <domen@dev.si>
From upstream's NEWS:
> Changes from 5.6.1 to 5.6.2
> ---------------------------
>
> Fix another SIGTTOU case.
>
> Fix SIGWINCH being ignored when zsh is not in the foreground.
>
> The release tarballs are now compressed using xz(1), whereas previously
> both xz(1) and gzip(1) versions were available. If this gets in your way,
> give us a shout.
>
(cherry picked from commit ef21db9bd3)
Various improvements -- if nothing else the 'kmstest'
utility now explains properly when permissions problem :).
(cherry picked from commit 4c1f27a4fc)
No successful hydra build since 2018-02-14.
The build fails in the ruby gem json-1.8.3, which we cannot
mark directly as broken in nixpkgs, so we mark this as broken.
(cherry picked from commit 15fedb51b3)
Build failed because of dependency Flask-PyMongo, which actually
isn't a dependency of eve as they have their own flask_pymongo code.
- Drop incorrect dependency flask-pymongo
- Drop redunant dependencies already propageted by flask:
itsdangerous, werkzeug, jinja2
- Drop dependency markupsafe, not required any more
(cherry picked from commit 9e4b0b795c)
Looks CoreFoundation related.
Undefined symbols for architecture x86_64:
"_CFURLResourceIsReachable", referenced from:
fsevent_sys::core_foundation::str_path_to_cfstring_ref::h0ea4bd94e2c613f2 in libfsevent_sys-ef30b6879660a6c1.rlib(fsevent_sys-ef30b6879660a6c1.fsevent_sys7-49ce33334334dd3a5c7883bf4070f954.rs.rcgu.o)
ld: symbol(s) not found for architecture x86_64
/cc ZHF #45961
(cherry picked from commit 327edb4b1d)
While it's possible to build on darwin, we never had a successful build.
Undefined symbols for architecture x86_64:
"sf::microseconds(long long)", referenced from:
_sfSleep in Sleep.cpp.o
/cc ZHF #45961
(cherry picked from commit 54b4000d54)
Using gccStdenv makes the build go further, but then it fails with.
kuipc /private/tmp/nix-build-cernlib-2006.drv-0/2006/src/packlib/kuip/code_kuip/kuipcdf.cdf kuipcdf.c
Makefile:413: archive/kuipcdf.d: No such file or directory
make[3]: *** [Makefile:473: kuipcdf.c] Abort trap: 6
/cc ZHF #45961
(cherry picked from commit 68f7e75517)
The Basel BUILD file conflicts with the cmake build directory on
case-insensitive filesystems, eg. darwin.
/cc ZHF #45961
(cherry picked from commit 807b0105b4)
The test binaries depend use open_memstream which isn't available on
darwin.
error: use of undeclared identifier 'open_memstream'
/cc ZHF #45961
(cherry picked from commit 04cba83e7f)
This reverts commit 0b124c1e91. We
should really stop adding things that are not packages to
all-packages.nix. For example, having nixos-rebuild.nix in
all-packages.nix causes 'nix-env -qa' to evaluate a NixOS
configuration, which obviously is not good for performance. (We should
probably also remove the 'nixos' attribute from all-packages.nix, but
at least that's a function so nix-env will ignore it.)
(cherry picked from commit f1c978aca0)
See https://hydra.nixos.org/build/81125645
`tamarin-prover' upstream has a patch to fix GHC 8.4 compilation (and
uses stack lts-12.1 now), but it's not released yet:
a08f6e4007
The build is divided in several derivations, therefore the patch had to
be splitted and rebased for `lib/term', `lib/theory' and `lib/utils' to
ensure that the patch applies properly during the `patchPhase'.
Addresses #45960
(cherry picked from commit 2cfc0bb7ee)
- remove obsolete patch that caused build failure
- no tests were run because required data files were missing in pypi
version; use github version instead
(cherry picked from commit 80ecef8e4b)
Fixed minor issue where kube-addon manager complaints about
/opt/namespace.yaml missing.
Added release notes with reference to Kubernetes 1.11 release notes.
closes#43882
(cherry picked from commit a49f56c3b1)
- add missing checkInputs
- apply upstream patch to fix some tests that failed due to
expired test ssl certs
- re-enable a previously disabled test case
(cherry picked from commit d225a91b7c)
Could be caused by our older 10.10.5 CoreFoundation.
# github.com/segmentio/aws-okta/vendor/github.com/keybase/go-keychain
go/src/github.com/segmentio/aws-okta/vendor/github.com/keybase/go-keychain/corefoundation_go110.go:35:33: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFDataCreate
go/src/github.com/segmentio/aws-okta/vendor/github.com/keybase/go-keychain/corefoundation_go110.go:61: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
go/src/github.com/segmentio/aws-okta/vendor/github.com/keybase/go-keychain/corefoundation_go110.go:98:41: cannot use nil as type _Ctype_CFAllocatorRef in argument to _Cfunc_CFStringCreateWithBytes
go/src/github.com/segmentio/aws-okta/vendor/github.com/keybase/go-keychain/corefoundation_go110.go:133: cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
/cc ZHF #45961
(cherry picked from commit b814881394)
In file included from qt/bitcoin.cpp:9:
./qt/bitcoingui.h:14:10: fatal error: 'QLabel' file not found
#include <QLabel>
^~~~~~~~
1 error generated.
/cc ZHF #45961
(cherry picked from commit 0dff3d5c73)
Could be caused by our older 10.10.5 CoreFoundation.
go/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_fsevents_cgo.go:51:216:
cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
go/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_fsevents_cgo.go:162:47:
cannot use nil as type _Ctype_CFAllocatorRef in argument to
_Cfunc_CFStringCreateWithCStringNoCopy
go/src/github.com/ethereum/go-ethereum/vendor/github.com/rjeczalik/notify/watcher_fsevents_cgo.go:163:225:
cannot use nil as type _Ctype_CFAllocatorRef in argument to func literal
/cc ZHF #45961
(cherry picked from commit e7ecf593e2)
Started to fail since 1.12.1.
Undefined symbols for architecture x86_64:
"boost::chrono::steady_clock::now()", referenced from:
pinger::start_send() in ping.o
asio::detail::timer_queue<asio::detail::chrono_time_traits<boost::chrono::steady_clock, asio::wait_traits<boost::chrono::steady_clock> > >::wait_duration_msec(long) const in ping.o
asio::detail::timer_queue<asio::detail::chrono_time_traits<boost::chrono::steady_clock, asio::wait_traits<boost::chrono::steady_clock> > >::wait_duration_usec(long) const in ping.o
asio::detail::timer_queue<asio::detail::chrono_time_traits<boost::chrono::steady_clock, asio::wait_traits<boost::chrono::steady_clock> > >::get_ready_timers(asio::detail::op_queue<asio::detail::scheduler_operation>&) in ping.o
pinger::handle_receive(unsigned long) in p
/cc ZHF #45961
(cherry picked from commit 80a4e48e45)
The darwin build fails and it's probably not particularly useful there.
utils.c:33:19: error: use of undeclared identifier 'CLOCK_MONOTONIC'
clock_gettime(CLOCK_MONOTONIC, &t);
/cc ZHF #45961
(cherry picked from commit 139eb11a6d)
- our version is from 2015
- it doesn't build
- upstream project is dead, last release 2012, last commit Oct 2016.
- used by only 1 nixpkgs package: `boo`, marked broken since 2016.
(cherry picked from commit dc84de59d1)
Added a wrapper package that creates symlinks to OpenSSL libraries with
the actual versions, because we have 1.0.2 with .so-suffix .1.0.0 and
cl-async-ssl is unhappy because of that.
I continue to dislike cl-postgres and simple-date upstream packaging.
This fixes cl-libuv and dependent packages.
Progress on #45960
(cherry picked from commit 5048b0d1f9)
See https://hydra.nixos.org/build/80727495
`pants' works with requests==2.19 to build successfully and `nixpkgs' currently uses `requests==2.19.1`.
Patching the version constraint in `setup.py' accordingly fixes the problem.
Addresses #45960
(cherry picked from commit 46d1166b80)
This hopefully makes patchShebangs respect cross compilation. It
introduces the concept of the HOST_PATH. Nothing is ever executed on
it but instead used as a way to get the proper path using ‘command
-v’. Needs more testing.
/cc @ericson2314 @dtzwill
Fixes#33956Fixes#21138
(Modified backport of f06942327a. See
previous commit to understand the differences between this and the
original.)
b785d4813e introduced breakage in Qt
modules for 5.6 and 5.9, especially visible is Qt Webkit.
This was manifested by having a non-sensical build log where it is using
the top-level `src` attribute as source instead of Qt Webkit's own
source.
Were it not for the `src` top-level attribute (which is a legit
package), the error would have been made obvious by passing `null` to
`src`.
This partily reverts newly introduced way `src` can be passed to a
qtModule, instead relying on extending the `srcs` attrset.
For ZHF #45960
(cherry picked from commit cb380983ed)
While building the tests LD gets called with -mmacosx-version-min=10.10
which is a CC flag, causing the build to fail with LD=ld. This is
pretty common with perl packages.
/cc ZHF #45961
(cherry picked from commit dc6b36b994)
1. Simplify the command by reading directly from /etc/machine-id which
is already a random, lower-case hex string
2. Previously, the command output could be too short because of missing
leading digits. This is now fixed.
(cherry picked from commit 7a42623c23)
See https://hydra.nixos.org/build/80828287
Moves `mahotas` out of `python-packages.nix` into its own file and fixes
broken test cases by skipping them using nosetest's `@nottest`
annotation.
These tests broke from time to time in a sandbox and are therefore
considered impure.
Addresses #45960
(cherry picked from commit ee897b64ff)
See https://hydra.nixos.org/build/80705916
To fix the linux build the patch `bsm-add-audit_token_to_pid.patch`
mustn't be applied during Linux builds, only for Darwin as it's an
Apple-only fix.
The compiler failure occurred because `audit_token_t` is part of
`<mach/mach.h>` which is not available on Linux.
Addresses #45960
(cherry picked from commit 080b2fdd8c)
The examples fail with an opengl related issue:
Undefined symbols for architecture x86_64:
"SimpleOpenGL3App::SimpleOpenGL3App(char const*, int, int, bool)", referenced from:
_main in main_opengl_single_example.o
"_useShadowMap", referenced from:
GL_ShapeDrawer::drawScene(btDiscreteDynamicsWorld const*, bool, int) in GL_ShapeDrawer.o
ld: symbol(s) not found for architecture x86_64
And the tests need an extra dependencly, possibley related to
https://github.com/bulletphysics/bullet3/issues/819
ld: library not found for -lBussIK
/cc ZHF #45961
(cherry picked from commit 939debc7a4)
Currently one would need to manually run the command to generate
completion and insert it into `.bashrc`/`.zshrc` to get the
autocompletion to work. This patch will automatically generate
both docs and save them to correct position so it should continue
to work even if user changes the shell at a later stage.
(cherry picked from commit d23c357f1c)
earlyoom's Makefile attempts to detect the current version using git,
but we don't keep .git in its source, so this fails. We can however
set the VERSION environment variable to override this, as we now do.
(cherry picked from commit ac679e721f)
They all depend on safe-money-0.7, but hackage2nix made them incorrectly
depend on safe-money-0.6
We should be able to remove this 'hack' as soon as the new Stackage LTS
comes out, as safe-money-0.7 will then be the default version.
The original build broke with the following linker issue:
```
CXXLD _PythonMagick.la
/nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/ld: cannot find -l-L/nix/store/4gh6ynzsd5ndx37hmkl62xa8z30k43y1-imagemagick-6.9.9-34/lib
collect2: error: ld returned 1 exit status
```
This happens since `BOOST_PYTHON_LIB` wasn't set properly, however
`_PythonMagick.la` was linked with `-l$(BOOST_PYTHON_LIB)
$(MAGICK_LIBS)`. With an empty `BOOST_PYTHON_LIB` the linker got
confused.
To work around this, the `boost` library directory needs to be specified
explicitly. To ensure that the changes take effect, the original
`configure` script shipped with `$src` needs to be removed and recreated
using the `autoreconfHook`.
Additionally the `imagemagick` license (https://spdx.org/licenses/ImageMagick.html)
needs to be added to `lib/licenses.nix` to document the proper license
of `pythonmagick` in the meta section.
(cherry picked from commit 6d6cbd316d)
This fixes an issue where the strongswan NM client is not able to
connect to a VPN. By default it tries to load the trust CA from
/usr/share/ca-certificates which doesn't exist in NixOS and most modern
distros.
See debian-related issue:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=835095
(cherry picked from commit 71e6dfdaea)
groff may not be installed system-wide or may be outside cgit's path
(e.g. when running cgit using uwsgi). As a result, the man2html filter
fails. Replace groff by its full path in man2html.
(cherry picked from commit bbcf5eb130)
This seems like a known issue as other distributions (ArchLinux here)
have patches fixing the issue.
This hopefully fixes more than one dependant builds for ZHF 18.09.
(cherry picked from commit 1ac912bf1b)
For update-mime-database to work, you must have to have some mime
packages installed. In some DEs like XFCE this is not guaranteed to
happen. In that case just skip the update-mime-database call.
Fixes#46162
Documentation-related tests failed, the docs didn't build correctly
because of missing dependencies sphinx and manuel. Add them.
(cherry picked from commit 113f4d7b44)
Some tests use O_DIRECT which is filesystem dependent and fails in a
sandbox as well as on some filesystems without sandboxing.
Patch out O_DIRECT and disable the 4 test cases that still fail in a
sandbox. See discussion in #46151.
(cherry picked from commit 8c6cf3de3d)
See https://hydra.nixos.org/build/80999410
Without a sandbox the `checkPhase` breaks with the following error:
```
/nix/store/i6vl5lwlz5jbkg4r6p340dwmj6fha3xq-stdenv-linux/setup: bin/jbake: /usr/bin/env: bad interpreter: No such file or directory
jbake did not return correct version
```
Running `patchShebangs` before fixes the issue.
Addresses #45960
(cherry picked from commit 6960a96a0c)
See https://hydra.nixos.org/build/80998335.
Upstream doesn't support QT 5.11 ATM which broke compilation:
```
src/dialogs/savedialog.cpp: In constructor ‘SaveDialog::SaveDialog(QWidget*, Qt::WindowFlags)’:
src/dialogs/savedialog.cpp:37:34: error: invalid use of incomplete type ‘class QButtonGroup’
group = new QButtonGroup(this);
```
The Arch community recommends to use an older QT version to fix
this (https://aur.archlinux.org/packages/chessx/).
Furthermore the `QT_PLUGIN_PATH` wasn't set properly which broke the
runtime since QT coudln't find the `xcb` plugin:
```
qt.qpa.plugin: Could not find the Qt platform plugin "xcb" in ""
This application failed to start because no Qt platform plugin could be initialized.
Reinstalling the application may fix this problem.
```
Finally, some minor style fixes were made for consistent indentation.
Addresses #45960
(cherry picked from commit 37032db583)
Fixes the build for `python3Packages.trio' for the next ZHF iteration.
Please refer to the Hydra build for further reference: https://hydra.nixos.org/build/80617356
`python3Packages.sniffio` is needed for the build, otherwise the build
aborts with an error like this:
```
Could not find a version that satisfies the requirement sniffio (from trio==0.6.0) (from versions: )
No matching distribution found for sniffio (from trio==0.6.0)
```
See #45960
(cherry picked from commit 218ac50fd4)
Update the model list to those models that are compatibly with the
latest spaCy version:
https://github.com/explosion/spacy-models/blob/master/compatibility.json
The updated model file evaluates to the following packages:
python2Packages.spacy_models.de_core_news_sm
python2Packages.spacy_models.en_core_web_lg
python2Packages.spacy_models.en_core_web_md
python2Packages.spacy_models.en_core_web_sm
python2Packages.spacy_models.en_vectors_web_lg
python2Packages.spacy_models.es_core_news_md
python2Packages.spacy_models.es_core_news_sm
python2Packages.spacy_models.fr_core_news_md
python2Packages.spacy_models.fr_core_news_sm
python2Packages.spacy_models.it_core_news_sm
python2Packages.spacy_models.nl_core_news_sm
python2Packages.spacy_models.pt_core_news_sm
python2Packages.spacy_models.xx_ent_wiki_sm
python3Packages.spacy_models.de_core_news_sm
python3Packages.spacy_models.en_core_web_lg
python3Packages.spacy_models.en_core_web_md
python3Packages.spacy_models.en_core_web_sm
python3Packages.spacy_models.en_vectors_web_lg
python3Packages.spacy_models.es_core_news_md
python3Packages.spacy_models.es_core_news_sm
python3Packages.spacy_models.fr_core_news_md
python3Packages.spacy_models.fr_core_news_sm
python3Packages.spacy_models.it_core_news_sm
python3Packages.spacy_models.nl_core_news_sm
python3Packages.spacy_models.pt_core_news_sm
python3Packages.spacy_models.xx_ent_wiki_sm
(cherry picked from commit d261df5fc1)
Fixing build issue. Additionally fetching from github becuase pypi
packge does not come with README which is a buildtime dependency.
(cherry picked from commit b5b48c0a91)
A shared exported guard `__NIXOS_SET_ENVIRONMENT_DONE` is introduced that can
be used to prevent child shells from sourcing `system.build.setEnvironment`
the second time.
This fixes e.g. `nix run derivation` when run from e.g. ZSH through the console or
ssh. Before this Bash would resource the common environment resetting the `PATH`
environment variable.
We also export `system.build.setEnvironment` to `/etc/set-environment` making it
easy to reset the common environment with `. /etc/set-environment` when
needed and to grep for environment variables in `/etc` (which was the
motivation of #30418).
This reverts changes made in b00a3fc6fd
(the original #30418).
(cherry picked from commit d273db48c6)
`all-functions` corresponds to `-fprof-auto` which places an SCC on every binding. It is well known that SCCs hinder GHC from doing its optimization magic and really slows down profiled code to a point where the profiling reports are completely skewed towards things that were completely optimized away in production settings. Concretely this shows up with things like lenses which do not carry runtime overhead when properly simplified.
`exported-functions` corresponds to GHCs `-fprof-auto-exported` which doesn't put SCCs on `INLINE`d code and in turn doesn't influence simplification of this basic but important stuff.
(cherry picked from commit 3c70c4c6af)
For each new release, the upstream developers of Gildas move the
source code of the previous release to a different directory. Add this
directory to the list of url to avoid build failure at each new
release.
(cherry picked from commit 5b58c6ac76)
Also 2018.8.10 does not build in previous state because auto updating
from pypi did not account for a new requirement `networkx`. This has
been added.
(cherry picked from commit 63cf879d6c)
This fixes an issue with shells like fish that are not fully POSIX
compliant. The syntax `ENV=val cmd' doesn't work properly in there.
This issue has been addressed in #45932 and #45945, however it has been
recommended to use a single shell (`stdenv.shell' which is either
`bash' or `sh') to significantly reduce the maintenance overload in the
future.
See https://github.com/NixOS/nixpkgs/issues/45897#issuecomment-417923464Fixes#45897
/cc @FRidh @xaverdh @etu
(cherry picked from commit df05618f2a)
A sporadic failure occured on Hydra because a request was sent
to smtpd after the systemd unit was started, but before the daemon
was actually listening. Fix by checking for open ports first.
(cherry picked from commit f70dc57ad3)
thinc uses Accelerate on macOS as a blas library. Add the necessary
dependencies to allow building on unsandboxed Darwin machines.
(cherry picked from commit 348a153781)
- mxnet now builds the open source MKL DNN library. However, this
was not included with the build.
- mxnet builds its own OpenMP library, even when OpenMP is provided
by the system. This leads to problems, so remove the included
OpenMP implementation:
https://github.com/apache/incubator-mxnet/pull/12160
(cherry picked from commit 590ce7d1ae)
Tests seem to fail on all recent versions even outside of our build
system. The upstream CI tests seem to somehow succeed but I wasn't able
to reproduce that locally. For not it is probably best to disable them.
(cherry picked from commit d814c98ec8)
A sporadic failure occured on Hydra because a request was sent
to the daemon after the systemd unit was started, but before the
daemon was actually listening. Fix by checking for open port first.
(cherry picked from commit 62086c6be6)
- Follow the same logic for finding boost-python as for Linux.
- Set DYLD_LIBRARY_PATH so that ctypes find_library can find
boost-python.
- Do not use '-mt' suffix when linking against boost-serialization
and boost-program_options on Darwin.
(cherry picked from commit fe7e89ba3b)
Out of bounds vulnerability in versions up to 0.62.
Generally, we use a newer poppler version but some pkgs still depend on
0.61. Patch named in https://nvd.nist.gov/vuln/detail/CVE-2018-13988.
(cherry picked from commit 542a86037d)
This adds a release notes entry to make users (and especially
developers) aware so they no longer need to use </para><para> in option
descriptions as this is now done automatically on every two consecutive
newlines.
More details can be found in the commit message of f865d0feab.
Signed-off-by: aszlig <aszlig@nix.build>
(cherry picked from commit 4652f2f87e)
What annoyed me for a long time was the fact, that in order to break
into a new paragraph, you need to insert </para><para> in the
description attribute of an option.
Now we will automatically create <para/> elements for every block that
is separated by two consecutive newlines.
I first tried to do this within options-to-docbook.xsl, but it turns
out[1] that this isn't directly possible with XSLT 1.0, so I added
another XSLT file that postprocesses the option descriptions that are
now enclosed in <nixos:option-description/> by options-to-docbook.xsl.
The splitting itself is a bit more involved, because we can't simply
split on every \n\n because we'd also split text nodes of elements, for
example:
<screen><![CDATA[
one line
another one
]]></screen>
This would create one <para/> element for "one line" and another for
"another line", which we obviously don't want because <screen/> is used
to display verbatim contents of what a user is seeing on the screen.
So what we do instead is splitting *only* the top-level text nodes
within the outermost <para/> and leave all elements as-is. If there are
more than one <para/> elements at the top-level, we simply don't process
it at all, because the description then already contains </para><para>.
https://www.mhonarc.org/archive/html/xsl-list/2012-09/msg00319.html
Signed-off-by: aszlig <aszlig@nix.build>
Cc: @edolstra, @domenkozar
(cherry picked from commit f865d0feab)
2018-09-02 18:31:36 -04:00
16239 changed files with 459426 additions and 716270 deletions
<!-- Nixpkgs has a lot of new incoming Pull Requests, but not enough people to review this constant stream. Even if you aren't a committer, we would appreciate reviews of other PRs, especially simple ones like package updates. Just testing the relevant package/service and leaving a comment saying what you tested, how you tested it and whether it worked would be great. List of open PRs: <https://github.com/NixOS/nixpkgs/pulls>, for more about reviewing contributions: <https://hydra.nixos.org/job/nixpkgs/trunk/manual/latest/download/1/nixpkgs/manual.html#sec-reviewing-contributions>. Reviewing isn't mandatory, but it would help out a lot and reduce the average time-to-merge for all of us. Thanks a lot if you do! -->
###### Motivation for this change
@@ -12,12 +11,10 @@
- [ ] macOS
- [ ] other Linux distributions
- [ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- [ ] Tested compilation of all pkgs that depend on this change using `nix-shell -p nix-review --run "nix-review wip"`
- [ ] Tested compilation of all pkgs that depend on this change using `nix-shell -p nox --run "nox-review wip"`
- [ ] Tested execution of all binary files (usually in `./result/bin/`)
- [ ] Determined the impact on package closure size (by running `nix path-info -S` before and after)
- [ ] Ensured that relevant documentation is up to date
Use 2 spaces of indentation per indentation level in Nix expressions, 4 spaces in shell scripts.
Use 2 spaces of indentation per indentation level in Nix expressions, 4
spaces in shell scripts.
</para>
</listitem>
<listitem>
<para>
Do not use tab characters, i.e. configure your editor to use soft tabs. For instance, use <literal>(setq-default indent-tabs-mode nil)</literal> in Emacs. Everybody has different tab settings so it’s asking for trouble.
Do not use tab characters, i.e. configure your editor to use soft tabs.
For instance, use <literal>(setq-default indent-tabs-mode nil)</literal>
in Emacs. Everybody has different tab settings so it’s asking for
trouble.
</para>
</listitem>
<listitem>
<para>
Use <literal>lowerCamelCase</literal> for variable names, not<literal>UpperCamelCase</literal>. Note, this rule does not apply to package attribute names, which instead follow the rules in <xreflinkend="sec-package-naming"/>.
Use <literal>lowerCamelCase</literal> for variable names, not
<literal>UpperCamelCase</literal>. Note, this rule does not apply to
package attribute names, which instead follow the rules in
<xreflinkend="sec-package-naming"/>.
</para>
</listitem>
<listitem>
@@ -45,33 +52,29 @@ foo { arg = ...; }
</listitem>
<listitem>
<para>
In attribute sets or lists that span multiple lines, the attribute names or list elements should be aligned:
In attribute sets or lists that span multiple lines, the attribute names
or list elements should be aligned:
<programlisting>
# A long list.
list = [
elem1
elem2
elem3
];
list =
[ elem1
elem2
elem3
];
# A long attribute set.
attrs =
{ attr1 = short_expr;
attr2 =
if true then big_expr else big_expr;
};
# Alternatively:
attrs = {
attr1 = short_expr;
attr2 =
if true then big_expr else big_expr;
};
# Combined
listOfAttrs = [
{
attr1 = 3;
attr2 = "fff";
}
{
attr1 = 5;
attr2 = "ggg";
}
];
</programlisting>
</para>
</listitem>
@@ -89,7 +92,8 @@ attrs = { x = 1280; y = 1024; };
</listitem>
<listitem>
<para>
Breaking in the middle of a function argument can give hard-to-read code, like
Breaking in the middle of a function argument can give hard-to-read code,
like
<programlisting>
someFunction { x = 1280;
y = 1024; } otherArg
@@ -114,7 +118,8 @@ in someFunction res otherArg yetAnotherArg
</listitem>
<listitem>
<para>
The bodies of functions, asserts, and withs are not indented to prevent a lot of superfluous indentation levels, i.e.
The bodies of functions, asserts, and withs are not indented to prevent a
lot of superfluous indentation levels, i.e.
<programlisting>
{ arg1, arg2 }:
assert system == "i686-linux";
@@ -146,7 +151,8 @@ stdenv.mkDerivation { ...
</listitem>
<listitem>
<para>
Functions should list their expected arguments as precisely as possible. That is, write
Functions should list their expected arguments as precisely as possible.
@@ -160,7 +166,9 @@ args: with args; <replaceable>...</replaceable>
</programlisting>
</para>
<para>
For functions that are truly generic in the number of arguments (such as wrappers around <varname>mkDerivation</varname>) that have some required arguments, you should write them using an <literal>@</literal>-pattern:
For functions that are truly generic in the number of arguments (such as
wrappers around <varname>mkDerivation</varname>) that have some required
arguments, you should write them using an <literal>@</literal>-pattern:
The key words <emphasis>must</emphasis>, <emphasis>must not</emphasis>, <emphasis>required</emphasis>, <emphasis>shall</emphasis>, <emphasis>shall not</emphasis>, <emphasis>should</emphasis>, <emphasis>should not</emphasis>, <emphasis>recommended</emphasis>, <emphasis>may</emphasis>, and <emphasis>optional</emphasis> in this section are to be interpreted as described in <linkxlink:href="https://tools.ietf.org/html/rfc2119">RFC 2119</link>. Only <emphasis>emphasized</emphasis> words are to be interpreted in this way.
</para>
<para>
In Nixpkgs, there are generally three different names associated with a package:
In Nixpkgs, there are generally three different names associated with a
package:
<itemizedlist>
<listitem>
<para>
The <varname>name</varname> attribute of the derivation (excluding the version part). This is what most users see, in particular when using <command>nix-env</command>.
The <varname>name</varname> attribute of the derivation (excluding the
version part). This is what most users see, in particular when using
<command>nix-env</command>.
</para>
</listitem>
<listitem>
<para>
The variable name used for the instantiated package in<filename>all-packages.nix</filename>, and when passing it as a dependency to other functions. Typically this is called the <emphasis>package attribute name</emphasis>. This is what Nix expression authors see. It can also be used when installing using <command>nix-env -iA</command>.
The variable name used for the instantiated package in
<filename>all-packages.nix</filename>, and when passing it as a
dependency to other functions. Typically this is called the
<emphasis>package attribute name</emphasis>. This is what Nix expression
authors see. It can also be used when installing using <command>nix-env
Most of the time, these are the same. For instance, the package<literal>e2fsprogs</literal> has a <varname>name</varname> attribute <literal>"e2fsprogs-<replaceable>version</replaceable>"</literal>, is bound to the variable name <varname>e2fsprogs</varname> in <filename>all-packages.nix</filename>, and the Nix expression is in <filename>pkgs/os-specific/linux/e2fsprogs/default.nix</filename>.
Most of the time, these are the same. For instance, the package
<literal>e2fsprogs</literal> has a <varname>name</varname> attribute
<literal>"e2fsprogs-<replaceable>version</replaceable>"</literal>, is bound
to the variable name <varname>e2fsprogs</varname> in
<filename>all-packages.nix</filename>, and the Nix expression is in
The <literal>name</literal> attribute <emphasis>should</emphasis> be identical to the upstream package name.
Generally, try to stick to the upstream package name.
</para>
</listitem>
<listitem>
<para>
The <literal>name</literal> attribute <emphasis>must not</emphasis> contain uppercase letters — e.g.,<literal>"mplayer-1.0rc2"</literal> instead of <literal>"MPlayer-1.0rc2"</literal>.
Don’t use uppercase letters in the<literal>name</literal> attribute
— e.g., <literal>"mplayer-1.0rc2"</literal> instead of
<literal>"MPlayer-1.0rc2"</literal>.
</para>
</listitem>
<listitem>
<para>
The version part of the <literal>name</literal> attribute<emphasis>must</emphasis> start with a digit (following a dash) — e.g., <literal>"hello-0.3.1rc2"</literal>.
The version part of the <literal>name</literal> attribute
<emphasis>must</emphasis> start with a digit (following a dash) — e.g.,
<literal>"hello-0.3.1rc2"</literal>.
</para>
</listitem>
<listitem>
<para>
If a package is not a release but a commit from a repository, then the version part of the name <emphasis>must</emphasis> be the date of that (fetched) commit. The date <emphasis>must</emphasis> be in <literal>"YYYY-MM-DD"</literal> format. Also append <literal>"unstable"</literal> to the name - e.g., <literal>"pkgname-unstable-2014-09-23"</literal>.
If a package is not a release but a commit from a repository, then the
version part of the name <emphasis>must</emphasis> be the date of that
(fetched) commit. The date must be in <literal>"YYYY-MM-DD"</literal>
format. Also append <literal>"unstable"</literal> to the name - e.g.,
<literal>"pkgname-unstable-2014-09-23"</literal>.
</para>
</listitem>
<listitem>
<para>
Dashes in the package name <emphasis>should</emphasis> be preserved in new variable names, rather than converted to underscores or camel cased — e.g., <varname>http-parser</varname> instead of <varname>http_parser</varname> or <varname>httpParser</varname>. The hyphenated style is preferred in all three package names.
Dashes in the package name should be preserved in new variable names,
rather than converted to underscores or camel cased — e.g.,
<varname>http-parser</varname> instead of <varname>http_parser</varname>
or <varname>httpParser</varname>. The hyphenated style is preferred in
all three package names.
</para>
</listitem>
<listitem>
<para>
If there are multiple versions of a package, this <emphasis>should</emphasis> be reflected in the variable names in <filename>all-packages.nix</filename>, e.g. <varname>json-c-0-9</varname> and <varname>json-c-0-11</varname>. If there is an obvious “default” version, make an attribute like <literal>json-c = json-c-0-9;</literal>. See also <xreflinkend="sec-versioning"/>
If there are multiple versions of a package, this should be reflected in
the variable names in <filename>all-packages.nix</filename>, e.g.
<varname>json-c-0-9</varname> and <varname>json-c-0-11</varname>. If
there is an obvious “default” version, make an attribute like
Names of files and directories should be in lowercase, with dashes between words — not in camel case. For instance, it should be <filename>all-packages.nix</filename>, not <filename>allPackages.nix</filename> or <filename>AllPackages.nix</filename>.
Names of files and directories should be in lowercase, with dashes between
words — not in camel case. For instance, it should be
<filename>all-packages.nix</filename>, not
<filename>allPackages.nix</filename> or
<filename>AllPackages.nix</filename>.
</para>
<sectionxml:id="sec-hierarchy">
<title>Hierarchy</title>
<para>
Each package should be stored in its own directory somewhere in the<filename>pkgs/</filename> tree, i.e. in <filename>pkgs/<replaceable>category</replaceable>/<replaceable>subcategory</replaceable>/<replaceable>...</replaceable>/<replaceable>pkgname</replaceable></filename>. Below are some rules for picking the right category for a package. Many packages fall under several categories; what matters is the <emphasis>primary</emphasis> purpose of a package. For example, the <literal>libxml2</literal> package builds both a library and some tools; but it’s a library foremost, so it goes under <filename>pkgs/development/libraries</filename>.
Each package should be stored in its own directory somewhere in the
Because every version of a package in Nixpkgs creates a potential maintenance burden, old versions of a package should not be kept unless there is a good reason to do so. For instance, Nixpkgs contains several versions of GCC because other packages don’t build with the latest version of GCC. Other examples are having both the latest stable and latest pre-release version of a package, or to keep several major releases of an application that differ significantly in functionality.
Because every version of a package in Nixpkgs creates a potential
maintenance burden, old versions of a package should not be kept unless
there is a good reason to do so. For instance, Nixpkgs contains several
versions of GCC because other packages don’t build with the latest
version of GCC. Other examples are having both the latest stable and latest
pre-release version of a package, or to keep several major releases of an
application that differ significantly in functionality.
</para>
<para>
If there is only one version of a package, its Nix expression should be named <filename>e2fsprogs/default.nix</filename>. If there are multiple versions, this should be reflected in the filename, e.g. <filename>e2fsprogs/1.41.8.nix</filename> and <filename>e2fsprogs/1.41.9.nix</filename>. The version in the filename should leave out unnecessary detail. For instance, if we keep the latest Firefox 2.0.x and 3.5.x versions in Nixpkgs, they should be named <filename>firefox/2.0.nix</filename> and <filename>firefox/3.5.nix</filename>, respectively (which, at a given point, might contain versions <literal>2.0.0.20</literal> and <literal>3.5.4</literal>). If a version requires many auxiliary files, you can use a subdirectory for each version, e.g. <filename>firefox/2.0/default.nix</filename> and <filename>firefox/3.5/default.nix</filename>.
If there is only one version of a package, its Nix expression should be
named <filename>e2fsprogs/default.nix</filename>. If there are multiple
versions, this should be reflected in the filename, e.g.
<filename>e2fsprogs/1.41.8.nix</filename> and
<filename>e2fsprogs/1.41.9.nix</filename>. The version in the filename
should leave out unnecessary detail. For instance, if we keep the latest
Firefox 2.0.x and 3.5.x versions in Nixpkgs, they should be named
<filename>firefox/2.0.nix</filename> and
<filename>firefox/3.5.nix</filename>, respectively (which, at a given
point, might contain versions <literal>2.0.0.20</literal> and
<literal>3.5.4</literal>). If a version requires many auxiliary files, you
can use a subdirectory for each version, e.g.
<filename>firefox/2.0/default.nix</filename> and
<filename>firefox/3.5/default.nix</filename>.
</para>
<para>
All versions of a package <emphasis>must</emphasis> be included in<filename>all-packages.nix</filename> to make sure that they evaluate correctly.
All versions of a package <emphasis>must</emphasis> be included in
<filename>all-packages.nix</filename> to make sure that they evaluate
There are multiple ways to fetch a package source in nixpkgs. The general guideline is that you should package reproducible sources with a high degree of availability. Right now there is only one fetcher which has mirroring support and that is <literal>fetchurl</literal>. Note that you should also prefer protocols which have a corresponding proxy environment variable.
There are multiple ways to fetch a package source in nixpkgs. The general
guideline is that you should package sources with a high degree of
availability. Right now there is only one fetcher which has mirroring
support and that is <literal>fetchurl</literal>. Note that you should also
prefer protocols which have a corresponding proxy environment variable.
</para>
<para>
You can find many source fetch helpers in<literal>pkgs/build-support/fetch*</literal>.
You can find many source fetch helpers in
<literal>pkgs/build-support/fetch*</literal>.
</para>
<para>
In the file <literal>pkgs/top-level/all-packages.nix</literal> you can find fetch helpers, these have names on the form <literal>fetchFrom*</literal>. The intention of these are to provide snapshot fetches but using the same api as some of the version controlled fetchers from <literal>pkgs/build-support/</literal>. As an example going from bad to good:
In the file <literal>pkgs/top-level/all-packages.nix</literal> you can find
fetch helpers, these have names on the form <literal>fetchFrom*</literal>.
The intention of these are to provide snapshot fetches but using the same
api as some of the version controlled fetchers from
<literal>pkgs/build-support/</literal>. As an example going from bad to
Find the value to put as <literal>sha256</literal> by running <literal>nix run -f '<nixpkgs>' nix-prefetch-github -c nix-prefetch-github --rev 1f795f9f44607cc5bec70d1300150bfefcef2aae NixOS nix</literal> or <literal>nix-prefetch-url --unpack https://github.com/NixOS/nix/archive/1f795f9f44607cc5bec70d1300150bfefcef2aae.tar.gz</literal>.
</para>
</listitem>
</itemizedlist>
</para>
</section>
<sectionxml:id="sec-source-hashes">
<title>Obtaining source hash</title>
<para>
Preferred source hash type is sha256. There are several ways to get it.
</para>
<orderedlist>
<listitem>
<para>
Prefetch URL (with <literal>nix-prefetch-<replaceable>XXX</replaceable><replaceable>URL</replaceable></literal>, where <replaceable>XXX</replaceable> is one of <literal>url</literal>, <literal>git</literal>, <literal>hg</literal>, <literal>cvs</literal>, <literal>bzr</literal>, <literal>svn</literal>). Hash is printed to stdout.
</para>
</listitem>
<listitem>
<para>
Prefetch by package source (with <literal>nix-prefetch-url '<nixpkgs>' -A <replaceable>PACKAGE</replaceable>.src</literal>, where <replaceable>PACKAGE</replaceable> is package attribute name). Hash is printed to stdout.
</para>
<para>
This works well when you've upgraded existing package version and want to find out new hash, but is useless if package can't be accessed by attribute or package has multiple sources (<literal>.srcs</literal>, architecture-dependent sources, etc).
</para>
</listitem>
<listitem>
<para>
Upstream provided hash: use it when upstream provides <literal>sha256</literal> or <literal>sha512</literal> (when upstream provides <literal>md5</literal>, don't use it, compute <literal>sha256</literal> instead).
</para>
<para>
A little nuance is that <literal>nix-prefetch-*</literal> tools produce hash encoded with <literal>base32</literal>, but upstream usually provides hexadecimal (<literal>base16</literal>) encoding. Fetchers understand both formats. Nixpkgs does not standardize on any one format.
</para>
<para>
You can convert between formats with nix-hash, for example:
Extracting hash from local source tarball can be done with <literal>sha256sum</literal>. Use <literal>nix-prefetch-url file:///path/to/tarball </literal> if you want base32 hash.
</para>
</listitem>
<listitem>
<para>
Fake hash: set fake hash in package expression, perform build and extract correct hash from error Nix prints.
</para>
<para>
For package updates it is enough to change one symbol to make hash fake. For new packages, you can use <literal>lib.fakeSha256</literal>, <literal>lib.fakeSha512</literal> or any other fake hash.
</para>
<para>
This is last resort method when reconstructing source URL is non-trivial and <literal>nix-prefetch-url -A</literal> isn't applicable (for example, <linkxlink:href="https://github.com/NixOS/nixpkgs/blob/d2ab091dd308b99e4912b805a5eb088dd536adb9/pkgs/applications/video/kodi/default.nix#L73"> one of <literal>kodi</literal> dependencies</link>). The easiest way then would be replace hash with a fake one and rebuild. Nix build will fail and error message will contain desired hash.
</para>
<warning>
<para>
This method has security problems. Check below for details.
</para>
</warning>
</listitem>
</orderedlist>
<sectionxml:id="sec-source-hashes-security">
<title>Obtaining hashes securely</title>
<para>
Let's say Man-in-the-Middle (MITM) sits close to your network. Then instead of fetching source you can fetch malware, and instead of source hash you get hash of malware. Here are security considerations for this scenario:
</para>
<itemizedlist>
<listitem>
<para>
<literal>http://</literal> URLs are not secure to prefetch hash from;
</para>
</listitem>
<listitem>
<para>
hashes from upstream (in method 3) should be obtained via secure protocol;
</para>
</listitem>
<listitem>
<para>
<literal>https://</literal> URLs are secure in methods 1, 2, 3;
</para>
</listitem>
<listitem>
<para>
<literal>https://</literal> URLs are not secure in method 5. When obtaining hashes with fake hash method, TLS checks are disabled. So refetch source hash from several different networks to exclude MITM scenario. Alternatively, use fake hash method to make Nix error, but instead of extracting hash from error, extract <literal>https://</literal> URL and prefetch it with method 1.
</para>
</listitem>
</itemizedlist>
</section>
</section>
<sectionxml:id="sec-patches">
<title>Patches</title>
<para>
Patches available online should be retrieved using<literal>fetchpatch</literal>.
Patches available online should be retrieved using
<literal>fetchpatch</literal>.
</para>
<para>
@@ -858,7 +871,10 @@ patches = [
</para>
<para>
Otherwise, you can add a <literal>.patch</literal> file to the<literal>nixpkgs</literal> repository. In the interest of keeping our maintenance burden to a minimum, only patches that are unique to <literal>nixpkgs</literal> should be added in this way.
Otherwise, you can add a <literal>.patch</literal> file to the
<literal>nixpkgs</literal> repository. In the interest of keeping our
maintenance burden to a minimum, only patches that are unique to
<literal>nixpkgs</literal> should be added in this way.
Nix comes with certain defaults about what packages can and cannot be installed, based on a package's metadata. By default, Nix will prevent installation if any of the following criteria are true:
Nix comes with certain defaults about what packages can and cannot be
installed, based on a package's metadata. By default, Nix will prevent
installation if any of the following criteria are true:
</para>
<itemizedlist>
<listitem>
<para>
The package is thought to be broken, and has had its<literal>meta.broken</literal> set to <literal>true</literal>.
The package is thought to be broken, and has had its
<literal>meta.broken</literal> set to <literal>true</literal>.
</para>
</listitem>
<listitem>
<para>
The package isn't intended to run on the given system, as none of its<literal>meta.platforms</literal> match the given system.
The package isn't intended to run on the given system, as none of its
<literal>meta.platforms</literal> match the given system.
</para>
</listitem>
<listitem>
<para>
The package's <literal>meta.license</literal> is set to a license which is considered to be unfree.
The package's <literal>meta.license</literal> is set to a license which is
considered to be unfree.
</para>
</listitem>
<listitem>
<para>
The package has known security vulnerabilities but has not or can not be updated for some reason, and a list of issues has been entered in to the package's <literal>meta.knownVulnerabilities</literal>.
The package has known security vulnerabilities but has not or can not be
updated for some reason, and a list of issues has been entered in to the
Note that all this is checked during evaluation already, and the check includes any package that is evaluated. In particular, all build-time dependencies are checked. <literal>nix-env -qa</literal> will (attempt to) hide any packages that would be refused.
Note that all this is checked during evaluation already, and the check
includes any package that is evaluated. In particular, all build-time
dependencies are checked. <literal>nix-env -qa</literal> will (attempt to)
hide any packages that would be refused.
</para>
<para>
Each of these criteria can be altered in the nixpkgs configuration.
</para>
<para>
The nixpkgs configuration for a NixOS system is set in the<literal>configuration.nix</literal>, as in the following example:
The nixpkgs configuration for a NixOS system is set in the
<literal>configuration.nix</literal>, as in the following example:
<programlisting>
{
nixpkgs.config = {
@@ -42,10 +53,13 @@
};
}
</programlisting>
However, this does not allow unfree software for individual users. Their configurations are managed separately.
However, this does not allow unfree software for individual users. Their
configurations are managed separately.
</para>
<para>
A user's of nixpkgs configuration is stored in a user-specific configuration file located at <filename>~/.config/nixpkgs/config.nix</filename>. For example:
A user's of nixpkgs configuration is stored in a user-specific configuration
file located at <filename>~/.config/nixpkgs/config.nix</filename>. For
example:
<programlisting>
{
allowUnfree = true;
@@ -53,25 +67,31 @@
</programlisting>
</para>
<para>
Note that we are not able to test or build unfree software on Hydra due to policy. Most unfree licenses prohibit us from either executing or distributing the software.
Note that we are not able to test or build unfree software on Hydra due to
policy. Most unfree licenses prohibit us from either executing or
distributing the software.
</para>
<sectionxml:id="sec-allow-broken">
<title>Installing broken packages</title>
<para>
There are two ways to try compiling a package which has been marked as broken.
There are two ways to try compiling a package which has been marked as
broken.
</para>
<itemizedlist>
<listitem>
<para>
For allowing the build of a broken package once, you can use an environment variable for a single invocation of the nix tools:
For allowing the build of a broken package once, you can use an
environment variable for a single invocation of the nix tools:
For permanently allowing broken packages to be built, you may add<literal>allowUnsupportedSystem = true;</literal> to your user's configuration file, like this:
For permanently allowing broken packages to be built, you may add
<literal>allowUnsupportedSystem = true;</literal> to your user's
configuration file, like this:
<programlisting>
{
allowUnsupportedSystem = true;
@@ -108,29 +132,42 @@
</itemizedlist>
<para>
The difference between a package being unsupported on some system and being broken is admittedly a bit fuzzy. If a program <emphasis>ought</emphasis> to work on a certain platform, but doesn't, the platform should be included in <literal>meta.platforms</literal>, but marked as broken with e.g. <literal>meta.broken = !hostPlatform.isWindows</literal>. Of course, this begs the question of what "ought" means exactly. That is left to the package maintainer.
The difference between an a package being unsupported on some system and
being broken is admittedly a bit fuzzy. If a program
<emphasis>ought</emphasis> to work on a certain platform, but doesn't, the
platform should be included in <literal>meta.platforms</literal>, but marked
as broken with e.g. <literal>meta.broken =
!hostPlatform.isWindows</literal>. Of course, this begs the question of what
"ought" means exactly. That is left to the package maintainer.
</para>
</section>
<sectionxml:id="sec-allow-unfree">
<title>Installing unfree packages</title>
<para>
There are several ways to tweak how Nix handles a package which has been marked as unfree.
There are several ways to tweak how Nix handles a package which has been
marked as unfree.
</para>
<itemizedlist>
<listitem>
<para>
To temporarily allow all unfree packages, you can use an environment variable for a single invocation of the nix tools:
To temporarily allow all unfree packages, you can use an environment
variable for a single invocation of the nix tools:
It is possible to permanently allow individual unfree packages, while still blocking unfree packages by default using the <literal>allowUnfreePredicate</literal> configuration option in the user configuration file.
It is possible to permanently allow individual unfree packages, while
still blocking unfree packages by default using the
<literal>allowUnfreePredicate</literal> configuration option in the user
configuration file.
</para>
<para>
This option is a function which accepts a package as a parameter, and returns a boolean. The following example configuration accepts a package and always returns false:
This option is a function which accepts a package as a parameter, and
returns a boolean. The following example configuration accepts a package
and always returns false:
<programlisting>
{
allowUnfreePredicate = (pkg: false);
@@ -138,24 +175,25 @@
</programlisting>
</para>
<para>
For a more useful example, try the following. This configuration only allows unfree packages named flash player and visual studio code:
A more useful example, the following configuration allows only allows
It is also possible to whitelist and blacklist licenses that are specifically acceptable or not acceptable, using <literal>whitelistedLicenses</literal> and <literal>blacklistedLicenses</literal>, respectively.
It is also possible to whitelist and blacklist licenses that are
It is possible to permanently allow individual insecure packages, while still blocking other insecure packages by default using the <literal>permittedInsecurePackages</literal> configuration option in the user configuration file.
It is possible to permanently allow individual insecure packages, while
still blocking other insecure packages by default using the
<literal>permittedInsecurePackages</literal> configuration option in the
user configuration file.
</para>
<para>
The following example configuration permits the installation of the hypothetically insecure package <literal>hello</literal>, version <literal>1.2.3</literal>:
The following example configuration permits the installation of the
hypothetically insecure package <literal>hello</literal>, version
<literal>1.2.3</literal>:
<programlisting>
{
permittedInsecurePackages = [
@@ -208,13 +255,18 @@
</listitem>
<listitem>
<para>
It is also possible to create a custom policy around which insecure packages to allow and deny, by overriding the <literal>allowInsecurePredicate</literal> configuration option.
It is also possible to create a custom policy around which insecure
The <literal>allowInsecurePredicate</literal> option is a function which accepts a package and returns a boolean, much like <literal>allowUnfreePredicate</literal>.
The <literal>allowInsecurePredicate</literal> option is a function which
accepts a package and returns a boolean, much like
<literal>allowUnfreePredicate</literal>.
</para>
<para>
The following configuration example only allows insecure packages with very short names:
The following configuration example only allows insecure packages with
Note that <literal>permittedInsecurePackages</literal> is only checked if<literal>allowInsecurePredicate</literal> is not specified.
Note that <literal>permittedInsecurePackages</literal> is only checked if
<literal>allowInsecurePredicate</literal> is not specified.
</para>
</listitem>
</itemizedlist>
@@ -232,7 +285,10 @@
<title>Modify packages via <literal>packageOverrides</literal></title>
<para>
You can define a function called <varname>packageOverrides</varname> in your local <filename>~/.config/nixpkgs/config.nix</filename> to override Nix packages. It must be a function that takes pkgs as an argument and returns a modified set of packages.
You can define a function called <varname>packageOverrides</varname> in your
local <filename>~/.config/nixpkgs/config.nix</filename> to override nix
packages. It must be a function that takes pkgs as an argument and return
modified set of packages.
<programlisting>
{
packageOverrides = pkgs: rec {
@@ -249,7 +305,15 @@
<title>Build an environment</title>
<para>
Using <literal>packageOverrides</literal>, it is possible to manage packages declaratively. This means that we can list all of our desired packages within a declarative Nix expression. For example, to have <literal>aspell</literal>, <literal>bc</literal>, <literal>ffmpeg</literal>, <literal>coreutils</literal>, <literal>gdb</literal>, <literal>nixUnstable</literal>, <literal>emscripten</literal>, <literal>jq</literal>, <literal>nox</literal>, and <literal>silver-searcher</literal>, we could use the following in <filename>~/.config/nixpkgs/config.nix</filename>:
Using <literal>packageOverrides</literal>, it is possible to manage
packages declaratively. This means that we can list all of our desired
packages within a declarative Nix expression. For example, to have
To install it into our environment, you can just run <literal>nix-env -iA nixpkgs.myPackages</literal>. If you want to load the packages to be built from a working copy of <literal>nixpkgs</literal> you just run <literal>nix-env -f. -iA myPackages</literal>. To explore what's been installed, just look through <filename>~/.nix-profile/</filename>. You can see that a lot of stuff has been installed. Some of this stuff is useful some of it isn't. Let's tell Nixpkgs to only link the stuff that we want:
To install it into our environment, you can just run <literal>nix-env -iA
nixpkgs.myPackages</literal>. If you want to load the packages to be built
from a working copy of <literal>nixpkgs</literal> you just run
<literal>nix-env -f. -iA myPackages</literal>. To explore what's been
installed, just look through <filename>~/.nix-profile/</filename>. You can
see that a lot of stuff has been installed. Some of this stuff is useful
some of it isn't. Let's tell Nixpkgs to only link the stuff that we want:
<literal>pathsToLink</literal> tells Nixpkgs to only link the paths listed which gets rid of the extra stuff in the profile. <filename>/bin</filename> and <filename>/share</filename> are good defaults for a user environment, getting rid of the clutter. If you are running on Nix on MacOS, you may want to add another path as well, <filename>/Applications</filename>, that makes GUI apps available.
<literal>pathsToLink</literal> tells Nixpkgs to only link the paths listed
which gets rid of the extra stuff in the profile. <filename>/bin</filename>
and <filename>/share</filename> are good defaults for a user environment,
getting rid of the clutter. If you are running on Nix on MacOS, you may
want to add another path as well, <filename>/Applications</filename>, that
makes GUI apps available.
</para>
</section>
@@ -310,7 +363,13 @@
<title>Getting documentation</title>
<para>
After building that new environment, look through<filename>~/.nix-profile</filename> to make sure everything is there that we wanted. Discerning readers will note that some files are missing. Look inside <filename>~/.nix-profile/share/man/man1/</filename> to verify this. There are no man pages for any of the Nix tools! This is because some packages like Nix have multiple outputs for things like documentation (see section 4). Let's make Nix install those as well.
After building that new environment, look through
<filename>~/.nix-profile</filename> to make sure everything is there that
we wanted. Discerning readers will note that some files are missing. Look
inside <filename>~/.nix-profile/share/man/man1/</filename> to verify this.
There are no man pages for any of the Nix tools! This is because some
packages like Nix have multiple outputs for things like documentation (see
This provides us with some useful documentation for using our packages. However, if we actually want those manpages to be detected by man, we need to set up our environment. This can also be managed within Nix expressions.
This provides us with some useful documentation for using our packages.
However, if we actually want those manpages to be detected by man, we need
to set up our environment. This can also be managed within Nix expressions.
For this to work fully, you must also have this script sourced when you are logged in. Try adding something like this to your <filename>~/.profile</filename> file:
For this to work fully, you must also have this script sourced when you are
logged in. Try adding something like this to your
<filename>~/.profile</filename> file:
</para>
<screen>
@@ -385,10 +438,11 @@ if [ -d $HOME/.nix-profile/etc/profile.d ]; then
fi
done
fi
</screen>
</screen>
<para>
Now just run <literal>source $HOME/.profile</literal> and you can starting loading man pages from your environent.
Now just run <literal>source $HOME/.profile</literal> and you can starting
loading man pages from your environent.
</para>
</section>
@@ -396,23 +450,25 @@ fi
<title>GNU info setup</title>
<para>
Configuring GNU info is a little bit trickier than man pages. To work correctly, info needs a database to be generated. This can be done with some small modifications to our environment scripts.
Configuring GNU info is a little bit trickier than man pages. To work
correctly, info needs a database to be generated. This can be done with
some small modifications to our environment scripts.
if [ -x $out/bin/install-info -a -w $out/share/info ]; then
shopt -s nullglob
for i in $out/share/info/*.info $out/share/info/*.info.gz; do
$out/bin/install-info $i $out/share/info/dir
done
fi
if [ -x $out/bin/install-info -a -w $out/share/info ]; then
shopt -s nullglob
for i in $out/share/info/*.info $out/share/info/*.info.gz; do
$out/bin/install-info $i $out/share/info/dir
done
fi
'';
};
};
}
</screen>
</screen>
<para>
<literal>postBuild</literal> tells Nixpkgs to run a command after building the environment. In this case, <literal>install-info</literal> adds the installed info pages to <literal>dir</literal> which is GNU info's default root node. Note that <literal>texinfoInteractive</literal> is added to the environment to give the <literal>install-info</literal> command.
<literal>postBuild</literal> tells Nixpkgs to run a command after building
the environment. In this case, <literal>install-info</literal> adds the
installed info pages to <literal>dir</literal> which is GNU info's default
root node. Note that <literal>texinfoInteractive</literal> is added to the
environment to give the <literal>install-info</literal> command.
"Cross-compilation" means compiling a program on one machine for another type of machine. For example, a typical use of cross-compilation is to compile programs for embedded devices. These devices often don't have the computing power and memory to compile their own programs. One might think that cross-compilation is a fairly niche concern. However, there are significant advantages to rigorously distinguishing between build-time and run-time environments! Significant, because the benefits apply even when one is developing and deploying on the same machine. Nixpkgs is increasingly adopting the opinion that packages should be written with cross-compilation in mind, and nixpkgs should evaluate in a similar way (by minimizing cross-compilation-specific special cases) whether or not one is cross-compiling.
"Cross-compilation" means compiling a program on one machine for another
type of machine. For example, a typical use of cross compilation is to
compile programs for embedded devices. These devices often don't have the
computing power and memory to compile their own programs. One might think
that cross-compilation is a fairly niche concern, but there are advantages
to being rigorous about distinguishing build-time vs run-time environments
even when one is developing and deploying on the same machine. Nixpkgs is
increasingly adopting the opinion that packages should be written with
cross-compilation in mind, and nixpkgs should evaluate in a similar way (by
minimizing cross-compilation-specific special cases) whether or not one is
cross-compiling.
</para>
<para>
This chapter will be organized in three parts. First, it will describe the basics of how to package software in a way that supports cross-compilation. Second, it will describe how to use Nixpkgs when cross-compiling. Third, it will describe the internal infrastructure supporting cross-compilation.
This chapter will be organized in three parts. First, it will describe the
basics of how to package software in a way that supports cross-compilation.
Second, it will describe how to use Nixpkgs when cross-compiling. Third, it
will describe the internal infrastructure supporting cross-compilation.
<title>Packaging in a cross-friendly manner</title>
<sectionxml:id="ssec-cross-platform-parameters">
<sectionxml:id="sec-cross-platform-parameters">
<title>Platform parameters</title>
<para>
Nixpkgs follows the<link
xlink:href="https://gcc.gnu.org/onlinedocs/gccint/Configure-Terms.html">conventions of GNU autoconf</link>. We distinguish between 3 types of platforms when building a derivation: <wordasword>build</wordasword>, <wordasword>host</wordasword>, and <wordasword>target</wordasword>. In summary, <wordasword>build</wordasword> is the platform on which a package is being built, <wordasword>host</wordasword> is the platform on which it will run. The third attribute, <wordasword>target</wordasword>, is relevant only for certain specific compilers and build tools.
historical convention of GNU autoconf</link> of distinguishing between 3
types of platform: <wordasword>build</wordasword>,
<wordasword>host</wordasword>, and <wordasword>target</wordasword>. In
summary, <wordasword>build</wordasword> is the platform on which a package
is being built, <wordasword>host</wordasword> is the platform on which it
is to run. The third attribute, <wordasword>target</wordasword>, is
relevant only for certain specific compilers and build tools.
</para>
<para>
In Nixpkgs, these three platforms are defined as attribute sets under the names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, and <literal>targetPlatform</literal>. They are always defined as attributes in the standard environment. That means one can access them like:
In Nixpkgs, these three platforms are defined as attribute sets under the
The "build platform" is the platform on which a package is built. Once someone has a built package, or pre-built binary package, the build platform should not matter and can be ignored.
The "build platform" is the platform on which a package is built. Once
someone has a built package, or pre-built binary package, the build
platform should not matter and be safe to ignore.
</para>
</listitem>
</varlistentry>
@@ -48,7 +74,9 @@
</term>
<listitem>
<para>
The "host platform" is the platform on which a package will be run. This is the simplest platform to understand, but also the one with the worst name.
The "host platform" is the platform on which a package will be run. This
is the simplest platform to understand, but also the one with the worst
name.
</para>
</listitem>
</varlistentry>
@@ -58,23 +86,44 @@
</term>
<listitem>
<para>
The "target platform" attribute is, unlike the other two attributes, not actually fundamental to the process of building software. Instead, it is only relevant for compatibility with building certain specific compilers and build tools. It can be safely ignored for all other packages.
The "target platform" attribute is, unlike the other two attributes, not
actually fundamental to the process of building software. Instead, it is
only relevant for compatibility with building certain specific compilers
and build tools. It can be safely ignored for all other packages.
</para>
<para>
The build process of certain compilers is written in such a way that the compiler resulting from a single build can itself only produce binaries for a single platform. The task of specifying this single "target platform" is thus pushed to build time of the compiler. The root cause of this is that the compiler (which will be run on the host) and the standard library/runtime (which will be run on the target) are built by a single build process.
The build process of certain compilers is written in such a way that the
compiler resulting from a single build can itself only produce binaries
for a single platform. The task specifying this single "target platform"
is thus pushed to build time of the compiler. The root cause of this
mistake is often that the compiler (which will be run on the host) and
the the standard library/runtime (which will be run on the target) are
built by a single build process.
</para>
<para>
There is no fundamental need to think about a single target ahead of time like this. If the tool supports modular or pluggable backends, both the need to specify the target at build time and the constraint of having only a single target disappear. An example of such a tool is LLVM.
There is no fundamental need to think about a single target ahead of
time like this. If the tool supports modular or pluggable backends, both
the need to specify the target at build time and the constraint of
having only a single target disappear. An example of such a tool is
LLVM.
</para>
<para>
Although the existence of a "target platfom" is arguably a historical mistake, it is a common one: examples of tools that suffer from it are GCC, Binutils, GHC and Autoconf. Nixpkgs tries to avoid sharing in the mistake where possible. Still, because the concept of a target platform is so ingrained, it is best to support it as is.
Although the existence of a "target platfom" is arguably a historical
mistake, it is a common one: examples of tools that suffer from it are
GCC, Binutils, GHC and Autoconf. Nixpkgs tries to avoid sharing in the
mistake where possible. Still, because the concept of a target platform
is so ingrained, it is best to support it as is.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
The exact schema these fields follow is a bit ill-defined due to a long and convoluted evolution, but this is slowly being cleaned up. You can see examples of ones used in practice in <literal>lib.systems.examples</literal>; note how they are not all very consistent. For now, here are few fields can count on them containing:
The exact schema these fields follow is a bit ill-defined due to a long and
convoluted evolution, but this is slowly being cleaned up. You can see
examples of ones used in practice in
<literal>lib.systems.examples</literal>; note how they are not all very
consistent. For now, here are few fields can count on them containing:
</para>
<variablelist>
@@ -84,7 +133,11 @@
</term>
<listitem>
<para>
This is a two-component shorthand for the platform. Examples of this would be "x86_64-darwin" and "i686-linux"; see <literal>lib.systems.doubles</literal> for more. The first component corresponds to the CPU architecture of the platform and the second to the operating system of the platform (<literal>[cpu]-[os]</literal>). This format has built-in support in Nix, such as the <varname>builtins.currentSystem</varname> impure string.
This is a two-component shorthand for the platform. Examples of this
would be "x86_64-darwin" and "i686-linux"; see
<literal>lib.systems.doubles</literal> for more. This format isn't very
standard, but has built-in support in Nix, such as the
This is a 3- or 4- component shorthand for the platform. Examples of this would be <literal>x86_64-unknown-linux-gnu</literal> and <literal>aarch64-apple-darwin14</literal>. This is a standard format called the "LLVM target triple", as they are pioneered by LLVM. In the 4-part form, this corresponds to <literal>[cpu]-[vendor]-[os]-[abi]</literal>. This format is strictly more informative than the "Nix host double", as the previous format could analogously be termed. This needs a better name than <varname>config</varname>!
This is a 3- or 4- component shorthand for the platform. Examples of
this would be "x86_64-unknown-linux-gnu" and "aarch64-apple-darwin14".
This is a standard format called the "LLVM target triple", as they are
pioneered by LLVM and traditionally just used for the
<varname>targetPlatform</varname>. This format is strictly more
informative than the "Nix host double", as the previous format could
analogously be termed. This needs a better name than
<varname>config</varname>!
</para>
</listitem>
</varlistentry>
@@ -104,7 +164,12 @@
</term>
<listitem>
<para>
This is a Nix representation of a parsed LLVM target triple with white-listed components. This can be specified directly, or actually parsed from the <varname>config</varname>. See <literal>lib.systems.parse</literal> for the exact representation.
This is a nix representation of a parsed LLVM target triple with
white-listed components. This can be specified directly, or actually
parsed from the <varname>config</varname>. [Technically, only one need
be specified and the others can be inferred, though the precision of
inference may not be very good.] See
<literal>lib.systems.parse</literal> for the exact representation.
</para>
</listitem>
</varlistentry>
@@ -114,7 +179,10 @@
</term>
<listitem>
<para>
This is a string identifying the standard C library used. Valid identifiers include "glibc" for GNU libc, "libSystem" for Darwin's Libsystem, and "uclibc" for µClibc. It should probably be refactored to use the module system, like <varname>parse</varname>.
This is a string identifying the standard C library used. Valid
identifiers include "glibc" for GNU libc, "libSystem" for Darwin's
Libsystem, and "uclibc" for µClibc. It should probably be refactored to
use the module system, like <varname>parse</varname>.
</para>
</listitem>
</varlistentry>
@@ -124,7 +192,10 @@
</term>
<listitem>
<para>
These predicates are defined in <literal>lib.systems.inspect</literal>, and slapped onto every platform. They are superior to the ones in <varname>stdenv</varname> as they force the user to be explicit about which platform they are inspecting. Please use these instead of those.
These predicates are defined in <literal>lib.systems.inspect</literal>,
and slapped on every platform. They are superior to the ones in
<varname>stdenv</varname> as they force the user to be explicit about
which platform they are inspecting. Please use these instead of those.
</para>
</listitem>
</varlistentry>
@@ -134,99 +205,120 @@
</term>
<listitem>
<para>
This is, quite frankly, a dumping ground of ad-hoc settings (it's an attribute set). See <literal>lib.systems.platforms</literal> for examples—there's hopefully one in there that will work verbatim for each platform that is working. Please help us triage these flags and give them better homes!
This is, quite frankly, a dumping ground of ad-hoc settings (it's an
attribute set). See <literal>lib.systems.platforms</literal> for
examples—there's hopefully one in there that will work verbatim for
each platform that is working. Please help us triage these flags and
In this section we explore the relationship between both runtime and
buildtime dependencies and the 3 Autoconf platforms.
</para>
<para>
A runtime dependency between 2 packages implies that between them both the
host and target platforms match. This is directly implied by the meaning of
"host platform" and "runtime dependency": The package dependency exists
while both packages are running on a single host platform.
</para>
<para>
A build time dependency, however, implies a shift in platforms between the
depending package and the depended-on package. The meaning of a build time
dependency is that to build the depending package we need to be able to run
the depended-on's package. The depending package's build platform is
therefore equal to the depended-on package's host platform. Analogously,
the depending package's host platform is equal to the depended-on package's
target platform.
</para>
<para>
In this manner, given the 3 platforms for one package, we can determine the
three platforms for all its transitive dependencies. This is the most
important guiding principle behind cross-compilation with Nixpkgs, and will
be called the <wordasword>sliding window principle</wordasword>.
</para>
<para>
Some examples will probably make this clearer. If a package is being built
with a <literal>(build, host, target)</literal> platform triple of
<literal>(foo, bar, bar)</literal>, then its build-time dependencies would
have a triple of <literal>(foo, foo, bar)</literal>, and <emphasis>those
packages'</emphasis> build-time dependencies would have triple of
<literal>(foo, foo, foo)</literal>. In other words, it should take two
"rounds" of following build-time dependency edges before one reaches a
fixed point where, by the sliding window principle, the platform triple no
longer changes. Indeed, this happens with cross compilation, where only
rounds of native dependencies starting with the second necessarily coincide
with native packages.
</para>
<note>
<para>
This is a rather philosophical description that isn't very Nixpkgs-specific. For an overview of all the relevant attributes given to <varname>mkDerivation</varname>, see <xref
linkend="ssec-stdenv-dependencies"/>. For a description of how everything is implemented, see <xreflinkend="ssec-cross-dependency-implementation"/>.
The depending package's target platform is unconstrained by the sliding
window principle, which makes sense in that one can in principle build
cross compilers targeting arbitrary platforms.
</para>
</note>
<para>
In this section we explore the relationship between both runtime and build-time dependencies and the 3 Autoconf platforms.
How does this work in practice? Nixpkgs is now structured so that
build-time dependencies are taken from <varname>buildPackages</varname>,
whereas run-time dependencies are taken from the top level attribute set.
For example, <varname>buildPackages.gcc</varname> should be used at build
time, while <varname>gcc</varname> should be used at run time. Now, for
most of Nixpkgs's history, there was no <varname>buildPackages</varname>,
and most packages have not been refactored to use it explicitly. Instead,
one can use the six (<emphasis>gasp</emphasis>) attributes used for
specifying dependencies as documented in
<xreflinkend="ssec-stdenv-dependencies"/>. We "splice" together the
run-time and build-time package sets with <varname>callPackage</varname>,
and then <varname>mkDerivation</varname> for each of four attributes pulls
the right derivation out. This splicing can be skipped when not cross
compiling as the package sets are the same, but is a bit slow for cross
compiling. Because of this, a best-of-both-worlds solution is in the works
with no splicing or explicit access of <varname>buildPackages</varname>
needed. For now, feel free to use either method.
</para>
<para>
A run time dependency between two packages requires that their host platforms match. This is directly implied by the meaning of "host platform" and "runtime dependency": The package dependency exists while both packages are running on a single host platform.
</para>
<para>
A build time dependency, however, has a shift in platforms between the depending package and the depended-on package. "build time dependency" means that to build the depending package we need to be able to run the depended-on's package. The depending package's build platform is therefore equal to the depended-on package's host platform.
</para>
<para>
If both the dependency and depending packages aren't compilers or other machine-code-producing tools, we're done. And indeed <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname> have covered these simpler build-time and run-time (respectively) changes for many years. But if the dependency does produce machine code, we might need to worry about its target platform too. In principle, that target platform might be any of the depending package's build, host, or target platforms, but we prohibit dependencies from a "later" platform to an earlier platform to limit confusion because we've never seen a legitimate use for them.
</para>
<para>
Finally, if the depending package is a compiler or other machine-code-producing tool, it might need dependencies that run at "emit time". This is for compilers that (regrettably) insist on being built together with their source langauges' standard libraries. Assuming build != host != target, a run-time dependency of the standard library cannot be run at the compiler's build time or run time, but only at the run time of code emitted by the compiler.
</para>
<para>
Putting this all together, that means we have dependencies in the form "host → target", in at most the following six combinations:
<table>
<caption>Possible dependency types</caption>
<thead>
<tr>
<th>Dependency's host platform</th>
<th>Dependency's target platform</th>
</tr>
</thead>
<tbody>
<tr>
<td>build</td>
<td>build</td>
</tr>
<tr>
<td>build</td>
<td>host</td>
</tr>
<tr>
<td>build</td>
<td>target</td>
</tr>
<tr>
<td>host</td>
<td>host</td>
</tr>
<tr>
<td>host</td>
<td>target</td>
</tr>
<tr>
<td>target</td>
<td>target</td>
</tr>
</tbody>
</table>
</para>
<para>
Some examples will make this table clearer. Suppose there's some package that is being built with a <literal>(build, host, target)</literal> platform triple of <literal>(foo, bar, baz)</literal>. If it has a build-time library dependency, that would be a "host → build" dependency with a triple of <literal>(foo, foo, *)</literal> (the target platform is irrelevant). If it needs a compiler to be built, that would be a "build → host" dependency with a triple of <literal>(foo, foo, *)</literal> (the target platform is irrelevant). That compiler, would be built with another compiler, also "build → host" dependency, with a triple of <literal>(foo, foo, foo)</literal>.
</para>
<note>
<para>
There is also a "backlink" <varname>targetPackages</varname>, yielding a
package set whose <varname>buildPackages</varname> is the current package
set. This is a hack, though, to accommodate compilers with lousy build
systems. Please do not use this unless you are absolutely sure you are
packaging such a compiler and there is no other way.
</para>
</note>
</section>
<sectionxml:id="ssec-cross-cookbook">
<sectionxml:id="sec-cross-cookbook">
<title>Cross packaging cookbook</title>
<para>
Some frequently encountered problems when packaging for cross-compilation should be answered here. Ideally, the information above is exhaustive, so this section cannot provide any new information, but it is ludicrous and cruel to expect everyone to spend effort working through the interaction of many features just to figure out the same answer to the same common problem. Feel free to add to this list!
Some frequently problems when packaging for crosscompilation are good to
just spell and answer. Ideally the information above is exhaustive, so this
section cannot provide any new information, but its ludicrous and cruel to
expect everyone to spend effort working through the interaction of many
features just to figure out the same answer to the same common problem.
What if my package's build system needs to build a C program to be run under the build environment?
What if my package's build system needs to build a C program to be run
under the build environment?
</para>
</question>
<answer>
@@ -244,7 +336,11 @@
</question>
<answer>
<para>
Many packages assume that an unprefixed <command>ar</command> is available, but Nix doesn't provide one. It only provides a prefixed one, just as it only does for all the other binutils programs. It may be necessary to patch the package to fix the build system to use a prefixed `ar`.
Many packages assume that an unprefixed <command>ar</command> is
available, but Nix doesn't provide one. It only provides a prefixed one,
just as it only does for all the other binutils programs. It may be
necessary to patch the package to fix the build system to use a prefixed
`ar`.
</para>
</answer>
</qandaentry>
@@ -268,127 +364,112 @@
<sectionxml:id="sec-cross-usage">
<title>Cross-building packages</title>
<note>
<para>
More information needs to moved from the old wiki, especially
<linkxlink:href="https://nixos.org/wiki/CrossCompiling"/>, for this
section.
</para>
</note>
<para>
Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in which case there is no cross-compiling and everything is built by and for that system, or also with <varname>crossSystem</varname>, in which case packages run on the latter, but all building happens on the former. Both parameters take the same schema as the 3 (build, host, and target) platforms defined in the previous section. As mentioned above, <literal>lib.systems.examples</literal> has some platforms which are used as arguments for these parameters in practice. You can use them programmatically, or on the command line:
Nixpkgs can be instantiated with <varname>localSystem</varname> alone, in
which case there is no cross compiling and everything is built by and for
that system, or also with <varname>crossSystem</varname>, in which case
packages run on the latter, but all building happens on the former. Both
parameters take the same schema as the 3 (build, host, and target) platforms
defined in the previous section. As mentioned above,
<literal>lib.systems.examples</literal> has some platforms which are used as
arguments for these parameters in practice. You can use them
programmatically, or on the command line:
<programlisting>
nix-build <nixpkgs> --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever</programlisting>
</para>
<note>
<para>
Eventually we would like to make these platform examples an unnecessary convenience so that
Eventually we would like to make these platform examples an unnecessary
works in the vast majority of cases. The problem today is dependencies on other sorts of configuration which aren't given proper defaults. We rely on the examples to crudely to set those configuration parameters in some vaguely sane manner on the users behalf. Issue <linkxlink:href="https://github.com/NixOS/nixpkgs/issues/34274">#34274</link> tracks this inconvenience along with its root cause in crufty configuration options.
nix-build <nixpkgs> --arg crossSystem.config '<arch>-<os>-<vendor>-<abi>' -A whatever</programlisting>
works in the vast majority of cases. The problem today is dependencies on
other sorts of configuration which aren't given proper defaults. We rely on
the examples to crudely to set those configuration parameters in some
tracks this inconvenience along with its root cause in crufty configuration
options.
</para>
</note>
<para>
While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields. As discussed in the previous section, only one of <varname>system</varname>, <varname>config</varname>, and <varname>parsed</varname> is needed to infer the other two. Additionally, <varname>libc</varname> will be inferred from <varname>parse</varname>. Finally, <literal>localSystem.system</literal> is also <emphasis>impurely</emphasis> inferred based on the platform evaluation occurs. This means it is often not necessary to pass <varname>localSystem</varname> at all, as in the command-line example in the previous paragraph.
While one is free to pass both parameters in full, there's a lot of logic to
fill in missing fields. As discussed in the previous section, only one of
<varname>system</varname>, <varname>config</varname>, and
<varname>parsed</varname> is needed to infer the other two. Additionally,
<varname>libc</varname> will be inferred from <varname>parse</varname>.
Finally, <literal>localSystem.system</literal> is also
<emphasis>impurely</emphasis> inferred based on the platform evaluation
occurs. This means it is often not necessary to pass
<varname>localSystem</varname> at all, as in the command-line example in the
previous paragraph.
</para>
<note>
<para>
Many sources (manual, wiki, etc) probably mention passing<varname>system</varname>, <varname>platform</varname>, along with the optional <varname>crossSystem</varname> to nixpkgs: <literal>import <nixpkgs> { system = ..; platform = ..; crossSystem = ..; }</literal>. Passing those two instead of <varname>localSystem</varname> is still supported for compatibility, but is discouraged. Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience.
Many sources (manual, wiki, etc) probably mention passing
<varname>system</varname>, <varname>platform</varname>, along with the
optional <varname>crossSystem</varname> to nixpkgs: <literal>import
}</literal>. Passing those two instead of <varname>localSystem</varname> is
still supported for compatibility, but is discouraged. Indeed, much of the
inference we do for these parameters is motivated by compatibility as much
as convenience.
</para>
</note>
<para>
One would think that <varname>localSystem</varname> and<varname>crossSystem</varname> overlap horribly with the three <varname>*Platforms</varname> (<varname>buildPlatform</varname>, <varname>hostPlatform,</varname> and <varname>targetPlatform</varname>; see <varname>stage.nix</varname> or the manual). Actually, those identifiers are purposefully not used here to draw a subtle but important distinction: While the granularity of having 3 platforms is necessary to properly *build* packages, it is overkill for specifying the user's *intent* when making a build plan or package set. A simple "build vs deploy" dichotomy is adequate: the sliding window principle described in the previous section shows how to interpolate between the these two "end points" to get the 3 platform triple for each bootstrapping stage. That means for any package a given package set, even those not bound on the top level but only reachable via dependencies or <varname>buildPackages</varname>, the three platforms will be defined as one of <varname>localSystem</varname> or <varname>crossSystem</varname>, with the former replacing the latter as one traverses build-time dependencies. A last simple difference is that <varname>crossSystem</varname> should be null when one doesn't want to cross-compile, while the <varname>*Platform</varname>s are always non-null. <varname>localSystem</varname> is always non-null.
One would think that <varname>localSystem</varname> and
<varname>crossSystem</varname> overlap horribly with the three
The categorizes of dependencies developed in <xref
linkend="ssec-cross-dependency-categorization"/> are specified as lists of derivations given to <varname>mkDerivation</varname>, as documented in <xreflinkend="ssec-stdenv-dependencies"/>. In short, each list of dependencies for "host → target" of "foo → bar" is called <varname>depsFooBar</varname>, with exceptions for backwards compatibility that <varname>depsBuildHost</varname> is instead called <varname>nativeBuildInputs</varname> and <varname>depsHostTarget</varname> is instead called <varname>buildInputs</varname>. Nixpkgs is now structured so that each <varname>depsFooBar</varname> is automatically taken from <varname>pkgsFooBar</varname>. (These <varname>pkgsFooBar</varname>s are quite new, so there is no special case for <varname>nativeBuildInputs</varname> and <varname>buildInputs</varname>.) For example, <varname>pkgsBuildHost.gcc</varname> should be used at build-time, while <varname>pkgsHostTarget.gcc</varname> should be used at run-time.
If one explores nixpkgs, they will see derivations with names like
<literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is
a holdover from before we properly distinguished between the host and
target platforms —the derivation with "Cross" in the name covered the
<literal>build = host != target</literal> case, while the other covered the
<literal>host = target</literal>, with build platform the same or not based
on whether one was using its <literal>.nativeDrv</literal> or
<literal>.crossDrv</literal>. This ugliness will disappear soon.
</para>
<para>
Now, for most of Nixpkgs's history, there were no <varname>pkgsFooBar</varname> attributes, and most packages have not been refactored to use it explicitly. Prior to those, there were just <varname>buildPackages</varname>, <varname>pkgs</varname>, and <varname>targetPackages</varname>. Those are now redefined as aliases to <varname>pkgsBuildHost</varname>, <varname>pkgsHostTarget</varname>, and <varname>pkgsTargetTarget</varname>. It is acceptable, even recommended, to use them for libraries to show that the host platform is irrelevant.
</para>
<para>
But before that, there was just <varname>pkgs</varname>, even though both <varname>buildInputs</varname> and <varname>nativeBuildInputs</varname> existed. [Cross barely worked, and those were implemented with some hacks on <varname>mkDerivation</varname> to override dependencies.] What this means is the vast majority of packages do not use any explicit package set to populate their dependencies, just using whatever <varname>callPackage</varname> gives them even if they do correctly sort their dependencies into the multiple lists described above. And indeed, asking that users both sort their dependencies, <emphasis>and</emphasis> take them from the right attribute set, is both too onerous and redundant, so the recommended approach (for now) is to continue just categorizing by list and not using an explicit package set.
</para>
<para>
To make this work, we "splice" together the six <varname>pkgsFooBar</varname> package sets and have <varname>callPackage</varname> actually take its arguments from that. This is currently implemented in <filename>pkgs/top-level/splice.nix</filename>. <varname>mkDerivation</varname> then, for each dependency attribute, pulls the right derivation out from the splice. This splicing can be skipped when not cross-compiling as the package sets are the same, but still is a bit slow for cross-compiling. We'd like to do something better, but haven't come up with anything yet.
</para>
</section>
<sectionxml:id="ssec-bootstrapping">
<title>Bootstrapping</title>
<para>
Each of the package sets described above come from a single bootstrapping stage. While <filename>pkgs/top-level/default.nix</filename>, coordinates the composition of stages at a high level, <filename>pkgs/top-level/stage.nix</filename> "ties the knot" (creates the fixed point) of each stage. The package sets are defined per-stage however, so they can be thought of as edges between stages (the nodes) in a graph. Compositions like <literal>pkgsBuildTarget.targetPackages</literal> can be thought of as paths to this graph.
</para>
<para>
While there are many package sets, and thus many edges, the stages can also be arranged in a linear chain. In other words, many of the edges are redundant as far as connectivity is concerned. This hinges on the type of bootstrapping we do. Currently for cross it is:
<orderedlist>
<listitem>
<para>
<literal>(native, native, native)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, native, foreign)</literal>
</para>
</listitem>
<listitem>
<para>
<literal>(native, foreign, foreign)</literal>
</para>
</listitem>
</orderedlist>
In each stage, <varname>pkgsBuildHost</varname> refers the the previous stage, <varname>pkgsBuildBuild</varname> refers to the one before that, and <varname>pkgsHostTarget</varname> refers to the current one, and <varname>pkgsTargetTarget</varname> refers to the next one. When there is no previous or next stage, they instead refer to the current stage. Note how all the invariants regarding the mapping between dependency and depending packages' build host and target platforms are preserved. <varname>pkgsBuildTarget</varname> and <varname>pkgsHostHost</varname> are more complex in that the stage fitting the requirements isn't always a fixed chain of "prevs" and "nexts" away (modulo the "saturating" self-references at the ends). We just special case each instead. All the primary edges are implemented is in <filename>pkgs/stdenv/booter.nix</filename>, and secondarily aliases in <filename>pkgs/top-level/stage.nix</filename>.
</para>
<note>
<para>
Note the native stages are bootstrapped in legacy ways that predate the current cross implementation. This is why the the bootstrapping stages leading up to the final stages are ignored inthe previous paragraph.
</para>
</note>
<para>
If one looks at the 3 platform triples, one can see that they overlap such that one could put them together into a chain like:
<programlisting>
(native, native, native, foreign, foreign)
</programlisting>
If one imagines the saturating self references at the end being replaced with infinite stages, and then overlays those platform triples, one ends up with the infinite tuple:
On can then imagine any sequence of platforms such that there are bootstrap stages with their 3 platforms determined by "sliding a window" that is the 3 tuple through the sequence. This was the original model for bootstrapping. Without a target platform (assume a better world where all compilers are multi-target and all standard libraries are built in their own derivation), this is sufficient. Conversely if one wishes to cross compile "faster", with a "Canadian Cross" bootstraping stage where <literal>build != host != target</literal>, more bootstrapping stages are needed since no sliding window providess the pesky <varname>pkgsBuildTarget</varname> package set since it skips the Canadian cross stage's "host".
</para>
<note>
<para>
It is much better to refer to <varname>buildPackages</varname> than <varname>targetPackages</varname>, or more broadly package sets that do not mention "target". There are three reasons for this.
</para>
<para>
First, it is because bootstrapping stages do not have a unique <varname>targetPackages</varname>. For example a <literal>(x86-linux, x86-linux, arm-linux)</literal> and <literal>(x86-linux, x86-linux, x86-windows)</literal> package set both have a <literal>(x86-linux, x86-linux, x86-linux)</literal> package set. Because there is no canonical <varname>targetPackages</varname> for such a native (<literal>build == host == target</literal>) package set, we set their <varname>targetPackages</varname>
</para>
<para>
Second, it is because this is a frequent source of hard-to-follow "infinite recursions" / cycles. When only package sets that don't mention target are used, the package set forms a directed acyclic graph. This means that all cycles that exist are confined to one stage. This means they are a lot smaller, and easier to follow in the code or a backtrace. It also means they are present in native and cross builds alike, and so more likely to be caught by CI and other users.
</para>
<para>
Thirdly, it is because everything target-mentioning only exists to accommodate compilers with lousy build systems that insist on the compiler itself and standard library being built together. Of course that is bad because bigger derivations means longer rebuilds. It is also problematic because it tends to make the standard libraries less like other libraries than they could be, complicating code and build systems alike. Because of the other problems, and because of these innate disadvantages, compilers ought to be packaged another way where possible.
</para>
</note>
<note>
<para>
If one explores Nixpkgs, they will see derivations with names like <literal>gccCross</literal>. Such <literal>*Cross</literal> derivations is a holdover from before we properly distinguished between the host and target platforms—the derivation with "Cross" in the name covered the <literal>build = host != target</literal> case, while the other covered the <literal>host = target</literal>, with build platform the same or not based on whether one was using its <literal>.nativeDrv</literal> or <literal>.crossDrv</literal>. This ugliness will disappear soon.
<varname>pkgs.appimageTools</varname> is a set of functions for extracting and wrapping <linkxlink:href="https://appimage.org/">AppImage</link> files. They are meant to be used if traditional packaging from source is infeasible, or it would take too long. To quickly run an AppImage file, <literal>pkgs.appimage-run</literal> can be used as well.
</para>
<warning>
<para>
The <varname>appimageTools</varname> API is unstable and may be subject to backwards-incompatible changes in the future.
</para>
</warning>
<sectionxml:id="ssec-pkgs-appimageTools-formats">
<title>AppImage formats</title>
<para>
There are different formats for AppImages, see <linkxlink:href="https://github.com/AppImage/AppImageSpec/blob/74ad9ca2f94bf864a4a0dac1f369dd4f00bd1c28/draft.md#image-format">the specification</link> for details.
</para>
<itemizedlist>
<listitem>
<para>
Type 1 images are ISO 9660 files that are also ELF executables.
</para>
</listitem>
<listitem>
<para>
Type 2 images are ELF executables with an appended filesystem.
</para>
</listitem>
</itemizedlist>
<para>
They can be told apart with <command>file -k</command>:
</para>
<screen>
<prompt>$ </prompt>file -k type1.AppImage
type1.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV) ISO 9660 CD-ROM filesystem data 'AppImage' (Lepton 3.x), scale 0-0,
spot sensor temperature 0.000000, unit celsius, color scheme 0, calibration: offset 0.000000, slope 0.000000, dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=d629f6099d2344ad82818172add1d38c5e11bc6d, stripped\012- data
<prompt>$ </prompt>file -k type2.AppImage
type2.AppImage: ELF 64-bit LSB executable, x86-64, version 1 (SYSV) (Lepton 3.x), scale 232-60668, spot sensor temperature -4.187500, color scheme 15, show scale bar, calibration: offset -0.000000, slope 0.000000 (Lepton 2.x), scale 4111-45000, spot sensor temperature 412442.250000, color scheme 3, minimum point enabled, calibration: offset -75402534979642766821519867692934234112.000000, slope 5815371847733706829839455140374904832.000000, dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 2.6.18, BuildID[sha1]=79dcc4e55a61c293c5e19edbd8d65b202842579f, stripped\012- data
</screen>
<para>
Note how the type 1 AppImage is described as an <literal>ISO 9660 CD-ROM filesystem</literal>, and the type 2 AppImage is not.
extraPkgs = pkgs: with pkgs; [ ]; <coxml:id='ex-appimageTools-wrapping-3'/>
}</programlisting>
<calloutlist>
<calloutarearefs='ex-appimageTools-wrapping-1'>
<para>
<varname>name</varname> specifies the name of the resulting image.
</para>
</callout>
<calloutarearefs='ex-appimageTools-wrapping-2'>
<para>
<varname>src</varname> specifies the AppImage file to extract.
</para>
</callout>
<calloutarearefs='ex-appimageTools-wrapping-2'>
<para>
<varname>extraPkgs</varname> allows you to pass a function to include additional packages inside the FHS environment your AppImage is going to run in. There are a few ways to learn which dependencies an application needs:
<itemizedlist>
<listitem>
<para>
Looking through the extracted AppImage files, reading its scripts and running <command>patchelf</command> and <command>ldd</command> on its executables. This can also be done in <command>appimage-run</command>, by setting <command>APPIMAGE_DEBUG_EXEC=bash</command>.
</para>
</listitem>
<listitem>
<para>
Running <command>strace -vfefile</command> on the wrapped executable, looking for libraries that can't be found.
Nix is a unityped, dynamic language, this means every value can potentially appear anywhere. Since it is also non-strict, evaluation order and what ultimately is evaluated might surprise you. Therefore it is important to be able to debug nix expressions.
Nix is a unityped, dynamic language, this means every value can potentially
appear anywhere. Since it is also non-strict, evaluation order and what
ultimately is evaluated might surprise you. Therefore it is important to be
able to debug nix expressions.
</para>
<para>
In the <literal>lib/debug.nix</literal> file you will find a number of functions that help (pretty-)printing values while evaluation is runnnig. You can even specify how deep these values should be printed recursively, and transform them on the fly. Please consult the docstrings in <literal>lib/debug.nix</literal> for usage information.
In the <literal>lib/debug.nix</literal> file you will find a number of
functions that help (pretty-)printing values while evaluation is runnnig. You
can even specify how deep these values should be printed recursively, and
transform them on the fly. Please consult the docstrings in
<literal>lib/debug.nix</literal> for usage information.
<varname>pkgs.dockerTools</varname> is a set of functions for creating and manipulating Docker images according to the <linkxlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120"> Docker Image Specification v1.2.0 </link>. Docker itself is not used to perform any of the operations done by these functions.
<varname>pkgs.dockerTools</varname> is a set of functions for creating and
Docker Image Specification v1.2.0 </link>. Docker itself is not used to
perform any of the operations done by these functions.
</para>
<warning>
<para>
The <varname>dockerTools</varname> API is unstable and may be subject to backwards-incompatible changes in the future.
The <varname>dockerTools</varname> API is unstable and may be subject to
backwards-incompatible changes in the future.
</para>
</warning>
@@ -18,11 +23,15 @@
<title>buildImage</title>
<para>
This function is analogous to the <command>docker build</command> command, in that it can be used to build a Docker-compatible repository tarball containing a single image with one or multiple layers. As such, the result is suitable for being loaded in Docker with <command>docker load</command>.
This function is analogous to the <command>docker build</command> command,
in that can used to build a Docker-compatible repository tarball containing
a single image with one or multiple layers. As such, the result is suitable
for being loaded in Docker with <command>docker load</command>.
</para>
<para>
The parameters of <varname>buildImage</varname> with relative example values are described below:
The parameters of <varname>buildImage</varname> with relative example values
The above example will build a Docker image <literal>redis/latest</literal> from the given base image. Loading and running this image in Docker results in <literal>redis-server</literal> being started automatically.
The above example will build a Docker image <literal>redis/latest</literal>
from the given base image. Loading and running this image in Docker results
in <literal>redis-server</literal> being started automatically.
</para>
<calloutlist>
<calloutarearefs='ex-dockerTools-buildImage-1'>
<para>
<varname>name</varname> specifies the name of the resulting image. This is the only required argument for <varname>buildImage</varname>.
<varname>name</varname> specifies the name of the resulting image. This is
the only required argument for <varname>buildImage</varname>.
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-2'>
<para>
<varname>tag</varname> specifies the tag of the resulting image. By default it's <literal>null</literal>, which indicates that the nix output hash will be used as tag.
<varname>tag</varname> specifies the tag of the resulting image. By
default it's <literal>null</literal>, which indicates that the nix output
hash will be used as tag.
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-3'>
<para>
<varname>fromImage</varname> is the repository tarball containing the base image. It must be a valid Docker image, such as exported by <command>docker save</command>. By default it's <literal>null</literal>, which can be seen as equivalent to <literal>FROM scratch</literal> of a <filename>Dockerfile</filename>.
<varname>fromImage</varname> is the repository tarball containing the base
image. It must be a valid Docker image, such as exported by
<command>docker save</command>. By default it's <literal>null</literal>,
which can be seen as equivalent to <literal>FROM scratch</literal> of a
<filename>Dockerfile</filename>.
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-4'>
<para>
<varname>fromImageName</varname> can be used to further specify the base image within the repository, in case it contains multiple images. By default it's <literal>null</literal>, in which case <varname>buildImage</varname> will peek the first image available in the repository.
<varname>fromImageName</varname> can be used to further specify the base
image within the repository, in case it contains multiple images. By
default it's <literal>null</literal>, in which case
<varname>buildImage</varname> will peek the first image available in the
repository.
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-5'>
<para>
<varname>fromImageTag</varname> can be used to further specify the tag of the base image within the repository, in case an image contains multiple tags. By default it's <literal>null</literal>, in which case <varname>buildImage</varname> will peek the first tag available for the base image.
<varname>fromImageTag</varname> can be used to further specify the tag of
the base image within the repository, in case an image contains multiple
tags. By default it's <literal>null</literal>, in which case
<varname>buildImage</varname> will peek the first tag available for the
base image.
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-6'>
<para>
<varname>contents</varname> is a derivation that will be copied in the new layer of the resulting image. This can be similarly seen as <command>ADD contents/ /</command> in a <filename>Dockerfile</filename>. By default it's <literal>null</literal>.
<varname>contents</varname> is a derivation that will be copied in the new
layer of the resulting image. This can be similarly seen as <command>ADD
contents/ /</command> in a <filename>Dockerfile</filename>. By default
<varname>runAsRoot</varname> is a bash script that will run as root in an environment that overlays the existing layers of the base image with the new resulting layer, including the previously copied <varname>contents</varname> derivation. This can be similarly seen as <command>RUN ...</command> in a <filename>Dockerfile</filename>.
<varname>runAsRoot</varname> is a bash script that will run as root in an
environment that overlays the existing layers of the base image with the
new resulting layer, including the previously copied
<varname>contents</varname> derivation. This can be similarly seen as
<command>RUN ...</command> in a <filename>Dockerfile</filename>.
<note>
<para>
Using this parameter requires the <literal>kvm</literal> device to be available.
Using this parameter requires the <literal>kvm</literal> device to be
available.
</para>
</note>
</para>
</callout>
<calloutarearefs='ex-dockerTools-buildImage-8'>
<para>
<varname>config</varname> is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the <linkxlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> Docker Image Specification v1.2.0 </link>.
<varname>config</varname> is used to specify the configuration of the
containers that will be started off the built image in Docker. The
After the new layer has been created, its closure (to which<varname>contents</varname>, <varname>config</varname> and <varname>runAsRoot</varname> contribute) will be copied in the layer itself. Only new dependencies that are not already in the existing layers will be copied.
After the new layer has been created, its closure (to which
<varname>contents</varname>, <varname>config</varname> and
<varname>runAsRoot</varname> contribute) will be copied in the layer itself.
Only new dependencies that are not already in the existing layers will be
copied.
</para>
<para>
At the end of the process, only one new single layer will be produced and added to the resulting image.
At the end of the process, only one new single layer will be produced and
added to the resulting image.
</para>
<para>
The resulting repository will only list the single image<varname>image/tag</varname>. In the case of <xreflinkend='ex-dockerTools-buildImage'/> it would be <varname>redis/latest</varname>.
The resulting repository will only list the single image
<varname>image/tag</varname>. In the case of
<xreflinkend='ex-dockerTools-buildImage'/> it would be
<varname>redis/latest</varname>.
</para>
<para>
It is possible to inspect the arguments with which an image was built using its <varname>buildArgs</varname> attribute.
It is possible to inspect the arguments with which an image was built using
its <varname>buildArgs</varname> attribute.
</para>
<note>
<para>
If you see errors similar to <literal>getProtocolByName: does not exist (no such protocol name: tcp)</literal> you may need to add <literal>pkgs.iana-etc</literal> to <varname>contents</varname>.
If you see errors similar to <literal>getProtocolByName: does not exist (no
such protocol name: tcp)</literal> you may need to add
<literal>pkgs.iana-etc</literal> to <varname>contents</varname>.
</para>
</note>
<note>
<para>
If you see errors similar to <literal>Error_Protocol ("certificate has unknown CA",True,UnknownCa)</literal> you may need to add <literal>pkgs.cacert</literal> to <varname>contents</varname>.
If you see errors similar to <literal>Error_Protocol ("certificate has
unknown CA",True,UnknownCa)</literal> you may need to add
<literal>pkgs.cacert</literal> to <varname>contents</varname>.
<title>Impurely Defining a Docker Layer's Creation Date</title>
<para>
By default <function>buildImage</function> will use a static date of one second past the UNIX Epoch. This allows <function>buildImage</function> to produce binary reproducible images. When listing images with <command>docker images</command>, the newly created images will be listed like this:
By default <function>buildImage</function> will use a static date of one
second past the UNIX Epoch. This allows <function>buildImage</function> to
produce binary reproducible images. When listing images with
<command>docker list images</command>, the newly created images will be
listed like this:
</para>
<screen><![CDATA[
$ docker images
$ docker image list
REPOSITORY TAG IMAGE ID CREATED SIZE
hello latest 08c791c7846e 48 years ago 25.2MB
]]></screen>
<para>
You can break binary reproducibility but have a sorted, meaningful<literal>CREATED</literal> column by setting <literal>created</literal> to <literal>now</literal>.
You can break binary reproducibility but have a sorted, meaningful
<literal>CREATED</literal> column by setting <literal>created</literal> to
@@ -218,7 +278,10 @@ hello latest de2bf4786de6 About a minute ago 25.2MB
</term>
<listitem>
<para>
Run-time configuration of the container. A full list of the options are available at in the <linkxlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> Docker Image Specification v1.2.0 </link>.
Run-time configuration of the container. A full list of the options are
Shell commands to run while building the final layer, without access to most of the layer contents. Changes to this layer are "on top" of all the other layers, so can create additional directories and files.
Increasing the <varname>maxLayers</varname> increases the number of layers which have a chance to be shared between different images.
Increasing the <varname>maxLayers</varname> increases the number of layers
which have a chance to be shared between different images.
</para>
<para>
Modern Docker installations support up to 128 layers, however older versions support as few as 42.
Modern Docker installations support up to 128 layers, however older
versions support as few as 42.
</para>
<para>
If the produced image will not be extended by other Docker builds, it is safe to set <varname>maxLayers</varname> to <literal>128</literal>. However it will be impossible to extend the image further.
If the produced image will not be extended by other Docker builds, it is
safe to set <varname>maxLayers</varname> to <literal>128</literal>. However
it will be impossible to extend the image further.
</para>
<para>
The first (<literal>maxLayers-2</literal>) most "popular" paths will have their own individual layers, then layer #<literal>maxLayers-1</literal> will contain all the remaining "unpopular" paths, and finally layer #<literal>maxLayers</literal> will contain the Image configuration.
The first (<literal>maxLayers-2</literal>) most "popular" paths will have
their own individual layers, then layer #<literal>maxLayers-1</literal>
will contain all the remaining "unpopular" paths, and finally layer
#<literal>maxLayers</literal> will contain the Image configuration.
</para>
<para>
Docker's Layers are not inherently ordered, they are content-addressable and are not explicitly layered until they are composed in to an Image.
Docker's Layers are not inherently ordered, they are content-addressable
and are not explicitly layered until they are composed in to an Image.
This function is analogous to the <command>docker pull</command> command, in that it can be used to pull a Docker image from a Docker registry. By default <linkxlink:href="https://hub.docker.com/">Docker Hub</link> is used to pull images.
This function is analogous to the <command>docker pull</command> command, in
that can be used to pull a Docker image from a Docker registry. By default
<linkxlink:href="https://hub.docker.com/">Docker Hub</link> is used to pull
<varname>imageName</varname> specifies the name of the image to be downloaded, which can also include the registry namespace (e.g. <literal>nixos</literal>). This argument is required.
<varname>imageName</varname> specifies the name of the image to be
downloaded, which can also include the registry namespace (e.g.
<literal>nixos</literal>). This argument is required.
</para>
</callout>
<calloutarearefs='ex-dockerTools-pullImage-2'>
<para>
<varname>imageDigest</varname> specifies the digest of the image to be downloaded. This argument is required.
<varname>imageDigest</varname> specifies the digest of the image to be
downloaded. Skopeo can be used to get the digest of an image, with its
<varname>inspect</varname> subcommand. Since a given
<varname>imageName</varname> may transparently refer to a manifest list of
images which support multiple architectures and/or operating systems,
supply the `--override-os` and `--override-arch` arguments to specify
exactly which image you want. By default it will match the OS and
<varname>finalImageName</varname>, if specified, this is the name of the image to be created. Note it is never used to fetch the image since we prefer to rely on the immutable digest ID. By default it's equal to <varname>imageName</varname>.
<varname>finalImageTag</varname>, if specified, this is the tag of the
image to be created. Note it is never used to fetch the image since we
prefer to rely on the immutable digest ID. By default it's
<literal>latest</literal>.
</para>
</callout>
<calloutarearefs='ex-dockerTools-pullImage-4'>
<para>
<varname>finalImageTag</varname>, if specified, this is the tag of the image to be created. Note it is never used to fetch the image since we prefer to rely on the immutable digest ID. By default it's <literal>latest</literal>.
<varname>sha256</varname> is the checksum of the whole fetched image. This
argument is required.
</para>
</callout>
<calloutarearefs='ex-dockerTools-pullImage-5'>
<para>
<varname>sha256</varname> is the checksum of the whole fetched image. This argument is required.
<varname>os</varname>, if specified, is the operating system of the
fetched image. By default it's <literal>linux</literal>.
</para>
</callout>
<calloutarearefs='ex-dockerTools-pullImage-6'>
<para>
<varname>os</varname>, if specified, is the operating system of the fetched image. By default it's <literal>linux</literal>.
</para>
</callout>
<calloutarearefs='ex-dockerTools-pullImage-7'>
<para>
<varname>arch</varname>, if specified, is the cpu architecture of the fetched image. By default it's <literal>x86_64</literal>.
<varname>arch</varname>, if specified, is the cpu architecture of the
fetched image. By default it's <literal>x86_64</literal>.
</para>
</callout>
</calloutlist>
<para>
<literal>nix-prefetch-docker</literal> command can be used to get required image parameters:
<screen>
<prompt>$ </prompt>nix run nixpkgs.nix-prefetch-docker -c nix-prefetch-docker --image-name mysql --image-tag 5
</screen>
Since a given <varname>imageName</varname> may transparently refer to a manifest list of images which support multiple architectures and/or operating systems, you can supply the <option>--os</option> and <option>--arch</option> arguments to specify exactly which image you want. By default it will match the OS and architecture of the host the command is run on.
<screen>
<prompt>$ </prompt>nix-prefetch-docker --image-name mysql --image-tag 5 --arch x86_64 --os linux
</screen>
Desired image name and tag can be set using <option>--final-image-name</option> and <option>--final-image-tag</option> arguments:
<screen>
<prompt>$ </prompt>nix-prefetch-docker --image-name mysql --image-tag 5 --final-image-name eu.gcr.io/my-project/mysql --final-image-tag prod
This function is analogous to the <command>docker export</command> command, in that it can be used to flatten a Docker image that contains multiple layers. It is in fact the result of the merge of all the layers of the image. As such, the result is suitable for being imported in Docker with <command>docker import</command>.
This function is analogous to the <command>docker export</command> command,
in that can used to flatten a Docker image that contains multiple layers. It
is in fact the result of the merge of all the layers of the image. As such,
the result is suitable for being imported in Docker with <command>docker
import</command>.
</para>
<note>
<para>
Using this function requires the <literal>kvm</literal> device to be available.
Using this function requires the <literal>kvm</literal> device to be
available.
</para>
</note>
@@ -440,15 +511,18 @@ exportImage {
name = someLayeredImage.name;
}
</programlisting>
</programlisting>
</example>
<para>
The parameters relative to the base image have the same synopsis as described in <xreflinkend='ssec-pkgs-dockerTools-buildImage'/>, except that <varname>fromImage</varname> is the only required argument in this case.
The parameters relative to the base image have the same synopsis as
described in <xreflinkend='ssec-pkgs-dockerTools-buildImage'/>, except that
<varname>fromImage</varname> is the only required argument in this case.
</para>
<para>
The <varname>name</varname> argument is the name of the derivation output, which defaults to <varname>fromImage.name</varname>.
The <varname>name</varname> argument is the name of the derivation output,
which defaults to <varname>fromImage.name</varname>.
</para>
</section>
@@ -456,7 +530,11 @@ exportImage {
<title>shadowSetup</title>
<para>
This constant string is a helper for setting up the base files for managing users and groups, only if such files don't exist already. It is suitable for being used in a <varname>runAsRoot</varname><xreflinkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like in the example below:
This constant string is a helper for setting up the base files for managing
users and groups, only if such files don't exist already. It is suitable for
being used in a <varname>runAsRoot</varname>
<xreflinkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like
in the example below:
</para>
<examplexml:id='ex-dockerTools-shadowSetup'>
@@ -466,7 +544,7 @@ buildImage {
name = "shadow-basic";
runAsRoot = ''
#!${pkgs.runtimeShell}
#!${stdenv.shell}
${shadowSetup}
groupadd -r redis
useradd -r -g redis redis
@@ -478,7 +556,9 @@ buildImage {
</example>
<para>
Creating base files like <literal>/etc/passwd</literal> or<literal>/etc/login.defs</literal> is necessary for shadow-utils to manipulate users and groups.
Creating base files like <literal>/etc/passwd</literal> or
<literal>/etc/login.defs</literal> are necessary for shadow-utils to
When using Nix, you will frequently need to download source code and other files from the internet. Nixpkgs comes with a few helper functions that allow you to fetch fixed-output derivations in a structured way.
</para>
<para>
The two fetcher primitives are <function>fetchurl</function> and <function>fetchzip</function>. Both of these have two required arguments, a URL and a hash. The hash is typically <literal>sha256</literal>, although many more hash algorithms are supported. Nixpkgs contributors are currently recommended to use <literal>sha256</literal>. This hash will be used by Nix to identify your source. A typical usage of fetchurl is provided below.
The main difference between <function>fetchurl</function> and <function>fetchzip</function> is in how they store the contents. <function>fetchurl</function> will store the unaltered contents of the URL within the Nix store. <function>fetchzip</function> on the other hand will decompress the archive for you, making files and directories directly accessible in the future. <function>fetchzip</function> can only be used with archives. Despite the name, <function>fetchzip</function> is not limited to .zip files and can also be used with any tarball.
</para>
<para>
<function>fetchpatch</function> works very similarly to <function>fetchurl</function> with the same arguments expected. It expects patch files as a source and and performs normalization on them before computing the checksum. For example it will remove comments or other unstable parts that are sometimes added by version control systems and can change over time.
</para>
<para>
Other fetcher functions allow you to add source code directly from a VCS such as subversion or git. These are mostly straightforward names based on the name of the command used with the VCS system. Because they give you a working repository, they act most like <function>fetchzip</function>.
</para>
<variablelist>
<varlistentry>
<term>
<literal>fetchsvn</literal>
</term>
<listitem>
<para>
Used with Subversion. Expects <literal>url</literal> to a Subversion directory, <literal>rev</literal>, and <literal>sha256</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchgit</literal>
</term>
<listitem>
<para>
Used with Git. Expects <literal>url</literal> to a Git repo, <literal>rev</literal>, and <literal>sha256</literal>. <literal>rev</literal> in this case can be full the git commit id (SHA1 hash) or a tag name like <literal>refs/tags/v1.0</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchfossil</literal>
</term>
<listitem>
<para>
Used with Fossil. Expects <literal>url</literal> to a Fossil archive, <literal>rev</literal>, and <literal>sha256</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchcvs</literal>
</term>
<listitem>
<para>
Used with CVS. Expects <literal>cvsRoot</literal>, <literal>tag</literal>, and <literal>sha256</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchhg</literal>
</term>
<listitem>
<para>
Used with Mercurial. Expects <literal>url</literal>, <literal>rev</literal>, and <literal>sha256</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
A number of fetcher functions wrap part of <function>fetchurl</function> and <function>fetchzip</function>. They are mainly convenience functions intended for commonly used destinations of source code in Nixpkgs. These wrapper fetchers are listed below.
</para>
<variablelist>
<varlistentry>
<term>
<literal>fetchFromGitHub</literal>
</term>
<listitem>
<para>
<function>fetchFromGitHub</function> expects four arguments. <literal>owner</literal> is a string corresponding to the GitHub user or organization that controls this repository. <literal>repo</literal> corresponds to the name of the software repository. These are located at the top of every GitHub HTML page as <literal>owner</literal>/<literal>repo</literal>. <literal>rev</literal> corresponds to the Git commit hash or tag (e.g <literal>v1.0</literal>) that will be downloaded from Git. Finally, <literal>sha256</literal> corresponds to the hash of the extracted directory. Again, other hash algorithms are also available but <literal>sha256</literal> is currently preferred.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchFromGitLab</literal>
</term>
<listitem>
<para>
This is used with GitLab repositories. The arguments expected are very similar to fetchFromGitHub above.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchFromBitbucket</literal>
</term>
<listitem>
<para>
This is used with BitBucket repositories. The arguments expected are very similar to fetchFromGitHub above.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchFromSavannah</literal>
</term>
<listitem>
<para>
This is used with Savannah repositories. The arguments expected are very similar to fetchFromGitHub above.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>fetchFromRepoOrCz</literal>
</term>
<listitem>
<para>
This is used with repo.or.cz repositories. The arguments expected are very similar to fetchFromGitHub above.
<function>buildFHSUserEnv</function> provides a way to build and run FHS-compatible lightweight sandboxes. It creates an isolated root with bound <filename>/nix/store</filename>, so its footprint in terms of disk space needed is quite small. This allows one to run software which is hard or unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions, games distributed as tarballs, software with integrity checking and/or external self-updated binaries. It uses Linux namespaces feature to create temporary lightweight environments which are destroyed after all child processes exit, without root user rights requirement. Accepted arguments are:
<function>buildFHSUserEnv</function> provides a way to build and run
FHS-compatible lightweight sandboxes. It creates an isolated root with bound
<filename>/nix/store</filename>, so its footprint in terms of disk space
needed is quite small. This allows one to run software which is hard or
unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions,
games distributed as tarballs, software with integrity checking and/or
external self-updated binaries. It uses Linux namespaces feature to create
temporary lightweight environments which are destroyed after all child
processes exit, without root user rights requirement. Accepted arguments are:
</para>
<variablelist>
@@ -25,7 +33,8 @@
</term>
<listitem>
<para>
Packages to be installed for the main host's architecture (i.e. x86_64 on x86_64 installations). Along with libraries binaries are also installed.
Packages to be installed for the main host's architecture (i.e. x86_64 on
x86_64 installations). Along with libraries binaries are also installed.
</para>
</listitem>
</varlistentry>
@@ -35,7 +44,9 @@
</term>
<listitem>
<para>
Packages to be installed for all architectures supported by a host (i.e. i686 and x86_64 on x86_64 installations). Only libraries are installed by default.
Packages to be installed for all architectures supported by a host (i.e.
i686 and x86_64 on x86_64 installations). Only libraries are installed by
default.
</para>
</listitem>
</varlistentry>
@@ -55,7 +66,8 @@
</term>
<listitem>
<para>
Like <literal>extraBuildCommands</literal>, but executed only on multilib architectures.
Like <literal>extraBuildCommands</literal>, but executed only on multilib
architectures.
</para>
</listitem>
</varlistentry>
@@ -65,7 +77,8 @@
</term>
<listitem>
<para>
Additional derivation outputs to be linked for both target and multi-architecture packages.
Additional derivation outputs to be linked for both target and
multi-architecture packages.
</para>
</listitem>
</varlistentry>
@@ -75,7 +88,8 @@
</term>
<listitem>
<para>
Additional commands to be executed for finalizing the derivation with runner script.
Additional commands to be executed for finalizing the derivation with
runner script.
</para>
</listitem>
</varlistentry>
@@ -85,14 +99,16 @@
</term>
<listitem>
<para>
A command that would be executed inside the sandbox and passed all the command line arguments. It defaults to <literal>bash</literal>.
A command that would be executed inside the sandbox and passed all the
command line arguments. It defaults to <literal>bash</literal>.
</para>
</listitem>
</varlistentry>
</variablelist>
<para>
One can create a simple environment using a <literal>shell.nix</literal> like that:
One can create a simple environment using a <literal>shell.nix</literal> like
that:
</para>
<programlisting><![CDATA[
@@ -117,6 +133,10 @@
]]></programlisting>
<para>
Running <literal>nix-shell</literal> would then drop you into a shell with these libraries and binaries available. You can use this to run closed-source applications which expect FHS structure without hassles: simply change <literal>runScript</literal> to the application path, e.g. <filename>./bin/start.sh</filename> -- relative paths are supported.
Running <literal>nix-shell</literal> would then drop you into a shell with
these libraries and binaries available. You can use this to run closed-source
applications which expect FHS structure without hassles: simply change
<literal>runScript</literal> to the application path, e.g.
<filename>./bin/start.sh</filename> -- relative paths are supported.
Generators are functions that create file formats from nix data structures, e.g. for configuration files. There are generators available for: <literal>INI</literal>, <literal>JSON</literal> and <literal>YAML</literal>
Generators are functions that create file formats from nix data structures,
e.g. for configuration files. There are generators available for:
<literal>INI</literal>, <literal>JSON</literal> and <literal>YAML</literal>
</para>
<para>
All generators follow a similar call interface: <code>generatorName configFunctions data</code>, where <literal>configFunctions</literal> is an attrset of user-defined functions that format nested parts of the content. They each have common defaults, so often they do not need to be set manually. An example is <code>mkSectionName ? (name: libStr.escape [ "[" "]" ] name)</code> from the <literal>INI</literal> generator. It receives the name of a section and sanitizes it. The default <literal>mkSectionName</literal> escapes <literal>[</literal> and <literal>]</literal> with a backslash.
All generators follow a similar call interface: <code>generatorName
configFunctions data</code>, where <literal>configFunctions</literal> is an
attrset of user-defined functions that format nested parts of the content.
They each have common defaults, so often they do not need to be set manually.
An example is <code>mkSectionName ? (name: libStr.escape [ "[" "]" ]
name)</code> from the <literal>INI</literal> generator. It receives the name
of a section and sanitizes it. The default <literal>mkSectionName</literal>
escapes <literal>[</literal> and <literal>]</literal> with a backslash.
</para>
<para>
Generators can be fine-tuned to produce exactly the file format required by your application/service. One example is an INI-file format which uses <literal>: </literal> as separator, the strings <literal>"yes"</literal>/<literal>"no"</literal> as boolean values and requires all string values to be quoted:
Generators can be fine-tuned to produce exactly the file format required by
your application/service. One example is an INI-file format which uses
<literal>: </literal> as separator, the strings
<literal>"yes"</literal>/<literal>"no"</literal> as boolean values and
requires all string values to be quoted:
</para>
<programlisting>
@@ -64,11 +77,13 @@ merge:"diff3"
<note>
<para>
Nix store paths can be converted to strings by enclosing a derivation attribute like so: <code>"${drv}"</code>.
Nix store paths can be converted to strings by enclosing a derivation
attribute like so: <code>"${drv}"</code>.
</para>
</note>
<para>
Detailed documentation for each generator can be found in<literal>lib/generators.nix</literal>.
Detailed documentation for each generator can be found in
Specialized <function>asserts.assertMsg</function> for checking if <varname>val</varname> is one of the elements of <varname>xs</varname>. Useful for checking enums.
</para>
<variablelist>
<varlistentry>
<term>
<varname>name</varname>
</term>
<listitem>
<para>
The name of the variable the user entered <varname>val</varname> into, for inclusion in the error message.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<varname>val</varname>
</term>
<listitem>
<para>
The value of what the user provided, to be compared against the values in <varname>xs</varname>.
Recursively collect sets that verify a given predicate named<varname>pred</varname> from the set <varname>attrs</varname>. The recursion stops when <varname>pred</varname> returns <literal>true</literal>.
Recursively collect sets that verify a given predicate named
<varname>pred</varname> from the set <varname>attrs</varname>. The recursion
stops when <varname>pred</varname> returns <literal>true</literal>.
Like <function>mapAttrs</function>, but allows the name of each attribute to be changed in addition to the value. The applied function should return both the new name and value as a <function>nameValuePair</function>.
Like <function>mapAttrs</function>, but allows the name of each attribute to
be changed in addition to the value. The applied function should return both
the new name and value as a <function>nameValuePair</function>.
</para>
<variablelist>
@@ -801,8 +829,10 @@ lib.attrsets.mapAttrs
<literal>String -> Any -> { name = String; value = Any }</literal>
</para>
<para>
Given an attribute's name and value, return a new<link
linkend="function-library-lib.attrsets.nameValuePair">name value pair</link>.
Like <function>mapAttrs</function>, except that it recursively applies itself to attribute sets. Also, the first argument of the argument function is a <emphasis>list</emphasis> of the names of the containing attributes.
Like <function>mapAttrs</function>, except that it recursively applies
itself to attribute sets. Also, the first argument of the argument function
is a <emphasis>list</emphasis> of the names of the containing attributes.
For example, the <varname>name_path</varname> for the<literal>example</literal> string in the attribute set <literal>{ foo = { bar = "example"; }; }</literal> is <literal>[ "foo" "bar" ]</literal>.
For example, the <varname>name_path</varname> for the
<literal>example</literal> string in the attribute set <literal>{ foo
= { bar = "example"; }; }</literal> is <literal>[ "foo" "bar"
Like <function>mapAttrsRecursive</function>, but it takes an additional predicate function that tells it whether to recursive into an attribute set. If it returns false, <function>mapAttrsRecursiveCond</function> does not recurse, but does apply the map function. It is returns true, it does recurse, and does not apply the map function.
Like <function>mapAttrsRecursive</function>, but it takes an additional
predicate function that tells it whether to recursive into an attribute set.
If it returns false, <function>mapAttrsRecursiveCond</function> does not
recurse, but does apply the map function. It is returns true, it does
recurse, and does not apply the map function.
</para>
<variablelist>
@@ -1036,7 +1076,8 @@ mapAttrsRecursive
<literal>(AttrSet -> Bool)</literal>
</para>
<para>
Determine if <function>mapAttrsRecursive</function> should recurse deeper in to the attribute set.
Determine if <function>mapAttrsRecursive</function> should recurse deeper
in to the attribute set.
</para>
<variablelist>
<varlistentry>
@@ -1073,7 +1114,10 @@ mapAttrsRecursive
The list of attribute names to this value.
</para>
<para>
For example, the <varname>name_path</varname> for the<literal>example</literal> string in the attribute set <literal>{ foo = { bar = "example"; }; }</literal> is <literal>[ "foo" "bar" ]</literal>.
For example, the <varname>name_path</varname> for the
<literal>example</literal> string in the attribute set <literal>{ foo
= { bar = "example"; }; }</literal> is <literal>[ "foo" "bar"
Merge sets of attributes and combine each attribute value in to a list. Similar to <xreflinkend="function-library-lib.attrsets.zipAttrsWith"/> where the merge function returns a list of all values.
Merge sets of attributes and combine each attribute value in to a list.
Similar to <xreflinkend="function-library-lib.attrsets.zipAttrsWith"/>
where the merge function returns a list of all values.
Does the same as the update operator <literal>//</literal> except that attributes are merged until the given predicate is verified. The predicate should accept 3 arguments which are the path to reach the attribute, a part of the first attribute set and a part of the second attribute set. When the predicate is verified, the value of the first attribute set is replaced by the value of the second attribute set.
Does the same as the update operator <literal>//</literal> except that
attributes are merged until the given predicate is verified. The predicate
should accept 3 arguments which are the path to reach the attribute, a part
of the first attribute set and a part of the second attribute set. When the
predicate is verified, the value of the first attribute set is replaced by
A recursive variant of the update operator <literal>//</literal>. The recursion stops when one of the attribute values is not an attribute set, in which case the right hand side value takes precedence over the left hand side value.
A recursive variant of the update operator <literal>//</literal>. The
recursion stops when one of the attribute values is not an attribute set, in
which case the right hand side value takes precedence over the left hand
<function>pkgs.nix-gitignore</function> is a function that acts similarly to <literal>builtins.filterSource</literal> but also allows filtering with the help of the gitignore format.
</para>
<sectionxml:id="sec-pkgs-nix-gitignore-usage">
<title>Usage</title>
<para>
<literal>pkgs.nix-gitignore</literal> exports a number of functions, but you'll most likely need either <literal>gitignoreSource</literal> or <literal>gitignoreSourcePure</literal>. As their first argument, they both accept either 1. a file with gitignore lines or 2. a string with gitignore lines, or 3. a list of either of the two. They will be concatenated into a single big string.
Those filter functions accept the same arguments the <literal>builtins.filterSource</literal> function would pass to its filters, thus <literal>fn: gitignoreFilterSourcePure fn ""</literal> should be extensionally equivalent to <literal>filterSource</literal>. The file is blacklisted iff it's blacklisted by either your filter or the gitignoreFilter.
</para>
<para>
If you want to make your own filter from scratch, you may use
<varname>pkgs.ociTools</varname> is a set of functions for creating containers according to the <linkxlink:href="https://github.com/opencontainers/runtime-spec">OCI container specification v1.0.0</link>. Beyond that it makes no assumptions about the container runner you choose to use to run the created container.
This function creates a simple OCI container that runs a single command inside of it. An OCI container consists of a <varname>config.json</varname> and a rootfs directory.The nix store of the container will contain all referenced dependencies of the given command.
</para>
<para>
The parameters of <varname>buildContainer</varname> with an example value are described below:
<varname>args</varname> specifies a set of arguments to run inside the container. This is the only required argument for <varname>buildContainer</varname>. All referenced packages inside the derivation will be made available inside the container
</para>
</callout>
<calloutarearefs='ex-ociTools-buildContainer-2'>
<para>
<varname>mounts</varname> specifies additional mount points chosen by the user. By default only a minimal set of necessary filesystems are mounted into the container (e.g procfs, cgroupfs)
</para>
</callout>
<calloutarearefs='ex-ociTools-buildContainer-3'>
<para>
<varname>readonly</varname> makes the container's rootfs read-only if it is set to true. The default value is false <literal>false</literal>.
Sometimes one wants to override parts of <literal>nixpkgs</literal>, e.g. derivation attributes, the results of derivations.
</para>
<para>
These functions are used to make changes to packages, returning only single packages. <linkxlink:href="#chap-overlays">Overlays</link>, on the other hand, can be used to combine the overridden packages across the entire package set of Nixpkgs.
Sometimes one wants to override parts of <literal>nixpkgs</literal>, e.g.
derivation attributes, the results of derivations or even the whole package
set.
</para>
<sectionxml:id="sec-pkg-override">
<title><pkg>.override</title>
<para>
The function <varname>override</varname> is usually available for all the derivations in the nixpkgs expression (<varname>pkgs</varname>).
The function <varname>override</varname> is usually available for all the
derivations in the nixpkgs expression (<varname>pkgs</varname>).
In the first example, <varname>pkgs.foo</varname> is the result of a function call with some default arguments, usually a derivation. Using <varname>pkgs.foo.override</varname> will call the same function with the given new arguments.
In the first example, <varname>pkgs.foo</varname> is the result of a
function call with some default arguments, usually a derivation. Using
<varname>pkgs.foo.override</varname> will call the same function with the
The function <varname>overrideAttrs</varname> allows overriding the attribute set passed to a <varname>stdenv.mkDerivation</varname> call, producing a new derivation based on the original one. This function is available on all derivations produced by the <varname>stdenv.mkDerivation</varname> function, which is most packages in the nixpkgs expression <varname>pkgs</varname>.
The function <varname>overrideAttrs</varname> allows overriding the
attribute set passed to a <varname>stdenv.mkDerivation</varname> call,
producing a new derivation based on the original one. This function is
available on all derivations produced by the
<varname>stdenv.mkDerivation</varname> function, which is most packages in
In the above example, the <varname>separateDebugInfo</varname> attribute is overridden to be true, thus building debug info for <varname>helloWithDebug</varname>, while all other attributes will be retained from the original <varname>hello</varname> package.
In the above example, the <varname>separateDebugInfo</varname> attribute is
overridden to be true, thus building debug info for
<varname>helloWithDebug</varname>, while all other attributes will be
retained from the original <varname>hello</varname> package.
</para>
<para>
The argument <varname>oldAttrs</varname> is conventionally used to refer to the attr set originally passed to <varname>stdenv.mkDerivation</varname>.
The argument <varname>oldAttrs</varname> is conventionally used to refer to
the attr set originally passed to <varname>stdenv.mkDerivation</varname>.
</para>
<note>
<para>
Note that <varname>separateDebugInfo</varname> is processed only by the<varname>stdenv.mkDerivation</varname> function, not the generated, raw Nix derivation. Thus, using <varname>overrideDerivation</varname> will not work in this case, as it overrides only the attributes of the final derivation. It is for this reason that <varname>overrideAttrs</varname> should be preferred in (almost) all cases to <varname>overrideDerivation</varname>, i.e. to allow using <varname>stdenv.mkDerivation</varname> to process input arguments, as well as the fact that it is easier to use (you can use the same attribute names you see in your Nix code, instead of the ones generated (e.g. <varname>buildInputs</varname> vs <varname>nativeBuildInputs</varname>), and it involves less typing).
Note that <varname>separateDebugInfo</varname> is processed only by the
<varname>stdenv.mkDerivation</varname> function, not the generated, raw Nix
derivation. Thus, using <varname>overrideDerivation</varname> will not work
in this case, as it overrides only the attributes of the final derivation.
It is for this reason that <varname>overrideAttrs</varname> should be
preferred in (almost) all cases to <varname>overrideDerivation</varname>,
i.e. to allow using <varname>sdenv.mkDerivation</varname> to process input
arguments, as well as the fact that it is easier to use (you can use the
same attribute names you see in your Nix code, instead of the ones
generated (e.g. <varname>buildInputs</varname> vs
<varname>nativeBuildInputs</varname>, and involves less typing.
You should prefer <varname>overrideAttrs</varname> in almost all cases, see its documentation for the reasons why. <varname>overrideDerivation</varname> is not deprecated and will continue to work, but is less nice to use and does not have as many abilities as <varname>overrideAttrs</varname>.
You should prefer <varname>overrideAttrs</varname> in almost all cases, see
its documentation for the reasons why.
<varname>overrideDerivation</varname> is not deprecated and will continue
to work, but is less nice to use and does not have as many abilities as
<varname>overrideAttrs</varname>.
</para>
</warning>
<warning>
<para>
Do not use this function in Nixpkgs as it evaluates a Derivation before modifying it, which breaks package abstraction and removes error-checking of function arguments. In addition, this evaluation-per-function application incurs a performance penalty, which can become a problem if many overrides are used. It is only intended for ad-hoc customisation, such as in <filename>~/.config/nixpkgs/config.nix</filename>.
Do not use this function in Nixpkgs as it evaluates a Derivation before
modifying it, which breaks package abstraction and removes error-checking
of function arguments. In addition, this evaluation-per-function
application incurs a performance penalty, which can become a problem if
many overrides are used. It is only intended for ad-hoc customisation, such
as in <filename>~/.config/nixpkgs/config.nix</filename>.
</para>
</warning>
<para>
The function <varname>overrideDerivation</varname> creates a new derivation based on an existing one by overriding the original's attributes with the attribute set produced by the specified function. This function is available on all derivations defined using the <varname>makeOverridable</varname> function. Most standard derivation-producing functions, such as <varname>stdenv.mkDerivation</varname>, are defined using this function, which means most packages in the nixpkgs expression, <varname>pkgs</varname>, have this function.
The function <varname>overrideDerivation</varname> creates a new derivation
based on an existing one by overriding the original's attributes with the
attribute set produced by the specified function. This function is available
on all derivations defined using the <varname>makeOverridable</varname>
function. Most standard derivation-producing functions, such as
<varname>stdenv.mkDerivation</varname>, are defined using this function,
which means most packages in the nixpkgs expression,
In the above example, the <varname>name</varname>, <varname>src</varname>, and <varname>patches</varname> of the derivation will be overridden, while all other attributes will be retained from the original derivation.
In the above example, the <varname>name</varname>, <varname>src</varname>,
and <varname>patches</varname> of the derivation will be overridden, while
all other attributes will be retained from the original derivation.
</para>
<para>
The argument <varname>oldAttrs</varname> is used to refer to the attribute set of the original derivation.
The argument <varname>oldAttrs</varname> is used to refer to the attribute
set of the original derivation.
</para>
<note>
<para>
A package's attributes are evaluated *before* being modified by the<varname>overrideDerivation</varname> function. For example, the <varname>name</varname> attribute reference in <varname>url = "mirror://gnu/hello/${name}.tar.gz";</varname> is filled-in *before* the <varname>overrideDerivation</varname> function modifies the attribute set. This means that overriding the <varname>name</varname> attribute, in this example, *will not* change the value of the <varname>url</varname> attribute. Instead, we need to override both the <varname>name</varname> *and* <varname>url</varname> attributes.
A package's attributes are evaluated *before* being modified by the
<varname>overrideDerivation</varname> function. For example, the
<varname>name</varname> attribute reference in <varname>url =
"mirror://gnu/hello/${name}.tar.gz";</varname> is filled-in *before* the
<varname>overrideDerivation</varname> function modifies the attribute set.
This means that overriding the <varname>name</varname> attribute, in this
example, *will not* change the value of the <varname>url</varname>
attribute. Instead, we need to override both the <varname>name</varname>
The function <varname>lib.makeOverridable</varname> is used to make the result of a function easily customizable. This utility only makes sense for functions that accept an argument set and return an attribute set.
The function <varname>lib.makeOverridable</varname> is used to make the
result of a function easily customizable. This utility only makes sense for
functions that accept an argument set and return an attribute set.
</para>
<para>
Example usage:
<programlisting>
f = { a, b }: { result = a+b; };
c = lib.makeOverridable f { a = 1; b = 2; };
</programlisting>
<programlisting>f = { a, b }: { result = a+b; }
c = lib.makeOverridable f { a = 1; b = 2; }</programlisting>
</para>
<para>
The variable <varname>c</varname> is the value of the <varname>f</varname> function applied with some default arguments. Hence the value of <varname>c.result</varname> is <literal>3</literal>, in this example.
The variable <varname>c</varname> is the value of the <varname>f</varname>
function applied with some default arguments. Hence the value of
<varname>c.result</varname> is <literal>3</literal>, in this example.
</para>
<para>
The variable <varname>c</varname> however also has some additional functions, like <linklinkend="sec-pkg-override">c.override</link> which can be used to override the default arguments. In this example the value of <varname>(c.override { a = 4; }).result</varname> is 6.
The variable <varname>c</varname> however also has some additional
functions, like <linklinkend="sec-pkg-override">c.override</link> which can
be used to override the default arguments. In this example the value of
<varname>(c.override { a = 4; }).result</varname> is 6.
<function>prefer-remote-fetch</function> is an overlay that download sources on remote builder. This is useful when the evaluating machine has a slow upload while the builder can fetch faster directly from the source. To use it, put the following snippet as a new overlay:
<programlisting>
self: super:
(super.prefer-remote-fetch self super)
</programlisting>
A full configuration example for that sets the overlay up for your own account, could look like this
<function>pkgs.mkShell</function> is a special kind of derivation that is only useful when using it combined with <command>nix-shell</command>. It will in fact fail to instantiate when invoked with <command>nix-build</command>.
<function>pkgs.mkShell</function> is a special kind of derivation that is
only useful when using it combined with <command>nix-shell</command>. It will
in fact fail to instantiate when invoked with <command>nix-build</command>.
<function>makeSnap</function> takes a single named argument, <parameter>meta</parameter>. This argument mirrors <linkxlink:href="https://docs.snapcraft.io/snap-format">the upstream <filename>snap.yaml</filename> format</link> exactly.
</para>
<para>
The <parameter>base</parameter> should not be be specified, as <function>makeSnap</function> will force set it.
</para>
<para>
Currently, <function>makeSnap</function> does not support creating GUI stubs.
<command>nix-build</command> this expression and install it with <command>snap install ./result --dangerous</command>. <command>hello</command> will now be the Snapcraft version of the package.
Graphical programs require many more integrations with the host. This example uses Firefox as an example, because it is one of the most complicated programs we could package.
<command>nix-build</command> this expression and install it with <command>snap install ./result --dangerous</command>. <command>nix-example-firefox</command> will now be the Snapcraft version of the Firefox package.
</para>
<para>
The specific meaning behind plugs can be looked up in the <linkxlink:href="https://docs.snapcraft.io/supported-interfaces">Snapcraft interface documentation</link>.
Nixpkgs provides a couple of functions that help with building derivations. The most important one, <function>stdenv.mkDerivation</function>, has already been documented above. The following functions wrap <function>stdenv.mkDerivation</function>, making it easier to use in certain cases.
</para>
<variablelist>
<varlistentry>
<term>
<literal>runCommand</literal>
</term>
<listitem>
<para>
This takes three arguments, <literal>name</literal>, <literal>env</literal>, and <literal>buildCommand</literal>. <literal>name</literal> is just the name that Nix will append to the store path in the same way that <literal>stdenv.mkDerivation</literal> uses its <literal>name</literal> attribute. <literal>env</literal> is an attribute set specifying environment variables that will be set for this derivation. These attributes are then passed to the wrapped <literal>stdenv.mkDerivation</literal>. <literal>buildCommand</literal> specifies the commands that will be run to create this derivation. Note that you will need to create <literal>$out</literal> for Nix to register the command as successful.
</para>
<para>
An example of using <literal>runCommand</literal> is provided below.
echo I can write data to the Nix store > $out/message
echo I can also run basic commands like:
echo ls
ls
echo whoami
whoami
echo date
date
''
</programlisting>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>runCommandCC</literal>
</term>
<listitem>
<para>
This works just like <literal>runCommand</literal>. The only difference is that it also provides a C compiler in <literal>buildCommand</literal>’s environment. To minimize your dependencies, you should only use this if you are sure you will need a C compiler as part of running your command.
These functions write <literal>text</literal> to the Nix store. This is useful for creating scripts from Nix expressions. <literal>writeTextFile</literal> takes an attribute set and expects two arguments, <literal>name</literal> and <literal>text</literal>. <literal>name</literal> corresponds to the name used in the Nix store path. <literal>text</literal> will be the contents of the file. You can also set <literal>executable</literal> to true to make this file have the executable bit set.
</para>
<para>
Many more commands wrap <literal>writeTextFile</literal> including <literal>writeText</literal>, <literal>writeTextDir</literal>, <literal>writeScript</literal>, and <literal>writeScriptBin</literal>. These are convenience functions over <literal>writeTextFile</literal>.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<literal>symlinkJoin</literal>
</term>
<listitem>
<para>
This can be used to put many derivations into the same directory structure. It works by creating a new derivation and adding symlinks to each of the paths listed. It expects two arguments, <literal>name</literal>, and <literal>paths</literal>. <literal>name</literal> is the name used in the Nix store path for the created derivation. <literal>paths</literal> is a list of paths that will be symlinked. These paths can be to Nix store derivations or any other subdirectory contained within.
In this document and related Nix expressions, we use the term,<emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction.
In this document and related Nix expressions, we use the term,
<emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name of
the Erlang Virtual Machine and, as far as we're concerned, from a packaging
perspective, all languages that run on the BEAM are interchangeable. That
which varies, like the build system, is transparent to users of any given
BEAM package, so we make no distinction.
</para>
</section>
@@ -15,36 +20,57 @@
<title>Structure</title>
<para>
All BEAM-related expressions are available via the top-level<literal>beam</literal> attribute, which includes:
All BEAM-related expressions are available via the top-level
<literal>beam</literal> attribute, which includes:
</para>
<itemizedlist>
<listitem>
<para>
<literal>interpreters</literal>: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (<literal>beam.interpreters.erlangR19</literal>, etc), Elixir (<literal>beam.interpreters.elixir</literal>) and LFE (<literal>beam.interpreters.lfe</literal>).
<literal>interpreters</literal>: a set of compilers running on the BEAM,
(<literal>beam.interpreters.elixir</literal>) and LFE
(<literal>beam.interpreters.lfe</literal>).
</para>
</listitem>
<listitem>
<para>
<literal>packages</literal>: a set of package sets, each compiled with a specific Erlang/OTP version, e.g. <literal>beam.packages.erlangR19</literal>.
<literal>packages</literal>: a set of package sets, each compiled with a
specific Erlang/OTP version, e.g.
<literal>beam.packages.erlangR19</literal>.
</para>
</listitem>
</itemizedlist>
<para>
The default Erlang compiler, defined by<literal>beam.interpreters.erlang</literal>, is aliased as <literal>erlang</literal>. The default BEAM package set is defined by <literal>beam.packages.erlang</literal> and aliased at the top level as <literal>beamPackages</literal>.
The default Erlang compiler, defined by
<literal>beam.interpreters.erlang</literal>, is aliased as
<literal>erlang</literal>. The default BEAM package set is defined by
<literal>beam.packages.erlang</literal> and aliased at the top level as
<literal>beamPackages</literal>.
</para>
<para>
To create a package set built with a custom Erlang version, use the lambda,<literal>beam.packagesWith</literal>, which accepts an Erlang/OTP derivation and produces a package set similar to <literal>beam.packages.erlang</literal>.
To create a package set built with a custom Erlang version, use the lambda,
<literal>beam.packagesWith</literal>, which accepts an Erlang/OTP derivation
and produces a package set similar to
<literal>beam.packages.erlang</literal>.
</para>
<para>
Many Erlang/OTP distributions available in<literal>beam.interpreters</literal> have versions with ODBC and/or Java enabled. For example, there's <literal>beam.interpreters.erlangR19_odbc_javac</literal>, which corresponds to <literal>beam.interpreters.erlangR19</literal>.
Many Erlang/OTP distributions available in
<literal>beam.interpreters</literal> have versions with ODBC and/or Java
enabled. For example, there's
<literal>beam.interpreters.erlangR19_odbc_javac</literal>, which corresponds
to <literal>beam.interpreters.erlangR19</literal>.
</para>
<paraxml:id="erlang-call-package">
We also provide the lambda,<literal>beam.packages.erlang.callPackage</literal>, which simplifies writing BEAM package definitions by injecting all packages from <literal>beam.packages.erlang</literal> into the top-level context.
We also provide the lambda,
<literal>beam.packages.erlang.callPackage</literal>, which simplifies
writing BEAM package definitions by injecting all packages from
<literal>beam.packages.erlang</literal> into the top-level context.
</para>
</section>
@@ -55,7 +81,28 @@
<title>Rebar3</title>
<para>
We provide a version of Rebar3, which is the normal, unmodified Rebar3, under <literal>rebar3</literal>. We also provide a helper to fetch Rebar3 dependencies from a lockfile under <literal>fetchRebar3Deps</literal>.
By default, Rebar3 wants to manage its own dependencies. This is perfectly
acceptable in the normal, non-Nix setup, but in the Nix world, it is not.
To rectify this, we provide two versions of Rebar3:
<itemizedlist>
<listitem>
<para>
<literal>rebar3</literal>: patched to remove the ability to download
anything. When not running it via <literal>nix-shell</literal> or
<literal>nix-build</literal>, it's probably not going to work as
desired.
</para>
</listitem>
<listitem>
<para>
<literal>rebar3-open</literal>: the normal, unmodified Rebar3. It should
work exactly as would any other version of Rebar3. Any Erlang package
should rely on <literal>rebar3</literal> instead. See
<xref
linkend="rebar3-packages"/>.
</para>
</listitem>
</itemizedlist>
</para>
</section>
@@ -63,7 +110,10 @@
<title>Mix & Erlang.mk</title>
<para>
Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the <literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations, respectively.
Both Mix and Erlang.mk work exactly as expected. There is a bootstrap
process that needs to be run for both, however, which is supported by the
<literal>buildMix</literal> and <literal>buildErlangMk</literal>
derivations, respectively.
</para>
</section>
</section>
@@ -72,11 +122,17 @@
<title>How to Install BEAM Packages</title>
<para>
BEAM packages are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. They are installable using the <literal>beam.packages.erlang</literal> attribute set (aliased as <literal>beamPackages</literal>), which points to packages built by the default Erlang/OTP version in Nixpkgs, as defined by <literal>beam.interpreters.erlang</literal>. To list the available packages in <literal>beamPackages</literal>, use the following command:
BEAM packages are not registered at the top level, simply because they are
not relevant to the vast majority of Nix users. They are installable using
the <literal>beam.packages.erlang</literal> attribute set (aliased as
<literal>beamPackages</literal>), which points to packages built by the
default Erlang/OTP version in Nixpkgs, as defined by
<literal>beam.interpreters.erlang</literal>. To list the available packages
in <literal>beamPackages</literal>, use the following command:
</para>
<screen>
<prompt>$ </prompt>nix-env -f "<nixpkgs>" -qaP -A beamPackages
<programlisting>
$ nix-env -f "<nixpkgs>" -qaP -A beamPackages
The attribute path of any BEAM package corresponds to the name of that particular package in <linkxlink:href="https://hex.pm">Hex</link> or its OTP Application/Release name.
The attribute path of any BEAM package corresponds to the name of that
particular package in <linkxlink:href="https://hex.pm">Hex</link> or its
The Nix function, <literal>buildRebar3</literal>, defined in<literal>beam.packages.erlang.buildRebar3</literal> and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. For example, we can build <link
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> as follows:
The Nix function, <literal>buildRebar3</literal>, defined in
<literal>beam.packages.erlang.buildRebar3</literal> and aliased at the top
level, can be used to build a derivation that understands how to build a
Such derivations are callable with<literal>beam.packages.erlang.callPackage</literal> (see <xref
linkend="erlang-call-package"/>). To call this package using the normal <literal>callPackage</literal>, refer to dependency packages via <literal>beamPackages</literal>, e.g. <literal>beamPackages.ibrowse</literal>.
Such derivations are callable with
<literal>beam.packages.erlang.callPackage</literal> (see
<xref
linkend="erlang-call-package"/>). To call this package using
the normal <literal>callPackage</literal>, refer to dependency packages
via <literal>beamPackages</literal>, e.g.
<literal>beamPackages.ibrowse</literal>.
</para>
<para>
Notably, <literal>buildRebar3</literal> includes<literal>beamDeps</literal>, while <literal>stdenv.mkDerivation</literal> does not. BEAM dependencies added there will be correctly handled by the system.
Notably, <literal>buildRebar3</literal> includes
<literal>beamDeps</literal>, while <literal>stdenv.mkDerivation</literal>
does not. BEAM dependencies added there will be correctly handled by the
system.
</para>
<para>
If a package needs to compile native code via Rebar3's port compilation mechanism, add <literal>compilePort = true;</literal> to the derivation.
If a package needs to compile native code via Rebar3's port compilation
mechanism, add <literal>compilePort = true;</literal> to the derivation.
</para>
</section>
@@ -149,93 +222,96 @@ buildRebar3 rec {
<title>Erlang.mk Packages</title>
<para>
Erlang.mk functions similarly to Rebar3, except we use<literal>buildErlangMk</literal> instead of <literal>buildRebar3</literal>.
Erlang.mk functions similarly to Rebar3, except we use
Often, we simply want to access a valid environment that contains a specific package and its dependencies. We can accomplish that with the <literal>env</literal> attribute of a derivation. For example, let's say we want to access an Erlang REPL with <literal>ibrowse</literal> loaded up. We could do the following:
Often, we simply want to access a valid environment that contains a
specific package and its dependencies. We can accomplish that with the
<literal>env</literal> attribute of a derivation. For example, let's say we
want to access an Erlang REPL with <literal>ibrowse</literal> loaded up. We
could do the following:
</para>
<screen>
<prompt>$ </prompt><userinput>nix-shell -A beamPackages.ibrowse.env --run "erl"</userinput>
Notice the <literal>-A beamPackages.ibrowse.env</literal>. That is the key to this functionality.
Notice the <literal>-A beamPackages.ibrowse.env</literal>. That is the key
to this functionality.
</para>
</section>
@@ -301,7 +382,11 @@ ok</computeroutput>
<title>Creating a Shell</title>
<para>
Getting access to an environment often isn't enough to do real development. Usually, we need to create a <literal>shell.nix</literal> file and do our development inside of the environment specified therein. This file looks a lot like the packaging described above, except that <literal>src</literal> points to the project root and we call the package directly.
Getting access to an environment often isn't enough to do real development.
Usually, we need to create a <literal>shell.nix</literal> file and do our
development inside of the environment specified therein. This file looks a
lot like the packaging described above, except that <literal>src</literal>
points to the project root and we call the package directly.
</para>
<programlisting>
@@ -323,13 +408,14 @@ let
in
drv
</programlisting>
</programlisting>
<sectionxml:id="building-in-a-shell">
<title>Building in a Shell (for Mix Projects)</title>
<para>
We can leverage the support of the derivation, irrespective of the build derivation, by calling the commands themselves.
We can leverage the support of the derivation, irrespective of the build
derivation, by calling the commands themselves.
</para>
<programlisting>
@@ -388,11 +474,15 @@ plt:
analyze: build plt
$(NIX_SHELL) --run "mix dialyzer --no-compile"
</programlisting>
</programlisting>
<para>
Using a <literal>shell.nix</literal> as described (see<xref
linkend="creating-a-shell"/>) should just work. Aside from <literal>test</literal>, <literal>plt</literal>, and <literal>analyze</literal>, the Make targets work just fine for all of the build derivations.
Using a <literal>shell.nix</literal> as described (see
<xref
linkend="creating-a-shell"/>) should just work. Aside from
<literal>test</literal>, <literal>plt</literal>, and
<literal>analyze</literal>, the Make targets work just fine for all of the
build derivations.
</para>
</section>
</section>
@@ -402,23 +492,37 @@ analyze: build plt
<title>Generating Packages from Hex with <literal>hex2nix</literal></title>
<para>
Updating the <linkxlink:href="https://hex.pm">Hex</link> package set requires <link
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link>. Given the path to the Erlang modules (usually <literal>pkgs/development/erlang-modules</literal>), it will dump a file called <literal>hex-packages.nix</literal>, containing all the packages that use a recognized build system in <link
xlink:href="https://hex.pm">Hex</link>. It can't be determined, however, whether every package is buildable.
Updating the <linkxlink:href="https://hex.pm">Hex</link> package set
<literal>pkgs/development/erlang-modules</literal>), it will dump a file
called <literal>hex-packages.nix</literal>, containing all the packages that
use a recognized build system in
<link
xlink:href="https://hex.pm">Hex</link>. It can't be determined,
however, whether every package is buildable.
</para>
<para>
To make life easier for our users, try to build every<link
xlink:href="https://hex.pm">Hex</link> package and remove those that fail. To do that, simply run the following command in the root of your <literal>nixpkgs</literal> repository:
To make life easier for our users, try to build every
<link
xlink:href="https://hex.pm">Hex</link> package and remove those
that fail. To do that, simply run the following command in the root of your
<literal>nixpkgs</literal> repository:
</para>
<screen>
<prompt>$ </prompt>nix-build -A beamPackages
</screen>
<programlisting>
$ nix-build -A beamPackages
</programlisting>
<para>
That will attempt to build every package in <literal>beamPackages</literal>. Then manually remove those that fail. Hopefully, someone will improve <link
xlink:href="https://github.com/erlang-nix/hex2nix">hex2nix</link> in the future to automate the process.
That will attempt to build every package in <literal>beamPackages</literal>.
Then manually remove those that fail. Hopefully, someone will improve
<linkxlink:href="http://bower.io">Bower</link> is a package manager for web site front-end components. Bower packages (comprising of build artefacts and sometimes sources) are stored in <command>git</command> repositories, typically on Github. The package registry is run by the Bower team with package metadata coming from the <filename>bower.json</filename> file within each package.
<linkxlink:href="http://bower.io">Bower</link> is a package manager for web
site front-end components. Bower packages (comprising of build artefacts and
sometimes sources) are stored in <command>git</command> repositories,
typically on Github. The package registry is run by the Bower team with
package metadata coming from the <filename>bower.json</filename> file within
each package.
</para>
<para>
The end result of running Bower is a <filename>bower_components</filename> directory which can be included in the web app's build process.
The end result of running Bower is a <filename>bower_components</filename>
directory which can be included in the web app's build process.
</para>
<para>
Bower can be run interactively, by installing<varname>nodePackages.bower</varname>. More interestingly, the Bower components can be declared in a Nix derivation, with the help of <varname>nodePackages.bower2nix</varname>.
Bower can be run interactively, by installing
<varname>nodePackages.bower</varname>. More interestingly, the Bower
components can be declared in a Nix derivation, with the help of
<varname>nodePackages.bower2nix</varname>.
</para>
<sectionxml:id="ssec-bower2nix-usage">
<title><command>bower2nix</command> usage</title>
<para>
Suppose you have a <filename>bower.json</filename> with the following contents:
Suppose you have a <filename>bower.json</filename> with the following
contents:
<examplexml:id="ex-bowerJson">
<title><filename>bower.json</filename></title>
<programlistinglanguage="json">
@@ -35,7 +45,8 @@
</para>
<para>
Running <command>bower2nix</command> will produce something like the following output:
Running <command>bower2nix</command> will produce something like the
Using the <command>bower2nix</command> command line arguments, the output can be redirected to a file. A name like <filename>bower-packages.nix</filename> would be fine.
Using the <command>bower2nix</command> command line arguments, the output
can be redirected to a file. A name like
<filename>bower-packages.nix</filename> would be fine.
</para>
<para>
The resulting derivation is a union of all the downloaded Bower packages (and their dependencies). To use it, they still need to be linked together by Bower, which is where <varname>buildBowerComponents</varname> is useful.
The resulting derivation is a union of all the downloaded Bower packages
(and their dependencies). To use it, they still need to be linked together
by Bower, which is where <varname>buildBowerComponents</varname> is useful.
The function is implemented in<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/bower-modules/generic/default.nix"><filename>pkgs/development/bower-modules/generic/default.nix</filename></link>. Example usage:
In <xreflinkend="ex-buildBowerComponents"/>, the following arguments are of special significance to the function:
In <xreflinkend="ex-buildBowerComponents"/>, the following arguments are
of special significance to the function:
<calloutlist>
<calloutarearefs="ex-buildBowerComponents-1">
<para>
<varname>generated</varname> specifies the file which was created by<command>bower2nix</command>.
<varname>generated</varname> specifies the file which was created by
<command>bower2nix</command>.
</para>
</callout>
<calloutarearefs="ex-buildBowerComponents-2">
<para>
<varname>src</varname> is your project's sources. It needs to contain a<filename>bower.json</filename> file.
<varname>src</varname> is your project's sources. It needs to contain a
<filename>bower.json</filename> file.
</para>
</callout>
</calloutlist>
</para>
<para>
<varname>buildBowerComponents</varname> will run Bower to link together the output of <command>bower2nix</command>, resulting in a <filename>bower_components</filename> directory which can be used.
<varname>buildBowerComponents</varname> will run Bower to link together the
output of <command>bower2nix</command>, resulting in a
<filename>bower_components</filename> directory which can be used.
</para>
<para>
Here is an example of a web frontend build process using<command>gulp</command>. You might use <command>grunt</command>, or anything else.
Here is an example of a web frontend build process using
<command>gulp</command>. You might use <command>grunt</command>, or anything
Whether to symlink or copy the <filename>bower_components</filename> directory depends on the build tool in use. In this case a copy is used to avoid <command>gulp</command> silliness with permissions.
Whether to symlink or copy the <filename>bower_components</filename>
directory depends on the build tool in use. In this case a copy is used
to avoid <command>gulp</command> silliness with permissions.
This means that Bower was looking for a package version which doesn't exist in the generated <filename>bower-packages.nix</filename>.
This means that Bower was looking for a package version which doesn't
exist in the generated <filename>bower-packages.nix</filename>.
</para>
<para>
If <filename>bower.json</filename> has been updated, then run<command>bower2nix</command> again.
If <filename>bower.json</filename> has been updated, then run
<command>bower2nix</command> again.
</para>
<para>
It could also be a bug in <command>bower2nix</command> or<command>fetchbower</command>. If possible, try reformulating the version specification in <filename>bower.json</filename>.
It could also be a bug in <command>bower2nix</command> or
<command>fetchbower</command>. If possible, try reformulating the version
Coq libraries should be installed in<literal>$(out)/lib/coq/${coq.coq-version}/user-contrib/</literal>. Such directories are automatically added to the <literal>$COQPATH</literal> environment variable by the hook defined in the Coq derivation.
Coq libraries should be installed in
<literal>$(out)/lib/coq/${coq.coq-version}/user-contrib/</literal>. Such
directories are automatically added to the <literal>$COQPATH</literal>
environment variable by the hook defined in the Coq derivation.
</para>
<para>
Some extensions (plugins) might require OCaml and sometimes other OCaml packages. The <literal>coq.ocamlPackages</literal> attribute can be used to depend on the same package set Coq was built against.
Some libraries require OCaml and sometimes also Camlp5 or findlib. The exact
versions that were used to build Coq are saved in the
<literal>coq.ocaml</literal> and <literal>coq.camlp5</literal> and
<literal>coq.findlib</literal> attributes.
</para>
<para>
Coq libraries may be compatible with some specific versions of Coq only. The<literal>compatibleCoqVersions</literal> attribute is used to precisely select those versions of Coq that are compatible with this derivation.
Coq libraries may be compatible with some specific versions of Coq only. The
<literal>compatibleCoqVersions</literal> attribute is used to precisely
select those versions of Coq that are compatible with this derivation.
</para>
<para>
Here is a simple package example. It is a pure Coq library, thus it depends on Coq. It builds on the Mathematical Components library, thus it also takes <literal>mathcomp</literal> as <literal>buildInputs</literal>. Its <literal>Makefile</literal> has been generated using <literal>coq_makefile</literal> so we only have to set the <literal>$COQLIB</literal> variable at install time.
Here is a simple package example. It is a pure Coq library, thus it depends
on Coq. It builds on the Mathematical Components library, thus it also takes
<literal>mathcomp</literal> as <literal>buildInputs</literal>. Its
<literal>Makefile</literal> has been generated using
<literal>coq_makefile</literal> so we only have to set the
<literal>$COQLIB</literal> variable at install time.
This section uses [Mint](https://github.com/mint-lang/mint) as an example for how to build a Crystal package.
If the Crystal project has any dependencies, the first step is to get a `shards.nix` file encoding those. Get a copy of the project and go to its root directory such that its `shard.lock` file is in the current directory, then run `crystal2nix` in it
```bash
$ git clone https://github.com/mint-lang/mint
$ cd mint
$ git checkout 0.5.0
$ nix-shell -p crystal2nix --run crystal2nix
```
This should have generated a `shards.nix` file.
Next create a Nix file for your derivation and use `pkgs.crystal.buildCrystalPackage` as follows:
This won't build anything yet, because we haven't told it what files build. We can specify a mapping from binary names to source files with the `crystalBinaries` attribute. The project's compilation instructions should show this. For Mint, the binary is called "mint", which is compiled from the source file `src/mint.cr`, so we'll specify this as follows:
```nix
crystalBinaries.mint.src="src/mint.cr";
# ...
```
Additionally you can override the default `crystal build` options (which are currently `--release --progress --no-debug --verbose`) with
Depending on the project, you might need additional steps to get it to compile successfully. In Mint's case, we need to link against openssl, so in the end the Nix file looks as follows:
Programs in the GNOME universe are written in various languages but they all use GObject-based libraries like GLib, GTK or GStreamer. These libraries are often modular, relying on looking into certain directories to find their modules. However, due to Nix’s specific file system organization, this will fail without our intervention. Fortunately, the libraries usually allow overriding the directories through environment variables, either natively or thanks to a patch in nixpkgs. <linkxlink:href="#fun-wrapProgram">Wrapping</link> the executables to ensure correct paths are available to the application constitutes a significant part of packaging a modern desktop application. In this section, we will describe various modules needed by such applications, environment variables needed to make the modules load, and finally a script that will do the work for us.
</para>
<sectionxml:id="ssec-gnome-settings">
<title>Settings</title>
<para>
<linkxlink:href="https://developer.gnome.org/gio/stable/GSettings.html">GSettings</link> API is often used for storing settings. GSettings schemas are required, to know the type and other metadata of the stored values. GLib looks for <filename>glib-2.0/schemas/gschemas.compiled</filename> files inside the directories of <envar>XDG_DATA_DIRS</envar>.
</para>
<para>
On Linux, GSettings API is implemented using <linkxlink:href="https://wiki.gnome.org/Projects/dconf">dconf</link> backend. You will need to add <literal>dconf</literal> GIO module to <envar>GIO_EXTRA_MODULES</envar> variable, otherwise the <literal>memory</literal> backend will be used and the saved settings will not be persistent.
</para>
<para>
Last you will need the dconf database D-Bus service itself. You can enable it using <option>programs.dconf.enable</option>.
</para>
<para>
Some applications will also require <package>gsettings-desktop-schemas</package> for things like reading proxy configuration or user interface customization. This dependency is often not mentioned by upstream, you should grep for <literal>org.gnome.desktop</literal> and <literal>org.gnome.system</literal> to see if the schemas are needed.
</para>
</section>
<sectionxml:id="ssec-gnome-icons">
<title>Icons</title>
<para>
When an application uses icons, an icon theme should be available in <envar>XDG_DATA_DIRS</envar>. The package for the default, icon-less <linkxlink:href="https://www.freedesktop.org/wiki/Software/icon-theme/">hicolor-icon-theme</link> contains <linklinkend="ssec-gnome-hooks-hicolor-icon-theme">a setup hook</link> that will pick up icon themes from <literal>buildInputs</literal> and pass it to our wrapper. Unfortunately, relying on that would mean every user has to download the theme included in the package expression no matter their preference. For that reason, we leave the installation of icon theme on the user. If you use one of the desktop environments, you probably already have an icon theme installed.
</para>
</section>
<sectionxml:id="ssec-gnome-themes">
<title>GTK Themes</title>
<para>
Previously, a GTK theme needed to be in <envar>XDG_DATA_DIRS</envar>. This is no longer necessary for most programs since GTK incorporated Adwaita theme. Some programs (for example, those designed for <linkxlink:href="https://elementary.io/docs/human-interface-guidelines#human-interface-guidelines">elementary HIG</link>) might require a special theme like <package>pantheon.elementary-gtk-theme</package>.
</para>
</section>
<sectionxml:id="ssec-gnome-typelibs">
<title>GObject introspection typelibs</title>
<para>
<linkxlink:href="https://wiki.gnome.org/Projects/GObjectIntrospection">GObject introspection</link> allows applications to use C libraries in other languages easily. It does this through <literal>typelib</literal> files searched in <envar>GI_TYPELIB_PATH</envar>.
</para>
</section>
<sectionxml:id="ssec-gnome-plugins">
<title>Various plug-ins</title>
<para>
If your application uses <linkxlink:href="https://gstreamer.freedesktop.org/">GStreamer</link> or <linkxlink:href="https://wiki.gnome.org/Projects/Grilo">Grilo</link>, you should set <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> and <envar>GRL_PLUGIN_PATH</envar>, respectively.
Fortunately, there is <package>wrapGAppsHook</package>, that does the wrapping for us. In particular, it works in conjunction with other setup hooks that will populate the variable:
<itemizedlist>
<listitemxml:id="ssec-gnome-hooks-wrapgappshook">
<para>
<package>wrapGAppsHook</package> itself will add the package’s <filename>share</filename> directory to <envar>XDG_DATA_DIRS</envar>.
</para>
</listitem>
<listitemxml:id="ssec-gnome-hooks-glib">
<para>
<package>glib</package> setup hook will populate <envar>GSETTINGS_SCHEMAS_PATH</envar> and then <package>wrapGAppsHook</package> will prepend it to <envar>XDG_DATA_DIRS</envar>.
</para>
</listitem>
<listitemxml:id="ssec-gnome-hooks-dconf">
<para>
<package>gnome3.dconf.lib</package> is a dependency of <package>wrapGAppsHook</package>, which then also adds it to the <envar>GIO_EXTRA_MODULES</envar> variable.
<package>hicolor-icon-theme</package>’s setup hook will add icon themes to <envar>XDG_ICON_DIRS</envar> which is prepended to <envar>XDG_DATA_DIRS</envar> by <package>wrapGAppsHook</package>.
<package>gobject-introspection</package> setup hook populates <envar>GI_TYPELIB_PATH</envar> variable with <filename>lib/girepository-1.0</filename> directories of dependencies, which is then added to wrapper by <package>wrapGAppsHook</package>. It also adds <filename>share</filename> directories of dependencies to <envar>XDG_DATA_DIRS</envar>, which is intended to promote GIR files but it also <linkxlink:href="https://github.com/NixOS/nixpkgs/issues/32790">pollutes the closures</link> of packages using <package>wrapGAppsHook</package>.
</para>
<warning>
<para>
The setup hook <linkxlink:href="https://github.com/NixOS/nixpkgs/issues/56943">currently</link> does not work in expressions with <literal>strictDeps</literal> enabled, like Python packages. In those cases, you will need to disable it with <code>strictDeps = false;</code>.
Setup hooks of <package>gst_all_1.gstreamer</package> and <package>gnome3.grilo</package> will populate the <envar>GST_PLUGIN_SYSTEM_PATH_1_0</envar> and <envar>GRL_PLUGIN_PATH</envar> variables, respectively, which will then be added to the wrapper by <literal>wrapGAppsHook</literal>.
</para>
</listitem>
</itemizedlist>
</para>
<para>
You can also pass additional arguments to <literal>makeWrapper</literal> using <literal>gappsWrapperArgs</literal> in <literal>preFixup</literal> hook:
Most GNOME package offer <linklinkend="var-passthru-updateScript"><literal>updateScript</literal></link>, it is therefore possible to update to latest source tarball by running <command>nix-shell maintainers/scripts/update.nix --argstr package gnome3.nautilus</command> or even en masse with <command>nix-shell maintainers/scripts/update.nix --argstr path gnome3</command>. Read the package’s <filename>NEWS</filename> file to see what changed.
<computeroutput>GLib-GIO-ERROR **: <replaceable>06:04:50.903</replaceable>: No GSettings schemas are installed on the system</computeroutput>
</term>
<listitem>
<para>
There are no schemas avalable in <envar>XDG_DATA_DIRS</envar>. Temporarily add a random package containing schemas like <package>gsettings-desktop-schemas</package> to <literal>buildInputs</literal>. <linklinkend="ssec-gnome-hooks-glib"><package>glib</package></link> and <linklinkend="ssec-gnome-hooks-wrapgappshook"><package>wrapGAppsHook</package></link> setup hooks will take care of making the schemas available to application and you will see the actual missing schemas with the <linklinkend="ssec-gnome-common-issues-missing-schema">next error</link>. Or you can try looking through the source code for the actual schemas used.
<computeroutput>GLib-GIO-ERROR **: <replaceable>06:04:50.903</replaceable>: Settings schema ‘<replaceable>org.gnome.foo</replaceable>’ is not installed</computeroutput>
</term>
<listitem>
<para>
Package is missing some GSettings schemas. You can find out the package containing the schema with <command>nix-locate <replaceable>org.gnome.foo</replaceable>.gschema.xml</command> and let the hooks handle the wrapping as <linklinkend="ssec-gnome-common-issues-no-schemas">above</link>.
When using <package>wrapGAppsHook</package> with special derivers you can end up with double wrapped binaries.
</term>
<listitem>
<para>
This is because derivers like <function>python.pkgs.buildPythonApplication</function> or <function>qt5.mkDerivation</function> have setup-hooks automatically added that produce wrappers with <package>makeWrapper</package>. The simplest way to workaround that is to disable the <package>wrapGAppsHook</package> automatic wrapping with <code>dontWrapGApps = true;</code> and pass the arguments it intended to pass to <package>makeWrapper</package> to another.
</para>
<para>
In the case of a Python application it could look like:
<programlisting>
python3.pkgs.buildPythonApplication {
pname = "gnome-music";
version = "3.32.2";
nativeBuildInputs = [
wrapGAppsHook
gobject-introspection
...
];
dontWrapGApps = true;
# Arguments to be passed to `makeWrapper`, only used by buildPython*
makeWrapperArgs = [
"\${gappsWrapperArgs[@]}"
];
}
</programlisting>
And for a QT app like:
<programlisting>
mkDerivation {
pname = "calibre";
version = "3.47.0";
nativeBuildInputs = [
wrapGAppsHook
qmake
...
];
dontWrapGApps = true;
# Arguments to be passed to `makeWrapper`, only used by qt5’s mkDerivation
I am packaging a project that cannot be wrapped, like a library or GNOME Shell extension.
</term>
<listitem>
<para>
You can rely on applications depending on the library set the necessary environment variables but that it often easy to miss. Instead we recommend to patch the paths in the source code whenever possible. Here are some examples:
<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/7bb8f05f12ca3cff9da72b56caa2f7472d5732bc/pkgs/desktops/gnome-3/core/gnome-shell-extensions/default.nix#L21-L24">Replacing a <envar>GI_TYPELIB_PATH</envar> in GNOME Shell extension</link>– we are using <function>substituteAll</function> to include the path to a typelib into a patch.
The following examples are hardcoding GSettings schema paths. To get the schema paths we use the functions
<itemizedlist>
<listitem>
<para>
<function>glib.getSchemaPath</function> Takes a nix package attribute as an argument.
</para>
</listitem>
<listitem>
<para>
<function>glib.makeSchemaPath</function> Takes a package output like <literal>$out</literal> and a derivation name. You should use this if the schemas you need to hardcode are in the same derivation.
<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/7bb8f05f12ca3cff9da72b56caa2f7472d5732bc/pkgs/desktops/pantheon/apps/elementary-files/default.nix#L78-L86">Hard-coding GSettings schema path in Vala plug-in (dynamically loaded library)</link>– here, <function>substituteAll</function> cannot be used since the schema comes from the same package preventing us from pass its path to the function, probably due to a <linkxlink:href="https://github.com/NixOS/nix/issues/1846">Nix bug</link>.
<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/29c120c065d03b000224872251bed93932d42412/pkgs/development/libraries/glib-networking/default.nix#L31-L34">Hard-coding GSettings schema path in C library</link>– nothing special other than using <linkxlink:href="https://github.com/NixOS/nixpkgs/pull/67957#issuecomment-527717467">Coccinelle patch</link> to generate the patch itself.
The function <varname>buildGoPackage</varname> builds standard Go programs.
</para>
<para>
The function <varname> buildGoModule </varname>builds Go programs managed with Go modules. It builds a <linkxlink:href="https://github.com/golang/go/wiki/Modules">Go modules</link> through a two phase build:
<itemizedlist>
<listitem>
<para>
An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module.
</para>
</listitem>
<listitem>
<para>
A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output.
description = "Simple command-line snippet manager, written in Go";
homepage = https://github.com/knqyf263/pet;
license = licenses.mit;
maintainers = with maintainers; [ kalbasit ];
platforms = platforms.linux ++ platforms.darwin;
};
}
</programlisting>
</example>
<para>
<xreflinkend='ex-buildGoModule'/> is an example expression using buildGoModule, the following arguments are of special significance to the function:
<calloutlist>
<calloutarearefs='ex-buildGoModule-1'>
<para>
<varname>modSha256</varname> is the hash of the output of the intermediate fetcher derivation.
</para>
</callout>
<calloutarearefs='ex-buildGoModule-2'>
<para>
<varname>subPackages</varname> limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built.
</para>
</callout>
</calloutlist>
</para>
</section>
<sectionxml:id="ssec-go-legacy">
<title>Go legacy</title>
<para>
The function <varname> buildGoPackage </varname> builds legacy Go programs, not supporting Go modules.
<xreflinkend='ex-buildGoPackage'/> is an example expression using buildGoPackage, the following arguments are of special significance to the function:
<calloutlist>
<calloutarearefs='ex-buildGoPackage-1'>
<para>
<varname>goPackagePath</varname> specifies the package's canonical Go import path.
</para>
</callout>
<calloutarearefs='ex-buildGoPackage-2'>
<para>
<varname>subPackages</varname> limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built.
</para>
<para>
In this example only <literal>github.com/deis/deis/client</literal> will be built.
</para>
</callout>
<calloutarearefs='ex-buildGoPackage-3'>
<para>
<varname>goDeps</varname> is where the Go dependencies of a Go program are listed as a list of package source identified by Go import path. It could be imported as a separate <varname>deps.nix</varname>file for readability. The dependency data structure is described below.
</para>
</callout>
<callout arearefs='ex-buildGoPackage-4'>
<para>
<varname>buildFlags</varname> is a list of flags passed to the go build command.
</para>
</callout>
</calloutlist>
</para>
<para>
<xreflinkend='ex-buildGoPackage'/> is an example expression using
buildGoPackage, the following arguments are of special significance to the
function:
<calloutlist>
<calloutarearefs='ex-buildGoPackage-1'>
<para>
<varname>goPackagePath</varname> specifies the package's canonical Go
import path.
</para>
</callout>
<calloutarearefs='ex-buildGoPackage-2'>
<para>
<varname>subPackages</varname> limits the builder from building child
packages that have not been listed. If <varname>subPackages</varname> is
not specified, all child packages will be built.
</para>
<para>
In this example only <literal>github.com/deis/deis/client</literal>will
be built.
</para>
</callout>
<calloutarearefs='ex-buildGoPackage-3'>
<para>
<varname>goDeps</varname> is where the Go dependencies of a Go program are
listed as a list of package source identified by Go import path. It could
be imported as a separate <varname>deps.nix</varname> file for
readability. The dependency data structure is described below.
</para>
</callout>
<calloutarearefs='ex-buildGoPackage-4'>
<para>
<varname>buildFlags</varname> is a list of flags passed to the go build
command.
</para>
</callout>
</calloutlist>
</para>
<para>
The <varname>goDeps</varname> attribute can be imported from a separate<varname>nix</varname> file that defines which Go libraries are needed and should be included in <varname>GOPATH</varname> for <varname>buildPhase</varname>.
</para>
<para>
The <varname>goDeps</varname> attribute can be imported from a separate
<varname>nix</varname> file that defines which Go libraries are needed and
should be included in <varname>GOPATH</varname> for
<varname>buildPhase</varname>.
</para>
<examplexml:id='ex-goDeps'>
<title>deps.nix</title>
<examplexml:id='ex-goDeps'>
<title>deps.nix</title>
<programlisting>
[ <coxml:id='ex-goDeps-1'/>
{
@@ -156,51 +101,60 @@ deis = buildGoPackage rec {
}
]
</programlisting>
</example>
</example>
<para>
<calloutlist>
<calloutarearefs='ex-goDeps-1'>
<para>
<varname>goDeps</varname> is a list of Go dependencies.
</para>
</callout>
<calloutarearefs='ex-goDeps-2'>
<para>
<varname>goPackagePath</varname> specifies Go package import path.
</para>
</callout>
<calloutarearefs='ex-goDeps-3'>
<para>
<varname>fetch type</varname> that needs to be used to get package source. If <varname>git</varname> is used there should be <varname>url</varname>, <varname>rev</varname> and <varname>sha256</varname> defined next to it.
</para>
</callout>
</calloutlist>
</para>
<para>
<calloutlist>
<calloutarearefs='ex-goDeps-1'>
<para>
<varname>goDeps</varname> is a list of Go dependencies.
</para>
</callout>
<calloutarearefs='ex-goDeps-2'>
<para>
<varname>goPackagePath</varname> specifies Go package import path.
</para>
</callout>
<calloutarearefs='ex-goDeps-3'>
<para>
<varname>fetch type</varname> that needs to be used to get package source.
If <varname>git</varname> is used there should be <varname>url</varname>,
<varname>rev</varname> and <varname>sha256</varname> defined next to it.
</para>
</callout>
</calloutlist>
</para>
<para>
To extract dependency information from a Go package in automated way use<linkxlink:href="https://github.com/kamilchm/go2nix">go2nix</link>. It can produce complete derivation and <varname>goDeps</varname> file for Go programs.
</para>
<para>
To extract dependency information from a Go package in automated way use
<linkxlink:href="https://github.com/kamilchm/go2nix">go2nix</link>. It can
produce complete derivation and <varname>goDeps</varname> file for Go
programs.
</para>
<para>
<varname>buildGoPackage</varname> produces<xreflinkend='chap-multiple-output'xrefstyle="select: title"/> where <varname>bin</varname> includes program binaries. You can test build a Go binary as follows:
<para>
<varname>buildGoPackage</varname> produces
<xreflinkend='chap-multiple-output'xrefstyle="select: title"/> where
<varname>bin</varname> includes program binaries. You can test build a Go
binary as follows:
<screen>
<prompt>$ </prompt>nix-build -A deis.bin
</screen>
or build all outputs with:
$ nix-build -A deis.bin
</screen>
or build all outputs with:
<screen>
<prompt>$ </prompt>nix-build -A deis.all
</screen>
<varname>bin</varname> output will be installed by default with<varname>nix-env -i</varname> or <varname>systemPackages</varname>.
</para>
$ nix-build -A deis.all
</screen>
<varname>bin</varname> output will be installed by default with
<varname>nix-env -i</varname> or <varname>systemPackages</varname>.
</para>
<para>
You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc:
<para>
You may use Go packages installed into the active Nix profiles by adding the
This directory contains build rules for idris packages. In addition,
it contains several functions to build and compose those packages.
Everything is exposed to the user via the `idrisPackages` attribute.
The easiest way to get a working idris version is to install the `idris` attribute:
callPackage
------------
```
$ # On NixOS
$ nix-env -i nixos.idris
$ # On non-NixOS
$ nix-env -i nixpkgs.idris
```
This is like the normal nixpkgs callPackage function, specialized to
idris packages.
This however only provides the `prelude` and `base` libraries. To install idris with additional libraries, you can use the `idrisPackages.with-packages` function, e.g. in an overlay in `~/.config/nixpkgs/overlays/my-idris.nix`:
in another file (say `default.nix`) to be able to build it with
```
$ nix-build -A yaml
```
## Passing options to `idris` commands
The `build-idris-package` function provides also optional input values to set additional options for the used `idris` commands.
Specifically, you can set `idrisBuildOptions`, `idrisTestOptions`, `idrisInstallOptions` and `idrisDocOptions` to provide additional options to the `idris` command respectively when building, testing, installing and generating docs for your package.
For example you could set
```
build-idris-package {
idrisBuildOptions = [ "--log" "1" "--verbose" ]
...
}
```
to require verbose output during `idris` build phase.
Bundle idris together with a list of packages. Because idris currently
only supports a single directory in its library path, you must include
all desired libraries here, including `prelude` and `base`.
<title>Support for specific programming languages and frameworks</title>
<para>
The <linklinkend="chap-stdenv">standard build environment</link> makes it easy to build typical Autotools-based packages with very little code. Any other kind of package can be accomodated by overriding the appropriate phases of <literal>stdenv</literal>. However, there are specialised functions in Nixpkgs to easily build packages for other programming languages, such as Perl or Haskell. These are described in this chapter.
The <linklinkend="chap-stdenv">standard build environment</link> makes it
easy to build typical Autotools-based packages with very little code. Any
other kind of package can be accomodated by overriding the appropriate phases
of <literal>stdenv</literal>. However, there are specialised functions in
Nixpkgs to easily build packages for other programming languages, such as
Perl or Haskell. These are described in this chapter.
Note that <varname>jdk</varname> is an alias for the OpenJDK (self-built where available, or pre-built via Zulu). Platforms with OpenJDK not (yet) in Nixpkgs (<literal>Aarch32</literal>, <literal>Aarch64</literal>) point to the (unfree) <literal>oraclejdk</literal>.
Note that <varname>jdk</varname> is an alias for the OpenJDK (self-built
where available, or pre-built via Zulu). Platforms with OpenJDK not (yet) in
Nixpkgs (<literal>Aarch32</literal>, <literal>Aarch64</literal>) point to the
(unfree) <literal>oraclejdk</literal>.
</para>
<para>
JAR files that are intended to be used by other packages should be installed in <filename>$out/share/java</filename>. JDKs have a stdenv setup hook that add any JARs in the <filename>share/java</filename> directories of the build inputs to the <envar>CLASSPATH</envar> environment variable. For instance, if the package <literal>libfoo</literal> installs a JAR named <filename>foo.jar</filename> in its <filename>share/java</filename> directory, and another package declares the attribute
JAR files that are intended to be used by other packages should be installed
in <filename>$out/share/java</filename>. JDKs have a stdenv setup hook that
add any JARs in the <filename>share/java</filename> directories of the build
inputs to the <envar>CLASSPATH</envar> environment variable. For instance, if
the package <literal>libfoo</literal> installs a JAR named
<filename>foo.jar</filename> in its <filename>share/java</filename>
directory, and another package declares the attribute
<programlisting>
buildInputs = [ libfoo ];
nativeBuildInputs = [ jdk ];
buildInputs = [ jdk libfoo ];
</programlisting>
then <envar>CLASSPATH</envar> will be set to<filename>/nix/store/...-libfoo/share/java/foo.jar</filename>.
If your Java package provides a program, you need to generate a wrapper script to run it using the OpenJRE. You can use <literal>makeWrapper</literal> for this:
If your Java package provides a program, you need to generate a wrapper
Note the use of <literal>jre</literal>, which is the part of the OpenJDK package that contains the Java Runtime Environment. By using <literal>${jre}/bin/java</literal> instead of <literal>${jdk}/bin/java</literal>, you prevent your package from depending on the JDK at runtime.
Note the use of <literal>jre</literal>, which is the part of the OpenJDK
package that contains the Java Runtime Environment. By using
<literal>${jre}/bin/java</literal> instead of
<literal>${jdk}/bin/java</literal>, you prevent your package from depending
on the JDK at runtime.
</para>
<para>
Note all JDKs passthru <literal>home</literal>, so if your application requires environment variables like <envar>JAVA_HOME</envar> being set, that can be done in a generic fashion with the <literal>--set</literal> argument of <literal>makeWrapper</literal>:
Note all JDKs passthru <literal>home</literal>, so if your application
requires environment variables like <envar>JAVA_HOME</envar> being set, that
can be done in a generic fashion with the <literal>--set</literal> argument
of <literal>makeWrapper</literal>:
<programlisting>
--set JAVA_HOME ${jdk.home}
--set JAVA_HOME ${jdk.home}
</programlisting>
</para>
<para>
It is possible to use a different Java compiler than <command>javac</command> from the OpenJDK. For instance, to use the GNU Java Compiler:
It is possible to use a different Java compiler than <command>javac</command>
from the OpenJDK. For instance, to use the GNU Java Compiler:
<programlisting>
nativeBuildInputs = [ gcj ant ];
buildInputs = [ gcj ant ];
</programlisting>
Here, Ant will automatically use <command>gij</command> (the GNU Java Runtime) instead of the OpenJRE.
Here, Ant will automatically use <command>gij</command> (the GNU Java
Lua packages are built by the <varname>buildLuaPackage</varname> function. This function is implemented in <linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/lua-modules/generic/default.nix"><filename>pkgs/development/lua-modules/generic/default.nix</filename></link> and works similarly to <varname>buildPerlPackage</varname>. (See <xreflinkend="sec-language-perl"/> for details.)
Lua packages are built by the <varname>buildLuaPackage</varname> function.
and works similarly to <varname>buildPerlPackage</varname>. (See
<xreflinkend="sec-language-perl"/> for details.)
</para>
<para>
Lua packages are defined in<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/lua-packages.nix"><filename>pkgs/top-level/lua-packages.nix</filename></link>. Most of them are simple. For example:
Lua packages accept additional parameter <varname>disabled</varname>, which defines the condition of disabling package from luaPackages. For example, if package has <varname>disabled</varname> assigned to <literal>lua.luaversion != "5.1"</literal>, it will not be included in any luaPackages except lua51Packages, making it only be built for lua 5.1.
Lua packages accept additional parameter <varname>disabled</varname>, which
defines the condition of disabling package from luaPackages. For example, if
package has <varname>disabled</varname> assigned to <literal>lua.luaversion
!= "5.1"</literal>, it will not be included in any luaPackages except
lua51Packages, making it only be built for lua 5.1.
OCaml libraries should be installed in <literal>$(out)/lib/ocaml/${ocaml.version}/site-lib/</literal>. Such directories are automatically added to the <literal>$OCAMLPATH</literal> environment variable when building another package that depends on them or when opening a <literal>nix-shell</literal>.
</para>
<para>
Given that most of the OCaml ecosystem is now built with dune, nixpkgs includes a convenience build support function called <literal>buildDunePackage</literal> that will build an OCaml package using dune, OCaml and findlib and any additional dependencies provided as <literal>buildInputs</literal> or <literal>propagatedBuildInputs</literal>.
</para>
<para>
Here is a simple package example. It defines an (optional) attribute <literal>minimumOCamlVersion</literal> that will be used to throw a descriptive evaluation error if building with an older OCaml is attempted. It uses the <literal>fetchFromGitHub</literal> fetcher to get its source. It sets the <literal>doCheck</literal> (optional) attribute to <literal>true</literal> which means that tests will be run with <literal>dune runtest -p angstrom</literal> after the build (<literal>dune build -p angstrom</literal>) is complete. It uses <literal>alcotest</literal> as a build input (because it is needed to run the tests) and <literal>bigstringaf</literal> and <literal>result</literal> as propagated build inputs (thus they will also be available to libraries depending on this library). The library will be installed using the <literal>angstrom.install</literal> file that dune generates.
description = "OCaml parser combinators built for speed and memory efficiency";
license = stdenv.lib.licenses.bsd3;
maintainers = with stdenv.lib.maintainers; [ sternenseemann ];
};
}
</programlisting>
<para>
Here is a second example, this time using a source archive generated with <literal>dune-release</literal>. It is a good idea to use this archive when it is available as it will usually contain substituted variables such as a <literal>%%VERSION%%</literal> field. This library does not depend on any other OCaml library and no tests are run after building it.
Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic package builder function for any Perl package that has a standard <varname>Makefile.PL</varname>. It’s implemented in <link
Nixpkgs provides a function <varname>buildPerlPackage</varname>, a generic
package builder function for any Perl package that has a standard
<varname>Makefile.PL</varname>. It’s implemented in
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link>, rather than <filename>pkgs/all-packages.nix</filename>. Most Perl packages are so straight-forward to build that they are defined here directly, rather than having a separate function for each package called from <filename>perl-packages.nix</filename>. However, more complicated packages should be put in a separate file, typically in <filename>pkgs/development/perl-modules</filename>. Here is an example of the former:
Note the use of <literal>mirror://cpan/</literal>, and the<literal>${name}</literal> in the URL definition to ensure that the name attribute is consistent with the source that we’re actually downloading. Perl packages are made available in <filename>all-packages.nix</filename> through the variable <varname>perlPackages</varname>. For instance, if you have a package that needs <varname>ClassC3</varname>, you would typically write
Note the use of <literal>mirror://cpan/</literal>, and the
<literal>${name}</literal> in the URL definition to ensure that the name
attribute is consistent with the source that we’re actually downloading.
Perl packages are made available in <filename>all-packages.nix</filename>
through the variable <varname>perlPackages</varname>. For instance, if you
have a package that needs <varname>ClassC3</varname>, you would typically
write
<programlisting>
foo = import ../path/to/foo.nix {
inherit stdenv fetchurl ...;
inherit (perlPackages) ClassC3;
};
</programlisting>
in <filename>all-packages.nix</filename>. You can test building a Perl package as follows:
in <filename>all-packages.nix</filename>. You can test building a Perl
package as follows:
<screen>
<prompt>$ </prompt>nix-build -A perlPackages.ClassC3
$ nix-build -A perlPackages.ClassC3
</screen>
<varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the start of the name attribute, so the package above is actually called <literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
<varname>buildPerlPackage</varname> adds <literal>perl-</literal> to the
start of the name attribute, so the package above is actually called
<literal>perl-Class-C3-0.21</literal>. So to install it, you can say:
<screen>
<prompt>$ </prompt>nix-env -i perl-Class-C3
$ nix-env -i perl-Class-C3
</screen>
(Of course you can also install using the attribute name: <literal>nix-env -i -A perlPackages.ClassC3</literal>.)
(Of course you can also install using the attribute name: <literal>nix-env -i
In the configure phase, it calls <literal>perl Makefile.PL</literal> to generate a Makefile. You can set the variable <varname>makeMakerFlags</varname> to pass flags to <filename>Makefile.PL</filename>
In the configure phase, it calls <literal>perl Makefile.PL</literal> to
generate a Makefile. You can set the variable
<varname>makeMakerFlags</varname> to pass flags to
<filename>Makefile.PL</filename>
</para>
</listitem>
<listitem>
<para>
It adds the contents of the <envar>PERL5LIB</envar> environment variable to <literal>#! .../bin/perl</literal> line of Perl scripts as <literal>-I<replaceable>dir</replaceable></literal> flags. This ensures that a script can find its dependencies. (This can cause this shebang line to become too long for Darwin to handle; see the note below.)
It adds the contents of the <envar>PERL5LIB</envar> environment variable
to <literal>#! .../bin/perl</literal> line of Perl scripts as
<literal>-I<replaceable>dir</replaceable></literal> flags. This ensures
that a script can find its dependencies.
</para>
</listitem>
<listitem>
<para>
In the fixup phase, it writes the propagated build inputs (<varname>propagatedBuildInputs</varname>) to the file <filename>$out/nix-support/propagated-user-env-packages</filename>. <command>nix-env</command> recursively installs all packages listed in this file when you install a package that has it. This ensures that a Perl package can find its dependencies.
In the fixup phase, it writes the propagated build inputs
(<varname>propagatedBuildInputs</varname>) to the file
<command>nix-env</command> recursively installs all packages listed in
this file when you install a package that has it. This ensures that a Perl
package can find its dependencies.
</para>
</listitem>
</orderedlist>
</para>
<para>
<varname>buildPerlPackage</varname> is built on top of<varname>stdenv</varname>, so everything can be customised in the usual way. For instance, the <literal>BerkeleyDB</literal> module has a <varname>preConfigure</varname> hook to generate a configuration file used by <filename>Makefile.PL</filename>:
<varname>buildPerlPackage</varname> is built on top of
<varname>stdenv</varname>, so everything can be customised in the usual way.
For instance, the <literal>BerkeleyDB</literal> module has a
<varname>preConfigure</varname> hook to generate a configuration file used by
<filename>Makefile.PL</filename>:
<programlisting>
{ buildPerlPackage, fetchurl, db }:
@@ -81,7 +117,12 @@ buildPerlPackage rec {
</para>
<para>
Dependencies on other Perl packages can be specified in the<varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname> attributes. If something is exclusively a build-time dependency, use <varname>buildInputs</varname>; if it’s (also) a runtime dependency, use <varname>propagatedBuildInputs</varname>. For instance, this builds a Perl module that has runtime dependencies on a bunch of other modules:
Dependencies on other Perl packages can be specified in the
<varname>buildInputs</varname> and <varname>propagatedBuildInputs</varname>
attributes. If something is exclusively a build-time dependency, use
<varname>buildInputs</varname>; if it’s (also) a runtime dependency, use
<varname>propagatedBuildInputs</varname>. For instance, this builds a Perl
module that has runtime dependencies on a bunch of other modules:
On Darwin, if a script has too many <literal>-I<replaceable>dir</replaceable></literal> flags in its first line (its “shebang line”), it will not run. This can be worked around by calling the <literal>shortenPerlShebang</literal> function from the <literal>postInstall</literal> phase:
This will remove the <literal>-I</literal> flags from the shebang line, rewrite them in the <literal>use lib</literal> form, and put them on the next line instead. This function can be given any number of Perl scripts as arguments; it will modify them in-place.
</para>
<sectionxml:id="ssec-generation-from-CPAN">
<title>Generation from CPAN</title>
<para>
Nix expressions for Perl packages can be generated (almost) automatically from CPAN. This is done by the program <command>nix-generate-from-cpan</command>, which can be installed as follows:
Nix expressions for Perl packages can be generated (almost) automatically
from CPAN. This is done by the program
<command>nix-generate-from-cpan</command>, which can be installed as
This program takes a Perl module name, looks it up on CPAN, fetches and unpacks the corresponding package, and prints a Nix expression on standard output. For example:
This program takes a Perl module name, looks it up on CPAN, fetches and
unpacks the corresponding package, and prints a Nix expression on standard
Nixpkgs has experimental support for cross-compiling Perl modules. In many cases, it will just work out of the box, even for modules with native extensions. Sometimes, however, the Makefile.PL for a module may (indirectly) import a native module. In that case, you will need to make a stub for that module that will satisfy the Makefile.PL and install it into <filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the <varname>postInstall</varname> for <varname>DBI</varname> for an example.
Nixpkgs has experimental support for cross-compiling Perl modules. In many
cases, it will just work out of the box, even for modules with native
extensions. Sometimes, however, the Makefile.PL for a module may
(indirectly) import a native module. In that case, you will need to make a
stub for that module that will satisfy the Makefile.PL and install it into
<filename>lib/perl5/site_perl/cross_perl/${perl.version}</filename>. See the
<varname>postInstall</varname> for <varname>DBI</varname> for an example.
@@ -474,18 +478,18 @@ don't explicitly define which `python` derivation should be used. In the above
example we use `buildPythonPackage` that is part of the set `python35Packages`,
and in this case the `python35` interpreter is automatically used.
## Reference
### Interpreters
Versions 2.7, 3.5, 3.6 and 3.7 of the CPython interpreter are available as
respectively `python27`, `python35`, `python36` and `python37`. The aliases
`python2` and `python3` correspond to respectively `python27` and
`python37`. The default interpreter, `python`, maps to `python2`. The PyPy
interpreters compatible with Python 2.7 and 3 are available as `pypy27` and
`pypy3`, with aliases `pypy2` mapping to `pypy27` and `pypy` mapping to
`pypy2`. The Nix expressions for the interpreters can be
found in `pkgs/development/interpreters/python`.
Versions 2.7, 3.4, 3.5, 3.6 and 3.7 of the CPython interpreter are available as
respectively `python27`, `python34`, `python35` and `python36`. The PyPy interpreter
is available as `pypy`. The aliases `python2` and `python3` correspond to respectively `python27` and
`python36`. The default interpreter, `python`, maps to `python2`.
The Nix expressions for the interpreters can be found in
`pkgs/development/interpreters/python`.
All packages depending on any Python interpreter get appended
`out/{python.sitePackages}` to `$PYTHONPATH` if such directory
@@ -504,13 +508,13 @@ Each interpreter has the following attributes:
-`buildEnv`. Function to build python interpreter environments with extra packages bundled together. See section *python.buildEnv function* for usage and documentation.
-`withPackages`. Simpler interface to `buildEnv`. See section *python.withPackages function* for usage and documentation.
-`sitePackages`. Alias for `lib/${libPrefix}/site-packages`.
-`executable`. Name of the interpreter executable, e.g. `python3.7`.
-`executable`. Name of the interpreter executable, e.g. `python3.4`.
-`pkgs`. Set of Python packages for that specific interpreter. The package set can be modified by overriding the interpreter and passing `packageOverrides`.
### Building packages and applications
Python libraries and applications that use `setuptools` or
`distutils` are typically built with respectively the `buildPythonPackage` and
`distutils` are typically build with respectively the `buildPythonPackage` and
`buildPythonApplication` functions. These two functions also support installing a `wheel`.
All Python packages reside in `pkgs/top-level/python-packages.nix` and all
@@ -526,6 +530,7 @@ attribute set is created for each available Python interpreter. The available
sets are
*`pkgs.python27Packages`
*`pkgs.python34Packages`
*`pkgs.python35Packages`
*`pkgs.python36Packages`
*`pkgs.python37Packages`
@@ -534,42 +539,41 @@ sets are
and the aliases
*`pkgs.python2Packages` pointing to `pkgs.python27Packages`
*`pkgs.python3Packages` pointing to `pkgs.python37Packages`
*`pkgs.python3Packages` pointing to `pkgs.python36Packages`
*`pkgs.pythonPackages` pointing to `pkgs.python2Packages`
#### `buildPythonPackage` function
The `buildPythonPackage` function is implemented in
@@ -582,6 +586,11 @@ The `buildPythonPackage` mainly does four things:
environment variable and add dependent libraries to script's `sys.path`.
* In the `installCheck` phase, `${python.interpreter} setup.py test` is ran.
As in Perl, dependencies on other Python packages can be specified in the
`buildInputs` and `propagatedBuildInputs` attributes. If something is
exclusively a build-time dependency, use `buildInputs`; if it is (also) a runtime
dependency, use `propagatedBuildInputs`.
By default tests are run because `doCheck = true`. Test dependencies, like
e.g. the test runner, should be added to `checkInputs`.
@@ -593,28 +602,19 @@ as the interpreter unless overridden otherwise.
All parameters from `stdenv.mkDerivation` function are still supported. The following are specific to `buildPythonPackage`:
* `catchConflicts ? true`: If `true`, abort package build if a package name appears more than once in dependency tree. Default is `true`.
* `checkInputs ? []`: Dependencies needed for running the `checkPhase`. These are added to `buildInputs` when `doCheck = true`.
* `disabled` ? false: If `true`, package is not build for the particular Python interpreter version.
* `dontWrapPythonPrograms ? false`: Skip wrapping of python programs.
* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped programs.
* `installFlags ? []`: A list of strings. Arguments to be passed to `pip install`. To pass options to `python setup.py install`, use `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"]`.
* `format ? "setuptools"`: Format of the source. Valid options are `"setuptools"`, `"pyproject"`, `"flit"`, `"wheel"`, and `"other"`. `"setuptools"` is for when the source has a `setup.py` and `setuptools` is used to build a wheel, `flit`, in case `flit` should be used to build a wheel, and `wheel` in case a wheel is provided. Use `other` when a custom `buildPhase` and/or `installPhase` is needed.
* `installFlags ? []`: A list of strings. Arguments to be passed to `pip install`. To pass options to `python setup.py install`, use `--install-option`. E.g., `installFlags=["--install-option='--cpp_implementation'"].
*`format ? "setuptools"`: Format of the source. Valid options are `"setuptools"`, `"flit"`, `"wheel"`, and `"other"`. `"setuptools"` is for when the source has a `setup.py` and `setuptools` is used to build a wheel, `flit`, in case `flit` should be used to build a wheel, and `wheel` in case a wheel is provided. Use `other` when a custom `buildPhase` and/or `installPhase` is needed.
*`makeWrapperArgs ? []`: A list of strings. Arguments to be passed to `makeWrapper`, which wraps generated binaries. By default, the arguments to `makeWrapper` set `PATH` and `PYTHONPATH` environment variables before calling the binary. Additional arguments here can allow a developer to set environment variables which will be available when the binary is run. For example, `makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
*`namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this defaults to `"python3.5-"` for Python 3.5, etc., and in case of applications to `""`.
*`pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages in `pythonPath` are not propagated (contrary to `propagatedBuildInputs`).
*`preShellHook`: Hook to execute commands before `shellHook`.
*`postShellHook`: Hook to execute commands after `shellHook`.
*`removeBinByteCode ? true`: Remove bytecode from `/bin`. Bytecode is only created when the filenames end with `.py`.
* `setupPyGlobalFlags ? []`: List of flags passed to `setup.py` command.
*`setupPyBuildFlags ? []`: List of flags passed to `setup.py build_ext` command.
The `stdenv.mkDerivation` function accepts various parameters for describing build inputs (see "Specifying dependencies"). The following are of special
interest for Python packages, either because these are primarily used, or because their behaviour is different:
* `nativeBuildInputs ? []`: Build-time only dependencies. Typically executables as well as the items listed in `setup_requires`.
* `buildInputs ? []`: Build and/or run-time dependencies that need to be be compiled for the host machine. Typically non-Python libraries which are being linked.
* `checkInputs ? []`: Dependencies needed for running the `checkPhase`. These are added to `nativeBuildInputs` when `doCheck = true`. Items listed in `tests_require` go here.
* `propagatedBuildInputs ? []`: Aside from propagating dependencies, `buildPythonPackage` also injects code into and wraps executables with the paths included in this list. Items listed in `install_requires` go here.
##### Overriding Python packages
The `buildPythonPackage` function has a `overridePythonAttrs` method that
propagatedBuildInputs = with python3Packages; [ tornado_4 python-daemon ];
meta = with lib; {
...
};
}
```
This is then added to `all-packages.nix` just as any other application would be.
```nix
luigi = callPackage ../applications/networking/cluster/luigi { };
```
Since the package is an application, a consumer doesn't need to care about
python versions or modules, which is why they don't go in `pythonPackages`.
#### `toPythonApplication` function
A distinction is made between applications and libraries, however, sometimes a
@@ -727,7 +694,7 @@ Saving the following as `default.nix`
withimport<nixpkgs>{};
python.buildEnv.override{
extraLibs = [ pythonPackages.pyramid ];
extraLibs =[pkgs.pythonPackages.pyramid];
ignoreCollisions=true;
}
```
@@ -759,7 +726,6 @@ specified packages in its path.
*`extraLibs`: List of packages installed inside the environment.
*`postBuild`: Shell command executed after the build of environment.
*`ignoreCollisions`: Ignore file collisions inside the environment (default is `false`).
* `permitUserSite`: Skip setting the `PYTHONNOUSERSITE` environment variable in wrapped binaries in the environment.
#### `python.withPackages` function
@@ -798,22 +764,6 @@ such as `ignoreCollisions = true` or `postBuild`. If you need them, you have to
Python 2 namespace packages may provide `__init__.py` that collide. In that case `python.buildEnv`
should be used with `ignoreCollisions = true`.
#### Setup hooks
The following are setup hooks specifically for Python packages. Most of these are
used in `buildPythonPackage`.
- `flitBuildHook` to build a wheel using `flit`.
- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system (e.g. `setuptools` or `flit`) should still be added as `nativeBuildInput`.
- `pipInstallHook` to install wheels.
- `pytestCheckHook` to run tests with `pytest`.
- `pythonCatchConflictsHook` to check whether a Python package is not already existing.
- `pythonImportsCheckHook` to check whether importing the listed modules works.
- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
- `setuptoolsBuildHook` to build a wheel using `setuptools`.
- `setuptoolsCheckHook` to run tests with `python setup.py test`.
- `wheelUnpackHook` to move a wheel to the correct folder so it can be installed with the `pipInstallHook`.
### Development mode
Development or editable mode is supported. To develop Python packages
@@ -826,12 +776,11 @@ Given a `default.nix`:
```nix
withimport<nixpkgs>{};
pythonPackages.buildPythonPackage {
name = "myproject";
buildInputs = with pythonPackages; [ pyramid ];
buildPythonPackage{name="myproject";
src = ./.;
}
buildInputs=withpkgs.pythonPackages;[pyramid];
src=./.;}
```
Running `nix-shell` with no arguments should give you
@@ -856,7 +805,7 @@ community to help save time. No tool is preferred at the moment.
### Deterministic builds
The Python interpreters are now built deterministically.
Python 2.7, 3.5 and 3.6 are now built deterministically and 3.4 mostly.
Minor modifications had to be made to the interpreters in order to generate
deterministic bytecode. This has security implications and is relevant for
those using Python in a `nix-shell`.
@@ -892,6 +841,7 @@ example of such a situation is when `py.test` is used.
'';
}
```
- Unicode issues can typically be fixed by including `glibcLocales` in `buildInputs` and exporting `LC_ALL=en_US.utf-8`.
- Tests that attempt to access `$HOME` can be fixed by using the following work-around before running tests (e.g. `preCheck`): `export HOME=$(mktemp -d)`
## FAQ
@@ -1017,13 +967,10 @@ Create this `default.nix` file, together with a `requirements.txt` and simply ex
```nix
with import <nixpkgs> {};
with python27Packages;
with pkgs.python27Packages;
stdenv.mkDerivation {
name = "impurePythonEnv";
src = null;
buildInputs = [
# these packages are required for virtualenv and pip to work:
#
@@ -1043,15 +990,14 @@ stdenv.mkDerivation {
libxslt
libzip
stdenv
zlib
];
zlib ];
src = null;
shellHook = ''
# set SOURCE_DATE_EPOCH so that we can use python wheels
SOURCE_DATE_EPOCH=$(date +%s)
virtualenv --no-setuptools venv
export PATH=$PWD/venv/bin:$PATH
pip install -r requirements.txt
# set SOURCE_DATE_EPOCH so that we can use python wheels
SOURCE_DATE_EPOCH=$(date +%s)
virtualenv --no-setuptools venv
export PATH=$PWD/venv/bin:$PATH
pip install -r requirements.txt
'';
}
```
@@ -1101,7 +1047,8 @@ To modify only a Python package set instead of a whole Python derivation, use th
This section describes the differences between Nix expressions for Qt libraries and applications and Nix expressions for other C++ software. Some knowledge of the latter is assumed. There are primarily two problems which the Qt infrastructure is designed to address: ensuring consistent versioning of all dependencies and finding dependencies at runtime.
Qt is a comprehensive desktop and mobile application development toolkit for
C++. Legacy support is available for Qt 3 and Qt 4, but all current
development uses Qt 5. The Qt 5 packages in Nixpkgs are updated frequently to
take advantage of new features, but older versions are typically retained
until their support window ends. The most important consideration in
packaging Qt-based software is ensuring that each package and all its
dependencies use the same version of Qt 5; this consideration motivates most
of the tools described below.
</para>
<examplexml:id='qt-default-nix'>
<title>Nix expression for a Qt package (<filename>default.nix</filename>)</title>
Import <literal>mkDerivation</literal> and Qt (such as <literal>qtbase</literal> modules directly. <emphasis>Do not</emphasis> import Qt package sets; the Qt versions of dependencies may not be coherent, causing build and runtime failures.
</para>
</callout>
<calloutarearefs='qt-default-nix-co-2'>
<para>
Use <literal>mkDerivation</literal> instead of <literal>stdenv.mkDerivation</literal>. <literal>mkDerivation</literal> is a wrapper around <literal>stdenv.mkDerivation</literal> which applies some Qt-specific settings. This deriver accepts the same arguments as <literal>stdenv.mkDerivation</literal>; refer to <xreflinkend='chap-stdenv'/> for details.
</para>
<para>
To use another deriver instead of <literal>stdenv.mkDerivation</literal>, use <literal>mkDerivationWith</literal>:
<programlisting>
mkDerivationWith myDeriver {
# ...
}
</programlisting>
If you cannot use <literal>mkDerivationWith</literal>, please refer to <xreflinkend='qt-runtime-dependencies'/>.
</para>
</callout>
<calloutarearefs='qt-default-nix-co-3'>
<para>
<literal>mkDerivation</literal> accepts the same arguments as <literal>stdenv.mkDerivation</literal>, such as <literal>buildInputs</literal>.
</para>
</callout>
</calloutlist>
<formalparaxml:id='qt-runtime-dependencies'>
<title>Locating runtime dependencies</title>
<para>
Qt applications need to be wrapped to find runtime dependencies. If you cannot use <literal>mkDerivation</literal> or <literal>mkDerivationWith</literal> above, include <literal>wrapQtAppsHook</literal> in <literal>nativeBuildInputs</literal>:
<programlisting>
stdenv.mkDerivation {
# ...
nativeBuildInputs = [ wrapQtAppsHook ];
}
</programlisting>
Whenever possible, libraries that use Qt 5 should be built with each
available version. Packages providing libraries should be added to the
top-level function <varname>mkLibsForQt5</varname>, which is used to build a
set of libraries for every Qt 5 version. A special
<varname>callPackage</varname> function is used in this scope to ensure that
the entire dependency tree uses the same Qt 5 version. Import dependencies
unqualified, i.e., <literal>qtbase</literal> not
<literal>qt5.qtbase</literal>. <emphasis>Do not</emphasis> import a package
set such as <literal>qt5</literal> or <literal>libsForQt5</literal>.
</para>
</formalpara>
<para>
Entries added to <literal>qtWrapperArgs</literal> are used to modify the wrappers created by <literal>wrapQtAppsHook</literal>. The entries are passed as arguments to <xreflinkend='fun-wrapProgram'/>.
Set <literal>dontWrapQtApps</literal> to stop applications from being wrapped automatically. It is required to wrap applications manually with <literal>wrapQtApp</literal>, using the syntax of <xreflinkend='fun-wrapProgram'/>:
<literal>wrapQtAppsHook</literal> ignores files that are non-ELF executables. This means that scripts won't be automatically wrapped so you'll need to manually wrap them as previously mentioned. An example of when you'd always need to do this is with Python applications that use PyQT.
If a library does not support a particular version of Qt 5, it is best to
mark it as broken by setting its <literal>meta.broken</literal> attribute. A
package may be marked broken for certain versions by testing the
<literal>qtbase.version</literal> attribute, which will always give the
current Qt 5 version.
</para>
</note>
</section>
<para>
Libraries are built with every available version of Qt. Use the <literal>meta.broken</literal> attribute to disable the package for unsupported Qt versions:
Add a Qt library to <filename>all-packages.nix</filename> by adding it to the collection inside <literal>mkLibsForQt5</literal>. This ensures that the library is built with every available version of Qt as needed.
<examplexml:id='qt-library-all-packages-nix'>
<title>Adding a Qt library to <filename>all-packages.nix</filename></title>
<programlisting>
{
# ...
mkLibsForQt5 = self: with self; {
# ...
mylib = callPackage ../path/to/mylib {};
};
# ...
}
</programlisting>
</example>
Call your application expression using
<literal>libsForQt5.callPackage</literal> instead of
<literal>qtbase</literal> not <literal>qt5.qtbase</literal>. <emphasis>Do
not</emphasis> import a package set such as <literal>qt5</literal> or
<literal>libsForQt5</literal>.
</para>
</formalpara>
<formalpara>
<title>Adding an application to Nixpkgs</title>
<para>
Add a Qt application to <filename>all-packages.nix</filename> using <literal>libsForQt5.callPackage</literal> instead of the usual <literal>callPackage</literal>. The former ensures that all dependencies are built with the same version of Qt.
<examplexml:id='qt-application-all-packages-nix'>
<title>Adding a Qt application to <filename>all-packages.nix</filename></title>
There currently is support to bundle applications that are packaged as Ruby gems. The utility "bundix" allows you to write a <filename>Gemfile</filename>, let bundler create a <filename>Gemfile.lock</filename>, and then convert this into a nix expression that contains all Gem dependencies automatically.
There currently is support to bundle applications that are packaged as Ruby
gems. The utility "bundix" allows you to write a
<filename>Gemfile</filename>, let bundler create a
<filename>Gemfile.lock</filename>, and then convert this into a nix
expression that contains all Gem dependencies automatically.
</para>
<para>
@@ -41,22 +45,16 @@ bundlerEnv rec {
</screen>
<para>
Please check in the <filename>Gemfile</filename>,<filename>Gemfile.lock</filename> and the <filename>gemset.nix</filename> so future updates can be run easily.
Please check in the <filename>Gemfile</filename>,
<filename>Gemfile.lock</filename> and the <filename>gemset.nix</filename> so
future updates can be run easily.
</para>
<para>
Updating Ruby packages can then be done like this:
For tools written in Ruby - i.e. where the desire is to install a package and then execute e.g. <command>rake</command> at the command line, there is an alternative builder called <literal>bundlerApp</literal>. Set up the <filename>gemset.nix</filename> the same way, and then, for example:
For tools written in Ruby - i.e. where the desire is to install a package and
then execute e.g. <command>rake</command> at the command line, there is an
alternative builder called <literal>bundlerApp</literal>. Set up the
<filename>gemset.nix</filename> the same way, and then, for example:
</para>
<screen>
@@ -78,11 +76,29 @@ bundlerApp {
</screen>
<para>
The chief advantage of <literal>bundlerApp</literal> over<literal>bundlerEnv</literal> is the executables introduced in the environment are precisely those selected in the <literal>exes</literal> list, as opposed to <literal>bundlerEnv</literal> which adds all the executables made available by gems in the gemset, which can mean e.g. <command>rspec</command> or <command>rake</command> in unpredictable versions available from various packages.
The chief advantage of <literal>bundlerApp</literal> over
<literal>bundlerEnv</literal> is the executables introduced in the
environment are precisely those selected in the <literal>exes</literal> list,
as opposed to <literal>bundlerEnv</literal> which adds all the executables
made available by gems in the gemset, which can mean e.g.
<command>rspec</command> or <command>rake</command> in unpredictable versions
available from various packages.
</para>
<para>
Resulting derivations for both builders also have two helpful attributes,<literal>env</literal> and <literal>wrappedRuby</literal>. The first one allows one to quickly drop into <command>nix-shell</command> with the specified environment present. E.g. <command>nix-shell -A sensu.env</command> would give you an environment with Ruby preset so it has all the libraries necessary for <literal>sensu</literal> in its paths. The second one can be used to make derivations from custom Ruby scripts which have <filename>Gemfile</filename>s with their dependencies specified. It is a derivation with <command>ruby</command> wrapped so it can find all the needed dependencies. For example, to make a derivation <literal>my-script</literal> for a <filename>my-script.rb</filename> (which should be placed in <filename>bin</filename>) you should run <command>bundix</command> as specified above and then use <literal>bundlerEnv</literal> like this:
Resulting derivations for both builders also have two helpful attributes,
<literal>env</literal> and <literal>wrappedRuby</literal>. The first one
allows one to quickly drop into <command>nix-shell</command> with the
specified environment present. E.g. <command>nix-shell -A sensu.env</command>
would give you an environment with Ruby preset so it has all the libraries
necessary for <literal>sensu</literal> in its paths. The second one can be
used to make derivations from custom Ruby scripts which have
<filename>Gemfile</filename>s with their dependencies specified. It is a
derivation with <command>ruby</command> wrapped so it can find all the needed
dependencies. For example, to make a derivation <literal>my-script</literal>
for a <filename>my-script.rb</filename> (which should be placed in
<filename>bin</filename>) you should run <command>bundix</command> as
specified above and then use <literal>bundlerEnv</literal> like this:
There are all the schemes, collections and a few thousand packages, as defined upstream (perhaps with tiny differences).
</programlisting>
There are all the schemes, collections and a few thousand packages, as
defined upstream (perhaps with tiny differences).
</para>
</listitem>
<listitem>
<para>
By default you only get executables and files needed during runtime, and a little documentation for the core packages. To change that, you need to add <varname>pkgFilter</varname> function to <varname>combine</varname>.
By default you only get executables and files needed during runtime, and a
little documentation for the core packages. To change that, you need to
add <varname>pkgFilter</varname> function to <varname>combine</varname>.
Note that the wrapper assumes that the result has a chance to be useful. For example, the core executables should be present, as well as some core data files. The supported way of ensuring this is by including some scheme, for example <varname>scheme-basic</varname>, into the combination.
Note that the wrapper assumes that the result has a chance to be useful.
For example, the core executables should be present, as well as some core
data files. The supported way of ensuring this is by including some
scheme, for example <varname>scheme-basic</varname>, into the combination.
Nix expressions for Vim plugins are stored in [pkgs/misc/vim-plugins](/pkgs/misc/vim-plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`./update.py`](/pkgs/misc/vim-plugins/update.py). This creates a [generated.nix](/pkgs/misc/vim-plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names). Plugins are listed in alphabetical order in `vim-plugin-names` using the format `[github username]/[repository]`. For example https://github.com/scrooloose/nerdtree becomes `scrooloose/nerdtree`.
Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
dependencies = with super; [ deoplete-nvim vim-fish ];
});
```
Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `./update.py` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`.
To add a new plugin:
1. run `./update.py` and create a commit named "vimPlugins: Update",
2. add the new plugin to [vim-plugin-names](/pkgs/misc/vim-plugins/vim-plugin-names) and add overrides if required to [overrides.nix](/pkgs/misc/vim-plugins/overrides.nix),
3. run `./update.py` again and create a commit named "vimPlugins.[name]: init at [version]" (where `name` and `version` can be found in [generated.nix](/pkgs/misc/vim-plugins/generated.nix)), and
4. create a pull request.
## Important repositories
- [vim-pi](https://bitbucket.org/vimcommunity/vim-pi) is a plugin repository
from VAM plugin manager meant to be used by others as well used by
- [vim2nix](https://github.com/MarcWeber/vim-addon-vim2nix) which generates the
- [vim2nix](http://github.com/MarcWeber/vim-addon-vim2nix) which generates the
Nix packages can declare <emphasis>meta-attributes</emphasis> that contain information about a package such as a description, its homepage, its license, and so on. For instance, the GNU Hello package has a <varname>meta</varname> declaration like this:
Nix packages can declare <emphasis>meta-attributes</emphasis> that contain
information about a package such as a description, its homepage, its license,
and so on. For instance, the GNU Hello package has a <varname>meta</varname>
declaration like this:
<programlisting>
meta = with stdenv.lib; {
description = "A program that produces a familiar, friendly greeting";
@@ -11,7 +14,7 @@ meta = with stdenv.lib; {
GNU Hello is a program that prints "Hello, world!" when you run it.
Meta-attributes are not passed to the builder of the package. Thus, a change to a meta-attribute doesn’t trigger a recompilation of the package. The value of a meta-attribute must be a string.
Meta-attributes are not passed to the builder of the package. Thus, a change
to a meta-attribute doesn’t trigger a recompilation of the package. The
value of a meta-attribute must be a string.
</para>
<para>
The meta-attributes of a package can be queried from the command-line using<command>nix-env</command>:
The meta-attributes of a package can be queried from the command-line using
<command>nix-env</command>:
<screen>
<prompt>$ </prompt>nix-env -qa hello --json
$ nix-env -qa hello --json
{
"hello": {
"meta": {
"description": "A program that produces a familiar, friendly greeting",
hello-2.3 A program that produces a familiar, friendly greeting
</screen>
</para>
@@ -81,13 +88,18 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
A short (one-line) description of the package. This is shown by<command>nix-env -q --description</command> and also on the Nixpkgs release pages.
A short (one-line) description of the package. This is shown by
<command>nix-env -q --description</command> and also on the Nixpkgs
release pages.
</para>
<para>
Don’t include a period at the end. Don’t include newline characters. Capitalise the first character. For brevity, don’t repeat the name of package — just describe what it does.
Don’t include a period at the end. Don’t include newline characters.
Capitalise the first character. For brevity, don’t repeat the name of
package — just describe what it does.
</para>
<para>
Wrong: <literal>"libpng is a library that allows you to decode PNG images."</literal>
Wrong: <literal>"libpng is a library that allows you to decode PNG
images."</literal>
</para>
<para>
Right: <literal>"A library for decoding PNG images"</literal>
@@ -110,7 +122,9 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
Release branch. Used to specify that a package is not going to receive updates that are not in this branch; for example, Linux kernel 3.0 is supposed to be updated to 3.0.X, not 3.1.
Release branch. Used to specify that a package is not going to receive
updates that are not in this branch; for example, Linux kernel 3.0 is
supposed to be updated to 3.0.X, not 3.1.
</para>
</listitem>
</varlistentry>
@@ -120,7 +134,8 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
The package’s homepage. Example:<literal>https://www.gnu.org/software/hello/manual/</literal>
@@ -130,17 +145,8 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
The page where a link to the current version can be found. Example:<literal>https://ftp.gnu.org/gnu/hello/</literal>
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<varname>changelog</varname>
</term>
<listitem>
<para>
A link or a list of links to the location of Changelog for a package. A link may use expansion to refer to the correct changelog version. Example: <literal>"https://git.savannah.gnu.org/cgit/hello.git/plain/NEWS?h=v${version}"</literal>
The page where a link to the current version can be found. Example:
<literal>http://ftp.gnu.org/gnu/hello/</literal>
</para>
</listitem>
</varlistentry>
@@ -150,32 +156,46 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
The license, or licenses, for the package. One from the attribute set defined in <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/lib/licenses.nix"><filename>nixpkgs/lib/licenses.nix</filename></link>. At this moment using both a list of licenses and a single license is valid. If the license field is in the form of a list representation, then it means that parts of the package are licensed differently. Each license should preferably be referenced by their attribute. The non-list attribute value can also be a space delimited string representation of the contained attribute shortNames or spdxIds. The following are all valid examples:
The license, or licenses, for the package. One from the attribute set
@@ -189,8 +209,13 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
A list of names and e-mail addresses of the maintainers of this Nix expression. If you would like to be a maintainer of a package, you may want to add yourself to <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/maintainers/maintainer-list.nix"><filename>nixpkgs/maintainers/maintainer-list.nix</filename></link> and write something like <literal>[ stdenv.lib.maintainers.alice stdenv.lib.maintainers.bob ]</literal>.
A list of names and e-mail addresses of the maintainers of this Nix
expression. If you would like to be a maintainer of a package, you may
and write something like <literal>[ stdenv.lib.maintainers.alice
stdenv.lib.maintainers.bob ]</literal>.
</para>
</listitem>
</varlistentry>
@@ -200,7 +225,10 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
The <emphasis>priority</emphasis> of the package, used by<command>nix-env</command> to resolve file name conflicts between packages. See the Nix manual page for <command>nix-env</command> for details. Example: <literal>"10"</literal> (a low-priority package).
The <emphasis>priority</emphasis> of the package, used by
<command>nix-env</command> to resolve file name conflicts between
packages. See the Nix manual page for <command>nix-env</command> for
details. Example: <literal>"10"</literal> (a low-priority package).
</para>
</listitem>
</varlistentry>
@@ -210,48 +238,15 @@ hello-2.3 A program that produces a familiar, friendly greeting
</term>
<listitem>
<para>
The list of Nix platform types on which the package is supported. Hydra builds packages according to the platform specified. If no platform is specified, the package does not have prebuilt binaries. An example is:
The list of Nix platform types on which the package is supported. Hydra
builds packages according to the platform specified. If no platform is
specified, the package does not have prebuilt binaries. An example is:
<programlisting>
meta.platforms = stdenv.lib.platforms.linux;
</programlisting>
Attribute Set <varname>stdenv.lib.platforms</varname> defines<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/lib/systems/doubles.nix"> various common lists</link> of platforms types.
</para>
</listitem>
</varlistentry>
<varlistentry>
<term>
<varname>tests</varname>
</term>
<listitem>
<warning>
<para>
This attribute is special in that it is not actually under the <literal>meta</literal> attribute set but rather under the <literal>passthru</literal> attribute set. This is due to how <literal>meta</literal> attributes work, and the fact that they are supposed to contain only metadata, not derivations.
</para>
</warning>
<para>
An attribute set with as values tests. A test is a derivation, which builds successfully when the test passes, and fails to build otherwise. A derivation that is a test needs to have <literal>meta.timeout</literal> defined.
</para>
<para>
The NixOS tests are available as <literal>nixosTests</literal> in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, it can fail due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in <literal>nixpkgs</literal>.
Attribute Set <varname>stdenv.lib.platforms</varname> defines
The list of Nix platform types for which the Hydra instance at<literal>hydra.nixos.org</literal> will build the package. (Hydra is the Nix-based continuous build system.) It defaults to the value of <varname>meta.platforms</varname>. Thus, the only reason to set <varname>meta.hydraPlatforms</varname> is if you want <literal>hydra.nixos.org</literal> to build the package on a subset of <varname>meta.platforms</varname>, or not at all, e.g.
The list of Nix platform types for which the Hydra instance at
<literal>hydra.nixos.org</literal> will build the package. (Hydra is the
Nix-based continuous build system.) It defaults to the value of
<varname>meta.platforms</varname>. Thus, the only reason to set
<varname>meta.hydraPlatforms</varname> is if you want
<literal>hydra.nixos.org</literal> to build the package on a subset of
<varname>meta.platforms</varname>, or not at all, e.g.
<programlisting>
meta.platforms = stdenv.lib.platforms.linux;
meta.hydraPlatforms = [];
@@ -275,7 +276,10 @@ meta.hydraPlatforms = [];
</term>
<listitem>
<para>
If set to <literal>true</literal>, the package is marked as “broken”, meaning that it won’t show up in <literal>nix-env -qa</literal>, and cannot be built or installed. Such packages should be removed from Nixpkgs eventually unless they are fixed.
If set to <literal>true</literal>, the package is marked as “broken”,
meaning that it won’t show up in <literal>nix-env -qa</literal>, and
cannot be built or installed. Such packages should be removed from
Nixpkgs eventually unless they are fixed.
</para>
</listitem>
</varlistentry>
@@ -285,7 +289,12 @@ meta.hydraPlatforms = [];
</term>
<listitem>
<para>
If set to <literal>true</literal>, the package is tested to be updated correctly by the <literal>update-walker.sh</literal> script without additional settings. Such packages have <varname>meta.version</varname> set and their homepage (or the page specified by <varname>meta.downloadPage</varname>) contains a direct link to the package tarball.
If set to <literal>true</literal>, the package is tested to be updated
correctly by the <literal>update-walker.sh</literal> script without
additional settings. Such packages have <varname>meta.version</varname>
set and their homepage (or the page specified by
<varname>meta.downloadPage</varname>) contains a direct link to the
package tarball.
</para>
</listitem>
</varlistentry>
@@ -295,11 +304,17 @@ meta.hydraPlatforms = [];
<title>Licenses</title>
<para>
The <varname>meta.license</varname> attribute should preferrably contain a value from <varname>stdenv.lib.licenses</varname> defined in <linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/lib/licenses.nix"><filename>nixpkgs/lib/licenses.nix</filename></link>, or in-place license description of the same format if the license is unlikely to be useful in another expression.
The <varname>meta.license</varname> attribute should preferrably contain a
value from <varname>stdenv.lib.licenses</varname> defined in
<filename>nixpkgs/lib/licenses.nix</filename></link>, or in-place license
description of the same format if the license is unlikely to be useful in
another expression.
</para>
<para>
Although it's typically better to indicate the specific license, a few generic options are available:
Although it's typically better to indicate the specific license, a few
generic options are available:
<variablelist>
<varlistentry>
<term>
@@ -317,10 +332,18 @@ meta.hydraPlatforms = [];
</term>
<listitem>
<para>
Unfree package that can be redistributed in binary form. That is, it’s legal to redistribute the <emphasis>output</emphasis> of the derivation. This means that the package can be included in the Nixpkgs channel.
Unfree package that can be redistributed in binary form. That is, it’s
legal to redistribute the <emphasis>output</emphasis> of the derivation.
This means that the package can be included in the Nixpkgs channel.
</para>
<para>
Sometimes proprietary software can only be redistributed unmodified. Make sure the builder doesn’t actually modify the original binaries; otherwise we’re breaking the license. For instance, the NVIDIA X11 drivers can be redistributed unmodified, but our builder applies <command>patchelf</command> to make them work. Thus, its license is <varname>"unfree"</varname> and it cannot be included in the Nixpkgs channel.
Sometimes proprietary software can only be redistributed unmodified.
Make sure the builder doesn’t actually modify the original binaries;
otherwise we’re breaking the license. For instance, the NVIDIA X11
drivers can be redistributed unmodified, but our builder applies
<command>patchelf</command> to make them work. Thus, its license is
<varname>"unfree"</varname> and it cannot be included in the Nixpkgs
channel.
</para>
</listitem>
</varlistentry>
@@ -330,7 +353,9 @@ meta.hydraPlatforms = [];
</term>
<listitem>
<para>
Unfree package that cannot be redistributed. You can build it yourself, but you cannot redistribute the output of the derivation. Thus it cannot be included in the Nixpkgs channel.
Unfree package that cannot be redistributed. You can build it yourself,
but you cannot redistribute the output of the derivation. Thus it cannot
be included in the Nixpkgs channel.
</para>
</listitem>
</varlistentry>
@@ -340,7 +365,9 @@ meta.hydraPlatforms = [];
</term>
<listitem>
<para>
This package supplies unfree, redistributable firmware. This is a separate value from <varname>unfree-redistributable</varname> because not everybody cares whether firmware is free.
This package supplies unfree, redistributable firmware. This is a
separate value from <varname>unfree-redistributable</varname> because
The Nix language allows a derivation to produce multiple outputs, which is similar to what is utilized by other Linux distribution packaging systems. The outputs reside in separate Nix store paths, so they can be mostly handled independently of each other, including passing to build inputs, garbage collection or binary substitution. The exception is that building from source always produces all the outputs.
The Nix language allows a derivation to produce multiple outputs, which is
similar to what is utilized by other Linux distribution packaging systems.
The outputs reside in separate nix store paths, so they can be mostly
handled independently of each other, including passing to build inputs,
garbage collection or binary substitution. The exception is that building
from source always produces all the outputs.
</para>
<para>
The main motivation is to save disk space by reducing runtime closure sizes; consequently also sizes of substituted binaries get reduced. Splitting can be used to have more granular runtime dependencies, for example the typical reduction is to split away development-only files, as those are typically not needed during runtime. As a result, closure sizes of many packages can get reduced to a half or even much less.
The main motivation is to save disk space by reducing runtime closure sizes;
consequently also sizes of substituted binaries get reduced. Splitting can
be used to have more granular runtime dependencies, for example the typical
reduction is to split away development-only files, as those are typically
not needed during runtime. As a result, closure sizes of many packages can
get reduced to a half or even much less.
</para>
<note>
<para>
The reduction effects could be instead achieved by building the parts in completely separate derivations. That would often additionally reduce build-time closures, but it tends to be much harder to write such derivations, as build systems typically assume all parts are being built at once. This compromise approach of single source package producing multiple binary packages is also utilized often by rpm and deb.
The reduction effects could be instead achieved by building the parts in
completely separate derivations. That would often additionally reduce
build-time closures, but it tends to be much harder to write such
derivations, as build systems typically assume all parts are being built at
once. This compromise approach of single source package producing multiple
binary packages is also utilized often by rpm and deb.
</para>
</note>
</section>
@@ -27,18 +42,23 @@
<title>Installing a split package</title>
<para>
When installing a package via <varname>systemPackages</varname> or<command>nix-env</command> you have several options:
When installing a package via <varname>systemPackages</varname> or
<command>nix-env</command> you have several options:
</para>
<itemizedlist>
<listitem>
<para>
You can install particular outputs explicitly, as each is available in the Nix language as an attribute of the package. The <varname>outputs</varname> attribute contains a list of output names.
You can install particular outputs explicitly, as each is available in the
Nix language as an attribute of the package. The
<varname>outputs</varname> attribute contains a list of output names.
</para>
</listitem>
<listitem>
<para>
You can let it use the default outputs. These are handled by<varname>meta.outputsToInstall</varname> attribute that contains a list of output names.
You can let it use the default outputs. These are handled by
<varname>meta.outputsToInstall</varname> attribute that contains a list of
output names.
</para>
<para>
TODO: more about tweaking the attribute, etc.
@@ -46,11 +66,19 @@
</listitem>
<listitem>
<para>
NixOS provides configuration option<varname>environment.extraOutputsToInstall</varname> that allows adding extra outputs of <varname>environment.systemPackages</varname> atop the default ones. It's mainly meant for documentation and debug symbols, and it's also modified by specific options.
NixOS provides configuration option
<varname>environment.extraOutputsToInstall</varname> that allows adding
extra outputs of <varname>environment.systemPackages</varname> atop the
default ones. It's mainly meant for documentation and debug symbols, and
it's also modified by specific options.
</para>
<note>
<para>
At this moment there is no similar configurability for packages installed by <command>nix-env</command>. You can still use approach from <xreflinkend="sec-modify-via-packageOverrides"/> to override <varname>meta.outputsToInstall</varname> attributes, but that's a rather inconvenient way.
At this moment there is no similar configurability for packages installed
by <command>nix-env</command>. You can still use approach from
<xreflinkend="sec-modify-via-packageOverrides"/> to override
<varname>meta.outputsToInstall</varname> attributes, but that's a rather
inconvenient way.
</para>
</note>
</listitem>
@@ -60,15 +88,18 @@
<title>Using a split package</title>
<para>
In the Nix language the individual outputs can be reached explicitly as attributes, e.g. <varname>coreutils.info</varname>, but the typical case is just using packages as build inputs.
In the Nix language the individual outputs can be reached explicitly as
attributes, e.g. <varname>coreutils.info</varname>, but the typical case is
just using packages as build inputs.
</para>
<para>
When a multiple-output derivation gets into a build input of another derivation, the <varname>dev</varname> output is added if it exists, otherwise the first output is added. In addition to that, <varname>propagatedBuildOutputs</varname> of that package which by default contain <varname>$outputBin</varname> and <varname>$outputLib</varname> are also added. (See <xreflinkend="multiple-output-file-type-groups"/>.)
</para>
<para>
In some cases it may be desirable to combine different outputs under a single store path. A function <literal>symlinkJoin</literal>can be used to do this. (Note that it may negate some closure size benefits of using a multiple-output package.)
When a multiple-output derivation gets into a build input of another
derivation, the <varname>dev</varname> output is added if it exists,
otherwise the first output is added. In addition to that,
<varname>propagatedBuildOutputs</varname> of that package which by default
contain <varname>$outputBin</varname> and <varname>$outputLib</varname> are
also added. (See <xreflinkend="multiple-output-file-type-groups"/>.)
</para>
</section>
<sectionxml:id="sec-multiple-outputs-">
@@ -79,18 +110,29 @@
</para>
<para>
In nixpkgs there is a framework supporting multiple-output derivations. It tries to cover most cases by default behavior. You can find the source separated in <<filename>nixpkgs/pkgs/build-support/setup-hooks/multiple-outputs.sh</filename>>; it's relatively well-readable. The whole machinery is triggered by defining the <varname>outputs</varname> attribute to contain the list of desired output names (strings).
In nixpkgs there is a framework supporting multiple-output derivations. It
tries to cover most cases by default behavior. You can find the source
Often such a single line is enough. For each output an equally named environment variable is passed to the builder and contains the path in nix store for that output. Typically you also want to have the main <varname>out</varname> output, as it catches any files that didn't get elsewhere.
Often such a single line is enough. For each output an equally named
environment variable is passed to the builder and contains the path in nix
store for that output. Typically you also want to have the main
<varname>out</varname> output, as it catches any files that didn't get
elsewhere.
</para>
<note>
<para>
There is a special handling of the <varname>debug</varname> output, described at <xreflinkend="stdenv-separateDebugInfo"/>.
There is a special handling of the <varname>debug</varname> output,
described at <xreflinkend="stdenv-separateDebugInfo"/>.
</para>
</note>
@@ -98,15 +140,36 @@
<title><quote>Binaries first</quote></title>
<para>
A commonly adopted convention in <literal>nixpkgs</literal> is that executables provided by the package are contained within its first output. This convention allows the dependent packages to reference the executables provided by packages in a uniform manner. For instance, provided with the knowledge that the <literal>perl</literal> package contains a <literal>perl</literal> executable it can be referenced as <literal>${pkgs.perl}/bin/perl</literal> within a Nix derivation that needs to execute a Perl script.
A commonly adopted convention in <literal>nixpkgs</literal> is that
executables provided by the package are contained within its first output.
This convention allows the dependent packages to reference the executables
provided by packages in a uniform manner. For instance, provided with the
knowledge that the <literal>perl</literal> package contains a
<literal>perl</literal> executable it can be referenced as
<literal>${pkgs.perl}/bin/perl</literal> within a Nix derivation that needs
to execute a Perl script.
</para>
<para>
The <literal>glibc</literal> package is a deliberate single exception to the <quote>binaries first</quote> convention. The <literal>glibc</literal> has <literal>libs</literal> as its first output allowing the libraries provided by <literal>glibc</literal> to be referenced directly (e.g. <literal>${stdenv.glibc}/lib/ld-linux-x86-64.so.2</literal>). The executables provided by <literal>glibc</literal> can be accessed via its <literal>bin</literal> attribute (e.g. <literal>${stdenv.glibc.bin}/bin/ldd</literal>).
The <literal>glibc</literal> package is a deliberate single exception to
the <quote>binaries first</quote> convention. The <literal>glibc</literal>
has <literal>libs</literal> as its first output allowing the libraries
provided by <literal>glibc</literal> to be referenced directly (e.g.
<literal>${stdenv.glibc}/lib/ld-linux-x86-64.so.2</literal>). The
executables provided by <literal>glibc</literal> can be accessed via its
<literal>bin</literal> attribute (e.g.
<literal>${stdenv.glibc.bin}/bin/ldd</literal>).
</para>
<para>
The reason for why <literal>glibc</literal> deviates from the convention is because referencing a library provided by <literal>glibc</literal> is a very common operation among Nix packages. For instance, third-party executables packaged by Nix are typically patched and relinked with the relevant version of <literal>glibc</literal> libraries from Nix packages (please see the documentation on <linkxlink:href="https://nixos.org/patchelf.html">patchelf</link> for more details).
The reason for why <literal>glibc</literal> deviates from the convention is
because referencing a library provided by <literal>glibc</literal> is a
very common operation among Nix packages. For instance, third-party
executables packaged by Nix are typically patched and relinked with the
relevant version of <literal>glibc</literal> libraries from Nix packages
(please see the documentation on
<linkxlink:href="https://nixos.org/patchelf.html">patchelf</link> for more
details).
</para>
</section>
@@ -114,7 +177,13 @@
<title>File type groups</title>
<para>
The support code currently recognizes some particular kinds of outputs and either instructs the build system of the package to put files into their desired outputs or it moves the files during the fixup phase. Each group of file types has an <varname>outputFoo</varname> variable specifying the output name where they should go. If that variable isn't defined by the derivation writer, it is guessed – a default output name is defined, falling back to other possibilities if the output isn't defined.
The support code currently recognizes some particular kinds of outputs and
either instructs the build system of the package to put files into their
desired outputs or it moves the files during the fixup phase. Each group of
file types has an <varname>outputFoo</varname> variable specifying the
output name where they should go. If that variable isn't defined by the
derivation writer, it is guessed – a default output name is defined,
falling back to other possibilities if the output isn't defined.
</para>
<variablelist>
@@ -124,7 +193,9 @@
</term>
<listitem>
<para>
is for development-only files. These include C(++) headers, pkg-config, cmake and aclocal files. They go to <varname>dev</varname> or <varname>out</varname> by default.
is for development-only files. These include C(++) headers, pkg-config,
cmake and aclocal files. They go to <varname>dev</varname> or
<varname>out</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -134,7 +205,8 @@
</term>
<listitem>
<para>
is meant for user-facing binaries, typically residing in bin/. They go to <varname>bin</varname> or <varname>out</varname> by default.
is meant for user-facing binaries, typically residing in bin/. They go
to <varname>bin</varname> or <varname>out</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -144,7 +216,9 @@
</term>
<listitem>
<para>
is meant for libraries, typically residing in <filename>lib/</filename> and <filename>libexec/</filename>. They go to <varname>lib</varname> or <varname>out</varname> by default.
is meant for libraries, typically residing in <filename>lib/</filename>
and <filename>libexec/</filename>. They go to <varname>lib</varname> or
<varname>out</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -154,7 +228,9 @@
</term>
<listitem>
<para>
is for user documentation, typically residing in<filename>share/doc/</filename>. It goes to <varname>doc</varname> or <varname>out</varname> by default.
is for user documentation, typically residing in
<filename>share/doc/</filename>. It goes to <varname>doc</varname> or
<varname>out</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -164,7 +240,10 @@
</term>
<listitem>
<para>
is for <emphasis>developer</emphasis> documentation. Currently we count gtk-doc and devhelp books in there. It goes to <varname>devdoc</varname> or is removed (!) by default. This is because e.g. gtk-doc tends to be rather large and completely unused by nixpkgs users.
is for <emphasis>developer</emphasis> documentation. Currently we count
gtk-doc and devhelp books in there. It goes to <varname>devdoc</varname>
or is removed (!) by default. This is because e.g. gtk-doc tends to be
rather large and completely unused by nixpkgs users.
</para>
</listitem>
</varlistentry>
@@ -174,7 +253,8 @@
</term>
<listitem>
<para>
is for man pages (except for section 3). They go to<varname>man</varname> or <varname>$outputBin</varname> by default.
is for man pages (except for section 3). They go to
<varname>man</varname> or <varname>$outputBin</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -184,7 +264,8 @@
</term>
<listitem>
<para>
is for section 3 man pages. They go to <varname>devman</varname> or<varname>$outputMan</varname> by default.
is for section 3 man pages. They go to <varname>devman</varname> or
<varname>$outputMan</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -194,7 +275,8 @@
</term>
<listitem>
<para>
is for info pages. They go to <varname>info</varname> or<varname>$outputBin</varname> by default.
is for info pages. They go to <varname>info</varname> or
<varname>$outputBin</varname> by default.
</para>
</listitem>
</varlistentry>
@@ -207,22 +289,31 @@
<itemizedlist>
<listitem>
<para>
Some configure scripts don't like some of the parameters passed by default by the framework, e.g. <literal>--docdir=/foo/bar</literal>. You can disable this by setting <literal>setOutputFlags = false;</literal>.
Some configure scripts don't like some of the parameters passed by
default by the framework, e.g. <literal>--docdir=/foo/bar</literal>. You
can disable this by setting <literal>setOutputFlags = false;</literal>.
</para>
</listitem>
<listitem>
<para>
The outputs of a single derivation can retain references to each other, but note that circular references are not allowed. (And each strongly-connected component would act as a single output anyway.)
The outputs of a single derivation can retain references to each other,
but note that circular references are not allowed. (And each
strongly-connected component would act as a single output anyway.)
</para>
</listitem>
<listitem>
<para>
Most of split packages contain their core functionality in libraries. These libraries tend to refer to various kind of data that typically gets into <varname>out</varname>, e.g. locale strings, so there is often no advantage in separating the libraries into <varname>lib</varname>, as keeping them in <varname>out</varname> is easier.
Most of split packages contain their core functionality in libraries.
These libraries tend to refer to various kind of data that typically gets
into <varname>out</varname>, e.g. locale strings, so there is often no
advantage in separating the libraries into <varname>lib</varname>, as
keeping them in <varname>out</varname> is easier.
</para>
</listitem>
<listitem>
<para>
Some packages have hidden assumptions on install paths, which complicates splitting.
Some packages have hidden assumptions on install paths, which complicates
This chapter describes how to extend and change Nixpkgs using overlays. Overlays are used to add layers in the fixed-point used by Nixpkgs to compose the set of all packages.
This chapter describes how to extend and change Nixpkgs packages using
overlays. Overlays are used to add layers in the fix-point used by Nixpkgs to
compose the set of all packages.
</para>
<para>
Nixpkgs can be configured with a list of overlays, which are applied in order. This means that the order of the overlays can be significant if multiple layers override the same package.
Nixpkgs can be configured with a list of overlays, which are applied in
order. This means that the order of the overlays can be significant if
The list of overlays can be set either explicitly in a Nix expression, or through <literal><nixpkgs-overlays></literal> or user configuration files.
The list of overlays is determined as follows.
</para>
<sectionxml:id="sec-overlays-argument">
<title>Set overlays in NixOS or Nix expressions</title>
<para>
If the <varname>overlays</varname> argument is not provided explicitly, we
look for overlays in a path. The path is determined as follows:
<orderedlist>
<listitem>
<para>
First, if an <varname>overlays</varname> argument to the nixpkgs function
itself is given, then that is used.
</para>
<para>
This can be passed explicitly when importing nipxkgs, for example
Otherwise, if the Nix path entry <literal><nixpkgs-overlays></literal>
exists, we look for overlays at that path, as described below.
</para>
<para>
See the section on <literal>NIX_PATH</literal> in the Nix manual for more
details on how to set a value for
<literal><nixpkgs-overlays>.</literal>
</para>
</listitem>
<listitem>
<para>
If one of <filename>~/.config/nixpkgs/overlays.nix</filename> and
<filename>~/.config/nixpkgs/overlays/</filename> exists, then we look for
overlays at that path, as described below. It is an error if both exist.
</para>
</listitem>
</orderedlist>
</para>
<para>
On a NixOS system the value of the <literal>nixpkgs.overlays</literal> option, if present, is passed to the system Nixpkgs directly as an argument. Note that this does not affect the overlays for non-NixOS operations (e.g. <literal>nix-env</literal>), which are <linkxlink:href="#sec-overlays-lookup">looked</link> up independently.
</para>
<para>
If we are looking for overlays at a path, then there are two cases:
<itemizedlist>
<listitem>
<para>
If the path is a file, then the file is imported as a Nix expression and
used as the list of overlays.
</para>
</listitem>
<listitem>
<para>
If the path is a directory, then we take the content of the directory,
order it lexicographically, and attempt to interpret each as an overlay
by:
<itemizedlist>
<listitem>
<para>
Importing the file, if it is a <literal>.nix</literal> file.
</para>
</listitem>
<listitem>
<para>
Importing a top-level <filename>default.nix</filename> file, if it is
a directory.
</para>
</listitem>
</itemizedlist>
</para>
</listitem>
</itemizedlist>
</para>
<para>
The list of overlays can be passed explicitly when importing nixpkgs, for example <literal>import <nixpkgs> { overlays = [ overlay1 overlay2 ]; }</literal>.
</para>
<para>
On a NixOS system the value of the <literal>nixpkgs.overlays</literal>
option, if present, is passed to the system Nixpkgs directly as an argument.
Note that this does not affect the overlays for non-NixOS operations (e.g.
<literal>nix-env</literal>), which are looked up independently.
</para>
<para>
Further overlays can be added by calling the <literal>pkgs.extend</literal> or <literal>pkgs.appendOverlays</literal>, although it is often preferable to avoid these functions, because they recompute the Nixpkgs fixpoint, which is somewhat expensive to do.
</para>
</section>
<sectionxml:id="sec-overlays-lookup">
<title>Install overlays via configuration lookup</title>
<para>
The list of overlays is determined as follows.
</para>
<para>
<orderedlist>
<listitem>
<para>
First, if an <linkxlink:href="#sec-overlays-argument"><varname>overlays</varname> argument</link> to the Nixpkgs function itself is given, then that is used and no path lookup will be performed.
</para>
</listitem>
<listitem>
<para>
Otherwise, if the Nix path entry <literal><nixpkgs-overlays></literal> exists, we look for overlays at that path, as described below.
</para>
<para>
See the section on <literal>NIX_PATH</literal> in the Nix manual for more details on how to set a value for <literal><nixpkgs-overlays>.</literal>
</para>
</listitem>
<listitem>
<para>
If one of <filename>~/.config/nixpkgs/overlays.nix</filename> and <filename>~/.config/nixpkgs/overlays/</filename> exists, then we look for overlays at that path, as described below. It is an error if both exist.
</para>
</listitem>
</orderedlist>
</para>
<para>
If we are looking for overlays at a path, then there are two cases:
<itemizedlist>
<listitem>
<para>
If the path is a file, then the file is imported as a Nix expression and used as the list of overlays.
</para>
</listitem>
<listitem>
<para>
If the path is a directory, then we take the content of the directory, order it lexicographically, and attempt to interpret each as an overlay by:
<itemizedlist>
<listitem>
<para>
Importing the file, if it is a <literal>.nix</literal> file.
</para>
</listitem>
<listitem>
<para>
Importing a top-level <filename>default.nix</filename> file, if it is a directory.
</para>
</listitem>
</itemizedlist>
</para>
</listitem>
</itemizedlist>
</para>
<para>
Because overlays that are set in NixOS configuration do not affect non-NixOS operations such as <literal>nix-env</literal>, the <filename>overlays.nix</filename> option provides a convenient way to use the same overlays for a NixOS system configuration and user configuration: the same file can be used as <filename>overlays.nix</filename> and imported as the value of <literal>nixpkgs.overlays</literal>.
</para>
<!-- TODO: Example of sharing overlays between NixOS configuration
and configuration lookup. Also reference the example
from the sec-overlays-argument paragraph about NixOS.
-->
</section>
<para>
The <filename>overlays.nix</filename> option therefore provides a convenient
way to use the same overlays for a NixOS system configuration and user
configuration: the same file can be used as
<filename>overlays.nix</filename> and imported as the value of
Overlays are Nix functions which accept two arguments, conventionally called<varname>self</varname> and <varname>super</varname>, and return a set of packages. For example, the following is a valid overlay.
Overlays are Nix functions which accept two arguments, conventionally called
<varname>self</varname> and <varname>super</varname>, and return a set of
packages. For example, the following is a valid overlay.
</para>
<programlisting>
@@ -122,19 +127,38 @@ self: super:
</programlisting>
<para>
The first argument (<varname>self</varname>) corresponds to the final package set. You should use this set for the dependencies of all packages specified in your overlay. For example, all the dependencies of <varname>rr</varname> in the example above come from <varname>self</varname>, as well as the overridden dependencies used in the <varname>boost</varname> override.
The first argument (<varname>self</varname>) corresponds to the final
package set. You should use this set for the dependencies of all packages
specified in your overlay. For example, all the dependencies of
<varname>rr</varname> in the example above come from
<varname>self</varname>, as well as the overridden dependencies used in the
<varname>boost</varname> override.
</para>
<para>
The second argument (<varname>super</varname>) corresponds to the result of the evaluation of the previous stages of Nixpkgs. It does not contain any of the packages added by the current overlay, nor any of the following overlays. This set should be used either to refer to packages you wish to override, or to access functions defined in Nixpkgs. For example, the original recipe of <varname>boost</varname> in the above example, comes from <varname>super</varname>, as well as the <varname>callPackage</varname> function.
The second argument (<varname>super</varname>) corresponds to the result of
the evaluation of the previous stages of Nixpkgs. It does not contain any of
the packages added by the current overlay, nor any of the following
overlays. This set should be used either to refer to packages you wish to
override, or to access functions defined in Nixpkgs. For example, the
original recipe of <varname>boost</varname> in the above example, comes from
<varname>super</varname>, as well as the <varname>callPackage</varname>
function.
</para>
<para>
The value returned by this function should be a set similar to<filename>pkgs/top-level/all-packages.nix</filename>, containing overridden and/or new packages.
The value returned by this function should be a set similar to
Overlays are similar to other methods for customizing Nixpkgs, in particular the <literal>packageOverrides</literal> attribute described in <xreflinkend="sec-modify-via-packageOverrides"/>. Indeed, <literal>packageOverrides</literal> acts as an overlay with only the <varname>super</varname> argument. It is therefore appropriate for basic use, but overlays are more powerful and easier to distribute.
Overlays are similar to other methods for customizing Nixpkgs, in particular
the <literal>packageOverrides</literal> attribute described in
This chapter contains information about how to use and maintain the Nix expressions for a number of specific packages, such as the Linux kernel or X.org.
This chapter contains information about how to use and maintain the Nix
expressions for a number of specific packages, such as the Linux kernel or
The function that builds the kernel has an argument<varname>kernelPatches</varname> which should be a list of <literal>{name, patch, extraConfig}</literal> attribute sets, where <varname>name</varname> is the name of the patch (which is included in the kernel’s <varname>meta.description</varname> attribute), <varname>patch</varname> is the patch itself (possibly compressed), and <varname>extraConfig</varname> (optional) is a string specifying extra options to be concatenated to the kernel configuration file (<filename>.config</filename>).
The function that builds the kernel has an argument
<varname>kernelPatches</varname> which should be a list of <literal>{name,
patch, extraConfig}</literal> attribute sets, where <varname>name</varname>
is the name of the patch (which is included in the kernel’s
<varname>meta.description</varname> attribute), <varname>patch</varname> is
the patch itself (possibly compressed), and <varname>extraConfig</varname>
(optional) is a string specifying extra options to be concatenated to the
The kernel derivation exports an attribute <varname>features</varname> specifying whether optional functionality is or isn’t enabled. This is used in NixOS to implement kernel-specific behaviour. For instance, if the kernel has the <varname>iwlwifi</varname> feature (i.e. has built-in support for Intel wireless chipsets), then NixOS doesn’t have to build the external <varname>iwlwifi</varname> package:
The kernel derivation exports an attribute <varname>features</varname>
specifying whether optional functionality is or isn’t enabled. This is
used in NixOS to implement kernel-specific behaviour. For instance, if the
kernel has the <varname>iwlwifi</varname> feature (i.e. has built-in support
for Intel wireless chipsets), then NixOS doesn’t have to build the
Copy the old Nix expression (e.g. <filename>linux-2.6.21.nix</filename>) to the new one (e.g. <filename>linux-2.6.22.nix</filename>) and update it.
Copy the old Nix expression (e.g. <filename>linux-2.6.21.nix</filename>)
to the new one (e.g. <filename>linux-2.6.22.nix</filename>) and update
it.
</para>
</listitem>
<listitem>
<para>
Add the new kernel to <filename>all-packages.nix</filename> (e.g., create an attribute <varname>kernel_2_6_22</varname>).
Add the new kernel to <filename>all-packages.nix</filename> (e.g., create
an attribute <varname>kernel_2_6_22</varname>).
</para>
</listitem>
<listitem>
<para>
Now we’re going to update the kernel configuration. First unpack the kernel. Then for each supported platform (<literal>i686</literal>, <literal>x86_64</literal>, <literal>uml</literal>) do the following:
Now we’re going to update the kernel configuration. First unpack the
kernel. Then for each supported platform (<literal>i686</literal>,
<literal>x86_64</literal>, <literal>uml</literal>) do the following:
<orderedlist>
<listitem>
<para>
Make an copy from the old config (e.g.<filename>config-2.6.21-i686-smp</filename>) to the new one (e.g. <filename>config-2.6.22-i686-smp</filename>).
Make an copy from the old config (e.g.
<filename>config-2.6.21-i686-smp</filename>) to the new one (e.g.
<filename>config-2.6.22-i686-smp</filename>).
</para>
</listitem>
<listitem>
<para>
Copy the config file for this platform (e.g.<filename>config-2.6.22-i686-smp</filename>) to <filename>.config</filename> in the kernel source tree.
Copy the config file for this platform (e.g.
<filename>config-2.6.22-i686-smp</filename>) to
<filename>.config</filename> in the kernel source tree.
</para>
</listitem>
<listitem>
<para>
Run <literal>make oldconfig ARCH=<replaceable>{i386,x86_64,um}</replaceable></literal> and answer all questions. (For the uml configuration, also add <literal>SHELL=bash</literal>.) Make sure to keep the configuration consistent between platforms (i.e. don’t enable some feature on <literal>i686</literal> and disable it on <literal>x86_64</literal>).
Run <literal>make oldconfig
ARCH=<replaceable>{i386,x86_64,um}</replaceable></literal> and answer
all questions. (For the uml configuration, also add
<literal>SHELL=bash</literal>.) Make sure to keep the configuration
consistent between platforms (i.e. don’t enable some feature on
<literal>i686</literal> and disable it on <literal>x86_64</literal>).
</para>
</listitem>
<listitem>
<para>
If needed you can also run <literal>make menuconfig</literal>:
$ make menuconfig ARCH=<replaceable>arch</replaceable></screen>
</para>
</listitem>
<listitem>
<para>
Copy <filename>.config</filename> over the new config file (e.g.<filename>config-2.6.22-i686-smp</filename>).
Copy <filename>.config</filename> over the new config file (e.g.
<filename>config-2.6.22-i686-smp</filename>).
</para>
</listitem>
</orderedlist>
@@ -78,12 +108,18 @@ modulesTree = [kernel]
</listitem>
<listitem>
<para>
Test building the kernel: <literal>nix-build -A kernel_2_6_22</literal>. If it compiles, ship it! For extra credit, try booting NixOS with it.
Test building the kernel: <literal>nix-build -A kernel_2_6_22</literal>.
If it compiles, ship it! For extra credit, try booting NixOS with it.
</para>
</listitem>
<listitem>
<para>
It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the <varname>linuxPackagesFor</varname> function in <filename>all-packages.nix</filename> (such as the NVIDIA drivers, AUFS, etc.). If the updated packages aren’t backwards compatible with older kernels, you may need to keep the older versions around.
It may be that the new kernel requires updating the external kernel
modules and kernel-dependent packages listed in the
<varname>linuxPackagesFor</varname> function in
<filename>all-packages.nix</filename> (such as the NVIDIA drivers, AUFS,
etc.). If the updated packages aren’t backwards compatible with older
kernels, you may need to keep the older versions around.
</para>
</listitem>
</orderedlist>
@@ -94,32 +130,53 @@ modulesTree = [kernel]
<title>X.org</title>
<para>
The Nix expressions for the X.org packages reside in<filename>pkgs/servers/x11/xorg/default.nix</filename>. This file is automatically generated from lists of tarballs in an X.org release. As such it should not be modified directly; rather, you should modify the lists, the generator script or the file <filename>pkgs/servers/x11/xorg/overrides.nix</filename>, in which you can override or add to the derivations produced by the generator.
The Nix expressions for the X.org packages reside in
<filename>pkgs/servers/x11/xorg/default.nix</filename>. This file is
automatically generated from lists of tarballs in an X.org release. As such
it should not be modified directly; rather, you should modify the lists, the
generator script or the file
<filename>pkgs/servers/x11/xorg/overrides.nix</filename>, in which you can
override or add to the derivations produced by the generator.
For each of the tarballs in the <filename>.list</filename> files, the script downloads it, unpacks it, and searches its <filename>configure.ac</filename> and <filename>*.pc.in</filename> files for dependencies. This information is used to generate <filename>default.nix</filename>. The generator caches downloaded tarballs between runs. Pay close attention to the <literal>NOT FOUND: <replaceable>name</replaceable></literal> messages at the end of the run, since they may indicate missing dependencies. (Some might be optional dependencies, however.)
For each of the tarballs in the <filename>.list</filename> files, the script
downloads it, unpacks it, and searches its <filename>configure.ac</filename>
and <filename>*.pc.in</filename> files for dependencies. This information is
used to generate <filename>default.nix</filename>. The generator caches
downloaded tarballs between runs. Pay close attention to the <literal>NOT
FOUND: <replaceable>name</replaceable></literal> messages at the end of the
run, since they may indicate missing dependencies. (Some might be optional
dependencies, however.)
</para>
<para>
A file like <filename>tarballs-7.5.list</filename> contains all tarballs in a X.org release. It can be generated like this:
A file like <filename>tarballs-7.5.list</filename> contains all tarballs in
<filename>extra.list</filename> contains libraries that aren’t part of X.org proper, but are closely related to it, such as <literal>libxcb</literal>. <filename>old.list</filename> contains some packages that were removed from X.org, but are still needed by some people or by other packages (such as <varname>imake</varname>).
<filename>extra.list</filename> contains libraries that aren’t part of
X.org proper, but are closely related to it, such as
<literal>libxcb</literal>. <filename>old.list</filename> contains some
packages that were removed from X.org, but are still needed by some people
or by other packages (such as <varname>imake</varname>).
</para>
<para>
If the expression for a package requires derivation attributes that the generator cannot figure out automatically (say, <varname>patches</varname> or a <varname>postInstall</varname> hook), you should modify <filename>pkgs/servers/x11/xorg/overrides.nix</filename>.
If the expression for a package requires derivation attributes that the
generator cannot figure out automatically (say, <varname>patches</varname>
or a <varname>postInstall</varname> hook), you should modify
The Nix expressions related to the Eclipse platform and IDE are in<linkxlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/eclipse"><filename>pkgs/applications/editors/eclipse</filename></link>.
The Nix expressions related to the Eclipse platform and IDE are in
Nixpkgs provides a number of packages that will install Eclipse in its various forms. These range from the bare-bones Eclipse Platform to the more fully featured Eclipse SDK or Scala-IDE packages and multiple version are often available. It is possible to list available Eclipse packages by issuing the command:
Nixpkgs provides a number of packages that will install Eclipse in its
various forms, these range from the bare-bones Eclipse Platform to the more
fully featured Eclipse SDK or Scala-IDE packages and multiple version are
often available. It is possible to list available Eclipse packages by
issuing the command:
<screen>
<prompt>$ </prompt>nix-env -f '<nixpkgs>' -qaP -A eclipses --description
$ nix-env -f '<nixpkgs>' -qaP -A eclipses --description
</screen>
Once an Eclipse variant is installed it can be run using the<command>eclipse</command> command, as expected. From within Eclipse it is then possible to install plugins in the usual manner by either manually specifying an Eclipse update site or by installing the Marketplace Client plugin and using it to discover and install other plugins. This installation method provides an Eclipse installation that closely resemble a manually installed Eclipse.
Once an Eclipse variant is installed it can be run using the
<command>eclipse</command> command, as expected. From within Eclipse it is
then possible to install plugins in the usual manner by either manually
specifying an Eclipse update site or by installing the Marketplace Client
plugin and using it to discover and install other plugins. This installation
method provides an Eclipse installation that closely resemble a manually
installed Eclipse.
</para>
<para>
If you prefer to install plugins in a more declarative manner then Nixpkgs also offer a number of Eclipse plugins that can be installed in an <emphasis>Eclipse environment</emphasis>. This type of environment is created using the function <varname>eclipseWithPlugins</varname> found inside the <varname>nixpkgs.eclipses</varname> attribute set. This function takes as argument <literal>{ eclipse, plugins ? [], jvmArgs ? [] }</literal> where <varname>eclipse</varname> is a one of the Eclipse packages described above, <varname>plugins</varname> is a list of plugin derivations, and <varname>jvmArgs</varname> is a list of arguments given to the JVM running the Eclipse. For example, say you wish to install the latest Eclipse Platform with the popular Eclipse Color Theme plugin and also allow Eclipse to use more RAM. You could then add
If you prefer to install plugins in a more declarative manner then Nixpkgs
also offer a number of Eclipse plugins that can be installed in an
<emphasis>Eclipse environment</emphasis>. This type of environment is
created using the function <varname>eclipseWithPlugins</varname> found
inside the <varname>nixpkgs.eclipses</varname> attribute set. This function
where <varname>eclipse</varname> is a one of the Eclipse packages described
above, <varname>plugins</varname> is a list of plugin derivations, and
<varname>jvmArgs</varname> is a list of arguments given to the JVM running
the Eclipse. For example, say you wish to install the latest Eclipse
Platform with the popular Eclipse Color Theme plugin and also allow Eclipse
to use more RAM. You could then add
<screen>
packageOverrides = pkgs: {
myEclipse = with pkgs.eclipses; eclipseWithPlugins {
@@ -164,18 +243,38 @@ packageOverrides = pkgs: {
};
}
</screen>
to your Nixpkgs configuration (<filename>~/.config/nixpkgs/config.nix</filename>) and install it by running <command>nix-env -f '<nixpkgs>' -iA myEclipse</command> and afterward run Eclipse as usual. It is possible to find out which plugins are available for installation using <varname>eclipseWithPlugins</varname> by running
to your Nixpkgs configuration
(<filename>~/.config/nixpkgs/config.nix</filename>) and install it by
running <command>nix-env -f '<nixpkgs>' -iA myEclipse</command> and
afterward run Eclipse as usual. It is possible to find out which plugins are
available for installation using <varname>eclipseWithPlugins</varname> by
running
<screen>
<prompt>$ </prompt>nix-env -f '<nixpkgs>' -qaP -A eclipses.plugins --description
$ nix-env -f '<nixpkgs>' -qaP -A eclipses.plugins --description
</screen>
</para>
<para>
If there is a need to install plugins that are not available in Nixpkgs then it may be possible to define these plugins outside Nixpkgs using the <varname>buildEclipseUpdateSite</varname> and <varname>buildEclipsePlugin</varname> functions found in the <varname>nixpkgs.eclipses.plugins</varname> attribute set. Use the <varname>buildEclipseUpdateSite</varname> function to install a plugin distributed as an Eclipse update site. This function takes <literal>{ name, src }</literal> as argument where <literal>src</literal> indicates the Eclipse update site archive. All Eclipse features and plugins within the downloaded update site will be installed. When an update site archive is not available then the <varname>buildEclipsePlugin</varname> function can be used to install a plugin that consists of a pair of feature and plugin JARs. This function takes an argument <literal>{ name, srcFeature, srcPlugin }</literal> where <literal>srcFeature</literal> and <literal>srcPlugin</literal> are the feature and plugin JARs, respectively.
If there is a need to install plugins that are not available in Nixpkgs then
it may be possible to define these plugins outside Nixpkgs using the
<varname>buildEclipseUpdateSite</varname> and
<varname>buildEclipsePlugin</varname> functions found in the
<varname>nixpkgs.eclipses.plugins</varname> attribute set. Use the
<varname>buildEclipseUpdateSite</varname> function to install a plugin
distributed as an Eclipse update site. This function takes <literal>{ name,
src }</literal> as argument where <literal>src</literal> indicates the
Eclipse update site archive. All Eclipse features and plugins within the
downloaded update site will be installed. When an update site archive is not
available then the <varname>buildEclipsePlugin</varname> function can be
used to install a plugin that consists of a pair of feature and plugin JARs.
This function takes an argument <literal>{ name, srcFeature, srcPlugin
}</literal> where <literal>srcFeature</literal> and
<literal>srcPlugin</literal> are the feature and plugin JARs, respectively.
</para>
<para>
Expanding the previous example with two plugins using the above functions we have
Expanding the previous example with two plugins using the above functions we
have
<screen>
packageOverrides = pkgs: {
myEclipse = with pkgs.eclipses; eclipseWithPlugins {
@@ -212,34 +311,28 @@ packageOverrides = pkgs: {
<title>Elm</title>
<para>
To start a development environment do <command>nix-shell -p elmPackages.elm elmPackages.elm-format</command>
</para>
<para>
To update Elm compiler, see <filename>nixpkgs/pkgs/development/compilers/elm/README.md</filename>.
</para>
<para>
To package Elm applications, <linkxlink:href="https://github.com/hercules-ci/elm2nix#elm2nix">read about elm2nix</link>.
</para>
</section>
<sectionxml:id="sec-kakoune">
<title>Kakoune</title>
<para>
Kakoune can be built to autoload plugins:
<programlisting>(kakoune.override {
configure = {
plugins = with pkgs.kakounePlugins; [ parinfer-rust ];
};
})</programlisting>
The Nix expressions for Elm reside in
<filename>pkgs/development/compilers/elm</filename>. They are generated
automatically by <command>update-elm.rb</command> script. One should specify
versions of Elm packages inside the script, clear the
<filename>packages</filename> directory and run the script from inside it.
<literal>elm-reactor</literal> is special because it also has Elm package
dependencies. The process is not automated very much for now -- you should
get the <literal>elm-reactor</literal> source tree (e.g. with
<command>nix-shell</command>) and run <command>elm2nix.rb</command> inside
it. Place the resulting <filename>package.nix</filename> file into
Some packages provide the shell integration to be more useful. But unlike other systems, nix doesn't have a standard share directory location. This is why a bunch <command>PACKAGE-share</command> scripts are shipped that print the location of the corresponding shared folder. Current list of such packages is as following:
Some packages provide the shell integration to be more useful. But unlike
other systems, nix doesn't have a standard share directory location. This is
why a bunch <command>PACKAGE-share</command> scripts are shipped that print
the location of the corresponding shared folder. Current list of such
packages is as following:
<itemizedlist>
<listitem>
<para>
@@ -258,24 +351,343 @@ packageOverrides = pkgs: {
</screen>
</para>
</section>
<sectionxml:id="sec-steam">
<title>Steam</title>
<sectionxml:id="sec-steam-nix">
<title>Steam in Nix</title>
<para>
Steam is distributed as a <filename>.deb</filename> file, for now only as
an i686 package (the amd64 package only has documentation). When unpacked,
it has a script called <filename>steam</filename> that in ubuntu (their
target distro) would go to <filename>/usr/bin </filename>. When run for the
first time, this script copies some files to the user's home, which include
another script that is the ultimate responsible for launching the steam
binary, which is also in $HOME.
</para>
<para>
Nix problems and constraints:
<itemizedlist>
<listitem>
<para>
We don't have <filename>/bin/bash</filename> and many scripts point
there. Similarly for <filename>/usr/bin/python</filename> .
</para>
</listitem>
<listitem>
<para>
We don't have the dynamic loader in <filename>/lib </filename>.
</para>
</listitem>
<listitem>
<para>
The <filename>steam.sh</filename> script in $HOME can not be patched, as
it is checked and rewritten by steam.
</para>
</listitem>
<listitem>
<para>
The steam binary cannot be patched, it's also checked.
</para>
</listitem>
</itemizedlist>
</para>
<para>
The current approach to deploy Steam in NixOS is composing a FHS-compatible
This provides a fairly full Emacs start file. It will load in addition to
the user's presonal config. You can always disable it by passing
<command>-q</command> to the Emacs command.
</para>
<para>
Sometimes <varname>emacsWithPackages</varname> is not enough, as this
package set has some priorities imposed on packages (with the lowest
priority assigned to Melpa Unstable, and the highest for packages manually
defined in <filename>pkgs/top-level/emacs-packages.nix</filename>). But you
can't control this priorities when some package is installed as a
dependency. You can override it on per-package-basis, providing all the
required dependencies manually - but it's tedious and there is always a
possibility that an unwanted dependency will sneak in through some other
package. To completely override such a package you can use
<varname>overrideScope'</varname>.
</para>
<screen>
overrides = self: super: rec {
haskell-mode = self.melpaPackages.haskell-mode;
...
};
((emacsPackagesNgGen emacs).overrideScope' overrides).emacsWithPackages (p: with p; [
# here both these package will use haskell-mode of our own choice
ghc-mod
dante
])
</screen>
</section>
</section>
<sectionxml:id="sec-weechat">
<title>Weechat</title>
<para>
Weechat can be configured to include your choice of plugins, reducing its closure size from the default configuration which includes all available plugins. To make use of this functionality, install an expression that overrides its configuration such as
Weechat can be configured to include your choice of plugins, reducing its
closure size from the default configuration which includes all available
plugins. To make use of this functionality, install an expression that
If the <literal>configure</literal> function returns an attrset without the<literal>plugins</literal> attribute, <literal>availablePlugins</literal> will be used automatically.
If the <literal>configure</literal> function returns an attrset without the
The plugins currently available are <literal>python</literal>,<literal>perl</literal>, <literal>ruby</literal>, <literal>guile</literal>, <literal>tcl</literal> and <literal>lua</literal>.
The plugins currently available are <literal>python</literal>,
<literal>tcl</literal> and <literal>lua</literal>.
</para>
<para>
The python and perl plugins allows the addition of extra libraries. For instance, the<literal>inotify.py</literal> script in weechat-scripts requires D-Bus or libnotify, and the <literal>fish.py</literal> script requires pycrypto. To use these scripts, use the plugin's <literal>withPackages</literal> attribute:
The python plugin allows the addition of extra libraries. For instance, the
<literal>inotify.py</literal> script in weechat-scripts requires D-Bus or
libnotify, and the <literal>fish.py</literal> script requires pycrypto. To
use these scripts, use the <literal>python</literal> plugin's
python = availablePlugins.python.withPackages (ps: with ps; [ pycrypto python-dbus ]);
@@ -296,7 +709,9 @@ packageOverrides = pkgs: {
</para>
<para>
WeeChat allows to set defaults on startup using the<literal>--run-command</literal>. The <literal>configure</literal> method can be used to pass commands to the program:
WeeChat allows to set defaults on startup using the
<literal>--run-command</literal>. The <literal>configure</literal> method
can be used to pass commands to the program:
<programlisting>weechat.override {
configure = { availablePlugins, ... }: {
init = ''
@@ -305,11 +720,14 @@ packageOverrides = pkgs: {
'';
};
}</programlisting>
Further values can be added to the list of commands when running<literal>weechat --run-command "your-commands"</literal>.
Further values can be added to the list of commands when running
Additionally it's possible to specify scripts to be loaded when starting<literal>weechat</literal>. These will be loaded before the commands from <literal>init</literal>:
Additionally it's possible to specify scripts to be loaded when starting
<literal>weechat</literal>. These will be loaded before the commands from
<literal>init</literal>:
<programlisting>weechat.override {
configure = { availablePlugins, ... }: {
scripts = with pkgs.weechatScripts; [
@@ -323,7 +741,11 @@ packageOverrides = pkgs: {
</para>
<para>
In <literal>nixpkgs</literal> there's a subpackage which contains derivations for WeeChat scripts. Such derivations expect a <literal>passthru.scripts</literal> attribute which contains a list of all scripts inside the store path. Furthermore all scripts have to live in <literal>$out/share</literal>. An exemplary derivation looks like this:
In <literal>nixpkgs</literal> there's a subpackage which contains
derivations for WeeChat scripts. Such derivations expect a
<literal>passthru.scripts</literal> attribute which contains a list of all
scripts inside the store path. Furthermore all scripts have to live in
<literal>$out/share</literal>. An exemplary derivation looks like this:
<programlisting>{ stdenv, fetchurl }:
stdenv.mkDerivation {
@@ -341,81 +763,61 @@ stdenv.mkDerivation {
}</programlisting>
</para>
</section>
<sectionxml:id="sec-ibus-typing-booster">
<title>ibus-engines.typing-booster</title>
<sectionxml:id="sec-citrix">
<title>Citrix Receiver</title>
<para>
This package is an ibus-based completion method to speed up typing.
The <linkxlink:href="https://www.citrix.com/products/receiver/">Citrix
Receiver</link> is a remote desktop viewer which provides access to
IBus needs to be configured accordingly to activate <literal>typing-booster</literal>. The configuration depends on the desktop manager in use. For detailed instructions, please refer to the <linkxlink:href="https://mike-fabian.github.io/ibus-typing-booster/documentation.html">upstream docs</link>.
The tarball archive needs to be downloaded manually as the licenses
agreements of the vendor need to be accepted first. This is available at
page at citrix.com</link>. Then run <literal>nix-prefetch-url
file://$PWD/linuxx64-$version.tar.gz</literal>. With the archive available
in the store the package can be built and installed with Nix.
</para>
<para>
On NixOS you need to explicitly enable <literal>ibus</literal> with given engines before customizing your desktop to use <literal>typing-booster</literal>. This can be achieved using the <literal>ibus</literal> module:
<programlisting>{ pkgs, ... }: {
i18n.inputMethod = {
enabled = "ibus";
ibus.engines = with pkgs.ibus-engines; [ typing-booster ];
};
}</programlisting>
<emphasis>Note: it's recommended to install <literal>Citrix
Receiver</literal> using <literal>nix-env -i</literal> or globally to
ensure that the <literal>.desktop</literal> files are installed properly
into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to
open <literal>.ica</literal> files automatically from the browser to start
The IBus engine is based on <literal>hunspell</literal> to support completion in many languages. By default the dictionaries <literal>de-de</literal>, <literal>en-us</literal>, <literal>fr-moderne</literal><literal>es-es</literal>, <literal>it-it</literal>, <literal>sv-se</literal> and <literal>sv-fi</literal> are in use. To add another dictionary, the package can be overridden like this:
The <literal>ibus-engines.typing-booster</literal> package contains a program named <literal>emoji-picker</literal>. To display all emojis correctly, a special font such as <literal>noto-fonts-emoji</literal> is needed:
</para>
<para>
On NixOS it can be installed using the following expression:
<programlisting>{ pkgs, ... }: {
fonts.fonts = with pkgs; [ noto-fonts-emoji ];
}</programlisting>
</para>
</section>
</section>
<sectionxml:id="sec-nginx">
<title>Nginx</title>
<para>
<linkxlink:href="https://nginx.org/">Nginx</link> is a reverse proxy and lightweight webserver.
</para>
<sectionxml:id="sec-nginx-etag">
<title>ETags on static files served from the Nix store</title>
<para>
HTTP has a couple different mechanisms for caching to prevent clients from having to download the same content repeatedly if a resource has not changed since the last time it was requested. When nginx is used as a server for static files, it implements the caching mechanism based on the <linkxlink:href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Last-Modified"><literal>Last-Modified</literal></link> response header automatically; unfortunately, it works by using filesystem timestamps to determine the value of the <literal>Last-Modified</literal> header. This doesn't give the desired behavior when the file is in the Nix store, because all file timestamps are set to 0 (for reasons related to build reproducibility).
</para>
<para>
Fortunately, HTTP supports an alternative (and more effective) caching mechanism: the <linkxlink:href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag"><literal>ETag</literal></link> response header. The value of the <literal>ETag</literal> header specifies some identifier for the particular content that the server is sending (e.g. a hash). When a client makes a second request for the same resource, it sends that value back in an <literal>If-None-Match</literal> header. If the ETag value is unchanged, then the server does not need to resend the content.
</para>
<para>
As of NixOS 19.09, the nginx package in Nixpkgs is patched such that when nginx serves a file out of <filename>/nix/store</filename>, the hash in the store path is used as the <literal>ETag</literal> header in the HTTP response, thus providing proper caching functionality. This happens automatically; you do not need to do modify any configuration to get this behavior.
The <literal>Citrix Receiver</literal> in <literal>nixpkgs</literal> trusts
several certificates
<linkxlink:href="https://curl.haxx.se/docs/caextract.html">from the
Mozilla database</link> by default. However several companies using Citrix
might require their own corporate certificate. On distros with imperative
These chapters includes some notes that apply to specific packages and should answer some of the frequently asked questions related to Nixpkgs use. Some useful information related to package use can be found in <linklinkend="chap-package-notes">package-specific development notes</link>.
</para>
<sectionxml:id="opengl">
<title>OpenGL</title>
<para>
Packages that use OpenGL have NixOS desktop as their primary target. The current solution for loading the GPU-specific drivers is based on <literal>libglvnd</literal> and looks for the driver implementation in <literal>LD_LIBRARY_PATH</literal>. If you are using a non-NixOS GNU/Linux/X11 desktop with free software video drivers, consider launching OpenGL-dependent programs from Nixpkgs with Nixpkgs versions of <literal>libglvnd</literal> and <literal>mesa_drivers</literal> in <literal>LD_LIBRARY_PATH</literal>. For proprietary video drivers you might have luck with also adding the corresponding video driver package.
</para>
</section>
<sectionxml:id="locales">
<title>Locales</title>
<para>
To allow simultaneous use of packages linked against different versions of <literal>glibc</literal> with different locale archive formats Nixpkgs patches <literal>glibc</literal> to rely on <literal>LOCALE_ARCHIVE</literal> environment variable.
</para>
<para>
On non-NixOS distributions this variable is obviously not set. This can cause regressions in language support or even crashes in some Nixpkgs-provided programs. The simplest way to mitigate this problem is exporting the <literal>LOCALE_ARCHIVE</literal> variable pointing to <literal>${glibcLocales}/lib/locale/locale-archive</literal>. The drawback (and the reason this is not the default) is the relatively large (a hundred MiB) size of the full set of locales. It is possible to build a custom set of locales by overriding parameters <literal>allLocales</literal> and <literal>locales</literal> of the package.
</para>
</section>
<sectionxml:id="sec-emacs">
<title>Emacs</title>
<sectionxml:id="sec-emacs-config">
<title>Configuring Emacs</title>
<para>
The Emacs package comes with some extra helpers to make it easier to configure. <varname>emacsWithPackages</varname> allows you to manage packages from ELPA. This means that you will not have to install that packages from within Emacs. For instance, if you wanted to use <literal>company</literal>, <literal>counsel</literal>, <literal>flycheck</literal>, <literal>ivy</literal>, <literal>magit</literal>, <literal>projectile</literal>, and <literal>use-package</literal> you could use this as a <filename>~/.config/nixpkgs/config.nix</filename> override:
You can install it like any other packages via <command>nix-env -iA myEmacs</command>. However, this will only install those packages. It will not <literal>configure</literal> them for us. To do this, we need to provide a configuration file. Luckily, it is possible to do this from within Nix! By modifying the above example, we can make Emacs load a custom config file. The key is to create a package that provide a <filename>default.el</filename> file in <filename>/share/emacs/site-start/</filename>. Emacs knows to load this file automatically when it starts.
This provides a fairly full Emacs start file. It will load in addition to the user's presonal config. You can always disable it by passing <command>-q</command> to the Emacs command.
</para>
<para>
Sometimes <varname>emacsWithPackages</varname> is not enough, as this package set has some priorities imposed on packages (with the lowest priority assigned to Melpa Unstable, and the highest for packages manually defined in <filename>pkgs/top-level/emacs-packages.nix</filename>). But you can't control this priorities when some package is installed as a dependency. You can override it on per-package-basis, providing all the required dependencies manually - but it's tedious and there is always a possibility that an unwanted dependency will sneak in through some other package. To completely override such a package you can use <varname>overrideScope'</varname>.
</para>
<screen>
overrides = self: super: rec {
haskell-mode = self.melpaPackages.haskell-mode;
...
};
((emacsPackagesGen emacs).overrideScope' overrides).emacsWithPackages (p: with p; [
# here both these package will use haskell-mode of our own choice
ghc-mod
dante
])
</screen>
</section>
</section>
<sectionxml:id="dlib">
<title>DLib</title>
<para>
<linkxlink:href="http://dlib.net/">DLib</link> is a modern, C++-based toolkit which provides several machine learning algorithms.
</para>
<sectionxml:id="compiling-without-avx-support">
<title>Compiling without AVX support</title>
<para>
Especially older CPUs don't support <linkxlink:href="https://en.wikipedia.org/wiki/Advanced_Vector_Extensions">AVX</link> (<abbrev>Advanced Vector Extensions</abbrev>) instructions that are used by DLib to optimize their algorithms.
</para>
<para>
On the affected hardware errors like <literal>Illegal instruction</literal> will occur. In those cases AVX support needs to be disabled:
All users of Nixpkgs are free software users, and many users (and developers) of Nixpkgs want to limit and tightly control their exposure to unfree software. At the same time, many users need (or want) to run some specific pieces of proprietary software. Nixpkgs includes some expressions for unfree software packages. By default unfree software cannot be installed and doesn’t show up in searches. To allow installing unfree software in a single Nix invocation one can export <literal>NIXPKGS_ALLOW_UNFREE=1</literal>. For a persistent solution, users can set <literal>allowUnfree</literal> in the Nixpkgs configuration.
</para>
<para>
Fine-grained control is possible by defining <literal>allowUnfreePredicate</literal> function in config; it takes the <literal>mkDerivation</literal> parameter attrset and returns <literal>true</literal> for unfree packages that should be allowed.
</para>
</section>
<sectionxml:id="sec-steam">
<title>Steam</title>
<sectionxml:id="sec-steam-nix">
<title>Steam in Nix</title>
<para>
Steam is distributed as a <filename>.deb</filename> file, for now only as an i686 package (the amd64 package only has documentation). When unpacked, it has a script called <filename>steam</filename> that in Ubuntu (their target distro) would go to <filename>/usr/bin </filename>. When run for the first time, this script copies some files to the user's home, which include another script that is the ultimate responsible for launching the steam binary, which is also in $HOME.
</para>
<para>
Nix problems and constraints:
<itemizedlist>
<listitem>
<para>
We don't have <filename>/bin/bash</filename> and many scripts point there. Similarly for <filename>/usr/bin/python</filename> .
</para>
</listitem>
<listitem>
<para>
We don't have the dynamic loader in <filename>/lib </filename>.
</para>
</listitem>
<listitem>
<para>
The <filename>steam.sh</filename> script in $HOME can not be patched, as it is checked and rewritten by steam.
</para>
</listitem>
<listitem>
<para>
The steam binary cannot be patched, it's also checked.
</para>
</listitem>
</itemizedlist>
</para>
<para>
The current approach to deploy Steam in NixOS is composing a FHS-compatible chroot environment, as documented <linkxlink:href="http://sandervanderburg.blogspot.nl/2013/09/composing-fhs-compatible-chroot.html">here</link>. This allows us to have binaries in the expected paths without disrupting the system, and to avoid patching them to work in a non FHS environment.
if you are using PulseAudio - this will enable 32bit ALSA apps integration. To use the Steam controller or other Steam supported controllers such as the DualShock 4 or Nintendo Switch Pro, you need to add
The FHS-compatible chroot used for steam can also be used to run other linux games that expect a FHS environment. To do it, add
<programlisting>pkgs.(steam.override {
nativeOnly = true;
newStdcpp = true;
}).run</programlisting>
to your configuration, rebuild, and run the game with
<programlisting>steam-run ./foo</programlisting>
</para>
</section>
</section>
<sectionxml:id="sec-citrix">
<title>Citrix Receiver & Citrix Workspace App</title>
<para>
<note>
<para>
Please note that the <literal>citrix_receiver</literal> package has been deprecated since its development was <linkxlink:href="https://docs.citrix.com/en-us/citrix-workspace-app.html">discontinued by upstream</link> and has been replaced by <linkxlink:href="https://www.citrix.com/products/workspace-app/">the citrix workspace app</link>.
</para>
</note>
<linkxlink:href="https://www.citrix.com/products/receiver/">Citrix Receiver</link> and <linkxlink:href="https://www.citrix.com/products/workspace-app/">Citrix Workspace App</link> are a remote desktop viewers which provide access to <linkxlink:href="https://www.citrix.com/products/xenapp-xendesktop/">XenDesktop</link> installations.
</para>
<sectionxml:id="sec-citrix-base">
<title>Basic usage</title>
<para>
The tarball archive needs to be downloaded manually as the license agreements of the vendor for <linkxlink:href="https://www.citrix.com/downloads/citrix-receiver/">Citrix Receiver</link> or <linkxlink:href="https://www.citrix.de/downloads/workspace-app/linux/workspace-app-for-linux-latest.html">Citrix Workspace</link> need to be accepted first. Then run <command>nix-prefetch-url file://$PWD/linuxx64-$version.tar.gz</command>. With the archive available in the store the package can be built and installed with Nix.
</para>
<warning>
<title>Caution with <command>nix-shell</command> installs</title>
<para>
It's recommended to install <literal>Citrix Receiver</literal> and/or <literal>Citrix Workspace</literal> using <literal>nix-env -i</literal> or globally to ensure that the <literal>.desktop</literal> files are installed properly into <literal>$XDG_CONFIG_DIRS</literal>. Otherwise it won't be possible to open <literal>.ica</literal> files automatically from the browser to start a Citrix connection.
</para>
</warning>
</section>
<sectionxml:id="sec-citrix-custom-certs">
<title>Custom certificates</title>
<para>
The <literal>Citrix Workspace App</literal> in <literal>nixpkgs</literal> trust several certificates <linkxlink:href="https://curl.haxx.se/docs/caextract.html">from the Mozilla database</link> by default. However several companies using Citrix might require their own corporate certificate. On distros with imperative packaging these certs can be stored easily in <linkxlink:href="https://developer-docs.citrix.com/projects/receiver-for-linux-command-reference/en/13.7/"><literal>$ICAROOT</literal></link>, however this directory is a store path in <literal>nixpkgs</literal>. In order to work around this issue the package provides a simple mechanism to add custom certificates without rebuilding the entire package using <literal>symlinkJoin</literal>:
Some common issues when packaging software for Darwin:
Some common issues when packaging software for darwin:
</para>
<itemizedlist>
<listitem>
<para>
The Darwin <literal>stdenv</literal> uses clang instead of gcc. When referring to the compiler <varname>$CC</varname> or <command>cc</command> will work in both cases. Some builds hardcode gcc/g++ in their build scripts, that can usually be fixed with using something like <literal>makeFlags = [ "CC=cc" ];</literal> or by patching the build scripts.
The darwin <literal>stdenv</literal> uses clang instead of gcc. When
referring to the compiler <varname>$CC</varname> or <command>cc</command>
will work in both cases. Some builds hardcode gcc/g++ in their build
scripts, that can usually be fixed with using something like
<literal>makeFlags = [ "CC=cc" ];</literal> or by patching the build
scripts.
</para>
<programlisting>
stdenv.mkDerivation {
name = "libfoo-1.2.3";
# ...
buildPhase = ''
$CC -o hello hello.c
'';
}
</programlisting>
stdenv.mkDerivation {
name = "libfoo-1.2.3";
# ...
buildPhase = ''
$CC -o hello hello.c
'';
}
</programlisting>
</listitem>
<listitem>
<para>
On Darwin, libraries are linked using absolute paths, libraries are resolved by their <literal>install_name</literal> at link time. Sometimes packages won't set this correctly causing the library lookups to fail at runtime. This can be fixed by adding extra linker flags or by running <command>install_name_tool -id</command> during the <function>fixupPhase</function>.
On darwin libraries are linked using absolute paths, libraries are
resolved by their <literal>install_name</literal> at link time. Sometimes
packages won't set this correctly causing the library lookups to fail at
runtime. This can be fixed by adding extra linker flags or by running
<command>install_name_tool -id</command> during the
Even if the libraries are linked using absolute paths and resolved via their <literal>install_name</literal> correctly, tests can sometimes fail to run binaries. This happens because the <varname>checkPhase</varname> runs before the libraries are installed.
Even if the libraries are linked using absolute paths and resolved via
their <literal>install_name</literal> correctly, tests can sometimes fail
to run binaries. This happens because the <varname>checkPhase</varname>
runs before the libraries are installed.
</para>
<para>
This can usually be solved by running the tests after the<varname>installPhase</varname> or alternatively by using <varname>DYLD_LIBRARY_PATH</varname>. More information about this variable can be found in the <citerefentry>
This can usually be solved by running the tests after the
<varname>installPhase</varname> or alternatively by using
<varname>DYLD_LIBRARY_PATH</varname>. More information about this variable
can be found in the <citerefentry>
<refentrytitle>dyld</refentrytitle>
<manvolnum>1</manvolnum></citerefentry> manpage.
</para>
<programlisting>
dyld: Library not loaded: /nix/store/7hnmbscpayxzxrixrgxvvlifzlxdsdir-jq-1.5-lib/lib/libjq.1.dylib
Some packages assume xcode is available and use <command>xcrun</command> to resolve build tools like <command>clang</command>, etc. This causes errors like <code>xcode-select: error: no developer tools were found at '/Applications/Xcode.app'</code> while the build doesn't actually depend on xcode.
Some packages assume xcode is available and use <command>xcrun</command>
to resolve build tools like <command>clang</command>, etc. This causes
errors like <code>xcode-select: error: no developer tools were found at
'/Applications/Xcode.app'</code> while the build doesn't actually depend
on xcode.
</para>
<programlisting>
stdenv.mkDerivation {
name = "libfoo-1.2.3";
# ...
prePatch = ''
substituteInPlace Makefile \
--replace '/usr/bin/xcrun clang' clang
'';
}
</programlisting>
stdenv.mkDerivation {
name = "libfoo-1.2.3";
# ...
prePatch = ''
substituteInPlace Makefile \
--replace '/usr/bin/xcrun clang' clang
'';
}
</programlisting>
<para>
The package <literal>xcbuild</literal> can be used to build projects that really depend on Xcode. However, this replacement is not 100% compatible with Xcode and can occasionally cause issues.
The package <literal>xcbuild</literal> can be used to build projects that
really depend on Xcode, however projects that build some kind of graphical
interface won't work without using Xcode in an impure way.
Find a good place in the Nixpkgs tree to add the Nix expression for your package. For instance, a library package typically goes into <filename>pkgs/development/libraries/<replaceable>pkgname</replaceable></filename>, while a web browser goes into <filename>pkgs/applications/networking/browsers/<replaceable>pkgname</replaceable></filename>. See <xreflinkend="sec-organisation"/> for some hints on the tree organisation. Create a directory for your package, e.g.
Find a good place in the Nixpkgs tree to add the Nix expression for your
package. For instance, a library package typically goes into
In the package directory, create a Nix expression — a piece of code that describes how to build the package. In this case, it should be a <emphasis>function</emphasis> that is called with the package dependencies as arguments, and returns a build of the package in the Nix store. The expression should usually be called <filename>default.nix</filename>.
In the package directory, create a Nix expression — a piece of code that
describes how to build the package. In this case, it should be a
<emphasis>function</emphasis> that is called with the package dependencies
as arguments, and returns a build of the package in the Nix store. The
expression should usually be called <filename>default.nix</filename>.
You can have a look at the existing Nix expressions under<filename>pkgs/</filename> to see how it’s done. Here are some good ones:
You can have a look at the existing Nix expressions under
<filename>pkgs/</filename> to see how it’s done. Here are some good
ones:
<itemizedlist>
<listitem>
<para>
GNU Hello:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/misc/hello/default.nix"><filename>pkgs/applications/misc/hello/default.nix</filename></link>. Trivial package, which specifies some <varname>meta</varname> attributes which is good practice.
Trivial package, which specifies some <varname>meta</varname>
attributes which is good practice.
</para>
</listitem>
<listitem>
<para>
GNU cpio:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/archivers/cpio/default.nix"><filename>pkgs/tools/archivers/cpio/default.nix</filename></link>. Also a simple package. The generic builder in <varname>stdenv</varname> does everything for you. It has no dependencies beyond <varname>stdenv</varname>.
Also a simple package. The generic builder in <varname>stdenv</varname>
does everything for you. It has no dependencies beyond
<varname>stdenv</varname>.
</para>
</listitem>
<listitem>
<para>
GNU Multiple Precision arithmetic library (GMP):<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/libraries/gmp/5.1.x.nix"><filename>pkgs/development/libraries/gmp/5.1.x.nix</filename></link>. Also done by the generic builder, but has a dependency on <varname>m4</varname>.
Also done by the generic builder, but has a dependency on
<varname>m4</varname>.
</para>
</listitem>
<listitem>
<para>
Pan, a GTK-based newsreader:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/newsreaders/pan/default.nix"><filename>pkgs/applications/networking/newsreaders/pan/default.nix</filename></link>. Has an optional dependency on <varname>gtkspell</varname>, which is only built if <varname>spellCheck</varname> is <literal>true</literal>.
Has an optional dependency on <varname>gtkspell</varname>, which is
only built if <varname>spellCheck</varname> is <literal>true</literal>.
</para>
</listitem>
<listitem>
<para>
Apache HTTPD:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/servers/http/apache-httpd/2.4.nix"><filename>pkgs/servers/http/apache-httpd/2.4.nix</filename></link>. A bunch of optional features, variable substitutions in the configure flags, a post-install hook, and miscellaneous hackery.
A bunch of optional features, variable substitutions in the configure
flags, a post-install hook, and miscellaneous hackery.
</para>
</listitem>
<listitem>
<para>
Thunderbird:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/networking/mailreaders/thunderbird/default.nix"><filename>pkgs/applications/networking/mailreaders/thunderbird/default.nix</filename></link>. Lots of dependencies.
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/misc/jdiskreport/default.nix"><filename>pkgs/tools/misc/jdiskreport/default.nix</filename></link> (and the <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/tools/misc/jdiskreport/builder.sh">builder</link>). Nixpkgs doesn’t have a decent <varname>stdenv</varname> for Java yet so this is pretty ad-hoc.
Nixpkgs doesn’t have a decent <varname>stdenv</varname> for Java yet
so this is pretty ad-hoc.
</para>
</listitem>
<listitem>
<para>
XML::Simple, a Perl module:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/perl-packages.nix"><filename>pkgs/top-level/perl-packages.nix</filename></link> (search for the <varname>XMLSimple</varname> attribute). Most Perl modules are so simple to build that they are defined directly in <filename>perl-packages.nix</filename>; no need to make a separate file for them.
(search for the <varname>XMLSimple</varname> attribute). Most Perl
modules are so simple to build that they are defined directly in
<filename>perl-packages.nix</filename>; no need to make a separate file
for them.
</para>
</listitem>
<listitem>
<para>
Adobe Reader:<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/misc/adobe-reader/default.nix"><filename>pkgs/applications/misc/adobe-reader/default.nix</filename></link>. Shows how binary-only packages can be supported. In particular the <link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/misc/adobe-reader/builder.sh">builder</link> uses <command>patchelf</command> to set the RUNPATH and ELF interpreter of the executables so that the right libraries are found at runtime.
uses <command>patchelf</command> to set the RUNPATH and ELF interpreter
of the executables so that the right libraries are found at runtime.
</para>
</listitem>
</itemizedlist>
@@ -93,59 +138,80 @@
<itemizedlist>
<listitem>
<para>
All <varnamelinkend="chap-meta">meta</varname> attributes are optional, but it’s still a good idea to provide at least the <varname>description</varname>, <varname>homepage</varname> and <varname
All <varnamelinkend="chap-meta">meta</varname> attributes are
optional, but it’s still a good idea to provide at least the
<varname>description</varname>, <varname>homepage</varname> and
<varname
linkend="sec-meta-license">license</varname>.
</para>
</listitem>
<listitem>
<para>
You can use <command>nix-prefetch-url</command><replaceable>url</replaceable> to get the SHA-256 hash of source distributions. There are similar commands as <command>nix-prefetch-git</command> and <command>nix-prefetch-hg</command> available in <literal>nix-prefetch-scripts</literal> package.
You can use <command>nix-prefetch-url</command>(or similar
nix-prefetch-git, etc) <replaceable>url</replaceable> to get the
SHA-256 hash of source distributions. There are similar commands as
<command>nix-prefetch-git</command> and
<command>nix-prefetch-hg</command> available in
<literal>nix-prefetch-scripts</literal> package.
</para>
</listitem>
<listitem>
<para>
A list of schemes for <literal>mirror://</literal> URLs can be found in<link
A list of schemes for <literal>mirror://</literal> URLs can be found in
The exact syntax and semantics of the Nix expression language, including the built-in function, are described in the Nix manual in the <link
xlink:href="http://hydra.nixos.org/job/nix/trunk/tarball/latest/download-by-type/doc/manual/#chap-writing-nix-expressions">chapter on writing Nix expressions</link>.
The exact syntax and semantics of the Nix expression language, including
the built-in function, are described in the Nix manual in the
Add a call to the function defined in the previous step to<link
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/all-packages.nix"><filename>pkgs/top-level/all-packages.nix</filename></link> with some descriptive name for the variable, e.g. <varname>libfoo</varname>.
Add a call to the function defined in the previous step to
The attributes in that file are sorted by category (like “Development / Libraries”) that more-or-less correspond to the directory structure of Nixpkgs, and then by attribute name.
The attributes in that file are sorted by category (like “Development /
Libraries”) that more-or-less correspond to the directory structure of
Nixpkgs, and then by attribute name.
</para>
</listitem>
<listitem>
<para>
To test whether the package builds, run the following command from the root of the nixpkgs source tree:
To test whether the package builds, run the following command from the
root of the nixpkgs source tree:
<screen>
<prompt>$ </prompt>nix-build -A libfoo</screen>
where <varname>libfoo</varname> should be the variable name defined in the previous step. You may want to add the flag <option>-K</option> to keep the temporary build directory in case something fails. If the build succeeds, a symlink <filename>./result</filename> to the package in the Nix store is created.
$ nix-build -A libfoo</screen>
where <varname>libfoo</varname> should be the variable name defined in the
previous step. You may want to add the flag <option>-K</option> to keep
the temporary build directory in case something fails. If the build
succeeds, a symlink <filename>./result</filename> to the package in the
Nix store is created.
</para>
</listitem>
<listitem>
<para>
If you want to install the package into your profile (optional), do
Optionally commit the new package and open a pull request<link
xlink:href="https://github.com/NixOS/nixpkgs/pulls">to nixpkgs</link>, or use <link
xlink:href="https://discourse.nixos.org/t/about-the-patches-category/477"> the Patches category</link> on Discourse for sending a patch without a GitHub account.
Optionally commit the new package and open a pull request, or send a patch
to <literal>https://groups.google.com/forum/#!forum/nix-devel</literal>.
In preparation for the switch from Subversion to Git, this release is mainly the prevent the Nixpkgs version number from going backwards. (This would happen because prerelease version numbers produced for the Git repository are lower than those for the Subversion repository.)
In preparation for the switch from Subversion to Git, this release is mainly
the prevent the Nixpkgs version number from going backwards. (This would
happen because prerelease version numbers produced for the Git repository
are lower than those for the Subversion repository.)
</para>
<para>
Since the last release, there have been thousands of changes and new packages by numerous contributors. For details, see the commit logs.
Since the last release, there have been thousands of changes and new
packages by numerous contributors. For details, see the commit logs.
</para>
</section>
<sectionxml:id="release-notes-0.13">
@@ -51,11 +55,14 @@
<title>Release 0.12 (April 24, 2009)</title>
<para>
There are way too many additions to Nixpkgs since the last release to list here: for example, the number of packages on Linux has increased from 1002 to 2159. However, some specific improvements are worth listing:
There are way too many additions to Nixpkgs since the last release to list
here: for example, the number of packages on Linux has increased from 1002
to 2159. However, some specific improvements are worth listing:
<itemizedlist>
<listitem>
<para>
Nixpkgs now has a manual. In particular, it describes the standard build environment in detail.
Nixpkgs now has a manual. In particular, it describes the standard build
environment in detail.
</para>
</listitem>
<listitem>
@@ -115,7 +122,9 @@
</listitem>
<listitem>
<para>
Support for building derivations in a virtual machine, including RPM and Debian builds in automatically generated VM images. See <filename>pkgs/build-support/vm/default.nix</filename> for details.
Support for building derivations in a virtual machine, including RPM and
Debian builds in automatically generated VM images. See
<filename>pkgs/build-support/vm/default.nix</filename> for details.
</para>
</listitem>
<listitem>
@@ -127,7 +136,13 @@
</para>
<para>
The following people contributed to this release: Andres Löh, Arie Middelkoop, Armijn Hemel, Eelco Dolstra, Lluís Batlle, Ludovic Courtès, Marc Weber, Mart Kolthof, Martin Bravenboer, Michael Raskin, Nicolas Pierron, Peter Simons, Pjotr Prins, Rob Vermaas, Sander van der Burg, Tobias Hammerschmidt, Valentin David, Wouter den Breejen and Yury G. Kudryashov. In addition, several people contributed patches on the <literal>nix-dev</literal> mailing list.
The following people contributed to this release: Andres Löh, Arie
Marc Weber, Mart Kolthof, Martin Bravenboer, Michael Raskin, Nicolas
Pierron, Peter Simons, Pjotr Prins, Rob Vermaas, Sander van der Burg, Tobias
Hammerschmidt, Valentin David, Wouter den Breejen and Yury G. Kudryashov. In
addition, several people contributed patches on the
<literal>nix-dev</literal> mailing list.
</para>
</section>
<sectionxml:id="release-notes-0.11">
@@ -138,12 +153,25 @@
<itemizedlist>
<listitem>
<para>
The standard build environment (<literal>stdenv</literal>) is now pure on the <literal>x86_64-linux</literal> and <literal>powerpc-linux</literal> platforms, just as on <literal>i686-linux</literal>. (Purity means that building and using the standard environment has no dependencies outside of the Nix store. For instance, it doesn’t require an external C compiler such as <filename>/usr/bin/gcc</filename>.) Also, the statically linked binaries used in the bootstrap process are now automatically reproducible, making it easy to update the bootstrap tools and to add support for other Linux platforms. See <filename>pkgs/stdenv/linux/make-bootstrap-tools.nix</filename> for details.
The standard build environment (<literal>stdenv</literal>) is now pure on
the <literal>x86_64-linux</literal> and <literal>powerpc-linux</literal>
platforms, just as on <literal>i686-linux</literal>. (Purity means that
building and using the standard environment has no dependencies outside
of the Nix store. For instance, it doesn’t require an external C
compiler such as <filename>/usr/bin/gcc</filename>.) Also, the statically
linked binaries used in the bootstrap process are now automatically
reproducible, making it easy to update the bootstrap tools and to add
support for other Linux platforms. See
<filename>pkgs/stdenv/linux/make-bootstrap-tools.nix</filename> for
details.
</para>
</listitem>
<listitem>
<para>
Hook variables in the generic builder are now executed using the<function>eval</function> shell command. This has a major advantage: you can write hooks directly in Nix expressions. For instance, rather than writing a builder like this:
Hook variables in the generic builder are now executed using the
<function>eval</function> shell command. This has a major advantage: you
can write hooks directly in Nix expressions. For instance, rather than
writing a builder like this:
<programlisting>
source $stdenv/setup
@@ -154,57 +182,91 @@ postInstall() {
}
genericBuild</programlisting>
(the <literal>gzip</literal> builder), you can just add this attribute to the derivation:
(the <literal>gzip</literal> builder), you can just add this attribute to
and so a separate build script becomes unnecessary. This should allow us to get rid of most builders in Nixpkgs.
and so a separate build script becomes unnecessary. This should allow us
to get rid of most builders in Nixpkgs.
</para>
</listitem>
<listitem>
<para>
It is now possible to have the generic builder pass arguments to<command>configure</command> and <command>make</command> that contain whitespace. Previously, for example, you could say in a builder,
It is now possible to have the generic builder pass arguments to
<command>configure</command> and <command>make</command> that contain
whitespace. Previously, for example, you could say in a builder,
<programlisting>
configureFlags="CFLAGS=-O0"</programlisting>
but not
<programlisting>
configureFlags="CFLAGS=-O0 -g"</programlisting>
since the <literal>-g</literal> would be interpreted as a separate argument to <command>configure</command>. Now you can say
since the <literal>-g</literal> would be interpreted as a separate
argument to <command>configure</command>. Now you can say
which does the right thing. Idem for <literal>makeFlags</literal>,<literal>installFlags</literal>, <literal>checkFlags</literal> and <literal>distFlags</literal>.
which does the right thing. Idem for <literal>makeFlags</literal>,
<literal>installFlags</literal>, <literal>checkFlags</literal> and
<literal>distFlags</literal>.
</para>
<para>
Unfortunately you can't pass arrays to Bash through the environment, so you can't put the array above in a Nix expression, e.g.,
Unfortunately you can't pass arrays to Bash through the environment, so
you can't put the array above in a Nix expression, e.g.,
The function <function>fetchurl</function> now has support for two different kinds of mirroring of files. First, it has support for <emphasis>content-addressable mirrors</emphasis>. For example, given the <function>fetchurl</function> call
The function <function>fetchurl</function> now has support for two
different kinds of mirroring of files. First, it has support for
<emphasis>content-addressable mirrors</emphasis>. For example, given the
<function>fetchurl</function> will first try to download this file from<link
xlink:href="http://tarballs.nixos.org/sha1/eb72f55e4a8bf08e8c6ef227c0ade3d068ba1082"/>. If that file doesn’t exist, it will try the original URL. In general, the “content-addressed” location is <replaceable>mirror</replaceable><literal>/</literal><replaceable>hash-type</replaceable><literal>/</literal><replaceable>hash</replaceable>. There is currently only one content-addressable mirror (<link
xlink:href="http://tarballs.nixos.org"/>), but more can be specified in the <varname>hashedMirrors</varname> attribute in <filename>pkgs/build-support/fetchurl/mirrors.nix</filename>, or by setting the <envar>NIX_HASHED_MIRRORS</envar> environment variable to a whitespace-separated list of URLs.
<function>fetchurl</function> will first try to download this file from
There is currently only one content-addressable mirror
(<link
xlink:href="http://tarballs.nixos.org"/>), but more can be
specified in the <varname>hashedMirrors</varname> attribute in
<filename>pkgs/build-support/fetchurl/mirrors.nix</filename>, or by
setting the <envar>NIX_HASHED_MIRRORS</envar> environment variable to a
whitespace-separated list of URLs.
</para>
<para>
Second, <function>fetchurl</function> has support for widely-mirrored distribution sites such as SourceForge or the Linux kernel archives. Given a URL of the form <literal>mirror://<replaceable>site</replaceable>/<replaceable>path</replaceable></literal>, it will try to download <replaceable>path</replaceable> from a configurable list of mirrors for <replaceable>site</replaceable>. (This idea was borrowed from Gentoo Linux.) Example:
Second, <function>fetchurl</function> has support for widely-mirrored
distribution sites such as SourceForge or the Linux kernel archives.
Currently <replaceable>site</replaceable> can be<literal>sourceforge</literal>, <literal>gnu</literal> and <literal>kernel</literal>. The list of mirrors is defined in <filename>pkgs/build-support/fetchurl/mirrors.nix</filename>. You can override the list of mirrors for a particular site by setting the environment variable <envar>NIX_MIRRORS_<replaceable>site</replaceable></envar>, e.g.
Currently <replaceable>site</replaceable> can be
<literal>sourceforge</literal>, <literal>gnu</literal> and
<literal>kernel</literal>. The list of mirrors is defined in
<filename>pkgs/build-support/fetchurl/mirrors.nix</filename>. You can
override the list of mirrors for a particular site by setting the
environment variable
<envar>NIX_MIRRORS_<replaceable>site</replaceable></envar>, e.g.
The following people contributed to this release: Andres Löh, Arie Middelkoop, Armijn Hemel, Eelco Dolstra, Marc Weber, Mart Kolthof, Martin Bravenboer, Michael Raskin, Wouter den Breejen and Yury G. Kudryashov.
The following people contributed to this release: Andres Löh, Arie
Middelkoop, Armijn Hemel, Eelco Dolstra, Marc Weber, Mart Kolthof, Martin
Bravenboer, Michael Raskin, Wouter den Breejen and Yury G. Kudryashov.
@@ -297,15 +363,32 @@ xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
<itemizedlist>
<listitem>
<para>
<filename>pkgs/system/all-packages-generic.nix</filename> is gone, we now just have <filename>pkgs/top-level/all-packages.nix</filename> that contains all available packages. This should cause much less confusion with users. <filename>all-packages.nix</filename> is a function that by default returns packages for the current platform, but you can override this by specifying a different <varname>system</varname> argument.
<filename>pkgs/system/all-packages-generic.nix</filename> is gone, we now
just have <filename>pkgs/top-level/all-packages.nix</filename> that
contains all available packages. This should cause much less confusion
with users. <filename>all-packages.nix</filename> is a function that by
default returns packages for the current platform, but you can override
this by specifying a different <varname>system</varname> argument.
</para>
</listitem>
<listitem>
<para>
Certain packages in Nixpkgs are now user-configurable through a configuration file, i.e., without having to edit the Nix expressions in Nixpkgs. For instance, the Firefox provided in the Nixpkgs channel is built without the RealPlayer plugin (for legal reasons). Previously, you could easily enable RealPlayer support by editing the call to the Firefox function in <filename>all-packages.nix</filename>, but such changes are not respected when Firefox is subsequently updated through the Nixpkgs channel.
Certain packages in Nixpkgs are now user-configurable through a
configuration file, i.e., without having to edit the Nix expressions in
Nixpkgs. For instance, the Firefox provided in the Nixpkgs channel is
built without the RealPlayer plugin (for legal reasons). Previously, you
could easily enable RealPlayer support by editing the call to the Firefox
function in <filename>all-packages.nix</filename>, but such changes are
not respected when Firefox is subsequently updated through the Nixpkgs
channel.
</para>
<para>
The Nixpkgs configuration file (found in<filename>~/.nixpkgs/config.nix</filename> or through the <envar>NIXPKGS_CONFIG</envar> environment variable) is an attribute set that contains configuration options that <filename>all-packages.nix</filename> reads and uses for certain packages. For instance, the following configuration file:
The Nixpkgs configuration file (found in
<filename>~/.nixpkgs/config.nix</filename> or through the
<envar>NIXPKGS_CONFIG</envar> environment variable) is an attribute set
that contains configuration options that
<filename>all-packages.nix</filename> reads and uses for certain packages.
For instance, the following configuration file:
<programlisting>
{
firefox = {
@@ -315,7 +398,9 @@ xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
persistently enables RealPlayer support in the Firefox build.
</para>
<para>
(Actually, <literal>firefox.enableRealPlayer</literal> is the<emphasis>only</emphasis> configuration option currently available, but more are sure to be added.)
(Actually, <literal>firefox.enableRealPlayer</literal> is the
<emphasis>only</emphasis> configuration option currently available, but
more are sure to be added.)
</para>
</listitem>
<listitem>
@@ -324,8 +409,17 @@ xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
<itemizedlist>
<listitem>
<para>
<literal>i686-cygwin</literal>, i.e., Windows (using<linkxlink:href="http://www.cygwin.com/">Cygwin</link>). The standard environment on <literal>i686-cygwin</literal> by default builds binaries for the Cygwin environment (i.e., it uses Cygwin tools and produces executables that use the Cygwin library). However, there is also a standard environment that produces binaries that use <link
xlink:href="http://www.mingw.org/">MinGW</link>. You can use it by calling <filename>all-package.nix</filename> with the <varname>stdenvType</varname> argument set to <literal>"i686-mingw"</literal>.
<literal>i686-cygwin</literal>, i.e., Windows (using
<linkxlink:href="http://www.cygwin.com/">Cygwin</link>). The standard
environment on <literal>i686-cygwin</literal> by default builds
binaries for the Cygwin environment (i.e., it uses Cygwin tools and
produces executables that use the Cygwin library). However, there is
also a standard environment that produces binaries that use
<link
xlink:href="http://www.mingw.org/">MinGW</link>. You can
use it by calling <filename>all-package.nix</filename> with the
<varname>stdenvType</varname> argument set to
<literal>"i686-mingw"</literal>.
</para>
</listitem>
<listitem>
@@ -340,7 +434,9 @@ xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
</listitem>
<listitem>
<para>
<literal>x86_64-linux</literal>, i.e., Linux on 64-bit AMD/Intel CPUs. Unlike <literal>i686-linux</literal>, this platform doesn’t have a pure <literal>stdenv</literal> yet.
<literal>x86_64-linux</literal>, i.e., Linux on 64-bit AMD/Intel CPUs.
Unlike <literal>i686-linux</literal>, this platform doesn’t have a
pure <literal>stdenv</literal> yet.
</para>
</listitem>
</itemizedlist>
@@ -376,10 +472,21 @@ xlink:href='http://nixos.org/releases/nix/nix-0.10/'>Nix 0.10</link> or higher.
</listitem>
<listitem>
<para>
It is now <emphasis>much</emphasis> easier to override the default C compiler and other tools in <literal>stdenv</literal> for specific packages. <filename>all-packages.nix</filename> provides two utility functions for this purpose: <function>overrideGCC</function> and <function>overrideInStdenv</function>. Both take a <literal>stdenv</literal> and return an augmented <literal>stdenv</literal>; the formed changes the C compiler, and the latter adds additional packages to the front of <literal>stdenv</literal>’s initial <envar>PATH</envar>, allowing tools to be overridden.
It is now <emphasis>much</emphasis> easier to override the default C
compiler and other tools in <literal>stdenv</literal> for specific
packages. <filename>all-packages.nix</filename> provides two utility
functions for this purpose: <function>overrideGCC</function> and
<function>overrideInStdenv</function>. Both take a
<literal>stdenv</literal> and return an augmented
<literal>stdenv</literal>; the formed changes the C compiler, and the
For instance, the package <varname>strategoxt</varname> doesn’t build with the GNU Make in <literal>stdenv</literal> (version 3.81), so we call it with an augmented <literal>stdenv</literal> that uses GNU Make 3.80:
For instance, the package <varname>strategoxt</varname> doesn’t build
with the GNU Make in <literal>stdenv</literal> (version 3.81), so we call
it with an augmented <literal>stdenv</literal> that uses GNU Make 3.80:
It has also become much easier to experiment with changes to the<literal>stdenv</literal> setup script (which notably contains the generic builder). Since edits to <filename>pkgs/stdenv/generic/setup.sh</filename> trigger a rebuild of <emphasis>everything</emphasis>, this was formerly quite painful. But now <literal>stdenv</literal> contains a function to “regenerate” <literal>stdenv</literal> with a different setup script, allowing the use of a different setup script for specific packages:
It has also become much easier to experiment with changes to the
<literal>stdenv</literal> setup script (which notably contains the generic
builder). Since edits to <filename>pkgs/stdenv/generic/setup.sh</filename>
trigger a rebuild of <emphasis>everything</emphasis>, this was formerly
quite painful. But now <literal>stdenv</literal> contains a function to
“regenerate” <literal>stdenv</literal> with a different setup script,
allowing the use of a different setup script for specific packages:
Packages can now have a human-readable <emphasis>description</emphasis> field. Package descriptions are shown by <literal>nix-env -qa --description</literal>. In addition, they’re shown on the Nixpkgs release page. A description can be added to a package as follows:
Packages can now have a human-readable <emphasis>description</emphasis>
field. Package descriptions are shown by <literal>nix-env -qa
--description</literal>. In addition, they’re shown on the Nixpkgs
release page. A description can be added to a package as follows:
<programlisting>
stdenv.mkDerivation {
name = "exult-1.2";
@@ -416,26 +533,34 @@ stdenv.mkDerivation {
description = "A reimplementation of the Ultima VII game engine";
};
}</programlisting>
The <varname>meta</varname> attribute is not passed to the builder, so changes to the description do not trigger a rebuild. Additional <varname>meta</varname> attributes may be defined in the future (such as the URL of the package’s homepage, the license, etc.).
The <varname>meta</varname> attribute is not passed to the builder, so
changes to the description do not trigger a rebuild. Additional
<varname>meta</varname> attributes may be defined in the future (such as
the URL of the package’s homepage, the license, etc.).
</para>
</listitem>
</itemizedlist>
<para>
The following people contributed to this release: Andres Löh, Armijn Hemel, Christof Douma, Eelco Dolstra, Eelco Visser, Mart Kolthof, Martin Bravenboer, Merijn de Jonge, Rob Vermaas and Roy van den Broek.
The following people contributed to this release: Andres Löh, Armijn Hemel,
Christof Douma, Eelco Dolstra, Eelco Visser, Mart Kolthof, Martin
Bravenboer, Merijn de Jonge, Rob Vermaas and Roy van den Broek.
</para>
</section>
<sectionxml:id="release-notes-0.9">
<title>Release 0.9 (January 31, 2006)</title>
<para>
There have been zillions of changes since the last release of Nixpkgs. Many packages have been added or updated. The following are some of the more notable changes:
There have been zillions of changes since the last release of Nixpkgs. Many
packages have been added or updated. The following are some of the more
notable changes:
</para>
<itemizedlist>
<listitem>
<para>
Distribution files have been moved to<link
Distribution files have been moved to
<link
xlink:href="http://nixos.org/"/>.
</para>
</listitem>
@@ -451,17 +576,24 @@ stdenv.mkDerivation {
</listitem>
<listitem>
<para>
The old, unofficial Xlibs has been replaced by the official modularised X11 distribution from X.org, i.e., X11R7.0. X11R7.0 consists of 287 (!) packages, all of which are in Nixpkgs though not all have been tested. It is now possible to build a working X server (previously we only had X client libraries). We use a fully Nixified X server on NixOS.
The old, unofficial Xlibs has been replaced by the official modularised
X11 distribution from X.org, i.e., X11R7.0. X11R7.0 consists of 287 (!)
packages, all of which are in Nixpkgs though not all have been tested. It
is now possible to build a working X server (previously we only had X
client libraries). We use a fully Nixified X server on NixOS.
</para>
</listitem>
<listitem>
<para>
The Sun JDK 5 has been purified, i.e., it doesn’t require any non-Nix components such as <filename>/lib/ld-linux.so.2</filename>. This means that Java applications such as Eclipse and Azureus can run on NixOS.
The Sun JDK 5 has been purified, i.e., it doesn’t require any non-Nix
components such as <filename>/lib/ld-linux.so.2</filename>. This means
that Java applications such as Eclipse and Azureus can run on NixOS.
</para>
</listitem>
<listitem>
<para>
Hardware-accelerated OpenGL support, used by games like Quake 3 (which is now built from source).
Hardware-accelerated OpenGL support, used by games like Quake 3 (which is
now built from source).
</para>
</listitem>
<listitem>
@@ -476,7 +608,8 @@ stdenv.mkDerivation {
</listitem>
<listitem>
<para>
Some support for cross-compilation: cross-compiling builds of GCC and Binutils, and cross-compiled builds of the C library uClibc.
Some support for cross-compilation: cross-compiling builds of GCC and
Binutils, and cross-compiled builds of the C library uClibc.
</para>
</listitem>
<listitem>
@@ -485,7 +618,8 @@ stdenv.mkDerivation {
<itemizedlist>
<listitem>
<para>
teTeX, including support for building LaTeX documents using Nix (with automatic dependency determination).
teTeX, including support for building LaTeX documents using Nix (with
automatic dependency determination).
</para>
</listitem>
<listitem>
@@ -495,12 +629,14 @@ stdenv.mkDerivation {
</listitem>
<listitem>
<para>
System-level packages to support NixOS, e.g. Grub, GNU<literal>parted</literal> and so on.
System-level packages to support NixOS, e.g. Grub, GNU
<literal>parted</literal> and so on.
</para>
</listitem>
<listitem>
<para>
<literal>ecj</literal>, the Eclipse Compiler for Java, so we finally have a freely distributable compiler that supports Java 5.0.
<literal>ecj</literal>, the Eclipse Compiler for Java, so we finally
have a freely distributable compiler that supports Java 5.0.
</para>
</listitem>
<listitem>
@@ -525,7 +661,8 @@ stdenv.mkDerivation {
</listitem>
<listitem>
<para>
<literal>kdelibs</literal>. This allows us to add KDE-based packages (such as <literal>kcachegrind</literal>).
<literal>kdelibs</literal>. This allows us to add KDE-based packages
(such as <literal>kcachegrind</literal>).
</para>
</listitem>
</itemizedlist>
@@ -534,14 +671,17 @@ stdenv.mkDerivation {
</itemizedlist>
<para>
The following people contributed to this release: Andres Löh, Armijn Hemel, Bogdan Dumitriu, Christof Douma, Eelco Dolstra, Eelco Visser, Mart Kolthof, Martin Bravenboer, Rob Vermaas and Roy van den Broek.
The following people contributed to this release: Andres Löh, Armijn Hemel,
Bogdan Dumitriu, Christof Douma, Eelco Dolstra, Eelco Visser, Mart Kolthof,
Martin Bravenboer, Rob Vermaas and Roy van den Broek.
</para>
</section>
<sectionxml:id="release-notes-0.8">
<title>Release 0.8 (April 11, 2005)</title>
<para>
This release is mostly to remain synchronised with the changed hashing scheme in Nix 0.8.
This release is mostly to remain synchronised with the changed hashing
scheme in Nix 0.8.
</para>
<para>
@@ -566,10 +706,16 @@ stdenv.mkDerivation {
<itemizedlist>
<listitem>
<para>
The bootstrap process for the standard build environment on Linux (stdenv-linux) has been improved. It is no longer dependent in its initial bootstrap stages on the system Glibc, GCC, and other tools. Rather, Nixpkgs contains a statically linked bash and curl, and uses that to download other statically linked tools. These are then used to build a Glibc and dynamically linked versions of all other tools.
The bootstrap process for the standard build environment on Linux
(stdenv-linux) has been improved. It is no longer dependent in its initial
bootstrap stages on the system Glibc, GCC, and other tools. Rather,
Nixpkgs contains a statically linked bash and curl, and uses that to
download other statically linked tools. These are then used to build a
Glibc and dynamically linked versions of all other tools.
</para>
<para>
This change also makes the bootstrap process faster. For instance, GCC is built only once instead of three times.
This change also makes the bootstrap process faster. For instance, GCC is
built only once instead of three times.
</para>
<para>
(Contributed by Armijn Hemel.)
@@ -577,13 +723,17 @@ stdenv.mkDerivation {
</listitem>
<listitem>
<para>
Tarballs used by Nixpkgs are now obtained from the same server that hosts Nixpkgs (<link
xlink:href="http://catamaran.labs.cs.uu.nl/"/>). This reduces the risk of packages being unbuildable due to moved or deleted files on various servers.
Tarballs used by Nixpkgs are now obtained from the same server that hosts
Nixpkgs (<link
xlink:href="http://catamaran.labs.cs.uu.nl/"/>). This
reduces the risk of packages being unbuildable due to moved or deleted
files on various servers.
</para>
</listitem>
<listitem>
<para>
There now is a generic mechanism for building Perl modules. See the various Perl modules defined in pkgs/system/all-packages-generic.nix.
There now is a generic mechanism for building Perl modules. See the
various Perl modules defined in pkgs/system/all-packages-generic.nix.
The Nixpkgs project receives a fairly high number of contributions via GitHub pull requests. Reviewing and approving these is an important task and a way to contribute to the project.
The nixpkgs project receives a fairly high number of contributions via GitHub
pull-requests. Reviewing and approving these is an important task and a way
to contribute to the project.
</para>
<para>
The high change rate of Nixpkgs makes any pull request that remains open for too long subject to conflicts that will require extra work from the submitter or the merger. Reviewing pull requests in a timely manner and being responsive to the comments is the key to avoid this issue. GitHub provides sort filters that can be used to see the <link
xlink:href="https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-desc">most recently</link> and the <link
xlink:href="https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+sort%3Aupdated-asc">least recently</link> updated pull requests. We highly encourage looking at <linkxlink:href="https://github.com/NixOS/nixpkgs/pulls?q=is%3Apr+is%3Aopen+review%3Anone+status%3Asuccess+-label%3A%222.status%3A+work-in-progress%22+no%3Aproject+no%3Aassignee+no%3Amilestone"> this list of ready to merge, unreviewed pull requests</link>.
The high change rate of nixpkgs makes any pull request that remains open for
too long subject to conflicts that will require extra work from the submitter
or the merger. Reviewing pull requests in a timely manner and being
responsive to the comments is the key to avoid these. GitHub provides sort
this list of ready to merge, unreviewed pull requests</link>.
</para>
<para>
When reviewing a pull request, please always be nice and polite. Controversial changes can lead to controversial opinions, but it is important to respect every community member and their work.
When reviewing a pull request, please always be nice and polite.
Controversial changes can lead to controversial opinions, but it is important
to respect every community member and their work.
</para>
<para>
GitHub provides reactions as a simple and quick way to provide feedback to pull requests or any comments. The thumb-down reaction should be used with care and if possible accompanied with some explanation so the submitter has directions to improve their contribution.
GitHub provides reactions as a simple and quick way to provide feedback to
pull-requests or any comments. The thumb-down reaction should be used with
care and if possible accompanied with some explanation so the submitter has
directions to improve their contribution.
</para>
<para>
pullrequest reviews should include a list of what has been reviewed in a comment, so other reviewers and mergers can know the state of the review.
Pull-request reviews should include a list of what has been reviewed in a
comment, so other reviewers and mergers can know the state of the review.
</para>
<para>
All the review template samples provided in this section are generic and meant as examples. Their usage is optional and the reviewer is free to adapt them to their liking.
All the review template samples provided in this section are generic and
meant as examples. Their usage is optional and the reviewer is free to adapt
A package update is the most trivial and common type of pullrequest. These pull requests mainly consist of updating the version part of the package name and the source hash.
A package update is the most trivial and common type of pull-request. These
pull-requests mainly consist of updating the version part of the package
name and the source hash.
</para>
<para>
It can happen that non-trivial updates include patches or more complex changes.
It can happen that non-trivial updates include patches or more complex
changes.
</para>
<para>
@@ -49,12 +77,13 @@
<itemizedlist>
<listitem>
<para>
Add labels to the pullrequest. (Requires commit rights)
Add labels to the pull-request. (Requires commit rights)
</para>
<itemizedlist>
<listitem>
<para>
<literal>8.has: package (update)</literal> and any topic label that fit the updated package.
<literal>8.has: package (update)</literal> and any topic label that fit
the updated package.
</para>
</listitem>
</itemizedlist>
@@ -76,7 +105,9 @@
<itemizedlist>
<listitem>
<para>
<linkxlink:href="https://help.github.com/articles/about-codeowners/">CODEOWNERS</link> will make GitHub notify users based on the submitted changes, but it can happen that it misses some of the package maintainers.
will make GitHub notify users based on the submitted changes, but it can
happen that it misses some of the package maintainers.
</para>
</listitem>
</itemizedlist>
@@ -88,12 +119,15 @@
<itemizedlist>
<listitem>
<para>
License can change with version updates, so it should be checked to match the upstream license.
License can change with version updates, so it should be checked to
match the upstream license.
</para>
</listitem>
<listitem>
<para>
If the package has no maintainer, a maintainer must be set. This can be the update submitter or a community member that accepts to take maintainership of the package.
If the package has no maintainer, a maintainer must be set. This can be
the update submitter or a community member that accepts to take
maintainership of the package.
</para>
</listitem>
</itemizedlist>
@@ -110,22 +144,27 @@
<itemizedlist>
<listitem>
<para>
pullrequests are often targeted to the master or staging branch, and building the pull request locally when it is submitted can trigger many source builds.
Pull-requests are often targeted to the master or staging branch, and
building the pull-request locally when it is submitted can trigger many
source builds.
</para>
<para>
It is possible to rebase the changes on nixos-unstable or nixpkgs-unstable for easier review by running the following commands from a nixpkgs clone.
It is possible to rebase the changes on nixos-unstable or
nixpkgs-unstable for easier review by running the following commands
$ git rebase --onto nixos-unstable BASEBRANCH FETCH_HEAD <co
xml:id='reviewing-rebase-4'/>
</screen>
<calloutlist>
<calloutarearefs='reviewing-rebase-1'>
<para>
This should be done only once to be able to fetch channel branches from the nixpkgs-channels repository.
This should be done only once to be able to fetch channel branches
from the nixpkgs-channels repository.
</para>
</callout>
<calloutarearefs='reviewing-rebase-2'>
@@ -135,12 +174,14 @@
</callout>
<calloutarearefs='reviewing-rebase-3'>
<para>
Fetching the pullrequest changes, <varname>PRNUMBER</varname> is the number at the end of the pull request title and <varname>BASEBRANCH</varname> the base branch of the pull request.
Fetching the pull-request changes, <varname>PRNUMBER</varname> is the
number at the end of the pull-request title and
<varname>BASEBRANCH</varname> the base branch of the pull-request.
</para>
</callout>
<calloutarearefs='reviewing-rebase-4'>
<para>
Rebasing the pullrequest changes to the nixos-unstable branch.
Rebasing the pull-request changes to the nixos-unstable branch.
</para>
</callout>
</calloutlist>
@@ -148,10 +189,14 @@
</listitem>
<listitem>
<para>
The <linkxlink:href="https://github.com/Mic92/nix-review">nix-review</link> tool can be used to review a pull request content in a single command. <varname>PRNUMBER</varname> should be replaced by the number at the end of the pull request title. You can also provide the full github pull request url.
The <linkxlink:href="https://github.com/madjar/nox">nox</link> tool can
be used to review a pull-request content in a single command. It doesn't
rebase on a channel branch so it might trigger multiple source builds.
<varname>PRNUMBER</varname> should be replaced by the number at the end
New packages are a common type of pullrequests. These pull requests consists in adding a new nix-expression for a package.
New packages are a common type of pull-requests. These pull requests
consists in adding a new nix-expression for a package.
</para>
<para>
@@ -195,12 +241,13 @@
<itemizedlist>
<listitem>
<para>
Add labels to the pullrequest. (Requires commit rights)
Add labels to the pull-request. (Requires commit rights)
</para>
<itemizedlist>
<listitem>
<para>
<literal>8.has: package (new)</literal> and any topic label that fit the new package.
<literal>8.has: package (new)</literal> and any topic label that fit the
new package.
</para>
</listitem>
</itemizedlist>
@@ -232,7 +279,8 @@
</listitem>
<listitem>
<para>
A maintainer must be set. This can be the package submitter or a community member that accepts to take maintainership of the package.
A maintainer must be set, this can be the package submitter or a
community member that accepts to take maintainership of the package.
</para>
</listitem>
</itemizedlist>
@@ -254,7 +302,8 @@
</listitem>
<listitem>
<para>
The most appropriate function should be used (e.g. packages from GitHub should use <literal>fetchFromGitHub</literal>).
The most appropriate function should be used (e.g. packages from GitHub
should use <literal>fetchFromGitHub</literal>).
</para>
</listitem>
</itemizedlist>
@@ -301,7 +350,8 @@
<title>Module updates</title>
<para>
Module updates are submissions changing modules in some ways. These often contains changes to the options or introduce new options.
Module updates are submissions changing modules in some ways. These often
contains changes to the options or introduce new options.
</para>
<para>
@@ -311,12 +361,13 @@
<itemizedlist>
<listitem>
<para>
Add labels to the pullrequest. (Requires commit rights)
Add labels to the pull-request. (Requires commit rights)
</para>
<itemizedlist>
<listitem>
<para>
<literal>8.has: module (update)</literal> and any topic label that fit the module.
<literal>8.has: module (update)</literal> and any topic label that fit
the module.
</para>
</listitem>
</itemizedlist>
@@ -328,7 +379,9 @@
<itemizedlist>
<listitem>
<para>
<linkxlink:href="https://help.github.com/articles/about-codeowners/">CODEOWNERS</link> will make GitHub notify users based on the submitted changes, but it can happen that it misses some of the package maintainers.
will make GitHub notify users based on the submitted changes, but it can
happen that it misses some of the package maintainers.
</para>
</listitem>
</itemizedlist>
@@ -345,7 +398,9 @@
<itemizedlist>
<listitem>
<para>
Type should be appropriate (string related types differs in their merging capabilities, <literal>optionSet</literal> and <literal>string</literal> types are deprecated).
Type should be appropriate (string related types differs in their
merging capabilities, <literal>optionSet</literal> and
<literal>string</literal> types are deprecated).
</para>
</listitem>
<listitem>
@@ -362,19 +417,23 @@
<itemizedlist>
<listitem>
<para>
<literal>mkRenamedOptionModule</literal> and<literal>mkAliasOptionModule</literal> functions provide way to make option changes backward compatible.
<literal>mkRenamedOptionModule</literal> and
<literal>mkAliasOptionModule</literal> functions provide way to make
option changes backward compatible.
</para>
</listitem>
</itemizedlist>
</listitem>
<listitem>
<para>
Ensure that removed options are declared with<literal>mkRemovedOptionModule</literal>
Ensure that removed options are declared with
<literal>mkRemovedOptionModule</literal>
</para>
</listitem>
<listitem>
<para>
Ensure that changes that are not backward compatible are mentioned in release notes.
Ensure that changes that are not backward compatible are mentioned in
release notes.
</para>
</listitem>
<listitem>
@@ -415,12 +474,13 @@
<itemizedlist>
<listitem>
<para>
Add labels to the pullrequest. (Requires commit rights)
Add labels to the pull-request. (Requires commit rights)
</para>
<itemizedlist>
<listitem>
<para>
<literal>8.has: module (new)</literal> and any topic label that fit the module.
<literal>8.has: module (new)</literal> and any topic label that fit the
module.
</para>
</listitem>
</itemizedlist>
@@ -437,7 +497,9 @@
<itemizedlist>
<listitem>
<para>
Type should be appropriate (string related types differs in their merging capabilities, <literal>optionSet</literal> and <literal>string</literal> types are deprecated).
Type should be appropriate (string related types differs in their
merging capabilities, <literal>optionSet</literal> and
<literal>string</literal> types are deprecated).
</para>
</listitem>
<listitem>
@@ -459,7 +521,8 @@
</listitem>
<listitem>
<para>
Module documentation should be declared with<literal>meta.doc</literal>.
Module documentation should be declared with
<literal>meta.doc</literal>.
</para>
</listitem>
</itemizedlist>
@@ -471,7 +534,8 @@
<itemizedlist>
<listitem>
<para>
For example, enabling a module should not open firewall ports by default.
For example, enabling a module should not open firewall ports by
default.
</para>
</listitem>
</itemizedlist>
@@ -508,18 +572,25 @@
</para>
<para>
If you consider having enough knowledge and experience in a topic and would like to be a long-term reviewer for related submissions, please contact the current reviewers for that topic. They will give you information about the reviewing process. The main reviewers for a topic can be hard to find as there is no list, but checking past pull requests to see who reviewed or git-blaming the code to see who committed to that topic can give some hints.
If you consider having enough knowledge and experience in a topic and would
like to be a long-term reviewer for related submissions, please contact the
current reviewers for that topic. They will give you information about the
reviewing process. The main reviewers for a topic can be hard to find as
there is no list, but checking past pull-requests to see who reviewed or
git-blaming the code to see who committed to that topic can give some hints.
</para>
<para>
Container system, boot system and library changes are some examples of the pull requests fitting this category.
Container system, boot system and library changes are some examples of the
It is possible for community members that have enough knowledge and experience on a special topic to contribute by merging pull requests.
It is possible for community members that have enough knowledge and
experience on a special topic to contribute by merging pull requests.
</para>
<para>
@@ -536,8 +607,12 @@ policy.
-->
<para>
In a case a contributor definitively leaves the Nix community, they should create an issue or post on <link
xlink:href="https://discourse.nixos.org">Discourse</link> with references of packages and modules they maintain so the maintainership can be taken over by other contributors.
In a case a contributor leaves definitively the Nix community, he should
create an issue or post on
<link
xlink:href="https://discourse.nixos.org">Discourse</link> with
references of packages and modules he maintains so the maintainership can be
Read <linkxlink:href="https://nixos.org/nixpkgs/manual/">Manual (How to write packages for Nix)</link>.
Read <linkxlink:href="https://nixos.org/nixpkgs/manual/">Manual (How to
write packages for Nix)</link>.
</para>
</listitem>
<listitem>
@@ -22,17 +23,21 @@
<itemizedlist>
<listitem>
<para>
You can make branch from a commit of your local<command>nixos-version</command>. That will help you to avoid additional local compilations. Because you will receive packages from binary cache.
You can make branch from a commit of your local
<command>nixos-version</command>. That will help you to avoid
additional local compilations. Because you will receive packages from
binary cache.
<itemizedlist>
<listitem>
<para>
For example: <command>nixos-version</command> returns<command>15.05.git.0998212 (Dingo)</command>. So you can do:
For example: <command>nixos-version</command> returns
<command>15.05.git.0998212 (Dingo)</command>. So you can do:
<command>nix-env -i pkg-name -f <path to your local nixpkgs folder></command>
<command>nix-env -i pkg-name -f <path to your local nixpkgs
folder></command>
</para>
</listitem>
</itemizedlist>
@@ -138,11 +149,15 @@ Additional information.
</listitem>
<listitem>
<para>
<emphasis>If you don't want to install pkg in you profile</emphasis>.
<emphasis>If you don't want to install pkg in you
profile</emphasis>.
<itemizedlist>
<listitem>
<para>
<command>nix-build -A pkg-attribute-name <path to your local nixpkgs folder>/default.nix</command> and check results in the folder <command>result</command>. It will appear in the same directory where you did <command>nix-build</command>.
<command>nix-build -A pkg-attribute-name <path to your local
nixpkgs folder>/default.nix</command> and check results in the
folder <command>result</command>. It will appear in the same
directory where you did <command>nix-build</command>.
</para>
</listitem>
</itemizedlist>
@@ -150,7 +165,9 @@ Additional information.
</listitem>
<listitem>
<para>
If you did <command>nix-env -i pkg-name</command> you can do<command>nix-env -e pkg-name</command> to uninstall it from your system.
If you did <command>nix-env -i pkg-name</command> you can do
<command>nix-env -e pkg-name</command> to uninstall it from your
system.
</para>
</listitem>
</itemizedlist>
@@ -162,7 +179,10 @@ Additional information.
<itemizedlist>
<listitem>
<para>
You can add new module to your NixOS configuration file (usually it's <command>/etc/nixos/configuration.nix</command>). And do <command>sudo nixos-rebuild test -I nixpkgs=<path to your local nixpkgs folder> --fast</command>.
You can add new module to your NixOS configuration file (usually
it's <command>/etc/nixos/configuration.nix</command>). And do
<command>sudo nixos-rebuild test -I nixpkgs=<path to your local
nixpkgs folder> --fast</command>.
</para>
</listitem>
</itemizedlist>
@@ -173,7 +193,9 @@ Additional information.
</listitem>
<listitem>
<para>
If you have commits <command>pkg-name: oh, forgot to insert whitespace</command>: squash commits in this case. Use <command>git rebase -i</command>.
If you have commits <command>pkg-name: oh, forgot to insert
whitespace</command>: squash commits in this case. Use <command>git rebase
-i</command>.
</para>
</listitem>
<listitem>
@@ -198,7 +220,8 @@ Additional information.
<itemizedlist>
<listitem>
<para>
Write the title in format <command>(pkg-name | nixos/<module>): improvement</command>.
Write the title in format <command>(pkg-name | nixos/<module>):
improvement</command>.
<itemizedlist>
<listitem>
<para>
@@ -210,7 +233,8 @@ Additional information.
</listitem>
<listitem>
<para>
Write in comment if you have tested your patch. Do not rely much on<command>TravisCI</command>.
Write in comment if you have tested your patch. Do not rely much on
<command>TravisCI</command>.
</para>
</listitem>
<listitem>
@@ -220,7 +244,8 @@ Additional information.
</listitem>
<listitem>
<para>
Notify maintainers of the package. For example add to the message:<command>cc @jagajaga @domenkozar</command>.
Notify maintainers of the package. For example add to the message:
<command>cc @jagajaga @domenkozar</command>.
</para>
</listitem>
</itemizedlist>
@@ -232,39 +257,64 @@ Additional information.
<title>Pull Request Template</title>
<para>
The pull request template helps determine what steps have been made for a contribution so far, and will help guide maintainers on the status of a change. The motivation section of the PR should include any extra details the title does not address and link any existing issues related to the pull request.
The pull request template helps determine what steps have been made for a
contribution so far, and will help guide maintainers on the status of a
change. The motivation section of the PR should include any extra details
the title does not address and link any existing issues related to the pull
request.
</para>
<para>
When a PR is created, it will be pre-populated with some checkboxes detailed below:
When a PR is created, it will be pre-populated with some checkboxes detailed
When sandbox builds are enabled, Nix will setup an isolated environment for each build process. It is used to remove further hidden dependencies set by the build environment to improve reproducibility. This includes access to the network during the build outside of <function>fetch*</function> functions and files outside the Nix store. Depending on the operating system access to other resources are blocked as well (ex. inter process communication is isolated on Linux); see <link
xlink:href="https://nixos.org/nix/manual/#description-45">build-use-sandbox</link> in Nix manual for details.
When sandbox builds are enabled, Nix will setup an isolated environment for
each build process. It is used to remove further hidden dependencies set by
the build environment to improve reproducibility. This includes access to
the network during the build outside of <function>fetch*</function>
functions and files outside the Nix store. Depending on the operating
system access to other resources are blocked as well (ex. inter process
Sandboxing is not enabled by default in Nix due to a small performance hit on each build. In pull requests for <link
xlink:href="https://github.com/NixOS/nixpkgs/">nixpkgs</link> people are asked to test builds with sandboxing enabled (see <literal>Tested using sandboxing</literal> in the pull request template) because in<link
xlink:href="https://nixos.org/hydra/">https://nixos.org/hydra/</link> sandboxing is also used.
Sandboxing is not enabled by default in Nix due to a small performance hit
Depending if you use NixOS or other platforms you can use one of the following methods to enable sandboxing <emphasisrole="bold">before</emphasis> building the package:
Depending if you use NixOS or other platforms you can use one of the
following methods to enable sandboxing
<emphasisrole="bold">before</emphasis> building the package:
<itemizedlist>
<listitem>
<para>
<emphasisrole="bold">Globally enable sandboxing on NixOS</emphasis>: add the following to <filename>configuration.nix</filename>
<emphasisrole="bold">Globally enable sandboxing on NixOS</emphasis>:
add the following to <filename>configuration.nix</filename>
<screen>nix.useSandbox = true;</screen>
</para>
</listitem>
<listitem>
<para>
<emphasisrole="bold">Globally enable sandboxing on non-NixOS platforms</emphasis>: add the following to: <filename>/etc/nix/nix.conf</filename>
<emphasisrole="bold">Globally enable sandboxing on non-NixOS
platforms</emphasis>: add the following to:
<filename>/etc/nix/nix.conf</filename>
<screen>build-use-sandbox = true</screen>
</para>
</listitem>
@@ -276,7 +326,11 @@ Additional information.
<title>Built on platform(s)</title>
<para>
Many Nix packages are designed to run on multiple platforms. As such, it's important to let the maintainer know which platforms your changes have been tested on. It's not always practical to test a change on all platforms, and is not required for a pull request to be merged. Only check the systems you tested the build on in this section.
Many Nix packages are designed to run on multiple platforms. As such, it's
important to let the maintainer know which platforms your changes have been
tested on. It's not always practical to test a change on all platforms, and
is not required for a pull request to be merged. Only check the systems you
tested the build on in this section.
</para>
</section>
@@ -284,26 +338,38 @@ Additional information.
<title>Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)</title>
<para>
Packages with automated tests are much more likely to be merged in a timely fashion because it doesn't require as much manual testing by the maintainer to verify the functionality of the package. If there are existing tests for the package, they should be run to verify your changes do not break the tests. Tests only apply to packages with NixOS modules defined and can only be run on Linux. For more details on writing and running tests, see the <link
xlink:href="https://nixos.org/nixos/manual/index.html#sec-nixos-tests">section in the NixOS manual</link>.
Packages with automated tests are much more likely to be merged in a timely
fashion because it doesn't require as much manual testing by the maintainer
to verify the functionality of the package. If there are existing tests for
the package, they should be run to verify your changes do not break the
tests. Tests only apply to packages with NixOS modules defined and can only
be run on Linux. For more details on writing and running tests, see the
<title>Tested compilation of all pkgs that depend on this change using <command>nix-review</command></title>
<title>Tested compilation of all pkgs that depend on this change using <command>nox-review</command></title>
<para>
If you are updating a package's version, you can use nix-review to make sure all packages that depend on the updated package still compile correctly. The <command>nix-review</command> utility can look for and build all dependencies either based on uncommited changes with the <literal>wip</literal> option or specifying a github pull request number.
<title>Tested execution of all binary files (usually in <filename>./result/bin/</filename>)</title>
<para>
It's important to test any executables generated by a build when you change or create a package in nixpkgs. This can be done by looking in <filename>./result/bin</filename> and running any files in there, or at a minimum, the main executable for the package. For example, if you make a change to <package>texlive</package>, you probably would only check the binaries associated with the change you made rather than testing all of them.
It's important to test any executables generated by a build when you change
or create a package in nixpkgs. This can be done by looking in
<filename>./result/bin</filename> and running any files in there, or at a
minimum, the main executable for the package. For example, if you make a
change to <package>texlive</package>, you probably would only check the
binaries associated with the change you made rather than testing all of
xlink:href="https://github.com/NixOS/nixpkgs/blob/master/.github/CONTRIBUTING.md">CONTRIBUTING.md</link>. The contributing document has detailed information on standards the Nix community has for commit messages, reviews, licensing of contributions you make to the project, etc... Everyone should read and understand the standards the community has for contributing before submitting a pull request.
The contributing document has detailed information on standards the Nix
community has for commit messages, reviews, licensing of contributions you
make to the project, etc... Everyone should read and understand the
standards the community has for contributing before submitting a pull
request.
</para>
</section>
</section>
@@ -358,17 +436,22 @@ Additional information.
<itemizedlist>
<listitem>
<para>
Commits must be sufficiently tested before being merged, both for the master and staging branches.
Commits must be sufficiently tested before being merged, both for the
master and staging branches.
</para>
</listitem>
<listitem>
<para>
Hydra builds for master and staging should not be used as testing platform, it's a build farm for changes that have been already tested.
Hydra builds for master and staging should not be used as testing
platform, it's a build farm for changes that have been already tested.
</para>
</listitem>
<listitem>
<para>
When changing the bootloader installation process, extra care must be taken. Grub installations cannot be rolled back, hence changes may break people's installations forever. For any non-trivial change to the bootloader please file a PR asking for review, especially from @edolstra.
When changing the bootloader installation process, extra care must be
taken. Grub installations cannot be rolled back, hence changes may break
people's installations forever. For any non-trivial change to the
bootloader please file a PR asking for review, especially from @edolstra.
</para>
</listitem>
</itemizedlist>
@@ -391,12 +474,20 @@ Additional information.
<itemizedlist>
<listitem>
<para>
It's only for non-breaking mass-rebuild commits. That means it's not to be used for testing, and changes must have been well tested already. <linkxlink:href="https://web.archive.org/web/20160528180406/http://comments.gmane.org/gmane.linux.distributions.nixos/13447">Read policy here</link>.
It's only for non-breaking mass-rebuild commits. That means it's not to
be used for testing, and changes must have been well tested already.
If the branch is already in a broken state, please refrain from adding extra new breakages. Stabilize it for a few days, merge into master, then resume development on staging. <linkxlink:href="http://hydra.nixos.org/jobset/nixpkgs/staging#tabs-evaluations">Keep an eye on the staging evaluations here</link>. If any fixes for staging happen to be already in master, then master can be merged into staging.
If the branch is already in a broken state, please refrain from adding
extra new breakages. Stabilize it for a few days, merge into master, then
an eye on the staging evaluations here</link>. If any fixes for staging
happen to be already in master, then master can be merged into staging.
</para>
</listitem>
</itemizedlist>
@@ -408,7 +499,10 @@ Additional information.
<itemizedlist>
<listitem>
<para>
If you're cherry-picking a commit to a stable release branch, always use<command>git cherry-pick -xe</command> and ensure the message contains a clear description about why this needs to be included in the stable branch.
If you're cherry-picking a commit to a stable release branch, always use
<command>git cherry-pick -xe</command> and ensure the message contains a
clear description about why this needs to be included in the stable
branch.
</para>
<para>
An example of a cherry-picked commit would look like this:
@@ -421,7 +515,7 @@ The original commit message describing the reason why the world was torn apart.
(cherry picked from commit abcdef)
Reason: I just had a gut feeling that this would also be wanted by people from
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
