Compare commits

..

1 Commits

Author SHA1 Message Date
Weijia Wang
3c80acabe4 24.05 beta release 2024-05-22 18:11:14 +02:00
27065 changed files with 645784 additions and 1240396 deletions

View File

@@ -44,10 +44,6 @@ indent_size = 4
indent_size = 2
indent_style = space
# Match package.json, which are generally pulled from upstream and accept them as they are
[package.json]
indent_style = unset
# Disable file types or individual files
# some of these files may be auto-generated and/or require significant changes

View File

@@ -109,10 +109,6 @@ fb0e5be84331188a69b3edd31679ca6576edb75a
# postgresql: move packages.nix to ext/default.nix
719034f6f6749d624faa28dff259309fc0e3e730
# php ecosystem: reformat with nixfmt-rfc-style
75ae7621330ff8db944ce4dff4374e182d5d151f
c759efa5e7f825913f9a69ef20f025f50f56dc4d
# pkgs/os-specific/bsd: Reformat with nixfmt-rfc-style 2024-03-01
3fe3b055adfc020e6a923c466b6bcd978a13069a
@@ -121,56 +117,3 @@ c759efa5e7f825913f9a69ef20f025f50f56dc4d
# python3Packages: format with nixfmt
59b1aef59071cae6e87859dc65de973d2cc595c0
# treewide description changes (#317959)
bf995e3641950f4183c1dd9010349263dfa0123b
755b915a158c9d588f08e9b08da9f7f3422070cc
f8c4a98e8e138e21353a2c33b90db3359f539b37
# vscode-extensions.*: format with nixfmt (RFC 166)
7bf9febfa6271012b1ef86647a3a06f06875fdcf
# remove uses of mdDoc (#303841)
1a24330f792c8625746d07d842290e6fd95ae6f9
acd0e3898feb321cb9a71a0fd376f1157d0f4553
1b28414d2886c57343864326dbb745a634d3e37d
6afb255d976f85f3359e4929abd6f5149c323a02
# azure-cli: move to by-name, nixfmt #325950
96cd538b68bd1d0a0a37979356d669abbba32ebc
# poptracker: format with nixfmt-rfc-style (#326697)
ff5c8f6cc3d1f2e017e86d50965c14b71f00567b
# mangal: format with nixfmt-rfc-style #328284
3bb5e993cac3a6e1c3056d2bc9bf43eb2c7a5951
# pico-sdk: switch to finalAttrs (#329438)
8946018b0391ae594d167f1e58497b18de068968
# ollama: format with nixfmt-rfc-style (#329353)
bdfde18037f8d9f9b641a4016c8ada4dc4cbf856
# nixos/ollama: format with nixfmt-rfc-style (#329561)
246d1ee533810ac1946d863bbd9de9b525818d56
# steam: cleanup (#216972)
ad815aebfbfe1415ff6436521d545029c803c3fb
# nixos/nvidia: apply nixfmt-rfc-style (#313440)
fbdcdde04a7caa007e825a8b822c75fab9adb2d6
# step-cli: format package.nix with nixfmt (#331629)
fc7a83f8b62e90de5679e993d4d49ca014ea013d
# darwin.stdenv: format with nixfmt-rfc-style (#333962)
93c10ac9e561c6594d3baaeaff2341907390d9b8
# nrr: format with nixfmt-rfc-style (#334578)
cffc27daf06c77c0d76bc35d24b929cb9d68c3c9
# nixos/kanidm: inherit lib, nixfmt
8f18393d380079904d072007fb19dc64baef0a3a
# fetchurl: nixfmt-rfc-style
ce21e97a1f20dee15da85c084f9d1148d84f853b

135
.github/CODEOWNERS vendored
View File

@@ -11,15 +11,11 @@
# This also holds true for GitHub teams. Since almost none of our teams have write
# permissions, you need to list all members of the team with commit access individually.
# CI
# GitHub actions
/.github/workflows @NixOS/Security @Mic92 @zowoq
/.github/workflows/check-nix-format.yml @infinisil
/.github/workflows/nixpkgs-vet.yml @infinisil @philiptaron
/ci @infinisil @philiptaron @NixOS/Security
# Development support
# EditorConfig
/.editorconfig @Mic92 @zowoq
/shell.nix @infinisil @NixOS/Security
# Libraries
/lib @infinisil
@@ -44,12 +40,9 @@
/pkgs/top-level/stage.nix @Ericson2314
/pkgs/top-level/splice.nix @Ericson2314
/pkgs/top-level/release-cross.nix @Ericson2314
/pkgs/top-level/by-name-overlay.nix @infinisil @philiptaron
/pkgs/stdenv @philiptaron
/pkgs/stdenv/generic @Ericson2314
/pkgs/stdenv/generic/check-meta.nix @Ericson2314
/pkgs/stdenv/cross @Ericson2314
/pkgs/build-support @philiptaron
/pkgs/build-support/cc-wrapper @Ericson2314
/pkgs/build-support/bintools-wrapper @Ericson2314
/pkgs/build-support/setup-hooks @Ericson2314
@@ -57,9 +50,15 @@
/pkgs/build-support/setup-hooks/auto-patchelf.py @layus
/pkgs/pkgs-lib @infinisil
## Format generators/serializers
/pkgs/pkgs-lib/formats/libconfig @h7x4
/pkgs/pkgs-lib/formats/libconfig @ckiee @h7x4
/pkgs/pkgs-lib/formats/hocon @h7x4
# pkgs/by-name
/pkgs/test/check-by-name @infinisil
/pkgs/by-name/README.md @infinisil
/pkgs/top-level/by-name-overlay.nix @infinisil
/.github/workflows/check-by-name.yml @infinisil
# Nixpkgs build-support
/pkgs/build-support/writers @lassulus @Profpatsch
@@ -68,11 +67,8 @@
/nixos/lib/make-disk-image.nix @raitobezarius
# Nix, the package manager
# @raitobezarius is not "code owner", but is listed here to be notified of changes
# pertaining to the Nix package manager.
# i.e. no authority over those files.
pkgs/tools/package-management/nix/ @NixOS/nix-team @raitobezarius
nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobezarius
pkgs/tools/package-management/nix/ @raitobezarius @ma27
nixos/modules/installer/tools/nix-fallback-paths.nix @raitobezarius @ma27
# Nixpkgs documentation
/maintainers/scripts/db-to-md.sh @jtojnar @ryantm
@@ -87,7 +83,6 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
/doc/README.md @infinisil
/nixos/README.md @infinisil
/pkgs/README.md @infinisil
/pkgs/by-name/README.md @infinisil
/maintainers/README.md @infinisil
# User-facing development documentation
@@ -98,6 +93,7 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
/nixos/default.nix @infinisil
/nixos/lib/from-env.nix @infinisil
/nixos/lib/eval-config.nix @infinisil
/nixos/modules/system @dasJ
/nixos/modules/system/activation/bootspec.nix @grahamc @cole-h @raitobezarius
/nixos/modules/system/activation/bootspec.cue @grahamc @cole-h @raitobezarius
@@ -107,9 +103,6 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
# NixOS QEMU virtualisation
/nixos/virtualisation/qemu-vm.nix @raitobezarius
# ACME
/nixos/modules/security/acme @arianvp @flokli @aanderse @emilazy # no merge permission: @m1cr0man
# Systemd
/nixos/modules/system/boot/systemd.nix @NixOS/systemd
/nixos/modules/system/boot/systemd @NixOS/systemd
@@ -120,9 +113,8 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
/nixos/modules/system/boot/loader/systemd-boot @JulienMalka
# Images and installer media
/nixos/modules/profiles/installation-device.nix @ElvishJerricco
/nixos/modules/installer/cd-dvd/ @ElvishJerricco
/nixos/modules/installer/sd-card/
/nixos/modules/installer/cd-dvd/ @samueldr
/nixos/modules/installer/sd-card/ @samueldr
# Updaters
## update.nix
@@ -132,25 +124,22 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
/pkgs/common-updater/scripts/update-source-version @jtojnar
# Python-related code and docs
/doc/languages-frameworks/python.section.md @mweinelt @natsukium
/maintainers/scripts/update-python-libraries @natsukium
/pkgs/development/interpreters/python @natsukium
/pkgs/top-level/python-packages.nix @natsukium
/pkgs/top-level/release-python.nix @natsukium
/doc/languages-frameworks/python.section.md @mweinelt
/pkgs/development/interpreters/python/hooks @jonringer
# Haskell
/doc/languages-frameworks/haskell.section.md @sternenseemann @maralorn
/maintainers/scripts/haskell @sternenseemann @maralorn
/pkgs/development/compilers/ghc @sternenseemann @maralorn
/pkgs/development/haskell-modules @sternenseemann @maralorn
/pkgs/test/haskell @sternenseemann @maralorn
/pkgs/top-level/release-haskell.nix @sternenseemann @maralorn
/pkgs/top-level/haskell-packages.nix @sternenseemann @maralorn
/doc/languages-frameworks/haskell.section.md @sternenseemann @maralorn @ncfavier
/maintainers/scripts/haskell @sternenseemann @maralorn @ncfavier
/pkgs/development/compilers/ghc @sternenseemann @maralorn @ncfavier
/pkgs/development/haskell-modules @sternenseemann @maralorn @ncfavier
/pkgs/test/haskell @sternenseemann @maralorn @ncfavier
/pkgs/top-level/release-haskell.nix @sternenseemann @maralorn @ncfavier
/pkgs/top-level/haskell-packages.nix @sternenseemann @maralorn @ncfavier
# Perl
/pkgs/development/interpreters/perl @stigtsp @zakame @marcusramberg
/pkgs/top-level/perl-packages.nix @stigtsp @zakame @marcusramberg
/pkgs/development/perl-modules @stigtsp @zakame @marcusramberg
/pkgs/development/interpreters/perl @stigtsp @zakame @dasJ @marcusramberg
/pkgs/top-level/perl-packages.nix @stigtsp @zakame @dasJ @marcusramberg
/pkgs/development/perl-modules @stigtsp @zakame @dasJ @marcusramberg
# R
/pkgs/applications/science/math/R @jbedo
@@ -163,7 +152,6 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @NixOS/nix-team @raitobeza
# C compilers
/pkgs/development/compilers/gcc
/pkgs/development/compilers/llvm @alyssais @RossComputerGuy
/pkgs/development/compilers/emscripten @raitobezarius
/doc/languages-frameworks/emscripten.section.md @raitobezarius
@@ -231,15 +219,18 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/nixos/modules/services/networking/ntp @thoughtpolice
# Network
/pkgs/tools/networking/octodns @Janik-Haag
/pkgs/tools/networking/kea/default.nix @mweinelt
/pkgs/tools/networking/babeld/default.nix @mweinelt
/nixos/modules/services/networking/babeld.nix @mweinelt
/nixos/modules/services/networking/kea.nix @mweinelt
/nixos/modules/services/networking/knot.nix @mweinelt
nixos/modules/services/networking/networkmanager.nix @Janik-Haag
/nixos/modules/services/monitoring/prometheus/exporters/kea.nix @mweinelt
/nixos/tests/babeld.nix @mweinelt
/nixos/tests/kea.nix @mweinelt
/nixos/tests/knot.nix @mweinelt
/nixos/tests/networking/* @Janik-Haag
# Web servers
/doc/packages/nginx.section.md @raitobezarius
@@ -252,7 +243,6 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
# Idris
/pkgs/development/idris-modules @Infinisil
/pkgs/development/compilers/idris2 @mattpolzin
# Bazel
/pkgs/development/tools/build-managers/bazel @Profpatsch
@@ -268,17 +258,14 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/pkgs/applications/editors/emacs @adisbladis
/pkgs/top-level/emacs-packages.nix @adisbladis
# Kakoune
/pkgs/applications/editors/kakoune @philiptaron
# Neovim
/pkgs/applications/editors/neovim @figsoda @teto
/pkgs/applications/editors/neovim @figsoda @jonringer @teto
# VimPlugins
/pkgs/applications/editors/vim/plugins @figsoda
/pkgs/applications/editors/vim/plugins @figsoda @jonringer
# VsCode Extensions
/pkgs/applications/editors/vscode/extensions
/pkgs/applications/editors/vscode/extensions @jonringer
# PHP interpreter, packages, extensions, tests and documentation
/doc/languages-frameworks/php.section.md @aanderse @drupol @globin @ma27 @talyz
@@ -298,9 +285,9 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/pkgs/applications/blockchains @mmahut @RaghavSood
# Go
/doc/languages-frameworks/go.section.md @kalbasit @katexochen @Mic92 @zowoq
/pkgs/build-support/go @kalbasit @katexochen @Mic92 @zowoq
/pkgs/development/compilers/go @kalbasit @katexochen @Mic92 @zowoq
/doc/languages-frameworks/go.section.md @kalbasit @Mic92 @zowoq
/pkgs/build-support/go @kalbasit @Mic92 @zowoq
/pkgs/development/compilers/go @kalbasit @Mic92 @zowoq
# GNOME
/pkgs/desktops/gnome @jtojnar
@@ -308,11 +295,7 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/pkgs/build-support/make-hardcode-gsettings-patch @jtojnar
# Cinnamon
/pkgs/by-name/ci/cinnamon-* @mkg20001
/pkgs/by-name/cj/cjs @mkg20001
/pkgs/by-name/mu/muffin @mkg20001
/pkgs/by-name/ne/nemo @mkg20001
/pkgs/by-name/ne/nemo-* @mkg20001
/pkgs/desktops/cinnamon @mkg20001
# nim
/pkgs/development/compilers/nim @ehmry
@@ -323,19 +306,22 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/pkgs/applications/networking/cluster/terraform-providers @zowoq
# Forgejo
nixos/modules/services/misc/forgejo.nix @adamcstephens @bendlas @emilylange
pkgs/by-name/fo/forgejo/ @adamcstephens @bendlas @emilylange
nixos/modules/services/misc/forgejo.nix @bendlas @emilylange
pkgs/applications/version-management/forgejo @bendlas @emilylange
# Dotnet
/pkgs/build-support/dotnet @corngood
/pkgs/development/compilers/dotnet @corngood
/pkgs/test/dotnet @corngood
/doc/languages-frameworks/dotnet.section.md @corngood
/pkgs/build-support/dotnet @IvarWithoutBones
/pkgs/development/compilers/dotnet @IvarWithoutBones
/pkgs/test/dotnet @IvarWithoutBones
/doc/languages-frameworks/dotnet.section.md @IvarWithoutBones
# Node.js
/pkgs/build-support/node/build-npm-package @winterqt
/pkgs/build-support/node/fetch-npm-deps @winterqt
/doc/languages-frameworks/javascript.section.md @winterqt
/pkgs/build-support/node/build-npm-package @lilyinstarlight @winterqt
/pkgs/build-support/node/fetch-npm-deps @lilyinstarlight @winterqt
/doc/languages-frameworks/javascript.section.md @lilyinstarlight @winterqt
# environment.noXlibs option aka NoX
/nixos/modules/config/no-x-libs.nix @SuperSandro2000
# OCaml
/pkgs/build-support/ocaml @ulrikstrid
@@ -365,28 +351,15 @@ nixos/modules/services/web-apps/pretalx.nix @mweinelt
nixos/tests/web-apps/pretix.nix @mweinelt
nixos/tests/web-apps/pretalx.nix @mweinelt
# incus/lxc
nixos/maintainers/scripts/incus/ @adamcstephens
# incus/lxc/lxd
nixos/maintainers/scripts/lxd/ @adamcstephens
nixos/modules/virtualisation/incus.nix @adamcstephens
nixos/modules/virtualisation/lxc* @adamcstephens
nixos/modules/virtualisation/lxd* @adamcstephens
nixos/tests/incus/ @adamcstephens
nixos/tests/lxd/ @adamcstephens
pkgs/by-name/in/incus/ @adamcstephens
pkgs/by-name/lx/lxc* @adamcstephens
pkgs/by-name/lx/lxd* @adamcstephens
pkgs/os-specific/linux/lxc/ @adamcstephens
# ExpidusOS, Flutter
/pkgs/development/compilers/flutter @RossComputerGuy
/pkgs/desktops/expidus @RossComputerGuy
# GNU Tar & Zip
/pkgs/tools/archivers/gnutar @RossComputerGuy
/pkgs/tools/archivers/zip @RossComputerGuy
# SELinux
/pkgs/os-specific/linux/checkpolicy @RossComputerGuy
/pkgs/os-specific/linux/libselinux @RossComputerGuy
/pkgs/os-specific/linux/libsepol @RossComputerGuy
# installShellFiles
/pkgs/by-name/in/installShellFiles/* @Ericson2314
/pkgs/test/install-shell-files/* @Ericson2314
/doc/hooks/installShellFiles.section.md @Ericson2314

View File

@@ -21,10 +21,10 @@ For new packages please briefly describe the package or provide a link to its ho
- [NixOS test(s)](https://nixos.org/manual/nixos/unstable/index.html#sec-nixos-tests) (look inside [nixos/tests](https://github.com/NixOS/nixpkgs/blob/master/nixos/tests))
- and/or [package tests](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests)
- or, for functions and "core" functionality, tests in [lib/tests](https://github.com/NixOS/nixpkgs/blob/master/lib/tests) or [pkgs/test](https://github.com/NixOS/nixpkgs/blob/master/pkgs/test)
- made sure NixOS tests are [linked](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#linking-nixos-module-tests-to-a-package) to the relevant packages
- made sure NixOS tests are [linked](https://nixos.org/manual/nixpkgs/unstable/#ssec-nixos-tests-linking) to the relevant packages
- [ ] Tested compilation of all packages that depend on this change using `nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD"`. Note: all changes have to be committed, also see [nixpkgs-review usage](https://github.com/Mic92/nixpkgs-review#usage)
- [ ] Tested basic functionality of all binary files (usually in `./result/bin/`)
- [24.11 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2411.section.md) (or backporting [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) and [24.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) Release notes)
- [24.05 Release Notes](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2405.section.md) (or backporting [23.05](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2305.section.md) and [23.11](https://github.com/NixOS/nixpkgs/blob/master/nixos/doc/manual/release-notes/rl-2311.section.md) Release notes)
- [ ] (Package updates) Added a release notes entry if the change is major or breaking
- [ ] (Module updates) Added a release notes entry if the change is significant
- [ ] (Module addition) Added a release notes entry if adding a new NixOS module

52
.github/labeler.yml vendored
View File

@@ -12,25 +12,9 @@
- any:
- changed-files:
- any-glob-to-any-file:
- pkgs/desktops/cinnamon/**/*
- nixos/modules/services/x11/desktop-managers/cinnamon.nix
- nixos/tests/cinnamon.nix
- nixos/tests/cinnamon-wayland.nix
- pkgs/by-name/ci/cinnamon-*/**/*
- pkgs/by-name/cj/cjs/**/*
- pkgs/by-name/mu/muffin/**/*
- pkgs/by-name/ne/nemo/**/*
- pkgs/by-name/ne/nemo-*/**/*
"6.topic: dotnet":
- any:
- changed-files:
- any-glob-to-any-file:
- doc/languages-frameworks/dotnet.section.md
- maintainers/scripts/update-dotnet-lockfiles.nix
- pkgs/build-support/dotnet/**/*
- pkgs/development/compilers/dotnet/**/*
- pkgs/test/dotnet/**/*
- pkgs/top-level/dotnet-packages.nix
"6.topic: emacs":
- any:
@@ -39,9 +23,9 @@
- nixos/modules/services/editors/emacs.nix
- nixos/modules/services/editors/emacs.xml
- nixos/tests/emacs-daemon.nix
- pkgs/applications/editors/emacs/build-support/**/*
- pkgs/applications/editors/emacs/elisp-packages/**/*
- pkgs/applications/editors/emacs/**/*
- pkgs/build-support/emacs/**/*
- pkgs/top-level/emacs-packages.nix
"6.topic: Enlightenment DE":
@@ -79,13 +63,6 @@
- lib/systems/flake-systems.nix
- nixos/modules/config/nix-flakes.nix
"6.topic: flutter":
- any:
- changed-files:
- any-glob-to-any-file:
- pkgs/build-support/flutter/*.nix
- pkgs/development/compilers/flutter/**/*.nix
"6.topic: GNOME":
- any:
- changed-files:
@@ -157,12 +134,6 @@
- any-glob-to-any-file:
- lib/**
"6.topic: llvm/clang":
- any:
- changed-files:
- any-glob-to-any-file:
- pkgs/development/compilers/llvm/**/*
"6.topic: lua":
- any:
- changed-files:
@@ -230,7 +201,6 @@
- pkgs/development/node-packages/**/*
- pkgs/development/tools/yarn/*
- pkgs/development/tools/yarn2nix-moretea/**/*
- pkgs/development/tools/pnpm/**/*
- pkgs/development/web/nodejs/*
"6.topic: ocaml":
@@ -270,12 +240,6 @@
- changed-files:
- any-glob-to-any-file:
- .github/**/*
- CONTRIBUTING.md
- pkgs/README.md
- nixos/README.md
- maintainers/README.md
- lib/README.md
- doc/README.md
"6.topic: printing":
- any:
@@ -356,7 +320,6 @@
# *developed in this repo*;
# - not individual tests
# - not packages for test frameworks
- pkgs/build-support/testers/**
- nixos/lib/testing/**
- nixos/lib/test-driver/**
- nixos/tests/nixos-test-driver/**
@@ -380,17 +343,6 @@
- any-glob-to-any-file:
- pkgs/applications/editors/vscode/**/*
"6.topic: xen-project":
- any:
- changed-files:
- any-glob-to-any-file:
- nixos/modules/virtualisation/xen*
- pkgs/applications/virtualization/xen/**
- pkgs/by-name/xe/xen-guest-agent/*
- pkgs/by-name/xt/xtf/*
- pkgs/development/ocaml-modules/xen*/*
- pkgs/development/ocaml-modules/vchan/*
"6.topic: xfce":
- any:
- changed-files:

View File

@@ -20,11 +20,11 @@ jobs:
if: github.repository_owner == 'NixOS' && github.event.pull_request.merged == true && (github.event_name != 'labeled' || startsWith('backport', github.event.label.name))
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
ref: ${{ github.event.pull_request.head.sha }}
- name: Create backport PRs
uses: korthout/backport-action@bd410d37cdcae80be6d969823ff5a225fe5c833f # v3.0.2
uses: korthout/backport-action@ef20d86abccbac3ee3a73cb2efbdc06344c390e5 # v2.5.0
with:
# Config README: https://github.com/korthout/backport-action#backport-action
copy_labels_pattern: 'severity:\ssecurity'

View File

@@ -15,13 +15,12 @@ permissions:
jobs:
tests:
name: basic-eval-checks
runs-on: ubuntu-latest
# we don't limit this action to only NixOS repo since the checks are cheap and useful developer feedback
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
- uses: cachix/cachix-action@18cf96c7c98e048e10a83abd92116114cd8504be # v14
with:
# This cache is for the nixpkgs repo checks and should not be trusted or used elsewhere.
name: nixpkgs-ci

123
.github/workflows/check-by-name.yml vendored Normal file
View File

@@ -0,0 +1,123 @@
# Checks pkgs/by-name (see pkgs/by-name/README.md)
# using the nixpkgs-check-by-name tool (see https://github.com/NixOS/nixpkgs-check-by-name)
#
# When you make changes to this workflow, also update pkgs/test/check-by-name/run-local.sh adequately
name: Check pkgs/by-name
on:
# Using pull_request_target instead of pull_request avoids having to approve first time contributors
pull_request_target:
# This workflow depends on the base branch of the PR,
# but changing the base branch is not included in the default trigger events,
# which would be `opened`, `synchronize` or `reopened`.
# Instead it causes an `edited` event, so we need to add it explicitly here
# While `edited` is also triggered when the PR title/body is changed,
# this PR action is fairly quick, and PR's don't get edited that often,
# so it shouldn't be a problem
types: [opened, synchronize, reopened, edited]
permissions: {}
# Create a check-by-name concurrency group based on the pull request number. if
# an event triggers a run on the same PR while a previous run is still in
# progress, the previous run will be canceled and the new one will start.
concurrency:
group: check-by-name-${{ github.event.pull_request.number }}
cancel-in-progress: true
jobs:
check:
# This needs to be x86_64-linux, because we depend on the tooling being pre-built in the GitHub releases
runs-on: ubuntu-latest
# This should take 1 minute at most, but let's be generous.
# The default of 6 hours is definitely too long
timeout-minutes: 10
steps:
# This step has to be in this file,
# because it's needed to determine which revision of the repository to fetch,
# and we can only use other files from the repository once it's fetched.
- name: Resolving the merge commit
env:
GH_TOKEN: ${{ github.token }}
run: |
# This checks for mergeability of a pull request as recommended in
# https://docs.github.com/en/rest/guides/using-the-rest-api-to-interact-with-your-git-database?apiVersion=2022-11-28#checking-mergeability-of-pull-requests
# Retry the API query this many times
retryCount=5
# Start with 5 seconds, but double every retry
retryInterval=5
while true; do
echo "Checking whether the pull request can be merged"
prInfo=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/"$GITHUB_REPOSITORY"/pulls/${{ github.event.pull_request.number }})
mergeable=$(jq -r .mergeable <<< "$prInfo")
mergedSha=$(jq -r .merge_commit_sha <<< "$prInfo")
if [[ "$mergeable" == "null" ]]; then
if (( retryCount == 0 )); then
echo "Not retrying anymore, probably GitHub is having internal issues"
exit 1
else
(( retryCount -= 1 )) || true
# null indicates that GitHub is still computing whether it's mergeable
# Wait a couple seconds before trying again
echo "GitHub is still computing whether this PR can be merged, waiting $retryInterval seconds before trying again ($retryCount retries left)"
sleep "$retryInterval"
(( retryInterval *= 2 )) || true
fi
else
break
fi
done
if [[ "$mergeable" == "true" ]]; then
echo "The PR can be merged, checking the merge commit $mergedSha"
echo "mergedSha=$mergedSha" >> "$GITHUB_ENV"
else
echo "The PR cannot be merged, it has a merge conflict, skipping the rest.."
fi
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
if: env.mergedSha
with:
# pull_request_target checks out the base branch by default
ref: ${{ env.mergedSha }}
# Fetches the merge commit and its parents
fetch-depth: 2
- name: Checking out base branch
if: env.mergedSha
run: |
base=$(mktemp -d)
git worktree add "$base" "$(git rev-parse HEAD^1)"
echo "base=$base" >> "$GITHUB_ENV"
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
if: env.mergedSha
- name: Fetching the pinned tool
if: env.mergedSha
# Update the pinned version using pkgs/test/check-by-name/update-pinned-tool.sh
run: |
# The pinned version of the tooling to use
toolVersion=$(<pkgs/test/check-by-name/pinned-version.txt)
# Fetch the x86_64-linux-specific release artifact containing the Gzipped NAR of the pre-built tool
toolPath=$(curl -sSfL https://github.com/NixOS/nixpkgs-check-by-name/releases/download/"$toolVersion"/x86_64-linux.nar.gz \
| gzip -cd | nix-store --import | tail -1)
# Adds a result symlink as a GC root
nix-store --realise "$toolPath" --add-root result
- name: Running nixpkgs-check-by-name
if: env.mergedSha
env:
# Force terminal colors to be enabled. The library that
# nixpkgs-check-by-name uses respects: https://bixense.com/clicolors/
CLICOLOR_FORCE: 1
run: |
if result/bin/nixpkgs-check-by-name --base "$base" .; then
exit 0
else
exitCode=$?
echo "To run locally: ./maintainers/scripts/check-by-name.sh $GITHUB_BASE_REF https://github.com/$GITHUB_REPOSITORY.git"
exit "$exitCode"
fi

View File

@@ -4,17 +4,15 @@ on:
branches:
- 'release-**'
- 'staging-**'
- '!staging-next'
permissions: {}
jobs:
check:
name: cherry-pick-check
runs-on: ubuntu-latest
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
fetch-depth: 0
filter: blob:none

View File

@@ -9,11 +9,10 @@ permissions:
jobs:
nixos:
name: maintainer-list-check
runs-on: ubuntu-latest
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
@@ -21,7 +20,7 @@ jobs:
sparse-checkout: |
lib
maintainers
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true

View File

@@ -7,84 +7,56 @@ name: Check that Nix files are formatted
on:
pull_request_target:
# See the comment at the same location in ./nixpkgs-vet.yml
types: [opened, synchronize, reopened, edited]
permissions:
contents: read
jobs:
nixos:
name: nixfmt-check
runs-on: ubuntu-latest
if: "!contains(github.event.pull_request.title, '[skip treewide]')"
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
# Fetches the merge commit and its parents
fetch-depth: 2
- name: Checking out base branch
run: |
base=$(mktemp -d)
baseRev=$(git rev-parse HEAD^1)
git worktree add "$base" "$baseRev"
echo "baseRev=$baseRev" >> "$GITHUB_ENV"
echo "base=$base" >> "$GITHUB_ENV"
- name: Get Nixpkgs revision for nixfmt
run: |
# pin to a commit from nixpkgs-unstable to avoid e.g. building nixfmt
# from staging
# This should not be a URL, because it would allow PRs to run arbitrary code in CI!
rev=$(jq -r .rev ci/pinned-nixpkgs.json)
echo "url=https://github.com/NixOS/nixpkgs/archive/$rev.tar.gz" >> "$GITHUB_ENV"
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true
nix_path: nixpkgs=${{ env.url }}
# fix a commit from nixpkgs-unstable to avoid e.g. building nixfmt
# from staging
nix_path: nixpkgs=https://github.com/NixOS/nixpkgs/archive/4b455dc2048f73a79eb3713f342369ff58f93e0b.tar.gz
- name: Install nixfmt
run: "nix-env -f '<nixpkgs>' -iAP nixfmt-rfc-style"
- name: Check that Nix files are formatted according to the RFC style
# Each environment variable beginning with NIX_FMT_PATHS_ is a list of
# paths to check with nixfmt.
env:
NIX_FMT_PATHS_BSD: pkgs/os-specific/bsd
NIX_FMT_PATHS_MPVSCRIPTS: pkgs/applications/video/mpv/scripts
# Format paths related to the Nixpkgs CUDA ecosystem.
NIX_FMT_PATHS_CUDA: |
pkgs/development/cuda-modules
pkgs/test/cuda
pkgs/top-level/cuda-packages.nix
NIX_FMT_PATHS_K3S: |
nixos/modules/services/cluster/k3s
nixos/tests/k3s
pkgs/applications/networking/cluster/k3s
NIX_FMT_PATHS_VSCODE_EXTS: pkgs/applications/editors/vscode/extensions
NIX_FMT_PATHS_PHP_PACKAGES: pkgs/development/php-packages
NIX_FMT_PATHS_BUILD_SUPPORT_PHP: pkgs/build-support/php
# Iterate over all environment variables beginning with NIX_FMT_PATHS_.
run: |
unformattedFiles=()
# TODO: Make this more parallel
# Loop through all Nix files touched by the PR
while readarray -d '' -n 2 entry && (( ${#entry[@]} != 0 )); do
type=${entry[0]}
file=${entry[1]}
case $type in
A*)
source=""
dest=$file
;;
M*)
source=$file
dest=$file
;;
C*|R*)
source=$file
read -r -d '' dest
;;
*)
echo "Ignoring file $file with type $type"
continue
esac
# Ignore files that weren't already formatted
if [[ -n "$source" ]] && ! nixfmt --check ${{ env.base }}/"$source" 2>/dev/null; then
echo "Ignoring file $file because it's not formatted in the base commit"
elif ! nixfmt --check "$dest"; then
unformattedFiles+=("$dest")
for env_var in "${!NIX_FMT_PATHS_@}"; do
readarray -t paths <<< "${!env_var}"
if [[ "${paths[*]}" == "" ]]; then
echo "Error: $env_var is empty."
exit 1
fi
done < <(git diff -z --name-status ${{ env.baseRev }} -- '*.nix')
if (( "${#unformattedFiles[@]}" > 0 )); then
echo "Some new/changed Nix files are not properly formatted"
echo "Please go to the Nixpkgs root directory, run \`nix-shell\`, then:"
echo "nixfmt ${unformattedFiles[*]@Q}"
echo "If you're having trouble, please ping @NixOS/nix-formatting"
exit 1
fi
echo "Checking paths: ${paths[@]}"
if ! nixfmt --check "${paths[@]}"; then
echo "Error: nixfmt failed."
exit 1
fi
done

View File

@@ -1,129 +0,0 @@
name: Check changed Nix files with nixf-tidy (experimental)
on:
pull_request_target:
types: [opened, synchronize, reopened, edited]
permissions:
contents: read
jobs:
nixos:
name: exp-nixf-tidy-check
runs-on: ubuntu-latest
if: "!contains(github.event.pull_request.title, '[skip treewide]')"
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
# Fetches the merge commit and its parents
fetch-depth: 2
- name: Checking out base branch
run: |
base=$(mktemp -d)
baseRev=$(git rev-parse HEAD^1)
git worktree add "$base" "$baseRev"
echo "baseRev=$baseRev" >> "$GITHUB_ENV"
echo "base=$base" >> "$GITHUB_ENV"
- name: Get Nixpkgs revision for nixf
run: |
# pin to a commit from nixpkgs-unstable to avoid e.g. building nixf
# from staging
# This should not be a URL, because it would allow PRs to run arbitrary code in CI!
rev=$(jq -r .rev ci/pinned-nixpkgs.json)
echo "url=https://github.com/NixOS/nixpkgs/archive/$rev.tar.gz" >> "$GITHUB_ENV"
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true
nix_path: nixpkgs=${{ env.url }}
- name: Install nixf and jq
# provided jq is incompatible with our expression
run: "nix-env -f '<nixpkgs>' -iAP nixf jq"
- name: Check that Nix files pass nixf-tidy
run: |
# Filtering error messages we don't like
nixf_wrapper(){
nixf-tidy --variable-lookup < "$1" | jq -r '
[
"sema-escaping-with"
]
as $ignored_errors|[.[]|select(.sname as $s|$ignored_errors|index($s)|not)]
'
}
failedFiles=()
# Don't report errors to file overview
# to avoid duplicates when editing title and description
if [[ "${{ github.event.action }}" == 'edited' ]] && [[ -z "${{ github.event.edited.changes.base }}" ]]; then
DONT_REPORT_ERROR=1
else
DONT_REPORT_ERROR=
fi
# TODO: Make this more parallel
# Loop through all Nix files touched by the PR
while readarray -d '' -n 2 entry && (( ${#entry[@]} != 0 )); do
type=${entry[0]}
file=${entry[1]}
case $type in
A*)
source=""
dest=$file
;;
M*)
source=$file
dest=$file
;;
C*|R*)
source=$file
read -r -d '' dest
;;
*)
echo "Ignoring file $file with type $type"
continue
esac
if [[ -n "$source" ]] && [[ "$(nixf_wrapper ${{ env.base }}/"$source")" != '[]' ]] 2>/dev/null; then
echo "Ignoring file $file because it doesn't pass nixf-tidy in the base commit"
echo # insert blank line
else
nixf_report="$(nixf_wrapper "$dest")"
if [[ "$nixf_report" != '[]' ]]; then
echo "$dest doesn't pass nixf-tidy. Reported by nixf-tidy:"
errors=$(echo "$nixf_report" | jq -r --arg dest "$dest" '
def getLCur: "line=" + (.line+1|tostring) + ",col=" + (.column|tostring);
def getRCur: "endLine=" + (.line+1|tostring) + ",endColumn=" + (.column|tostring);
def getRange: "file=\($dest)," + (.lCur|getLCur) + "," + (.rCur|getRCur);
def getBody: . as $top|(.range|getRange) + ",title="+ .sname + "::" +
(.message|sub("{}" ; ($top.args.[]|tostring)));
def getNote: "\n::notice " + (.|getBody);
def getMessage: "::error " + (.|getBody) + (if (.notes|length)>0 then
([.notes.[]|getNote]|add) else "" end);
.[]|getMessage
')
if [[ -z "$DONT_REPORT_ERROR" ]]; then
echo "$errors"
else
# just print in plain text
echo "$errors" | sed 's/^:://'
echo # add one empty line
fi
failedFiles+=("$dest")
fi
fi
done < <(git diff -z --name-status ${{ env.baseRev }} -- '*.nix')
if [[ -n "$DONT_REPORT_ERROR" ]]; then
echo "Edited the PR but didn't change the base branch, only the description/title."
echo "Not reporting errors again to avoid duplication."
echo # add one empty line
fi
if (( "${#failedFiles[@]}" > 0 )); then
echo "Some new/changed Nix files don't pass nixf-tidy."
echo "See ${{ github.event.pull_request.html_url }}/files for reported errors."
echo "If you believe this is a false positive, ping @Aleksanaa and @inclyc in this PR."
exit 1
fi

View File

@@ -1,31 +0,0 @@
name: "Check shell"
on:
pull_request_target:
permissions: {}
jobs:
x86_64-linux:
name: shell-check-x86_64-linux
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- name: Build shell
run: nix-build shell.nix
aarch64-darwin:
name: shell-check-aarch64-darwin
runs-on: macos-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- name: Build shell
run: nix-build shell.nix

View File

@@ -10,7 +10,6 @@ on:
jobs:
tests:
name: editorconfig-check
runs-on: ubuntu-latest
if: "github.repository_owner == 'NixOS' && !contains(github.event.pull_request.title, '[skip treewide]')"
steps:
@@ -25,11 +24,11 @@ jobs:
- name: print list of changed files
run: |
cat "$HOME/changed_files"
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
# nixpkgs commit is pinned so that it doesn't break
# editorconfig-checker 2.4.0

View File

@@ -15,7 +15,6 @@ permissions:
jobs:
labels:
name: label-pr
runs-on: ubuntu-latest
if: "github.repository_owner == 'NixOS' && !contains(github.event.pull_request.title, '[skip treewide]')"
steps:

View File

@@ -11,19 +11,18 @@ on:
jobs:
nixos:
name: nixos-manual-build
runs-on: ubuntu-latest
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true
- uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
- uses: cachix/cachix-action@18cf96c7c98e048e10a83abd92116114cd8504be # v14
with:
# This cache is for the nixpkgs repo checks and should not be trusted or used elsewhere.
name: nixpkgs-ci

View File

@@ -13,19 +13,18 @@ on:
jobs:
nixpkgs:
name: nixpkgs-manual-build
runs-on: ubuntu-latest
if: github.repository_owner == 'NixOS'
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
# explicitly enable sandbox
extra_nix_config: sandbox = true
- uses: cachix/cachix-action@ad2ddac53f961de1989924296a1f236fcfbaa4fc # v15
- uses: cachix/cachix-action@18cf96c7c98e048e10a83abd92116114cd8504be # v14
with:
# This cache is for the nixpkgs repo checks and should not be trusted or used elsewhere.
name: nixpkgs-ci

View File

@@ -10,7 +10,6 @@ on:
jobs:
tests:
name: nix-files-parseable-check
runs-on: ubuntu-latest
if: "github.repository_owner == 'NixOS' && !contains(github.event.pull_request.title, '[skip treewide]')"
steps:
@@ -25,12 +24,12 @@ jobs:
if [[ -s "$HOME/changed_files" ]]; then
echo "CHANGED_FILES=$HOME/changed_files" > "$GITHUB_ENV"
fi
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
with:
# pull_request_target checks out the base branch by default
ref: refs/pull/${{ github.event.pull_request.number }}/merge
if: ${{ env.CHANGED_FILES && env.CHANGED_FILES != '' }}
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
nix_path: nixpkgs=channel:nixpkgs-unstable
- name: Parse all changed or added nix files

View File

@@ -1,116 +0,0 @@
# `nixpkgs-vet` is a tool to vet Nixpkgs: its architecture, package structure, and more.
# Among other checks, it makes sure that `pkgs/by-name` (see `../../pkgs/by-name/README.md`) follows the validity rules outlined in [RFC 140](https://github.com/NixOS/rfcs/pull/140).
# When you make changes to this workflow, please also update `ci/nixpkgs-vet.sh` to reflect the impact of your work to the CI.
# See https://github.com/NixOS/nixpkgs-vet for details on the tool and its checks.
name: Vet nixpkgs
on:
# Using pull_request_target instead of pull_request avoids having to approve first time contributors.
pull_request_target:
# This workflow depends on the base branch of the PR, but changing the base branch is not included in the default trigger events, which would be `opened`, `synchronize` or `reopened`.
# Instead it causes an `edited` event, so we need to add it explicitly here.
# While `edited` is also triggered when the PR title/body is changed, this PR action is fairly quick, and PRs don't get edited **that** often, so it shouldn't be a problem.
# There is a feature request for adding a `base_changed` event: https://github.com/orgs/community/discussions/35058
types: [opened, synchronize, reopened, edited]
permissions: {}
# We don't use a concurrency group here, because the action is triggered quite often (due to the PR edit trigger), and contributors would get notified on any canceled run.
# There is a feature request for suppressing notifications on concurrency-canceled runs: https://github.com/orgs/community/discussions/13015
jobs:
check:
name: nixpkgs-vet
# This needs to be x86_64-linux, because we depend on the tooling being pre-built in the GitHub releases.
runs-on: ubuntu-latest
# This should take 1 minute at most, but let's be generous. The default of 6 hours is definitely too long.
timeout-minutes: 10
steps:
# This step has to be in this file, because it's needed to determine which revision of the repository to fetch, and we can only use other files from the repository once it's fetched.
- name: Resolving the merge commit
env:
GH_TOKEN: ${{ github.token }}
run: |
# This checks for mergeability of a pull request as recommended in
# https://docs.github.com/en/rest/guides/using-the-rest-api-to-interact-with-your-git-database?apiVersion=2022-11-28#checking-mergeability-of-pull-requests
# Retry the API query this many times
retryCount=5
# Start with 5 seconds, but double every retry
retryInterval=5
while true; do
echo "Checking whether the pull request can be merged"
prInfo=$(gh api \
-H "Accept: application/vnd.github+json" \
-H "X-GitHub-Api-Version: 2022-11-28" \
/repos/"$GITHUB_REPOSITORY"/pulls/${{ github.event.pull_request.number }})
mergeable=$(jq -r .mergeable <<< "$prInfo")
mergedSha=$(jq -r .merge_commit_sha <<< "$prInfo")
if [[ "$mergeable" == "null" ]]; then
if (( retryCount == 0 )); then
echo "Not retrying anymore. It's likely that GitHub is having internal issues: check https://www.githubstatus.com/"
exit 1
else
(( retryCount -= 1 )) || true
# null indicates that GitHub is still computing whether it's mergeable
# Wait a couple seconds before trying again
echo "GitHub is still computing whether this PR can be merged, waiting $retryInterval seconds before trying again ($retryCount retries left)"
sleep "$retryInterval"
(( retryInterval *= 2 )) || true
fi
else
break
fi
done
if [[ "$mergeable" == "true" ]]; then
echo "The PR can be merged, checking the merge commit $mergedSha"
echo "mergedSha=$mergedSha" >> "$GITHUB_ENV"
else
echo "The PR cannot be merged, it has a merge conflict, skipping the rest.."
fi
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
if: env.mergedSha
with:
# pull_request_target checks out the base branch by default
ref: ${{ env.mergedSha }}
# Fetches the merge commit and its parents
fetch-depth: 2
- name: Checking out base branch
if: env.mergedSha
run: |
base=$(mktemp -d)
git worktree add "$base" "$(git rev-parse HEAD^1)"
echo "base=$base" >> "$GITHUB_ENV"
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
if: env.mergedSha
- name: Fetching the pinned tool
if: env.mergedSha
# Update the pinned version using ci/nixpkgs-vet/update-pinned-tool.sh
run: |
# The pinned version of the tooling to use.
toolVersion=$(<ci/nixpkgs-vet/pinned-version.txt)
# Fetch the x86_64-linux-specific release artifact containing the gzipped NAR of the pre-built tool.
toolPath=$(curl -sSfL https://github.com/NixOS/nixpkgs-vet/releases/download/"$toolVersion"/x86_64-linux.nar.gz \
| gzip -cd | nix-store --import | tail -1)
# Adds a result symlink as a GC root.
nix-store --realise "$toolPath" --add-root result
- name: Running nixpkgs-vet
if: env.mergedSha
env:
# Force terminal colors to be enabled. The library that `nixpkgs-vet` uses respects https://bixense.com/clicolors/
CLICOLOR_FORCE: 1
run: |
if result/bin/nixpkgs-vet --base "$base" .; then
exit 0
else
exitCode=$?
echo "To run locally: ./ci/nixpkgs-vet.sh $GITHUB_BASE_REF https://github.com/$GITHUB_REPOSITORY.git"
echo "If you're having trouble, ping @NixOS/nixpkgs-vet"
exit "$exitCode"
fi

View File

@@ -16,7 +16,6 @@ permissions:
jobs:
action:
name: set-ofborg-pending
if: github.repository_owner == 'NixOS'
permissions:
statuses: write

View File

@@ -35,13 +35,13 @@ jobs:
pairs:
- from: master
into: haskell-updates
- from: release-24.05
into: staging-next-24.05
- from: staging-next-24.05
into: staging-24.05
- from: release-23.11
into: staging-next-23.11
- from: staging-next-23.11
into: staging-23.11
name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
- name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
uses: devmasx/merge-branch@854d3ac71ed1e9deb668e0074781b81fdd6e771f # 1.4.0

View File

@@ -39,7 +39,7 @@ jobs:
into: staging
name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
- name: ${{ matrix.pairs.from }} → ${{ matrix.pairs.into }}
uses: devmasx/merge-branch@854d3ac71ed1e9deb668e0074781b81fdd6e771f # 1.4.0

View File

@@ -16,8 +16,8 @@ jobs:
if: github.repository_owner == 'NixOS' && github.ref == 'refs/heads/master' # ensure workflow_dispatch only runs on master
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@692973e3d937129bcbf40652eb9f2f61becf3332 # v4.1.7
- uses: cachix/install-nix-action@ba0dd844c9180cbf77aa72a116d6fbc515d0e87b # v27
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
- uses: cachix/install-nix-action@8887e596b4ee1134dae06b98d573bd674693f47c # v26
with:
nix_path: nixpkgs=channel:nixpkgs-unstable
- name: setup
@@ -46,7 +46,7 @@ jobs:
run: |
git clean -f
- name: create PR
uses: peter-evans/create-pull-request@6cd32fd93684475c31847837f87bb135d40a2b79 # v7.0.3
uses: peter-evans/create-pull-request@9153d834b60caba6d51c9b9510b087acf9f33f83 # v6.0.4
with:
body: |
Automatic update by [update-terraform-providers](https://github.com/NixOS/nixpkgs/blob/master/.github/workflows/update-terraform-providers.yml) action.

3
.gitignore vendored
View File

@@ -7,7 +7,6 @@
.idea/
.nixos-test-history
.vscode/
.helix/
outputs/
result-*
result
@@ -20,8 +19,6 @@ tags
/doc/manual.pdf
/source/
.version-suffix
.direnv
.envrc
.DS_Store
.mypy_cache

View File

@@ -1,23 +1,15 @@
ajs124 <git@ajs124.de> <ajs124@users.noreply.github.com>
Anderson Torres <torres.anderson.85@protonmail.com>
Atemu <git@atemu.net> <atemu.main@gmail.com>
Christina Sørensen <christina@cafkafk.com>
Christina Sørensen <christina@cafkafk.com> <christinaafk@gmail.com>
Christina Sørensen <christina@cafkafk.com> <89321978+cafkafk@users.noreply.github.com>
Daniel Løvbrøtte Olsen <me@dandellion.xyz> <daniel.olsen99@gmail.com>
Fabian Affolter <mail@fabian-affolter.ch> <fabian@affolter-engineering.ch>
goatastronaut0212 <goatastronaut0212@outlook.com> <goatastronaut0212@proton.me>
Janne Heß <janne@hess.ooo> <dasJ@users.noreply.github.com>
Jörg Thalheim <joerg@thalheim.io> <Mic92@users.noreply.github.com>
Lin Jian <me@linj.tech> <linj.dev@outlook.com>
Lin Jian <me@linj.tech> <75130626+jian-lin@users.noreply.github.com>
Martin Weinelt <hexa@darmstadt.ccc.de> <mweinelt@users.noreply.github.com>
R. RyanTM <ryantm-bot@ryantm.com>
Robert Hensing <robert@roberthensing.nl> <roberth@users.noreply.github.com>
Sandro Jäckel <sandro.jaeckel@gmail.com>
Sandro Jäckel <sandro.jaeckel@gmail.com> <sandro.jaeckel@sap.com>
superherointj <5861043+superherointj@users.noreply.github.com>
Tomodachi94 <tomodachi94@protonmail.com> Tomo <68489118+Tomodachi94@users.noreply.github.com>
Vladimír Čunát <v@cunat.cz> <vcunat@gmail.com>
Vladimír Čunát <v@cunat.cz> <vladimir.cunat@nic.cz>
Yifei Sun <ysun@hey.com> StepBroBD <Hi@StepBroBD.com>

View File

@@ -315,22 +315,6 @@ When reviewing a pull request, please always be nice and polite. Controversial c
GitHub provides reactions as a simple and quick way to provide feedback to pull requests or any comments. The thumb-down reaction should be used with care and if possible accompanied with some explanation so the submitter has directions to improve their contribution.
When doing a review:
- Aim to drive the proposal to a timely conclusion.
- Focus on the proposed changes to keep the scope of the discussion narrow.
- Help the contributor prioritise their efforts towards getting their change merged.
If you find anything related that could be improved but is not immediately required for acceptance, consider
- Implementing the changes yourself in a follow-up pull request (and request review from the person who inspired you)
- Tracking your idea in an issue
- Offering the original contributor to review a follow-up pull request
- Making concrete [suggestions](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/incorporating-feedback-in-your-pull-request) in the same pull request.
For example, follow-up changes could involve refactoring code in the affected files.
But please remember not to make such additional considerations a blocker, and communicate that to the contributor, for example by following the [conventional comments](https://conventionalcomments.org/) pattern.
If the related change is essential for the contribution at hand, make clear why you think it is important to address that first.
Pull request reviews should include a list of what has been reviewed in a comment, so other reviewers and mergers can know the state of the review.
All the review template samples provided in this section are generic and meant as examples. Their usage is optional and the reviewer is free to adapt them to their liking.
@@ -346,14 +330,7 @@ Container system, boot system and library changes are some examples of the pull
## How to merge pull requests
[pr-merge]: #how-to-merge-pull-requests
To streamline automated updates, leverage the nixpkgs-merge-bot by simply commenting `@NixOS/nixpkgs-merge-bot merge`. The bot will verify if the following conditions are met, refusing to merge otherwise:
- the commenter that issued the command should be among the package maintainers;
- the package should reside in `pkgs/by-name`.
Further, nixpkgs-merge-bot will ensure all ofBorg checks (except the Darwin-related ones) are successfully completed before merging the pull request. Should the checks still be underway, the bot patiently waits for ofBorg to finish before attempting the merge again.
For other pull requests, the *Nixpkgs committers* are people who have been given
The *Nixpkgs committers* are people who have been given
permission to merge.
It is possible for community members that have enough knowledge and experience on a special topic to contribute by merging pull requests.
@@ -370,7 +347,7 @@ The following paragraphs about how to deal with unactive contributors is just a
Please note that contributors with commit rights unactive for more than three months will have their commit rights revoked.
-->
Please see the discussion in [GitHub nixpkgs issue #321665](https://github.com/NixOS/nixpkgs/issues/321665) for information on how to proceed to be granted this level of access.
Please see the discussion in [GitHub nixpkgs issue #50105](https://github.com/NixOS/nixpkgs/issues/50105) for information on how to proceed to be granted this level of access.
In a case a contributor definitively leaves the Nix community, they should create an issue or post on [Discourse](https://discourse.nixos.org) with references of packages and modules they maintain so the maintainership can be taken over by other contributors.
@@ -382,7 +359,7 @@ See [Nix Channel Status](https://status.nixos.org/) for the current channels and
Here's a brief overview of the main Git branches and what channels they're used for:
- `master`: The main branch, used for the unstable channels such as `nixpkgs-unstable`, `nixos-unstable` and `nixos-unstable-small`.
- `release-YY.MM` (e.g. `release-24.05`): The NixOS release branches, used for the stable channels such as `nixos-24.05`, `nixos-24.05-small` and `nixpkgs-24.05-darwin`.
- `release-YY.MM` (e.g. `release-23.11`): The NixOS release branches, used for the stable channels such as `nixos-23.11`, `nixos-23.11-small` and `nixpkgs-23.11-darwin`.
When a channel is updated, a corresponding Git branch is also updated to point to the corresponding commit.
So e.g. the [`nixpkgs-unstable` branch](https://github.com/nixos/nixpkgs/tree/nixpkgs-unstable) corresponds to the Git commit from the [`nixpkgs-unstable` channel](https://channels.nixos.org/nixpkgs-unstable).
@@ -395,12 +372,10 @@ See [this section][branch] to know when to use the release branches.
[staging]: #staging
The staging workflow exists to batch Hydra builds of many packages together.
It is coordinated in the [Staging room](https://matrix.to/#/#staging:nixos.org) on Matrix.
It works by directing commits that cause [mass rebuilds][mass-rebuild] to a separate `staging` branch that isn't directly built by Hydra.
Regularly, the `staging` branch is _manually_ merged into a `staging-next` branch to be built by Hydra using the [`nixpkgs:staging-next` jobset](https://hydra.nixos.org/jobset/nixpkgs/staging-next).
The `staging-next` branch should then only receive changes that fix Hydra builds;
**for anything else, ask the [Staging room](https://matrix.to/#/#staging:nixos.org) first**.
The `staging-next` branch should then only receive direct commits in order to fix Hydra builds.
Once it is verified that there are no major regressions, it is merged into `master` using [a pull request](https://github.com/NixOS/nixpkgs/pulls?q=head%3Astaging-next).
This is done manually in order to ensure it's a good use of Hydra's computing resources.
By keeping the `staging-next` branch separate from `staging`, this batching does not block developers from merging changes into `staging`.
@@ -573,11 +548,138 @@ Names of files and directories should be in lowercase, with dashes between words
### Syntax
- Set up [editorconfig](https://editorconfig.org/) for your editor, such that [the settings](./.editorconfig) are automatically applied.
- Use 2 spaces of indentation per indentation level in Nix expressions, 4 spaces in shell scripts.
- Do not use tab characters, i.e. configure your editor to use soft tabs. For instance, use `(setq-default indent-tabs-mode nil)` in Emacs. Everybody has different tab settings so its asking for trouble.
- Use `lowerCamelCase` for variable names, not `UpperCamelCase`. Note, this rule does not apply to package attribute names, which instead follow the rules in [package naming](./pkgs/README.md#package-naming).
- New files must be formatted by entering the `nix-shell` from the repository root and running `nixfmt`.
- Function calls with attribute set arguments are written as
```nix
foo {
arg = <...>;
}
```
not
```nix
foo
{
arg = <...>;
}
```
Also fine is
```nix
foo { arg = <...>; }
```
if it's a short call.
- In attribute sets or lists that span multiple lines, the attribute names or list elements should be aligned:
```nix
{
# A long list.
list = [
elem1
elem2
elem3
];
# A long attribute set.
attrs = {
attr1 = short_expr;
attr2 =
if true then big_expr else big_expr;
};
# Combined
listOfAttrs = [
{
attr1 = 3;
attr2 = "fff";
}
{
attr1 = 5;
attr2 = "ggg";
}
];
}
```
- Short lists or attribute sets can be written on one line:
```nix
{
# A short list.
list = [ elem1 elem2 elem3 ];
# A short set.
attrs = { x = 1280; y = 1024; };
}
```
- Breaking in the middle of a function argument can give hard-to-read code, like
```nix
someFunction { x = 1280;
y = 1024; } otherArg
yetAnotherArg
```
(especially if the argument is very large, spanning multiple lines).
Better:
```nix
someFunction
{ x = 1280; y = 1024; }
otherArg
yetAnotherArg
```
or
```nix
let res = { x = 1280; y = 1024; };
in someFunction res otherArg yetAnotherArg
```
- The bodies of functions, asserts, and withs are not indented to prevent a lot of superfluous indentation levels, i.e.
```nix
{ arg1, arg2 }:
assert system == "i686-linux";
stdenv.mkDerivation { /* ... */ }
```
not
```nix
{ arg1, arg2 }:
assert system == "i686-linux";
stdenv.mkDerivation { /* ... */ }
```
- Function formal arguments are written as:
```nix
{ arg1, arg2, arg3 }: { /* ... */ }
```
but if they don't fit on one line they're written as:
```nix
{ arg1, arg2, arg3
, arg4
# Some comment...
, argN
}: { }
```
- Functions should list their expected arguments as precisely as possible. That is, write

View File

@@ -52,9 +52,9 @@ Nixpkgs and NixOS are built and tested by our continuous integration
system, [Hydra](https://hydra.nixos.org/).
* [Continuous package builds for unstable/master](https://hydra.nixos.org/jobset/nixos/trunk-combined)
* [Continuous package builds for the NixOS 24.05 release](https://hydra.nixos.org/jobset/nixos/release-24.05)
* [Continuous package builds for the NixOS 23.11 release](https://hydra.nixos.org/jobset/nixos/release-23.11)
* [Tests for unstable/master](https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents)
* [Tests for the NixOS 24.05 release](https://hydra.nixos.org/job/nixos/release-24.05/tested#tabs-constituents)
* [Tests for the NixOS 23.11 release](https://hydra.nixos.org/job/nixos/release-23.11/tested#tabs-constituents)
Artifacts successfully built with Hydra are published to cache at
https://cache.nixos.org/. When successful build and test criteria are

View File

@@ -1,43 +0,0 @@
# CI support files
This directory contains files to support CI, such as [GitHub Actions](https://github.com/NixOS/nixpkgs/tree/master/.github/workflows) and [Ofborg](https://github.com/nixos/ofborg).
This is in contrast with [`maintainers/scripts`](../maintainers/scripts) which is for human use instead.
## Pinned Nixpkgs
CI may need certain packages from Nixpkgs.
In order to ensure that the needed packages are generally available without building,
[`pinned-nixpkgs.json`](./pinned-nixpkgs.json) contains a pinned Nixpkgs version tested by Hydra.
Run [`update-pinned-nixpkgs.sh`](./update-pinned-nixpkgs.sh) to update it.
## `ci/nixpkgs-vet.sh BASE_BRANCH [REPOSITORY]`
Runs the [`nixpkgs-vet` tool](https://github.com/NixOS/nixpkgs-vet) on the HEAD commit, closely matching what CI does. This can't do exactly the same as CI, because CI needs to rely on GitHub's server-side Git history to compute the mergeability of PRs before the check can be started.
In turn, when contributors are running this tool locally, we don't want to have to push commits to test them, and we can also rely on the local Git history to do the mergeability check.
Arguments:
- `BASE_BRANCH`: The base branch to use, e.g. master or release-24.05
- `REPOSITORY`: The repository from which to fetch the base branch. Defaults to <https://github.com/NixOS/nixpkgs.git>.
## `ci/nixpkgs-vet`
This directory contains scripts and files used and related to [`nixpkgs-vet`](https://github.com/NixOS/nixpkgs-vet/), which the CI uses to implement `pkgs/by-name` checks, along with many other Nixpkgs architecture rules.
See also the [CI GitHub Action](../.github/workflows/nixpkgs-vet.yml).
## `ci/nixpkgs-vet/update-pinned-tool.sh`
Updates the pinned [`nixpkgs-vet` tool](https://github.com/NixOS/nixpkgs-vet) in [`ci/nixpkgs-vet/pinned-version.txt`](./nixpkgs-vet/pinned-version.txt) to the latest [release](https://github.com/NixOS/nixpkgs-vet/releases).
Each release contains a pre-built `x86_64-linux` version of the tool which is used by CI.
This script currently needs to be called manually when the CI tooling needs to be updated.
Why not just build the tooling right from the PRs Nixpkgs version?
- Because it allows CI to check all PRs, even if they would break the CI tooling.
- Because it makes the CI check very fast, since no Nix builds need to be done, even for mass rebuilds.
- Because it improves security, since we don't have to build potentially untrusted code from PRs.
The tool only needs a very minimal Nix evaluation at runtime, which can work with [readonly-mode](https://nixos.org/manual/nix/stable/command-ref/opt-common.html#opt-readonly-mode) and [restrict-eval](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval).

View File

@@ -1,71 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p jq
set -o pipefail -o errexit -o nounset
trace() { echo >&2 "$@"; }
tmp=$(mktemp -d)
cleanup() {
# Don't exit early if anything fails to cleanup
set +o errexit
trace -n "Cleaning up.. "
[[ -e "$tmp/base" ]] && git worktree remove --force "$tmp/base"
[[ -e "$tmp/merged" ]] && git worktree remove --force "$tmp/merged"
rm -rf "$tmp"
trace "Done"
}
trap cleanup exit
repo=https://github.com/NixOS/nixpkgs.git
if (( $# != 0 )); then
baseBranch=$1
shift
else
trace "Usage: $0 BASE_BRANCH [REPOSITORY]"
trace "BASE_BRANCH: The base branch to use, e.g. master or release-23.11"
trace "REPOSITORY: The repository to fetch the base branch from, defaults to $repo"
exit 1
fi
if (( $# != 0 )); then
repo=$1
shift
fi
if [[ -n "$(git status --porcelain)" ]]; then
trace -e "\e[33mWarning: Dirty tree, uncommitted changes won't be taken into account\e[0m"
fi
headSha=$(git rev-parse HEAD)
trace -e "Using HEAD commit \e[34m$headSha\e[0m"
trace -n "Creating Git worktree for the HEAD commit in $tmp/merged.. "
git worktree add --detach -q "$tmp/merged" HEAD
trace "Done"
trace -n "Fetching base branch $baseBranch to compare against.. "
git fetch -q "$repo" refs/heads/"$baseBranch"
baseSha=$(git rev-parse FETCH_HEAD)
trace -e "\e[34m$baseSha\e[0m"
trace -n "Creating Git worktree for the base branch in $tmp/base.. "
git worktree add -q "$tmp/base" "$baseSha"
trace "Done"
trace -n "Merging base branch into the HEAD commit in $tmp/merged.. "
git -C "$tmp/merged" merge -q --no-edit "$baseSha"
trace -e "\e[34m$(git -C "$tmp/merged" rev-parse HEAD)\e[0m"
trace -n "Reading pinned nixpkgs-vet version from pinned-version.txt.. "
toolVersion=$(<"$tmp/merged/ci/nixpkgs-vet/pinned-version.txt")
trace -e "\e[34m$toolVersion\e[0m"
trace -n "Building tool.. "
nix-build https://github.com/NixOS/nixpkgs-vet/tarball/"$toolVersion" -o "$tmp/tool" -A build
trace "Running nixpkgs-vet.."
"$tmp/tool/bin/nixpkgs-vet" --base "$tmp/base" "$tmp/merged"

View File

@@ -1 +0,0 @@
0.1.4

View File

@@ -1,4 +0,0 @@
{
"rev": "4de4818c1ffa76d57787af936e8a23648bda6be4",
"sha256": "0l3b9jr5ydzqgvd10j12imc9jqb6jv5v2bdi1gyy5cwkwplfay67"
}

View File

@@ -1,17 +0,0 @@
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p jq
set -euo pipefail
# https://stackoverflow.com/a/246128
SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
repo=https://github.com/nixos/nixpkgs
branch=nixpkgs-unstable
file=$SCRIPT_DIR/pinned-nixpkgs.json
defaultRev=$(git ls-remote "$repo" refs/heads/"$branch" | cut -f1)
rev=${1:-$defaultRev}
sha256=$(nix-prefetch-url --unpack "$repo/archive/$rev.tar.gz" --name source)
jq -n --arg rev "$rev" --arg sha256 "$sha256" '$ARGS.named' | tee /dev/stderr > $file

View File

@@ -2,20 +2,10 @@
This directory houses the sources files for the Nixpkgs reference manual.
> [!IMPORTANT]
> We are actively restructuring our documentation to follow the [Diátaxis framework](https://diataxis.fr/)
>
> Going forward, this directory should **only** contain [reference documentation](https://nix.dev/contributing/documentation/diataxis#reference).
> For tutorials, guides and explanations, contribute to <https://nix.dev/> instead.
>
> We are actively working to generate **all** reference documentation from the [doc-comments](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md) present in code.
> This also provides the benefit of using `:doc` in the `nix repl` to view reference documentation locally on the fly.
Going forward, it should only contain [reference](https://nix.dev/contributing/documentation/diataxis#reference) documentation.
For tutorials, guides and explanations, contribute to <https://nix.dev/> instead.
For documentation only relevant for contributors, use Markdown files next to the source and regular code comments.
> [!TIP]
> Feedback for improving support for parsing and rendering doc-comments is highly appreciated.
> [Open an issue](https://github.com/NixOS/nixpkgs/issues/new?labels=6.topic%3A+documentation&title=Doc%3A+) to request bugfixes or new features.
For documentation only relevant for contributors, use Markdown files and code comments in the source code.
Rendered documentation:
- [Unstable (from master)](https://nixos.org/manual/nixpkgs/unstable/)
@@ -261,42 +251,25 @@ You, as the writer of documentation, are still in charge of its content.
For example:
```markdown
# pkgs.coolFunction {#pkgs.coolFunction}
# pkgs.coolFunction
`pkgs.coolFunction` *`name`* *`config`*
Description of what `coolFunction` does.
Description of what `callPackage` does.
## Inputs
`coolFunction` expects a single argument which should be an attribute set, with the following possible attributes:
## Inputs {#pkgs-coolFunction-inputs}
If something's special about `coolFunction`'s general argument handling, you can say so here.
Otherwise, just describe the single argument or start the arguments' definition list without introduction.
*`name`* (String)
`name` (String)
: The name of the resulting image.
*`config`* (Attribute set)
`tag` (String; _optional_)
: Introduce the parameter. Maybe you have a test to make sure `{ }` is a sensible default; then you can say: these attributes are optional; `{ }` is a valid argument.
: Tag of the generated image.
`outputHash` (String; _optional_)
: A brief explanation including when and when not to pass this attribute.
: _Default:_ the output path's hash.
_Default:_ the output path's hash.
```
Checklist:
- Start with a synopsis, to show the order of positional arguments.
- Metavariables are in emphasized code spans: ``` *`arg1`* ```. Metavariables are placeholders where users may write arbitrary expressions. This includes positional arguments.
- Attribute names are regular code spans: ``` `attr1` ```. These identifiers can _not_ be picked freely by users, so they are _not_ metavariables.
- _optional_ attributes have a _`Default:`_ if it's easily described as a value.
- _optional_ attributes have a _`Default behavior:`_ if it's not easily described using a value.
- Nix types aren't in code spans, because they are not code
- Nix types are capitalized, to distinguish them from the camelCase Module System types, which _are_ code and behave like functions.
#### Examples
To define a referenceable figure use the following fencing:
@@ -320,7 +293,7 @@ Though this is not shown in the rendered documentation on nixos.org.
#### Figures
To define a referenceable figure use the following fencing:
To define a referencable figure use the following fencing:
```markdown
::: {.figure #nixos-logo}

View File

@@ -20,7 +20,6 @@ There is no uniform interface for build helpers.
build-helpers/fetchers.chapter.md
build-helpers/trivial-build-helpers.chapter.md
build-helpers/testers.chapter.md
build-helpers/dev-shell-tools.chapter.md
build-helpers/special.md
build-helpers/images.md
hooks/index.md

View File

@@ -1,75 +0,0 @@
# Development Shell helpers {#chap-devShellTools}
The `nix-shell` command has popularized the concept of transient shell environments for development or testing purposes.
<!--
We should try to document the product, not its development process in the Nixpkgs reference manual,
but *something* needs to be said to provide context for this library.
This is the most future proof sentence I could come up with while Nix itself does yet make use of this.
Relevant is the current status of the devShell attribute "project": https://github.com/NixOS/nix/issues/7501
-->
However, `nix-shell` is not the only way to create such environments, and even `nix-shell` itself can indirectly benefit from this library.
This library provides a set of functions that help create such environments.
## `devShellTools.valueToString` {#sec-devShellTools-valueToString}
Converts Nix values to strings in the way the [`derivation` built-in function](https://nix.dev/manual/nix/2.23/language/derivations) does.
:::{.example}
## `valueToString` usage examples
```nix
devShellTools.valueToString (builtins.toFile "foo" "bar")
=> "/nix/store/...-foo"
```
```nix
devShellTools.valueToString false
=> ""
```
:::
## `devShellTools.unstructuredDerivationInputEnv` {#sec-devShellTools-unstructuredDerivationInputEnv}
Convert a set of derivation attributes (as would be passed to [`derivation`]) to a set of environment variables that can be used in a shell script.
This function does not support `__structuredAttrs`, but does support `passAsFile`.
:::{.example}
## `unstructuredDerivationInputEnv` usage example
```nix
devShellTools.unstructuredDerivationInputEnv {
drvAttrs = {
name = "foo";
buildInputs = [ hello figlet ];
builder = bash;
args = [ "-c" "${./builder.sh}" ];
};
}
=> {
name = "foo";
buildInputs = "/nix/store/...-hello /nix/store/...-figlet";
builder = "/nix/store/...-bash";
}
```
Note that `args` is not included, because Nix does not added it to the builder process environment.
:::
## `devShellTools.derivationOutputEnv` {#sec-devShellTools-derivationOutputEnv}
Takes the relevant parts of a derivation and returns a set of environment variables, that would be present in the derivation.
:::{.example}
## `derivationOutputEnv` usage example
```nix
let
pkg = hello;
in
devShellTools.derivationOutputEnv { outputList = pkg.outputs; outputMap = pkg; }
```
:::

View File

@@ -157,12 +157,6 @@ Here are security considerations for this scenario:
In more concrete terms, if you use any other hash, the [`--insecure` flag](https://curl.se/docs/manpage.html#-k) will be passed to the underlying call to `curl` when downloading content.
## Proxy usage {#sec-pkgs-fetchers-proxy}
Nixpkgs fetchers can make use of a http(s) proxy. Each fetcher will automatically inherit proxy-related environment variables (`http_proxy`, `https_proxy`, etc) via [impureEnvVars](https://nixos.org/manual/nix/stable/language/advanced-attributes#adv-attr-impureEnvVars).
The environment variable `NIX_SSL_CERT_FILE` is also inherited in fetchers, and can be used to provide a custom certificate bundle to fetchers. This is usually required for a https proxy to work without certificate validation errors.
[]{#fetchurl}
## `fetchurl` {#sec-pkgs-fetchers-fetchurl}
@@ -371,8 +365,8 @@ If `pname` and `version` are specified, `fetchurl` will use those values and wil
_Default value:_ `{}`.
`passthru` (Attribute Set; _optional_)
: Specifies any extra [`passthru`](#chap-passthru) attributes for the derivation returned by `fetchurl`.
Note that `fetchurl` defines [`passthru` attributes of its own](#ssec-pkgs-fetchers-fetchurl-passthru-outputs).
: Specifies any extra [passthru](#var-stdenv-passthru) attributes for the derivation returned by `fetchurl`.
Note that `fetchurl` defines [passthru attributes of its own](#ssec-pkgs-fetchers-fetchurl-passthru-outputs).
Attributes specified in `passthru` can override the default attributes returned by `fetchurl`.
_Default value:_ `{}`.
@@ -393,7 +387,7 @@ If `pname` and `version` are specified, `fetchurl` will use those values and wil
### Passthru outputs {#ssec-pkgs-fetchers-fetchurl-passthru-outputs}
`fetchurl` also defines its own [`passthru`](#chap-passthru) attributes:
`fetchurl` also defines its own [`passthru`](#var-stdenv-passthru) attributes:
`url` (String)
@@ -875,7 +869,7 @@ It produces packages that cannot be built automatically.
fetchtorrent {
config = { peer-limit-global = 100; };
url = "magnet:?xt=urn:btih:dd8255ecdc7ca55fb0bbf81323d87062db1f6d1c";
hash = "";
sha256 = "";
}
```

View File

@@ -64,7 +64,7 @@ let
src = fetchurl {
url = "https://github.com/irccloud/irccloud-desktop/releases/download/v${version}/IRCCloud-${version}-linux-x86_64.AppImage";
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
sha256 = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
in appimageTools.wrapType2 {
inherit pname version src;
@@ -100,7 +100,7 @@ let
src = fetchurl {
url = "https://github.com/irccloud/irccloud-desktop/releases/download/v${version}/IRCCloud-${version}-linux-x86_64.AppImage";
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
sha256 = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
appimageContents = appimageTools.extract {
@@ -141,7 +141,7 @@ let
src = fetchurl {
url = "https://github.com/irccloud/irccloud-desktop/releases/download/v${version}/IRCCloud-${version}-linux-x86_64.AppImage";
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
sha256 = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
appimageContents = appimageTools.extract {

View File

@@ -50,10 +50,6 @@ Similarly, if you encounter errors similar to `Error_Protocol ("certificate has
If specified, the layer created by `buildImage` will be appended to the layers defined in the base image, resulting in an image with at least two layers (one or more layers from the base image, and the layer created by `buildImage`).
Otherwise, the resulting image with contain the single layer created by `buildImage`.
:::{.note}
Only **Env** configuration is inherited from the base image.
:::
_Default value:_ `null`.
`fromImageName` (String or Null; _optional_)
@@ -189,26 +185,13 @@ Similarly, if you encounter errors similar to `Error_Protocol ("certificate has
_Default value:_ `"gz"`.\
_Possible values:_ `"none"`, `"gz"`, `"zstd"`.
`includeNixDB` (Boolean; _optional_)
: Populate the nix database in the image with the dependencies of `copyToRoot`.
The main purpose is to be able to use nix commands in the container.
:::{.caution}
Be careful since this doesn't work well in combination with `fromImage`. In particular, in a multi-layered image, only the Nix paths from the lower image will be in the database.
This also neglects to register the store paths that are pulled into the image as a dependency of one of the other values, but aren't a dependency of `copyToRoot`.
:::
_Default value:_ `false`.
`contents` **DEPRECATED**
: This attribute is deprecated, and users are encouraged to use `copyToRoot` instead.
### Passthru outputs {#ssec-pkgs-dockerTools-buildImage-passthru-outputs}
`buildImage` defines a few [`passthru`](#chap-passthru) attributes:
`buildImage` defines a few [`passthru`](#var-stdenv-passthru) attributes:
`buildArgs` (Attribute Set)
@@ -453,7 +436,7 @@ See [](#ex-dockerTools-streamLayeredImage-exploringlayers) to understand how the
`streamLayeredImage` allows scripts to be run when creating the additional layer with symlinks, allowing custom behaviour to affect the final results of the image (see the documentation of the `extraCommands` and `fakeRootCommands` attributes).
The resulting repository tarball will list a single image as specified by the `name` and `tag` attributes.
By default, that image will use a static creation date (see documentation for the `created` and `mtime` attributes).
By default, that image will use a static creation date (see documentation for the `created` attribute).
This allows the function to produce reproducible images.
### Inputs {#ssec-pkgs-dockerTools-streamLayeredImage-inputs}
@@ -516,7 +499,6 @@ This allows the function to produce reproducible images.
`created` (String; _optional_)
: Specifies the time of creation of the generated image.
This date will be used for the image metadata.
This should be either a date and time formatted according to [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601) or `"now"`, in which case the current date will be used.
:::{.caution}
@@ -525,18 +507,6 @@ This allows the function to produce reproducible images.
_Default value:_ `"1970-01-01T00:00:01Z"`.
`mtime` (String; _optional_)
: Specifies the time used for the modification timestamp of files within the layers of the generated image.
This should be either a date and time formatted according to [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601) or `"now"`, in which case the current date will be used.
:::{.caution}
Using a non-constant date will cause built layers to have a different hash each time, preventing deduplication.
Using `"now"` also means that the generated image will not be reproducible anymore (because the date will always change whenever it's built).
:::
_Default value:_ `"1970-01-01T00:00:01Z"`.
`uid` (Number; _optional_) []{#dockerTools-buildLayeredImage-arg-uid}
`gid` (Number; _optional_) []{#dockerTools-buildLayeredImage-arg-gid}
`uname` (String; _optional_) []{#dockerTools-buildLayeredImage-arg-uname}
@@ -604,28 +574,15 @@ This allows the function to produce reproducible images.
_Default value:_ `true`
`includeNixDB` (Boolean; _optional_)
: Populate the nix database in the image with the dependencies of `copyToRoot`.
The main purpose is to be able to use nix commands in the container.
:::{.caution}
Be careful since this doesn't work well in combination with `fromImage`. In particular, in a multi-layered image, only the Nix paths from the lower image will be in the database.
This also neglects to register the store paths that are pulled into the image as a dependency of one of the other values, but aren't a dependency of `copyToRoot`.
:::
_Default value:_ `false`.
`passthru` (Attribute Set; _optional_)
: Use this to pass any attributes as [`passthru`](#chap-passthru) for the resulting derivation.
: Use this to pass any attributes as [passthru](#var-stdenv-passthru) for the resulting derivation.
_Default value:_ `{}`
### Passthru outputs {#ssec-pkgs-dockerTools-streamLayeredImage-passthru-outputs}
`streamLayeredImage` also defines its own [`passthru`](#chap-passthru) attributes:
`streamLayeredImage` also defines its own [`passthru`](#var-stdenv-passthru) attributes:
`imageTag` (String)

View File

@@ -85,14 +85,14 @@ let
in
make-disk-image {
inherit pkgs lib;
inherit (evalConfig {
config = evalConfig {
modules = [
{
fileSystems."/" = { device = "/dev/vda"; fsType = "ext4"; autoFormat = true; };
boot.grub.device = "/dev/vda";
}
];
}) config;
};
format = "qcow2";
onlyNixStore = false;
partitionTableType = "legacy+gpt";
@@ -104,3 +104,5 @@ in
memSize = 2048; # Qemu VM memory size in megabytes. Defaults to 1024M.
}
```

View File

@@ -9,40 +9,22 @@ pkgs.makeSetupHook {
name = "something-hook";
propagatedBuildInputs = [ pkgs.commandsomething ];
depsTargetTargetPropagated = [ pkgs.libsomething ];
} ./script.sh;
} ./script.sh
```
### setup hook that depends on the hello package and runs hello and @shell@ is substituted with path to bash {#sec-pkgs.makeSetupHook-usage-example}
```nix
pkgs.makeSetupHook
{
pkgs.makeSetupHook {
name = "run-hello-hook";
# Put dependencies here if they have hooks or necessary dependencies propagated
# otherwise prefer direct paths to executables.
propagatedBuildInputs = [
pkgs.hello
pkgs.cowsay
];
substitutions = {
shell = "${pkgs.bash}/bin/bash";
cowsay = "${pkgs.cowsay}/bin/cowsay";
};
}
(
writeScript "run-hello-hook.sh" ''
#!@shell@
# the direct path to the executable has to be here because
# this will be run when the file is sourced
# at which point '$PATH' has not yet been populated with inputs
@cowsay@ cow
_printHelloHook() {
hello
}
preConfigureHooks+=(_printHelloHook)
''
);
propagatedBuildInputs = [ pkgs.hello ];
substitutions = { shell = "${pkgs.bash}/bin/bash"; };
passthru.tests.greeting = callPackage ./test { };
meta.platforms = lib.platforms.linux;
} (writeScript "run-hello-hook.sh" ''
#!@shell@
hello
'')
```
## Attributes {#sec-pkgs.makeSetupHook-attributes}

View File

@@ -40,139 +40,13 @@ If the `moduleNames` argument is omitted, `hasPkgConfigModules` will use `meta.p
:::
## `lycheeLinkCheck` {#tester-lycheeLinkCheck}
Check a packaged static site's links with the [`lychee` package](https://search.nixos.org/packages?show=lychee&type=packages&query=lychee).
You may use Nix to reproducibly build static websites, such as for software documentation.
Some packages will install documentation in their `out` or `doc` outputs, or maybe you have dedicated package where you've made your static site reproducible by running a generator, such as [Hugo](https://gohugo.io/) or [mdBook](https://rust-lang.github.io/mdBook/), in a derivation.
If you have a static site that can be built with Nix, you can use `lycheeLinkCheck` to check that the hyperlinks in your site are correct, and do so as part of your Nix workflow and CI.
:::{.example #ex-lycheelinkcheck}
# Check hyperlinks in the `nix` documentation
```nix
testers.lycheeLinkCheck {
site = nix.doc + "/share/doc/nix/manual";
}
```
:::
### Return value {#tester-lycheeLinkCheck-return}
This tester produces a package that does not produce useful outputs, but only succeeds if the hyperlinks in your site are correct. The build log will list the broken links.
It has two modes:
- Build the returned derivation; its build process will check that internal hyperlinks are correct. This runs in the sandbox, so it will not check external hyperlinks, but it is quick and reliable.
- Invoke the `.online` attribute with [`nix run`](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-run) ([experimental](https://nixos.org/manual/nix/stable/contributing/experimental-features#xp-feature-nix-command)). This runs outside the sandbox, and checks that both internal and external hyperlinks are correct.
Example:
```shell
nix run nixpkgs#lychee.tests.ok.online
```
### Inputs {#tester-lycheeLinkCheck-inputs}
`site` (path or derivation) {#tester-lycheeLinkCheck-param-site}
: The path to the files to check.
`remap` (attribe set, optional) {#tester-lycheeLinkCheck-param-remap}
: An attribute set where the attribute names are regular expressions.
The values should be strings, derivations, or path values.
In the returned check's default configuration, external URLs are only checked when you run the `.online` attribute.
By adding remappings, you can check offline that URLs to external resources are correct, by providing a stand-in from the file system.
Before checking the existence of a URL, the regular expressions are matched and replaced by their corresponding values.
Example:
```nix
{
"https://nix\\.dev/manual/nix/[a-z0-9.-]*" = "${nix.doc}/share/doc/nix/manual";
"https://nixos\\.org/manual/nix/(un)?stable" = "${emptyDirectory}/placeholder-to-disallow-old-nix-docs-urls";
}
```
Store paths in the attribute values are automatically prefixed with `file://`, because lychee requires this for paths in the file system.
If this is a problem, or if you need to control the order in which replacements are performed, use `extraConfig.remap` instead.
`extraConfig` (attribute set) {#tester-lycheeLinkCheck-param-extraConfig}
: Extra configuration to pass to `lychee` in its [configuration file](https://github.com/lycheeverse/lychee/blob/master/lychee.example.toml).
It is automatically [translated](https://nixos.org/manual/nixos/stable/index.html#sec-settings-nix-representable) to TOML.
Example: `{ "include_verbatim" = true; }`
`lychee` (derivation, optional) {#tester-lycheeLinkCheck-param-lychee}
: The `lychee` package to use.
## `shellcheck` {#tester-shellcheck}
Runs files through `shellcheck`, a static analysis tool for shell scripts.
:::{.example #ex-shellcheck}
# Run `testers.shellcheck`
A single script
```nix
testers.shellcheck {
name = "shellcheck";
src = ./script.sh;
}
```
Multiple files
```nix
let
inherit (lib) fileset;
in
testers.shellcheck {
name = "shellcheck";
src = fileset.toSource {
root = ./.;
fileset = fileset.unions [
./lib.sh
./nixbsd-activate
];
};
}
```
:::
### Inputs {#tester-shellcheck-inputs}
[`src` (path or string)]{#tester-shellcheck-param-src}
: The path to the shell script(s) to check.
This can be a single file or a directory containing shell files.
All files in `src` will be checked, so you may want to provide `fileset`-based source instead of a whole directory.
### Return value {#tester-shellcheck-return}
A derivation that runs `shellcheck` on the given script(s).
The build will fail if `shellcheck` finds any issues.
## `testVersion` {#tester-testVersion}
Checks that the output from running a command contains the specified version string in it as a whole word.
NOTE: In most cases, [`versionCheckHook`](#versioncheckhook) should be preferred, but this function is provided and documented here anyway. The motivation for adding either tests would be:
- Catch dynamic linking errors and such and missing environment variables that should be added by wrapping.
- Probable protection against accidentally building the wrong version, for example when using an "old" hash in a fixed-output derivation.
Although simplistic, this test assures that the main program can run.
While there's no substitute for a real test case, it does catch dynamic linking errors and such.
It also provides some protection against accidentally building the wrong version, for example when using an "old" hash in a fixed-output derivation.
By default, the command to be run will be inferred from the given `package` attribute:
it will check `meta.mainProgram` first, and fall back to `pname` or `name`.
@@ -255,7 +129,7 @@ runCommand "example" {
:::
## `testEqualContents` {#tester-testEqualContents}
## `testEqualContents` {#tester-equalContents}
Check that two paths have the same contents.
@@ -339,41 +213,6 @@ once to get a derivation hash, and again to produce the final fixed output deriv
:::
## `runCommand` {#tester-runCommand}
`runCommand :: { name, script, stdenv ? stdenvNoCC, hash ? "...", ... } -> Derivation`
This is a wrapper around `pkgs.runCommandWith`, which
- produces a fixed-output derivation, enabling the command(s) to access the network ;
- salts the derivation's name based on its inputs, ensuring the command is re-run whenever the inputs changes.
It accepts the following attributes:
- the derivation's `name` ;
- the `script` to be executed ;
- `stdenv`, the environment to use, defaulting to `stdenvNoCC` ;
- the derivation's output `hash`, defaulting to the empty file's.
The derivation's `outputHashMode` is set by default to recursive, so the `script` can output a directory as well.
All other attributes are passed through to [`mkDerivation`](#sec-using-stdenv),
including `nativeBuildInputs` to specify dependencies available to the `script`.
:::{.example #ex-tester-runCommand-nix}
# Run a command with network access
```nix
testers.runCommand {
name = "access-the-internet";
command = ''
curl -o /dev/null https://example.com
touch $out
'';
nativeBuildInputs = with pkgs; [ cacert curl ];
}
```
:::
## `runNixOSTest` {#tester-runNixOSTest}
A helper function that behaves exactly like the NixOS `runTest`, except it also assigns this Nixpkgs package set as the `pkgs` of the test and makes the `nixpkgs.*` options read-only.

View File

@@ -3,122 +3,32 @@
Nixpkgs provides a variety of wrapper functions that help build commonly useful derivations.
Like [`stdenv.mkDerivation`](#sec-using-stdenv), each of these build helpers creates a derivation, but the arguments passed are different (usually simpler) from those required by `stdenv.mkDerivation`.
## `runCommand` {#trivial-builder-runCommand}
## `runCommandWith` {#trivial-builder-runCommandWith}
`runCommand :: String -> AttrSet -> String -> Derivation`
The function `runCommandWith` returns a derivation built using the specified command(s), in a specified environment.
The result of `runCommand name drvAttrs buildCommand` is a derivation that is built by running the specified shell commands.
It is the underlying base function of all [`runCommand*` variants].
The general behavior is controlled via a single attribute set passed
as the first argument, and allows specifying `stdenv` freely.
By default `runCommand` runs in a stdenv with no compiler environment, whereas [`runCommandCC`](#trivial-builder-runCommandCC) uses the default stdenv, `pkgs.stdenv`.
The following [`runCommand*` variants] exist: `runCommand`, `runCommandCC`, and `runCommandLocal`.
`name :: String`
: The name that Nix will append to the store path in the same way that `stdenv.mkDerivation` uses its `name` attribute.
[`runCommand*` variants]: #trivial-builder-runCommand
`drvAttr :: AttrSet`
: Attributes to pass to the underlying call to [`stdenv.mkDerivation`](#chap-stdenv).
### Type {#trivial-builder-runCommandWith-Type}
```
runCommandWith :: {
name :: name;
stdenv? :: Derivation;
runLocal? :: Bool;
derivationArgs? :: { ... };
} -> String -> Derivation
```
### Inputs {#trivial-builder-runCommandWith-Inputs}
`name` (String)
: The derivation's name, which Nix will append to the store path; see [`mkDerivation`](#sec-using-stdenv).
`runLocal` (Boolean)
: If set to `true` this forces the derivation to be built locally, not using [substitutes] nor remote builds.
This is intended for very cheap commands (<1s execution time) which can be sped up by avoiding the network round-trip(s).
Its effect is to set [`preferLocalBuild = true`][preferLocalBuild] and [`allowSubstitutes = false`][allowSubstitutes].
::: {.note}
This prevents the use of [substituters][substituter], so only set `runLocal` (or use `runCommandLocal`) when certain the user will
always have a builder for the `system` of the derivation. This should be true for most trivial use cases
(e.g., just copying some files to a different location or adding symlinks) because there the `system`
is usually the same as `builtins.currentSystem`.
:::
`stdenv` (Derivation)
: The [standard environment](#chap-stdenv) to use, defaulting to `pkgs.stdenv`
`derivationArgs` (Attribute set)
: Additional arguments for [`mkDerivation`](#sec-using-stdenv).
`buildCommand` (String)
`buildCommand :: String`
: Shell commands to run in the derivation builder.
::: {.note}
You have to create a file or directory `$out` for Nix to be able to run the builder successfully.
:::
[allowSubstitutes]: https://nixos.org/nix/manual/#adv-attr-allowSubstitutes
[preferLocalBuild]: https://nixos.org/nix/manual/#adv-attr-preferLocalBuild
[substituter]: https://nix.dev/manual/nix/latest/glossary#gloss-substituter
[substitutes]: https://nix.dev/manual/nix/2.23/glossary#gloss-substitute
::: {.example #ex-runcommandwith}
# Invocation of `runCommandWith`
```nix
runCommandWith {
name = "example";
derivationArgs.nativeBuildInputs = [ cowsay ];
} ''
cowsay > $out <<EOMOO
'runCommandWith' is a bit cumbersome,
so we have more ergonomic wrappers.
EOMOO
''
```
:::
## `runCommand` and `runCommandCC` {#trivial-builder-runCommand}
The function `runCommand` returns a derivation built using the specified command(s), in the `stdenvNoCC` environment.
`runCommandCC` is similar but uses the default compiler environment. To minimize dependencies, `runCommandCC`
should only be used when the build command needs a C compiler.
`runCommandLocal` is also similar to `runCommand`, but forces the derivation to be built locally.
See the note on [`runCommandWith`] about `runLocal`.
[`runCommandWith`]: #trivial-builder-runCommandWith
### Type {#trivial-builder-runCommand-Type}
```
runCommand :: String -> AttrSet -> String -> Derivation
runCommandCC :: String -> AttrSet -> String -> Derivation
runCommandLocal :: String -> AttrSet -> String -> Derivation
```
### Input {#trivial-builder-runCommand-Input}
While the type signature(s) differ from [`runCommandWith`], individual arguments with the same name will have the same type and meaning:
`name` (String)
: The derivation's name
`derivationArgs` (Attribute set)
: Additional parameters passed to [`mkDerivation`]
`buildCommand` (String)
: The command(s) run to build the derivation.
::: {.example #ex-runcommand-simple}
# Invocation of `runCommand`
```nix
runCommand "my-example" {} ''
(import <nixpkgs> {}).runCommand "my-example" {} ''
echo My example command is running
mkdir $out
@@ -139,24 +49,18 @@ runCommand "my-example" {} ''
```
:::
## `runCommandCC` {#trivial-builder-runCommandCC}
This works just like `runCommand`. The only difference is that it also provides a C compiler in `buildCommand`'s environment. To minimize your dependencies, you should only use this if you are sure you will need a C compiler as part of running your command.
## `runCommandLocal` {#trivial-builder-runCommandLocal}
Variant of `runCommand` that forces the derivation to be built locally, it is not substituted. This is intended for very cheap commands (<1s execution time). It saves on the network round-trip and can speed up a build.
::: {.note}
`runCommand name derivationArgs buildCommand` is equivalent to
```nix
runCommandWith {
inherit name derivationArgs;
stdenv = stdenvNoCC;
} buildCommand
```
Likewise, `runCommandCC name derivationArgs buildCommand` is equivalent to
```nix
runCommandWith {
inherit name derivationArgs;
} buildCommand
```
This sets [`allowSubstitutes` to `false`](https://nixos.org/nix/manual/#adv-attr-allowSubstitutes), so only use `runCommandLocal` if you are certain the user will always have a builder for the `system` of the derivation. This should be true for most trivial use cases (e.g., just copying some files to a different location or adding symlinks) because there the `system` is usually the same as `builtins.currentSystem`.
:::
## Writing text files {#trivial-builder-text-writing}
Nixpkgs provides the following functions for producing derivations which write text files or executable scripts into the Nix store.
@@ -337,7 +241,7 @@ Write a text file to the Nix store.
`allowSubstitutes` (Bool, _optional_)
: Whether to allow substituting from a binary cache.
Passed through to [`allowSubstitutes`](https://nixos.org/manual/nix/stable/language/advanced-attributes#adv-attr-allowSubstitutes) of the underlying call to `builtins.derivation`.
Passed through to [`allowSubsitutes`](https://nixos.org/manual/nix/stable/language/advanced-attributes#adv-attr-allowSubstitutes) of the underlying call to `builtins.derivation`.
It defaults to `false`, as running the derivation's simple `builder` executable locally is assumed to be faster than network operations.
Set it to true if the `checkPhase` step is expensive.
@@ -501,7 +405,7 @@ writeTextFile {
text = ''
Contents of File
'';
destination = "/share/my-file";
destination = "share/my-file";
}
```
@@ -533,6 +437,7 @@ writeScript "my-file"
Contents of File
''
```
:::
This is equivalent to:
@@ -545,11 +450,10 @@ writeTextFile {
executable = true;
}
```
:::
### `writeScriptBin` {#trivial-builder-writeScriptBin}
Write a script within a `bin` subdirectory of a directory in the Nix store.
Write a script within a `bin` subirectory of a directory in the Nix store.
This is for consistency with the convention of software packages placing executables under `bin`.
`writeScriptBin` takes the following arguments:
@@ -564,7 +468,7 @@ This is for consistency with the convention of software packages placing executa
The created file is marked as executable.
The file's contents will be put into `/nix/store/<store path>/bin/<name>`.
The store path will include the name, and it will be a directory.
The store path will include the the name, and it will be a directory.
::: {.example #ex-writeScriptBin}
# Usage of `writeScriptBin`
@@ -586,7 +490,7 @@ writeTextFile {
echo "hi"
'';
executable = true;
destination = "/bin/my-script";
destination = "bin/my-script";
}
```
@@ -674,7 +578,7 @@ writeTextFile {
echo "hi"
'';
executable = true;
destination = "/bin/my-script";
destination = "bin/my-script";
}
```

4
doc/common.nix Normal file
View File

@@ -0,0 +1,4 @@
{
outputPath = "share/doc/nixpkgs";
indexPath = "manual.html";
}

View File

@@ -1,6 +1,176 @@
{
pkgs ? (import ./.. { }),
nixpkgs ? { },
}:
{ pkgs ? (import ./.. { }), nixpkgs ? { }}:
let
inherit (pkgs) lib;
inherit (lib) hasPrefix removePrefix;
pkgs.nixpkgs-manual.override { inherit nixpkgs; }
common = import ./common.nix;
lib-docs = import ./doc-support/lib-function-docs.nix {
inherit pkgs nixpkgs;
libsets = [
{ name = "asserts"; description = "assertion functions"; }
{ name = "attrsets"; description = "attribute set functions"; }
{ name = "strings"; description = "string manipulation functions"; }
{ name = "versions"; description = "version string functions"; }
{ name = "trivial"; description = "miscellaneous functions"; }
{ name = "fixedPoints"; baseName = "fixed-points"; description = "explicit recursion functions"; }
{ name = "lists"; description = "list manipulation functions"; }
{ name = "debug"; description = "debugging functions"; }
{ name = "options"; description = "NixOS / nixpkgs option handling"; }
{ name = "path"; description = "path functions"; }
{ name = "filesystem"; description = "filesystem functions"; }
{ name = "fileset"; description = "file set functions"; }
{ name = "sources"; description = "source filtering functions"; }
{ name = "cli"; description = "command-line serialization functions"; }
{ name = "gvariant"; description = "GVariant formatted string serialization functions"; }
{ name = "customisation"; description = "Functions to customise (derivation-related) functions, derivatons, or attribute sets"; }
{ name = "meta"; description = "functions for derivation metadata"; }
{ name = "derivations"; description = "miscellaneous derivation-specific functions"; }
];
};
epub = pkgs.runCommand "manual.epub" {
nativeBuildInputs = with pkgs; [ libxslt zip ];
epub = ''
<book xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="nixpkgs-manual">
<info>
<title>Nixpkgs Manual</title>
<subtitle>Version ${pkgs.lib.version}</subtitle>
</info>
<chapter>
<title>Temporarily unavailable</title>
<para>
The Nixpkgs manual is currently not available in EPUB format,
please use the <link xlink:href="https://nixos.org/nixpkgs/manual">HTML manual</link>
instead.
</para>
<para>
If you've used the EPUB manual in the past and it has been useful to you, please
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/237234">let us know</link>.
</para>
</chapter>
</book>
'';
passAsFile = [ "epub" ];
} ''
mkdir scratch
xsltproc \
--param chapter.autolabel 0 \
--nonet \
--output scratch/ \
${pkgs.docbook_xsl_ns}/xml/xsl/docbook/epub/docbook.xsl \
$epubPath
echo "application/epub+zip" > mimetype
zip -0Xq "$out" mimetype
cd scratch && zip -Xr9D "$out" *
'';
# NB: This file describes the Nixpkgs manual, which happens to use module
# docs infra originally developed for NixOS.
optionsDoc = pkgs.nixosOptionsDoc {
inherit (pkgs.lib.evalModules {
modules = [ ../pkgs/top-level/config.nix ];
class = "nixpkgsConfig";
}) options;
documentType = "none";
transformOptions = opt:
opt // {
declarations =
map
(decl:
if hasPrefix (toString ../..) (toString decl)
then
let subpath = removePrefix "/" (removePrefix (toString ../.) (toString decl));
in { url = "https://github.com/NixOS/nixpkgs/blob/master/${subpath}"; name = subpath; }
else decl)
opt.declarations;
};
};
in pkgs.stdenv.mkDerivation {
name = "nixpkgs-manual";
nativeBuildInputs = with pkgs; [
nixos-render-docs
];
src = ./.;
postPatch = ''
ln -s ${optionsDoc.optionsJSON}/share/doc/nixos/options.json ./config-options.json
'';
buildPhase = ''
cat \
./functions/library.md.in \
${lib-docs}/index.md \
> ./functions/library.md
substitute ./manual.md.in ./manual.md \
--replace-fail '@MANUAL_VERSION@' '${pkgs.lib.version}'
mkdir -p out/media
mkdir -p out/highlightjs
cp -t out/highlightjs \
${pkgs.documentation-highlighter}/highlight.pack.js \
${pkgs.documentation-highlighter}/LICENSE \
${pkgs.documentation-highlighter}/mono-blue.css \
${pkgs.documentation-highlighter}/loader.js
cp -t out ./style.css ./anchor.min.js ./anchor-use.js
nixos-render-docs manual html \
--manpage-urls ./manpage-urls.json \
--revision ${pkgs.lib.trivial.revisionWithDefault (pkgs.rev or "master")} \
--stylesheet style.css \
--stylesheet highlightjs/mono-blue.css \
--script ./highlightjs/highlight.pack.js \
--script ./highlightjs/loader.js \
--script ./anchor.min.js \
--script ./anchor-use.js \
--toc-depth 1 \
--section-toc-depth 1 \
manual.md \
out/index.html
'';
installPhase = ''
dest="$out/${common.outputPath}"
mkdir -p "$(dirname "$dest")"
mv out "$dest"
mv "$dest/index.html" "$dest/${common.indexPath}"
cp ${epub} "$dest/nixpkgs-manual.epub"
mkdir -p $out/nix-support/
echo "doc manual $dest ${common.indexPath}" >> $out/nix-support/hydra-build-products
echo "doc manual $dest nixpkgs-manual.epub" >> $out/nix-support/hydra-build-products
'';
passthru.tests.manpage-urls = with pkgs; testers.invalidateFetcherByDrvHash
({ name ? "manual_check-manpage-urls"
, script
, urlsFile
}: runCommand name {
nativeBuildInputs = [
cacert
(python3.withPackages (p: with p; [
aiohttp
rich
structlog
]))
];
outputHash = "sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU="; # Empty output
} ''
python3 ${script} ${urlsFile}
touch $out
'') {
script = ./tests/manpage-urls.py;
urlsFile = ./manpage-urls.json;
};
}

View File

@@ -1,54 +0,0 @@
# To build this derivation, run `nix-build -A nixpkgs-manual.epub`
{
lib,
runCommand,
docbook_xsl_ns,
libxslt,
zip,
}:
runCommand "manual.epub"
{
nativeBuildInputs = [
libxslt
zip
];
epub = ''
<book xmlns="http://docbook.org/ns/docbook"
xmlns:xlink="http://www.w3.org/1999/xlink"
version="5.0"
xml:id="nixpkgs-manual">
<info>
<title>Nixpkgs Manual</title>
<subtitle>Version ${lib.version}</subtitle>
</info>
<chapter>
<title>Temporarily unavailable</title>
<para>
The Nixpkgs manual is currently not available in EPUB format,
please use the <link xlink:href="https://nixos.org/nixpkgs/manual">HTML manual</link>
instead.
</para>
<para>
If you've used the EPUB manual in the past and it has been useful to you, please
<link xlink:href="https://github.com/NixOS/nixpkgs/issues/237234">let us know</link>.
</para>
</chapter>
</book>
'';
passAsFile = [ "epub" ];
}
''
mkdir scratch
xsltproc \
--param chapter.autolabel 0 \
--nonet \
--output scratch/ \
${docbook_xsl_ns}/xml/xsl/docbook/epub/docbook.xsl \
$epubPath
echo "application/epub+zip" > mimetype
zip -0Xq -b "$TMPDIR" "$out" mimetype
cd scratch && zip -Xr9D -b "$TMPDIR" "$out" *
''

View File

@@ -1,123 +1,27 @@
# Generates the documentation for library functions via nixdoc.
# To build this derivation, run `nix-build -A nixpkgs-manual.lib-docs`
{
lib,
stdenvNoCC,
nixdoc,
nix,
nixpkgs ? { },
libsets ? [
{
name = "asserts";
description = "assertion functions";
}
{
name = "attrsets";
description = "attribute set functions";
}
{
name = "strings";
description = "string manipulation functions";
}
{
name = "versions";
description = "version string functions";
}
{
name = "trivial";
description = "miscellaneous functions";
}
{
name = "fixedPoints";
baseName = "fixed-points";
description = "explicit recursion functions";
}
{
name = "lists";
description = "list manipulation functions";
}
{
name = "debug";
description = "debugging functions";
}
{
name = "options";
description = "NixOS / nixpkgs option handling";
}
{
name = "path";
description = "path functions";
}
{
name = "filesystem";
description = "filesystem functions";
}
{
name = "fileset";
description = "file set functions";
}
{
name = "sources";
description = "source filtering functions";
}
{
name = "cli";
description = "command-line serialization functions";
}
{
name = "generators";
description = "functions that create file formats from nix data structures";
}
{
name = "gvariant";
description = "GVariant formatted string serialization functions";
}
{
name = "customisation";
description = "Functions to customise (derivation-related) functions, derivatons, or attribute sets";
}
{
name = "meta";
description = "functions for derivation metadata";
}
{
name = "derivations";
description = "miscellaneous derivation-specific functions";
}
],
}:
stdenvNoCC.mkDerivation {
{ pkgs, nixpkgs, libsets }:
with pkgs;
let
locationsJSON = import ./lib-function-locations.nix { inherit pkgs nixpkgs libsets; };
in
stdenv.mkDerivation {
name = "nixpkgs-lib-docs";
src = ../../lib;
src = lib.fileset.toSource {
root = ../..;
fileset = ../../lib;
};
buildInputs = [
nixdoc
nix
];
buildInputs = [ nixdoc ];
installPhase = ''
export NIX_STATE_DIR=$(mktemp -d)
nix-instantiate --eval --strict --json ${./lib-function-locations.nix} \
--arg nixpkgsPath "./." \
--argstr revision ${nixpkgs.rev or "master"} \
--argstr libsetsJSON ${lib.escapeShellArg (builtins.toJSON libsets)} \
--store $(mktemp -d) \
> locations.json
function docgen {
name=$1
baseName=$2
description=$3
# TODO: wrap lib.$name in <literal>, make nixdoc not escape it
if [[ -e "lib/$baseName.nix" ]]; then
nixdoc -c "$name" -d "lib.$name: $description" -l locations.json -f "lib/$baseName.nix" > "$out/$name.md"
if [[ -e "../lib/$baseName.nix" ]]; then
nixdoc -c "$name" -d "lib.$name: $description" -l ${locationsJSON} -f "$baseName.nix" > "$out/$name.md"
else
nixdoc -c "$name" -d "lib.$name: $description" -l locations.json -f "lib/$baseName/default.nix" > "$out/$name.md"
nixdoc -c "$name" -d "lib.$name: $description" -l ${locationsJSON} -f "$baseName/default.nix" > "$out/$name.md"
fi
echo "$out/$name.md" >> "$out/index.md"
}
@@ -128,16 +32,9 @@ stdenvNoCC.mkDerivation {
```{=include=} sections auto-id-prefix=auto-generated
EOF
${lib.concatMapStrings (
{
name,
baseName ? name,
description,
}:
''
docgen ${name} ${baseName} ${lib.escapeShellArg description}
''
) libsets}
${lib.concatMapStrings ({ name, baseName ? name, description }: ''
docgen ${name} ${baseName} ${lib.escapeShellArg description}
'') libsets}
echo '```' >> "$out/index.md"
'';

View File

@@ -1,14 +1,13 @@
{ nixpkgsPath, revision, libsetsJSON }:
{ pkgs, nixpkgs ? { }, libsets }:
let
lib = import (nixpkgsPath + "/lib");
libsets = builtins.fromJSON libsetsJSON;
revision = pkgs.lib.trivial.revisionWithDefault (nixpkgs.rev or "master");
libDefPos = prefix: set:
builtins.concatMap
(name: [{
name = builtins.concatStringsSep "." (prefix ++ [name]);
location = builtins.unsafeGetAttrPos name set;
}] ++ lib.optionals
}] ++ nixpkgsLib.optionals
(builtins.length prefix == 0 && builtins.isAttrs set.${name})
(libDefPos (prefix ++ [name]) set.${name})
) (builtins.attrNames set);
@@ -21,6 +20,8 @@ let
})
(builtins.map (x: x.name) libsets);
nixpkgsLib = pkgs.lib;
flattenedLibSubset = { subsetname, functions }:
builtins.map
(fn: {
@@ -37,13 +38,13 @@ let
substr = builtins.substring prefixLen filenameLen filename;
in substr;
removeNixpkgs = removeFilenamePrefix (builtins.toString nixpkgsPath);
removeNixpkgs = removeFilenamePrefix (builtins.toString pkgs.path);
liblocations =
builtins.filter
(elem: elem.value != null)
(lib.lists.flatten
(locatedlibsets lib));
(nixpkgsLib.lists.flatten
(locatedlibsets nixpkgsLib));
fnLocationRelative = { name, value }:
{
@@ -71,4 +72,4 @@ let
relativeLocs);
in
jsonLocs
pkgs.writeText "locations.json" (builtins.toJSON jsonLocs)

View File

@@ -1,28 +0,0 @@
# To build this derivation, run `nix-build -A nixpkgs-manual.optionsDoc`
{ lib, nixosOptionsDoc }:
let
modules = lib.evalModules {
modules = [ ../../pkgs/top-level/config.nix ];
class = "nixpkgsConfig";
};
root = toString ../..;
transformDeclaration =
decl:
let
declStr = toString decl;
subpath = lib.removePrefix "/" (lib.removePrefix root declStr);
in
assert lib.hasPrefix root declStr;
{
url = "https://github.com/NixOS/nixpkgs/blob/master/${subpath}";
name = subpath;
};
in
nixosOptionsDoc {
inherit (modules) options;
documentType = "none";
transformOptions = opt: opt // { declarations = map transformDeclaration opt.declarations; };
}

View File

@@ -1,106 +0,0 @@
# This file describes the Nixpkgs manual, which happens to use module docs infra originally
# developed for NixOS. To build this derivation, run `nix-build -A nixpkgs-manual`.
#
{
lib,
stdenvNoCC,
callPackage,
documentation-highlighter,
nixos-render-docs,
nixpkgs ? { },
}:
stdenvNoCC.mkDerivation (
finalAttrs:
let
inherit (finalAttrs.finalPackage.optionsDoc) optionsJSON;
inherit (finalAttrs.finalPackage) epub lib-docs pythonInterpreterTable;
in
{
name = "nixpkgs-manual";
nativeBuildInputs = [ nixos-render-docs ];
src = lib.fileset.toSource {
root = ../.;
fileset = lib.fileset.unions [
(lib.fileset.fileFilter (file: file.hasExt "md" || file.hasExt "md.in") ../.)
../style.css
../anchor-use.js
../anchor.min.js
../manpage-urls.json
];
};
postPatch = ''
ln -s ${optionsJSON}/share/doc/nixos/options.json ./config-options.json
'';
buildPhase = ''
substituteInPlace ./languages-frameworks/python.section.md \
--subst-var-by python-interpreter-table "$(<"${pythonInterpreterTable}")"
cat \
./functions/library.md.in \
${lib-docs}/index.md \
> ./functions/library.md
substitute ./manual.md.in ./manual.md \
--replace-fail '@MANUAL_VERSION@' '${lib.version}'
mkdir -p out/media
mkdir -p out/highlightjs
cp -t out/highlightjs \
${documentation-highlighter}/highlight.pack.js \
${documentation-highlighter}/LICENSE \
${documentation-highlighter}/mono-blue.css \
${documentation-highlighter}/loader.js
cp -t out ./style.css ./anchor.min.js ./anchor-use.js
nixos-render-docs manual html \
--manpage-urls ./manpage-urls.json \
--revision ${nixpkgs.rev or "master"} \
--stylesheet style.css \
--stylesheet highlightjs/mono-blue.css \
--script ./highlightjs/highlight.pack.js \
--script ./highlightjs/loader.js \
--script ./anchor.min.js \
--script ./anchor-use.js \
--toc-depth 1 \
--section-toc-depth 1 \
manual.md \
out/index.html
'';
installPhase = ''
dest="$out/share/doc/nixpkgs"
mkdir -p "$(dirname "$dest")"
mv out "$dest"
mv "$dest/index.html" "$dest/manual.html"
cp ${epub} "$dest/nixpkgs-manual.epub"
mkdir -p $out/nix-support/
echo "doc manual $dest manual.html" >> $out/nix-support/hydra-build-products
echo "doc manual $dest nixpkgs-manual.epub" >> $out/nix-support/hydra-build-products
'';
passthru = {
lib-docs = callPackage ./lib-function-docs.nix { inherit nixpkgs; };
epub = callPackage ./epub.nix { };
optionsDoc = callPackage ./options-doc.nix { };
pythonInterpreterTable = callPackage ./python-interpreter-table.nix { };
shell = callPackage ../../pkgs/tools/nix/web-devmode.nix {
buildArgs = "./.";
open = "/share/doc/nixpkgs/manual.html";
};
tests.manpage-urls = callPackage ../tests/manpage-urls.nix { };
};
}
)

View File

@@ -1,64 +0,0 @@
# To build this derivation, run `nix-build -A nixpkgs-manual.pythonInterpreterTable`
{
lib,
writeText,
pkgs,
pythonInterpreters,
}:
let
isPythonInterpreter =
name:
/*
NB: Package names that don't follow the regular expression:
- `python-cosmopolitan` is not part of `pkgs.pythonInterpreters`.
- `_prebuilt` interpreters are used for bootstrapping internally.
- `python3Minimal` contains python packages, left behind conservatively.
- `rustpython` lacks `pythonVersion` and `implementation`.
*/
(lib.strings.match "(pypy|python)([[:digit:]]*)" name) != null;
interpreterName =
pname:
let
cuteName = {
cpython = "CPython";
pypy = "PyPy";
};
interpreter = pkgs.${pname};
in
"${cuteName.${interpreter.implementation}} ${interpreter.pythonVersion}";
interpreters = lib.reverseList (
lib.naturalSort (lib.filter isPythonInterpreter (lib.attrNames pythonInterpreters))
);
aliases =
pname:
lib.attrNames (
lib.filterAttrs (
name: value:
isPythonInterpreter name && name != pname && interpreterName name == interpreterName pname
) pkgs
);
result = map (pname: {
inherit pname;
aliases = aliases pname;
interpreter = interpreterName pname;
}) interpreters;
toMarkdown =
data:
let
line = package: ''
| ${package.pname} | ${lib.concatStringsSep ", " package.aliases or [ ]} | ${package.interpreter} |
'';
in
lib.concatStringsSep "" (map line data);
in
writeText "python-interpreter-table.md" ''
| Package | Aliases | Interpeter |
|---------|---------|------------|
${toMarkdown result}
''

View File

@@ -54,4 +54,4 @@ merge:"diff3"
Nix store paths can be converted to strings by enclosing a derivation attribute like so: `"${drv}"`.
:::
Detailed documentation for each generator can be found [here](#sec-functions-library-generators)
Detailed documentation for each generator can be found in `lib/generators.nix`.

View File

@@ -1,3 +0,0 @@
# CERNLIB {#cernlib-hook}
This hook sets the `CERN`, `CERN_LEVEL`, and `CERN_ROOT` environment variables. They are part of [CERNLIB's build system](https://cernlib.web.cern.ch/install/install.html), and are are needed for some programs to compile correctly.

View File

@@ -1,29 +0,0 @@
# `haredo` {#haredo-hook}
This hook uses [the `haredo` command runner](https://sr.ht/~autumnull/haredo/) to build, check, and install the package. It overrides `buildPhase`, `checkPhase`, and `installPhase` by default.
The hook builds its targets in parallel if [`config.enableParallelBuilding`](#var-stdenv-enableParallelBuilding) is set to `true`.
## `buildPhase` {#haredo-hook-buildPhase}
This phase attempts to build the default target.
[]{#haredo-hook-haredoBuildTargets} Targets can be explicitly set by adding a string to the `haredoBuildTargets` list.
[]{#haredo-hook-dontUseHaredoBuild} This behavior can be disabled by setting `dontUseHaredoBuild` to `true`.
## `checkPhase` {#haredo-hook-checkPhase}
This phase searches for the `check.do` or `test.do` targets, running them if they exist.
[]{#haredo-hook-haredoCheckTargets} Targets can be explicitly set by adding a string to the `haredoCheckTargets` list.
[]{#haredo-hook-dontUseHaredoCheck} This behavior can be disabled by setting `dontUseHaredoCheck` to `true`.
## `installPhase` {#haredo-hook-installPhase}
This phase attempts to build the `install.do` target, if it exists.
[]{#haredo-hook-haredoInstallTargets} Targets can be explicitly set by adding a string to the `haredoInstallTargets` list.
[]{#haredo-hook-dontUseHaredoInstall} This behavior can be disabled by setting `dontUseHaredoInstall` to `true`.

View File

@@ -10,14 +10,11 @@ automake.section.md
autopatchelf.section.md
bmake.section.md
breakpoint.section.md
cernlib.section.md
cmake.section.md
gdk-pixbuf.section.md
ghc.section.md
gnome.section.md
haredo.section.md
installShellFiles.section.md
just.section.md
libiconv.section.md
libxml2.section.md
meson.section.md
@@ -27,13 +24,11 @@ patch-rc-path-hooks.section.md
perl.section.md
pkg-config.section.md
postgresql-test-hook.section.md
premake.section.md
python.section.md
scons.section.md
tetex-tex-live.section.md
unzip.section.md
validatePkgConfig.section.md
versionCheckHook.section.md
waf.section.md
zig.section.md
xcbuild.section.md

View File

@@ -1,105 +1,22 @@
# `installShellFiles` {#installshellfiles}
This hook adds helpers that install artifacts like executable files, manpages
and shell completions.
This hook helps with installing manpages and shell completion files. It exposes 2 shell functions `installManPage` and `installShellCompletion` that can be used from your `postInstall` hook.
It exposes the following functions that can be used from your `postInstall`
hook:
The `installManPage` function takes one or more paths to manpages to install. The manpages must have a section suffix, and may optionally be compressed (with `.gz` suffix). This function will place them into the correct `share/man/man<section>/` directory, in [`outputMan`](#outputman).
## `installBin` {#installshellfiles-installbin}
The `installBin` function takes one or more paths to files to install as
executable files.
This function will place them into [`outputBin`](#outputbin).
### Example Usage {#installshellfiles-installbin-exampleusage}
```nix
{
nativeBuildInputs = [ installShellFiles ];
# Sometimes the file has an undersirable name. It should be renamed before
# being installed via installBin
postInstall = ''
mv a.out delmar
installBin foobar delmar
'';
}
```
## `installManPage` {#installshellfiles-installmanpage}
The `installManPage` function takes one or more paths to manpages to install.
The manpages must have a section suffix, and may optionally be compressed (with
`.gz` suffix). This function will place them into the correct
`share/man/man<section>/` directory in [`outputMan`](#outputman).
### Example Usage {#installshellfiles-installmanpage-exampleusage}
```nix
{
nativeBuildInputs = [ installShellFiles ];
# Sometimes the manpage file has an undersirable name; e.g. it conflicts with
# another software with an equal name. It should be renamed before being
# installed via installManPage
postInstall = ''
mv fromsea.3 delmar.3
installManPage foobar.1 delmar.3
'';
}
```
## `installShellCompletion` {#installshellfiles-installshellcompletion}
The `installShellCompletion` function takes one or more paths to shell
completion files.
By default it will autodetect the shell type from the completion file extension,
but you may also specify it by passing one of `--bash`, `--fish`, or
`--zsh`. These flags apply to all paths listed after them (up until another
shell flag is given). Each path may also have a custom installation name
provided by providing a flag `--name NAME` before the path. If this flag is not
provided, zsh completions will be renamed automatically such that `foobar.zsh`
becomes `_foobar`. A root name may be provided for all paths using the flag
`--cmd NAME`; this synthesizes the appropriate name depending on the shell
(e.g. `--cmd foo` will synthesize the name `foo.bash` for bash and `_foo` for
zsh).
### Example Usage {#installshellfiles-installshellcompletion-exampleusage}
The `installShellCompletion` function takes one or more paths to shell completion files. By default it will autodetect the shell type from the completion file extension, but you may also specify it by passing one of `--bash`, `--fish`, or `--zsh`. These flags apply to all paths listed after them (up until another shell flag is given). Each path may also have a custom installation name provided by providing a flag `--name NAME` before the path. If this flag is not provided, zsh completions will be renamed automatically such that `foobar.zsh` becomes `_foobar`. A root name may be provided for all paths using the flag `--cmd NAME`; this synthesizes the appropriate name depending on the shell (e.g. `--cmd foo` will synthesize the name `foo.bash` for bash and `_foo` for zsh). The path may also be a fifo or named fd (such as produced by `<(cmd)`), in which case the shell and name must be provided.
```nix
{
nativeBuildInputs = [ installShellFiles ];
postInstall = ''
installManPage doc/foobar.1 doc/barfoo.3
# explicit behavior
installShellCompletion --bash --name foobar.bash share/completions.bash
installShellCompletion --fish --name foobar.fish share/completions.fish
installShellCompletion --zsh --name _foobar share/completions.zsh
# implicit behavior
installShellCompletion share/completions/foobar.{bash,fish,zsh}
'';
}
```
The path may also be a fifo or named fd (such as produced by `<(cmd)`), in which
case the shell and name must be provided (see below).
If the destination shell completion file is not actually present or consists of
zero bytes after calling `installShellCompletion` this is treated as a build
failure. In particular, if completion files are not vendored but are generated
by running an executable, this is likely to fail in cross compilation
scenarios. The result will be a zero byte completion file and hence a build
failure. To prevent this, guard the completion generation commands.
### Example Usage {#installshellfiles-installshellcompletion-exampleusage-guarded}
```nix
{
nativeBuildInputs = [ installShellFiles ];
postInstall = lib.optionalString (stdenv.buildPlatform.canExecute stdenv.hostPlatform) ''
# using named fd
installShellCompletion --cmd foobar \
--bash <($out/bin/foobar --bash-completion) \

View File

@@ -1,23 +0,0 @@
# `just` {#just-hook}
This setup hook attempts to use [the `just` command runner](https://just.systems/man/en/) to build, check, and install the package. The hook overrides `buildPhase`, `checkPhase`, and `installPhase` by default.
[]{#just-hook-justFlags} The `justFlags` variable can be set to a list of strings to add additional flags passed to all invocations of `just`.
## `buildPhase` {#just-hook-buildPhase}
This phase attempts to invoke `just` with [the default recipe](https://just.systems/man/en/chapter_23.html).
[]{#just-hook-dontUseJustBuild} This behavior can be disabled by setting `dontUseJustBuild` to `true`.
## `checkPhase` {#just-hook-checkPhase}
This phase attempts to invoke the `just test` recipe, if it is available. This can be overrided by setting `checkTarget` to a string.
[]{#just-hook-dontUseJustCheck} This behavior can be disabled by setting `dontUseJustCheck` to `true`.
## `installPhase` {#just-hook-installPhase}
This phase attempts to invoke the `just install` recipe.
[]{#just-hook-dontUseJustInstall} This behavior can be disabled by setting `dontUseJustInstall` to `true`.

View File

@@ -1,7 +0,0 @@
# Premake {#premake-hook}
This setup hook attempts to configure the package using [the Premake build configuration system](https://premake.github.io/). It overrides the `configurePhase` by default, if none exists.
[]{#premake-hook-premakefile} The Premakefile to use can be specified by setting `premakefile` in the derivation.
[]{#premake-hook-premakeFlagsArray} The flags passed to Premake can be configured by adding strings to the `premakeFlags` list.

View File

@@ -1,35 +0,0 @@
# versionCheckHook {#versioncheckhook}
This hook adds a `versionCheckPhase` to the [`preInstallCheckHooks`](#ssec-installCheck-phase) that runs the main program of the derivation with a `--help` or `--version` argument, and checks that the `${version}` string is found in that output. You use it like this:
```nix
{
lib,
stdenv,
versionCheckHook,
# ...
}:
stdenv.mkDerivation (finalAttrs: {
# ...
nativeInstallCheckInputs = [
versionCheckHook
];
doInstallCheck = true;
# ...
})
```
Note that for [`buildPythonPackage`](#buildpythonpackage-function) and [`buildPythonApplication`](#buildpythonapplication-function), `doInstallCheck` is enabled by default.
It does so in a clean environment (using `env --ignore-environment`), and it checks for the `${version}` string in both the `stdout` and the `stderr` of the command. It will report to you in the build log the output it received and it will fail the build if it failed to find `${version}`.
The variables that this phase control are:
- `dontVersionCheck`: Disable adding this hook to the [`preDistPhases`](#var-stdenv-preDist). Useful if you do want to load the bash functions of the hook, but run them differently.
- `versionCheckProgram`: The full path to the program that should print the `${version}` string. Defaults roughly to `${placeholder "out"}/bin/${pname}`. Using `$out` in the value of this variable won't work, as environment variables from this variable are not expanded by the hook. Hence using `placeholder` is unavoidable.
- `versionCheckProgramArg`: The argument that needs to be passed to `versionCheckProgram`. If undefined the hook tries first `--help` and then `--version`. Examples: `version`, `-V`, `-v`.
- `preVersionCheck`: A hook to run before the check is done.
- `postVersionCheck`: A hook to run after the check is done.

View File

@@ -20,6 +20,10 @@ If `wafPath` doesn't exist, then `wafHook` will copy the `waf` provided from Nix
Controls the flags passed to waf tool during build and install phases. For settings specific to build or install phases, use `wafBuildFlags` or `wafInstallFlags` respectively.
#### `dontAddWafCrossFlags` {#dont-add-waf-cross-flags}
When set to `true`, don't add cross compilation flags during configure phase.
#### `dontUseWafConfigure` {#dont-use-waf-configure}
When set to true, don't use the predefined `wafConfigurePhase`.

View File

@@ -1,5 +0,0 @@
# Interoperability Standards {#part-interoperability}
```{=include=} chapters
interoperability/cyclonedx.md
```

View File

@@ -1,79 +0,0 @@
# CycloneDX {#chap-interop-cyclonedx}
[OWASP](https://owasp.org/) [CycloneDX](https://cyclonedx.org/) is a Software [Bill of Materials](https://en.wikipedia.org/wiki/Bill_of_materials) (SBOM) standard.
The standards described here are for including Nix specific information within SBOMs in a way that is interoperable with external SBOM tooling.
## `nix` Namespace Property Taxonomy {#sec-interop.cylonedx-nix}
The following tables describe namespaces for [properties](https://cyclonedx.org/docs/1.6/json/#components_items_properties) that may be attached to components within SBOMs.
Component properties are lists of name-value-pairs where values must be strings.
Properties with the same name may appear more than once.
Names and values are case-sensitive.
| Property | Description |
|------------------|-------------|
| `nix:store_path` | A Nix store path for the given component. This property should be contextualized by additional properties that describe the production of the store path, such as those from the `nix:narinfo:` and `nix:fod` namespaces. |
| Namespace | Description |
|---------------|-------------|
| [`nix:narinfo`](#sec-interop.cylonedx-narinfo) | Namespace for properties that are specific to how a component is stored as a [Nix archive](https://nixos.org/manual/nix/stable/glossary#gloss-nar) (NAR) in a [binary cache](https://nixos.org/manual/nix/stable/glossary#gloss-binary-cache). |
| [`nix:fod`](#sec-interop.cylonedx-fod) | Namespace for properties that describe a [fixed-output derivation](https://nixos.org/manual/nix/stable/glossary#gloss-fixed-output-derivation). |
### `nix:narinfo` {#sec-interop.cylonedx-narinfo}
Narinfo properties describe component archives that may be available from binary caches.
The `nix:narinfo` properties should be accompanied by a `nix:store_path` property within the same property list.
| Property | Description |
|---------------------------|-------------|
| `nix:narinfo:store_path` | Store path for the given store component. |
| `nix:narinfo:url` | URL path component. |
| `nix:narinfo:nar_hash` | Hash of the file system object part of the component when serialized as a Nix Archive. |
| `nix:narinfo:nar_size` | Size of the component when serialized as a Nix Archive. |
| `nix:narinfo:compression` | The compression format that component archive is in. |
| `nix:narinfo:file_hash` | A digest for the compressed component archive itself, as opposed to the data contained within. |
| `nix:narinfo:file_size` | The size of the compressed component archive itself. |
| `nix:narinfo:deriver` | The path to the derivation from which this component is produced. |
| `nix:narinfo:system` | The hardware and software platform on which this component is produced. |
| `nix:narinfo:sig` | Signatures claiming that this component is what it claims to be. |
| `nix:narinfo:ca` | Content address of this store object's file system object, used to compute its store path. |
| `nix:narinfo:references` | A whitespace separated array of store paths that this component references. |
### `nix:fod` {#sec-interop.cylonedx-fod}
FOD properties describe a [fixed-output derivation](https://nixos.org/manual/nix/stable/glossary#gloss-fixed-output-derivation).
The `nix:fod:method` property is required and must be accompanied by a `nix:store_path` property within the same property list.
All other properties in this namespace are method-specific.
To reproduce the build of a component the `nix:fod:method` value is resolved to an [appropriate function](#chap-pkgs-fetchers) within Nixpkgs whose arguments intersect with the given properties.
When generating `nix:fod` properties the method selected should be a stable function with a minimal number arguments.
For example, the `fetchFromGitHub` is commonly used within Nixpkgs but should be reduced to a call to the function by which it is implemented, `fetchzip`.
| Property | Description |
|------------------|-------------|
| `nix:fod:method` | Nixpkg function that produces this FOD. Required. Examples: `"fetchzip"`, `"fetchgit"` |
| `nix:fod:name` | Derivation name, present when method is `"fetchzip"` |
| `nix:fod:ref` | [Git ref](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefrefaref), present when method is `"fetchgit"` |
| `nix:fod:rev` | [Git rev](https://git-scm.com/docs/gitglossary#Documentation/gitglossary.txt-aiddefrevisionarevision), present when method is `"fetchgit"` |
| `nix:fod:sha256` | FOD hash |
| `nix:fod:url` | URL to fetch |
`nix:fod` properties may be extracted and evaluated to a derivation using code similar to the following, assuming a fictitious function `filterPropertiesToAttrs`:
```nix
{ pkgs, filterPropertiesToAttrs, properties }:
let
fodProps = filterPropertiesToAttrs "nix:fod:" properties;
methods = {
fetchzip =
{ name, url, sha256, ... }:
pkgs.fetchzip {
inherit name url sha256;
};
};
in methods.${fodProps.method} fodProps
```

View File

@@ -3,36 +3,10 @@
The Android build environment provides three major features and a number of
supporting features.
## Using androidenv with Android Studio {#using-androidenv-with-android-studio}
Use the `android-studio-full` attribute for a very complete Android SDK, including system images:
```nix
buildInputs = [ android-studio-full ];
```
This is identical to:
```nix
buildInputs = [ androidStudioPackages.stable.full ];
```
Alternatively, you can pass composeAndroidPackages to the `withSdk` passthru:
```nix
buildInputs = [
(android-studio.withSdk (androidenv.composeAndroidPackages {
includeNDK = true;
}).androidsdk)
];
```
These will export ANDROID_SDK_ROOT and ANDROID_NDK_ROOT to the SDK and NDK directories
in the specified Android build environment.
## Deploying an Android SDK installation with plugins {#deploying-an-android-sdk-installation-with-plugins}
Alternatively, you can deploy the SDK separately with a desired set of plugins, or subsets of an SDK.
The first use case is deploying the SDK with a desired set of plugins or subsets
of an SDK.
```nix
with import <nixpkgs> {};
@@ -171,14 +145,16 @@ androidComposition.platform-tools
## Using predefined Android package compositions {#using-predefined-android-package-compositions}
In addition to composing an Android package set manually, it is also possible
to use a predefined composition that contains a fairly complete set of Android packages:
to use a predefined composition that contains all basic packages for a specific
Android version, such as version 9.0 (API-level 28).
The following Nix expression can be used to deploy the entire SDK:
The following Nix expression can be used to deploy the entire SDK with all basic
plugins:
```nix
with import <nixpkgs> {};
androidenv.androidPkgs.androidsdk
androidenv.androidPkgs_9_0.androidsdk
```
It is also possible to use one plugin only:
@@ -186,9 +162,50 @@ It is also possible to use one plugin only:
```nix
with import <nixpkgs> {};
androidenv.androidPkgs.platform-tools
androidenv.androidPkgs_9_0.platform-tools
```
## Building an Android application {#building-an-android-application}
In addition to the SDK, it is also possible to build an Ant-based Android
project and automatically deploy all the Android plugins that a project
requires.
```nix
with import <nixpkgs> {};
androidenv.buildApp {
name = "MyAndroidApp";
src = ./myappsources;
release = true;
# If release is set to true, you need to specify the following parameters
keyStore = ./keystore;
keyAlias = "myfirstapp";
keyStorePassword = "mykeystore";
keyAliasPassword = "myfirstapp";
# Any Android SDK parameters that install all the relevant plugins that a
# build requires
platformVersions = [ "24" ];
# When we include the NDK, then ndk-build is invoked before Ant gets invoked
includeNDK = true;
}
```
Aside from the app-specific build parameters (`name`, `src`, `release` and
keystore parameters), the `buildApp {}` function supports all the function
parameters that the SDK composition function (the function shown in the
previous section) supports.
This build function is particularly useful when it is desired to use
[Hydra](https://nixos.org/hydra): the Nix-based continuous integration solution
to build Android apps. An Android APK gets exposed as a build product and can be
installed on any Android device with a web browser by navigating to the build
result page.
## Spawning emulator instances {#spawning-emulator-instances}
For testing purposes, it can also be quite convenient to automatically generate
@@ -232,11 +249,11 @@ In addition to prebuilt APKs, you can also bind the APK parameter to a
## Notes on environment variables in Android projects {#notes-on-environment-variables-in-android-projects}
* `ANDROID_HOME` should point to the Android SDK. In your Nix expressions, this should be
`${androidComposition.androidsdk}/libexec/android-sdk`. Note that `ANDROID_SDK_ROOT` is deprecated,
* `ANDROID_SDK_ROOT` should point to the Android SDK. In your Nix expressions, this should be
`${androidComposition.androidsdk}/libexec/android-sdk`. Note that `ANDROID_HOME` is deprecated,
but if you rely on tools that need it, you can export it too.
* `ANDROID_NDK_ROOT` should point to the Android NDK, if you're doing NDK development.
In your Nix expressions, this should be `${ANDROID_HOME}/ndk-bundle`.
In your Nix expressions, this should be `${ANDROID_SDK_ROOT}/ndk-bundle`.
If you are running the Android Gradle plugin, you need to export GRADLE_OPTS to override aapt2
to point to the aapt2 binary in the Nix store as well, or use a FHS environment so the packaged
@@ -250,11 +267,11 @@ let
androidComposition = <...>;
in
pkgs.mkShell rec {
ANDROID_HOME = "${androidComposition.androidsdk}/libexec/android-sdk";
ANDROID_NDK_ROOT = "${ANDROID_HOME}/ndk-bundle";
ANDROID_SDK_ROOT = "${androidComposition.androidsdk}/libexec/android-sdk";
ANDROID_NDK_ROOT = "${ANDROID_SDK_ROOT}/ndk-bundle";
# Use the same buildToolsVersion here
GRADLE_OPTS = "-Dorg.gradle.project.android.aapt2FromMavenOverride=${ANDROID_HOME}/build-tools/${buildToolsVersion}/aapt2";
GRADLE_OPTS = "-Dorg.gradle.project.android.aapt2FromMavenOverride=${ANDROID_SDK_ROOT}/build-tools/${buildToolsVersion}/aapt2";
}
```
@@ -270,18 +287,18 @@ let
androidComposition = <...>;
in
pkgs.mkShell rec {
ANDROID_HOME = "${androidComposition.androidsdk}/libexec/android-sdk";
ANDROID_NDK_ROOT = "${ANDROID_HOME}/ndk-bundle";
ANDROID_SDK_ROOT = "${androidComposition.androidsdk}/libexec/android-sdk";
ANDROID_NDK_ROOT = "${ANDROID_SDK_ROOT}/ndk-bundle";
# Use the same cmakeVersion here
shellHook = ''
export PATH="$(echo "$ANDROID_HOME/cmake/${cmakeVersion}".*/bin):$PATH"
export PATH="$(echo "$ANDROID_SDK_ROOT/cmake/${cmakeVersion}".*/bin):$PATH"
'';
}
```
Note that running Android Studio with ANDROID_HOME set will automatically write a
`local.properties` file with `sdk.dir` set to $ANDROID_HOME if one does not already
Note that running Android Studio with ANDROID_SDK_ROOT set will automatically write a
`local.properties` file with `sdk.dir` set to $ANDROID_SDK_ROOT if one does not already
exist. If you are using the NDK as well, you may have to add `ndk.dir` to this file.
An example shell.nix that does all this for you is provided in examples/shell.nix.
@@ -332,44 +349,3 @@ To update the expressions run the `generate.sh` script that is stored in the
```bash
./generate.sh
```
## Building an Android application with Ant {#building-an-android-application-with-ant}
In addition to the SDK, it is also possible to build an Ant-based Android
project and automatically deploy all the Android plugins that a project
requires. Most newer Android projects use Gradle, and this is included for historical
purposes.
```nix
with import <nixpkgs> {};
androidenv.buildApp {
name = "MyAndroidApp";
src = ./myappsources;
release = true;
# If release is set to true, you need to specify the following parameters
keyStore = ./keystore;
keyAlias = "myfirstapp";
keyStorePassword = "mykeystore";
keyAliasPassword = "myfirstapp";
# Any Android SDK parameters that install all the relevant plugins that a
# build requires
platformVersions = [ "24" ];
# When we include the NDK, then ndk-build is invoked before Ant gets invoked
includeNDK = true;
}
```
Aside from the app-specific build parameters (`name`, `src`, `release` and
keystore parameters), the `buildApp {}` function supports all the function
parameters that the SDK composition function (the function shown in the
previous section) supports.
This build function is particularly useful when it is desired to use
[Hydra](https://nixos.org/hydra): the Nix-based continuous integration solution
to build Android apps. An Android APK gets exposed as a build product and can be
installed on any Android device with a web browser by navigating to the build
result page.

View File

@@ -23,13 +23,12 @@ The recommended way of defining a derivation for a Coq library, is to use the `c
* if it is a string of the form `owner:branch` then it tries to download the `branch` of owner `owner` for a project of the same name using the same vcs, and the `version` attribute of the resulting derivation is set to `"dev"`, additionally if the owner is not provided (i.e. if the `owner:` prefix is missing), it defaults to the original owner of the package (see below),
* if it is a string of the form `"#N"`, and the domain is github, then it tries to download the current head of the pull request `#N` from github,
* `defaultVersion` (optional). Coq libraries may be compatible with some specific versions of Coq only. The `defaultVersion` attribute is used when no `version` is provided (or if `version = null`) to select the version of the library to use by default, depending on the context. This selection will mainly depend on a `coq` version number but also possibly on other packages versions (e.g. `mathcomp`). If its value ends up to be `null`, the package is marked for removal in end-user `coqPackages` attribute set.
* `release` (optional, defaults to `{}`), lists all the known releases of the library and for each of them provides an attribute set with at least a `hash` attribute (you may put the empty string `""` in order to automatically insert a fake hash, this will trigger an error which will allow you to find the correct hash), each attribute set of the list of releases also takes optional overloading arguments for the fetcher as below (i.e.`domain`, `owner`, `repo`, `rev`, `artifact` assuming the default fetcher is used) and optional overrides for the result of the fetcher (i.e. `version` and `src`).
* `release` (optional, defaults to `{}`), lists all the known releases of the library and for each of them provides an attribute set with at least a `sha256` attribute (you may put the empty string `""` in order to automatically insert a fake sha256, this will trigger an error which will allow you to find the correct sha256), each attribute set of the list of releases also takes optional overloading arguments for the fetcher as below (i.e.`domain`, `owner`, `repo`, `rev` assuming the default fetcher is used) and optional overrides for the result of the fetcher (i.e. `version` and `src`).
* `fetcher` (optional, defaults to a generic fetching mechanism supporting github or gitlab based infrastructures), is a function that takes at least an `owner`, a `repo`, a `rev`, and a `hash` and returns an attribute set with a `version` and `src`.
* `repo` (optional, defaults to the value of `pname`),
* `owner` (optional, defaults to `"coq-community"`).
* `domain` (optional, defaults to `"github.com"`), domains including the strings `"github"` or `"gitlab"` in their names are automatically supported, otherwise, one must change the `fetcher` argument to support them (cf `pkgs/development/coq-modules/heq/default.nix` for an example),
* `releaseRev` (optional, defaults to `(v: v)`), provides a default mapping from release names to revision hashes/branch names/tags,
* `releaseArtifact` (optional, defaults to `(v: null)`), provides a default mapping from release names to artifact names (only works for github artifact for now),
* `displayVersion` (optional), provides a way to alter the computation of `name` from `pname`, by explaining how to display version numbers,
* `namePrefix` (optional, defaults to `[ "coq" ]`), provides a way to alter the computation of `name` from `pname`, by explaining which dependencies must occur in `name`,
* `nativeBuildInputs` (optional), is a list of executables that are required to build the current derivation, in addition to the default ones (namely `which`, `dune` and `ocaml` depending on whether `useDune`, `useDuneifVersion` and `mlPlugin` are set).
@@ -70,22 +69,22 @@ mkCoqDerivation {
{ cases = [ (isEq "8.6") (range "1.6" "1.7") ]; out = "1.1"; }
] null;
release = {
"1.5.2".hash = "sha256-mjCx9XKa38Nz9E6wNK7YSqHdJ7YTua5fD3d6J4e7WpU=";
"1.5.1".hash = "sha256-Q8tm0y2FQAt2V1kZYkDlHWRia/lTvXAMVjdmzEV11I4=";
"1.5.0".hash = "sha256-HIK0f21G69oEW8JG46gSBde/Q2LR3GiBCv680gHbmRg=";
"1.5.0".rev = "1.5";
"1.4".hash = "sha256-F9g3MSIr3B6UZ3p8QWjz3/Jpw9sudJ+KRlvjiHSO024=";
"1.3".hash = "sha256-BPJTlAL0ETHvLMBslE0KFVt3DNoaGuMrHt2SBGyJe1A=";
"1.2".hash = "sha256-mHXBXSLYO4BN+jfN50y/+XCx0Qq5g4Ac2Y/qlsbgAdY=";
"1.1".hash = "sha256-ejAsMQbB/LtU9j+g160VdGXULrCe9s0gBWzyhKqmCuE=";
"1.0".hash = "sha256-tZTOltEBBKWciDxDMs/Ye4Jnq/33CANrHJ4FBMPtq+I=";
"1.5.2".sha256 = "15aspf3jfykp1xgsxf8knqkxv8aav2p39c2fyirw7pwsfbsv2c4s";
"1.5.1".sha256 = "13nlfm2wqripaq671gakz5mn4r0xwm0646araxv0nh455p9ndjs3";
"1.5.0".sha256 = "064rvc0x5g7y1a0nip6ic91vzmq52alf6in2bc2dmss6dmzv90hw";
"1.5.0".rev = "1.5";
"1.4".sha256 = "0vnkirs8iqsv8s59yx1fvg1nkwnzydl42z3scya1xp1b48qkgn0p";
"1.3".sha256 = "0l3vi5n094nx3qmy66hsv867fnqm196r8v605kpk24gl0aa57wh4";
"1.2".sha256 = "1mh1w339dslgv4f810xr1b8v2w7rpx6fgk9pz96q0fyq49fw2xcq";
"1.1".sha256 = "1q8alsm89wkc0lhcvxlyn0pd8rbl2nnxg81zyrabpz610qqjqc3s";
"1.0".sha256 = "1qmbxp1h81cy3imh627pznmng0kvv37k4hrwi2faa101s6bcx55m";
};
propagatedBuildInputs =
[ mathcomp.ssreflect mathcomp.algebra mathcomp-finmap mathcomp-bigenough ];
meta = {
description = "Coq/SSReflect Library for Monoidal Rings and Multinomials";
description = "A Coq/SSReflect Library for Monoidal Rings and Multinomials";
license = lib.licenses.cecill-c;
};
}
@@ -127,7 +126,7 @@ For example, here is how you could locally add a new release of the `multinomial
coqPackages.lib.overrideCoqDerivation
{
defaultVersion = "2.0";
release."2.0".hash = "sha256-czoP11rtrIM7+OLdMisv2EF7n/IbGuwFxHiPtg3qCNM=";
release."2.0".sha256 = "1lq8x86vd3vqqh2yq6hvyagpnhfq5wmk5pg2z0xq7b7dbbbhyfkk";
}
coqPackages.multinomials
```

View File

@@ -98,12 +98,10 @@ The function `buildFlutterApplication` builds Flutter applications.
See the [Dart documentation](#ssec-dart-applications) for more details on required files and arguments.
`flutter` in Nixpkgs always points to `flutterPackages.stable`, which is the latest packaged version. To avoid unforeseen breakage during upgrade, packages in Nixpkgs should use a specific flutter version, such as `flutter319` and `flutter322`, instead of using `flutter` directly.
```nix
{ flutter322, fetchFromGitHub }:
{ flutter, fetchFromGitHub }:
flutter322.buildFlutterApplication {
flutter.buildFlutterApplication {
pname = "firmware-updater";
version = "0-unstable-2023-04-30";
@@ -114,7 +112,7 @@ flutter322.buildFlutterApplication {
owner = "canonical";
repo = "firmware-updater";
rev = "6e7dbdb64e344633ea62874b54ff3990bd3b8440";
hash = "sha256-s5mwtr5MSPqLMN+k851+pFIFFPa0N1hqz97ys050tFA=";
sha256 = "sha256-s5mwtr5MSPqLMN+k851+pFIFFPa0N1hqz97ys050tFA=";
fetchSubmodules = true;
};

View File

@@ -93,18 +93,18 @@ The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given
To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions:
* `projectFile` is used for specifying the dotnet project file, relative to the source root. These have `.sln` (entire solution) or `.csproj` (single project) file extensions. This can be a list of multiple projects as well. When omitted, will attempt to find and build the solution (`.sln`). If running into problems, make sure to set it to a file (or a list of files) with the `.csproj` extension - building applications as entire solutions is not fully supported by the .NET CLI.
* `nugetDeps` takes either a path to a `deps.nix` file, or a derivation. The `deps.nix` file can be generated using the script attached to `passthru.fetch-deps`. For compatibility, if the argument is a list of derivations, they will be added to `buildInputs`.
* `nugetDeps` takes either a path to a `deps.nix` file, or a derivation. The `deps.nix` file can be generated using the script attached to `passthru.fetch-deps`. If the argument is a derivation, it will be used directly and assume it has the same output as `mkNugetDeps`.
::: {.note}
For more detail about managing the `deps.nix` file, see [Generating and updating NuGet dependencies](#generating-and-updating-nuget-dependencies)
:::
* `packNupkg` is used to pack project as a `nupkg`, and installs it to `$out/share`. If set to `true`, the derivation can be used as a dependency for another dotnet project by adding it to `buildInputs`.
* `buildInputs` can be used to resolve `ProjectReference` project items. Referenced projects can be packed with `buildDotnetModule` by setting the `packNupkg = true` attribute and passing a list of derivations to `buildInputs`. Since we are sharing referenced projects as NuGets they must be added to csproj/fsproj files as `PackageReference` as well.
* `packNupkg` is used to pack project as a `nupkg`, and installs it to `$out/share`. If set to `true`, the derivation can be used as a dependency for another dotnet project by adding it to `projectReferences`.
* `projectReferences` can be used to resolve `ProjectReference` project items. Referenced projects can be packed with `buildDotnetModule` by setting the `packNupkg = true` attribute and passing a list of derivations to `projectReferences`. Since we are sharing referenced projects as NuGets they must be added to csproj/fsproj files as `PackageReference` as well.
For example, your project has a local dependency:
```xml
<ProjectReference Include="../foo/bar.fsproj" />
```
To enable discovery through `buildInputs` you would need to add:
To enable discovery through `projectReferences` you would need to add:
```xml
<ProjectReference Include="../foo/bar.fsproj" />
<PackageReference Include="bar" Version="*" Condition=" '$(ContinuousIntegrationBuild)'=='true' "/>
@@ -117,8 +117,8 @@ For more detail about managing the `deps.nix` file, see [Generating and updating
* `useDotnetFromEnv` will change the binary wrapper so that it uses the .NET from the environment. The runtime specified by `dotnet-runtime` is given as a fallback in case no .NET is installed in the user's environment. This is most useful for .NET global tools and LSP servers, which often extend the .NET CLI and their runtime should match the users' .NET runtime.
* `dotnet-sdk` is useful in cases where you need to change what dotnet SDK is being used. You can also set this to the result of `dotnetSdkPackages.combinePackages`, if the project uses multiple SDKs to build.
* `dotnet-runtime` is useful in cases where you need to change what dotnet runtime is being used. This can be either a regular dotnet runtime, or an aspnetcore.
* `dotnet-test-sdk` is useful in cases where unit tests expect a different dotnet SDK. By default, this is set to the `dotnet-sdk` attribute.
* `testProjectFile` is useful in cases where the regular project file does not contain the unit tests. It gets restored and build, but not installed. You may need to regenerate your nuget lockfile after setting this. Note that if set, only tests from this project are executed.
* `testFilters` is used to disable running unit tests based on various [filters](https://docs.microsoft.com/en-us/dotnet/core/tools/dotnet-test#filter-option-details). This gets passed as: `dotnet test --filter "{}"`, with each filter being concatenated using `"&"`.
* `disabledTests` is used to disable running specific unit tests. This gets passed as: `dotnet test --filter "FullyQualifiedName!={}"`, to ensure compatibility with all unit test frameworks.
* `dotnetRestoreFlags` can be used to pass flags to `dotnet restore`.
* `dotnetBuildFlags` can be used to pass flags to `dotnet build`.
@@ -142,9 +142,11 @@ in buildDotnetModule rec {
src = ./.;
projectFile = "src/project.sln";
nugetDeps = ./deps.nix; # see "Generating and updating NuGet dependencies" section for details
# File generated with `nix-build -A package.passthru.fetch-deps`.
# To run fetch-deps when this file does not yet exist, set nugetDeps to null
nugetDeps = ./deps.nix;
buildInputs = [ referencedProject ]; # `referencedProject` must contain `nupkg` in the folder structure.
projectReferences = [ referencedProject ]; # `referencedProject` must contain `nupkg` in the folder structure.
dotnet-sdk = dotnetCorePackages.sdk_6_0;
dotnet-runtime = dotnetCorePackages.runtime_6_0;
@@ -195,7 +197,7 @@ This helper has the same arguments as `buildDotnetModule`, with a few difference
* `pname` and `version` are required, and will be used to find the NuGet package of the tool
* `nugetName` can be used to override the NuGet package name that will be downloaded, if it's different from `pname`
* `nugetHash` is the hash of the fetched NuGet package. `nugetSha256` is also supported, but not recommended. Set this to `lib.fakeHash` for the first build, and it will error out, giving you the proper hash. Also remember to update it during version updates (it will not error out if you just change the version while having a fetched package in `/nix/store`)
* `nugetSha256` is the hash of the fetched NuGet package. Set this to `lib.fakeHash256` for the first build, and it will error out, giving you the proper hash. Also remember to update it during version updates (it will not error out if you just change the version while having a fetched package in `/nix/store`)
* `dotnet-runtime` is set to `dotnet-sdk` by default. When changing this, remember that .NET tools fetched from NuGet require an SDK.
Here is an example of packaging `pbm`, an unfree binary without source available:
@@ -206,7 +208,7 @@ buildDotnetGlobalTool {
pname = "pbm";
version = "1.3.1";
nugetHash = "sha256-ZG2HFyKYhVNVYd2kRlkbAjZJq88OADe3yjxmLuxXDUo=";
nugetSha256 = "sha256-ZG2HFyKYhVNVYd2kRlkbAjZJq88OADe3yjxmLuxXDUo=";
meta = {
homepage = "https://cmd.petabridge.com/index.html";
@@ -218,12 +220,6 @@ buildDotnetGlobalTool {
```
## Generating and updating NuGet dependencies {#generating-and-updating-nuget-dependencies}
When writing a new expression, you can use the generated `fetch-deps` script to initialise the lockfile.
After setting `nugetDeps` to the desired location of the lockfile (e.g. `./deps.nix`),
build the script with `nix-build -A package.fetch-deps` and then run the result.
(When the root attr is your package, it's simply `nix-build -A fetch-deps`.)
There is also a manual method:
First, restore the packages to the `out` directory, ensure you have cloned
the upstream repository and you are inside it.
@@ -242,15 +238,15 @@ $ nuget-to-nix out > deps.nix
Which `nuget-to-nix` will generate an output similar to below
```nix
{ fetchNuGet }: [
(fetchNuGet { pname = "FosterFramework"; version = "0.1.15-alpha"; hash = "sha256-lM6eYgOGjl1fx6WFD7rnRi/YAQieM0mx60h0p5dr+l8="; })
(fetchNuGet { pname = "Microsoft.AspNetCore.App.Runtime.linux-x64"; version = "8.0.1"; hash = "sha256-QbUQXjCzr8j8u/5X0af9jE++EugdoxMhT08F49MZX74="; })
(fetchNuGet { pname = "Microsoft.NET.ILLink.Tasks"; version = "8.0.1"; hash = "sha256-SopZpGaZ48/8dpUwDFDM3ix+g1rP4Yqs1PGuzRp+K7c="; })
(fetchNuGet { pname = "Microsoft.NETCore.App.Runtime.linux-x64"; version = "8.0.1"; hash = "sha256-jajBI5GqG2IIcsIMgxTHfXbMapoXrZGl/EEhShwYq7w="; })
(fetchNuGet { pname = "SharpGLTF.Core"; version = "1.0.0-alpha0031"; hash = "sha256-Bs4baD5wNIH6wAbGK4Xaem0i3luQkOQs37izBWdFx1I="; })
(fetchNuGet { pname = "SharpGLTF.Runtime"; version = "1.0.0-alpha0031"; hash = "sha256-TwJO6b8ubmwBQh6NyHha8+JT5zHDJ4dROBbsEbUaa1M="; })
(fetchNuGet { pname = "Sledge.Formats"; version = "1.2.2"; hash = "sha256-0Ddhuwpu3wwIzA4NuPaEVdMkx6tUukh8uKD6nKoxFPg="; })
(fetchNuGet { pname = "Sledge.Formats.Map"; version = "1.1.5"; hash = "sha256-hkYJ2iWIz7vhPWlDOw2fvTenlh+4/D/37Z71tCEwnK8="; })
(fetchNuGet { pname = "System.Numerics.Vectors"; version = "4.5.0"; hash = "sha256-qdSTIFgf2htPS+YhLGjAGiLN8igCYJnCCo6r78+Q+c8="; })
(fetchNuGet { pname = "FosterFramework"; version = "0.1.15-alpha"; sha256 = "0pzsdfbsfx28xfqljcwy100xhbs6wyx0z1d5qxgmv3l60di9xkll"; })
(fetchNuGet { pname = "Microsoft.AspNetCore.App.Runtime.linux-x64"; version = "8.0.1"; sha256 = "1gjz379y61ag9whi78qxx09bwkwcznkx2mzypgycibxk61g11da1"; })
(fetchNuGet { pname = "Microsoft.NET.ILLink.Tasks"; version = "8.0.1"; sha256 = "1drbgqdcvbpisjn8mqfgba1pwb6yri80qc4mfvyczqwrcsj5k2ja"; })
(fetchNuGet { pname = "Microsoft.NETCore.App.Runtime.linux-x64"; version = "8.0.1"; sha256 = "1g5b30f4l8a1zjjr3b8pk9mcqxkxqwa86362f84646xaj4iw3a4d"; })
(fetchNuGet { pname = "SharpGLTF.Core"; version = "1.0.0-alpha0031"; sha256 = "0ln78mkhbcxqvwnf944hbgg24vbsva2jpih6q3x82d3h7rl1pkh6"; })
(fetchNuGet { pname = "SharpGLTF.Runtime"; version = "1.0.0-alpha0031"; sha256 = "0lvb3asi3v0n718qf9y367km7qpkb9wci38y880nqvifpzllw0jg"; })
(fetchNuGet { pname = "Sledge.Formats"; version = "1.2.2"; sha256 = "1y0l66m9rym0p1y4ifjlmg3j9lsmhkvbh38frh40rpvf1axn2dyh"; })
(fetchNuGet { pname = "Sledge.Formats.Map"; version = "1.1.5"; sha256 = "1bww60hv9xcyxpvkzz5q3ybafdxxkw6knhv97phvpkw84pd0jil6"; })
(fetchNuGet { pname = "System.Numerics.Vectors"; version = "4.5.0"; sha256 = "1kzrj37yzawf1b19jq0253rcs8hsq1l2q8g69d7ipnhzb0h97m59"; })
]
```
@@ -259,5 +255,6 @@ Finally, you move the `deps.nix` file to the appropriate location to be used by
If you ever need to update the dependencies of a package, you instead do
* `nix-build -A package.fetch-deps` to generate the update script for `package`
* Run `./result` to regenerate the lockfile to the path passed for `nugetDeps` (keep in mind if it can't be resolved to a local path, the script will write to `$1` or a temporary path instead)
* Finally, ensure the correct file was written and the derivation can be built.
* Run `./result deps.nix` to regenerate the lockfile to `deps.nix`, keep in mind if a location isn't provided, it will write to a temporary path instead
* Finally, move the file where needed and look at its contents to confirm it has updated the dependencies.

View File

@@ -143,7 +143,7 @@ You can also pass additional arguments to `makeWrapper` using `gappsWrapperArgs`
## Updating GNOME packages {#ssec-gnome-updating}
Most GNOME package offer [`updateScript`](#var-passthru-updateScript), it is therefore possible to update to latest source tarball by running `nix-shell maintainers/scripts/update.nix --argstr package nautilus` or even en masse with `nix-shell maintainers/scripts/update.nix --argstr path gnome`. Read the packages `NEWS` file to see what changed.
Most GNOME package offer [`updateScript`](#var-passthru-updateScript), it is therefore possible to update to latest source tarball by running `nix-shell maintainers/scripts/update.nix --argstr package gnome.nautilus` or even en masse with `nix-shell maintainers/scripts/update.nix --argstr path gnome`. Read the packages `NEWS` file to see what changed.
## Frequently encountered issues {#ssec-gnome-common-issues}

View File

@@ -62,88 +62,10 @@ The following is an example expression using `buildGoModule`:
}
```
### Obtaining and overriding `vendorHash` for `buildGoModule` {#buildGoModule-vendorHash}
We can use `nix-prefetch` to obtain the actual hash. The following command gets the value of `vendorHash` for package `pet`:
```sh
cd path/to/nixpkgs
nix-prefetch -E "{ sha256 }: ((import ./. { }).my-package.overrideAttrs { vendorHash = sha256; }).goModules"
```
To obtain the hash without external tools, set `vendorHash = lib.fakeHash;` and run the build. ([more details here](#sec-source-hashes)).
`vendorHash` can be overridden with `overrideAttrs`. Override the above example like this:
```nix
{
pet_0_4_0 = pet.overrideAttrs (
finalAttrs: previousAttrs: {
version = "0.4.0";
src = fetchFromGitHub {
inherit (previousAttrs.src) owner repo;
rev = "v${finalAttrs.version}";
hash = "sha256-gVTpzmXekQxGMucDKskGi+e+34nJwwsXwvQTjRO6Gdg=";
};
vendorHash = "sha256-dUvp7FEW09V0xMuhewPGw3TuAic/sD7xyXEYviZ2Ivs=";
}
);
}
```
### Overriding `goModules` {#buildGoModule-goModules-override}
Overriding `<pkg>.goModules` by calling `goModules.overrideAttrs` is unsupported. Still, it is possible to override the `vendorHash` (`goModules`'s `outputHash`) and the `pre`/`post` hooks for both the build and patch phases of the primary and `goModules` derivation. Alternatively, the primary derivation provides an overridable `passthru.overrideModAttrs` function to store the attribute overlay implicitly taken by `goModules.overrideAttrs`. Here's an example usage of `overrideModAttrs`:
```nix
{
pet-overridden = pet.overrideAttrs (
finalAttrs: previousAttrs: {
passthru = previousAttrs.passthru // {
# If the original package has an `overrideModAttrs` attribute set, you'd
# want to extend it, and not replace it. Hence we use
# `lib.composeExtensions`. If you are sure the `overrideModAttrs` of the
# original package trivially does nothing, you can safely replace it
# with your own by not using `lib.composeExtensions`.
overrideModAttrs = lib.composeExtensions previousAttrs.passthru.overrideModAttrs (
finalModAttrs: previousModAttrs: {
# goModules-specific overriding goes here
postBuild = ''
# Here you have access to the `vendor` directory.
substituteInPlace vendor/github.com/example/repo/file.go \
--replace-fail "panic(err)" ""
'';
}
);
};
}
);
}
```
## `buildGoPackage` (legacy) {#ssec-go-legacy}
The function `buildGoPackage` builds legacy Go programs, not supporting Go modules.
::: {.warning}
`buildGoPackage` is deprecated and will be removed for the 25.05 release.
:::
### Migrating from `buildGoPackage` to `buildGoModule` {#buildGoPackage-migration}
Go modules, released 6y ago, are now widely adopted in the ecosystem.
Most upstream projects are using Go modules, and the tooling previously used for dependency management in Go is mostly deprecated, archived or at least unmaintained at this point.
In case a project doesn't have external dependencies or dependencies are vendored in a way understood by `go mod init`, migration can be done with a few changes in the package.
- Switch the builder from `buildGoPackage` to `buildGoModule`
- Remove `goPackagePath` and other attributes specific to `buildGoPackage`
- Set `vendorHash = null;`
- Run `go mod init <module name>` in `postPatch`
In case the package has external dependencies that aren't vendored or the build setup is more complex the upstream source might need to be patched.
Examples for the migration can be found in the [issue tracking migration withing nixpkgs](https://github.com/NixOS/nixpkgs/issues/318069).
### Example for `buildGoPackage` {#example-for-buildgopackage}
In the following is an example expression using `buildGoPackage`, the following arguments are of special significance to the function:

View File

@@ -1,189 +0,0 @@
# Gradle {#gradle}
Gradle is a popular build tool for Java/Kotlin. Gradle itself doesn't
currently provide tools to make dependency resolution reproducible, so
nixpkgs has a proxy designed for intercepting Gradle web requests to
record dependencies so they can be restored in a reproducible fashion.
## Building a Gradle package {#building-a-gradle-package}
Here's how a typical derivation will look like:
```nix
stdenv.mkDerivation (finalAttrs: {
pname = "pdftk";
version = "3.3.3";
src = fetchFromGitLab {
owner = "pdftk-java";
repo = "pdftk";
rev = "v${finalAttrs.version}";
hash = "sha256-ciKotTHSEcITfQYKFZ6sY2LZnXGChBJy0+eno8B3YHY=";
};
nativeBuildInputs = [ gradle ];
# if the package has dependencies, mitmCache must be set
mitmCache = gradle.fetchDeps {
inherit (finalAttrs) pname;
data = ./deps.json;
};
# this is required for using mitm-cache on Darwin
__darwinAllowLocalNetworking = true;
gradleFlags = [ "-Dfile.encoding=utf-8" ];
# defaults to "assemble"
gradleBuildTask = "shadowJar";
# will run the gradleCheckTask (defaults to "test")
doCheck = true;
installPhase = ''
mkdir -p $out/{bin,share/pdftk}
cp build/libs/pdftk-all.jar $out/share/pdftk
makeWrapper ${jre}/bin/java $out/bin/pdftk \
--add-flags "-jar $out/share/pdftk/pdftk-all.jar"
cp ${finalAttrs.src}/pdftk.1 $out/share/man/man1
'';
meta.sourceProvenance = with lib.sourceTypes; [
fromSource
binaryBytecode # mitm cache
];
})
```
To update (or initialize) dependencies, run the update script via
something like `$(nix-build -A <pname>.mitmCache.updateScript)`
(`nix-build` builds the `updateScript`, `$(...)` runs the script at the
path printed by `nix-build`).
If your package can't be evaluated using a simple `pkgs.<pname>`
expression (for example, if your package isn't located in nixpkgs, or if
you want to override some of its attributes), you will usually have to
pass `pkg` instead of `pname` to `gradle.fetchDeps`. There are two ways
of doing it.
The first is to add the derivation arguments required for getting the
package. Using the pdftk example above:
```nix
{ lib
, stdenv
# ...
, pdftk
}:
stdenv.mkDerivation (finalAttrs: {
# ...
mitmCache = gradle.fetchDeps {
pkg = pdftk;
data = ./deps.json;
};
})
```
This allows you to `override` any arguments of the `pkg` used for
the update script (for example, `pkg = pdftk.override { enableSomeFlag =
true };`), so this is the preferred way.
The second is to create a `let` binding for the package, like this:
```nix
let self = stdenv.mkDerivation {
# ...
mitmCache = gradle.fetchDeps {
pkg = self;
data = ./deps.json;
};
}; in self
```
This is useful if you can't easily pass the derivation as its own
argument, or if your `mkDerivation` call is responsible for building
multiple packages.
In the former case, the update script will stay the same even if the
derivation is called with different arguments. In the latter case, the
update script will change depending on the derivation arguments. It's up
to you to decide which one would work best for your derivation.
## Update Script {#gradle-update-script}
The update script does the following:
- Build the derivation's source via `pkgs.srcOnly`
- Enter a `nix-shell` for the derivation in a `bwrap` sandbox (the
sandbox is only used on Linux)
- Set the `IN_GRADLE_UPDATE_DEPS` environment variable to `1`
- Run the derivation's `unpackPhase`, `patchPhase`, `configurePhase`
- Run the derivation's `gradleUpdateScript` (the Gradle setup hook sets
a default value for it, which runs `preBuild`, `preGradleUpdate`
hooks, fetches the dependencies using `gradleUpdateTask`, and finally
runs the `postGradleUpdate` hook)
- Finally, store all of the fetched files' hashes in the lockfile. They
may be `.jar`/`.pom` files from Maven repositories, or they may be
files otherwise used for building the package.
`fetchDeps` takes the following arguments:
- `attrPath` - the path to the package in nixpkgs (for example,
`"javaPackages.openjfx22"`). Used for update script metadata.
- `pname` - an alias for `attrPath` for convenience. This is what you
will generally use instead of `pkg` or `attrPath`.
- `pkg` - the package to be used for fetching the dependencies. Defaults
to `getAttrFromPath (splitString "." attrPath) pkgs`.
- `bwrapFlags` - allows you to override bwrap flags (only relevant for
downstream, non-nixpkgs projects)
- `data` - path to the dependencies lockfile (can be relative to the
package, can be absolute). In nixpkgs, it's discouraged to have the
lockfiles be named anything other `deps.json`, consider creating
subdirectories if your package requires multiple `deps.json` files.
## Environment {#gradle-environment}
The Gradle setup hook accepts the following environment variables:
- `mitmCache` - the MITM proxy cache imported using `gradle.fetchDeps`
- `gradleFlags` - command-line flags to be used for every Gradle
invocation (this simply registers a function that uses the necessary
flags).
- You can't use `gradleFlags` for flags that contain spaces, in that
case you must add `gradleFlagsArray+=("-flag with spaces")` to the
derivation's bash code instead.
- If you want to build the package using a specific Java version, you
can pass `"-Dorg.gradle.java.home=${jdk}"` as one of the flags.
- `gradleBuildTask` - the Gradle task (or tasks) to be used for building
the package. Defaults to `assemble`.
- `gradleCheckTask` - the Gradle task (or tasks) to be used for checking
the package if `doCheck` is set to `true`. Defaults to `test`.
- `gradleUpdateTask` - the Gradle task (or tasks) to be used for
fetching all of the package's dependencies in
`mitmCache.updateScript`. Defaults to `nixDownloadDeps`.
- `gradleUpdateScript` - the code to run for fetching all of the
package's dependencies in `mitmCache.updateScript`. Defaults to
running the `preBuild` and `preGradleUpdate` hooks, running the
`gradleUpdateTask`, and finally running the `postGradleUpdate` hook.
- `gradleInitScript` - path to the `--init-script` to pass to Gradle. By
default, a simple init script that enables reproducible archive
creation is used.
- Note that reproducible archives might break some builds. One example
of an error caused by it is `Could not create task ':jar'. Replacing
an existing task that may have already been used by other plugins is
not supported`. If you get such an error, the easiest "fix" is
disabling reproducible archives altogether by setting
`gradleInitScript` to something like `writeText
"empty-init-script.gradle" ""`
- `enableParallelBuilding` / `enableParallelChecking` /
`enableParallelUpdating` - pass `--parallel` to Gradle in the
build/check phase or in the update script. Defaults to true. If the
build fails for mysterious reasons, consider setting this to false.
- `dontUseGradleConfigure` / `dontUseGradleBuild` / `dontUseGradleCheck`
\- force disable the Gradle setup hook for certain phases.
- Note that if you disable the configure hook, you may face issues
such as `Failed to load native library 'libnative-platform.so'`,
because the configure hook is responsible for initializing Gradle.

View File

@@ -1,53 +0,0 @@
# Hare {#sec-language-hare}
## Building Hare programs with `hareHook` {#ssec-language-hare}
The `hareHook` package sets up the environment for building Hare programs by
doing the following:
1. Setting the `HARECACHE`, `HAREPATH` and `NIX_HAREFLAGS` environment variables;
1. Propagating `harec`, `qbe` and two wrapper scripts for the hare binary.
It is not a function as is the case for some other languages --- *e. g.*, Go or
Rust ---, but a package to be added to `nativeBuildInputs`.
## Attributes of `hareHook` {#hareHook-attributes}
The following attributes are accepted by `hareHook`:
1. `hareBuildType`: Either `release` (default) or `debug`. It controls if the
`-R` flag is added to `NIX_HAREFLAGS`.
## Example for `hareHook` {#ex-hareHook}
```nix
{
hareHook,
lib,
stdenv,
}: stdenv.mkDerivation {
pname = "<name>";
version = "<version>";
src = "<src>";
nativeBuildInputs = [ hareHook ];
meta = {
description = "<description>";
inherit (hareHook) badPlatforms platforms;
};
}
```
## Cross Compilation {#hareHook-cross-compilation}
`hareHook` should handle cross compilation out of the box. This is the main
purpose of `NIX_HAREFLAGS`: In it, the `-a` flag is passed with the architecture
of the `hostPlatform`.
However, manual intervention may be needed when a binary compiled by the build
process must be run for the build to complete --- *e. g.*, when using Hare's
`hare` module for code generation.
In those cases, `hareHook` provides the `hare-native` script, which is a wrapper
around the hare binary for using the native (`buildPlatform`) toolchain.

View File

@@ -21,14 +21,25 @@ Many “normal” user facing packages written in Haskell, like `niv` or `cachix
are also exposed at the top level, and there is nothing Haskell specific to
installing and using them.
All of these packages are originally defined in the `haskellPackages` package set.
The same packages are re-exposed with a reduced dependency closure for convenience (see `justStaticExecutables` or `separateBinOutput` below).
All of these packages are originally defined in the `haskellPackages` package
set and are re-exposed with a reduced dependency closure for convenience.
(see `justStaticExecutables` or `separateBinOutput` below)
:::{.note}
See [](#chap-language-support) for techniques to explore package sets.
:::
The `haskellPackages` set includes at least one version of every package from
Hackage as well as some manually injected packages. This amounts to a lot of
packages, so it is hidden from `nix-env -qa` by default for performance reasons.
You can still list all packages in the set like this:
The `haskellPackages` set includes at least one version of every package from [Hackage](https://hackage.haskell.org/) as well as some manually injected packages.
```console
$ nix-env -f '<nixpkgs>' -qaP -A haskellPackages
haskellPackages.a50 a50-0.5
haskellPackages.AAI AAI-0.2.0.1
haskellPackages.aasam aasam-0.2.0.0
haskellPackages.abacate abacate-0.0.0.0
haskellPackages.abc-puzzle abc-puzzle-0.2.1
```
Also, the `haskellPackages` set is included on [search.nixos.org].
The attribute names in `haskellPackages` always correspond with their name on
Hackage. Since Hackage allows names that are not valid Nix without escaping,
@@ -38,7 +49,8 @@ For packages that are part of [Stackage] (a curated set of known to be
compatible packages), we use the version prescribed by a Stackage snapshot
(usually the current LTS one) as the default version. For all other packages we
use the latest version from [Hackage](https://hackage.org) (the repository of
basically all open source Haskell packages). See [below](#haskell-available-versions) for a few more details on this.
basically all open source Haskell packages). See [below](#haskell-available-
versions) for a few more details on this.
Roughly half of the 16K packages contained in `haskellPackages` dont actually
build and are [marked as broken semi-automatically](https://github.com/NixOS/nixpkgs/blob/haskell-updates/pkgs/development/haskell-modules/configuration-hackage2nix/broken.yaml).
@@ -51,15 +63,68 @@ How you can help with that is
described in [Fixing a broken package](#haskell-fixing-a-broken-package).
-->
`haskellPackages` is built with our default compiler, but we also provide other releases of GHC and package sets built with them.
Available compilers are collected under `haskell.compiler`.
`haskellPackages` is built with our default compiler, but we also provide other
releases of GHC and package sets built with them. You can list all available
compilers like this:
Each of those compiler versions has a corresponding attribute set `packages` built with
```console
$ nix-env -f '<nixpkgs>' -qaP -A haskell.compiler
haskell.compiler.ghc810 ghc-8.10.7
haskell.compiler.ghc90 ghc-9.0.2
haskell.compiler.ghc925 ghc-9.2.5
haskell.compiler.ghc926 ghc-9.2.6
haskell.compiler.ghc927 ghc-9.2.7
haskell.compiler.ghc92 ghc-9.2.8
haskell.compiler.ghc945 ghc-9.4.5
haskell.compiler.ghc946 ghc-9.4.6
haskell.compiler.ghc947 ghc-9.4.7
haskell.compiler.ghc94 ghc-9.4.8
haskell.compiler.ghc963 ghc-9.6.3
haskell.compiler.ghc96 ghc-9.6.4
haskell.compiler.ghc98 ghc-9.8.1
haskell.compiler.ghcHEAD ghc-9.9.20231121
haskell.compiler.ghc8107Binary ghc-binary-8.10.7
haskell.compiler.ghc865Binary ghc-binary-8.6.5
haskell.compiler.ghc924Binary ghc-binary-9.2.4
haskell.compiler.integer-simple.ghc8107 ghc-integer-simple-8.10.7
haskell.compiler.integer-simple.ghc810 ghc-integer-simple-8.10.7
haskell.compiler.native-bignum.ghc90 ghc-native-bignum-9.0.2
haskell.compiler.native-bignum.ghc902 ghc-native-bignum-9.0.2
haskell.compiler.native-bignum.ghc925 ghc-native-bignum-9.2.5
haskell.compiler.native-bignum.ghc926 ghc-native-bignum-9.2.6
haskell.compiler.native-bignum.ghc927 ghc-native-bignum-9.2.7
haskell.compiler.native-bignum.ghc92 ghc-native-bignum-9.2.8
haskell.compiler.native-bignum.ghc928 ghc-native-bignum-9.2.8
haskell.compiler.native-bignum.ghc945 ghc-native-bignum-9.4.5
haskell.compiler.native-bignum.ghc946 ghc-native-bignum-9.4.6
haskell.compiler.native-bignum.ghc947 ghc-native-bignum-9.4.7
haskell.compiler.native-bignum.ghc94 ghc-native-bignum-9.4.8
haskell.compiler.native-bignum.ghc948 ghc-native-bignum-9.4.8
haskell.compiler.native-bignum.ghc963 ghc-native-bignum-9.6.3
haskell.compiler.native-bignum.ghc96 ghc-native-bignum-9.6.4
haskell.compiler.native-bignum.ghc964 ghc-native-bignum-9.6.4
haskell.compiler.native-bignum.ghc98 ghc-native-bignum-9.8.1
haskell.compiler.native-bignum.ghc981 ghc-native-bignum-9.8.1
haskell.compiler.native-bignum.ghcHEAD ghc-native-bignum-9.9.20231121
haskell.compiler.ghcjs ghcjs-8.10.7
```
Each of those compiler versions has a corresponding attribute set built using
it. However, the non-standard package sets are not tested regularly and, as a
result, contain fewer working packages. The corresponding package set for GHC
9.4.5 is `haskell.packages.ghc945`. In fact `haskellPackages` is just an alias
for `haskell.packages.ghc964`:
```console
$ nix-env -f '<nixpkgs>' -qaP -A haskell.packages.ghc927
haskell.packages.ghc927.a50 a50-0.5
haskell.packages.ghc927.AAI AAI-0.2.0.1
haskell.packages.ghc927.aasam aasam-0.2.0.0
haskell.packages.ghc927.abacate abacate-0.0.0.0
haskell.packages.ghc927.abc-puzzle abc-puzzle-0.2.1
```
Every package set also re-exposes the GHC used to build its packages as `haskell.packages.*.ghc`.
### Available package versions {#haskell-available-versions}
@@ -858,61 +923,14 @@ for this to work.
`justStaticExecutables drv`
: Only build and install the executables produced by `drv`, removing everything
that may refer to other Haskell packages' store paths (like libraries and
documentation). This dramatically reduces the closure size of the resulting
derivation. Note that the executables are only statically linked against their
Haskell dependencies, but will still link dynamically against libc, GMP and
other system library dependencies.
If a library or its dependencies use their Cabal-generated
`Paths_*` module, this may not work as well if GHC's dead code elimination is
unable to remove the references to the dependency's store path that module
contains.
As a consequence, an unused reference may be created from the static binary to such a _library_ store path.
(See [nixpkgs#164630][164630] for more information.)
Importing the `Paths_*` module may cause builds to fail with this message:
```
error: output '/nix/store/64k8iw0ryz76qpijsnl9v87fb26v28z8-my-haskell-package-1.0.0.0' is not allowed to refer to the following paths:
/nix/store/5q5s4a07gaz50h04zpfbda8xjs8wrnhg-ghc-9.6.3
```
If that happens, first disable the check for GHC references and rebuild the
derivation:
```nix
pkgs.haskell.lib.overrideCabal
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
(drv: {
disallowGhcReference = false;
})
```
Then use `strings` to determine which libraries are responsible:
```
$ nix-build ...
$ strings result/bin/my-haskell-binary | grep /nix/store/
...
/nix/store/n7ciwdlg8yyxdhbrgd6yc2d8ypnwpmgq-hs-opentelemetry-sdk-0.0.3.6/bin
...
```
Finally, use `remove-references-to` to delete those store paths from the produced output:
```nix
pkgs.haskell.lib.overrideCabal
(pkgs.haskell.lib.justStaticExecutables my-haskell-package)
(drv: {
postInstall = ''
${drv.postInstall or ""}
remove-references-to -t ${pkgs.haskellPackages.hs-opentelemetry-sdk}
'';
})
```
[164630]: https://github.com/NixOS/nixpkgs/issues/164630
that may refer to other Haskell packages' store paths (like libraries and
documentation). This dramatically reduces the closure size of the resulting
derivation. Note that the executables are only statically linked against their
Haskell dependencies, but will still link dynamically against libc, GMP and
other system library dependencies. If dependencies use their Cabal-generated
`Paths_*` module, this may not work as well if GHC's dead code elimination
is unable to remove the references to the dependency's store path that module
contains.
`enableSeparateBinOutput drv`
: Install executables produced by `drv` to a separate `bin` output. This

View File

@@ -19,7 +19,7 @@ let lspLibPkg = idris2Packages.buildIdris {
};
idrisLibraries = [ ];
};
in lspLibPkg.library { withSource = true; }
in lspLibPkg.library
```
The above results in a derivation with the installed library results (with sourcecode).
@@ -30,7 +30,6 @@ A slightly more involved example of a fully packaged executable would be the [`i
# Assuming the previous example lives in `lsp-lib.nix`:
let lspLib = callPackage ./lsp-lib.nix { };
inherit (idris2Packages) idris2Api;
lspPkg = idris2Packages.buildIdris {
ipkgName = "idris2-lsp";
src = fetchFromGitHub {
@@ -39,9 +38,10 @@ let lspLib = callPackage ./lsp-lib.nix { };
rev = "main";
hash = "sha256-vQTzEltkx7uelDtXOHc6QRWZ4cSlhhm5ziOqWA+aujk=";
};
idrisLibraries = [idris2Api lspLib];
idrisLibraries = [(idris2Packages.idris2Api { }) (lspLib { })];
};
in lspPkg.executable
```
The above uses the default value of `withSource = false` for the `idris2Api` but could be modified to include that library's source by passing `(idris2Api { withSource = true; })` to `idrisLibraries` instead. `idris2Api` in the above derivation comes built in with `idris2Packages`. This library exposes many of the otherwise internal APIs of the Idris2 compiler.
The above uses the default value of `withSource = false` for both of the two required Idris libraries that the `idris2-lsp` executable depends on. `idris2Api` in the above derivation comes built in with `idris2Packages`. This library exposes many of the otherwise internal APIs of the Idris2 compiler.

View File

@@ -2,54 +2,6 @@
The [standard build environment](#chap-stdenv) makes it easy to build typical Autotools-based packages with very little code. Any other kind of package can be accommodated by overriding the appropriate phases of `stdenv`. However, there are specialised functions in Nixpkgs to easily build packages for other programming languages, such as Perl or Haskell. These are described in this chapter.
Each supported language or software ecosystem has its own package set named `<language or ecosystem>Packages`, which can be explored in various ways:
- Search on [search.nixos.org](https://search.nixos.org/packages)
For example, search for [`haskellPackages`](https://search.nixos.org/packages?query=haskellPackages) or [`rubyPackages`](https://search.nixos.org/packages?query=rubyPackages).
- Navigate attribute sets with [`nix repl`](https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-repl).
This technique is generally useful to inspect Nix language data structures.
:::{.example #example-navigte-nix-repl}
# Navigate Java compiler variants in `javaPackages` with `nix repl`
```shell-session
$ nix repl '<nixpkgs>' -I nixpkgs=channel:nixpkgs-unstable
nix-repl> javaPackages.<tab>
javaPackages.compiler javaPackages.openjfx15 javaPackages.openjfx21 javaPackages.recurseForDerivations
javaPackages.jogl_2_4_0 javaPackages.openjfx17 javaPackages.openjfx22
javaPackages.mavenfod javaPackages.openjfx19 javaPackages.override
javaPackages.openjfx11 javaPackages.openjfx20 javaPackages.overrideDerivation
```
:::
- List all derivations on the command line with [`nix-env --query`](https://nixos.org/manual/nix/stable/command-ref/nix-env/query).
`nix-env` is the only convenient way to do that, as it will skip attributes that fail [assertions](https://nixos.org/manual/nix/stable/language/constructs#assertions), such as when a package is [marked as broken](#var-meta-broken), rather than failing the entire evaluation.
:::{.example #example-list-haskellPackages}
# List all Python packages in Nixpkgs
The following command lists all [derivations names](https://nixos.org/manual/nix/stable/language/derivations#attr-name) with their attribute path from the latest Nixpkgs rolling release (`nixpkgs-unstable`).
```shell-session
$ nix-env -qaP -f '<nixpkgs>' -A pythonPackages -I nixpkgs=channel:nixpkgs-unstable
```
```console
pythonPackages.avahi avahi-0.8
pythonPackages.boost boost-1.81.0
pythonPackages.caffe caffe-1.0
pythonPackages.caffeWithCuda caffe-1.0
pythonPackages.cbeams cbeams-1.0.3
```
:::
```{=include=} sections
agda.section.md
android.section.md
@@ -67,8 +19,6 @@ dotnet.section.md
emscripten.section.md
gnome.section.md
go.section.md
gradle.section.md
hare.section.md
haskell.section.md
hy.section.md
idris.section.md
@@ -91,7 +41,6 @@ qt.section.md
r.section.md
ruby.section.md
rust.section.md
scheme.section.md
swift.section.md
texlive.section.md
titanium.section.md

View File

@@ -206,7 +206,7 @@ buildNpmPackage rec {
NODE_OPTIONS = "--openssl-legacy-provider";
meta = {
description = "Modern web UI for various torrent clients with a Node.js backend and React frontend";
description = "A modern web UI for various torrent clients with a Node.js backend and React frontend";
homepage = "https://flood.js.org";
license = lib.licenses.gpl3Only;
maintainers = with lib.maintainers; [ winter ];
@@ -233,7 +233,7 @@ If these are not defined, `npm pack` may miss some files, and no binaries will b
* `npmPruneFlags`: Flags to pass to `npm prune`. Defaults to the value of `npmInstallFlags`.
* `makeWrapperArgs`: Flags to pass to `makeWrapper`, added to executable calling the generated `.js` with `node` as an interpreter. These scripts are defined in `package.json`.
* `nodejs`: The `nodejs` package to build against, using the corresponding `npm` shipped with that version of `node`. Defaults to `pkgs.nodejs`.
* `npmDeps`: The dependencies used to build the npm package. Especially useful to not have to recompute workspace dependencies.
* `npmDeps`: The dependencies used to build the npm package. Especially useful to not have to recompute workspace depedencies.
#### prefetch-npm-deps {#javascript-buildNpmPackage-prefetch-npm-deps}
@@ -258,39 +258,26 @@ It returns a derivation with all `package-lock.json` dependencies downloaded int
#### importNpmLock {#javascript-buildNpmPackage-importNpmLock}
This function replaces the npm dependency references in `package.json` and `package-lock.json` with paths to the Nix store.
How each dependency is fetched can be customized with the `fetcherOpts` argument.
`importNpmLock` is a Nix function that requires the following optional arguments:
This is a simpler and more convenient alternative to [`fetchNpmDeps`](#javascript-buildNpmPackage-fetchNpmDeps) for managing npm dependencies in Nixpkgs.
There is no need to specify a `hash`, since it relies entirely on the integrity hashes already present in the `package-lock.json` file.
##### Inputs {#javascript-buildNpmPackage-inputs}
- `npmRoot`: Path to package directory containing the source tree.
If this is omitted, the `package` and `packageLock` arguments must be specified instead.
- `npmRoot`: Path to package directory containing the source tree
- `package`: Parsed contents of `package.json`
- `packageLock`: Parsed contents of `package-lock.json`
- `pname`: Package name
- `version`: Package version
- `fetcherOpts`: An attribute set of arguments forwarded to the underlying fetcher.
It returns a derivation with a patched `package.json` & `package-lock.json` with all dependencies resolved to Nix store paths.
:::{.note}
`npmHooks.npmConfigHook` cannot be used with `importNpmLock`.
Use `importNpmLock.npmConfigHook` instead.
:::
This function is analogous to using `fetchNpmDeps`, but instead of specifying `hash` it uses metadata from `package.json` & `package-lock.json`.
:::{.example}
Note that `npmHooks.npmConfigHook` cannot be used with `importNpmLock`. You will instead need to use `importNpmLock.npmConfigHook`:
##### `pkgs.importNpmLock` usage example {#javascript-buildNpmPackage-example}
```nix
{ buildNpmPackage, importNpmLock }:
buildNpmPackage {
pname = "hello";
version = "0.1.0";
src = ./.;
npmDeps = importNpmLock {
npmRoot = ./.;
@@ -299,75 +286,6 @@ buildNpmPackage {
npmConfigHook = importNpmLock.npmConfigHook;
}
```
:::
:::{.example}
##### `pkgs.importNpmLock` usage example with `fetcherOpts` {#javascript-buildNpmPackage-example-fetcherOpts}
`importNpmLock` uses the following fetchers:
- `pkgs.fetchurl` for `http(s)` dependencies
- `builtins.fetchGit` for `git` dependencies
It is possible to provide additional arguments to individual fetchers as needed:
```nix
{ buildNpmPackage, importNpmLock }:
buildNpmPackage {
pname = "hello";
version = "0.1.0";
src = ./.;
npmDeps = importNpmLock {
npmRoot = ./.;
fetcherOpts = {
# Pass 'curlOptsList' to 'pkgs.fetchurl' while fetching 'axios'
{ "node_modules/axios" = { curlOptsList = [ "--verbose" ]; }; }
};
};
npmConfigHook = importNpmLock.npmConfigHook;
}
```
:::
#### importNpmLock.buildNodeModules {#javascript-buildNpmPackage-importNpmLock.buildNodeModules}
`importNpmLock.buildNodeModules` returns a derivation with a pre-built `node_modules` directory, as imported by `importNpmLock`.
This is to be used together with `importNpmLock.hooks.linkNodeModulesHook` to facilitate `nix-shell`/`nix develop` based development workflows.
It accepts an argument with the following attributes:
`npmRoot` (Path; optional)
: Path to package directory containing the source tree. If not specified, the `package` and `packageLock` arguments must both be specified.
`package` (Attrset; optional)
: Parsed contents of `package.json`, as returned by `lib.importJSON ./my-package.json`. If not specified, the `package.json` in `npmRoot` is used.
`packageLock` (Attrset; optional)
: Parsed contents of `package-lock.json`, as returned `lib.importJSON ./my-package-lock.json`. If not specified, the `package-lock.json` in `npmRoot` is used.
`derivationArgs` (`mkDerivation` attrset; optional)
: Arguments passed to `stdenv.mkDerivation`
For example:
```nix
pkgs.mkShell {
packages = [
importNpmLock.hooks.linkNodeModulesHook
nodejs
];
npmDeps = importNpmLock.buildNodeModules {
npmRoot = ./.;
inherit nodejs;
};
}
```
will create a development shell where a `node_modules` directory is created & packages symlinked to the Nix store when activated.
### corepack {#javascript-corepack}
@@ -392,193 +310,8 @@ See `node2nix` [docs](https://github.com/svanderburg/node2nix) for more info.
- `node2nix` has some [bugs](https://github.com/svanderburg/node2nix/issues/238) related to working with lock files from npm distributed with `nodejs_16`.
- `node2nix` does not like missing packages from npm. If you see something like `Cannot resolve version: vue-loader-v16@undefined` then you might want to try another tool. The package might have been pulled off of npm.
### pnpm {#javascript-pnpm}
Pnpm is available as the top-level package `pnpm`. Additionally, there are variants pinned to certain major versions, like `pnpm_8` and `pnpm_9`, which support different sets of lock file versions.
When packaging an application that includes a `pnpm-lock.yaml`, you need to fetch the pnpm store for that project using a fixed-output-derivation. The functions `pnpm_8.fetchDeps` and `pnpm_9.fetchDeps` can create this pnpm store derivation. In conjunction, the setup hooks `pnpm_8.configHook` and `pnpm_9.configHook` will prepare the build environment to install the prefetched dependencies store. Here is an example for a package that contains a `package.json` and a `pnpm-lock.yaml` files using the above `pnpm_` attributes:
```nix
{
stdenv,
nodejs,
# This is pinned as { pnpm = pnpm_9; }
pnpm
}:
stdenv.mkDerivation (finalAttrs: {
pname = "foo";
version = "0-unstable-1980-01-01";
src = ...;
nativeBuildInputs = [
nodejs
pnpm.configHook
];
pnpmDeps = pnpm.fetchDeps {
inherit (finalAttrs) pname version src;
hash = "...";
};
})
```
NOTE: It is highly recommended to use a pinned version of pnpm (i.e. `pnpm_8` or `pnpm_9`), to increase future reproducibility. It might also be required to use an older version, if the package needs support for a certain lock file version.
In case you are patching `package.json` or `pnpm-lock.yaml`, make sure to pass `finalAttrs.patches` to the function as well (i.e. `inherit (finalAttrs) patches`.
`pnpm.configHook` supports adding additional `pnpm install` flags via `pnpmInstallFlags` which can be set to a Nix string array.
#### Dealing with `sourceRoot` {#javascript-pnpm-sourceRoot}
If the pnpm project is in a subdirectory, you can just define `sourceRoot` or `setSourceRoot` for `fetchDeps`.
If `sourceRoot` is different between the parent derivation and `fetchDeps`, you will have to set `pnpmRoot` to effectively be the same location as it is in `fetchDeps`.
Assuming the following directory structure, we can define `sourceRoot` and `pnpmRoot` as follows:
```
.
├── frontend
│   ├── ...
│   ├── package.json
│   └── pnpm-lock.yaml
└── ...
```
```nix
...
pnpmDeps = pnpm.fetchDeps {
...
sourceRoot = "${finalAttrs.src.name}/frontend";
};
# by default the working directory is the extracted source
pnpmRoot = "frontend";
```
#### PNPM Workspaces {#javascript-pnpm-workspaces}
If you need to use a PNPM workspace for your project, then set `pnpmWorkspace = "<workspace project name>"` in your `pnpm.fetchDeps` call,
which will make PNPM only install dependencies for that workspace package.
For example:
```nix
...
pnpmWorkspace = "@astrojs/language-server";
pnpmDeps = pnpm.fetchDeps {
inherit (finalAttrs) pnpmWorkspace;
...
}
```
The above would make `pnpm.fetchDeps` call only install dependencies for the `@astrojs/language-server` workspace package.
Note that you do not need to set `sourceRoot` to make this work.
Usually in such cases, you'd want to use `pnpm --filter=$pnpmWorkspace build` to build your project, as `npmHooks.npmBuildHook` probably won't work. A `buildPhase` based on the following example will probably fit most workspace projects:
```nix
buildPhase = ''
runHook preBuild
pnpm --filter=@astrojs/language-server build
runHook postBuild
'';
```
#### Additional PNPM Commands and settings {#javascript-pnpm-extraCommands}
If you require setting an additional PNPM configuration setting (such as `dedupe-peer-dependents` or similar),
set `prePnpmInstall` to the right commands to run. For example:
```nix
prePnpmInstall = ''
pnpm config set dedupe-peer-dependants false
'';
pnpmDeps = pnpm.fetchDeps {
inherit (finalAttrs) prePnpmInstall;
...
};
```
In this example, `prePnpmInstall` will be run by both `pnpm.configHook` and by the `pnpm.fetchDeps` builder.
### Yarn {#javascript-yarn}
Yarn based projects use a `yarn.lock` file instead of a `package-lock.json` to pin dependencies. Nixpkgs provides the Nix function `fetchYarnDeps` which fetches an offline cache suitable for running `yarn install` before building the project. In addition, Nixpkgs provides the hooks:
- `yarnConfigHook`: Fetches the dependencies from the offline cache and installs them into `node_modules`.
- `yarnBuildHook`: Runs `yarn build` or a specified `yarn` command that builds the project.
- `yarnInstallHook`: Runs `yarn install --production` to prune dependencies and installs the project into `$out`.
An example usage of the above attributes is:
```nix
{
lib,
stdenv,
fetchFromGitHub,
fetchYarnDeps,
yarnConfigHook,
yarnBuildHook,
yarnInstallHook,
nodejs,
}:
stdenv.mkDerivation (finalAttrs: {
pname = "...";
version = "...";
src = fetchFromGitHub {
owner = "...";
repo = "...";
rev = "v${finalAttrs.version}";
hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
};
yarnOfflineCache = fetchYarnDeps {
yarnLock = finalAttrs.src + "/yarn.lock";
hash = "sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=";
};
nativeBuildInputs = [
yarnConfigHook
yarnBuildHook
yarnInstallHook
# Needed for executing package.json scripts
nodejs
];
meta = {
# ...
};
})
```
#### `yarnConfigHook` arguments {#javascript-yarnconfighook}
By default, `yarnConfigHook` relies upon the attribute `${yarnOfflineCache}` (or `${offlineCache}` if the former is not set) to find the location of the offline cache produced by `fetchYarnDeps`. To disable this phase, you can set `dontYarnInstallDeps = true` or override the `configurePhase`.
#### `yarnBuildHook` arguments {#javascript-yarnbuildhook}
This script by default runs `yarn --offline build`, and it relies upon the project's dependencies installed at `node_modules`. Below is a list of additional `mkDerivation` arguments read by this hook:
- `yarnBuildScript`: Sets a different `yarn --offline` subcommand (defaults to `build`).
- `yarnBuildFlags`: Single string list of additional flags to pass the above command, or a Nix list of such additional flags.
#### `yarnInstallHook` arguments {#javascript-yarninstallhook}
To install the package `yarnInstallHook` uses both `npm` and `yarn` to cleanup project files and dependencies. To disable this phase, you can set `dontYarnInstall = true` or override the `installPhase`. Below is a list of additional `mkDerivation` arguments read by this hook:
- `yarnKeepDevDeps`: Disables the removal of devDependencies from `node_modules` before installation.
### yarn2nix {#javascript-yarn2nix}
WARNING: The `yarn2nix` functions have been deprecated in favor of the new `yarnConfigHook`, `yarnBuildHook` and `yarnInstallHook`. Documentation for them still appears here for the sake of the packages that still use them. See also a tracking issue [#324246](https://github.com/NixOS/nixpkgs/issues/324246).
#### Preparation {#javascript-yarn2nix-preparation}
You will need at least a `yarn.lock` file. If upstream does not have one you need to generate it and reference it in your package definition.

View File

@@ -219,10 +219,10 @@ stdenv.mkDerivation {
# don't do any fixup
dontFixup = true;
outputHashAlgo = null;
outputHashAlgo = "sha256";
outputHashMode = "recursive";
# replace this with the correct SHA256
outputHash = lib.fakeHash;
outputHash = lib.fakeSha256;
}
```

View File

@@ -7,7 +7,7 @@ The following example shows a Nim program that depends only on Nim libraries:
```nix
{ lib, buildNimPackage, fetchFromGitHub }:
buildNimPackage (finalAttrs: {
buildNimPackage { } (finalAttrs: {
pname = "ttop";
version = "1.2.7";
@@ -80,6 +80,7 @@ For example, to propagate a dependency on SDL2 for lockfiles that select the Nim
/* */
sdl2 =
lockAttrs:
finalAttrs:
{ buildInputs ? [ ], ... }:
{
buildInputs = buildInputs ++ [ SDL2 ];
@@ -88,8 +89,9 @@ For example, to propagate a dependency on SDL2 for lockfiles that select the Nim
}
```
The annotations in the `nim-overrides.nix` set are functions that take two arguments and return a new attrset to be overlaid on the package being built.
The annotations in the `nim-overrides.nix` set are functions that take three arguments and return a new attrset to be overlayed on the package being built.
- lockAttrs: the attrset for this library from within a lockfile. This can be used to implement library version constraints, such as marking libraries as broken or insecure.
- finalAttrs: the final attrset passed by `buildNimPackage` to `stdenv.mkDerivation`.
- prevAttrs: the attrset produced by initial arguments to `buildNimPackage` and any preceding lockfile overlays.
### Overriding an Nim library override {#nim-lock-overrides-overrides}

View File

@@ -113,13 +113,21 @@ buildDunePackage rec {
meta = {
homepage = "https://github.com/flowtype/ocaml-wtf8";
description = "WTF-8 is a superset of UTF-8 that allows unpaired surrogates";
description = "WTF-8 is a superset of UTF-8 that allows unpaired surrogates.";
license = lib.licenses.mit;
maintainers = [ lib.maintainers.eqyiel ];
};
}
```
Note about `minimalOCamlVersion`. A deprecated version of this argument was
spelled `minimumOCamlVersion`; setting the old attribute wrongly modifies the
derivation hash and is therefore inappropriate. As a technical dept, currently
packaged libraries may still use the old spelling: maintainers are invited to
fix this when updating packages. Massive renaming is strongly discouraged as it
would be challenging to review, difficult to test, and will cause unnecessary
rebuild.
The build will automatically fail if two distinct versions of the same library
are added to `buildInputs` (which usually happens transitively because of
`propagatedBuildInputs`). Set `dontDetectOcamlConflicts` to true to disable this

View File

@@ -158,7 +158,7 @@ $ nix-generate-from-cpan XML::Simple
};
propagatedBuildInputs = [ XMLNamespaceSupport XMLSAX XMLSAXExpat ];
meta = {
description = "API for simple XML files";
description = "An API for simple XML files";
license = with lib.licenses; [ artistic1 gpl1Plus ];
};
};

View File

@@ -283,10 +283,7 @@ in {
];
composerRepository = php.mkComposerRepository {
inherit (finalAttrs) pname version src;
composerNoDev = true;
composerNoPlugins = true;
composerNoScripts = true;
inherit (finalAttrs) src;
# Specifying a custom composer.lock since it is not present in the sources.
composerLock = ./composer.lock;
# The composer vendor hash

View File

@@ -4,7 +4,16 @@
### Interpreters {#interpreters}
@python-interpreter-table@
| Package | Aliases | Interpreter |
|------------|-----------------|-------------|
| python27 | python2, python | CPython 2.7 |
| python39 | | CPython 3.9 |
| python310 | | CPython 3.10 |
| python311 | python3 | CPython 3.11 |
| python312 | | CPython 3.12 |
| python313 | | CPython 3.13 |
| pypy27 | pypy2, pypy | PyPy2.7 |
| pypy39 | pypy3 | PyPy 3.9 |
The Nix expressions for the interpreters can be found in
`pkgs/development/interpreters/python`.
@@ -31,8 +40,8 @@ Each interpreter has the following attributes:
### Building packages and applications {#building-packages-and-applications}
Python libraries and applications that use tools to follow PEP 517 (e.g. `setuptools` or `hatchling`, etc.) or
previous tools such as `distutils` are typically built with respectively the [`buildPythonPackage`](#buildpythonpackage-function) and
Python libraries and applications that use `setuptools` or
`distutils` are typically built with respectively the [`buildPythonPackage`](#buildpythonpackage-function) and
[`buildPythonApplication`](#buildpythonapplication-function) functions. These two functions also support installing a `wheel`.
All Python packages reside in `pkgs/top-level/python-packages.nix` and all
@@ -55,19 +64,13 @@ sets are
* `pkgs.python311Packages`
* `pkgs.python312Packages`
* `pkgs.python313Packages`
* `pkgs.pypy27Packages`
* `pkgs.pypy39Packages`
* `pkgs.pypy310Packages`
* `pkgs.pypyPackages`
and the aliases
* `pkgs.python2Packages` pointing to `pkgs.python27Packages`
* `pkgs.python3Packages` pointing to `pkgs.python312Packages`
* `pkgs.python3Packages` pointing to `pkgs.python311Packages`
* `pkgs.pythonPackages` pointing to `pkgs.python2Packages`
* `pkgs.pypy2Packages` pointing to `pkgs.pypy27Packages`
* `pkgs.pypy3Packages` pointing to `pkgs.pypy39Packages`
* `pkgs.pypyPackages` pointing to `pkgs.pypy2Packages`
#### `buildPythonPackage` function {#buildpythonpackage-function}
@@ -84,7 +87,6 @@ The following is an example:
, fetchPypi
# build-system
, setuptools
, setuptools-scm
# dependencies
@@ -114,7 +116,6 @@ buildPythonPackage rec {
'';
build-system = [
setuptools
setuptools-scm
];
@@ -142,13 +143,13 @@ buildPythonPackage rec {
The `buildPythonPackage` mainly does four things:
* In the [`buildPhase`](#build-phase), it calls `${python.pythonOnBuildForHost.interpreter} -m build --wheel` to
* In the [`buildPhase`](#build-phase), it calls `${python.pythonOnBuildForHost.interpreter} setup.py bdist_wheel` to
build a wheel binary zipfile.
* In the [`installPhase`](#ssec-install-phase), it installs the wheel file using `${python.pythonOnBuildForHost.interpreter} -m installer *.whl`.
* In the [`installPhase`](#ssec-install-phase), it installs the wheel file using `pip install *.whl`.
* In the [`postFixup`](#var-stdenv-postFixup) phase, the `wrapPythonPrograms` bash function is called to
wrap all programs in the `$out/bin/*` directory to include `$PATH`
environment variable and add dependent libraries to script's `sys.path`.
* In the [`installCheck`](#ssec-installCheck-phase) phase, `${python.interpreter} -m pytest` is run.
* In the [`installCheck`](#ssec-installCheck-phase) phase, `${python.interpreter} setup.py test` is run.
By default tests are run because [`doCheck = true`](#var-stdenv-doCheck). Test dependencies, like
e.g. the test runner, should be added to [`nativeCheckInputs`](#var-stdenv-nativeCheckInputs).
@@ -168,8 +169,7 @@ following are specific to `buildPythonPackage`:
* `dontWrapPythonPrograms ? false`: Skip wrapping of Python programs.
* `permitUserSite ? false`: Skip setting the `PYTHONNOUSERSITE` environment
variable in wrapped programs.
* `pyproject`: Whether the pyproject format should be used. As all other formats
are deprecated, you are recommended to set this to `true`. When you do so,
* `pyproject`: Whether the pyproject format should be used. When set to `true`,
`pypaBuildHook` will be used, and you can add the required build dependencies
from `build-system.requires` to `build-system`. Note that the pyproject
format falls back to using `setuptools`, so you can use `pyproject = true`
@@ -186,6 +186,10 @@ following are specific to `buildPythonPackage`:
`makeWrapperArgs = ["--set FOO BAR" "--set BAZ QUX"]`.
* `namePrefix`: Prepends text to `${name}` parameter. In case of libraries, this
defaults to `"python3.8-"` for Python 3.8, etc., and in case of applications to `""`.
* `pipInstallFlags ? []`: A list of strings. Arguments to be passed to `pip
install`. To pass options to `python setup.py install`, use
`--install-option`. E.g., `pipInstallFlags=["--install-option='--cpp_implementation'"]`.
* `pipBuildFlags ? []`: A list of strings. Arguments to be passed to `pip wheel`.
* `pypaBuildFlags ? []`: A list of strings. Arguments to be passed to `python -m build --wheel`.
* `pythonPath ? []`: List of packages to be added into `$PYTHONPATH`. Packages
in `pythonPath` are not propagated (contrary to [`propagatedBuildInputs`](#var-stdenv-propagatedBuildInputs)).
@@ -214,6 +218,9 @@ because their behaviour is different:
paths included in this list. Items listed in `install_requires` go here.
* `optional-dependencies ? { }`: Optional feature flagged dependencies. Items listed in `extras_requires` go here.
Aside from propagating dependencies,
`buildPythonPackage` also injects code into and wraps executables with the
paths included in this list. Items listed in `extras_requires` go here.
##### Overriding Python packages {#overriding-python-packages}
@@ -300,6 +307,7 @@ python3Packages.buildPythonApplication rec {
build-system = with python3Packages; [
setuptools
wheel
];
dependencies = with python3Packages; [
@@ -313,7 +321,13 @@ python3Packages.buildPythonApplication rec {
}
```
This is then added to `pkgs/by-name` just as any other application would be.
This is then added to `all-packages.nix` just as any other application would be.
```nix
{
luigi = callPackage ../applications/networking/cluster/luigi { };
}
```
Since the package is an application, a consumer doesn't need to care about
Python versions or modules, which is why they don't go in `python3Packages`.
@@ -322,27 +336,25 @@ Python versions or modules, which is why they don't go in `python3Packages`.
A distinction is made between applications and libraries, however, sometimes a
package is used as both. In this case the package is added as a library to
`python-packages.nix` and as an application to `pkgs/by-name`. To reduce
`python-packages.nix` and as an application to `all-packages.nix`. To reduce
duplication the `toPythonApplication` can be used to convert a library to an
application.
The Nix expression shall use [`buildPythonPackage`](#buildpythonpackage-function) and be called from
`python-packages.nix`. A reference shall be created from `pkgs/by-name` to
`python-packages.nix`. A reference shall be created from `all-packages.nix` to
the attribute in `python-packages.nix`, and the `toPythonApplication` shall be
applied to the reference:
```nix
{
python3Packages,
}:
python3Packages.toPythonApplication python3Packages.youtube-dl
youtube-dl = with python3Packages; toPythonApplication youtube-dl;
}
```
#### `toPythonModule` function {#topythonmodule-function}
In some cases, such as bindings, a package is created using
[`stdenv.mkDerivation`](#sec-using-stdenv) and added as attribute in `pkgs/by-name` or in `all-packages.nix`. The Python
[`stdenv.mkDerivation`](#sec-using-stdenv) and added as attribute in `all-packages.nix`. The Python
bindings should be made available from `python-packages.nix`. The
`toPythonModule` function takes a derivation and makes certain Python-specific
modifications.
@@ -358,66 +370,6 @@ modifications.
Do pay attention to passing in the right Python version!
#### `mkPythonMetaPackage` function {#mkpythonmetapackage-function}
This will create a meta package containing [metadata files](https://packaging.python.org/en/latest/specifications/recording-installed-packages/) to satisfy a dependency on a package, without it actually having been installed into the environment.
In nixpkgs this is used to package Python packages with split binary/source distributions such as [psycopg2](https://pypi.org/project/psycopg2/)/[psycopg2-binary](https://pypi.org/project/psycopg2-binary/).
```nix
mkPythonMetaPackage {
pname = "psycopg2-binary";
inherit (psycopg2) optional-dependencies version;
dependencies = [ psycopg2 ];
meta = {
inherit (psycopg2.meta) description homepage;
};
}
```
#### `mkPythonEditablePackage` function {#mkpythoneditablepackage-function}
When developing Python packages it's common to install packages in [editable mode](https://setuptools.pypa.io/en/latest/userguide/development_mode.html).
Like `mkPythonMetaPackage` this function exists to create an otherwise empty package, but also containing a pointer to an impure location outside the Nix store that can be changed without rebuilding.
The editable root is passed as a string. Normally `.pth` files contains absolute paths to the mutable location. This isn't always ergonomic with Nix, so environment variables are expanded at runtime.
This means that a shell hook setting up something like a `$REPO_ROOT` variable can be used as the relative package root.
As an implementation detail, the [PEP-518](https://peps.python.org/pep-0518/) `build-system` specified won't be used, but instead the editable package will be built using [hatchling](https://pypi.org/project/hatchling/).
The `build-system`'s provided will instead become runtime dependencies of the editable package.
Note that overriding packages deeper in the dependency graph _can_ work, but it's not the primary use case and overriding existing packages can make others break in unexpected ways.
``` nix
{ pkgs ? import <nixpkgs> { } }:
let
pyproject = pkgs.lib.importTOML ./pyproject.toml;
myPython = pkgs.python.override {
self = myPython;
packageOverrides = pyfinal: pyprev: {
# An editable package with a script that loads our mutable location
my-editable = pyfinal.mkPythonEditablePackage {
# Inherit project metadata from pyproject.toml
pname = pyproject.project.name;
inherit (pyproject.project) version;
# The editable root passed as a string
root = "$REPO_ROOT/src"; # Use environment variable expansion at runtime
# Inject a script (other PEP-621 entrypoints are also accepted)
inherit (pyproject.project) scripts;
};
};
};
pythonEnv = testPython.withPackages (ps: [ ps.my-editable ]);
in pkgs.mkShell {
packages = [ pythonEnv ];
}
```
#### `python.buildEnv` function {#python.buildenv-function}
Python environments can be created using the low-level `pkgs.buildEnv` function.
@@ -522,11 +474,13 @@ are used in [`buildPythonPackage`](#buildpythonpackage-function).
with the `eggInstallHook`
- `eggBuildHook` to skip building for eggs.
- `eggInstallHook` to install eggs.
- `pipBuildHook` to build a wheel using `pip` and PEP 517. Note a build system
(e.g. `setuptools` or `flit`) should still be added as `build-system`.
- `pypaBuildHook` to build a wheel using
[`pypa/build`](https://pypa-build.readthedocs.io/en/latest/index.html) and
PEP 517/518. Note a build system (e.g. `setuptools` or `flit`) should still
be added as `build-system`.
- `pypaInstallHook` to install wheels.
- `pipInstallHook` to install wheels.
- `pytestCheckHook` to run tests with `pytest`. See [example usage](#using-pytestcheckhook).
- `pythonCatchConflictsHook` to fail if the package depends on two different versions of the same dependency.
- `pythonImportsCheckHook` to check whether importing the listed modules works.
@@ -534,6 +488,7 @@ are used in [`buildPythonPackage`](#buildpythonpackage-function).
See [example usage](#using-pythonrelaxdepshook).
- `pythonRemoveBinBytecode` to remove bytecode from the `/bin` folder.
- `setuptoolsBuildHook` to build a wheel using `setuptools`.
- `setuptoolsCheckHook` to run tests with `python setup.py test`.
- `sphinxHook` to build documentation and manpages using Sphinx.
- `venvShellHook` to source a Python 3 `venv` at the `venvDir` location. A
`venv` is created if it does not yet exist. `postVenvCreation` can be used to
@@ -663,8 +618,7 @@ that sets up an interpreter pointing to them. This matters much more for "big"
modules like `pytorch` or `tensorflow`.
Module names usually match their names on [pypi.org](https://pypi.org/), but
normalized according to PEP 503/508. (e.g. Foo__Bar.baz -> foo-bar-baz)
You can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html)
you can use the [Nixpkgs search website](https://nixos.org/nixos/packages.html)
to find them as well (along with non-python packages).
At this point we can create throwaway experimental Python environments with
@@ -892,6 +846,7 @@ building Python libraries is [`buildPythonPackage`](#buildpythonpackage-function
, buildPythonPackage
, fetchPypi
, setuptools
, wheel
}:
buildPythonPackage rec {
@@ -906,6 +861,7 @@ buildPythonPackage rec {
build-system = [
setuptools
wheel
];
# has no tests
@@ -929,7 +885,7 @@ buildPythonPackage rec {
What happens here? The function [`buildPythonPackage`](#buildpythonpackage-function) is called and as argument
it accepts a set. In this case the set is a recursive set, `rec`. One of the
arguments is the name of the package, which consists of a basename (generally
following the name on PyPI) and a version. Another argument, `src` specifies the
following the name on PyPi) and a version. Another argument, `src` specifies the
source, which in this case is fetched from PyPI using the helper function
`fetchPypi`. The argument `doCheck` is used to set whether tests should be run
when building the package. Since there are no tests, we rely on [`pythonImportsCheck`](#using-pythonimportscheck)
@@ -964,6 +920,7 @@ with import <nixpkgs> {};
build-system = [
python311.pkgs.setuptools
python311.pkgs.wheel
];
# has no tests
@@ -1016,13 +973,13 @@ order to build [`datashape`](https://github.com/blaze/datashape).
, fetchPypi
# build dependencies
, setuptools
, setuptools, wheel
# dependencies
, numpy, multipledispatch, python-dateutil
# tests
, pytestCheckHook
, pytest
}:
buildPythonPackage rec {
@@ -1037,6 +994,7 @@ buildPythonPackage rec {
build-system = [
setuptools
wheel
];
dependencies = [
@@ -1046,21 +1004,21 @@ buildPythonPackage rec {
];
nativeCheckInputs = [
pytestCheckHook
pytest
];
meta = {
changelog = "https://github.com/blaze/datashape/releases/tag/${version}";
homepage = "https://github.com/ContinuumIO/datashape";
description = "Data description language";
description = "A data description language";
license = lib.licenses.bsd2;
};
}
```
We can see several runtime dependencies, `numpy`, `multipledispatch`, and
`python-dateutil`. Furthermore, we have [`nativeCheckInputs`](#var-stdenv-nativeCheckInputs) with `pytestCheckHook`.
`pytestCheckHook` is a test runner hook and is only used during the [`checkPhase`](#ssec-check-phase) and is
`python-dateutil`. Furthermore, we have [`nativeCheckInputs`](#var-stdenv-nativeCheckInputs) with `pytest`.
`pytest` is a test runner and is only used during the [`checkPhase`](#ssec-check-phase) and is
therefore not added to `dependencies`.
In the previous case we had only dependencies on other Python packages to consider.
@@ -1073,6 +1031,7 @@ when building the bindings and are therefore added as [`buildInputs`](#var-stden
, buildPythonPackage
, fetchPypi
, setuptools
, wheel
, libxml2
, libxslt
}:
@@ -1089,6 +1048,7 @@ buildPythonPackage rec {
build-system = [
setuptools
wheel
];
buildInputs = [
@@ -1096,14 +1056,6 @@ buildPythonPackage rec {
libxslt
];
# tests are meant to be ran "in-place" in the same directory as src
doCheck = false;
pythonImportsCheck = [
"lxml"
"lxml.etree"
];
meta = {
changelog = "https://github.com/lxml/lxml/releases/tag/lxml-${version}";
description = "Pythonic binding for the libxml2 and libxslt libraries";
@@ -1131,6 +1083,7 @@ therefore we have to set `LDFLAGS` and `CFLAGS`.
# build dependencies
, setuptools
, wheel
# dependencies
, fftw
@@ -1141,7 +1094,7 @@ therefore we have to set `LDFLAGS` and `CFLAGS`.
}:
buildPythonPackage rec {
pname = "pyfftw";
pname = "pyFFTW";
version = "0.9.2";
pyproject = true;
@@ -1152,6 +1105,7 @@ buildPythonPackage rec {
build-system = [
setuptools
wheel
];
buildInputs = [
@@ -1173,11 +1127,9 @@ buildPythonPackage rec {
# Tests cannot import pyfftw. pyfftw works fine though.
doCheck = false;
pythonImportsCheck = [ "pyfftw" ];
meta = {
changelog = "https://github.com/pyFFTW/pyFFTW/releases/tag/v${version}";
description = "Pythonic wrapper around FFTW, the FFT library, presenting a unified interface for all the supported transforms";
description = "A pythonic wrapper around FFTW, the FFT library, presenting a unified interface for all the supported transforms";
homepage = "http://hgomersall.github.com/pyFFTW";
license = with lib.licenses; [ bsd2 bsd3 ];
};
@@ -1190,8 +1142,10 @@ Note also the line [`doCheck = false;`](#var-stdenv-doCheck), we explicitly disa
It is highly encouraged to have testing as part of the package build. This
helps to avoid situations where the package was able to build and install,
but is not usable at runtime.
Your package should provide its own [`checkPhase`](#ssec-check-phase).
but is not usable at runtime. Currently, all packages will use the `test`
command provided by the setup.py (i.e. `python setup.py test`). However,
this is currently deprecated https://github.com/pypa/setuptools/pull/1878
and your package should provide its own [`checkPhase`](#ssec-check-phase).
::: {.note}
The [`checkPhase`](#ssec-check-phase) for python maps to the `installCheckPhase` on a
@@ -1262,11 +1216,9 @@ been removed, in this case, it's recommended to use `pytestCheckHook`.
#### Using pytestCheckHook {#using-pytestcheckhook}
`pytestCheckHook` is a convenient hook which will set up (or configure)
a [`checkPhase`](#ssec-check-phase) to run `pytest`. This is also beneficial
`pytestCheckHook` is a convenient hook which will substitute the setuptools
`test` command for a [`checkPhase`](#ssec-check-phase) which runs `pytest`. This is also beneficial
when a package may need many items disabled to run the test suite.
Most packages use `pytest` or `unittest`, which is compatible with `pytest`,
so you will most likely use `pytestCheckHook`.
Using the example above, the analogous `pytestCheckHook` usage would be:
@@ -1374,6 +1326,9 @@ we can do:
```nix
{
nativeBuildInputs = [
pythonRelaxDepsHook
];
pythonRelaxDeps = [
"pkg1"
"pkg3"
@@ -1396,6 +1351,7 @@ example:
```nix
{
nativeBuildInputs = [ pythonRelaxDepsHook ];
pythonRelaxDeps = true;
}
```
@@ -1418,18 +1374,9 @@ Keep in mind that while the examples above are done with `requirements.txt`,
`pythonRelaxDepsHook` works by modifying the resulting wheel file, so it should
work with any of the [existing hooks](#setup-hooks).
The `pythonRelaxDepsHook` has no effect on build time dependencies, such as
those specified in `build-system`. If a package requires incompatible build
time dependencies, they should be removed in `postPatch` through
`substituteInPlace` or similar.
For ease of use, both `buildPythonPackage` and `buildPythonApplication` will
automatically add `pythonRelaxDepsHook` if either `pythonRelaxDeps` or
`pythonRemoveDeps` is specified.
#### Using unittestCheckHook {#using-unittestcheckhook}
`unittestCheckHook` is a hook which will set up (or configure) a [`checkPhase`](#ssec-check-phase) to run `python -m unittest discover`:
`unittestCheckHook` is a hook which will substitute the setuptools `test` command for a [`checkPhase`](#ssec-check-phase) which runs `python -m unittest discover`:
```nix
{
@@ -1443,8 +1390,6 @@ automatically add `pythonRelaxDepsHook` if either `pythonRelaxDeps` or
}
```
`pytest` is compatible with `unittest`, so in most cases you can use `pytestCheckHook` instead.
#### Using sphinxHook {#using-sphinxhook}
The `sphinxHook` is a helpful tool to build documentation and manpages
@@ -1523,6 +1468,7 @@ We first create a function that builds `toolz` in `~/path/to/toolz/release.nix`
, buildPythonPackage
, fetchPypi
, setuptools
, wheel
}:
buildPythonPackage rec {
@@ -1537,6 +1483,7 @@ buildPythonPackage rec {
build-system = [
setuptools
wheel
];
meta = {
@@ -1556,9 +1503,10 @@ with import <nixpkgs> {};
( let
toolz = callPackage /path/to/toolz/release.nix {
buildPythonPackage = python3Packages.buildPythonPackage;
buildPythonPackage = python310
Packages.buildPythonPackage;
};
in python3.withPackages (ps: [
in python310.withPackages (ps: [
ps.numpy
toolz
])
@@ -1979,8 +1927,6 @@ because we can only provide security support for non-vendored dependencies.
We recommend [nix-init](https://github.com/nix-community/nix-init) for creating new python packages within nixpkgs,
as it already prefetches the source, parses dependencies for common formats and prefills most things in `meta`.
See also [contributing section](#contributing).
### Are Python interpreters built deterministically? {#deterministic-builds}
The Python interpreters are now built deterministically. Minor modifications had
@@ -1998,15 +1944,16 @@ Both are also exported in `nix-shell`.
It is recommended to test packages as part of the build process.
Source distributions (`sdist`) often include test files, but not always.
The best practice today is to pass a test hook (e.g. pytestCheckHook, unittestCheckHook) into nativeCheckInputs.
This will reconfigure the checkPhase to make use of that particular test framework.
Occasionally packages don't make use of a common test framework, which may then require a custom checkPhase.
By default the command `python setup.py test` is run as part of the
[`checkPhase`](#ssec-check-phase), but often it is necessary to pass a custom [`checkPhase`](#ssec-check-phase). An
example of such a situation is when `py.test` is used.
#### Common issues {#common-issues}
* Non-working tests can often be deselected. Most Python modules
do follow the standard test protocol where the pytest runner can be used.
`pytest` supports the `-k` and `--ignore` parameters to ignore test
* Non-working tests can often be deselected. By default [`buildPythonPackage`](#buildpythonpackage-function)
runs `python setup.py test`. which is deprecated. Most Python modules however
do follow the standard test protocol where the pytest runner can be used
instead. `pytest` supports the `-k` and `--ignore` parameters to ignore test
methods or classes as well as whole files. For `pytestCheckHook` these are
conveniently exposed as `disabledTests` and `disabledTestPaths` respectively.
@@ -2047,25 +1994,14 @@ The following rules are desired to be respected:
* Python applications live outside of `python-packages.nix` and are packaged
with [`buildPythonApplication`](#buildpythonapplication-function).
* Make sure libraries build for all Python interpreters.
If it fails to build on some Python versions, consider disabling them by setting `disable = pythonAtLeast "3.x"` along with a comment.
* The two parameters, `pyproject` and `build-system` are set to avoid the legacy setuptools/distutils build.
* Only unversioned attributes (e.g. `pydantic`, but not `pypdantic_1`) can be included in `dependencies`,
since due to `PYTHONPATH` limitations we can only ever support a single version for libraries
without running into duplicate module name conflicts.
* The version restrictions of `dependencies` can be relaxed by [`pythonRelaxDepsHook`](#using-pythonrelaxdepshook).
* Make sure the tests are enabled using for example [`pytestCheckHook`](#using-pytestcheckhook) and, in the case of
* By default we enable tests. Make sure the tests are found and, in the case of
libraries, are passing for all interpreters. If certain tests fail they can be
disabled individually. Try to avoid disabling the tests altogether. In any
case, when you disable tests, leave a comment explaining why.
* `pythonImportsCheck` is set. This is still a good smoke test even if `pytestCheckHook` is set.
* `meta.platforms` takes the default value in many cases.
It does not need to be set explicitly unless the package requires a specific platform.
* The file is formatted with `nixfmt-rfc-style`.
* Commit names of Python libraries should reflect that they are Python
libraries, so write for example `python311Packages.numpy: 1.11 -> 1.12`.
It is highly recommended to specify the current default version to enable
automatic build by ofborg.
Note that `pythonPackages` is an alias for `python27Packages`.
* Attribute names in `python-packages.nix` as well as `pname`s should match the
library's name on PyPI, but be normalized according to [PEP
0503](https://www.python.org/dev/peps/pep-0503/#normalized-names). This means
@@ -2079,8 +2015,6 @@ The following rules are desired to be respected:
* Attribute names in `python-packages.nix` should be sorted alphanumerically to
avoid merge conflicts and ease locating attributes.
This list is useful for reviewers as well as for self-checking when submitting packages.
## Package set maintenance {#python-package-set-maintenance}
The whole Python package set has a lot of packages that do not see regular
@@ -2090,8 +2024,8 @@ no maintainer, so maintenance falls back to the package set maintainers.
### Updating packages in bulk {#python-package-bulk-updates}
A tool to bulk-update numerous Python libraries is available in the
repository at `maintainers/scripts/update-python-libraries`.
There is a tool to update alot of python libraries in bulk, it exists at
`maintainers/scripts/update-python-libraries` with this repository.
It can quickly update minor or major versions for all packages selected
and create update commits, and supports the `fetchPypi`, `fetchurl` and

View File

@@ -104,27 +104,24 @@ directory and executed as follows:
```bash
nix-shell generate-shell.nix
Rscript generate-r-packages.R cran > cran-packages.json.new
mv cran-packages.json.new cran-packages.json
Rscript generate-r-packages.R cran > cran-packages.nix.new
mv cran-packages.nix.new cran-packages.nix
Rscript generate-r-packages.R bioc > bioc-packages.json.new
mv bioc-packages.json.new bioc-packages.json
Rscript generate-r-packages.R bioc > bioc-packages.nix.new
mv bioc-packages.nix.new bioc-packages.nix
Rscript generate-r-packages.R bioc-annotation > bioc-annotation-packages.json.new
mv bioc-annotation-packages.json.new bioc-annotation-packages.json
Rscript generate-r-packages.R bioc-annotation > bioc-annotation-packages.nix.new
mv bioc-annotation-packages.nix.new bioc-annotation-packages.nix
Rscript generate-r-packages.R bioc-experiment > bioc-experiment-packages.json.new
mv bioc-experiment-packages.json.new bioc-experiment-packages.json
Rscript generate-r-packages.R bioc-experiment > bioc-experiment-packages.nix.new
mv bioc-experiment-packages.nix.new bioc-experiment-packages.nix
```
`generate-r-packages.R <repo>` reads `<repo>-packages.json`, therefore
`generate-r-packages.R <repo>` reads `<repo>-packages.nix`, therefore
the renaming.
The contents of a generated `*-packages.json` file will be used to
create a package derivation for each R package listed in the file.
Some packages require overrides to specify external dependencies or other
patches and special requirements. These overrides are specified in the
`pkgs/development/r-modules/default.nix` file. As the `*-packages.json`
`pkgs/development/r-modules/default.nix` file. As the `*-packages.nix`
contents are automatically generated it should not be edited and broken
builds should be addressed using overrides.

View File

@@ -2,7 +2,7 @@
## Using Ruby {#using-ruby}
Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. The attribute `ruby` refers to the default Ruby interpreter, which is currently MRI 3.3. It's also possible to refer to specific versions, e.g. `ruby_3_y`, `jruby`, or `mruby`.
Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. The attribute `ruby` refers to the default Ruby interpreter, which is currently MRI 3.1. It's also possible to refer to specific versions, e.g. `ruby_3_y`, `jruby`, or `mruby`.
In the Nixpkgs tree, Ruby packages can be found throughout, depending on what they do, and are called from the main package set. Ruby gems, however are separate sets, and there's one default set for each interpreter (currently MRI only).
@@ -154,7 +154,7 @@ let
defaultGemConfig = pkgs.defaultGemConfig // {
pg = attrs: {
buildFlags =
[ "--with-pg-config=${lib.getDev pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
[ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
};
};
};
@@ -172,7 +172,7 @@ let
gemConfig = pkgs.defaultGemConfig // {
pg = attrs: {
buildFlags =
[ "--with-pg-config=${lib.getDev pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
[ "--with-pg-config=${pkgs."postgresql_${pg_version}"}/bin/pg_config" ];
};
};
};
@@ -190,7 +190,9 @@ let
defaultGemConfig = super.defaultGemConfig // {
pg = attrs: {
buildFlags = [
"--with-pg-config=${lib.getDev pkgs."postgresql_${pg_version}"}/bin/pg_config"
"--with-pg-config=${
pkgs."postgresql_${pg_version}"
}/bin/pg_config"
];
};
};

View File

@@ -38,24 +38,19 @@ rustPlatform.buildRustPackage rec {
cargoHash = "sha256-jtBw4ahSl88L0iuCXxQgZVm1EcboWRJMNtjxLVTtzts=";
meta = {
description = "Fast line-oriented regex search tool, similar to ag and ack";
description = "A fast line-oriented regex search tool, similar to ag and ack";
homepage = "https://github.com/BurntSushi/ripgrep";
license = lib.licenses.unlicense;
maintainers = [ ];
maintainers = [];
};
}
```
`buildRustPackage` requires a `cargoHash` attribute, computed over all crate sources of this package.
::: {.warning}
`cargoSha256` is already deprecated, and is subject to removal in favor of
`cargoHash` which supports [SRI](https://www.w3.org/TR/SRI/) hashes.
If you are still using `cargoSha256`, you can simply replace it with
`cargoHash` and recompute the hash, or convert the original sha256 to SRI
hash using `nix-hash --to-sri --type sha256 "<original sha256>"`.
:::
`buildRustPackage` requires either a `cargoHash` (preferred) or a
`cargoSha256` attribute, computed over all crate sources of this package.
`cargoHash` supports [SRI](https://www.w3.org/TR/SRI/) hashes and should be
preferred over `cargoSha256` which was used for traditional Nix SHA-256 hashes.
For example:
```nix
{
@@ -63,7 +58,7 @@ hash using `nix-hash --to-sri --type sha256 "<original sha256>"`.
}
```
Exception: If the application has cargo `git` dependencies, the `cargoHash`
Exception: If the application has cargo `git` dependencies, the `cargoHash`/`cargoSha256`
approach will not work, and you will need to copy the `Cargo.lock` file of the application
to nixpkgs and continue with the next section for specifying the options of the `cargoLock`
section.
@@ -81,6 +76,14 @@ then be taken from the failed build. A fake hash can be used for
}
```
For `cargoSha256` you can use:
```nix
{
cargoSha256 = lib.fakeSha256;
}
```
Per the instructions in the [Cargo Book](https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html)
best practices guide, Rust applications should always commit the `Cargo.lock`
file in git to ensure a reproducible build. However, a few packages do not, and
@@ -95,7 +98,7 @@ directory into a tar.gz archive.
The tarball with vendored dependencies contains a directory with the
package's `name`, which is normally composed of `pname` and
`version`. This means that the vendored dependencies hash
(`cargoHash`) is dependent on the package name and
(`cargoHash`/`cargoSha256`) is dependent on the package name and
version. The `cargoDepsName` attribute can be used to use another name
for the directory of vendored dependencies. For example, the hash can
be made invariant to the version by setting `cargoDepsName` to
@@ -120,7 +123,7 @@ rustPlatform.buildRustPackage rec {
### Importing a `Cargo.lock` file {#importing-a-cargo.lock-file}
Using a vendored hash (`cargoHash`) is tedious when using
Using a vendored hash (`cargoHash`/`cargoSha256`) is tedious when using
`buildRustPackage` within a project, since it requires that the hash
is updated after every change to `Cargo.lock`. Therefore,
`buildRustPackage` also supports vendoring dependencies directly from
@@ -251,7 +254,7 @@ By default, it takes the `stdenv.hostPlatform.config` and replaces components
where they are known to differ. But there are ways to customize the argument:
- To choose a different target by name, define
`stdenv.hostPlatform.rust.rustcTarget` as that name (a string), and that
`stdenv.hostPlatform.rustc.config` as that name (a string), and that
name will be used instead.
For example:
@@ -259,7 +262,7 @@ where they are known to differ. But there are ways to customize the argument:
```nix
import <nixpkgs> {
crossSystem = (import <nixpkgs/lib>).systems.examples.armhf-embedded // {
rust.rustcTarget = "thumbv7em-none-eabi";
rustc.config = "thumbv7em-none-eabi";
};
}
```
@@ -271,10 +274,10 @@ where they are known to differ. But there are ways to customize the argument:
```
- To pass a completely custom target, define
`stdenv.hostPlatform.rust.rustcTarget` with its name, and
`stdenv.hostPlatform.rust.platform` with the value. The value will be
`stdenv.hostPlatform.rustc.config` with its name, and
`stdenv.hostPlatform.rustc.platform` with the value. The value will be
serialized to JSON in a file called
`${stdenv.hostPlatform.rust.rustcTarget}.json`, and the path of that file
`${stdenv.hostPlatform.rustc.config}.json`, and the path of that file
will be used instead.
For example:
@@ -282,8 +285,8 @@ where they are known to differ. But there are ways to customize the argument:
```nix
import <nixpkgs> {
crossSystem = (import <nixpkgs/lib>).systems.examples.armhf-embedded // {
rust.rustcTarget = "thumb-crazy";
rust.platform = { foo = ""; bar = ""; };
rustc.config = "thumb-crazy";
rustc.platform = { foo = ""; bar = ""; };
};
}
```
@@ -567,7 +570,8 @@ buildPythonPackage rec {
};
cargoDeps = rustPlatform.fetchCargoTarball {
inherit pname version src sourceRoot;
inherit src sourceRoot;
name = "${pname}-${version}";
hash = "sha256-miW//pnOmww2i6SOGbkrAIdc/JMDT4FJLqdMFojZeoY=";
};
@@ -610,8 +614,9 @@ buildPythonPackage rec {
};
cargoDeps = rustPlatform.fetchCargoTarball {
inherit pname version src;
inherit src;
sourceRoot = "${pname}-${version}/${cargoRoot}";
name = "${pname}-${version}";
hash = "sha256-PS562W4L1NimqDV2H0jl5vYhL08H9est/pbIxSdYVfo=";
};
@@ -640,7 +645,6 @@ builds the `retworkx` Python package. `fetchCargoTarball` and
buildPythonPackage rec {
pname = "retworkx";
version = "0.6.0";
pyproject = true;
src = fetchFromGitHub {
owner = "Qiskit";
@@ -650,10 +654,13 @@ buildPythonPackage rec {
};
cargoDeps = rustPlatform.fetchCargoTarball {
inherit pname version src;
inherit src;
name = "${pname}-${version}";
hash = "sha256-heOBK8qi2nuc/Ib+I/vLzZ1fUUD/G/KTw9d7M4Hz5O0=";
};
format = "pyproject";
nativeBuildInputs = with rustPlatform; [ cargoSetupHook maturinBuildHook ];
# ...
@@ -694,7 +701,8 @@ stdenv.mkDerivation rec {
};
cargoDeps = rustPlatform.fetchCargoTarball {
inherit pname version src;
inherit src;
name = "${pname}-${version}";
hash = "sha256-8fa3fa+sFi5H+49B5sr2vYPkp9C9s6CcE0zv4xB8gww=";
};
@@ -992,7 +1000,7 @@ rustPlatform.buildRustPackage rec {
doCheck = false;
meta = {
description = "Fast line-oriented regex search tool, similar to ag and ack";
description = "A fast line-oriented regex search tool, similar to ag and ack";
homepage = "https://github.com/BurntSushi/ripgrep";
license = with lib.licenses; [ mit unlicense ];
maintainers = with lib.maintainers; [];

View File

@@ -1,35 +0,0 @@
# Scheme {#sec-scheme}
## Package Management {#sec-scheme-package-management}
### Akku {#sec-scheme-package-management-akku}
About two hundred R6RS & R7RS libraries from [Akku](https://akkuscm.org/)
(which also mirrors [snow-fort](https://snow-fort.org/pkg))
are available inside the `akkuPackages` attrset, and the Akku executable
itself is at the top level as `akku`. The packages could be used
in a derivation's `buildInputs`, work inside of `nix-shell`, and
are tested using [Chez](https://www.scheme.com/) &
[Chibi](https://synthcode.com/wiki/chibi-scheme)
Scheme during build time.
Including a package as a build input is done in the typical Nix fashion.
For example, to include
[a bunch of SRFIs](https://akkuscm.org/packages/chez-srfi/)
primarily for Chez Scheme in a derivation, one might write:
```nix
{
buildInputs = [
chez
akkuPackages.chez-srfi
];
}
```
The package index is located in `pkgs/tools/package-management/akku`
as `deps.toml`, and should be updated occasionally by running `./update.sh`
in the directory. Doing so will pull the source URLs for new packages and
more recent versions, then write them to the TOML.

View File

@@ -83,13 +83,12 @@ Release 23.11 ships with a new interface that will eventually replace `texlive.c
```nix
stdenvNoCC.mkDerivation rec {
src = texlive.pkgs.iwona;
dontUnpack = true;
inherit (src) pname version;
installPhase = ''
runHook preInstall
install -Dm644 $src/fonts/opentype/nowacki/iwona/*.otf -t $out/share/fonts/opentype
install -Dm644 fonts/opentype/nowacki/iwona/*.otf -t $out/share/fonts/opentype
runHook postInstall
'';
}
@@ -183,7 +182,7 @@ let
'';
meta = {
description = "LaTeX2e class for overhead transparencies";
description = "A LaTeX2e class for overhead transparencies";
license = lib.licenses.unfreeRedistributable;
maintainers = with lib.maintainers; [ veprbl ];
platforms = lib.platforms.all;

View File

@@ -214,7 +214,7 @@ Note: this is not possible anymore for Neovim.
Nix expressions for Vim plugins are stored in [pkgs/applications/editors/vim/plugins](https://github.com/NixOS/nixpkgs/tree/master/pkgs/applications/editors/vim/plugins). For the vast majority of plugins, Nix expressions are automatically generated by running [`nix-shell -p vimPluginsUpdater --run vim-plugins-updater`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/updater.nix). This creates a [generated.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/generated.nix) file based on the plugins listed in [vim-plugin-names](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/vim-plugin-names).
When the vim updater detects an nvim-treesitter update, it also runs [`nvim-treesitter/update.py $(nix-build -A vimPlugins.nvim-treesitter)`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/update.py) to update the tree sitter grammars for `nvim-treesitter`.
After running the updater, if nvim-treesitter received an update, also run [`nvim-treesitter/update.py`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/update.py) to update the tree sitter grammars for `nvim-treesitter`.
Some plugins require overrides in order to function properly. Overrides are placed in [overrides.nix](https://github.com/NixOS/nixpkgs/blob/master/pkgs/applications/editors/vim/plugins/overrides.nix). Overrides are most often required when a plugin requires some dependencies, or extra steps are required during the build process. For example `deoplete-fish` requires both `deoplete-nvim` and `vim-fish`, and so the following override was added:
@@ -226,26 +226,18 @@ Some plugins require overrides in order to function properly. Overrides are plac
}
```
Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `nix-shell -p vimPluginsUpdater --run vim-plugins-updater` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`.
Sometimes plugins require an override that must be changed when the plugin is updated. This can cause issues when Vim plugins are auto-updated but the associated override isn't updated. For these plugins, the override should be written so that it specifies all information required to install the plugin, and running `./update.py` doesn't change the derivation for the plugin. Manually updating the override is required to update these types of plugins. An example of such a plugin is `LanguageClient-neovim`.
To add a new plugin, run `nix-shell -p vimPluginsUpdater --run 'vim-plugins-updater add "[owner]/[name]"'`. **NOTE**: This script automatically commits to your git repository. Be sure to check out a fresh branch before running.
Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `vimPluginsUpdater` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of the Language Server Protocol integration with Vim/Neovim.
### Plugin optional configuration {#vim-plugin-required-snippet}
Some plugins require specific configuration to work. We choose not to
patch those plugins but expose the necessary configuration under
`PLUGIN.passthru.initLua` for neovim plugins. For instance, the `unicode-vim` plugin
needs the path towards a unicode database so we expose the following snippet `vim.g.Unicode_data_directory="${self.unicode-vim}/autoload/unicode"` under `vimPlugins.unicode-vim.passthru.initLua`.
To add a new plugin, run `./update.py add "[owner]/[name]"`. **NOTE**: This script automatically commits to your git repository. Be sure to check out a fresh branch before running.
Finally, there are some plugins that are also packaged in nodePackages because they have Javascript-related build steps, such as running webpack. Those plugins are not listed in `vim-plugin-names` or managed by `update.py` at all, and are included separately in `overrides.nix`. Currently, all these plugins are related to the `coc.nvim` ecosystem of the Language Server Protocol integration with Vim/Neovim.
## Updating plugins in nixpkgs {#updating-plugins-in-nixpkgs}
Run the update script with a GitHub API token that has at least `public_repo` access. Running the script without the token is likely to result in rate-limiting (429 errors). For steps on creating an API token, please refer to [GitHub's token documentation](https://docs.github.com/en/free-pro-team@latest/github/authenticating-to-github/creating-a-personal-access-token).
```sh
nix-shell -p vimPluginsUpdater --run 'vim-plugins-updater --github-token=mytoken' # or set GITHUB_API_TOKEN environment variable
GITHUB_API_TOKEN=my_token ./pkgs/applications/editors/vim/plugins/update.py
```
Alternatively, set the number of processes to a lower count to avoid rate-limiting.

View File

@@ -12,5 +12,4 @@ stdenv.md
build-helpers.md
development.md
contributing.md
interoperability.md
```

View File

@@ -1,102 +0,0 @@
# Build Support {#sec-build-support}
## `pkgs.substitute` {#pkgs-substitute}
`pkgs.substitute` is a wrapper around [the `substitute` Bash function](#fun-substitute) in the standard environment.
It replaces strings in `src` as specified by the `substitutions` argument.
:::{.example #ex-pkgs-substitute}
# Usage of `pkgs.substitute`
In a build script, the line:
```bash
substitute $infile $outfile --replace-fail @foo@ ${foopkg}/bin/foo
```
is equivalent to:
```nix
{ substitute, foopkg }:
substitute {
src = ./sourcefile.txt;
substitutions = [
"--replace"
"@foo@"
"${foopkg}/bin/foo"
];
}
```
:::
## `pkgs.substituteAll` {#pkgs-substituteall}
`pkgs.substituteAll` substitutes all instances of `@varName@` (`@`s included) in file `src` with the value of the corresponding environment variable.
As this uses the [`substituteAll`] (#fun-substitute) function, its limitations regarding variable names that will or will not be replaced also apply here.
:::{.example #ex-pkgs-substituteAll}
# Usage of `pkgs.substituteAll`
If `say-goodbye.sh` contains the following:
```bash
#! @bash@/bin/bash
echo @unchanged@
@hello@/bin/hello --greeting @greeting@
```
the following derivation will make substitutions to `@bash@`, `@hello@`, and `@greeting@`:
```nix
{
substituteAll,
bash,
hello,
}:
substituteAll {
src = ./say-goodbye.sh;
env = {
inherit bash hello;
greeting = "goodbye";
};
}
```
such that `$out` will result in something like the following:
```
#! /nix/store/s30jrpgav677fpc9yvkqsib70xfmx7xi-bash-5.2p26/bin/bash
echo @unchanged@
/nix/store/566f5isbvw014h7knmzmxa5l6hshx43k-hello-2.12.1/bin/hello --greeting goodbye
```
:::
## `pkgs.substituteAllFiles` {#pkgs-substituteallfiles}
`pkgs.substituteAllFiles` replaces `@varName@` with the value of the environment variable `varName`.
It expects `src` to be a directory and requires a `files` argument that specifies which files will be subject to replacements; only these files will be placed in `$out`.
As it also uses the `substituteAll` function, it is subject to the same limitations on environment variables as discussed in [pkgs.substituteAll](#pkgs-substituteall).
:::{.example #ex-pkgs-substitute-all-files}
# Usage of `pkgs.substituteAllFiles`
If the current directory contains `{foo,bar,baz}.txt` and the following `default.nix`
```nix
{ substituteAllFiles }:
substituteAllFiles {
src = ./.;
files = [
"foo.txt"
"bar.txt"
];
hello = "there";
}
```
in the resulting derivation, every instance of `@hello@` will be replaced with `there` in `$out/foo.txt` and` `$out/bar.txt`; `baz.txt` will not be processed nor will it appear in `$out`.
:::

View File

@@ -1,15 +0,0 @@
# Geant4 {#geant4}
[Geant4](https://www.geant4.org/) is a toolkit for simulating how particles pass through matter. It is available through the `geant4` package.
## Setup hook {#geant4-hook}
The setup hook included in the package applies the environment variables set by the [`geant4.sh` script](https://github.com/Geant4/geant4/blob/master/cmake/Modules/G4ConfigureGNUMakeHelpers.cmake#L4-L55), which is typically necessary for compiling `make`-based programs that depend on Geant4.
## Datasets {#geant4-datasets}
All of [the Geant4 datasets provided by CERN](https://geant4.web.cern.ch/support/download) are available through the `geant4.data` attrset.
### Setup hook {#geant4-datasets-hook}
The hook provided by the packages in `geant4.data` will set an appropriate environment variable in the form of `G4[...]DATA`. For example, for the `G4RadioactiveDecay` dataset, the `G4RADIOACTIVEDATA` environment variable is set to the value expected by Geant4.

View File

@@ -11,8 +11,7 @@ On NixOS, you need to explicitly enable `ibus` with given engines before customi
```nix
{ pkgs, ... }: {
i18n.inputMethod = {
enable = true;
type = "ibus";
enabled = "ibus";
ibus.engines = with pkgs.ibus-engines; [ typing-booster ];
};
}

View File

@@ -12,12 +12,10 @@ emacs.section.md
firefox.section.md
fish.section.md
fuse.section.md
geant4.section.md
ibus.section.md
kakoune.section.md
krita.section.md
linux.section.md
lhapdf.section.md
locales.section.md
etc-files.section.md
nginx.section.md
@@ -26,8 +24,6 @@ shell-helpers.section.md
steam.section.md
cataclysm-dda.section.md
urxvt.section.md
vcpkg.section.md
weechat.section.md
xorg.section.md
build-support.md
```

View File

@@ -1,11 +0,0 @@
# LHAPDF {#lhapdf}
[LHAPDF](https://lhapdf.hepforge.org/) is a tool for evaluating parton distribution functions (PDFs) in high-energy physics. LHAPDF is available in the `lhapdf` package.
## PDF sets {#lhapdf-sets}
All of [the PDF sets made available by the LHAPDF project](https://lhapdf.hepforge.org/pdfsets.html) are available through the `lhapdf.pdf_sets` attrset.
### Setup hook {#lhapdf-sets-hook}
Each package provided in the `lhapdf.pdf_sets` attrset contains a setup hook which adds itself to [the `LHAPDF_DATA_PATH` environment variable](https://lhapdf.hepforge.org/#sets).

View File

@@ -40,42 +40,48 @@ pkgs.linux_latest.override {
## Manual kernel configuration {#sec-manual-kernel-configuration}
Sometimes it may not be desirable to use kernels built with `pkgs.buildLinux`, especially if most of the common configuration has to be altered or disabled to achieve a kernel as expected by the target use case.
An example of this is building a kernel for use in a VM or micro VM. You can use `pkgs.linuxPackages_custom` in these cases. It requires the `src`, `version`, and `configfile` attributes to be specified.
An example of this is building a kernel for use in a VM or micro VM. You can use `pkgs.linuxManualConfig` in these cases. It requires the `src`, `version`, and `configfile` attributes to be specified.
:::{.example #ex-using-linux-manual-config}
# Using `pkgs.linuxPackages_custom` with a specific source, version, and config file
# Using `pkgs.linuxManualConfig` with a specific source, version, and config file
```nix
{ pkgs, ... }:
pkgs.linuxPackages_custom {
{ pkgs, ... }: {
version = "6.1.55";
src = pkgs.fetchurl {
url = "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-${version}.tar.xz";
hash = "sha256-qH4kHsFdU0UsTv4hlxOjdp2IzENrW5jPbvsmLEr/FcA=";
hash = "sha256:1h0mzx52q9pvdv7rhnvb8g68i7bnlc9rf8gy9qn4alsxq4g28zm8";
};
configfile = ./path_to_config_file;
linux = pkgs.linuxManualConfig {
inherit version src configfile;
allowImportFromDerivation = true;
};
}
```
If necessary, the version string can be slightly modified to explicitly mark it as a custom version. If you do so, ensure the `modDirVersion` attribute matches the source's version, otherwise the build will fail.
```nix
{ pkgs, ... }:
pkgs.linuxPackages_custom {
{ pkgs, ... }: {
version = "6.1.55-custom";
modDirVersion = "6.1.55";
src = pkgs.fetchurl {
url = "https://cdn.kernel.org/pub/linux/kernel/v6.x/linux-${modDirVersion}.tar.xz";
hash = "sha256-qH4kHsFdU0UsTv4hlxOjdp2IzENrW5jPbvsmLEr/FcA=";
hash = "sha256:1h0mzx52q9pvdv7rhnvb8g68i7bnlc9rf8gy9qn4alsxq4g28zm8";
};
configfile = ./path_to_config_file;
linux = pkgs.linuxManualConfig {
inherit version modDirVersion src configfile;
allowImportFromDerivation = true;
};
}
```
:::
Additional attributes can be used with `linuxManualConfig` for further customisation instead of `linuxPackages_custom`. You're encouraged to read [the `pkgs.linuxManualConfig` source code](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/manual-config.nix) to understand how to use them.
Additional attributes can be used with `linuxManualConfig` for further customisation. You're encouraged to read [the `pkgs.linuxManualConfig` source code](https://github.com/NixOS/nixpkgs/blob/d77bda728d5041c1294a68fb25c79e2d161f62b9/pkgs/os-specific/linux/kernel/manual-config.nix) to understand how to use them.
To edit the `.config` file for Linux X.Y from within Nix, proceed as follows:

View File

@@ -40,6 +40,20 @@ Use `programs.steam.enable = true;` if you want to add steam to `systemPackages`
have a look at [this pull request](https://github.com/NixOS/nixpkgs/pull/20269).
- **Java**
1. There is no java in steam chrootenv by default. If you get a message like:
```
/home/foo/.local/share/Steam/SteamApps/common/towns/towns.sh: line 1: java: command not found
```
you need to add:
```nix
steam.override { withJava = true; }
```
## steam-run {#sec-steam-run}
The FHS-compatible chroot used for Steam can also be used to run other Linux games that expect a FHS environment. To use it, install the `steam-run` package and run the game with:

View File

@@ -1,24 +0,0 @@
# VCPKG {#sec-vcpkg}
The `vcpkg-tool` package has a wrapper around the `vcpkg` executable to avoid writing to the nix store.
The wrapper will also be present in `vcpkg`, unless you specify `vcpkg.override { vcpkg-tool = vcpkg-tool-unwrapped; }`
The wrapper has been made in a way so that it will provide default cli arguments, but tries not to interfere if the user provides the same arguments.
The arguments also have corresponding environment variables that can be used as an alternative way of overriding these paths.
Run the wrapper with the environment variable `NIX_VCPKG_DEBUG_PRINT_ENVVARS=true` to get a full list of corresponding environment variables.
## Nix specific environment variables {#sec-vcpkg-nix-envvars}
The wrapper also provides some new nix-specific environment variables that lets you control some of the wrapper functionality.
- `NIX_VCPKG_WRITABLE_PATH = <path>`
Set this environment variable to specify the path where `vcpkg` will store buildtime artifacts.
This will become the base path for all of the other paths.
- `NIX_VCPKG_DEBUG_PRINT_ENVVARS = true | false`
Set this to `true` for the wrapper to print the corresponding environment variables for the arguments that will be provided to the unwrapped executable.
The list of variables will be printed right before invoking `vcpkg`.
This can be useful if you suspect that the wrapper for some reason was unable to prioritize user-provided cli args over its default ones, or for fixing other issues like typos or unexpanded environment variables.

View File

@@ -1,7 +1,20 @@
let
pkgs = import ../. {
config = { };
overlays = [ ];
config = {};
overlays = [];
};
common = import ./common.nix;
inherit (common) outputPath indexPath;
web-devmode = import ../pkgs/tools/nix/web-devmode.nix {
inherit pkgs;
buildArgs = "./.";
open = "/${outputPath}/${indexPath}";
};
in
pkgs.nixpkgs-manual.shell
pkgs.mkShell {
packages = [
web-devmode
];
}

View File

@@ -3,7 +3,6 @@
```{=include=} chapters
stdenv/stdenv.chapter.md
stdenv/meta.chapter.md
stdenv/passthru.chapter.md
stdenv/multiple-output.chapter.md
stdenv/cross-compilation.chapter.md
stdenv/platform-notes.chapter.md

View File

@@ -5,7 +5,7 @@ Nix packages can declare *meta-attributes* that contain information about a pack
```nix
{
meta = {
description = "Program that produces a familiar, friendly greeting";
description = "A program that produces a familiar, friendly greeting";
longDescription = ''
GNU Hello is a program that prints "Hello, world!" when you run it.
It is fully customizable.
@@ -22,10 +22,6 @@ Meta-attributes are not passed to the builder of the package. Thus, a change to
## Standard meta-attributes {#sec-standard-meta-attributes}
If the package is to be submitted to Nixpkgs, please check out the
[requirements for meta attributes](https://github.com/NixOS/nixpkgs/tree/master/pkgs#meta-attributes)
in the contributing documentation.
It is expected that each meta-attribute is one of the following:
### `description` {#var-meta-description}
@@ -33,21 +29,11 @@ It is expected that each meta-attribute is one of the following:
A short (one-line) description of the package.
This is displayed on [search.nixos.org](https://search.nixos.org/packages).
The general requirements of a description are:
- Be short, just one sentence.
- Be capitalized.
- Not start with definite ("The") or indefinite ("A"/"An") article.
- Not start with the package name.
- More generally, it should not refer to the package name.
- Not end with a period (or any punctuation for that matter).
- Provide factual information.
- Avoid subjective language.
Dont include a period at the end. Dont include newline characters. Capitalise the first character. For brevity, dont repeat the name of package --- just describe what it does.
Wrong: `"libpng is a library that allows you to decode PNG images."`
Right: `"Library for decoding PNG images"`
Right: `"A library for decoding PNG images"`
### `longDescription` {#var-meta-longDescription}
@@ -81,12 +67,6 @@ The license, or licenses, for the package. One from the attribute set defined in
For details, see [Licenses](#sec-meta-license).
### `sourceProvenance` {#var-meta-sourceProvenance}
A list containing the type or types of source inputs from which the package is built, e.g. original source code, pre-built binaries, etc.
For details, see [Source provenance](#sec-meta-sourceProvenance).
### `maintainers` {#var-meta-maintainers}
A list of the maintainers of this Nix expression. Maintainers are defined in [`nixpkgs/maintainers/maintainer-list.nix`](https://github.com/NixOS/nixpkgs/blob/master/maintainers/maintainer-list.nix). There is no restriction to becoming a maintainer, just add yourself to that list in a separate commit titled “maintainers: add alice” in the same pull request, and reference maintainers with `maintainers = with lib.maintainers; [ alice bob ]`.
@@ -130,6 +110,83 @@ Some packages use this to automatically detect the maximum set of features with
For example, `systemd` [requires dynamic linking](https://github.com/systemd/systemd/issues/20600#issuecomment-912338965), and [has a `meta.badPlatforms` setting](https://github.com/NixOS/nixpkgs/blob/b03ac42b0734da3e7be9bf8d94433a5195734b19/pkgs/os-specific/linux/systemd/default.nix#L752) similar to the one above.
Packages which can be built with or without `systemd` support will use `lib.meta.availableOn` to detect whether or not `systemd` is available on the [`hostPlatform`](#ssec-cross-platform-parameters) for which they are being built; if it is not available (e.g. due to a statically-linked host platform like `pkgsStatic`) this support will be disabled by default.
### `tests` {#var-meta-tests}
::: {.warning}
This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations.
:::
An attribute set with tests as values. A test is a derivation that builds when the test passes and fails to build otherwise.
You can run these tests with:
```ShellSession
$ cd path/to/nixpkgs
$ nix-build -A your-package.tests
```
#### Package tests {#var-meta-tests-packages}
Tests that are part of the source package are often executed in the `installCheckPhase`.
Prefer `passthru.tests` for tests that are introduced in nixpkgs because:
* `passthru.tests` tests the 'real' package, independently from the environment in which it was built
* we can run `passthru.tests` independently
* `installCheckPhase` adds overhead to each build
For more on how to write and run package tests, see [](#sec-package-tests).
#### NixOS tests {#var-meta-tests-nixos}
The NixOS tests are available as `nixosTests` in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
```nix
{ /* ... , */ nixosTests }:
{
# ...
passthru.tests = {
basic-functionality-and-dovecot-integration = nixosTests.opensmtpd;
};
}
```
NixOS tests run in a VM, so they are slower than regular package tests. For more information see [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
Alternatively, you can specify other derivations as tests. You can make use of
the optional parameter to inject the correct package without
relying on non-local definitions, even in the presence of `overrideAttrs`.
Here that's `finalAttrs.finalPackage`, but you could choose a different name if
`finalAttrs` already exists in your scope.
`(mypkg.overrideAttrs f).passthru.tests` will be as expected, as long as the
definition of `tests` does not rely on the original `mypkg` or overrides it in
all places.
```nix
# my-package/default.nix
{ stdenv, callPackage }:
stdenv.mkDerivation (finalAttrs: {
# ...
passthru.tests.example = callPackage ./example.nix { my-package = finalAttrs.finalPackage; };
})
```
```nix
# my-package/example.nix
{ runCommand, lib, my-package, ... }:
runCommand "my-package-test" {
nativeBuildInputs = [ my-package ];
src = lib.sources.sourcesByRegex ./. [ ".*.in" ".*.expected" ];
} ''
my-package --help
my-package <example.in >example.actual
diff -U3 --color=auto example.expected example.actual
mkdir $out
''
```
### `timeout` {#var-meta-timeout}
A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, Hydra will fail it due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in `nixpkgs`.

View File

@@ -1,134 +0,0 @@
# Passthru-attributes {#chap-passthru}
[]{#var-stdenv-passthru} []{#special-variables} <!-- legacy anchors -->
As opposed to most other `mkDerivation` input attributes, `passthru` is not passed to the derivation's [`builder` executable](https://nixos.org/manual/nix/stable/expressions/derivations.html#attr-builder).
Changing it will not trigger a rebuild it is "passed through".
Its value can be accessed as if it was set inside a derivation.
::: {.note}
`passthru` attributes follow no particular schema, but there are a few [conventional patterns](#sec-common-passthru-attributes).
:::
:::{.example #ex-accessing-passthru}
## Setting and accessing `passthru` attributes
```nix
{ stdenv, fetchGit }:
let
hello = stdenv.mkDerivation {
pname = "hello";
src = fetchGit { /* ... */ };
passthru = {
foo = "bar";
baz = {
value1 = 4;
value2 = 5;
};
};
};
in
hello.baz.value1
```
```
4
```
:::
## Common `passthru`-attributes {#sec-common-passthru-attributes}
Many `passthru` attributes are situational, so this section only lists recurring patterns.
They fall in one of these categories:
- Global conventions, which are applied almost universally in Nixpkgs.
Generally these don't entail any special support built into the derivation they belong to.
Common examples of this type are [`passthru.tests`](#var-passthru-tests) and [`passthru.updateScript`](#var-passthru-updateScript).
- Conventions for adding extra functionality to a derivation.
These tend to entail support from the derivation or the `passthru` attribute in question.
Common examples of this type are `passthru.optional-dependencies`, `passthru.withPlugins`, and `passthru.withPackages`.
All of those allow associating the package with a set of components built for that specific package, such as when building Python runtime environments using (`python.withPackages`)[#python.withpackages-function].
Attributes that apply only to particular [build helpers](#part-builders) or [language ecosystems](#chap-language-support) are documented there.
### `passthru.tests` {#var-passthru-tests}
[]{#var-meta-tests} <!-- legacy anchor -->
An attribute set with tests as values.
A test is a derivation that builds when the test passes and fails to build otherwise.
Run these tests with:
```ShellSession
$ cd path/to/nixpkgs
$ nix-build -A your-package.tests
```
:::{.note}
The Nixpkgs systems for continuous integration [Hydra](https://hydra.nixos.org/) and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) don't build these derivations by default, and ([`@ofborg`](https://github.com/NixOS/ofborg)) only builds them when evaluating pull requests for that particular package, or when manually instructed.
:::
#### Package tests {#var-passthru-tests-packages}
[]{#var-meta-tests-packages} <!-- legacy anchor -->
Besides tests provided by upstream, that you run in the [`checkPhase`](#ssec-check-phase), you may want to define tests derivations in the `passthru.tests` attribute, which won't change the build. `passthru.tests` have several advantages over running tests during any of the [standard phases](#sec-stdenv-phases):
- They access the package as consumers would, independently from the environment in which it was built
- They can be run and debugged without rebuilding the package, which is useful if that takes a long time
- They don't add overhead to each build, as opposed checks added to the [`distPhase`](#ssec-distribution-phase), such as [`versionCheckHook`](#versioncheckhook).
It is also possible to use `passthru.tests` to test the version with [`testVersion`](#tester-testVersion), but since that is pretty trivial and recommended thing to do, we recommend using [`versionCheckHook`](#versioncheckhook) for that, which has the following advantages over `passthru.tests`:
- If the `versionCheckPhase` (the phase defined by [`versionCheckHook`](#versioncheckhook)) fails, it triggers a failure which can't be ignored if you use the package, or if you find out about it in a [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) report.
- Sometimes packages become silently broken - meaning they fail to launch but their build passes because they don't perform any tests in the `checkPhase`. If you use this tool infrequently, such a silent breakage may rot in your system / profile configuration, and you will not notice the failure until you will want to use this package. Testing such basic functionality ensures you have to deal with the failure when you update your system / profile.
- When you open a PR, [ofborg](https://github.com/NixOS/ofborg)'s CI _will_ run `passthru.tests` of [packages that are directly changed by your PR (according to your commits' messages)](https://github.com/NixOS/ofborg?tab=readme-ov-file#automatic-building), but if you'd want to use the [`@ofborg build`](https://github.com/NixOS/ofborg?tab=readme-ov-file#build) command for dependent packages, you won't have to specify in addition the `.tests` attribute of the packages you want to build, and no body will be able to avoid these tests.
<!-- NOTE(@fricklerhandwerk): one may argue whether that testing guide should rather be in the user's manual -->
For more on how to write and run package tests for Nixpkgs, see the [testing section in the package contributor guide](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests).
#### NixOS tests {#var-passthru-tests-nixos}
[]{#var-meta-tests-nixos} <!-- legacy anchor -->
Tests written for NixOS are available as the `nixosTests` argument to package recipes.
For instance, the [OpenSMTPD derivation](https://search.nixos.org/packages?show=opensmtpd) includes lines similar to:
```nix
{ nixosTests, ... }:
{
# ...
passthru.tests = {
basic-functionality-and-dovecot-integration = nixosTests.opensmtpd;
};
}
```
NixOS tests run in a virtual machine (VM), so they are slower than regular package tests.
For more information see the NixOS manual on [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
### `passthru.updateScript` {#var-passthru-updateScript}
<!-- legacy anchors -->
[]{#var-passthru-updateScript-command}
[]{#var-passthru-updateScript-set-command}
[]{#var-passthru-updateScript-set-attrPath}
[]{#var-passthru-updateScript-set-supportedFeatures}
[]{#var-passthru-updateScript-env-UPDATE_NIX_NAME}
[]{#var-passthru-updateScript-env-UPDATE_NIX_PNAME}
[]{#var-passthru-updateScript-env-UPDATE_NIX_OLD_VERSION}
[]{#var-passthru-updateScript-env-UPDATE_NIX_ATTR_PATH}
[]{#var-passthru-updateScript-execution}
[]{#var-passthru-updateScript-supported-features}
[]{#var-passthru-updateScript-commit}
[]{#var-passthru-updateScript-commit-attrPath}
[]{#var-passthru-updateScript-commit-oldVersion}
[]{#var-passthru-updateScript-commit-newVersion}
[]{#var-passthru-updateScript-commit-files}
[]{#var-passthru-updateScript-commit-commitBody}
[]{#var-passthru-updateScript-commit-commitMessage}
[]{#var-passthru-updateScript-example-commit}
Nixpkgs tries to automatically update all packages that have an `passthru.updateScript` attribute.
See the [section on automatic package updates in the package contributor guide](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#automatic-package-updates) for details.

View File

@@ -442,6 +442,145 @@ If set to `true`, `stdenv` will pass specific flags to `make` and other build to
Unless set to `false`, some build systems with good support for parallel building including `cmake`, `meson`, and `qmake` will set it to `true`.
### Special variables {#special-variables}
#### `passthru` {#var-stdenv-passthru}
This is an attribute set which can be filled with arbitrary values. For example:
```nix
{
passthru = {
foo = "bar";
baz = {
value1 = 4;
value2 = 5;
};
};
}
```
Values inside it are not passed to the builder, so you can change them without triggering a rebuild. However, they can be accessed outside of a derivation directly, as if they were set inside a derivation itself, e.g. `hello.baz.value1`. We dont specify any usage or schema of `passthru` - it is meant for values that would be useful outside the derivation in other parts of a Nix expression (e.g. in other derivations). An example would be to convey some specific dependency of your derivation which contains a program with plugins support. Later, others who make derivations with plugins can use passed-through dependency to ensure that their plugin would be binary-compatible with built program.
#### `passthru.updateScript` {#var-passthru-updateScript}
A script to be run by `maintainers/scripts/update.nix` when the package is matched. The attribute can contain one of the following:
- []{#var-passthru-updateScript-command} an executable file, either on the file system:
```nix
{
passthru.updateScript = ./update.sh;
}
```
or inside the expression itself:
```nix
{
passthru.updateScript = writeScript "update-zoom-us" ''
#!/usr/bin/env nix-shell
#!nix-shell -i bash -p curl pcre2 common-updater-scripts
set -eu -o pipefail
version="$(curl -sI https://zoom.us/client/latest/zoom_x86_64.tar.xz | grep -Fi 'Location:' | pcre2grep -o1 '/(([0-9]\.?)+)/')"
update-source-version zoom-us "$version"
'';
}
```
- a list, a script followed by arguments to be passed to it:
```nix
{
passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ];
}
```
- an attribute set containing:
- [`command`]{#var-passthru-updateScript-set-command} a string or list in the [format expected by `passthru.updateScript`](#var-passthru-updateScript-command).
- [`attrPath`]{#var-passthru-updateScript-set-attrPath} (optional) a string containing the canonical attribute path for the package. If present, it will be passed to the update script instead of the attribute path on which the package was discovered during Nixpkgs traversal.
- [`supportedFeatures`]{#var-passthru-updateScript-set-supportedFeatures} (optional) a list of the [extra features](#var-passthru-updateScript-supported-features) the script supports.
```nix
{
passthru.updateScript = {
command = [ ../../update.sh pname ];
attrPath = pname;
supportedFeatures = [ /* ... */ ];
};
}
```
::: {.tip}
A common pattern is to use the [`nix-update-script`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/common-updater/nix-update.nix) attribute provided in Nixpkgs, which runs [`nix-update`](https://github.com/Mic92/nix-update):
```nix
{
passthru.updateScript = nix-update-script { };
}
```
For simple packages, this is often enough, and will ensure that the package is updated automatically by [`nixpkgs-update`](https://ryantm.github.io/nixpkgs-update) when a new version is released. The [update bot](https://nix-community.org/update-bot) runs periodically to attempt to automatically update packages, and will run `passthru.updateScript` if set. While not strictly necessary if the project is listed on [Repology](https://repology.org), using `nix-update-script` allows the package to update via many more sources (e.g. GitHub releases).
:::
##### How update scripts are executed? {#var-passthru-updateScript-execution}
Update scripts are to be invoked by `maintainers/scripts/update.nix` script. You can run `nix-shell maintainers/scripts/update.nix` in the root of Nixpkgs repository for information on how to use it. `update.nix` offers several modes for selecting packages to update (e.g. select by attribute path, traverse Nixpkgs and filter by maintainer, etc.), and it will execute update scripts for all matched packages that have an `updateScript` attribute.
Each update script will be passed the following environment variables:
- [`UPDATE_NIX_NAME`]{#var-passthru-updateScript-env-UPDATE_NIX_NAME} content of the `name` attribute of the updated package.
- [`UPDATE_NIX_PNAME`]{#var-passthru-updateScript-env-UPDATE_NIX_PNAME} content of the `pname` attribute of the updated package.
- [`UPDATE_NIX_OLD_VERSION`]{#var-passthru-updateScript-env-UPDATE_NIX_OLD_VERSION} content of the `version` attribute of the updated package.
- [`UPDATE_NIX_ATTR_PATH`]{#var-passthru-updateScript-env-UPDATE_NIX_ATTR_PATH} attribute path the `update.nix` discovered the package on (or the [canonical `attrPath`](#var-passthru-updateScript-set-attrPath) when available). Example: `pantheon.elementary-terminal`
::: {.note}
An update script will be usually run from the root of the Nixpkgs repository but you should not rely on that. Also note that `update.nix` executes update scripts in parallel by default so you should avoid running `git commit` or any other commands that cannot handle that.
:::
::: {.tip}
While update scripts should not create commits themselves, `maintainers/scripts/update.nix` supports automatically creating commits when running it with `--argstr commit true`. If you need to customize commit message, you can have the update script implement [`commit`](#var-passthru-updateScript-commit) feature.
:::
##### Supported features {#var-passthru-updateScript-supported-features}
###### `commit` {#var-passthru-updateScript-commit}
This feature allows update scripts to *ask* `update.nix` to create Git commits.
When support of this feature is declared, whenever the update script exits with `0` return status, it is expected to print a JSON list containing an object described below for each updated attribute to standard output.
When `update.nix` is run with `--argstr commit true` arguments, it will create a separate commit for each of the objects. An empty list can be returned when the script did not update any files, for example, when the package is already at the latest version.
The commit object contains the following values:
- [`attrPath`]{#var-passthru-updateScript-commit-attrPath} a string containing attribute path.
- [`oldVersion`]{#var-passthru-updateScript-commit-oldVersion} a string containing old version.
- [`newVersion`]{#var-passthru-updateScript-commit-newVersion} a string containing new version.
- [`files`]{#var-passthru-updateScript-commit-files} a non-empty list of file paths (as strings) to add to the commit.
- [`commitBody`]{#var-passthru-updateScript-commit-commitBody} (optional) a string with extra content to be appended to the default commit message (useful for adding changelog links).
- [`commitMessage`]{#var-passthru-updateScript-commit-commitMessage} (optional) a string to use instead of the default commit message.
If the returned array contains exactly one object (e.g. `[{}]`), all values are optional and will be determined automatically.
::: {.example #var-passthru-updateScript-example-commit}
# Standard output of an update script using commit feature
```json
[
{
"attrPath": "volume_key",
"oldVersion": "0.3.11",
"newVersion": "0.3.12",
"files": [
"/path/to/nixpkgs/pkgs/development/libraries/volume-key/default.nix"
]
}
]
```
:::
### Fixed-point arguments of `mkDerivation` {#mkderivation-recursive-attributes}
If you pass a function to `mkDerivation`, it will receive as its argument the final arguments, including the overrides when reinvoked via `overrideAttrs`. For example:
@@ -494,7 +633,7 @@ in pkg
Unlike the `pkg` binding in the above example, the `finalAttrs` parameter always references the final attributes. For instance `(pkg.overrideAttrs(x)).finalAttrs.finalPackage` is identical to `pkg.overrideAttrs(x)`, whereas `(pkg.overrideAttrs(x)).original` is the same as the original `pkg`.
See also the section about [`passthru.tests`](#var-passthru-tests).
See also the section about [`passthru.tests`](#var-meta-tests).
## Phases {#sec-stdenv-phases}
@@ -580,7 +719,7 @@ After unpacking all of `src` and `srcs`, if neither of `sourceRoot` and `setSour
If `unpackPhase` produces multiple source directories, you should set `sourceRoot` to the name of the intended directory.
You can also set `sourceRoot = ".";` if you want to control it yourself in a later phase.
For example, if you want your build to start in a sub-directory inside your sources, and you are using `fetchzip`-derived `src` (like `fetchFromGitHub` or similar), you need to set `sourceRoot = "${src.name}/my-sub-directory"`.
For example, if your want your build to start in a sub-directory inside your sources, and you are using `fetchzip`-derived `src` (like `fetchFromGitHub` or similar), you need to set `sourceRoot = "${src.name}/my-sub-directory"`.
##### `setSourceRoot` {#var-stdenv-setSourceRoot}
@@ -762,8 +901,6 @@ Before and after running `make`, the hooks `preBuild` and `postBuild` are called
The check phase checks whether the package was built correctly by running its test suite. The default `checkPhase` calls `make $checkTarget`, but only if the [`doCheck` variable](#var-stdenv-doCheck) is enabled.
It is highly recommended, for packages' sources that are not distributed with any tests, to at least use [`versionCheckHook`](#versioncheckhook) to test that the resulting executable is basically functional.
#### Variables controlling the check phase {#variables-controlling-the-check-phase}
##### `doCheck` {#var-stdenv-doCheck}
@@ -1008,7 +1145,7 @@ This setup works as follows:
The installCheck phase checks whether the package was installed correctly by running its test suite against the installed directories. The default `installCheck` calls `make installcheck`.
It is often better to add tests that are not part of the source distribution to `passthru.tests` (see
[](#var-passthru-tests)). This avoids adding overhead to every build and enables us to run them independently.
[](#var-meta-tests)). This avoids adding overhead to every build and enables us to run them independently.
#### Variables controlling the installCheck phase {#variables-controlling-the-installcheck-phase}
@@ -1136,12 +1273,6 @@ Example removing all references to the compiler in the output:
}
```
### `runHook` \<hook\> {#fun-runHook}
Execute \<hook\> and the values in the array associated with it. The array's name is determined by removing `Hook` from the end of \<hook\> and appending `Hooks`.
For example, `runHook postHook` would run the hook `postHook` and all of the values contained in the `postHooks` array, if it exists.
### `substitute` \<infile\> \<outfile\> \<subs\> {#fun-substitute}
Performs string substitution on the contents of \<infile\>, writing the result to \<outfile\>. The substitutions in \<subs\> are of the following form:
@@ -1427,8 +1558,6 @@ Both parameters take a list of flags as strings. The special `"all"` flag can be
For more in-depth information on these hardening flags and hardening in general, refer to the [Debian Wiki](https://wiki.debian.org/Hardening), [Ubuntu Wiki](https://wiki.ubuntu.com/Security/Features), [Gentoo Wiki](https://wiki.gentoo.org/wiki/Project:Hardened), and the [Arch Wiki](https://wiki.archlinux.org/title/Security).
Note that support for some hardening flags varies by compiler, CPU architecture, target OS and libc. Combinations of these that don't support a particular hardening flag will silently ignore attempts to enable it. To see exactly which hardening flags are being employed in any invocation, the `NIX_DEBUG` environment variable can be used.
### Hardening flags enabled by default {#sec-hardening-flags-enabled-by-default}
The following flags are enabled by default and might require disabling with `hardeningDisable` if the program to package is incompatible.
@@ -1478,16 +1607,6 @@ installwatch.c:3751:5: error: conflicting types for '__open_2'
fcntl2.h:50:4: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments
```
Disabling `fortify` implies disablement of `fortify3`
#### `fortify3` {#fortify3}
Adds the `-O2 -D_FORTIFY_SOURCE=3` compiler options. This expands the cases that can be protected by fortify-checks to include some situations with dynamic-length buffers whose length can be inferred at runtime using compiler hints.
Enabling this flag implies enablement of `fortify`. Disabling this flag does not imply disablement of `fortify`.
This flag can sometimes conflict with a build-system's own attempts at enabling fortify support and result in errors complaining about `redefinition of _FORTIFY_SOURCE`.
#### `pic` {#pic}
Adds the `-fPIC` compiler options. This options adds support for position independent code in shared libraries and thus making ASLR possible.
@@ -1523,61 +1642,19 @@ This flag can break dynamic shared object loading. For instance, the module syst
intel_drv.so: undefined symbol: vgaHWFreeHWRec
```
#### `zerocallusedregs` {#zerocallusedregs}
Adds the `-fzero-call-used-regs=used-gpr` compiler option. This causes the general-purpose registers that an architecture's calling convention considers "call-used" to be zeroed on return from the function. This can make it harder for attackers to construct useful ROP gadgets and also reduces the chance of data leakage from a function call.
### Hardening flags disabled by default {#sec-hardening-flags-disabled-by-default}
The following flags are disabled by default and should be enabled with `hardeningEnable` for packages that take untrusted input like network services.
#### `pie` {#pie}
This flag is disabled by default for normal `glibc` based NixOS package builds, but enabled by default for
- `musl`-based package builds, except on Aarch64 and Aarch32, where there are issues.
- Statically-linked for OpenBSD builds, where it appears to be required to get a working binary.
This flag is disabled by default for normal `glibc` based NixOS package builds, but enabled by default for `musl` based package builds.
Adds the `-fPIE` compiler and `-pie` linker options. Position Independent Executables are needed to take advantage of Address Space Layout Randomization, supported by modern kernel versions. While ASLR can already be enforced for data areas in the stack and heap (brk and mmap), the code areas must be compiled as position-independent. Shared libraries already do this with the `pic` flag, so they gain ASLR automatically, but binary .text regions need to be build with `pie` to gain ASLR. When this happens, ROP attacks are much harder since there are no static locations to bounce off of during a memory corruption attack.
Static libraries need to be compiled with `-fPIE` so that executables can link them in with the `-pie` linker option.
If the libraries lack `-fPIE`, you will get the error `recompile with -fPIE`.
#### `shadowstack` {#shadowstack}
Adds the `-fcf-protection=return` compiler option. This enables the Shadow Stack feature supported by some newer processors, which maintains a user-inaccessible copy of the program's stack containing only return-addresses. When returning from a function, the processor compares the return-address value on the two stacks and throws an error if they do not match, considering it a sign of corruption and possible tampering. This should significantly increase the difficulty of ROP attacks.
For the Shadow Stack to be enabled at runtime, all code linked into a process must be built with Shadow Stack enabled, so this is probably only useful to enable on a wide scale, so that all of a packages dependencies also have the feature enabled.
This is currently only supported on some newer Intel and AMD processors as part of the Intel CET set of features. However, the generated code should continue to work on older processors which will simply omit any of this checking.
This breaks some code that does advanced stack management or exception handling. If enabling this hardening flag it is important to test the result on a system that has known working and enabled CET support, so that any such breakage can be discovered.
#### `trivialautovarinit` {#trivialautovarinit}
Adds the `-ftrivial-auto-var-init=pattern` compiler option. This causes "trivially-initializable" uninitialized stack variables to be forcibly initialized with a nonzero value that is likely to cause a crash (and therefore be noticed). Uninitialized variables generally take on their values based on fragments of previous program state, and attackers can carefully manipulate that state to craft malicious initial values for these variables.
Use of this flag is controversial as it can prevent tools that detect uninitialized variable use (such as valgrind) from operating correctly.
This should be turned off or fixed for build errors such as:
```
sorry, unimplemented: __builtin_clear_padding not supported for variable length aggregates
```
#### `stackclashprotection` {#stackclashprotection}
This flag adds the `-fstack-clash-protection` compiler option, which causes growth of a program's stack to access each successive page in order. This should force the guard page to be accessed and cause an attempt to "jump over" this guard page to crash.
#### `pacret` {#pacret}
This flag adds the `-mbranch-protection=pac-ret` compiler option on aarch64-linux targets. This uses ARM v8.3's Pointer Authentication feature to sign function return pointers before adding them to the stack. The pointer's authenticity is then validated before returning to its destination. This dramatically increases the difficulty of ROP exploitation techniques.
This may cause problems with code that does advanced stack manipulation, and debugging/stack-unwinding tools need to be pac-ret aware to work correctly when these features are in operation.
Pre-ARM v8.3 processors will ignore Pointer Authentication instructions, so code built with this flag will continue to work on older processors, though without any of the intended protections. If enabling this flag, it is recommended to ensure the resultant packages are tested against an ARM v8.3+ linux system with known-working Pointer Authentication support so that any breakage caused by this feature is actually detected.
[^footnote-stdenv-ignored-build-platform]: The build platform is ignored because it is a mere implementation detail of the package satisfying the dependency: As a general programming principle, dependencies are always *specified* as interfaces, not concrete implementation.
[^footnote-stdenv-native-dependencies-in-path]: Currently, this means for native builds all dependencies are put on the `PATH`. But in the future that may not be the case for sake of matching cross: the platforms would be assumed to be unique for native and cross builds alike, so only the `depsBuild*` and `nativeBuildInputs` would be added to the `PATH`.
[^footnote-stdenv-propagated-dependencies]: Nix itself already takes a packages transitive dependencies into account, but this propagation ensures nixpkgs-specific infrastructure like [setup hooks](#ssec-setup-hooks) also are run as if it were a propagated dependency.

Some files were not shown because too many files have changed in this diff Show More