Compare commits

..

21 Commits

Author SHA1 Message Date
Vladimír Čunát
77636a6a16 stdenv-bootstrap-tools: Fix lib*_asneeded problem on gcc16 (#545658) 2026-07-27 12:43:03 +02:00
whispers
bee3b6c52f texlive-bin-big: autoreconf to fix build with gcc 16
(texlive-bin-big is not a real attribute, but it does not appear to be
actually exposed as an attribute. we're building this as a dependency of
`texlivePackages.texlive-scripts`.)

autoconf 2.72 has a bug where AC_PROG_CXX would reject c++20 compilers,
and attempts to switch to c++98 or c++11 instead of using the compiler's
default. when using gcc 16, which defaults to c++20, this causes
texlive-bin-big to be built with c++20, causing issues with some of the
included icu4c headers. this bug was fixed in autoconf 2.73, so we
regenerate the autoconf declarations with the newest version of autconf
(currently 2.73). once the configure scripts distributed with the source
are regenerated with autoconf 2.73+, the `reautoconf` call should be
removed.
2026-07-27 08:54:46 +02:00
whispers
cedf46f815 apache-orc: explicitly specify C++17 for abseil-cpp
abseil exposes different interfaces depending on what is available in
`std` in a given C++ standard. gcc 16 defaults to C++20, thus causing
the default abseil-cpp to expose a different interface than the one
apache-orc (built with C++17) expects. this leads to a
great deal of "error: 'partial_ordering' has not been declared in 'std'"
and similar originating from abseil's types/compare.h. thus, we
explicitly override abseil to specify the desired C++ standard. this
happens through protobuf as abseil-cpp is in its propagatedBuildInputs.
2026-07-27 08:54:33 +02:00
dramforever
cd01953447 stdenv-bootstrap-tools: Fix lib*_asneeded problem on gcc16
GCC 16 added and starts using -latomic_asneeded and -lgcc_s_asneeded to
pull in the corresponding libraries as if --as-needed. Add these linker
scripts.

After fixing this, gcc in turn wants libatomic, so reuse the existing
line copying it for riscv, and use it for all platforms.

See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=123650 for why this was
added.
2026-07-26 00:09:13 +08:00
whispers
a06d109d48 glog: disable optimization-dependent stacktrace test
This test fails under GCC 16. It appears to be because it makes
assumptions that certain compiler optimizations do not occur. It clearly
is compiler-specific and tries to work around specific compiler
optimizations upstream. In particular, an analysis of the test
53d58e4531/src/stacktrace_unittest.cc (L218)
shows that it would be very plausible/correct for GCC to place labels
such that the `INIT_ADDRESS_RANGE` macro has `start` and `end` labels
that have the same address. While the functions are labeled NOINLINE,
interprocedural analysis could *plausibly* produce this result. While
we are not completely certain that this is the case, it seems likely
that this is not a genuine issue and just a questionable test.
2026-07-24 12:59:10 -04:00
whispers
c465430f82 assimp: never treat warnings as fatal
assimp sometimes has warnings that are triggered by new compiler or
library versions. since assimp builds with -Werror by default, this
requires workaround in Nixpkgs until they cut a release. this is the
case with unused variable warnings in GCC 16. instead of dealing with
this, we simply disable -Werror via the cmake flag it offers to do so.
2026-07-24 12:59:10 -04:00
whispers
1e10b67e35 protobufc: unpin standard version to fix build with gcc 16
protobufc pins a specific version of the C++ standard and does so using
an ancient vendored macro from the autoconf archive. this causes a
failure to build on gcc 16, as it defaults to C++20 and protobufc uses
C++17. this particularly causes problems with abseil, which has headers
which depend on the C++ standard to compile. accordingly, to avoid
having to manually specify and update a version each time the default
standard version updates, we unpin it completely and allow the compiler
to choose what it uses by default.

alternatively, we could override the abseil that ends up in protobufc
by way of protobuf_33 to use the C++17 standard instead. this would
work, but this seems more fragile and subject to compiler version churn.
it is also our understanding that mixing and matching versions of
standards in dependents can be messy, and using the default seems the
least likely to cause problems.
2026-07-24 10:52:32 -04:00
whispers
6aa36277ff webrtc-audio-processing: explicitly specify C++17 for abseil-cpp
abseil exposes different interfaces depending on what is available in
`std` in a given C++ standard. gcc 16 defaults to C++20, thus causing
the default abseil-cpp to expose a different interface than the one
webrtc-audio-processing (built with C++17) expects. this leads to a
great deal of "error: 'partial_ordering' has not been declared in 'std'"
and similar originating from abseil's types/compare.h. thus, we
explicitly override abseil to specify the desired C++ standard.
2026-07-24 10:52:28 -04:00
whispers
d457a15860 grub2_efi: apply patch for gcc 16
gcc 16 gains stricter analysis of whether attributes are ignored,
leading to build failures when built with -Werror. we could silence the
error, but this was fixed in a trivial commit upstream that is obviously
correct, so we pull it in:
https://cgit.git.savannah.gnu.org/cgit/grub.git/commit/?id=9922ed133c2c754ec9f37198da2b3e3e8a4fd5ff
2026-07-24 10:52:15 -04:00
whispers
75c304b0e0 toml11: make maybe-uninitialized non-fatal for gcc 16
GCC 16 flags various calls here as if they're uninitialized, but they
appear (to me as a non-C++ expert) to be false positives. Accordingly,
we make those non-fatal. This fixes the build on GCC 16.
2026-07-24 10:52:05 -04:00
whispers
2012d325a3 onetbb: never treat warnings as fatal
onetbb often triggers compiler warnings upstream, and works around them
in an ad-hoc, per version manner:
88482f5f1a
bdbec20606
we encountered this failing while preparing for a gcc 16 upgrade in
Nixpkgs, and the same issue came up for gcc 15 and similar (#446139). it
is likely to come up again, as -Werror is extremely susceptible to
compiler and library changes (though onetbb admittedly has few
dependencies). additionally, while it may be useful for upstream onetbb,
it seems to provide little value to us downstream; we just end up
working around it. thus, we never treat warnings as errors by disabling
upstream's cmake flag for this purpose.
2026-07-24 10:52:00 -04:00
whispers
1dd0a5d9e5 libfaketime: add patch for gcc 16
gcc 16's unused variable analysis is more advanced than previous
versions, and detects that some variables used in tests are unused.
upstream patched this away, so we fetch their patch. alternatively, we
could pass `-Wno-error=unused-but-set-variable` if that is preferred.
2026-07-24 10:51:38 -04:00
whispers
fb7e01a3b6 sbsigntool: make unused-but-set-variable non-fatal for gcc 16
since sbsigntool builds with -Werror by default, and gcc 16's unused
variable analysis is better than previous versions, this causes a build
failure.
2026-07-24 10:50:54 -04:00
whispers
80c9a1708e libsystemtap: 5.3 -> 5.5
Log: https://sourceware.org/git/?p=systemtap.git;a=shortlog;h=refs/tags/release-5.5
2026-07-24 10:48:57 -04:00
whispers
b57ec0fcb2 systemtap-unwrapped: 5.4 -> 5.5
Log: https://sourceware.org/git/?p=systemtap.git;a=shortlog;h=refs/tags/release-5.5
2026-07-24 10:48:57 -04:00
whispers
f7b16131d9 usrsctp: make unused-but-set-variable non-fatal for gcc 16
since usrsctp builds with -Werror by default, and gcc 16's unused
variable analysis is better than previous versions, this causes a build
failure. a fix for this particular variable has been submitted upstream,
but this is sufficient in the interim.
2026-07-24 13:31:31 +02:00
whispers
6c37551652 jemalloc: add patch to fix build under gcc 16
jemalloc used the nonstandard `std::__throw_bad_alloc`, which is no
longer visible in GCC 16. this upstream patch makes it conditional on
exceptions and defers to either `throw std::bad_alloc()` or
`std::terminate` as appropriate, and fixes the build.
2026-07-24 13:31:18 +02:00
whispers
aab37bd258 sourceHighlight: patch to fix build with gcc 16
The `ranges` name in the test here conflicts with the
`namespace std::ranges { }` from GCC 16, causing this test to fail to
build with "error: reference to 'ranges' is ambiguous". To avoid this,
we simply rename the variable.
2026-07-24 13:31:06 +02:00
whispers
017fed1d63 gcc: 15 -> 16
changes: https://gcc.gnu.org/gcc-16/changes.html
porting guide: https://gcc.gnu.org/gcc-16/porting_to.html
2026-07-24 13:23:18 +02:00
whispers
f46e80f6c5 minimal-bootstrap.gcc-glibc: 15.3.0 -> 16.1.0
https://gcc.gnu.org/gcc-16/changes.html
2026-07-24 13:23:18 +02:00
whispers
8c8bcc8565 minimal-bootstrap.gcc-latest: 15.3.0 -> 16.1.0
https://gcc.gnu.org/gcc-16/changes.html
2026-07-24 13:23:14 +02:00
14843 changed files with 296294 additions and 406672 deletions

View File

@@ -23,15 +23,15 @@ insert_final_newline = false
# see https://nixos.org/nixpkgs/manual/#chap-conventions
[*.{bash,css,js,json,lock,md,nix,pl,pm,py,rb,sh,ts,xml}]
[*.{bash,css,js,json,lock,md,nix,pl,pm,py,rb,sh,xml}]
indent_style = space
# Match docbook files, set indent width of one
[*.xml]
indent_size = 1
# Match js/json/lockfiles/markdown/nix/ruby/ts files, set indent width of two
[*.{js,json,lock,md,nix,rb,ts}]
# Match json/lockfiles/markdown/nix/ruby files, set indent width of two
[*.{js,json,lock,md,nix,rb}]
indent_size = 2
# Match all the Bash code in Nix files, set indent width of two
@@ -79,7 +79,7 @@ indent_size = unset
trim_trailing_whitespace = true
# binaries
[*.{nib,torrent}]
[*.nib]
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset

3
.gitattributes vendored
View File

@@ -62,6 +62,3 @@ ci/OWNERS linguist-language=CODEOWNERS
# patching CRLF line endings from an upstream source package.
*.diff !text !eol
*.patch !text !eol
# Torrent files are binary files and should not be re-encoded.
*.torrent !text !eol

View File

@@ -13,14 +13,7 @@ inputs:
runs:
using: composite
steps:
# We don't actually need anything in this directory, but we need a small
# sparse checkout, and this directory is small.
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
- uses: actions/github-script@ed597411d8f924073f98dfc5c65a23a2325f34cd # v8.0.0
env:
MERGED_SHA: ${{ inputs.merged-as-untrusted-at }}
TARGET_SHA: ${{ inputs.target-as-trusted-at }}
@@ -44,7 +37,7 @@ runs:
})
}
// These are set automatically by the sparse checkout for .github/actions.
// These are set automatically by the spare checkout for .github/actions.
// Undo them, otherwise git fetch below will not do anything.
await run('git', 'config', 'unset', 'remote.origin.promisor')
await run('git', 'config', 'unset', 'remote.origin.partialclonefilter')

View File

@@ -1,9 +1,7 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directories:
- "/"
- ".github/actions/*/*"
directory: "/"
schedule:
interval: "weekly"
labels: []

20
.github/labeler.yml vendored
View File

@@ -43,6 +43,14 @@
- .github/**/*
- ci/**/*.*
"6.topic: coq":
- any:
- changed-files:
- any-glob-to-any-file:
- pkgs/applications/science/logic/coq/**/*
- pkgs/development/coq-modules/**/*
- pkgs/top-level/coq-packages.nix
"6.topic: COSMIC":
- any:
- changed-files:
@@ -458,18 +466,6 @@
- any-glob-to-any-file:
- pkgs/development/rocm-modules/**/*
"6.topic: rocq":
- any:
- changed-files:
- any-glob-to-any-file:
- pkgs/applications/science/logic/coq/**/*
- pkgs/applications/science/logic/rocq-core/**/*
- pkgs/build-support/coq/**/*
- pkgs/build-support/rocq/**/*
- pkgs/development/rocq-modules/**/*
- pkgs/top-level/coq-packages.nix
- pkgs/top-level/rocq-packages.nix
"6.topic: ruby":
- any:
- changed-files:

View File

@@ -39,8 +39,6 @@ jobs:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
ref: ${{ github.event.pull_request.head.sha }}
# Avoid materializing full nixpkgs tree
sparse-checkout: .
token: ${{ steps.app-token.outputs.token }}
persist-credentials: true
@@ -51,7 +49,7 @@ jobs:
- name: Create backport PRs
id: backport
uses: korthout/backport-action@6b65649031ac6d18ffdfd0c0820e9436f3fde22b # v4.6.1
uses: korthout/backport-action@2e830a1d0b8269505846ddd407a70876913ad1f8 # v4.6.0
with:
# Config README: https://github.com/korthout/backport-action#backport-action
add_author_as_reviewer: true

View File

@@ -49,7 +49,7 @@ jobs:
ci/github-script
- name: Install dependencies
run: npm ci --package-lock-only=false --no-audit @actions/artifact bottleneck
run: npm ci --package-lock-only=false @actions/artifact bottleneck
working-directory: ci/github-script
# Use a GitHub App, because it has much higher rate limits: 12,500 instead of 5,000 req / hour.
@@ -76,8 +76,7 @@ jobs:
github-token: ${{ steps.app-token.outputs.token || github.token }}
retries: 3
script: |
const { default: bot } = await import('${{ github.workspace }}/ci/github-script/bot.js')
await bot({
require('./ci/github-script/bot.js')({
github,
context,
core,
@@ -89,7 +88,7 @@ jobs:
GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }}
run: gh api /rate_limit | jq
- uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13 # v7.0.0
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0
name: Labels from touched files
if: |
github.event_name == 'pull_request_target' &&
@@ -99,7 +98,7 @@ jobs:
configuration-path: .github/labeler.yml # default
sync-labels: true
- uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13 # v7.0.0
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0
name: Labels from touched files (no sync)
if: |
github.event_name == 'pull_request_target' &&
@@ -109,7 +108,7 @@ jobs:
configuration-path: .github/labeler-no-sync.yml
sync-labels: false
- uses: actions/labeler@bf12e9b00b37c5c0ca2b87b79b2daf7891dbda13 # v7.0.0
- uses: actions/labeler@b8dd2d9be0f68b860e7dae5dae7d772984eacd6d # v6.2.0
name: Labels from touched files (development branches)
# Development branches like staging-next, haskell-updates and python-updates get special labels.
# This is to avoid the mass of labels there, which is mostly useless - and really annoying for

View File

@@ -49,13 +49,17 @@ jobs:
runs-on: ${{ matrix.runner }}
timeout-minutes: 60
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout the merge commit
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
target-as-trusted-at: ${{ inputs.targetSha }}
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
with:
# Sandbox is disabled on MacOS by default.
extra_nix_config: sandbox = true

View File

@@ -24,10 +24,6 @@ on:
# not evaluate untrusted code.
NIXPKGS_MANUAL_EDIT_CHECK_APP_PRIVATE_KEY:
required: false
# Can be provided in pull requests because the job it is used in does
# not evaluate untrusted code.
NIXPKGS_CI_APP_PRIVATE_KEY:
required: false
# Should only be provided in the merge queue, not in pull requests,
# where we're evaluating untrusted code.
CACHIX_AUTH_TOKEN_GHA:
@@ -55,7 +51,7 @@ jobs:
ci/github-script
- name: Install dependencies
run: npm ci --package-lock-only=false --no-audit bottleneck
run: npm ci --package-lock-only=false bottleneck
working-directory: trusted/ci/github-script
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
@@ -80,8 +76,7 @@ jobs:
github-token: ${{ steps.app-token.outputs.token || github.token }}
script: |
const targetsStable = JSON.parse(process.env.TARGETS_STABLE)
const { default: commits } = await import('${{ github.workspace }}/trusted/ci/github-script/commits.ts')
await commits({
require('./trusted/ci/github-script/commits.js')({
github,
context,
core,
@@ -126,8 +121,7 @@ jobs:
with:
github-token: ${{ steps.app-token.outputs.token || github.token }}
script: |
const { default: checkManualFileEdits } = await import('${{ github.workspace }}/trusted/ci/github-script/manual-file-edits.ts')
await checkManualFileEdits({
require('./trusted/ci/github-script/manual-file-edits.js')({
github,
context,
core,
@@ -140,87 +134,21 @@ jobs:
GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }}
run: gh api /rate_limit | jq
reminders:
if: inputs.baseBranch && inputs.headBranch
permissions:
pull-requests: write
runs-on: ubuntu-24.04-arm
timeout-minutes: 8
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
path: trusted
sparse-checkout: |
ci/github-script
- uses: actions/create-github-app-token@bcd2ba49218906704ab6c1aa796996da409d3eb1 # v3.2.0
if: github.event_name != 'pull_request' && vars.NIXPKGS_CI_CLIENT_ID
id: app-token
with:
client-id: ${{ vars.NIXPKGS_CI_CLIENT_ID }}
private-key: ${{ secrets.NIXPKGS_CI_APP_PRIVATE_KEY }}
permission-pull-requests: write
- name: Log current API rate limits
env:
GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }}
run: gh api /rate_limit | jq
- name: Post reminders
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
github-token: ${{ steps.app-token.outputs.token || github.token }}
script: |
const { default: postReminders } = await import('${{ github.workspace }}/trusted/ci/github-script/reminders.ts')
await postReminders({
github,
context,
core,
dry: context.eventName == 'pull_request',
repoPath: 'trusted',
})
- name: Log current API rate limits
env:
GH_TOKEN: ${{ steps.app-token.outputs.token || github.token }}
run: gh api /rate_limit | jq
github-script:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
steps:
- name: Checkout merge and target commits
uses: $/.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
target-as-trusted-at: ${{ inputs.targetSha }}
- name: Install dependencies to trusted/
run: |
npm ci --package-lock-only=false --no-audit
echo "$PWD/node_modules/.bin" >> "$GITHUB_PATH"
working-directory: nixpkgs/trusted/ci/github-script
- name: Link trusted/ dependencies to untrusted/
run: ln -s "$PWD/trusted/ci/github-script/node_modules" untrusted/ci/github-script/node_modules
working-directory: nixpkgs
- name: Check ci/github-script
run: npm test
working-directory: nixpkgs/untrusted/ci/github-script
owners:
runs-on: ubuntu-24.04-arm
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout merge and target commits
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
target-as-trusted-at: ${{ inputs.targetSha }}
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
continue-on-error: true

View File

@@ -43,7 +43,7 @@ jobs:
github-token: ${{ steps.app-token.outputs.token || github.token }}
retries: 3
script: |
const { handleMergeComment } = await import('${{ github.workspace }}/ci/github-script/merge.js')
const { handleMergeComment } = require('./ci/github-script/merge.js')
const { body, node_id } = context.payload.comment
await handleMergeComment({

View File

@@ -139,7 +139,7 @@ jobs:
core.info(`Found pinned.json commit: ${ciPinBumpCommit}`)
- name: Install Nix
uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- name: Load supported versions
id: versions
@@ -174,8 +174,12 @@ jobs:
sudo mkswap /swap
sudo swapon /swap
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Check out the PR at merged and target commits
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
# For versioned evals, use the target as the untrusted base and apply the pin-bump commit
merged-as-untrusted-at: ${{ matrix.version && inputs.targetSha || inputs.mergedSha }}
@@ -183,7 +187,7 @@ jobs:
target-as-trusted-at: ${{ inputs.targetSha }}
- name: Install Nix
uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
continue-on-error: true
@@ -255,8 +259,12 @@ jobs:
statuses: write # creating 'Eval Summary' commit statuses
timeout-minutes: 5
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Check out the PR at the target commit
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
target-as-trusted-at: ${{ inputs.targetSha }}
@@ -269,7 +277,7 @@ jobs:
merge-multiple: true
- name: Install Nix
uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- name: Combine all output paths and eval stats
run: |
@@ -367,8 +375,7 @@ jobs:
with:
github-token: ${{ steps.app-token.outputs.token || github.token }}
script: |
const { checkTargetBranch } = await import('${{ github.workspace }}/nixpkgs/trusted/ci/github-script/check-target-branch.ts')
await checkTargetBranch({
require('./nixpkgs/trusted/ci/github-script/check-target-branch.js')({
github,
context,
core,
@@ -469,13 +476,17 @@ jobs:
runs-on: ubuntu-24.04-arm
timeout-minutes: 10
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout the merge commit
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
- name: Install Nix
uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- name: Ensure flake outputs on all systems still evaluate
run: nix flake check --all-systems --no-build './nixpkgs/untrusted?shallow=1'

View File

@@ -26,12 +26,16 @@ jobs:
runs-on: ubuntu-24.04-arm
timeout-minutes: 10
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout the merge commit
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
# TODO: Figure out how to best enable caching for the treefmt job. Cachix won't work well,
# because the cache would be invalidated on every commit - treefmt checks every file.
@@ -57,12 +61,16 @@ jobs:
runs-on: ubuntu-24.04-arm
timeout-minutes: 10
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout the merge commit
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
continue-on-error: true
@@ -82,13 +90,17 @@ jobs:
runs-on: ubuntu-24.04-arm
timeout-minutes: 10
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
sparse-checkout: .github/actions
- name: Checkout merge and target commits
uses: $/.github/actions/checkout
uses: ./.github/actions/checkout
with:
merged-as-untrusted-at: ${{ inputs.mergedSha }}
target-as-trusted-at: ${{ inputs.targetSha }}
- uses: cachix/install-nix-action@13d8dd58da0234aa297dedd986986ccb8e7f3e24 # v31.11.1
- uses: cachix/install-nix-action@630ae543ea3a38a9a4166f03376c02c50f408342 # v31.11.0
- uses: cachix/cachix-action@5f2d7c5294214f71b873db4b969586b980625e71 # v17
continue-on-error: true
@@ -130,9 +142,9 @@ jobs:
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
with:
script: |
const { default: checkCommitMessages } = await import('${{ github.workspace }}/trusted/ci/github-script/lint-commits.ts')
const checkCommitMessages = require('./trusted/ci/github-script/lint-commits.js')
await checkCommitMessages({
checkCommitMessages({
github,
context,
core,

View File

@@ -29,7 +29,7 @@ jobs:
with:
persist-credentials: false
sparse-checkout: |
ci/github-script
ci/github-script/supportedSystems.js
- id: prepare
uses: actions/github-script@3a2844b7e9c422d3c10d287c895573f7108da1b3 # v9.0.0
@@ -38,8 +38,8 @@ jobs:
TARGET_SHA: ${{ inputs.targetSha }}
with:
script: |
const { classify } = await import('${{ github.workspace }}/ci/github-script/supportedBranches.ts')
const { default: supportedSystems } = await import('${{ github.workspace }}/ci/github-script/supportedSystems.ts')
const { classify } = require('./ci/supportedBranches.js')
const supportedSystems = require('./ci/github-script/supportedSystems.js')
const baseBranch = (
context.payload.merge_group?.base_ref ??
@@ -63,7 +63,7 @@ jobs:
check:
name: Check
needs: [prepare]
uses: $/.github/workflows/check.yml
uses: ./.github/workflows/check.yml
permissions:
pull-requests: write # cherry-picks: unused in merge queue but required for check workflow
secrets:
@@ -75,7 +75,7 @@ jobs:
lint:
name: Lint
needs: [prepare]
uses: $/.github/workflows/lint.yml
uses: ./.github/workflows/lint.yml
secrets:
CACHIX_AUTH_TOKEN_GHA: ${{ secrets.CACHIX_AUTH_TOKEN_GHA }}
with:
@@ -85,7 +85,7 @@ jobs:
eval:
name: Eval
needs: [prepare]
uses: $/.github/workflows/eval.yml
uses: ./.github/workflows/eval.yml
# The eval workflow requests these permissions so we must explicitly allow them,
# even though they are unused when working with the merge queue.
permissions:
@@ -103,7 +103,7 @@ jobs:
build:
name: Build
needs: [prepare]
uses: $/.github/workflows/build.yml
uses: ./.github/workflows/build.yml
secrets:
CACHIX_AUTH_TOKEN_GHA: ${{ secrets.CACHIX_AUTH_TOKEN_GHA }}
with:

View File

@@ -40,7 +40,7 @@ jobs:
- name: merge-base(master,staging) → haskell-updates
from: master staging
into: haskell-updates
uses: $/.github/workflows/periodic-merge.yml
uses: ./.github/workflows/periodic-merge.yml
with:
from: ${{ matrix.pairs.from }}
into: ${{ matrix.pairs.into }}

View File

@@ -37,7 +37,7 @@ jobs:
into: staging
- from: master
into: staging-nixos
uses: $/.github/workflows/periodic-merge.yml
uses: ./.github/workflows/periodic-merge.yml
with:
from: ${{ matrix.pairs.from }}
into: ${{ matrix.pairs.into }}

View File

@@ -32,7 +32,7 @@ jobs:
client-id: ${{ vars.NIXPKGS_CI_CLIENT_ID }}
private-key: ${{ secrets.NIXPKGS_CI_APP_PRIVATE_KEY }}
permission-contents: write
permission-issues: write
permission-pull-requests: write
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
@@ -66,4 +66,4 @@ jobs:
Periodic merge from `${{ inputs.from }}` into [`${{ inputs.into }}`](https://github.com/NixOS/nixpkgs/tree/${{ inputs.into }}) has [failed](https://github.com/NixOS/nixpkgs/actions/runs/${{ github.run_id }}).
GH_TOKEN: ${{ steps.app-token.outputs.token }}
run: |
gh issue comment 562905 --body "$BODY_TEXT"
gh pr comment 105153 --body "$BODY_TEXT"

View File

@@ -64,8 +64,7 @@ jobs:
# https://github.com/octokit/plugin-retry.js/blob/9a2443746c350b3beedec35cf26e197ea318a261/src/index.ts#L14
retry-exempt-status-codes: 400,401,403,404
script: |
const { default: prepare } = await import('${{ github.workspace }}/ci/github-script/prepare.js')
await prepare({
require('./ci/github-script/prepare.js')({
github,
context,
core,
@@ -75,14 +74,13 @@ jobs:
check:
name: Check
needs: [prepare]
uses: $/.github/workflows/check.yml
uses: ./.github/workflows/check.yml
permissions:
# cherry-picks
pull-requests: write
secrets:
NIXPKGS_COMMIT_CHECK_APP_PRIVATE_KEY: ${{ secrets.NIXPKGS_COMMIT_CHECK_APP_PRIVATE_KEY }}
NIXPKGS_MANUAL_EDIT_CHECK_APP_PRIVATE_KEY: ${{ secrets.NIXPKGS_MANUAL_EDIT_CHECK_APP_PRIVATE_KEY }}
NIXPKGS_CI_APP_PRIVATE_KEY: ${{ secrets.NIXPKGS_CI_APP_PRIVATE_KEY }}
with:
baseBranch: ${{ needs.prepare.outputs.baseBranch }}
headBranch: ${{ needs.prepare.outputs.headBranch }}
@@ -92,7 +90,7 @@ jobs:
lint:
name: Lint
needs: [prepare]
uses: $/.github/workflows/lint.yml
uses: ./.github/workflows/lint.yml
with:
mergedSha: ${{ needs.prepare.outputs.mergedSha }}
targetSha: ${{ needs.prepare.outputs.targetSha }}
@@ -100,7 +98,7 @@ jobs:
eval:
name: Eval
needs: [prepare]
uses: $/.github/workflows/eval.yml
uses: ./.github/workflows/eval.yml
permissions:
# compare
pull-requests: write
@@ -119,7 +117,7 @@ jobs:
bot:
name: Bot
needs: [prepare, eval]
uses: $/.github/workflows/bot.yml
uses: ./.github/workflows/bot.yml
permissions:
issues: write
pull-requests: write
@@ -131,7 +129,7 @@ jobs:
build:
name: Build
needs: [prepare]
uses: $/.github/workflows/build.yml
uses: ./.github/workflows/build.yml
with:
artifact-prefix: ${{ inputs.artifact-prefix }}
baseBranch: ${{ needs.prepare.outputs.baseBranch }}

View File

@@ -40,7 +40,7 @@ jobs:
github-token: ${{ steps.app-token.outputs.token || github.token }}
retries: 3
script: |
const { handleMergeComment } = await import('${{ github.workspace }}/ci/github-script/merge.js')
const { handleMergeComment } = require('./ci/github-script/merge.js')
// PRs from forks don't have any PRs associated by default.
// Thus, we request the PR number with an API call *to* the fork's repo.

View File

@@ -38,7 +38,7 @@ jobs:
maintainers/github-teams.json
- name: Install dependencies
run: npm ci --package-lock-only=false --no-audit bottleneck
run: npm ci --package-lock-only=false bottleneck
working-directory: ci/github-script
- name: Synchronise teams
@@ -46,8 +46,7 @@ jobs:
with:
github-token: ${{ steps.app-token.outputs.token }}
script: |
const { default: getTeams } = await import('${{ github.workspace }}/ci/github-script/get-teams.js')
await getTeams({
require('./ci/github-script/get-teams.js')({
github,
context,
core,
@@ -79,3 +78,4 @@ jobs:
This is an automated PR to sync the GitHub teams with access to this repository to the `lib.teams` list.
This PR can be merged without taking any further action.

View File

@@ -35,8 +35,7 @@ jobs:
# https://github.com/octokit/plugin-retry.js/blob/9a2443746c350b3beedec35cf26e197ea318a261/src/index.ts#L14
retry-exempt-status-codes: 400,401,403,404
script: |
const { default: prepare } = await import('${{ github.workspace }}/ci/github-script/prepare.js')
await prepare({
require('./ci/github-script/prepare.js')({
github,
context,
core,
@@ -62,11 +61,9 @@ jobs:
'.github/workflows/lint.yml',
'.github/workflows/merge-group.yml',
'.github/workflows/test.yml',
'ci/github-script/package.json',
'ci/github-script/package-lock.json',
'ci/github-script/supportedBranches.ts',
'ci/github-script/supportedSystems.ts',
'ci/github-script/supportedSystems.js',
'ci/pinned.json',
'ci/supportedBranches.js',
'pkgs/top-level/release-supported-systems.json',
].includes(file))) core.setOutput('merge-group', true)
@@ -80,30 +77,26 @@ jobs:
'.github/workflows/pull-request-target.yml',
'.github/workflows/test.yml',
'ci/github-script/bot.js',
'ci/github-script/check-target-branch.ts',
'ci/github-script/commits.ts',
'ci/github-script/get-pr-commit-details.ts',
'ci/github-script/lint-commits.ts',
'ci/github-script/manual-file-edits.ts',
'ci/github-script/check-target-branch.js',
'ci/github-script/commits.js',
'ci/github-script/get-pr-commit-details.js',
'ci/github-script/lint-commits.js',
'ci/github-script/merge.js',
'ci/github-script/package.json',
'ci/github-script/package-lock.json',
'ci/github-script/prepare.js',
'ci/github-script/reminders.ts',
'ci/github-script/reviewers.js',
'ci/github-script/reviews.ts',
'ci/github-script/supportedBranches.ts',
'ci/github-script/supportedSystems.ts',
'ci/github-script/reviews.js',
'ci/github-script/supportedSystems.js',
'ci/github-script/withRateLimit.js',
'ci/pinned.json',
'ci/supportedBranches.js',
'pkgs/top-level/release-supported-systems.json',
].includes(file) || file.startsWith('ci/github-script/reminders/'))) core.setOutput('pr', true)
].includes(file))) core.setOutput('pr', true)
merge-group:
if: needs.prepare.outputs.merge-group
name: Merge Group
needs: [prepare]
uses: $/.github/workflows/merge-group.yml
uses: ./.github/workflows/merge-group.yml
# Those are actually only used on the merge_group event, but will throw an error if not set.
permissions:
pull-requests: write # unused on pull_request, required by merge-group workflow
@@ -117,7 +110,7 @@ jobs:
if: needs.prepare.outputs.pr
name: PR
needs: [prepare]
uses: $/.github/workflows/pull-request-target.yml
uses: ./.github/workflows/pull-request-target.yml
# Those are actually only used on the pull_request_target event, but will throw an error if not set.
permissions:
issues: write # unused on pull_request, required by bot workflow

View File

@@ -6,7 +6,6 @@ Christina Sørensen <christina@cafkafk.com> <christinaafk@gmail.com>
Christina Sørensen <christina@cafkafk.com> <89321978+cafkafk@users.noreply.github.com>
Daniel Løvbrøtte Olsen <me@dandellion.xyz> <daniel.olsen99@gmail.com>
Ethan Carter Edwards <ethan@ethancedwards.com> Ethan Edwards <ethancarteredwards@gmail.com>
Ethan Carter Edwards <ethan@ethancedwards.com> <ethancedwards8@users.noreply.github.com>
Fabian Affolter <mail@fabian-affolter.ch> <fabian@affolter-engineering.ch>
Fiona Behrens <me@kloenk.dev>
Fiona Behrens <me@kloenk.dev> <me@kloenk.de>
@@ -21,8 +20,6 @@ Jörg Thalheim <joerg@thalheim.io> <Mic92@users.noreply.github.com>
Katalin Rebhan <me@dblsaiko.net>
Lin Jian <me@linj.tech> <linj.dev@outlook.com>
Lin Jian <me@linj.tech> <75130626+jian-lin@users.noreply.github.com>
Marie Ramlow <marie@marie.cologne> <me@nycode.dev>
Marie Ramlow <marie@marie.cologne> <tabmeier12@gmail.com>
Martin Weinelt <hexa@darmstadt.ccc.de> <mweinelt@users.noreply.github.com>
Martin Häcker <spamfaenger@gmx.de> <spamfaenger@gmx.de>
moni <lythe1107@gmail.com> <lythe1107@icloud.com>

View File

@@ -571,10 +571,7 @@ If a contributor does not want committers to push to their branch, they must unc
### Release notes
If you add or remove a NixOS module, or make other breaking or significant NixOS changes, write about it in the next NixOS release notes in [`nixos/doc/manual/release-notes`](./nixos/doc/manual/release-notes).
If you make major or breaking changes to a package (other than removal), write about it in the next Nixpkgs release notes in [`doc/release-notes`](./doc/release-notes).
Package removals should not get a Nixpkgs release note, [a throwing alias should be added instead](./pkgs/README.md#steps-to-remove-a-package-from-nixpkgs).
If you removed packages or made some major NixOS changes, write about it in the next release notes in [`nixos/doc/manual/release-notes`](./nixos/doc/manual/release-notes).
### File naming and organisation
@@ -587,9 +584,9 @@ CI [enforces](./.github/workflows/lint.yml) all Nix files to be formatted using
You can ensure this locally using either of these commands:
```
nix fmt
nix develop --command treefmt
nix-shell --run treefmt
nix develop --command treefmt
nix fmt
```
If you're starting your editor in `nix-shell` or `nix develop`, you can also set it up to automatically run `treefmt` on save.
@@ -957,6 +954,7 @@ The following situations are fully or partially exempt:
If you believe that someone is using automation without appropriate disclosure and review, you can politely ask them if that’s the case and point them to this policy as appropriate.
Please assume good faith and remain civil; it’s not always possible to determine, and it is more likely that someone overlooked this policy than deliberately violated it.
If you think someone is continuing to break the policy after this, please escalate to the [Nixpkgs core team](https://nixos.org/community/teams/nixpkgs-core/) rather than fighting over it.
If a contribution is clearly in violation of the policy (e.g. the contributor admits it was not followed, or there are AI tool attributions that do not meet our required format), it can be closed or hidden, preferably after informing the contributor of the policy and giving them a chance to address the violations.
Deliberate violations of this policy are considered to break the [Code of Conduct](https://github.com/NixOS/.github/blob/master/CODE_OF_CONDUCT.md) clause against “Wasting other people’s time with low quality contributions, including but not limited to LLM and bot spam”.

View File

@@ -32,8 +32,8 @@
/lib/asserts.nix @infinisil @hsjobeki @llakala
/lib/path/* @infinisil @hsjobeki @llakala
/lib/fileset @infinisil @hsjobeki @llakala
/maintainers/github-teams.json @infinisil @llakala @NixOS/nixpkgs-ci
/maintainers/computed-team-list.nix @infinisil @llakala @NixOS/nixpkgs-ci
/maintainers/github-teams.json @infinisil @llakala
/maintainers/computed-team-list.nix @infinisil @llakala
## Standard environment–related libraries
/lib/customisation.nix @alyssais @NixOS/stdenv @llakala
/lib/derivations.nix @NixOS/stdenv @llakala
@@ -97,7 +97,8 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @Artturin @Ericson2314 @lo
# Contributor documentation
/CONTRIBUTING.md
/.github/PULL_REQUEST_TEMPLATE.md
/doc/contributing.md @infinisil
/doc/contributing/
/doc/contributing/contributing-to-documentation.chapter.md @jtojnar
/lib/README.md
/doc/README.md
/nixos/README.md
@@ -105,6 +106,10 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @Artturin @Ericson2314 @lo
/pkgs/by-name/README.md
/maintainers/README.md
# User-facing development documentation
/doc/development.md @infinisil
/doc/development @infinisil
# NixOS Internals
/nixos/default.nix @infinisil
/nixos/lib/from-env.nix @infinisil
@@ -194,7 +199,7 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @Artturin @Ericson2314 @lo
/doc/languages-frameworks/haskell.section.md @sternenseemann @maralorn @wolfgangwalther
/maintainers/scripts/haskell @sternenseemann @maralorn @wolfgangwalther
/pkgs/development/compilers/ghc @sternenseemann @maralorn @wolfgangwalther
/pkgs/development/compilers/ghc/9.6.6-debian-binary.nix @sternenseemann @maralorn @wolfgangwalther @OPNA2608 @liberodark @NixOS/loongarch64
/pkgs/development/compilers/ghc/9.6.6-debian-binary.nix @sternenseemann @maralorn @wolfgangwalther @OPNA2608
/pkgs/development/haskell-modules @sternenseemann @maralorn @wolfgangwalther
/pkgs/test/haskell @sternenseemann @maralorn @wolfgangwalther
/pkgs/top-level/release-haskell.nix @sternenseemann @maralorn @wolfgangwalther
@@ -206,7 +211,7 @@ nixos/modules/installer/tools/nix-fallback-paths.nix @Artturin @Ericson2314 @lo
/pkgs/development/perl-modules @stigtsp @marcusramberg
# R
/pkgs/by-name/r/R @jbedo
/pkgs/applications/science/math/R @jbedo
/pkgs/development/r-modules @jbedo
# Rust
@@ -258,15 +263,15 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/lib/licenses @alyssais @emilazy @jopejoe1
# Qt
/pkgs/development/libraries/qt-5 @NixOS/qt-kde
/pkgs/development/libraries/qt-6 @NixOS/qt-kde
/pkgs/development/libraries/qt-5 @K900 @NickCao @SuperSandro2000
/pkgs/development/libraries/qt-6 @K900 @NickCao @SuperSandro2000
# KDE Frameworks 5
/pkgs/development/libraries/kde-frameworks @NixOS/qt-kde
/pkgs/development/libraries/kde-frameworks @K900 @NickCao @SuperSandro2000
# KDE / Plasma 6
/pkgs/kde @NixOS/qt-kde
/maintainers/scripts/kde @NixOS/qt-kde
/pkgs/kde @K900 @NickCao @SuperSandro2000
/maintainers/scripts/kde @K900 @NickCao @SuperSandro2000
# PostgreSQL and related stuff
/pkgs/by-name/po/postgresqlTestHook @NixOS/postgres
@@ -356,10 +361,7 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/pkgs/applications/editors/kakoune @philiptaron
# LuaPackages
/pkgs/development/interpreters/lua-5 @NixOS/lua
/pkgs/development/interpreters/luajit @NixOS/lua
/pkgs/development/lua-modules @NixOS/lua
/pkgs/top-level/lua-packages.nix @NixOS/lua
# Neovim
/pkgs/applications/editors/neovim @NixOS/neovim
@@ -388,9 +390,9 @@ pkgs/development/python-modules/buildcatrust/ @ajs124 @lukegb @mweinelt
/doc/build-helpers/images/dockertools.section.md @roberth @jhol
# Go
/doc/languages-frameworks/go.section.md @kalbasit @Mic92
/pkgs/build-support/go @kalbasit @Mic92
/pkgs/development/compilers/go @kalbasit @Mic92
/doc/languages-frameworks/go.section.md @kalbasit @katexochen @Mic92
/pkgs/build-support/go @kalbasit @katexochen @Mic92
/pkgs/development/compilers/go @kalbasit @katexochen @Mic92
# GNOME
/pkgs/desktops/gnome @NixOS/gnome
@@ -444,9 +446,9 @@ nixos/tests/forgejo.nix @adamcstephens @bendlas @christoph-heiss @
/doc/hooks/zig.section.md @RossComputerGuy
# Buildbot
nixos/modules/services/continuous-integration/buildbot @Mic92
nixos/tests/buildbot.nix @Mic92
pkgs/development/tools/continuous-integration/buildbot @Mic92
nixos/modules/services/continuous-integration/buildbot @Mic92 @zowoq
nixos/tests/buildbot.nix @Mic92 @zowoq
pkgs/development/tools/continuous-integration/buildbot @Mic92 @zowoq
# Pretix
pkgs/by-name/pr/pretix/ @mweinelt
@@ -464,8 +466,9 @@ nixos/tests/incus/ @adamcstephens
pkgs/by-name/in/incus/ @adamcstephens
pkgs/by-name/lx/lxc* @adamcstephens
# Flutter
# ExpidusOS, Flutter
/pkgs/development/compilers/flutter @RossComputerGuy
/pkgs/desktops/expidus @RossComputerGuy
# GNU Tar & Zip
/pkgs/by-name/gn/gnutar @RossComputerGuy
@@ -523,7 +526,3 @@ pkgs/by-name/wa/warp-terminal/ @emilytrau @imadnyc @4evy @johnrtitor
/nixos/lib/testing @NixOS/test-driver
/nixos/tests/nixos-test-driver @NixOS/test-driver
/nixos/modules/virtualisation/nspawn-container/run-nspawn @NixOS/test-driver
# Boot security
/pkgs/by-name/au/autopen @NixOS/boot-security
/pkgs/misc/signed-packages @NixOS/boot-security

View File

@@ -104,7 +104,7 @@ For the purposes of CI, branches in the NixOS/nixpkgs repository are classified
Some branches also have a version component, which is either `unstable` or `YY.MM`.
`ci/github-script/supportedBranches.ts` is a script imported by CI to classify the base and head branches of a Pull Request.
`ci/supportedBranches.js` is a script imported by CI to classify the base and head branches of a Pull Request.
This classification will then be used to skip certain jobs.
This script can also be run locally to print basic test cases.

View File

@@ -1,5 +1,2 @@
comparison
comparison.zip
node_modules
step-summary.md
*.tsbuildinfo

View File

@@ -15,9 +15,3 @@ Run `./run commits OWNER REPO PR`, where OWNER is your username or "NixOS", REPO
## Labeler
Run `./run labels OWNER REPO`, where OWNER is your username or "NixOS" and REPO the name of your fork or "nixpkgs".
## Reminders
Run `./run reminders OWNER REPO PR` to post the reminders that apply to the paths a pull request touches.
To add a reminder, write the review body to `reminders/KEY.md` and list KEY with its path regexes in `reminders.js`.

View File

@@ -1,13 +1,12 @@
// @ts-nocheck
import { readFile, writeFile } from 'node:fs/promises'
import path from 'node:path'
import { DefaultArtifactClient } from '@actions/artifact'
import { handleMerge } from './merge.js'
import { handleReviewers } from './reviewers.js'
import { classify } from './supportedBranches.ts'
import withRateLimit from './withRateLimit.js'
module.exports = async ({ github, context, core, dry }) => {
const path = require('node:path')
const { DefaultArtifactClient } = await import('@actions/artifact')
const { readFile, writeFile } = require('node:fs/promises')
const withRateLimit = require('./withRateLimit.js')
const { classify } = require('../supportedBranches.js')
const { handleMerge } = require('./merge.js')
const { handleReviewers } = require('./reviewers.js')
export default async ({ github, context, core, dry }) => {
const artifactClient = new DefaultArtifactClient()
// Detect if running in a fork (not NixOS/nixpkgs)
@@ -397,11 +396,11 @@ export default async ({ github, context, core, dry }) => {
per_page: 100,
})
// label llm-assisted PRs accordingly, retaining it if manually set
// label llm-assisted PRs accordingly
const assistedByPattern = /Assisted-by: (?!nix-init)/i
if (prCommits.some((c) => assistedByPattern.test(c.commit.message))) {
evalLabels['llm-assisted'] = true
}
evalLabels['llm-assisted'] = prCommits.some((c) =>
assistedByPattern.test(c.commit.message),
)
const commitSubjects = prCommits.map(
(c) => c.commit.message.split('\n')[0],
@@ -465,10 +464,9 @@ export default async ({ github, context, core, dry }) => {
let owners = []
try {
// TODO: Create owner map similar to maintainer map.
owners =
(await readFile(`${pull_number}/owners.txt`, 'utf-8')).match(
/[^\n]+/g,
) || []
owners = (await readFile(`${pull_number}/owners.txt`, 'utf-8')).split(
'\n',
)
} catch (e) {
// Older artifacts don't have the owners.txt, yet.
if (e.code !== 'ENOENT') throw e
@@ -484,18 +482,6 @@ export default async ({ github, context, core, dry }) => {
if (e.code !== 'ENOENT') throw e
}
// TODO: Use maintainer map instead of the artifact.
const user_maintainers = Object.keys(
JSON.parse(
await readFile(`${pull_number}/maintainers.json`, 'utf-8'),
),
).map((id) => parseInt(id))
prLabels['7.no default reviewers'] =
user_maintainers.length === 0 &&
team_maintainers.length === 0 &&
owners.length === 0
// We set this label earlier already, but the current PR state can be very different
// after handleReviewers has requested reviews, so update it in this case to prevent
// this label from flip-flopping.
@@ -507,7 +493,12 @@ export default async ({ github, context, core, dry }) => {
dry,
pull_request,
reviews,
user_maintainers,
// TODO: Use maintainer map instead of the artifact.
user_maintainers: Object.keys(
JSON.parse(
await readFile(`${pull_number}/maintainers.json`, 'utf-8'),
),
).map((id) => parseInt(id)),
team_maintainers,
owners,
getUser,
@@ -697,21 +688,16 @@ export default async ({ github, context, core, dry }) => {
if (context.payload.pull_request) {
await handle({ item: context.payload.pull_request, stats })
} else {
// We don't use filters here because that causes GitHub to use an often-outdated index,
// resulting in the cursor not being updated, and therefore causing the same PRs
// to use up our rate limit over and over again.
const lastRun = (
await github.rest.actions.listWorkflowRuns({
...context.repo,
workflow_id: 'bot.yml',
event: 'schedule',
status: 'success',
exclude_pull_requests: true,
per_page: 1,
})
).data.workflow_runs.find(
(run) => run.event === 'schedule' && run.conclusion === 'success',
)
core.info(
`Last successful run created at: ${lastRun?.created_at ?? '<n/a>'}`,
)
).data.workflow_runs[0]
const cutoff = new Date(
Math.max(
@@ -807,7 +793,6 @@ export default async ({ github, context, core, dry }) => {
} else {
// No stats.artifacts++, because this does not allow passing a custom token.
// Thus, the upload will not happen with the app token, but the default github.token.
core.info(`pagination-cursor: ${cursor}`)
await artifactClient.uploadArtifact(
'pagination-cursor',
[uploadPath],
@@ -817,8 +802,6 @@ export default async ({ github, context, core, dry }) => {
},
)
}
} else {
core.info('pagination-cursor: <n/a>')
}
// Some items might be in both search results, so filtering out duplicates as well.

View File

@@ -1,242 +0,0 @@
import assert from 'node:assert/strict'
import test from 'node:test'
import { evaluateTargetBranchPolicy } from './check-target-branch-policy.ts'
type DecisionFacts = {
base: string
head: string
maxRebuildCount: number
rebuildsAllTests: boolean
onlyChangedFile: string | null
}
type Decision =
| 'mass-rebuild'
| 'nixos-rebuild'
| 'possible-mass-rebuild'
| 'skip-development-merge'
| 'dismiss'
const defaults: DecisionFacts = {
base: 'master',
head: 'topic-branch',
maxRebuildCount: 0,
rebuildsAllTests: false,
onlyChangedFile: null,
}
const cases: Array<{
name: string
facts: Partial<DecisionFacts>
expected: Decision
}> = [
{
name: 'allows fewer than 500 rebuilds on master',
facts: { maxRebuildCount: 499 },
expected: 'dismiss',
},
{
name: 'flags 500 rebuilds on master as a possible mass rebuild',
facts: { maxRebuildCount: 500 },
expected: 'possible-mass-rebuild',
},
{
name: 'flags 999 rebuilds on master as a possible mass rebuild',
facts: { maxRebuildCount: 999 },
expected: 'possible-mass-rebuild',
},
{
name: 'flags 1000 rebuilds on master as a mass rebuild',
facts: { maxRebuildCount: 1000 },
expected: 'mass-rebuild',
},
{
name: 'flags a mass rebuild on staging-nixos',
facts: { base: 'staging-nixos', maxRebuildCount: 24_000 },
expected: 'mass-rebuild',
},
{
name: 'flags a mass rebuild on a release staging-nixos branch',
facts: { base: 'staging-nixos-26.05', maxRebuildCount: 1000 },
expected: 'mass-rebuild',
},
{
name: 'allows mass rebuilds on staging',
facts: { base: 'staging', maxRebuildCount: 1000 },
expected: 'dismiss',
},
{
name: 'flags a mass rebuild on a release branch',
facts: { base: 'release-26.05', maxRebuildCount: 1000 },
expected: 'mass-rebuild',
},
{
name: 'flags NixOS test rebuilds on master',
facts: { rebuildsAllTests: true },
expected: 'nixos-rebuild',
},
{
name: 'flags NixOS test rebuilds on a release branch',
facts: { base: 'release-26.05', rebuildsAllTests: true },
expected: 'nixos-rebuild',
},
{
name: 'allows NixOS test rebuilds on staging-nixos',
facts: { base: 'staging-nixos', rebuildsAllTests: true },
expected: 'dismiss',
},
{
name: 'does not flag a possible mass rebuild when staging-nixos rebuilds all NixOS tests',
facts: {
base: 'staging-nixos',
maxRebuildCount: 500,
rebuildsAllTests: true,
},
expected: 'dismiss',
},
{
name: 'flags other possible mass rebuilds on staging-nixos',
facts: { base: 'staging-nixos', maxRebuildCount: 500 },
expected: 'possible-mass-rebuild',
},
{
name: 'skips staging into master',
facts: {
head: 'staging',
maxRebuildCount: 24_000,
},
expected: 'skip-development-merge',
},
{
name: 'skips staging-nixos into master',
facts: {
head: 'staging-nixos',
maxRebuildCount: 24_000,
},
expected: 'skip-development-merge',
},
{
name: 'skips master into staging-nixos',
facts: {
base: 'staging-nixos',
head: 'master',
maxRebuildCount: 24_000,
},
expected: 'skip-development-merge',
},
{
name: 'checks staging into staging-nixos',
facts: {
base: 'staging-nixos',
head: 'staging',
maxRebuildCount: 24_000,
},
expected: 'mass-rebuild',
},
{
name: 'skips release staging-nixos into its release branch',
facts: {
base: 'release-26.05',
head: 'staging-nixos-26.05',
maxRebuildCount: 24_000,
},
expected: 'skip-development-merge',
},
{
name: 'skips a release branch into its staging-nixos branch',
facts: {
base: 'staging-nixos-26.05',
head: 'release-26.05',
maxRebuildCount: 24_000,
},
expected: 'skip-development-merge',
},
{
name: 'checks release staging into its staging-nixos branch',
facts: {
base: 'staging-nixos-26.05',
head: 'staging-26.05',
maxRebuildCount: 24_000,
},
expected: 'mass-rebuild',
},
{
name: 'kernels-org exemption suppresses a possible mass rebuild',
facts: {
maxRebuildCount: 999,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/kernels-org.json',
},
expected: 'dismiss',
},
{
name: 'kernels-org exemption suppresses a definite mass rebuild',
facts: {
maxRebuildCount: 1000,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/kernels-org.json',
},
expected: 'dismiss',
},
{
name: 'kernels-org exemption suppresses a NixOS test rebuild',
facts: {
rebuildsAllTests: true,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/kernels-org.json',
},
expected: 'dismiss',
},
{
name: 'xanmod kernel exemption suppresses a possible mass rebuild',
facts: {
maxRebuildCount: 999,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/xanmod-kernels.nix',
},
expected: 'dismiss',
},
{
name: 'xanmod kernel exemption suppresses a definite mass rebuild',
facts: {
maxRebuildCount: 1000,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/xanmod-kernels.nix',
},
expected: 'dismiss',
},
{
name: 'xanmod kernel exemption suppresses a NixOS test rebuild',
facts: {
rebuildsAllTests: true,
onlyChangedFile: 'pkgs/os-specific/linux/kernel/xanmod-kernels.nix',
},
expected: 'dismiss',
},
{
name: 'Home Assistant exemption suppresses a mass rebuild',
facts: {
head: 'wip-home-assistant',
maxRebuildCount: 1500,
},
expected: 'dismiss',
},
{
name: 'does not exempt a Home Assistant update above 1500 rebuilds',
facts: { head: 'wip-home-assistant', maxRebuildCount: 1501 },
expected: 'mass-rebuild',
},
{
name: 'Home Assistant exemption does not suppress a NixOS test rebuild',
facts: {
head: 'wip-home-assistant',
maxRebuildCount: 1500,
rebuildsAllTests: true,
},
expected: 'nixos-rebuild',
},
]
for (const { name, facts, expected } of cases) {
test(name, () => {
assert.equal(
evaluateTargetBranchPolicy({ ...defaults, ...facts }).decision,
expected,
)
})
}

View File

@@ -1,116 +0,0 @@
import { classify, split } from './supportedBranches.ts'
type TargetBranchPolicyFacts = {
base: string
head: string
maxRebuildCount: number
rebuildsAllTests: boolean
onlyChangedFile: string | null
}
type TargetBranchReviewDecision =
| 'mass-rebuild'
| 'nixos-rebuild'
| 'possible-mass-rebuild'
| 'skip-development-merge'
| 'dismiss'
type TargetBranchPolicyResult = {
decision: TargetBranchReviewDecision
details: {
isExemptKernelUpdate: boolean
isExemptHomeAssistantUpdate: boolean
}
}
export function getTargetBranchPolicy({
base,
head,
}: {
base: string
head: string
}) {
const baseClassification = classify(base)
const headClassification = classify(head)
const isPrimaryBase = baseClassification.type.includes('primary')
const isPrimaryHead = headClassification.type.includes('primary')
const isStagingNixosBase = split(base).prefix === 'staging-nixos'
const isDevelopmentHead = headClassification.type.includes('development')
const shouldSkipDevelopmentMerge =
isDevelopmentHead && (!isStagingNixosBase || isPrimaryHead)
return {
isStagingNixosBase,
shouldSkipDevelopmentMerge,
shouldCheckMassRebuild:
!shouldSkipDevelopmentMerge && (isPrimaryBase || isStagingNixosBase),
shouldCheckNixosRebuild: !shouldSkipDevelopmentMerge && isPrimaryBase,
}
}
export function evaluateTargetBranchPolicy({
base,
head,
maxRebuildCount,
rebuildsAllTests,
onlyChangedFile,
}: TargetBranchPolicyFacts): TargetBranchPolicyResult {
const {
isStagingNixosBase,
shouldSkipDevelopmentMerge,
shouldCheckMassRebuild,
shouldCheckNixosRebuild,
} = getTargetBranchPolicy({ base, head })
// https://github.com/NixOS/nixpkgs/pull/553786#issuecomment-5510286851
// kernels-org should go to staging-nixos (or master) and staging-nixos-xx.xx (or release-xx.xx) when backported
// https://github.com/NixOS/nixpkgs/pull/521157
// xanmod should go to master and release-xx.xx when backported
const isExemptKernelUpdate =
onlyChangedFile === 'pkgs/os-specific/linux/kernel/kernels-org.json' ||
onlyChangedFile === 'pkgs/os-specific/linux/kernel/xanmod-kernels.nix'
// https://github.com/NixOS/nixpkgs/pull/483194#issuecomment-3793393218
const isExemptHomeAssistantUpdate =
maxRebuildCount <= 1500 && head === 'wip-home-assistant'
const details = {
isExemptKernelUpdate,
isExemptHomeAssistantUpdate,
}
const result = (decision: TargetBranchReviewDecision) => ({
decision,
details,
})
const isMassRebuild =
maxRebuildCount >= 1000 &&
!isExemptKernelUpdate &&
!isExemptHomeAssistantUpdate
if (shouldCheckMassRebuild && isMassRebuild) {
return result('mass-rebuild')
}
if (shouldCheckNixosRebuild && rebuildsAllTests && !isExemptKernelUpdate) {
return result('nixos-rebuild')
}
const isPossibleMassRebuild =
maxRebuildCount >= 500 &&
!isMassRebuild &&
!isExemptKernelUpdate &&
!isExemptHomeAssistantUpdate
if (
shouldCheckMassRebuild &&
isPossibleMassRebuild &&
!(rebuildsAllTests && isStagingNixosBase)
) {
return result('possible-mass-rebuild')
}
return result(
shouldSkipDevelopmentMerge ? 'skip-development-merge' : 'dismiss',
)
}

View File

@@ -0,0 +1,221 @@
/// @ts-check
// TODO: should this be combined with the branch checks in prepare.js?
// They do seem quite similar, but this needs to run after eval,
// and prepare.js obviously doesn't.
const { classify, split } = require('../supportedBranches.js')
const { readFile } = require('node:fs/promises')
const { postReview, dismissReviews } = require('./reviews.js')
const reviewKey = 'check-target-branch'
/**
* @param {{
* github: InstanceType<import('@actions/github/lib/utils').GitHub>,
* context: import('@actions/github/lib/context').Context
* core: import('@actions/core')
* dry: boolean
* }} CheckTargetBranchProps
*/
async function checkTargetBranch({ github, context, core, dry }) {
/**
* @type {{
* attrdiff: {
* added: string[],
* changed: string[],
* removed: string[],
* },
* attrdiffByKernel: Record<string, {
* added: string[],
* changed: string[],
* removed: string[],
* }>,
* attrdiffByPlatform: Record<string, {
* added: string[],
* changed: string[],
* removed: string[],
* }>,
* labels: Record<string, boolean>,
* rebuildCountByKernel: Record<string, number>,
* rebuildsByKernel: Record<string, string[]>,
* rebuildsByPlatform: Record<string, string[]>,
* }}
*/
const changed = JSON.parse(
await readFile('comparison/changed-paths.json', 'utf-8'),
)
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning(
'Skipping checkTargetBranch: no pull_request number (is this being run as part of a merge group?)',
)
return
}
const prInfo = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
const base = prInfo.base.ref
const head = prInfo.head.ref
const baseClassification = classify(base)
const headClassification = classify(head)
// Don't run on, e.g., staging-nixos to master merges.
if (headClassification.type.includes('development')) {
core.info(
`Skipping checkTargetBranch: PR is from a development branch (${head})`,
)
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
return
}
// Don't run on PRs against staging branches, wip branches, haskell-updates, etc.
if (!baseClassification.type.includes('primary')) {
core.info(
`Skipping checkTargetBranch: PR is against a non-primary base branch (${base})`,
)
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
return
}
const maxRebuildCount = Math.max(
...Object.values(changed.rebuildCountByKernel),
)
const rebuildsAllTests =
changed.attrdiff.changed.includes('nixosTests.simple-container') ||
changed.attrdiff.changed.includes('nixosTests.simple-vm')
// https://github.com/NixOS/nixpkgs/pull/521157
// These should go to master and release-xx.xx when backported
let isExemptKernelUpdate = false
if (prInfo.changed_files === 1) {
const changedFiles = (
await github.rest.pulls.listFiles({
...context.repo,
pull_number,
})
).data
isExemptKernelUpdate =
changedFiles.length === 1 &&
changedFiles[0].filename ===
'pkgs/os-specific/linux/kernel/xanmod-kernels.nix'
}
// https://github.com/NixOS/nixpkgs/pull/483194#issuecomment-3793393218
const isExemptHomeAssistantUpdate =
maxRebuildCount <= 1500 && head === 'wip-home-assistant'
core.info(
[
`checkTargetBranch: this PR:`,
` * causes ${maxRebuildCount} rebuilds`,
` * ${rebuildsAllTests ? 'rebuilds' : 'does not rebuild'} all NixOS tests`,
` * ${isExemptKernelUpdate ? 'is' : 'is not'} an exempt kernel update`,
` * ${isExemptHomeAssistantUpdate ? 'is' : 'is not'} an exempt home-assistant update`,
].join('\n'),
)
if (
maxRebuildCount >= 1000 &&
!isExemptHomeAssistantUpdate &&
!isExemptKernelUpdate
) {
const desiredBranch =
base === 'master' ? 'staging' : `staging-${split(base).version}`
const body = [
`The PR's base branch is set to \`${base}\`, but this PR causes ${maxRebuildCount} rebuilds.`,
'It is therefore considered a mass rebuild.',
`Please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) (probably \`${desiredBranch}\`).`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
} else if (rebuildsAllTests && !isExemptKernelUpdate) {
let branchText
if (base === 'master' && maxRebuildCount >= 500) {
branchText = '(probably either `staging-nixos` or `staging`)'
} else if (base === 'master') {
branchText = '(probably `staging-nixos`)'
} else if (maxRebuildCount >= 500) {
branchText = `(probably either \`staging-nixos-${split(base).version}\` or \`staging-${split(base).version}\`)`
} else {
branchText = `(probably \`staging-nixos-${split(base).version}\`)`
}
const body = [
`The PR's base branch is set to \`${base}\`, but this PR rebuilds all NixOS tests.`,
base === 'master' && maxRebuildCount >= 500
? `Since this PR also causes ${maxRebuildCount} rebuilds, it may also be considered a mass rebuild.`
: '',
`Please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) ${branchText}.`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
} else if (
maxRebuildCount >= 500 &&
!isExemptKernelUpdate &&
!isExemptHomeAssistantUpdate
) {
const stagingBranch =
base === 'master' ? 'staging' : `staging-${split(base).version}`
const body = [
`The PR's base branch is set to \`${base}\`, and this PR causes ${maxRebuildCount} rebuilds.`,
`Please consider whether this PR causes a mass rebuild according to [our conventions](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions).`,
`If it does cause a mass rebuild, please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) (probably \`${stagingBranch}\`).`,
`If it does not cause a mass rebuild, this message can be ignored.`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
} else {
core.info('checkTargetBranch: this PR is against an appropriate branch.')
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
}
}
module.exports = checkTargetBranch

View File

@@ -1,242 +0,0 @@
import { readFile } from 'node:fs/promises'
import type * as actionsCore from '@actions/core'
import type { context as actionsContext } from '@actions/github'
import type { GitHub } from '@actions/github/lib/utils'
import {
evaluateTargetBranchPolicy,
getTargetBranchPolicy,
} from './check-target-branch-policy.ts'
import { dismissReviews, postReview } from './reviews.ts'
import { split } from './supportedBranches.ts'
// TODO: should this be combined with the branch checks in prepare.js?
// They do seem quite similar, but this needs to run after eval,
// and prepare.js obviously doesn't.
const reviewKey = 'check-target-branch'
type ChangedPaths = {
attrdiff: {
added: string[]
changed: string[]
removed: string[]
}
attrdiffByKernel: Record<
string,
{
added: string[]
changed: string[]
removed: string[]
}
>
attrdiffByPlatform: Record<
string,
{
added: string[]
changed: string[]
removed: string[]
}
>
labels: Record<string, boolean>
rebuildCountByKernel: Record<string, number>
rebuildsByKernel: Record<string, string[]>
rebuildsByPlatform: Record<string, string[]>
}
type TargetBranchReviewFacts = {
github: InstanceType<typeof GitHub>
context: typeof actionsContext
core: typeof actionsCore
dry: boolean
base: string
maxRebuildCount: number
}
function getStagingBranch(base: string) {
const version = split(base).version
return version ? `staging-${version}` : 'staging'
}
async function postMassRebuildReview(facts: TargetBranchReviewFacts) {
const { github, context, core, dry, base, maxRebuildCount } = facts
const desiredBranch = getStagingBranch(base)
const body = [
`The PR's base branch is set to \`${base}\`, but this PR causes ${maxRebuildCount} rebuilds.`,
'It is therefore considered a mass rebuild.',
`Please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) (probably \`${desiredBranch}\`).`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
}
async function postNixosRebuildReview(facts: TargetBranchReviewFacts) {
const { github, context, core, dry, base, maxRebuildCount } = facts
let branchText: string
if (base === 'master' && maxRebuildCount >= 500) {
branchText = '(probably either `staging-nixos` or `staging`)'
} else if (base === 'master') {
branchText = '(probably `staging-nixos`)'
} else if (maxRebuildCount >= 500) {
branchText = `(probably either \`staging-nixos-${split(base).version}\` or \`staging-${split(base).version}\`)`
} else {
branchText = `(probably \`staging-nixos-${split(base).version}\`)`
}
const body = [
`The PR's base branch is set to \`${base}\`, but this PR rebuilds all NixOS tests.`,
base === 'master' && maxRebuildCount >= 500
? `Since this PR also causes ${maxRebuildCount} rebuilds, it may also be considered a mass rebuild.`
: '',
`Please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) ${branchText}.`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
}
async function postPossibleMassRebuildReview(facts: TargetBranchReviewFacts) {
const { github, context, core, dry, base, maxRebuildCount } = facts
const stagingBranch = getStagingBranch(base)
const body = [
`The PR's base branch is set to \`${base}\`, and this PR causes ${maxRebuildCount} rebuilds.`,
`Please consider whether this PR causes a mass rebuild according to [our conventions](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions).`,
`If it does cause a mass rebuild, please [change the base branch](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/changing-the-base-branch-of-a-pull-request) to [the right base branch for your changes](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#branch-conventions) (probably \`${stagingBranch}\`).`,
`If it does not cause a mass rebuild, this message can be ignored.`,
].join('\n')
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
}
export async function checkTargetBranch({
github,
context,
core,
dry,
}: {
github: InstanceType<typeof GitHub>
context: typeof actionsContext
core: typeof actionsCore
dry: boolean
}) {
const changed: ChangedPaths = JSON.parse(
await readFile('comparison/changed-paths.json', 'utf-8'),
)
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning(
'Skipping checkTargetBranch: no pull_request number (is this being run as part of a merge group?)',
)
return
}
const prInfo = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
const base = prInfo.base.ref
const head = prInfo.head.ref
const { shouldCheckMassRebuild } = getTargetBranchPolicy({ base, head })
const maxRebuildCount = Math.max(
...Object.values(changed.rebuildCountByKernel),
)
const rebuildsAllTests =
changed.attrdiff.changed.includes('nixosTests.simple-container') ||
changed.attrdiff.changed.includes('nixosTests.simple-vm')
let onlyChangedFile: string | null = null
if (shouldCheckMassRebuild && prInfo.changed_files === 1) {
const changedFiles = (
await github.rest.pulls.listFiles({
...context.repo,
pull_number,
})
).data
onlyChangedFile =
changedFiles.length === 1 ? changedFiles[0].filename : null
}
const {
decision,
details: { isExemptKernelUpdate, isExemptHomeAssistantUpdate },
} = evaluateTargetBranchPolicy({
base,
head,
maxRebuildCount,
rebuildsAllTests,
onlyChangedFile,
})
core.info(
[
`checkTargetBranch: this PR:`,
` * causes ${maxRebuildCount} rebuilds`,
` * ${rebuildsAllTests ? 'rebuilds' : 'does not rebuild'} all NixOS tests`,
` * ${isExemptKernelUpdate ? 'is' : 'is not'} an exempt kernel update`,
` * ${isExemptHomeAssistantUpdate ? 'is' : 'is not'} an exempt home-assistant update`,
].join('\n'),
)
const reviewFacts: TargetBranchReviewFacts = {
github,
context,
core,
dry,
base,
maxRebuildCount,
}
if (decision === 'mass-rebuild') {
await postMassRebuildReview(reviewFacts)
return
}
if (decision === 'nixos-rebuild') {
await postNixosRebuildReview(reviewFacts)
return
}
if (decision === 'possible-mass-rebuild') {
await postPossibleMassRebuildReview(reviewFacts)
return
}
if (decision === 'skip-development-merge') {
core.info(
`Skipping checkTargetBranch: PR merges the development branch ${head} into ${base}`,
)
} else {
core.info('checkTargetBranch: this PR is against an appropriate branch.')
}
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
}

322
ci/github-script/commits.js Normal file
View File

@@ -0,0 +1,322 @@
module.exports = async ({ github, context, core, dry, cherryPicks }) => {
const { execFileSync } = require('node:child_process')
const { classify } = require('../supportedBranches.js')
const withRateLimit = require('./withRateLimit.js')
const { dismissReviews, postReview } = require('./reviews.js')
const reviewKey = 'check-commits'
await withRateLimit({ github, core }, async (stats) => {
stats.prs = 1
const pull_number = context.payload.pull_request.number
const job_url =
context.runId &&
(
await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
...context.repo,
run_id: context.runId,
per_page: 100,
})
).find(({ name }) => name.endsWith('Check / commits')).html_url +
'?pr=' +
pull_number
async function extract({ sha, commit }) {
const noCherryPick = Array.from(
commit.message.matchAll(/^Not-cherry-picked-because: (.*)$/gm),
).at(0)
if (noCherryPick)
return {
sha,
commit,
severity: 'important',
message: `${sha} is not a cherry-pick, because: ${noCherryPick[1]}. Please review this commit manually.`,
type: 'no-cherry-pick',
}
// Using the last line with "cherry" + hash, because a chained backport
// can result in multiple of those lines. Only the last one counts.
const cherry = Array.from(
commit.message.matchAll(/cherry.*([0-9a-f]{40})/g),
).at(-1)
if (!cherry)
return {
sha,
commit,
severity: 'warning',
message: `Couldn't locate the cherry-picked commit's hash in the commit message of ${sha}.`,
type: 'no-commit-hash',
}
const original_sha = cherry[1]
let branches
try {
branches = (
await github.request({
// This is an undocumented endpoint to fetch the branches a commit is part of.
// There is no equivalent in neither the REST nor the GraphQL API.
// The endpoint itself is unlikely to go away, because GitHub uses it to display
// the list of branches on the detail page of a commit.
url: `https://github.com/${context.repo.owner}/${context.repo.repo}/branch_commits/${original_sha}`,
headers: {
accept: 'application/json',
},
})
).data.branches
.map(({ branch }) => branch)
.filter((branch) => classify(branch).type.includes('development'))
} catch (e) {
// For some unknown reason a 404 error comes back as 500 without any more details in a GitHub Actions runner.
// Ignore these to return a regular error message below.
if (![404, 500].includes(e.status)) throw e
}
if (!branches?.length)
return {
sha,
commit,
severity: 'error',
message: `${original_sha} given in ${sha} not found in any pickable branch.`,
}
return {
sha,
commit,
original_sha,
}
}
function diff({ sha, commit, original_sha }) {
const diff = execFileSync('git', [
'-C',
__dirname,
'range-diff',
'--no-color',
'--ignore-all-space',
'--no-notes',
// 100 means "any change will be reported"; 0 means "no change will be reported"
'--creation-factor=100',
`${original_sha}~..${original_sha}`,
`${sha}~..${sha}`,
])
.toString()
.split('\n')
// First line contains commit SHAs, which we'll print separately.
.slice(1)
// # The output of `git range-diff` is indented with 4 spaces, but we'll control indentation manually.
.map((line) => line.replace(/^ {4}/, ''))
if (!diff.some((line) => line.match(/^[+-]{2}/)))
return {
sha,
commit,
severity: 'info',
message: `✔ ${original_sha} is highly similar to ${sha}.`,
}
const colored_diff = execFileSync('git', [
'-C',
__dirname,
'range-diff',
'--color',
'--no-notes',
'--creation-factor=100',
`${original_sha}~..${original_sha}`,
`${sha}~..${sha}`,
]).toString()
return {
sha,
commit,
diff,
colored_diff,
severity: 'warning',
message: `Difference between ${sha} and original ${original_sha} may warrant inspection.`,
type: 'diff',
}
}
// For now we short-circuit the list of commits when cherryPicks should not be checked.
// This will not run any checks, but still trigger the "dismiss reviews" part below.
const commits = !cherryPicks
? []
: await github.paginate(github.rest.pulls.listCommits, {
...context.repo,
pull_number,
})
const extracted = await Promise.all(commits.map(extract))
const fetch = extracted
.filter(({ severity }) => !severity)
.flatMap(({ sha, original_sha }) => [sha, original_sha])
if (fetch.length > 0) {
// Fetching all commits we need for diff at once is much faster than any other method.
execFileSync('git', [
'-C',
__dirname,
'fetch',
'--depth=2',
'origin',
...fetch,
])
}
const results = extracted.map((result) =>
result.severity ? result : diff(result),
)
// Log all results without truncation, with better highlighting and all whitespace changes to the job log.
results.forEach(({ sha, commit, severity, message, colored_diff }) => {
core.startGroup(`Commit ${sha}`)
core.info(`Author: ${commit.author.name} ${commit.author.email}`)
core.info(`Date: ${new Date(commit.author.date)}`)
switch (severity) {
case 'error':
core.error(message)
break
case 'warning':
core.warning(message)
break
default:
core.info(message)
}
core.endGroup()
if (colored_diff) core.info(colored_diff)
})
// Only create step summary below in case of warnings or errors.
// Also clean up older reviews, when all checks are good now.
// An empty results array will always trigger this condition, which is helpful
// to clean up reviews created by the prepare step when on the wrong branch.
if (results.every(({ severity }) => severity === 'info')) {
await dismissReviews({ github, context, dry, reviewKey })
return
}
// In the case of "error" severity, we also fail the job.
// Those should be considered blocking and not be dismissable via review.
if (results.some(({ severity }) => severity === 'error'))
process.exitCode = 1
core.summary.addRaw(
'This report is automatically generated by the `PR / Check / cherry-pick` CI workflow.',
true,
)
core.summary.addEOL()
core.summary.addRaw(
"Some of the commits in this PR require the author's and reviewer's attention.",
true,
)
core.summary.addEOL()
if (results.some(({ type }) => type === 'no-commit-hash')) {
core.summary.addRaw(
'Please follow the [backporting guidelines](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#how-to-backport-pull-requests) and cherry-pick with the `-x` flag.',
true,
)
core.summary.addRaw(
'This requires changes to the unstable `master` and `staging` branches first, before backporting them.',
true,
)
core.summary.addEOL()
core.summary.addRaw(
'Occasionally, commits are not cherry-picked at all, for example when updating minor versions of packages which have already advanced to the next major on unstable.',
true,
)
core.summary.addRaw(
'These commits can optionally be marked with a `Not-cherry-picked-because: <reason>` footer.',
true,
)
core.summary.addEOL()
}
if (results.some(({ type }) => type === 'diff')) {
core.summary.addRaw(
'Sometimes it is not possible to cherry-pick exactly the same patch.',
true,
)
core.summary.addRaw(
'This most frequently happens when resolving merge conflicts.',
true,
)
core.summary.addRaw(
'The range-diff will help to review the resolution of conflicts.',
true,
)
core.summary.addEOL()
}
core.summary.addRaw(
'If you need to merge this PR despite the warnings, please [dismiss](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/dismissing-a-pull-request-review) this review shortly before merging.',
true,
)
results.forEach(({ severity, message, diff }) => {
if (severity === 'info') return
// The docs for markdown alerts only show examples with markdown blockquote syntax, like this:
// > [!WARNING]
// > message
// However, our testing shows that this also works with a `<blockquote>` html tag, as long as there
// is an empty line:
// <blockquote>
//
// [!WARNING]
// message
// </blockquote>
// Whether this is intended or just an implementation detail is unclear.
core.summary.addRaw('<blockquote>')
core.summary.addRaw(
`\n\n[!${{ important: 'IMPORTANT', warning: 'WARNING', error: 'CAUTION' }[severity]}]`,
true,
)
core.summary.addRaw(`${message}`, true)
if (diff) {
// Limit the output to 10k bytes and remove the last, potentially incomplete line, because GitHub
// comments are limited in length. The value of 10k is arbitrary with the assumption, that after
// the range-diff becomes a certain size, a reviewer is better off reviewing the regular diff in
// GitHub's UI anyway, thus treating the commit as "new" and not cherry-picked.
// Note: if multiple commits are close to the limit, this approach could still lead to a comment
// that's too long. We think this is unlikely to happen, and so don't deal with it explicitly.
const truncated = []
let total_length = 0
for (line of diff) {
total_length += line.length
if (total_length > 10000) {
truncated.push('', '[...truncated...]')
break
} else {
truncated.push(line)
}
}
core.summary.addRaw('<details><summary>Show diff</summary>')
core.summary.addRaw('\n\n``````````diff', true)
core.summary.addRaw(truncated.join('\n'), true)
core.summary.addRaw('``````````', true)
core.summary.addRaw('</details>')
}
core.summary.addRaw('</blockquote>')
})
if (job_url)
core.summary.addRaw(
`\n\n_Hint: The full diffs are also available in the [runner logs](${job_url}) with slightly better highlighting._`,
)
const body = core.summary.stringify()
core.summary.write()
// Posting a review could fail for very long comments. This can only happen with
// multiple commits all hitting the truncation limit for the diff. If you ever hit
// this case, consider just splitting up those commits into multiple PRs.
await postReview({ github, context, core, dry, body, reviewKey })
})
}

View File

@@ -1,397 +0,0 @@
import { execFileSync } from 'node:child_process'
import type * as actionsCore from '@actions/core'
import type { context as actionsContext } from '@actions/github'
import type { GitHub } from '@actions/github/lib/utils'
import { dismissReviews, postReview } from './reviews.ts'
import { classify } from './supportedBranches.ts'
import withRateLimit from './withRateLimit.js'
const dirname = import.meta.dirname
type PullRequestCommit = Awaited<
ReturnType<InstanceType<typeof GitHub>['rest']['pulls']['listCommits']>
>['data'][number]
type CommitDetails = Pick<PullRequestCommit, 'sha' | 'commit'>
type CherryPick = CommitDetails & {
original_sha: string
severity?: undefined
}
type CheckResult = CommitDetails & {
diff?: string[]
colored_diff?: string
severity: 'important' | 'warning' | 'error' | 'info'
message: string
type?: 'no-cherry-pick' | 'no-commit-hash' | 'diff'
}
type ExtractedCommit = CheckResult | CherryPick
type CheckCommitsProps = {
github: InstanceType<typeof GitHub>
context: typeof actionsContext
core: typeof actionsCore
dry: boolean
cherryPicks: boolean
}
type CommitBranches = {
branches: {
branch: string
}[]
}
type RateLimitStats = {
prs: number
}
function isCherryPick(result: ExtractedCommit): result is CherryPick {
return !result.severity
}
function isIgnoredBranchError(error: unknown) {
return (
typeof error === 'object' &&
error !== null &&
'status' in error &&
(error.status === 404 || error.status === 500)
)
}
export default async ({
github,
context,
core,
dry,
cherryPicks,
}: CheckCommitsProps) => {
const reviewKey = 'check-commits'
await withRateLimit({ github, core }, async (stats: RateLimitStats) => {
stats.prs = 1
const pull_number = context.payload.pull_request!.number
const job_url =
context.runId &&
(
await github.paginate(github.rest.actions.listJobsForWorkflowRun, {
...context.repo,
run_id: context.runId,
per_page: 100,
})
).find(({ name }) => name.endsWith('Check / commits'))!.html_url +
'?pr=' +
pull_number
async function extract({
sha,
commit,
}: CommitDetails): Promise<ExtractedCommit> {
const noCherryPick = Array.from(
commit.message.matchAll(/^Not-cherry-picked-because: (.*)$/gm),
).at(0)
if (noCherryPick)
return {
sha,
commit,
severity: 'important',
message: `${sha} is not a cherry-pick, because: ${noCherryPick[1]}. Please review this commit manually.`,
type: 'no-cherry-pick',
}
// Using the last line with "cherry" + hash, because a chained backport
// can result in multiple of those lines. Only the last one counts.
const cherry = Array.from(
commit.message.matchAll(/cherry.*([0-9a-f]{40})/g),
).at(-1)
if (!cherry)
return {
sha,
commit,
severity: 'warning',
message: `Couldn't locate the cherry-picked commit's hash in the commit message of ${sha}.`,
type: 'no-commit-hash',
}
const original_sha = cherry[1]
let branches: string[] | undefined
try {
branches = (
await github.request<CommitBranches>({
// This is an undocumented endpoint to fetch the branches a commit is part of.
// There is no equivalent in neither the REST nor the GraphQL API.
// The endpoint itself is unlikely to go away, because GitHub uses it to display
// the list of branches on the detail page of a commit.
url: `https://github.com/${context.repo.owner}/${context.repo.repo}/branch_commits/${original_sha}`,
headers: {
accept: 'application/json',
},
})
).data.branches
.map(({ branch }) => branch)
.filter((branch) => classify(branch).type.includes('development'))
} catch (e) {
// For some unknown reason a 404 error comes back as 500 without any more details in a GitHub Actions runner.
// Ignore these to return a regular error message below.
if (!isIgnoredBranchError(e)) throw e
}
if (!branches?.length)
return {
sha,
commit,
severity: 'error',
message: `${original_sha} given in ${sha} not found in any pickable branch.`,
}
return {
sha,
commit,
original_sha,
}
}
function diff({ sha, commit, original_sha }: CherryPick): CheckResult {
const diff = execFileSync('git', [
'-C',
dirname,
'range-diff',
'--no-color',
'--ignore-all-space',
'--no-notes',
// 100 means "any change will be reported"; 0 means "no change will be reported"
'--creation-factor=100',
`${original_sha}~..${original_sha}`,
`${sha}~..${sha}`,
])
.toString()
.split('\n')
// First line contains commit SHAs, which we'll print separately.
.slice(1)
// # The output of `git range-diff` is indented with 4 spaces, but we'll control indentation manually.
.map((line) => line.replace(/^ {4}/, ''))
if (!diff.some((line) => line.match(/^[+-]{2}/)))
return {
sha,
commit,
severity: 'info',
message: `✔ ${original_sha} is highly similar to ${sha}.`,
}
const colored_diff = execFileSync('git', [
'-C',
dirname,
'range-diff',
'--color',
'--no-notes',
'--creation-factor=100',
`${original_sha}~..${original_sha}`,
`${sha}~..${sha}`,
]).toString()
return {
sha,
commit,
diff,
colored_diff,
severity: 'warning',
message: `Difference between ${sha} and original ${original_sha} may warrant inspection.`,
type: 'diff',
}
}
// For now we short-circuit the list of commits when cherryPicks should not be checked.
// This will not run any checks, but still trigger the "dismiss reviews" part below.
const commits = !cherryPicks
? []
: await github.paginate(github.rest.pulls.listCommits, {
...context.repo,
pull_number,
})
const extracted = await Promise.all(commits.map(extract))
const fetch = extracted
.filter(isCherryPick)
.flatMap(({ sha, original_sha }) => [sha, original_sha])
if (fetch.length > 0) {
// Fetching all commits we need for diff at once is much faster than any other method.
execFileSync('git', [
'-C',
dirname,
'fetch',
'--depth=2',
'origin',
...fetch,
])
}
const results = extracted.map((result) =>
result.severity ? result : diff(result),
)
// Log all results without truncation, with better highlighting and all whitespace changes to the job log.
results.forEach(({ sha, commit, severity, message, colored_diff }) => {
core.startGroup(`Commit ${sha}`)
core.info(`Author: ${commit.author!.name} ${commit.author!.email}`)
core.info(`Date: ${new Date(commit.author!.date!)}`)
switch (severity) {
case 'error':
core.error(message)
break
case 'warning':
core.warning(message)
break
default:
core.info(message)
}
core.endGroup()
if (colored_diff) core.info(colored_diff)
})
// Only create step summary below in case of warnings or errors.
// Also clean up older reviews, when all checks are good now.
// An empty results array will always trigger this condition, which is helpful
// to clean up reviews created by the prepare step when on the wrong branch.
if (results.every(({ severity }) => severity === 'info')) {
await dismissReviews({ github, context, core, dry, reviewKey })
return
}
// In the case of "error" severity, we also fail the job.
// Those should be considered blocking and not be dismissable via review.
if (results.some(({ severity }) => severity === 'error'))
process.exitCode = 1
core.summary.addRaw(
'This report is automatically generated by the `PR / Check / cherry-pick` CI workflow.',
true,
)
core.summary.addEOL()
core.summary.addRaw(
"Some of the commits in this PR require the author's and reviewer's attention.",
true,
)
core.summary.addEOL()
if (results.some(({ type }) => type === 'no-commit-hash')) {
core.summary.addRaw(
'Please follow the [backporting guidelines](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#how-to-backport-pull-requests) and cherry-pick with the `-x` flag.',
true,
)
core.summary.addRaw(
'This requires changes to the unstable `master` and `staging` branches first, before backporting them.',
true,
)
core.summary.addEOL()
core.summary.addRaw(
'Occasionally, commits are not cherry-picked at all, for example when updating minor versions of packages which have already advanced to the next major on unstable.',
true,
)
core.summary.addRaw(
'These commits can optionally be marked with a `Not-cherry-picked-because: <reason>` footer.',
true,
)
core.summary.addEOL()
}
if (results.some(({ type }) => type === 'diff')) {
core.summary.addRaw(
'Sometimes it is not possible to cherry-pick exactly the same patch.',
true,
)
core.summary.addRaw(
'This most frequently happens when resolving merge conflicts.',
true,
)
core.summary.addRaw(
'The range-diff will help to review the resolution of conflicts.',
true,
)
core.summary.addEOL()
}
core.summary.addRaw(
'If you need to merge this PR despite the warnings, please [dismiss](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/dismissing-a-pull-request-review) this review shortly before merging.',
true,
)
results.forEach(({ severity, message, diff }) => {
if (severity === 'info') return
// The docs for markdown alerts only show examples with markdown blockquote syntax, like this:
// > [!WARNING]
// > message
// However, our testing shows that this also works with a `<blockquote>` html tag, as long as there
// is an empty line:
// <blockquote>
//
// [!WARNING]
// message
// </blockquote>
// Whether this is intended or just an implementation detail is unclear.
core.summary.addRaw('<blockquote>')
core.summary.addRaw(
`\n\n[!${{ important: 'IMPORTANT', warning: 'WARNING', error: 'CAUTION' }[severity]}]`,
true,
)
core.summary.addRaw(`${message}`, true)
if (diff) {
// Limit the output to 10k bytes and remove the last, potentially incomplete line, because GitHub
// comments are limited in length. The value of 10k is arbitrary with the assumption, that after
// the range-diff becomes a certain size, a reviewer is better off reviewing the regular diff in
// GitHub's UI anyway, thus treating the commit as "new" and not cherry-picked.
// Note: if multiple commits are close to the limit, this approach could still lead to a comment
// that's too long. We think this is unlikely to happen, and so don't deal with it explicitly.
const truncated = []
let total_length = 0
for (const line of diff) {
total_length += line.length
if (total_length > 10000) {
truncated.push('', '[...truncated...]')
break
} else {
truncated.push(line)
}
}
core.summary.addRaw('<details><summary>Show diff</summary>')
core.summary.addRaw('\n\n``````````diff', true)
core.summary.addRaw(truncated.join('\n'), true)
core.summary.addRaw('``````````', true)
core.summary.addRaw('</details>')
}
core.summary.addRaw('</blockquote>')
})
if (job_url)
core.summary.addRaw(
`\n\n_Hint: The full diffs are also available in the [runner logs](${job_url}) with slightly better highlighting._`,
)
const body = core.summary.stringify()
core.summary.write()
// Posting a review could fail for very long comments. This can only happen with
// multiple commits all hitting the truncation limit for the diff. If you ever hit
// this case, consider just splitting up those commits into multiple PRs.
await postReview({
github,
context,
core,
dry,
body,
event: 'REQUEST_CHANGES',
reviewKey,
})
})
}

View File

@@ -0,0 +1,117 @@
// @ts-check
const { promisify } = require('node:util')
const execFile = promisify(require('node:child_process').execFile)
/**
* @typedef {{
* subject: string,
* sha: string,
* author: { name: string, email: string },
* committer: { name: string, email: string}
* changedPaths: string[],
* changedPathSegments: Set<string>,
* }} Commit
*/
/**
* @param {{
* args: string[]
* core: import('@actions/core'),
* quiet?: boolean,
* repoPath?: string,
* }} RunGitProps
*/
async function runGit({ args, repoPath, core, quiet }) {
if (repoPath) {
args = ['-C', repoPath, ...args]
}
if (!quiet) {
core.info(`About to run \`git ${args.map((s) => `'${s}'`).join(' ')}\``)
}
return await execFile('git', args)
}
/**
* Gets the SHA, subject and changed files for each commit in the given PR.
*
* Don't use GitHub API at all: the "list commits on PR" endpoint has a limit
* of 250 commits and doesn't return the changed files.
*
* @param {{
* core: import('@actions/core'),
* pr: Awaited<ReturnType<InstanceType<import('@actions/github/lib/utils').GitHub>["rest"]["pulls"]["get"]>>["data"]
* repoPath?: string,
* }} GetCommitMessagesForPRProps
*
* @returns {Promise<Commit[]>}
*/
async function getCommitDetailsForPR({ core, pr, repoPath }) {
await runGit({
args: ['fetch', `--depth=1`, 'origin', pr.base.sha],
repoPath,
core,
})
await runGit({
args: ['fetch', `--depth=${pr.commits + 1}`, 'origin', pr.head.sha],
repoPath,
core,
})
const shas = (
await runGit({
args: [
'rev-list',
`--max-count=${pr.commits}`,
`${pr.base.sha}..${pr.head.sha}`,
],
repoPath,
core,
})
).stdout
.split('\n')
.map((s) => s.trim())
.filter(Boolean)
return Promise.all(
shas.map(async (sha) => {
// Subject, author name, author email, committer name, committer email (all tab-seperated)
// then a blank line, then filenames.
const result = (
await runGit({
args: [
'log',
'--format=%s\t%aN\t%aE\t%cN\t%cE',
'--name-only',
'-1',
sha,
],
repoPath,
core,
quiet: true,
})
).stdout.split('\n')
const [subject, authorName, authorEmail, committerName, committerEmail] =
result[0].split('\t')
const changedPaths = result.slice(2, -1)
const changedPathSegments = new Set(
changedPaths.flatMap((path) => path.split('/')),
)
return {
sha,
subject,
author: { name: authorName, email: authorEmail },
committer: { name: committerName, email: committerEmail },
changedPaths,
changedPathSegments,
}
}),
)
}
module.exports = { getCommitDetailsForPR }

View File

@@ -1,121 +0,0 @@
import { execFile as nodeExecFile } from 'node:child_process'
import { promisify } from 'node:util'
const execFile = promisify(nodeExecFile)
export type Commit = {
subject: string
sha: string
author: { name: string; email: string }
committer: { name: string; email: string }
changedPaths: string[]
changedPathSegments: Set<string>
}
interface RunGitProps {
args: string[]
core: typeof import('@actions/core')
quiet?: boolean
repoPath?: string
}
async function runGit({ args, repoPath, core, quiet }: RunGitProps) {
if (repoPath) {
args = ['-C', repoPath, ...args]
}
if (!quiet) {
core.info(`About to run \`git ${args.map((s) => `'${s}'`).join(' ')}\``)
}
return await execFile('git', args)
}
interface GetCommitMessagesForPRProps {
core: typeof import('@actions/core')
pr: Awaited<
ReturnType<
InstanceType<
typeof import('@actions/github/lib/utils').GitHub
>['rest']['pulls']['get']
>
>['data']
repoPath?: string
}
/**
* Gets the SHA, subject and changed files for each commit in the given PR.
*
* Don't use GitHub API at all: the "list commits on PR" endpoint has a limit
* of 250 commits and doesn't return the changed files.
*/
export async function getCommitDetailsForPR({
core,
pr,
repoPath,
}: GetCommitMessagesForPRProps): Promise<Commit[]> {
await runGit({
args: ['fetch', `--depth=1`, 'origin', pr.base.sha],
repoPath,
core,
})
await runGit({
args: ['fetch', `--depth=${pr.commits + 1}`, 'origin', pr.head.sha],
repoPath,
core,
})
const shas = (
await runGit({
args: [
'rev-list',
`--max-count=${pr.commits}`,
`${pr.base.sha}..${pr.head.sha}`,
],
repoPath,
core,
})
).stdout
.split('\n')
.map((s) => s.trim())
.filter(Boolean)
return Promise.all(
shas.map(async (sha) => {
// Subject, author name, author email, committer name, committer email (all tab-separated)
// then a blank line, then filenames.
const result = (
await runGit({
args: [
'log',
'--format=%s\t%aN\t%aE\t%cN\t%cE',
'--name-only',
'-1',
sha,
],
repoPath,
core,
quiet: true,
})
).stdout.split('\n')
const [subject, authorName, authorEmail, committerName, committerEmail] =
result[0].split('\t')
const changedPaths = result.slice(2, -1)
const changedPathSegments = new Set(
changedPaths.flatMap((path) => path.split('/')),
)
return {
sha,
subject,
author: { name: authorName, email: authorEmail },
committer: { name: committerName, email: committerEmail },
changedPaths,
changedPathSegments,
}
}),
)
}

View File

@@ -1,14 +1,13 @@
// @ts-nocheck
import { writeFileSync } from 'node:fs'
import withRateLimit from './withRateLimit.js'
const excludeTeams = [
/^voters.*$/,
/^nixpkgs-maintainers$/,
/^nixpkgs-committers$/,
]
export default async ({ github, context, core, outFile }) => {
module.exports = async ({ github, context, core, outFile }) => {
const withRateLimit = require('./withRateLimit.js')
const { writeFileSync } = require('node:fs')
const org = context.repo.owner
const result = {}

View File

@@ -0,0 +1,225 @@
// @ts-check
const { classify } = require('../supportedBranches.js')
const { getCommitDetailsForPR } = require('./get-pr-commit-details.js')
/** @typedef {import('./get-pr-commit-details.js').Commit} Commit */
/**
* @param {{
* github: InstanceType<import('@actions/github/lib/utils').GitHub>,
* context: typeof import('@actions/github').context,
* core: import('@actions/core'),
* repoPath?: string,
* }} LintCommitsProps
*/
async function lintCommits({ github, context, core, repoPath }) {
// This check should only be run when we have the pull_request context.
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.info('This is not a pull request. Skipping checks.')
return
}
const pr = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
const baseBranchType = classify(
pr.base.ref.replace(/^refs\/heads\//, ''),
).type
const headBranchType = classify(
pr.head.ref.replace(/^refs\/heads\//, ''),
).type
if (
baseBranchType.includes('development') &&
headBranchType.includes('development') &&
pr.base.repo.id === pr.head.repo?.id
) {
// This matches, for example, PRs from NixOS:staging-next to NixOS:master, or vice versa.
// Ignore them: we should only care about PRs introducing *new* commits.
// We still want to run on PRs from, e.g., Someone:master to NixOS:master, though.
core.info(
'This PR is from one development branch to another. Skipping checks.',
)
return
}
const commits = await getCommitDetailsForPR({ core, pr, repoPath })
await checkCommitMessages({ commits, core })
await checkCommitMetadata({ commits, core })
}
/**
* @param {{
* commits: Commit[],
* core: import('@actions/core'),
* }} CheckCommitMessagesProps
*/
async function checkCommitMessages({ commits, core }) {
const failures = new Set()
const conventionalCommitTypes = [
'build',
'chore',
'ci',
'doc',
'docs',
'feat',
'feature',
'fix',
'perf',
'refactor',
'services',
'style',
'test',
'update',
]
/**
* @param {string[]} types e.g. ["fix", "feat"]
* @param {string?} sha commit hash
*/
function makeConventionalCommitRegex(types, sha = null) {
core.info(
`${
sha
? `Conventional commit types for ${sha?.slice(0, 16)}`
: 'Default conventional commit types'
}: ${JSON.stringify(types)}`,
)
return new RegExp(`^(${types.join('|')})!?(\\(.*\\))?!?:`)
}
// Optimize for the common case that we don't have path segments with the
// same name as a conventional commit type.
const fullConventionalCommitRegex = makeConventionalCommitRegex(
conventionalCommitTypes,
)
for (const commit of commits) {
const logMsgStart = `Commit ${commit.sha}'s message's subject ("${commit.subject}")`
// If we have a commit `perf: ...`, and we touch a file containing the path
// segment "perf", we don't want to flag this.
const filteredTypes = conventionalCommitTypes.filter(
(type) => !commit.changedPathSegments.has(type),
)
const conventionalCommitRegex =
filteredTypes.length === conventionalCommitTypes.length
? fullConventionalCommitRegex
: makeConventionalCommitRegex(filteredTypes, commit.sha)
if (!commit.subject.includes(': ')) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it does not contain a colon followed by a whitespace. ' +
'There are likely other issues as well.',
)
failures.add(commit.sha)
}
if (commit.subject.endsWith('.')) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it ends in a period. There may be other issues as well.',
)
failures.add(commit.sha)
}
const fixups = ['amend!', 'fixup!', 'squash!']
if (fixups.some((s) => commit.subject.startsWith(s))) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
`it begins with "${fixups.find((s) => commit.subject.startsWith(s))}". ` +
'Did you forget to run `git rebase -i --autosquash`?',
)
failures.add(commit.sha)
}
if (conventionalCommitRegex.test(commit.subject)) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it seems to use conventional commit (conventionalcommits.org) ' +
'formatting. Nixpkgs has its own, different, commit message ' +
'formatting standards.',
)
failures.add(commit.sha)
}
if (!failures.has(commit.sha)) {
core.info(`${logMsgStart} passed our automated checks!`)
}
}
if (failures.size !== 0) {
core.error(
'Please review the guidelines at ' +
'<https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#commit-conventions>, ' +
'as well as the applicable area-specific guidelines linked there.',
)
core.setFailed('Committers: merging is discouraged.')
}
}
/**
* @param {{
* commits: Commit[],
* core: import('@actions/core'),
* }} CheckGitFieldsProps
*/
async function checkCommitMetadata({ commits, core }) {
const failures = new Set()
/** @type {(s: string) => boolean} */
const isEmail = (s) => /^.+@.*$/.test(s)
for (const commit of commits) {
if (!commit.author.name) {
core.error(`Commit ${commit.sha} author's name field is missing`)
failures.add(commit.sha)
}
if (!commit.author.email || !isEmail(commit.author.email)) {
core.error(
`Commit ${commit.sha} author's email field is missing or invalid`,
)
failures.add(commit.sha)
}
if (!commit.committer.name) {
core.error(`Commit ${commit.sha} committer's name field is missing`)
failures.add(commit.sha)
}
if (!commit.committer.email || !isEmail(commit.committer.email)) {
core.error(
`Commit ${commit.sha} committer's email field is missing or invalid`,
)
failures.add(commit.sha)
}
if (!failures.has(commit.sha)) {
core.info(
`Commit ${commit.sha}'s git fields passed our automated checks!`,
)
}
}
if (failures.size !== 0) {
core.error(
'Please add the missing commit fields. ' +
'You can use the noreply email address generated for you by GitHub ' +
'(https://docs.github.com/en/account-and-profile/reference/email-addresses-reference#your-noreply-email-address) ' +
"if you'd like.",
)
core.setFailed('Committers: merging is discouraged.')
}
}
module.exports = lintCommits

View File

@@ -1,231 +0,0 @@
import { type Commit, getCommitDetailsForPR } from './get-pr-commit-details.ts'
import { classify } from './supportedBranches.ts'
type GitHub = InstanceType<typeof import('@actions/github/lib/utils').GitHub>
type Context = typeof import('@actions/github').context
type Core = typeof import('@actions/core')
interface LintCommitsProps {
github: GitHub
context: Context
core: Core
repoPath?: string
}
export default async function lintCommits({
github,
context,
core,
repoPath,
}: LintCommitsProps) {
// This check should only be run when we have the pull_request context.
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.info('This is not a pull request. Skipping checks.')
return
}
const pr = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
const baseBranchType = classify(
pr.base.ref.replace(/^refs\/heads\//, ''),
).type
const headBranchType = classify(
pr.head.ref.replace(/^refs\/heads\//, ''),
).type
if (
baseBranchType.includes('development') &&
headBranchType.includes('development') &&
pr.base.repo.id === pr.head.repo?.id
) {
// This matches, for example, PRs from NixOS:staging-next to NixOS:master, or vice versa.
// Ignore them: we should only care about PRs introducing *new* commits.
// We still want to run on PRs from, e.g., Someone:master to NixOS:master, though.
core.info(
'This PR is from one development branch to another. Skipping checks.',
)
return
}
const commits = await getCommitDetailsForPR({ core, pr, repoPath })
await checkCommitMessages({ commits, core })
await checkCommitMetadata({ commits, core })
}
interface CheckCommitMessagesProps {
commits: Commit[]
core: Core
}
async function checkCommitMessages({
commits,
core,
}: CheckCommitMessagesProps) {
const failures = new Set()
const conventionalCommitTypes = [
'build',
'chore',
'ci',
'doc',
'docs',
'feat',
'feature',
'fix',
'perf',
'refactor',
'services',
'style',
'test',
'update',
]
/**
* @param types e.g. ["fix", "feat"]
* @param sha commit hash
*/
function makeConventionalCommitRegex(
types: string[],
sha: string | null = null,
) {
core.info(
`${
sha
? `Conventional commit types for ${sha?.slice(0, 16)}`
: 'Default conventional commit types'
}: ${JSON.stringify(types)}`,
)
return new RegExp(`^(${types.join('|')})!?(\\(.*\\))?!?:`)
}
// Optimize for the common case that we don't have path segments with the
// same name as a conventional commit type.
const fullConventionalCommitRegex = makeConventionalCommitRegex(
conventionalCommitTypes,
)
for (const commit of commits) {
const logMsgStart = `Commit ${commit.sha}'s message's subject ("${commit.subject}")`
// If we have a commit `perf: ...`, and we touch a file containing the path
// segment "perf", we don't want to flag this.
const filteredTypes = conventionalCommitTypes.filter(
(type) => !commit.changedPathSegments.has(type),
)
const conventionalCommitRegex =
filteredTypes.length === conventionalCommitTypes.length
? fullConventionalCommitRegex
: makeConventionalCommitRegex(filteredTypes, commit.sha)
if (!commit.subject.includes(': ')) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it does not contain a colon followed by a whitespace. ' +
'There are likely other issues as well.',
)
failures.add(commit.sha)
}
if (commit.subject.endsWith('.')) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it ends in a period. There may be other issues as well.',
)
failures.add(commit.sha)
}
const fixups = ['amend!', 'fixup!', 'squash!']
if (fixups.some((s) => commit.subject.startsWith(s))) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
`it begins with "${fixups.find((s) => commit.subject.startsWith(s))}". ` +
'Did you forget to run `git rebase -i --autosquash`?',
)
failures.add(commit.sha)
}
if (conventionalCommitRegex.test(commit.subject)) {
core.error(
`${logMsgStart} was detected as not meeting our guidelines because ` +
'it seems to use conventional commit (conventionalcommits.org) ' +
'formatting. Nixpkgs has its own, different, commit message ' +
'formatting standards.',
)
failures.add(commit.sha)
}
if (!failures.has(commit.sha)) {
core.info(`${logMsgStart} passed our automated checks!`)
}
}
if (failures.size !== 0) {
core.error(
'Please review the guidelines at ' +
'<https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md#commit-conventions>, ' +
'as well as the applicable area-specific guidelines linked there.',
)
core.setFailed('Committers: merging is discouraged.')
}
}
interface CheckGitFieldsProps {
commits: Commit[]
core: Core
}
async function checkCommitMetadata({ commits, core }: CheckGitFieldsProps) {
const failures = new Set()
const isEmail = (s: string) => /^.+@.*$/.test(s)
for (const commit of commits) {
if (!commit.author.name) {
core.error(`Commit ${commit.sha} author's name field is missing`)
failures.add(commit.sha)
}
if (!commit.author.email || !isEmail(commit.author.email)) {
core.error(
`Commit ${commit.sha} author's email field is missing or invalid`,
)
failures.add(commit.sha)
}
if (!commit.committer.name) {
core.error(`Commit ${commit.sha} committer's name field is missing`)
failures.add(commit.sha)
}
if (!commit.committer.email || !isEmail(commit.committer.email)) {
core.error(
`Commit ${commit.sha} committer's email field is missing or invalid`,
)
failures.add(commit.sha)
}
if (!failures.has(commit.sha)) {
core.info(
`Commit ${commit.sha}'s git fields passed our automated checks!`,
)
}
}
if (failures.size !== 0) {
core.error(
'Please add the missing commit fields. ' +
'You can use the noreply email address generated for you by GitHub ' +
'(https://docs.github.com/en/account-and-profile/reference/email-addresses-reference#your-noreply-email-address) ' +
"if you'd like.",
)
core.setFailed('Committers: merging is discouraged.')
}
}

View File

@@ -0,0 +1,95 @@
// @ts-check
const { classify } = require('../supportedBranches.js')
const { getCommitDetailsForPR } = require('./get-pr-commit-details')
/**
* @param {{
* github: InstanceType<import('@actions/github/lib/utils').GitHub>,
* context: import('@actions/github/lib/context').Context,
* core: import('@actions/core'),
* repoPath?: string,
* dry: boolean,
* }} CheckManualFileEditsProps
*/
async function checkManualFileEdits({ github, context, core, repoPath, dry }) {
const { dismissReviews, postReview } = require('./reviews.js')
const reviewKey = 'manual-file-edits'
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.info('This is not a pull request. Skipping checks.')
return
}
const pr = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
if (pr.user.login.endsWith('[bot]')) {
core.info('This is a bot, so these checks do not apply.')
return
}
const baseBranchType = classify(
pr.base.ref.replace(/^refs\/heads\//, ''),
).type
const headBranchType = classify(
pr.head.ref.replace(/^refs\/heads\//, ''),
).type
if (
baseBranchType.includes('development') &&
headBranchType.includes('development') &&
pr.base.repo.id === pr.head.repo?.id
) {
// This matches, for example, PRs from NixOS:staging-next to NixOS:master, or vice versa.
// Ignore them: we should only care about PRs introducing *new* commits.
// We still want to run on PRs from, e.g., Someone:master to NixOS:master, though.
core.info(
'This PR is from one development branch to another. Skipping checks.',
)
return
}
const details = await getCommitDetailsForPR({ core, pr, repoPath })
if (
details.some(({ changedPaths }) =>
changedPaths.includes('maintainers/github-teams.json'),
)
) {
postReview({
github,
context,
core,
dry,
event: 'REQUEST_CHANGES',
body: [
'maintainers/github-teams.json is supposed to accurately reflect the state of the teams in GitHub.\n',
'Therefore, it should not be edited manually.\n',
'All changes to teams listed in maintainers/github-teams.json should be performed in GitHub by a team maintainer.\n',
"Team maintainers are listed in the github-teams.json file and in GitHub's UI.\n",
'If there is no team maintainer available, an org owner can make the needed change, please contact one by',
'following the instructions at https://github.com/NixOS/org/blob/main/doc/github-org-owners.md#how-to-contact-the-team.\n',
'Thank you!',
].reduce(
(prev, curr) => prev + (!prev || prev.endsWith('\n') ? '' : ' ') + curr,
'',
),
reviewKey,
})
} else {
dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
}
}
module.exports = checkManualFileEdits

View File

@@ -1,95 +0,0 @@
import { getCommitDetailsForPR } from './get-pr-commit-details.ts'
import { dismissReviews, postReview } from './reviews.ts'
import { classify } from './supportedBranches.ts'
export default async function checkManualFileEdits({
github,
context,
core,
repoPath,
dry,
}: {
github: InstanceType<typeof import('@actions/github/lib/utils').GitHub>
context: typeof import('@actions/github').context
core: typeof import('@actions/core')
repoPath?: string
dry: boolean
}) {
const reviewKey = 'manual-file-edits'
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.info('This is not a pull request. Skipping checks.')
return
}
const pr = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
if (pr.user.login.endsWith('[bot]')) {
core.info('This is a bot, so these checks do not apply.')
return
}
const baseBranchType = classify(
pr.base.ref.replace(/^refs\/heads\//, ''),
).type
const headBranchType = classify(
pr.head.ref.replace(/^refs\/heads\//, ''),
).type
if (
baseBranchType.includes('development') &&
headBranchType.includes('development') &&
pr.base.repo.id === pr.head.repo?.id
) {
// This matches, for example, PRs from NixOS:staging-next to NixOS:master, or vice versa.
// Ignore them: we should only care about PRs introducing *new* commits.
// We still want to run on PRs from, e.g., Someone:master to NixOS:master, though.
core.info(
'This PR is from one development branch to another. Skipping checks.',
)
return
}
const details = await getCommitDetailsForPR({ core, pr, repoPath })
if (
details.some(({ changedPaths }) =>
changedPaths.includes('maintainers/github-teams.json'),
)
) {
postReview({
github,
context,
core,
dry,
event: 'REQUEST_CHANGES',
body: [
'maintainers/github-teams.json is supposed to accurately reflect the state of the teams in GitHub.\n',
'Therefore, it should not be edited manually.\n',
'All changes to teams listed in maintainers/github-teams.json should be performed in GitHub by a team maintainer.\n',
"Team maintainers are listed in the github-teams.json file and in GitHub's UI.\n",
'If there is no team maintainer available, an org owner can make the needed change, please contact one by',
'following the instructions at https://github.com/NixOS/org/blob/main/doc/github-org-owners.md#how-to-contact-the-team.\n',
'Thank you!',
].reduce(
(prev, curr) => prev + (!prev || prev.endsWith('\n') ? '' : ' ') + curr,
'',
),
reviewKey,
})
} else {
dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
}
}

View File

@@ -1,5 +1,4 @@
// @ts-nocheck
import { classify } from './supportedBranches.ts'
const { classify } = require('../supportedBranches.js')
function runChecklist({
committers,
@@ -123,7 +122,7 @@ function hasMergeCommand(body) {
.match(/^@NixOS\/nixpkgs-merge-bot merge\s*$/im)
}
export async function handleMergeComment({ github, body, node_id, reaction }) {
async function handleMergeComment({ github, body, node_id, reaction }) {
if (!hasMergeCommand(body)) return
await github.graphql(
@@ -138,7 +137,7 @@ export async function handleMergeComment({ github, body, node_id, reaction }) {
)
}
export async function handleMerge({
async function handleMerge({
github,
context,
core,
@@ -409,3 +408,8 @@ export async function handleMerge({
// This is used to set the respective label in bot.js.
return result
}
module.exports = {
handleMerge,
handleMergeComment,
}

View File

@@ -10,10 +10,6 @@
"@actions/github": "9.1.0",
"bottleneck": "2.19.5",
"commander": "14.0.3"
},
"devDependencies": {
"@types/node": "24.13.3",
"typescript": "7.0.2"
}
},
"node_modules/@actions/artifact": {
@@ -592,19 +588,6 @@
"protoc-gen-ts": "bin/protoc-gen-ts"
}
},
"node_modules/@protobuf-ts/plugin/node_modules/typescript": {
"version": "3.9.10",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.9.10.tgz",
"integrity": "sha512-w6fIxVE/H1PkLKcCPsFqKE7Kv7QUwhU8qQY2MueZXWx5cPZdwFupLgKK3vntcK98BtNHZtAF4LA/yl2a7k8R6Q==",
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=4.2.0"
}
},
"node_modules/@protobuf-ts/protoc": {
"version": "2.11.1",
"resolved": "https://registry.npmjs.org/@protobuf-ts/protoc/-/protoc-2.11.1.tgz",
@@ -629,336 +612,6 @@
"@protobuf-ts/runtime": "^2.11.1"
}
},
"node_modules/@types/node": {
"version": "24.13.3",
"resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.3.tgz",
"integrity": "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q==",
"dev": true,
"license": "MIT",
"dependencies": {
"undici-types": "~7.18.0"
}
},
"node_modules/@typescript/typescript-aix-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-aix-ppc64/-/typescript-aix-ppc64-7.0.2.tgz",
"integrity": "sha512-MTKKkWB7p/0E9xi1d1tHtZ5PiLkGEMIq88pK2CubZjOsLtYTLqhgIgi6zepFa+9GHZ6h05NMCkQxGKiPXMxXtQ==",
"cpu": [
"ppc64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"aix"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-arm64/-/typescript-darwin-arm64-7.0.2.tgz",
"integrity": "sha512-gowzar9MwS/aRWp6f3a4KUqzRjAZjOsmGNCM6LcTgXum+dBfgsBVMN+AgvOCCbguXyick6LJhpBszxMebJ8syA==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-darwin-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-darwin-x64/-/typescript-darwin-x64-7.0.2.tgz",
"integrity": "sha512-SZ9xZInqApNlNGc9s0W1VSsktYSOe9cFqNOIqmN1Gs8SmkjKZYFt017G4VwPxASInODuAdbTW7sXiFUf893RgA==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"darwin"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-arm64/-/typescript-freebsd-arm64-7.0.2.tgz",
"integrity": "sha512-W5NH4y/J0plIIS5b2xvTEkU7JFxyqdMAOgf+Ilhl0vHQXKO5dZoxd+C/jEtq56c4F3wk71RB4BMRQ2XdI+bwYQ==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-freebsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-freebsd-x64/-/typescript-freebsd-x64-7.0.2.tgz",
"integrity": "sha512-UMGDx5sTpzNw3WiPebH7l90IWfJggEd+egHt/q6p7/Cm3zqoV7VxkGXt+3DxPIw8CcmvAB0j3sVVfbhX+M4Tpw==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"freebsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm/-/typescript-linux-arm-7.0.2.tgz",
"integrity": "sha512-gffT3xPz9sR7j/YJExkyPntrI0P2EP9XbOyWzth2/Gs0RstK+90RBcO0ncXoXy/beYll1SXw846Nf2zdnEz0QQ==",
"cpu": [
"arm"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-arm64/-/typescript-linux-arm64-7.0.2.tgz",
"integrity": "sha512-Qh4eU4/y3yDjnfjjyPYihMj5/ODIlmt+Bzu17OI+fiSRDW57QmU5SiN63exPRNJPKUzcc1INa1NXdrJ+MqHjUQ==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-loong64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-loong64/-/typescript-linux-loong64-7.0.2.tgz",
"integrity": "sha512-uEHck9i8hoAzXPiYRib1O7miOnz23SxIeVl6F4LXox+qov1K35jHcEW6VHKvZI+pyvl7fZEP4MCU5LYvIq1GuQ==",
"cpu": [
"loong64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-mips64el": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-mips64el/-/typescript-linux-mips64el-7.0.2.tgz",
"integrity": "sha512-R4KvAMnE43W5Qeqb0Ly56O3mWMWIAgsMyz36DCaycd5nbg/9kzm0liw3JocfRqyJY0KPmzFjbswozXyW0DnIYA==",
"cpu": [
"mips64el"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-ppc64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-ppc64/-/typescript-linux-ppc64-7.0.2.tgz",
"integrity": "sha512-DORx5b3sd/4S7eayxm4FQv+A7CrkUIGRaHiwI8oiHTAI1fAPWhF4J0vAlkC8biAlHSVVwxMQ3tjZ2/DVbnQiiA==",
"cpu": [
"ppc64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-riscv64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-riscv64/-/typescript-linux-riscv64-7.0.2.tgz",
"integrity": "sha512-wf0jqEDOjrPRnKwYRyyJDRo11KMbvMFrU+q4zqKyChODBzvlkbhNQfKvLxQCcwTpdDaXSHZTVuh0JoCrKCUMHQ==",
"cpu": [
"riscv64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-s390x": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-s390x/-/typescript-linux-s390x-7.0.2.tgz",
"integrity": "sha512-IkwJc3L7yhytWd/ewjyxNDfOmswCm9GWMJT/ue/dU4aZNbwZeYAetq42VyLmsmSjvoX7z74X6ZaYCtzAr0EuGw==",
"cpu": [
"s390x"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-linux-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-linux-x64/-/typescript-linux-x64-7.0.2.tgz",
"integrity": "sha512-EYdf2cNg7rgCWJnxCdJ+F3V39O8ihb37eHAu1LK8oAFizgTQbPOK7zHHXbPt8rX24COqODXeI3sIf0fCXG7H/A==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"linux"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-arm64/-/typescript-netbsd-arm64-7.0.2.tgz",
"integrity": "sha512-+polYF4MF04aPpO5FTkHran9yUQDSXqy5GiSDKpsll5jy3l3+g9QLhpf39T+ePtefhXLOGrLl0QIjkQP6VnelA==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-netbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-netbsd-x64/-/typescript-netbsd-x64-7.0.2.tgz",
"integrity": "sha512-8YIT0EHM/3dq10ZOVF/A7pc/YSMtbcecct4rWtexrnSCHOPcpC2KTLXfTCR6vDpnSiY12heNb1GiN/wu+T/FyA==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"netbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-arm64/-/typescript-openbsd-arm64-7.0.2.tgz",
"integrity": "sha512-APT8+ClYnuYm1u9+kgGXoMj2VzWzcymwh2gNSQVySHfkRDGOTVkoWLjCmOQSaO+PoqQ57B0flRp9SA+7GnnkzQ==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-openbsd-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-openbsd-x64/-/typescript-openbsd-x64-7.0.2.tgz",
"integrity": "sha512-yX7s+Q0Dln0Dt9tEzZsAjXXR/+ytBM7AlglaqyeMPxQszJ1JhlJdZ6jLA+IzldHtflX81em7lDao1xXu+aRRkg==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"openbsd"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-sunos-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-sunos-x64/-/typescript-sunos-x64-7.0.2.tgz",
"integrity": "sha512-dLJDGaLZ1D4HPQn62u1n8mBDkJREwMsAkCdkwd4Ieqw+x3TUyTsqY0YiBCtE6H6OzzgGk3iuZ3vFWRS+E8/d1g==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"sunos"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-arm64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-arm64/-/typescript-win32-arm64-7.0.2.tgz",
"integrity": "sha512-Gyl1Vy6OsWesLzmq+EP0Fb7b4Nid5232AvcA2SFcdYreldpNtYFFofPjnt62y9hQy7VTaZp65ICJjuAQRaVcIQ==",
"cpu": [
"arm64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/typescript-win32-x64": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/@typescript/typescript-win32-x64/-/typescript-win32-x64-7.0.2.tgz",
"integrity": "sha512-0BQ3HkAHHlKLSp1qRvf3SUhGpGsDuhB/jgFw75guyqbxJqEaS0Cw/VFO8i2nHglJUzQCRtMMR/IBAKE3ETMC4g==",
"cpu": [
"x64"
],
"license": "Apache-2.0",
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">=16.20.0"
}
},
"node_modules/@typescript/vfs": {
"version": "1.6.1",
"resolved": "https://registry.npmjs.org/@typescript/vfs/-/vfs-1.6.1.tgz",
@@ -2004,37 +1657,16 @@
}
},
"node_modules/typescript": {
"version": "7.0.2",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-7.0.2.tgz",
"integrity": "sha512-8FYau96o3NKOhbjKi/qNvG/W5jhzxkbdm5sj9AbZ/5T5sWqn3hJgLfGx27sRKZWTvyzCP8dLRBTf5tBTSRVUNA==",
"version": "3.9.10",
"resolved": "https://registry.npmjs.org/typescript/-/typescript-3.9.10.tgz",
"integrity": "sha512-w6fIxVE/H1PkLKcCPsFqKE7Kv7QUwhU8qQY2MueZXWx5cPZdwFupLgKK3vntcK98BtNHZtAF4LA/yl2a7k8R6Q==",
"license": "Apache-2.0",
"bin": {
"tsc": "bin/tsc"
"tsc": "bin/tsc",
"tsserver": "bin/tsserver"
},
"engines": {
"node": ">=16.20.0"
},
"optionalDependencies": {
"@typescript/typescript-aix-ppc64": "7.0.2",
"@typescript/typescript-darwin-arm64": "7.0.2",
"@typescript/typescript-darwin-x64": "7.0.2",
"@typescript/typescript-freebsd-arm64": "7.0.2",
"@typescript/typescript-freebsd-x64": "7.0.2",
"@typescript/typescript-linux-arm": "7.0.2",
"@typescript/typescript-linux-arm64": "7.0.2",
"@typescript/typescript-linux-loong64": "7.0.2",
"@typescript/typescript-linux-mips64el": "7.0.2",
"@typescript/typescript-linux-ppc64": "7.0.2",
"@typescript/typescript-linux-riscv64": "7.0.2",
"@typescript/typescript-linux-s390x": "7.0.2",
"@typescript/typescript-linux-x64": "7.0.2",
"@typescript/typescript-netbsd-arm64": "7.0.2",
"@typescript/typescript-netbsd-x64": "7.0.2",
"@typescript/typescript-openbsd-arm64": "7.0.2",
"@typescript/typescript-openbsd-x64": "7.0.2",
"@typescript/typescript-sunos-x64": "7.0.2",
"@typescript/typescript-win32-arm64": "7.0.2",
"@typescript/typescript-win32-x64": "7.0.2"
"node": ">=4.2.0"
}
},
"node_modules/undici": {
@@ -2049,13 +1681,6 @@
"node": ">=14.0"
}
},
"node_modules/undici-types": {
"version": "7.18.2",
"resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz",
"integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==",
"dev": true,
"license": "MIT"
},
"node_modules/universal-user-agent": {
"version": "7.0.3",
"resolved": "https://registry.npmjs.org/universal-user-agent/-/universal-user-agent-7.0.3.tgz",

View File

@@ -1,11 +1,5 @@
{
"private": true,
"scripts": {
"typecheck": "tsc --build",
"testsuite": "node --test",
"test": "npm run typecheck && npm run testsuite"
},
"type": "module",
"//": [
"Keep `@actions/core` and `@actions/github` in sync with",
"https://github.com/actions/github-script/blob/main/package.json."
@@ -16,9 +10,5 @@
"@actions/github": "9.1.0",
"bottleneck": "2.19.5",
"commander": "14.0.3"
},
"devDependencies": {
"@types/node": "24.13.3",
"typescript": "7.0.2"
}
}

View File

@@ -1,11 +1,9 @@
// @ts-nocheck
import { dismissReviews, postReview } from './reviews.ts'
import { classify } from './supportedBranches.ts'
import supportedSystems from './supportedSystems.ts'
const { classify } = require('../supportedBranches.js')
const { postReview, dismissReviews } = require('./reviews.js')
const reviewKey = 'prepare'
const supportedSystems = require('./supportedSystems.js')
export default async ({ github, context, core, dry }) => {
module.exports = async ({ github, context, core, dry }) => {
const pull_number = context.payload.pull_request.number
for (const retryInterval of [5, 10, 20, 40, 80]) {
@@ -66,7 +64,7 @@ export default async ({ github, context, core, dry }) => {
// commits between that base and head is the real base. We can query for this via GitHub's
// REST API. There can be multiple candidates for the real base with the same number of
// commits. In this case we pick the "best" candidate by a fixed ordering of branches,
// as defined in ci/github-script/supportedBranches.ts.
// as defined in ci/supportedBranches.js.
//
// These requests take a while, when comparing against the wrong release - they need
// to look at way more than 10k commits in that case. Thus, we try to minimize the

View File

@@ -1,111 +0,0 @@
import fs from 'node:fs'
import path from 'node:path'
import type * as actionsCore from '@actions/core'
import type { context as actionsContext } from '@actions/github'
import type { GitHub } from '@actions/github/lib/utils'
import { getCommitDetailsForPR } from './get-pr-commit-details.ts'
import { dismissReviews, postReview } from './reviews.ts'
import { classify } from './supportedBranches.ts'
/**
* Reminders to post as a non-blocking review when a pull request touches
* certain matching paths.
*/
export const reminders: { key: string; paths: RegExp[] }[] = [
{
key: 'docs-styleguide',
paths: [/^doc\//, /^nixos\/doc\//, /^nixos\/modules\/.*\.md$/],
},
]
export function matchesAny(changedPaths: string[], patterns: RegExp[]) {
return changedPaths.some((changedPath) =>
patterns.some((pattern) => pattern.test(changedPath)),
)
}
function reminderBody(key: string) {
return fs
.readFileSync(path.join(import.meta.dirname, 'reminders', `${key}.md`), {
encoding: 'utf8',
})
.trim()
}
export default async function postReminders({
github,
context,
core,
repoPath,
dry,
}: {
github: InstanceType<typeof GitHub>
context: typeof actionsContext
core: typeof actionsCore
repoPath?: string
dry: boolean
}) {
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.info('This is not a pull request. Skipping reminders.')
return
}
const pr = (
await github.rest.pulls.get({
...context.repo,
pull_number,
})
).data
if (pr.user.login.endsWith('[bot]')) {
core.info('This is a bot, so reminders do not apply.')
return
}
const baseBranchType = classify(
pr.base.ref.replace(/^refs\/heads\//, ''),
).type
const headBranchType = classify(
pr.head.ref.replace(/^refs\/heads\//, ''),
).type
if (
baseBranchType.includes('development') &&
headBranchType.includes('development') &&
pr.base.repo.id === pr.head.repo?.id
) {
// This matches, for example, PRs from NixOS:staging-next to NixOS:master, or vice versa.
// We ignore them, we should only care about PRs introducing new commits.
// We still want to run on PRs from, e.g., Someone:master to NixOS:master though.
core.info(
'This PR is from one development branch to another. Skipping reminders.',
)
return
}
const details = await getCommitDetailsForPR({ core, pr, repoPath })
const changedPaths = details.flatMap(({ changedPaths }) => changedPaths)
for (const { key, paths } of reminders) {
if (matchesAny(changedPaths, paths)) {
await postReview({
github,
context,
core,
dry,
event: 'COMMENT',
body: reminderBody(key),
reviewKey: key,
})
} else {
await dismissReviews({
github,
context,
core,
dry,
reviewKey: key,
})
}
}
}

View File

@@ -1,12 +0,0 @@
Thanks for contributing to the documentation
Make sure you follow the [documentation style guide](https://github.com/NixOS/nixpkgs/blob/master/doc/styleguide.md), most notably:
- Show, don't tell: lead with a minimal working example; explanation follows the code.
- No meta-commentary: don't write "This section explains how to…", just do it.
- Imperative mood and active voice: "Run the command", not "The user should run the following command".
- Present tense: "This creates a folder", not "This will create a folder".
- Be confident: no hedging with "should", "might", "typically", "usually".
- Cut filler words: "simply", "just", "easily", "basically"; "to", not "in order to".
For larger changes, like adding or removing whole sections, ask the NixOS documentation team for a review.

View File

@@ -1,5 +1,4 @@
// @ts-nocheck
export async function handleReviewers({
async function handleReviewers({
github,
context,
core,
@@ -183,3 +182,7 @@ export async function handleReviewers({
teams_reached.size === 0
)
}
module.exports = {
handleReviewers,
}

271
ci/github-script/reviews.js Normal file
View File

@@ -0,0 +1,271 @@
// @ts-check
const eventToState = {
COMMENT: 'COMMENTED',
REQUEST_CHANGES: 'CHANGES_REQUESTED',
}
// Use substring checks in order to allow testing in forks
// Usernames must also end in "[bot]"
const reviewUsers = [
'github-actions',
'nixpkgs-ci',
'branch-check',
'commit-check',
'manual-edit',
]
/**
* @typedef {InstanceType<import('@actions/github/lib/utils').GitHub>} GitHub
* @typedef {typeof import('@actions/github').context} Context
*
* @typedef {Awaited<ReturnType<GitHub['rest']['pulls']['listReviews']>>['data'][number]} Review
* @typedef {Review & { user: NonNullable<Review['user']> }} ReviewWithNonNullUser
*/
/**
* @param {{
* github: GitHub,
* context: Context,
* core: import('@actions/core'),
* dry: boolean,
* reviewKey?: string,
* }} DismissReviewsProps
*/
async function dismissReviews({ github, context, core, dry, reviewKey }) {
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning('dismissReviews called outside of pull_request context')
return
}
if (dry) {
return
}
const allReviews = await github.paginate(github.rest.pulls.listReviews, {
...context.repo,
pull_number,
})
const reviews = /** @type {ReviewWithNonNullUser[]} */ (
allReviews.filter(
(review) =>
review.user &&
review.state !== 'DISMISSED' &&
review.user.login.endsWith('[bot]') &&
reviewUsers.some((substr) => review.user?.login.includes(substr)),
)
)
const reviewsByUser = reviews.reduce(
(prev, curr) => {
if (!(curr.user.login in prev)) {
prev[curr.user.login] = []
}
prev[curr.user.login].push(curr)
return prev
},
/** @type {Record<string, ReviewWithNonNullUser[]> } */ ({}),
)
const commentRegex = new RegExp(
/<!-- nixpkgs review key: (.*)(?:; resolved: .*)? -->/,
)
const reviewKeyRegex = new RegExp(
`<!-- (nixpkgs review key: ${reviewKey})(?:; resolved: .*)? -->`,
)
const commentResolvedRegex = new RegExp(
/<!-- nixpkgs review key: .*; resolved: true -->/,
)
let reviewsToMinimize = reviews
const /** @type {ReviewWithNonNullUser[]} */ reviewsToDismiss = []
const /** @type {ReviewWithNonNullUser[]} */ reviewsToResolve = []
if (reviewKey && reviews.every((review) => commentRegex.test(review.body))) {
reviewsToMinimize = reviews.filter((review) =>
reviewKeyRegex.test(review.body),
)
}
for (const reviewsForUser of Object.values(reviewsByUser)) {
// Make sure that we don't dismiss all reviews by a user if they
// have any reviews we don't want to dismiss.
if (
reviewsForUser.every(
(review) =>
commentResolvedRegex.test(review.body) ||
(reviewKey && reviewKeyRegex.test(review.body)) ||
// If we are called by check-commits and the review body is clearly
// from `commits.js`, then we can safely dismiss the review.
// This helps with pre-existing reviews (before the comments were added).
(reviewKey &&
reviewKey === 'check-commits' &&
review.body.includes('PR / Check / cherry-pick')),
)
) {
reviewsToDismiss.push(
...reviewsForUser.filter(
(review) => review.state === 'CHANGES_REQUESTED',
),
)
} else {
reviewsToResolve.push(
...reviewsForUser.filter(
(review) =>
review.state === 'CHANGES_REQUESTED' &&
!commentResolvedRegex.test(review.body) &&
reviewsToMinimize.some(
(toMinimize) => toMinimize.node_id === review.node_id,
),
),
)
}
}
await Promise.all([
...reviewsToMinimize.map(async (review) =>
github.graphql(
`mutation($node_id:ID!) {
minimizeComment(input: {
classifier: OUTDATED,
subjectId: $node_id
})
{ clientMutationId }
}`,
{ node_id: review.node_id },
),
),
...reviewsToDismiss.map(async (review) =>
github.rest.pulls.dismissReview({
...context.repo,
pull_number,
review_id: review.id,
message: 'Review dismissed automatically',
}),
),
...reviewsToResolve.map(async (review) =>
github.rest.pulls.updateReview({
...context.repo,
pull_number,
review_id: review.id,
body: review.body.replace(
reviewKeyRegex,
`<!-- nixpkgs review key: ${reviewKey}; resolved: true -->`,
),
}),
),
])
}
/**
* @param {{
* github: GitHub,
* context: Context,
* core: import('@actions/core'),
* dry: boolean,
* body: string,
* event: keyof eventToState,
* reviewKey: string,
* }} PostReviewProps
*/
async function postReview({
github,
context,
core,
dry,
body,
event = 'REQUEST_CHANGES',
reviewKey,
}) {
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning('postReview called outside of pull_request context')
return
}
const reviewKeyRegex = new RegExp(
`<!-- (nixpkgs review key: ${reviewKey})(?:; resolved: .*)? -->`,
)
const reviewKeyComment = `<!-- nixpkgs review key: ${reviewKey}; resolved: false -->`
body = body + '\n\n' + reviewKeyComment
const reviews = (
await github.paginate(github.rest.pulls.listReviews, {
...context.repo,
pull_number,
})
).filter(
(review) =>
review.user &&
review.state !== 'DISMISSED' &&
review.user.login.endsWith('[bot]') &&
reviewUsers.some((substr) => review.user?.login.includes(substr)),
)
/** @type {null | Review} */
let pendingReview
const matchingReviews = reviews.filter((review) =>
reviewKeyRegex.test(review.body),
)
if (matchingReviews.length === 0) {
pendingReview = null
} else if (
matchingReviews.length === 1 &&
matchingReviews[0].state === eventToState[event]
) {
pendingReview = matchingReviews[0]
} else {
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
pendingReview = null
}
if (dry) {
if (pendingReview)
core.info(`pending review found: ${pendingReview.html_url}`)
else core.info('no pending review found')
core.info(body)
} else {
if (pendingReview) {
await Promise.all([
github.rest.pulls.updateReview({
...context.repo,
pull_number,
review_id: pendingReview.id,
body,
}),
github.graphql(
`mutation($node_id:ID!) {
unminimizeComment(input: {
subjectId: $node_id
})
{ clientMutationId }
}`,
{ node_id: pendingReview.node_id },
),
])
} else {
await github.rest.pulls.createReview({
...context.repo,
pull_number,
event,
body,
})
}
}
}
module.exports = {
dismissReviews,
postReview,
}

View File

@@ -1,272 +0,0 @@
const eventToState = {
COMMENT: 'COMMENTED',
REQUEST_CHANGES: 'CHANGES_REQUESTED',
}
// Use substring checks in order to allow testing in forks
// Usernames must also end in "[bot]"
const reviewUsers = [
'github-actions',
'nixpkgs-ci',
'branch-check',
'commit-check',
'manual-edit',
]
type GitHub = InstanceType<typeof import('@actions/github/lib/utils').GitHub>
type Context = typeof import('@actions/github').context
type Review = Awaited<
ReturnType<GitHub['rest']['pulls']['listReviews']>
>['data'][number]
type ReviewWithNonNullUser = Review & { user: NonNullable<Review['user']> }
interface DismissReviewsProps {
github: GitHub
context: Context
core: typeof import('@actions/core')
dry: boolean
reviewKey?: string
}
export async function dismissReviews({
github,
context,
core,
dry,
reviewKey,
}: DismissReviewsProps) {
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning('dismissReviews called outside of pull_request context')
return
}
if (dry) {
return
}
const allReviews: Review[] = await github.paginate(
github.rest.pulls.listReviews,
{
...context.repo,
pull_number,
},
)
const reviews = allReviews
.filter((review): review is ReviewWithNonNullUser => !!review.user)
.filter(
(review) =>
review.state !== 'DISMISSED' &&
review.user.login.endsWith('[bot]') &&
reviewUsers.some((substr) => review.user?.login.includes(substr)),
)
const reviewsByUser = reviews.reduce(
(prev, curr) => {
if (!curr.user) {
return prev
}
if (!(curr.user.login in prev)) {
prev[curr.user.login] = []
}
prev[curr.user.login].push(curr)
return prev
},
{} as Record<string, ReviewWithNonNullUser[]>,
)
const commentRegex = new RegExp(
/<!-- nixpkgs review key: (.*)(?:; resolved: .*)? -->/,
)
const reviewKeyRegex = new RegExp(
`<!-- (nixpkgs review key: ${reviewKey})(?:; resolved: .*)? -->`,
)
const commentResolvedRegex = new RegExp(
/<!-- nixpkgs review key: .*; resolved: true -->/,
)
let reviewsToMinimize = reviews
const reviewsToDismiss: ReviewWithNonNullUser[] = []
const reviewsToResolve: ReviewWithNonNullUser[] = []
if (reviewKey && reviews.every((review) => commentRegex.test(review.body))) {
reviewsToMinimize = reviews.filter((review) =>
reviewKeyRegex.test(review.body),
)
}
for (const reviewsForUser of Object.values(reviewsByUser)) {
// Make sure that we don't dismiss all reviews by a user if they
// have any reviews we don't want to dismiss.
if (
reviewsForUser.every(
(review) =>
commentResolvedRegex.test(review.body) ||
(reviewKey && reviewKeyRegex.test(review.body)) ||
// If we are called by check-commits and the review body is clearly
// from `commits.ts`, then we can safely dismiss the review.
// This helps with pre-existing reviews (before the comments were added).
(reviewKey &&
reviewKey === 'check-commits' &&
review.body.includes('PR / Check / cherry-pick')),
)
) {
reviewsToDismiss.push(
...reviewsForUser.filter(
(review) => review.state === 'CHANGES_REQUESTED',
),
)
} else {
reviewsToResolve.push(
...reviewsForUser.filter(
(review) =>
review.state === 'CHANGES_REQUESTED' &&
!commentResolvedRegex.test(review.body) &&
reviewsToMinimize.some(
(toMinimize) => toMinimize.node_id === review.node_id,
),
),
)
}
}
await Promise.all([
...reviewsToMinimize.map(async (review) =>
github.graphql(
`mutation($node_id:ID!) {
minimizeComment(input: {
classifier: OUTDATED,
subjectId: $node_id
})
{ clientMutationId }
}`,
{ node_id: review.node_id },
),
),
...reviewsToDismiss.map(async (review) =>
github.rest.pulls.dismissReview({
...context.repo,
pull_number,
review_id: review.id,
message: 'Review dismissed automatically',
}),
),
...reviewsToResolve.map(async (review) =>
github.rest.pulls.updateReview({
...context.repo,
pull_number,
review_id: review.id,
body: review.body.replace(
reviewKeyRegex,
`<!-- nixpkgs review key: ${reviewKey}; resolved: true -->`,
),
}),
),
])
}
interface PostReviewProps {
github: GitHub
context: Context
core: typeof import('@actions/core')
dry: boolean
body: string
event: keyof typeof eventToState
reviewKey: string
}
export async function postReview({
github,
context,
core,
dry,
body,
event = 'REQUEST_CHANGES',
reviewKey,
}: PostReviewProps) {
const pull_number = context.payload.pull_request?.number
if (!pull_number) {
core.warning('postReview called outside of pull_request context')
return
}
const reviewKeyRegex = new RegExp(
`<!-- (nixpkgs review key: ${reviewKey})(?:; resolved: .*)? -->`,
)
const reviewKeyComment = `<!-- nixpkgs review key: ${reviewKey}; resolved: false -->`
body = body + '\n\n' + reviewKeyComment
const reviews = (
await github.paginate(github.rest.pulls.listReviews, {
...context.repo,
pull_number,
})
).filter(
(review) =>
review.user &&
review.state !== 'DISMISSED' &&
review.user.login.endsWith('[bot]') &&
reviewUsers.some((substr) => review.user?.login.includes(substr)),
)
let pendingReview: null | Review
const matchingReviews = reviews.filter((review) =>
reviewKeyRegex.test(review.body),
)
if (matchingReviews.length === 0) {
pendingReview = null
} else if (
matchingReviews.length === 1 &&
matchingReviews[0].state === eventToState[event]
) {
pendingReview = matchingReviews[0]
} else {
await dismissReviews({
github,
context,
core,
dry,
reviewKey,
})
pendingReview = null
}
if (dry) {
if (pendingReview)
core.info(`pending review found: ${pendingReview.html_url}`)
else core.info('no pending review found')
core.info(body)
} else {
if (pendingReview) {
await Promise.all([
github.rest.pulls.updateReview({
...context.repo,
pull_number,
review_id: pendingReview.id,
body,
}),
github.graphql(
`mutation($node_id:ID!) {
unminimizeComment(input: {
subjectId: $node_id
})
{ clientMutationId }
}`,
{ node_id: pendingReview.node_id },
),
])
} else {
await github.rest.pulls.createReview({
...context.repo,
pull_number,
event,
body,
})
}
}
}

View File

@@ -1,4 +1,4 @@
#!/usr/bin/env node
#!/usr/bin/env -S node --import ./run
import { execSync } from 'node:child_process'
import { closeSync, mkdtempSync, openSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
@@ -60,7 +60,7 @@ program
.argument('<pr>', 'Number of the Pull Request to check')
.option('--no-cherry-picks', 'Do not expect cherry-picks.')
.action(async (owner, repo, pr, options) => {
const commits = (await import('./commits.ts')).default
const commits = (await import('./commits.js')).default
await run(commits, owner, repo, pr, options)
})
@@ -101,7 +101,7 @@ program
.argument('<repo>', 'Name of the GitHub repository to run on (Example: nixpkgs)')
.argument('<pr>', 'Number of the Pull Request to run on')
.action(async (owner, repo, pr, options) => {
const checkCommitMessages = (await import('./lint-commits.ts')).default
const checkCommitMessages = (await import('./lint-commits.js')).default
await run(checkCommitMessages, owner, repo, pr, options)
})
@@ -112,8 +112,8 @@ program
.argument('<repo>', 'Name of the GitHub repository to run on (Example: nixpkgs)')
.argument('<pr>', 'Number of the Pull Request to run on')
.action(async (owner, repo, pr, options) => {
const checkTargetBranch = (await import('./check-target-branch.ts')).checkTargetBranch
await run(checkTargetBranch, owner, repo, pr, options)
const checkCommitMessages = (await import('./check-target-branch.js')).default
await run(checkCommitMessages, owner, repo, pr, options)
})
program
@@ -123,19 +123,8 @@ program
.argument('<repo>', 'Name of the GitHub repository to run on (Example: nixpkgs)')
.argument('<pr>', 'Number of the Pull Request to run on')
.action(async (owner, repo, pr, options) => {
const checkManualFileEdits = (await import('./manual-file-edits.ts')).default
const checkManualFileEdits = (await import('./manual-file-edits.js')).default
await run(checkManualFileEdits, owner, repo, pr, options)
})
program
.command('reminders')
.description('Post reminders for the paths a PR touches')
.argument('<owner>', 'Owner of the GitHub repository to run on (Example: NixOS)')
.argument('<repo>', 'Name of the GitHub repository to run on (Example: nixpkgs)')
.argument('<pr>', 'Number of the Pull Request to run on')
.action(async (owner, repo, pr, options) => {
const postReminders = (await import('./reminders.ts')).default
await run(postReminders, owner, repo, pr, options)
})
await program.parse()

View File

@@ -1,104 +0,0 @@
#!/usr/bin/env nix-shell
/*
#!nix-shell -i node -p nodejs
*/
import { resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
type BranchType = 'channel' | 'development' | 'primary' | 'secondary'
const typeConfig: Record<string, BranchType[]> = {
master: ['development', 'primary'],
release: ['development', 'primary'],
staging: ['development', 'secondary'],
'staging-next': ['development', 'secondary'],
'staging-nixos': ['development', 'secondary'],
'haskell-updates': ['development', 'secondary'],
nixos: ['channel'],
nixpkgs: ['channel'],
}
// "order" ranks the development branches by how likely they are the intended base branch
// when they are an otherwise equally good fit according to ci/github-script/prepare.js.
const orderConfig: Record<string, number> = {
master: 0,
release: 1,
staging: 2,
'staging-nixos': 2,
'haskell-updates': 3,
'staging-next': 4,
}
type Digit = 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9
type Version = `${Digit}${Digit}.${Digit}${Digit}` | 'unstable'
interface SplitResult {
prefix: string
version: Version
suffix?: string
}
function split(branch: string) {
const groups = branch.match(
/(?<prefix>.+?)(-(?<version>\d{2}\.\d{2}|unstable)(?:-(?<suffix>.*))?)?$/,
)!.groups!
return groups as unknown as SplitResult
}
interface BranchClassification {
branch: string
order: number
stable: boolean
type: BranchType[]
version: Version
}
function classify(branch: string): BranchClassification {
const { prefix, version } = split(branch)
return {
branch,
order: orderConfig[prefix] ?? Infinity,
stable: (version ?? 'unstable') !== 'unstable',
type: typeConfig[prefix] ?? ['wip'],
version: version ?? 'unstable',
}
}
export { classify, split }
// If called directly via CLI, runs the following tests:
if (
process.argv[1] &&
fileURLToPath(import.meta.url) === resolve(process.argv[1])
) {
console.log('split(branch)')
function testSplit(branch: string) {
console.log(branch, split(branch))
}
testSplit('master')
testSplit('release-25.05')
testSplit('staging')
testSplit('staging-next')
testSplit('staging-25.05')
testSplit('staging-next-25.05')
testSplit('nixpkgs-25.05-darwin')
testSplit('nixpkgs-unstable')
testSplit('haskell-updates')
testSplit('backport-123-to-release-25.05')
console.log('')
console.log('classify(branch)')
function testClassify(branch: string) {
console.log(branch, classify(branch))
}
testClassify('master')
testClassify('release-25.05')
testClassify('staging')
testClassify('staging-next')
testClassify('staging-25.05')
testClassify('staging-next-25.05')
testClassify('nixpkgs-25.05-darwin')
testClassify('nixpkgs-unstable')
testClassify('haskell-updates')
testClassify('backport-123-to-release-25.05')
}

View File

@@ -0,0 +1,10 @@
module.exports = async ({ github, context, targetSha }) => {
const { content, encoding } = (
await github.rest.repos.getContent({
...context.repo,
path: 'pkgs/top-level/release-supported-systems.json',
ref: targetSha,
})
).data
return JSON.parse(Buffer.from(content, encoding).toString())
}

View File

@@ -1,30 +0,0 @@
interface SupportedSystemsProps {
github: InstanceType<typeof import('@actions/github/lib/utils').GitHub>
context: typeof import('@actions/github').context
targetSha: string
}
export default async ({
github,
context,
targetSha,
}: SupportedSystemsProps) => {
const contentObject = (
await github.rest.repos.getContent({
...context.repo,
path: 'pkgs/top-level/release-supported-systems.json',
ref: targetSha,
})
).data
if ('type' in contentObject && contentObject.type === 'file') {
const { content, encoding } = contentObject
return JSON.parse(
Buffer.from(content, encoding as BufferEncoding).toString(),
)
} else {
throw new Error(
'Fetched pkgs/top-level/release-supported-systems.json is not a file',
)
}
}

View File

@@ -1,26 +0,0 @@
{
"compilerOptions": {
"lib": [
"es2024",
"ESNext.Array",
"ESNext.Collection",
"ESNext.Error",
"ESNext.Iterator",
"ESNext.Promise"
],
"module": "nodenext",
"target": "es2024",
"strict": true,
"esModuleInterop": true,
"skipLibCheck": true,
"moduleResolution": "nodenext",
"allowImportingTsExtensions": true,
"allowJs": true,
"checkJs": true,
"erasableSyntaxOnly": true,
"verbatimModuleSyntax": true,
"noEmit": true,
}
}

View File

@@ -1,7 +1,6 @@
// @ts-nocheck
import Bottleneck from 'bottleneck'
module.exports = async ({ github, core, maxConcurrent = 1 }, callback) => {
const Bottleneck = require('bottleneck')
export default async ({ github, core, maxConcurrent = 1 }, callback) => {
const stats = {
issues: 0,
prs: 0,

View File

@@ -31,7 +31,6 @@ runCommand "nixpkgs-vet"
env.NIXPKGS_VET_NIX_PACKAGE = nix;
}
''
export NIX_STORE_DIR=$(mktemp -d)
export NIX_STATE_DIR=$(mktemp -d)
$NIXPKGS_VET_NIX_PACKAGE/bin/nix-store --init

View File

@@ -9,22 +9,9 @@
},
"branch": "nixpkgs-unstable",
"submodules": false,
"revision": "7d5589bbf421c7b6f4185371abe3c465b1b557e9",
"url": "https://github.com/NixOS/nixpkgs/archive/7d5589bbf421c7b6f4185371abe3c465b1b557e9.tar.gz",
"hash": "sha256-8emM5Z42GzMSLLvjJt7UkX1j2k2TKXQIS7FnPTfeHno="
},
"nixpkgs-26.05-darwin": {
"type": "Git",
"repository": {
"type": "GitHub",
"owner": "NixOS",
"repo": "nixpkgs"
},
"branch": "nixpkgs-26.05-darwin",
"submodules": false,
"revision": "7486293a941f7b0ec123f0c431517027f705f60f",
"url": "https://github.com/NixOS/nixpkgs/archive/7486293a941f7b0ec123f0c431517027f705f60f.tar.gz",
"hash": "sha256-WQhNWIW3PPijuZexyl5Zf0tJuQ7sKt5C9WPXVO4pMuM="
"revision": "421eebfd0ec7bccd4abe826ce62d7e6e83129493",
"url": "https://github.com/NixOS/nixpkgs/archive/421eebfd0ec7bccd4abe826ce62d7e6e83129493.tar.gz",
"hash": "sha256:1lxfhfgiv1sz2v7fg43gny57sa6wf59n98q7ldsyb2p06f4sal7w"
}
},
"version": 8

82
ci/supportedBranches.js Executable file
View File

@@ -0,0 +1,82 @@
#!/usr/bin/env nix-shell
/*
#!nix-shell -i node -p nodejs
*/
const typeConfig = {
master: ['development', 'primary'],
release: ['development', 'primary'],
staging: ['development', 'secondary'],
'staging-next': ['development', 'secondary'],
'staging-nixos': ['development', 'secondary'],
'haskell-updates': ['development', 'secondary'],
nixos: ['channel'],
nixpkgs: ['channel'],
}
// "order" ranks the development branches by how likely they are the intended base branch
// when they are an otherwise equally good fit according to ci/github-script/prepare.js.
const orderConfig = {
master: 0,
release: 1,
staging: 2,
'staging-nixos': 2,
'haskell-updates': 3,
'staging-next': 4,
}
function split(branch) {
return {
...branch.match(
/(?<prefix>.+?)(-(?<version>\d{2}\.\d{2}|unstable)(?:-(?<suffix>.*))?)?$/,
).groups,
}
}
function classify(branch) {
const { prefix, version } = split(branch)
return {
branch,
order: orderConfig[prefix] ?? Infinity,
stable: (version ?? 'unstable') !== 'unstable',
type: typeConfig[prefix] ?? ['wip'],
version: version ?? 'unstable',
}
}
module.exports = { classify, split }
// If called directly via CLI, runs the following tests:
if (!module.parent) {
console.log('split(branch)')
function testSplit(branch) {
console.log(branch, split(branch))
}
testSplit('master')
testSplit('release-25.05')
testSplit('staging')
testSplit('staging-next')
testSplit('staging-25.05')
testSplit('staging-next-25.05')
testSplit('nixpkgs-25.05-darwin')
testSplit('nixpkgs-unstable')
testSplit('haskell-updates')
testSplit('backport-123-to-release-25.05')
console.log('')
console.log('classify(branch)')
function testClassify(branch) {
console.log(branch, classify(branch))
}
testClassify('master')
testClassify('release-25.05')
testClassify('staging')
testClassify('staging-next')
testClassify('staging-25.05')
testClassify('staging-next-25.05')
testClassify('nixpkgs-25.05-darwin')
testClassify('nixpkgs-unstable')
testClassify('haskell-updates')
testClassify('backport-123-to-release-25.05')
}

View File

@@ -39,11 +39,6 @@
".github/workflows/*.yml"
".github/workflows/*.yaml"
];
options = [
# Support self-repository syntax
''-ignore=reusable workflow call "\$/.+" at "uses" is not following the format''
''-ignore=specifying action "\$/.+" in invalid format because ref is missing.''
];
};
biome = {

View File

@@ -7,26 +7,24 @@ This directory houses the source files for the Nixpkgs manual.
> We are actively restructuring our documentation to be more beginner friendly.
>
When writing new docs use **Progressive Disclosure:**
When writing new docs use **Progressive Disclosure**
- Start simple, pick up beginners.
- Use **examples** first to show how to get something done.
- Keep **explanation** lean.
Start simple, pick up beginners.
Use **examples** first to show how to get something done. Keep **Explanation** lean.
Use our [style guide](./styleguide.md) for more in depth guidance on writing good documentation.
Use our [styleguide](./styleguide.md) for more in depth guidance on writing good documentation.
Documentation about Nixpkgs belongs here.
This includes getting started guides and onboarding guides for *using* Nixpkgs and the language frameworks it ships.
This directory contains **guides** and **reference** documentation for Nixpkgs.
Write **guides** task-first: lead with a working example, then explain in prose.
Write **reference** as the specification of functions and attributes.
Borrowing from [Diátaxis framework](https://diataxis.fr/) what suits our needs:
We are actively working to generate reference documentation from the [doc-comments](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md) present in code, which also lets you view it locally with the `:doc` command in `nix repl`, e.g.:
**Guides** are task-oriented. They can be tutorial-style walkthroughs or how-to sections.
Explanations appear as prose after examples.
```
nix-repl> :l <nixpkgs>
nix-repl> :doc lib.mapAttrsToList
```
**Reference** documentation is the specification of functions and attributes.
We are actively working to generate **all** reference documentation from the [doc-comments](https://github.com/NixOS/rfcs/blob/master/rfcs/0145-doc-strings.md) present in code.
This also provides the benefit of using `:doc` in the `nix repl` to view reference documentation locally on the fly.
See [Document structure](#document-structure) for a structural template.
@@ -49,23 +47,23 @@ If the build succeeds, the manual will be in `./result/share/doc/nixpkgs/manual.
### Development environment
To reduce repetition, consider using tools from the documentation development environment:
To reduce repetition, consider using tools from the provided development environment:
Load it from the Nixpkgs documentation directory with
```ShellSession
$ cd /path/to/nixpkgs/doc
$ nix-shell
```
To load the documentation development environment automatically when entering that directory:
To load the development utilities automatically when entering that directory, [set up `nix-direnv`](https://nix.dev/guides/recipes/direnv).
1. Install [`nix-direnv`](https://search.nixos.org/packages?channel=unstable&query=nix-direnv#show=nix-direnv)
1. Set up direnv in the documentation directory:
Make sure that your local files aren't added to Git history by adding the following lines to `.git/info/exclude` at the root of the Nixpkgs repository:
```ShellSession
$ cd doc
$ echo "use nix" > .envrc
$ direnv allow
```
```
/**/.envrc
/**/.direnv
```
#### Live preview
@@ -140,12 +138,14 @@ A few markups for other kinds of literals are also available:
- `` {env}`XDG_DATA_DIRS` ``
- `` {file}`/etc/passwd` ``
- `` {option}`networking.useDHCP` ``
- `` {var}`pkgs` ``
The values will be formatted as inline `<code>` elements.
- `` {var}`/etc/passwd` ``
These literal kinds are used mostly in NixOS option documentation.
This syntax is taken from [MyST](https://myst-parser.readthedocs.io/en/latest/syntax/syntax.html#roles-an-in-line-extension-point).
Though, the feature originates from [reStructuredText](https://www.sphinx-doc.org/en/master/usage/restructuredtext/roles.html#role-manpage) with slightly different syntax.
They are handled by `myst_role` defined per renderer. <!-- reverse references in code -->
#### Admonitions
Set off from the text to bring attention to something.
@@ -168,7 +168,7 @@ The following are supported:
- `example`
Example admonitions require a title to work.
If you don't provide one, the manual won't build.
If you don't provide one, the manual won't be built.
```markdown
::: {.example #ex-showing-an-example}
@@ -184,11 +184,11 @@ Text for the example.
For defining a group of terms:
```markdown
Pear
: Green or yellow bulbous fruit
pear
: green or yellow bulbous fruit
Watermelon
: Green fruit with red flesh
watermelon
: green fruit with red flesh
```
## Commit conventions
@@ -198,15 +198,15 @@ Watermelon
- If creating a commit purely for documentation changes, format the commit message in the following way:
```
doc/component: (documentation summary)
doc: (documentation summary)
(Motivation for change, relevant links, additional information.)
```
Examples:
* doc/stdenv: update the kernel config documentation to use `nix-shell`
* doc/getting-started: add information about `nix-update-script`
* doc: update the kernel config documentation to use `nix-shell`
* doc: add information about `nix-update-script`
Closes #216321.
@@ -220,7 +220,7 @@ When needed, each convention explains why it exists, so you can make a decision
Note that these conventions are about the **structure** of the manual (and its source files), not about the content that goes in it.
You, as the writer of documentation, are still in charge of its content.
**For prose style, see the [documentation style guide](./styleguide.md).**
**For prose style, see the [documentation styleguide](./styleguide.md).**
### Document structure
@@ -290,7 +290,7 @@ When changing existing content, update formatting if possible, but avoid excessi
### Examples first
Put examples before detailed explanations (see the [style guide](./styleguide.md) for the rationale).
Put examples before detailed explanations (see the [styleguide](./styleguide.md) for the rationale).
Use this structure for each documented item:

View File

@@ -15,3 +15,15 @@ In addition, it offers various options to customize parts of the builds.
There is no uniform interface for build helpers.
[Trivial build helpers](#chap-trivial-builders) and [fetchers](#chap-pkgs-fetchers) have various input types for convenience.
[Language- or framework-specific build helpers](#chap-language-support) usually follow the style of `stdenv.mkDerivation`, which accepts an attribute set or a fixed-point function taking an attribute set.
```{=include=} chapters
build-helpers/fixed-point-arguments.chapter.md
build-helpers/fetchers.chapter.md
build-helpers/trivial-build-helpers.chapter.md
build-helpers/testers.chapter.md
build-helpers/dev-shell-tools.chapter.md
build-helpers/special.md
build-helpers/images.md
hooks/index.md
packages/index.md
```

View File

@@ -853,7 +853,7 @@ Used with CVS. Expects `cvsRoot`, `tag`, and `hash`.
Used with Mercurial. Expects `url`, `rev`, `hash`, overridable with [`<pkg>.overrideAttrs`](#sec-pkg-overrideAttrs).
A number of fetcher functions wrap lower-level fetchers such as `fetchurl`, `fetchzip`, and `fetchgit`. They are mainly convenience functions intended for commonly used destinations of source code in Nixpkgs. These wrapper fetchers are listed below.
A number of fetcher functions wrap part of `fetchurl` and `fetchzip`. They are mainly convenience functions intended for commonly used destinations of source code in Nixpkgs. These wrapper fetchers are listed below.
## `fetchFromGitea`, `fetchFromForgejo` and `fetchFromCodeberg` {#fetchfromgitea}
@@ -876,45 +876,6 @@ However, `fetchFromGitHub` will automatically switch to using `fetchgit` in any
When `fetchgit` is used, refer to the `fetchgit` section for documentation of its available options.
## `fetchFromHuggingFace` {#fetchfromhuggingface}
`fetchFromHuggingFace` fetches repositories from Hugging Face Hub. It expects
`repoId`, exactly one of `rev` or `tag`, and `hash`.
`repoId` must be in the form `repo` or `owner/repo`, so repositories such as
`gpt2` work as well.
::: {.example #ex-fetchfromhuggingface}
# Fetching a model repository from Hugging Face
```nix
fetchFromHuggingFace {
repoId = "hf-internal-testing/tiny-random-gpt2";
rev = "71034c5d8bde858ff824298bdedc65515b97d2b9";
backend = "lfs";
hash = "sha256-8K9B/C62GW5lXC0c8QQpQ9QAE1UMoG+kYqvGhnWIp64=";
}
```
:::
The optional `repoType` argument selects which Hugging Face Hub repository type
to use:
- `"model"` (default) fetches from `https://huggingface.co/<repo-id>`
- `"dataset"` fetches from `https://huggingface.co/datasets/<repo-id>`
- `"space"` fetches from `https://huggingface.co/spaces/<repo-id>`
To use a different Hugging Face Hub instance, use `domain`
(defaults to `"huggingface.co"`).
The optional `backend` argument defaults to `"xet"`. Because the Xet backend is
not implemented yet, callers must currently set `backend = "lfs"`, which uses
`fetchgit` with Git LFS enabled and defaults `fetchSubmodules` to `false`.
`rootDir`, `sparseCheckout`, and low-level `fetchgit` options such as
`deepClone`, `fetchTags`, `leaveDotGit`, and `branchName` are also supported.
## `fetchFromGitLab` {#fetchfromgitlab}
This is used with GitLab repositories. It behaves similarly to `fetchFromGitHub`, and expects `owner`, `repo`, `rev`, and `hash`.
@@ -988,20 +949,6 @@ fetchRadiclePatch {
}
```
## `fetchFromTangled` {#fetchfromtangled}
This is to be used with tangled repositories. `fetchFromTangled` works with
very similar arguments to `fetchFromGithub`. However, instead of a `owner` and
`repo`, a `did` argument is expected.
```nix
fetchFromTangled {
did = "did:plc:jj6ajj6duxnlthwtnob4qyuv"; # tranquil.farm/tranquil-pds
tag = "v6.6.0";
hash = "sha256-cfTsjmK/IMqT5kMKOGpwwWbBlvtrCDOerUJJ8AVI3kY=";
}
```
## `requireFile` {#requirefile}
`requireFile` allows requesting files that cannot be fetched automatically, but whose content is known.

View File

@@ -1,3 +1,12 @@
# Images {#chap-images}
This chapter describes tools for creating various types of images.
```{=include=} sections
images/appimagetools.section.md
images/dockertools.section.md
images/ocitools.section.md
images/portableservice.section.md
images/makediskimage.section.md
images/binarycache.section.md
```

View File

@@ -28,7 +28,7 @@ However, [those were unified early 2020](https://github.com/NixOS/nixpkgs/pull/8
```nix
{ appimageTools, fetchurl }:
appimageTools.wrapType2 {
let
pname = "nuclear";
version = "0.6.30";
@@ -36,7 +36,8 @@ appimageTools.wrapType2 {
url = "https://github.com/nukeop/nuclear/releases/download/v${version}/nuclear-v${version}.AppImage";
hash = "sha256-he1uGC1M/nFcKpMM9JKY4oeexJcnzV0ZRxhTjtJz6xw=";
};
}
in
appimageTools.wrapType2 { inherit pname version src; }
```
:::
@@ -55,7 +56,7 @@ There are a few ways to learn which dependencies an application needs:
```nix
{ appimageTools, fetchurl }:
appimageTools.wrapType2 {
let
pname = "irccloud";
version = "0.16.0";
@@ -63,7 +64,9 @@ appimageTools.wrapType2 {
url = "https://github.com/irccloud/irccloud-desktop/releases/download/v${version}/IRCCloud-${version}-linux-x86_64.AppImage";
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
in
appimageTools.wrapType2 {
inherit pname version src;
extraPkgs = pkgs: [ pkgs.at-spi2-core ];
}
```
@@ -85,12 +88,12 @@ However, [those were unified early 2020](https://github.com/NixOS/nixpkgs/pull/8
# Extracting an AppImage to install extra files
`wrapType2` automatically extracts the AppImage for you and makes it available via the `contents` attribute.
Note how `finalAttrs.contents` is used in `extraInstallCommands` to install additional files that were extracted from the AppImage.
This example was adapted from a real package in Nixpkgs to show how `extract` is usually used in combination with `wrapType2`.
Note how `appimageContents` is used in `extraInstallCommands` to install additional files that were extracted from the AppImage.
```nix
{ appimageTools, fetchurl }:
appimageTools.wrapType2 (finalAttrs: {
let
pname = "irccloud";
version = "0.16.0";
@@ -99,24 +102,27 @@ appimageTools.wrapType2 (finalAttrs: {
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
appimageContents = appimageTools.extract { inherit pname version src; };
in
appimageTools.wrapType2 {
inherit pname version src;
extraPkgs = pkgs: [ pkgs.at-spi2-core ];
extraInstallCommands = ''
mv $out/bin/irccloud-${version} $out/bin/irccloud
install -m 444 -D ${finalAttrs.contents}/irccloud.desktop $out/share/applications/irccloud.desktop
install -m 444 -D ${finalAttrs.contents}/usr/share/icons/hicolor/512x512/apps/irccloud.png \
install -m 444 -D ${appimageContents}/irccloud.desktop $out/share/applications/irccloud.desktop
install -m 444 -D ${appimageContents}/usr/share/icons/hicolor/512x512/apps/irccloud.png \
$out/share/icons/hicolor/512x512/apps/irccloud.png
substituteInPlace $out/share/applications/irccloud.desktop \
--replace-fail 'Exec=AppRun' 'Exec=irccloud'
'';
})
}
```
:::
`appimageTools` also exposes the `extract` function should you need to do it manually, requiring `pname`, `version`, and `src` arguments (`src` being the AppImage file to extract).
The arguments passed to `extract` can also contain a `postExtract` attribute, which allows you to execute additional commands after the files are extracted from the AppImage.
The argument passed to `extract` can also contain a `postExtract` attribute, which allows you to execute additional commands after the files are extracted from the AppImage.
`postExtract` must be a string with commands to run.
:::{.warning}
@@ -132,7 +138,7 @@ This is a rewrite of [](#ex-extracting-appimage) to use `postExtract` and `wrapA
```nix
{ appimageTools, fetchurl }:
appimageTools.wrapAppImage (finalAttrs: {
let
pname = "irccloud";
version = "0.16.0";
@@ -141,22 +147,30 @@ appimageTools.wrapAppImage (finalAttrs: {
hash = "sha256-/hMPvYdnVB1XjKgU2v47HnVvW4+uC3rhRjbucqin4iI=";
};
contents = appimageTools.extract {
inherit (finalAttrs) pname version src;
appimageContents = appimageTools.extract {
inherit pname version src;
postExtract = ''
substituteInPlace $out/irccloud.desktop --replace-fail 'Exec=AppRun' 'Exec=irccloud'
'';
};
in
appimageTools.wrapAppImage {
inherit pname version;
src = appimageContents;
extraPkgs = pkgs: [ pkgs.at-spi2-core ];
extraInstallCommands = ''
mv $out/bin/irccloud-${version} $out/bin/irccloud
install -m 444 -D ${finalAttrs.contents}/irccloud.desktop $out/share/applications/irccloud.desktop
install -m 444 -D ${finalAttrs.contents}/usr/share/icons/hicolor/512x512/apps/irccloud.png \
install -m 444 -D ${appimageContents}/irccloud.desktop $out/share/applications/irccloud.desktop
install -m 444 -D ${appimageContents}/usr/share/icons/hicolor/512x512/apps/irccloud.png \
$out/share/icons/hicolor/512x512/apps/irccloud.png
'';
})
# specify src archive for nix-update
passthru.src = src;
}
```
:::

View File

@@ -68,7 +68,7 @@ See [](#ex-portableService-hello) to understand how to use the output of `portab
: Allows you to override the package that provides {manpage}`mksquashfs(1)`, which is used internally by `portableService`.
_Default value:_ `pkgs.squashfs-tools`.
_Default value:_ `pkgs.squashfsTools`.
`squash-compression` (String; _optional_)

View File

@@ -1,3 +1,13 @@
# Special build helpers {#chap-special}
This chapter describes several special build helpers.
```{=include=} sections
special/buildenv.section.md
special/fakenss.section.md
special/fhs-environments.section.md
special/makesetuphook.section.md
special/mkshell.section.md
special/vm-tools.section.md
special/checkpoint-build.section.md
```

View File

@@ -3,35 +3,6 @@
Nixpkgs provides a variety of wrapper functions that help build commonly useful derivations.
Like [`stdenv.mkDerivation`](#sec-using-stdenv), each of these build helpers creates a derivation, but the arguments passed are different (usually simpler) from those required by `stdenv.mkDerivation`.
## Arguments with finalAttrs {#trivial-builder-finalAttrs}
In parameters that reference this section, you may either pass the value itself,
or a function that produces it.
When it's a function the argument value is [`finalAttrs`] from [`mkDerivation`].
Typically both the *attributes* and *script* arguments support this, simultaneously if needed.
::: {.example #ex-trivial-builder-finalAttrs}
# Using `finalAttrs` in a build helper
```nix
runCommand "hi" (finalAttrs: { passthru.exe = "${finalAttrs.finalPackage}/bin/hi"; }) ''
mkdir -p $out/bin
substitute ${./hi.foo} $out/bin/hi --replace-fail "@foo@" ${lib.getExe foo}
''
```
This creates a package with an executable script that's in the standard `bin/` directory,
but also convenient to interpolate without reliance on `$PATH`, e.g assuming the result of the above is in binding `hi`:
```nix
''
echo START_GREETING
${hi.exe} --rude
echo END_GREETING
''
```
:::
## `runCommandWith` {#trivial-builder-runCommandWith}
@@ -52,10 +23,8 @@ runCommandWith :: {
name :: name;
stdenv? :: Derivation;
runLocal? :: Bool;
derivationArgs? :: { ... } | finalAttrs@{ finalPackage :: Derivation, ... } -> { ... };
}
-> (String | finalAttrs@{ finalPackage :: Derivation, ... } -> String)
-> Derivation
derivationArgs? :: { ... };
} -> String -> Derivation
```
### Inputs {#trivial-builder-runCommandWith-Inputs}
@@ -78,10 +47,10 @@ runCommandWith :: {
`stdenv` (Derivation)
: The [standard environment](#chap-stdenv) to use, defaulting to `pkgs.stdenv`.
`derivationArgs` (Attribute set *or* [function from `finalAttrs`](#trivial-builder-finalAttrs))
`derivationArgs` (Attribute set)
: Additional arguments for [`mkDerivation`](#sec-using-stdenv).
`buildCommand` (String *or* [function from `finalAttrs`](#trivial-builder-finalAttrs))
`buildCommand` (String)
: Shell commands to run in the derivation builder.
::: {.note}
@@ -140,10 +109,10 @@ While the type signature(s) differ from [`runCommandWith`], individual arguments
`name` (String)
: The derivation's name
`derivationArgs` (Attribute set *or* [function from `finalAttrs`](#trivial-builder-finalAttrs))
`derivationArgs` (Attribute set)
: Additional parameters passed to [`mkDerivation`]
`buildCommand` (String *or* [function from `finalAttrs`](#trivial-builder-finalAttrs))
`buildCommand` (String)
: The command(s) run to build the derivation.
@@ -933,6 +902,3 @@ produces an output path `/nix/store/<hash>-runtime-references` containing
but none of `hello`'s dependencies because those are not referenced directly
by `hi`'s output.
[`finalAttrs`]: #mkderivation-recursive-attributes
[`mkDerivation`]: #sec-using-stdenv

View File

@@ -1,84 +1,10 @@
# Contributing {#part-contributing}
# Contributing to Nixpkgs {#part-contributing}
<!--
Legacy anchors. The sections behind these ids used to live in this manual and are
now maintained elsewhere. We keep these invisible anchor ids so external URIs would link to this page.
-->
<!-- moved to CONTRIBUTING.md -->
[]{#chap-conventions}
[]{#sec-syntax}
[]{#sec-organisation}
[]{#chap-submitting-changes}
[]{#submitting-changes-submitting-changes}
[]{#submitting-changes-pull-request-template}
[]{#submitting-changes-tested-with-sandbox}
[]{#submitting-changes-platform-diversity}
[]{#submitting-changes-nixos-tests}
[]{#submitting-changes-tested-compilation}
[]{#submitting-changes-tested-execution}
[]{#submitting-changes-contribution-standards}
[]{#submitting-changes-hotfixing-pull-requests}
[]{#submitting-changes-commit-policy}
[]{#submitting-changes-branches}
[]{#submitting-changes-master-branch}
[]{#submitting-changes-staging-branch}
[]{#submitting-changes-staging-next-branch}
[]{#submitting-changes-stable-release-branches}
[]{#submitting-changes-stable-release-branches-automatic-backports}
[]{#submitting-changes-stable-release-branches-manual-backports}
[]{#acceptable-backport-criteria}
[]{#chap-reviewing-contributions}
[]{#reviewing-contributions-other-submissions}
[]{#reviewing-contributions--merging-pull-requests}
[]{#part-development}
<!-- moved to pkgs/README.md -->
[]{#chap-quick-start}
[]{#sec-package-naming}
[]{#sec-versioning}
[]{#sec-sources}
[]{#sec-source-hashes}
[]{#sec-source-hashes-security}
[]{#sec-patches}
[]{#sec-package-tests}
[]{#ssec-inline-package-tests-writing}
[]{#ssec-package-tests-writing}
[]{#ssec-package-tests-running}
[]{#ssec-package-tests-examples}
[]{#ssec-nixos-tests-linking}
[]{#ssec-import-from-derivation}
[]{#submitting-changes-submitting-security-fixes}
[]{#submitting-changes-deprecating-packages}
[]{#steps-to-remove-a-package-from-nixpkgs}
[]{#chap-vulnerability-roundup}
[]{#vulnerability-roundup-issues}
[]{#vulnerability-roundup-triaging-and-fixing}
[]{#reviewing-contributions-package-updates}
[]{#reviewing-contributions-new-packages}
<!-- moved to nixos/README.md -->
[]{#reviewing-contributions-module-updates}
[]{#reviewing-contributions-new-modules}
<!-- moved to maintainers/README.md -->
[]{#reviewing-contributions-individual-maintainer-list}
[]{#reviewing-contributions-maintainer-teams}
<!-- moved to doc/README.md -->
[]{#chap-contributing}
Contribution documentation lives along the repository scope it applies to:
- [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md): coding conventions, file organisation, submitting changes, commit policy, branches and backports, reviewing contributions.
- [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md): adding and updating packages, package naming and versioning, source fetching and hashes, patches, package tests, deprecating packages, security fixes.
- [nixos/README.md](https://github.com/NixOS/nixpkgs/blob/master/nixos/README.md): NixOS modules and reviewing module changes.
- [maintainers/README.md](https://github.com/NixOS/nixpkgs/blob/master/maintainers/README.md): maintainer entries and maintainer teams.
- [doc/README.md](https://github.com/NixOS/nixpkgs/blob/master/doc/README.md): writing and building this manual.
## Opening issues {#sec-opening-issues}
- Make sure you have a [GitHub account](https://github.com/signup/free)
- Make sure there is no open issue on the topic
- [Submit a new issue](https://github.com/NixOS/nixpkgs/issues/new/choose) by choosing the kind of topic and filling out the template
<!-- In the future this section could also include more detailed information on the issue templates -->
```{=include=} chapters
contributing/quick-start.chapter.md
contributing/coding-conventions.chapter.md
contributing/submitting-changes.chapter.md
contributing/vulnerability-roundup.chapter.md
contributing/reviewing-contributions.chapter.md
contributing/contributing-to-documentation.chapter.md
```

View File

@@ -0,0 +1,63 @@
# Coding conventions {#chap-conventions}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Syntax {#sec-syntax}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Package naming {#sec-package-naming}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## File naming and organisation {#sec-organisation}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Versioning {#sec-versioning}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Fetching Sources {#sec-sources}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Obtaining source hash {#sec-source-hashes}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Obtaining hashes securely {#sec-source-hashes-security}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Patches {#sec-patches}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Package tests {#sec-package-tests}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Writing inline package tests {#ssec-inline-package-tests-writing}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Writing larger package tests {#ssec-package-tests-writing}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Running package tests {#ssec-package-tests-running}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Examples of package tests {#ssec-package-tests-examples}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Linking NixOS module tests to a package {#ssec-nixos-tests-linking}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Import From Derivation {#ssec-import-from-derivation}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).

View File

@@ -0,0 +1,3 @@
# Contributing to Nixpkgs documentation {#chap-contributing}
This section has been moved to [doc/README.md](https://github.com/NixOS/nixpkgs/blob/master/doc/README.md).

View File

@@ -0,0 +1,3 @@
# Quick Start to Adding a Package {#chap-quick-start}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).

View File

@@ -0,0 +1,35 @@
# Reviewing contributions {#chap-reviewing-contributions}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Package updates {#reviewing-contributions-package-updates}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## New packages {#reviewing-contributions-new-packages}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Module updates {#reviewing-contributions-module-updates}
This section has been moved to [nixos/README.md](https://github.com/NixOS/nixpkgs/blob/master/nixos/README.md).
## New modules {#reviewing-contributions-new-modules}
This section has been moved to [nixos/README.md](https://github.com/NixOS/nixpkgs/blob/master/nixos/README.md#new-modules).
## Individual maintainer list {#reviewing-contributions-individual-maintainer-list}
This section has been moved to [maintainers/README.md](https://github.com/NixOS/nixpkgs/blob/master/maintainers/README.md).
## Maintainer teams {#reviewing-contributions-maintainer-teams}
This section has been moved to [maintainers/README.md](https://github.com/NixOS/nixpkgs/blob/master/maintainers/README.md).
## Other submissions {#reviewing-contributions-other-submissions}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Merging pull requests {#reviewing-contributions--merging-pull-requests}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).

View File

@@ -0,0 +1,88 @@
# Submitting changes {#chap-submitting-changes}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Submitting changes {#submitting-changes-submitting-changes}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Submitting security fixes {#submitting-changes-submitting-security-fixes}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Deprecating/removing packages {#submitting-changes-deprecating-packages}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
### Steps to remove a package from Nixpkgs {#steps-to-remove-a-package-from-nixpkgs}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Pull Request Template {#submitting-changes-pull-request-template}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Tested using sandboxing {#submitting-changes-tested-with-sandbox}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Built on platform(s) {#submitting-changes-platform-diversity}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests) {#submitting-changes-nixos-tests}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Tested compilation of all pkgs that depend on this change using `nixpkgs-review` {#submitting-changes-tested-compilation}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Tested execution of all binary files (usually in `./result/bin/`) {#submitting-changes-tested-execution}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Meets Nixpkgs contribution standards {#submitting-changes-contribution-standards}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Hotfixing pull requests {#submitting-changes-hotfixing-pull-requests}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
## Commit policy {#submitting-changes-commit-policy}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
### Branches {#submitting-changes-branches}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Master branch {#submitting-changes-master-branch}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Staging branch {#submitting-changes-staging-branch}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Staging-next branch {#submitting-changes-staging-next-branch}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Stable release branches {#submitting-changes-stable-release-branches}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Automatically backporting a Pull Request {#submitting-changes-stable-release-branches-automatic-backports}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Manually backporting changes {#submitting-changes-stable-release-branches-manual-backports}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
#### Acceptable backport criteria {#acceptable-backport-criteria}
This section has been moved to [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).

View File

@@ -0,0 +1,11 @@
# Vulnerability Roundup {#chap-vulnerability-roundup}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Issues {#vulnerability-roundup-issues}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).
## Triaging and Fixing {#vulnerability-roundup-triaging-and-fixing}
This section has been moved to [pkgs/README.md](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md).

10
doc/development.md Normal file
View File

@@ -0,0 +1,10 @@
# Development of Nixpkgs {#part-development}
This section shows you how Nixpkgs is developed and how you can interact with the contributors and the latest updates.
If you are interested in contributing yourself, see [CONTRIBUTING.md](https://github.com/NixOS/nixpkgs/blob/master/CONTRIBUTING.md).
<!-- In the future this section should also include: How to test pull requests, how to know if pull requests are available in channels, etc. -->
```{=include=} chapters
development/opening-issues.chapter.md
```

View File

@@ -0,0 +1,7 @@
# Opening issues {#sec-opening-issues}
* Make sure you have a [GitHub account](https://github.com/signup/free)
* Make sure there is no open issue on the topic
* [Submit a new issue](https://github.com/NixOS/nixpkgs/issues/new/choose) by choosing the kind of topic and filling out the template
<!-- In the future this section could also include more detailed information on the issue templates -->

View File

@@ -1,205 +0,0 @@
import json
import re
import sys
from pathlib import Path
from typing import TypedDict
# Coupled to where this file lives!
# Needed to resolve the relative includes file paths
REPO_ROOT = Path(__file__).resolve().parent.parent.parent
ROOT_FILE = REPO_ROOT / "nixos/doc/manual/manual.md"
DOC_ROOT = ROOT_FILE.parent
INCLUDE_RE = re.compile(r"^```\{=include=\}(?P<rest>.*)$")
FENCE_RE = re.compile(r"^(```|~~~)")
HEADING_RE = re.compile(r"^(#{1,6})\s+(.*?)\s*(?:\{#([^}]+)\})?\s*$")
class TokenIncludeBlock:
"""Represents a complete include block like:
```{=include=} sections
special/buildenv.section.md
```
As
typ = "sections"
files = [ "special/buildenv.section.md" ]
"""
kind = "include"
def __repr__(self):
# For debugging
return f"```{{=include=}} {self.typ}\n" "\n".join(self.files) + "\n```"
def __init__(self, typ, args, files, start, end):
self.typ: str = typ
self.args: list[str] = args
self.files: list[str] = files
self.start: int = start
self.end: int = end
def is_option_block(self) -> bool:
# option blocks have some special syntax which is part of another migration
# 8 coccurrences, seperate migration
return self.typ == "options"
def into_file(self) -> bool:
# into-file is trivial after the nav migration
# 2 coccurrences, can be migrated by hand
return any("html:into-file=" in arg for arg in self.args)
def keep(self) -> bool:
# Keep the include blocks that require seperate migration
return self.is_option_block() or self.into_file()
class TokenHeading:
kind = "heading"
def __init__(self, level, title, anchor):
self.level: int = level
self.title: str = title
self.anchor: str = anchor
def tokenize(src: str) -> list[TokenIncludeBlock|TokenHeading]:
"""Finds all Includes and Headings"""
lines = src.splitlines()
items = []
fenced = False
i = 0
while i < len(lines):
line = lines[i]
m = INCLUDE_RE.match(line)
if m and not fenced:
typ, *args = m.group("rest").split()
end = i + 1
while end < len(lines) and not lines[end].startswith("```"):
end += 1
if end == len(lines):
sys.exit(f"unterminated include block at line {i + 1}")
files = [f.strip() for f in lines[i + 1 : end] if f.strip()]
items.append(TokenIncludeBlock(typ, args, files, i, end))
i = end + 1
continue
if FENCE_RE.match(line):
fenced = not fenced
i += 1
continue
if not fenced:
m = HEADING_RE.match(line)
if m:
items.append(TokenHeading(len(m.group(1)), m.group(2), m.group(3)))
i += 1
return items
def read_source(source: Path) -> None | tuple[str, Path]:
"""Read a return [content,Path]
library.md is called .md.in; provided at build time via nixdoc
So the filename might differ
"""
if source.exists():
return source.read_text(), source
in_file = Path(str(source) + ".in")
if in_file.exists():
return in_file.read_text(), in_file
return None
# Make sure we process every file only once
visited = set()
# Map from filename -> line_nrs
# These lines will be deleted from the file
rewrites: dict[str, tuple[int,int]] = {}
group_ids: set[str] = set()
class Green(TypedDict):
label: str
id: str
children: list["Green"]
file: str | None
def convert_md_in(md_in: Path) -> Green:
"""Build up items for the nav.json as we recurse.
Collecting line-areas and group_ids
"""
if md_in in visited:
sys.exit(f"{md_in}: reached twice")
visited.add(md_in)
res = read_source(md_in)
rel_file = md_in.relative_to(DOC_ROOT).as_posix()
if not res:
return {"file": rel_file}
text, actual = res
token: list[TokenHeading|TokenIncludeBlock] = tokenize(text)
# Collect a list of actual files to be processed
# Recurse for files
# Do not recurse for "options", "into-file"
all_file_includes: list[TokenIncludeBlock] = [it for it in token if it.kind == "include" and not (it.is_option_block() or it.into_file())]
children = [convert_md_in((md_in.parent / f).resolve()) for b in all_file_includes for f in b.files]
# Lines to rewrite
include_blocks_to_rewrite: list[TokenIncludeBlock] = [it for it in all_file_includes if not it.keep()]
rewrites[actual] = [(b.start, b.end) for b in include_blocks_to_rewrite]
if not children:
return {"file": rel_file}
title = next((it for it in token if it.kind == "heading" and it.level == 1), None)
if title is None:
sys.exit(f"{md_in}: no level-1 heading to label its group")
if not title.anchor:
sys.exit(f"{md_in}: level-1 heading has no id")
if title.anchor in group_ids:
sys.exit(f"{md_in}: duplicate group id {title.anchor}")
group_ids.add(title.anchor)
return Green({
"label": title.title.replace("`", ""),
"id": title.anchor,
"children": [{"file": rel_file}] + children,
})
def remove_includes(path: Path, drop_lines: tuple[int,int]):
if not drop_lines:
return
lines = path.read_text().splitlines()
for start, end in sorted(drop_lines, reverse=True):
del lines[start : end + 1]
if 0 < start < len(lines) and not lines[start - 1].strip() and not lines[start].strip():
del lines[start]
while lines and not lines[-1].strip():
lines.pop()
path.write_text("\n".join(lines) + "\n")
def main():
nav = DOC_ROOT / "nav.json"
if json.loads(nav.read_text())["items"]:
sys.exit("nav.json already has items; This tool can only run once")
root = convert_md_in(ROOT_FILE)
items = root["children"][1:]
nav.write_text(json.dumps({"open": [], "items": items}, indent=2) + "\n")
for path, lines in rewrites.items():
remove_includes(path, lines)
if __name__ == "__main__":
main()

View File

@@ -1,27 +1,151 @@
# Generates the documentation for library functions via nixdoc.
# To build this derivation, run `nix-build -A nixpkgs-manual.lib-docs`
{
lib,
stdenvNoCC,
nixdoc,
nix,
nixpkgs ? { },
libsets ? [
{
name = "asserts";
description = "assertion functions";
}
{
name = "attrsets";
description = "attribute set functions";
}
{
name = "strings";
description = "string manipulation functions";
}
{
name = "versions";
description = "version string functions";
}
{
name = "trivial";
description = "miscellaneous functions";
}
{
name = "fixedPoints";
baseName = "fixed-points";
description = "explicit recursion functions";
}
{
name = "lists";
description = "list manipulation functions";
}
{
name = "debug";
description = "debugging functions";
}
{
name = "options";
description = "NixOS / nixpkgs option handling";
}
{
name = "path";
description = "path functions";
}
{
name = "fetchers";
description = "functions which can be reused across fetchers";
}
{
name = "filesystem";
description = "filesystem functions";
}
{
name = "fileset";
description = "file set functions";
}
{
name = "sources";
description = "source filtering functions";
}
{
name = "cli";
description = "command-line serialization functions";
}
{
name = "generators";
description = "functions that create file formats from nix data structures";
}
{
name = "gvariant";
description = "GVariant formatted string serialization functions";
}
{
name = "customisation";
description = "Functions to customise (derivation-related) functions, derivations, or attribute sets";
}
{
name = "meta";
description = "functions for derivation metadata";
}
{
name = "derivations";
description = "miscellaneous derivation-specific functions";
}
],
}:
stdenvNoCC.mkDerivation {
name = "nixpkgs-lib-docs";
src = ../../lib;
nativeBuildInputs = [ nixdoc ];
buildPhase = ''
mkdir -p src
cp -r ${../../lib} src/lib
mkdir -p $out
nixdoc --manifest ${../function-catalog.json} --root src --output $out/lib-functions.json
'';
nativeBuildInputs = [
nixdoc
nix
];
installPhase = ''
runHook preInstall
cd ..
export NIX_STATE_DIR=$(mktemp -d)
nix-instantiate --eval --strict --json ${./lib-function-locations.nix} \
--arg nixpkgsPath "./." \
--argstr revision ${nixpkgs.rev or "master"} \
--argstr libsetsJSON ${lib.escapeShellArg (builtins.toJSON libsets)} \
--store $(mktemp -d) \
> locations.json
function docgen {
name=$1
baseName=$2
description=$3
# TODO: wrap lib.$name in <literal>, make nixdoc not escape it
if [[ -e "lib/$baseName.nix" ]]; then
nixdoc -c "$name" -d "lib.$name: $description" -l locations.json -f "lib/$baseName.nix" > "$out/$name.md"
else
nixdoc -c "$name" -d "lib.$name: $description" -l locations.json -f "lib/$baseName/default.nix" > "$out/$name.md"
fi
echo "$out/$name.md" >> "$out/index.md"
}
mkdir -p "$out"
cat > "$out/index.md" << 'EOF'
```{=include=} sections auto-id-prefix=auto-generated
EOF
${lib.concatMapStrings (
{
name,
baseName ? name,
description,
}:
''
docgen ${name} ${baseName} ${lib.escapeShellArg description}
''
) libsets}
echo '```' >> "$out/index.md"
runHook postInstall
'';
}

View File

@@ -0,0 +1,82 @@
{
nixpkgsPath,
revision,
libsetsJSON,
}:
let
lib = import (nixpkgsPath + "/lib");
libsets = builtins.fromJSON libsetsJSON;
libDefPos =
prefix: set:
builtins.concatMap (
name:
[
{
name = builtins.concatStringsSep "." (prefix ++ [ name ]);
location = builtins.unsafeGetAttrPos name set;
}
]
++ lib.optionals (builtins.length prefix == 0 && builtins.isAttrs set.${name}) (
libDefPos (prefix ++ [ name ]) set.${name}
)
) (builtins.attrNames set);
libset =
toplib:
map (subsetname: {
subsetname = subsetname;
functions = libDefPos [ ] toplib.${subsetname};
}) (map (x: x.name) libsets);
flattenedLibSubset =
{ subsetname, functions }:
map (fn: {
name = "lib.${subsetname}.${fn.name}";
value = fn.location;
}) functions;
locatedlibsets = libs: map flattenedLibSubset (libset libs);
removeFilenamePrefix =
prefix: filename:
let
prefixLen = (builtins.stringLength prefix) + 1; # +1 to remove the leading /
filenameLen = builtins.stringLength filename;
substr = builtins.substring prefixLen filenameLen filename;
in
substr;
removeNixpkgs = removeFilenamePrefix (toString nixpkgsPath);
liblocations = builtins.filter (elem: elem.value != null) (lib.lists.flatten (locatedlibsets lib));
fnLocationRelative =
{ name, value }:
{
inherit name;
value = value // {
file = removeNixpkgs value.file;
};
};
relativeLocs = (map fnLocationRelative liblocations);
sanitizeId = builtins.replaceStrings [ "'" ] [ "-prime" ];
urlPrefix = "https://github.com/NixOS/nixpkgs/blob/${revision}";
jsonLocs = builtins.listToAttrs (
map (
{ name, value }:
{
name = sanitizeId name;
value =
let
text = "${value.file}:${toString value.line}";
target = "${urlPrefix}/${value.file}#L${toString value.line}";
in
"[${text}](${target}) in `<nixpkgs>`";
}
) relativeLocs
);
in
jsonLocs

View File

@@ -91,7 +91,7 @@ stdenvNoCC.mkDerivation (
substituteInPlace ./languages-frameworks/python.section.md \
--subst-var-by python-interpreter-table "$(<"${pythonInterpreterTable}")"
cp ${lib-docs}/lib-functions.json ./lib-functions.json
cat ./functions/library.md.in ${lib-docs}/index.md > ./functions/library.md
substitute ./manual.md.in ./manual.md \
--replace-fail '@MANUAL_VERSION@' '${lib.version}'
@@ -118,7 +118,7 @@ stdenvNoCC.mkDerivation (
--script ./anchor.min.js \
--script ./anchor-use.js \
--sidebar-depth 3 \
--experimental-config ./nav.json \
--nav ./nav.json \
--header ${./header.html}\
--no-navheader \
manual.md \
@@ -147,7 +147,7 @@ stdenvNoCC.mkDerivation (
'';
passthru = {
lib-docs = callPackage ./lib-function-docs.nix { };
lib-docs = callPackage ./lib-function-docs.nix { inherit nixpkgs; };
epub = callPackage ./epub.nix { };
@@ -176,7 +176,5 @@ stdenvNoCC.mkDerivation (
manpage-urls = callPackage ../tests/manpage-urls.nix { };
};
};
meta.license = lib.licenses.mit;
}
)

View File

@@ -53,7 +53,7 @@ Pin Nixpkgs and call the package from `default.nix`:
```nix
# default.nix
let
nixpkgs = fetchTarball "https://channels.nixos.org/nixos-unstable/nixexprs.tar.zst";
nixpkgs = fetchTarball "https://github.com/NixOS/nixpkgs/archive/nixos-unstable.tar.gz";
pkgs = import nixpkgs { };
in
pkgs.callPackage ./package.nix { }

View File

@@ -1,124 +0,0 @@
{
"version": 1,
"groups": [
{ "id": "asserts", "description": "assertion functions" },
{ "id": "attrsets", "description": "attribute set functions" },
{ "id": "strings", "description": "string manipulation functions" },
{ "id": "versions", "description": "version string functions" },
{ "id": "trivial", "description": "miscellaneous functions" },
{ "id": "fixedPoints", "description": "explicit recursion functions" },
{ "id": "lists", "description": "list manipulation functions" },
{ "id": "debug", "description": "debugging functions" },
{ "id": "options", "description": "NixOS / nixpkgs option handling" },
{ "id": "path", "description": "path functions" },
{
"id": "fetchers",
"description": "functions which can be reused across fetchers"
},
{ "id": "filesystem", "description": "filesystem functions" },
{ "id": "fileset" },
{ "id": "sources", "description": "source filtering functions" },
{ "id": "cli", "description": "command-line serialization functions" },
{
"id": "generators",
"description": "functions that create file formats from nix data structures"
},
{
"id": "gvariant",
"description": "GVariant formatted string serialization functions"
},
{
"id": "customisation",
"description": "Functions to customise (derivation-related) functions, derivations, or attribute sets"
},
{ "id": "meta", "description": "functions for derivation metadata" },
{
"id": "derivations",
"description": "miscellaneous derivation-specific functions"
}
],
"sources": [
{ "path": "lib.trivial", "file": "lib/trivial.nix", "groups": ["trivial"] },
{
"path": "lib.fixedPoints",
"file": "lib/fixed-points.nix",
"groups": ["fixedPoints"]
},
{
"path": "lib.attrsets",
"file": "lib/attrsets.nix",
"groups": ["attrsets"]
},
{ "path": "lib.lists", "file": "lib/lists.nix", "groups": ["lists"] },
{ "path": "lib.strings", "file": "lib/strings.nix", "groups": ["strings"] },
{
"path": "lib.customisation",
"file": "lib/customisation.nix",
"groups": ["customisation"]
},
{
"path": "lib.derivations",
"file": "lib/derivations.nix",
"groups": ["derivations"]
},
{ "path": "lib.meta", "file": "lib/meta.nix", "groups": ["meta"] },
{
"path": "lib.versions",
"file": "lib/versions.nix",
"groups": ["versions"]
},
{ "path": "lib.cli", "file": "lib/cli.nix", "groups": ["cli"] },
{
"path": "lib.gvariant",
"file": "lib/gvariant.nix",
"groups": ["gvariant"]
},
{
"path": "lib.generators",
"file": "lib/generators.nix",
"groups": ["generators"]
},
{ "path": "lib.asserts", "file": "lib/asserts.nix", "groups": ["asserts"] },
{ "path": "lib.debug", "file": "lib/debug.nix", "groups": ["debug"] },
{
"path": "lib.fetchers",
"file": "lib/fetchers.nix",
"groups": ["fetchers"]
},
{ "path": "lib.path", "file": "lib/path/default.nix", "groups": ["path"] },
{
"path": "lib.filesystem",
"file": "lib/filesystem.nix",
"groups": ["filesystem"]
},
{
"path": "lib.fileset",
"file": "lib/fileset/default.nix",
"groups": ["fileset"]
},
{
"path": "lib.sources",
"file": "lib/sources.nix",
"groups": ["sources"],
"mode": "deep",
"include": [
"pathIsGitRepo",
"commitIdFromGitRepo",
"cleanSource",
"cleanSourceWith",
"cleanSourceFilter",
"pathHasContext",
"canCleanSource",
"urlToName",
"shortRev",
"revOrTag",
"repoRevToName",
"sourceByRegex",
"sourceFilesBySuffices",
"sourceByGlobs",
"trace"
]
},
{ "path": "lib.options", "file": "lib/options.nix", "groups": ["options"] }
]
}

View File

@@ -1,3 +1,11 @@
# Functions reference {#chap-functions}
The Nixpkgs repository has several utility functions to manipulate Nix expressions.
```{=include=} sections
functions/library.md
functions/generators.section.md
functions/debug.section.md
functions/prefer-remote-fetch.section.md
functions/nix-gitignore.section.md
```

View File

@@ -1,3 +0,0 @@
# Nixpkgs Library Functions {#sec-functions-library}
Nixpkgs provides a standard library at `pkgs.lib`, or through `import <nixpkgs/lib>`.

View File

@@ -0,0 +1,5 @@
# Nixpkgs Library Functions {#sec-functions-library}
Nixpkgs provides a standard library at `pkgs.lib`, or through `import <nixpkgs/lib>`.
<!-- nixdoc-generated documentation must be appended here during build! -->

View File

@@ -1,49 +0,0 @@
# Dev environments {#dev-environments}
Create a `shell.nix` with the following:
```nix
# shell.nix
let
nixpkgs = fetchTarball "https://channels.nixos.org/nixos-unstable/nixexprs.tar.zst";
pkgs = import nixpkgs { };
in
pkgs.mkShell {
packages = [ pkgs.python3 ];
shellHook = ''
echo "Welcome in my nix shell"
'';
}
```
run
```sh
nix-shell
```
This activates your `shell.nix` and you should see:
```sh
unpacking 'https://channels.nixos.org/nixos-unstable/nixexprs.tar.zst' into the Git cache...
Welcome in your nix shell
```
python3 is available
```sh
$ python3 --version
```
To leave the shell
```bash
ctrl+D
```
:::{.note}
You should use [pinned nixpkgs](https://nix.dev/guides/recipes/dependency-management.html).
The example used `unstable` here for demonstration purposes only
:::
For further information check out [nix-shell](https://nix.dev/manual/nix/stable/command-ref/nix-shell)

View File

@@ -1 +0,0 @@
# Getting started {#getting-started}

View File

@@ -3,7 +3,7 @@
This hook defaults a variety of environment variables known
to control thread counts to 1. Many of these otherwise default
to `$(nproc)`, which causes massive overloads on build machines
if nix build jobs and build cores are already tuned to fully use
if nix build jobs and build cores are already tuned to fully utilize
compute capacity of a builder without additional parallelism.
Currently sets the following environment variables:

3
doc/hooks/ghc.section.md Normal file
View File

@@ -0,0 +1,3 @@
# GHC {#ghc}
Creates a temporary package database and registers every Haskell build input in it (TODO: how?).

View File

@@ -4,3 +4,54 @@ Nixpkgs has several hook packages that augment the stdenv phases.
The stdenv built-in hooks are documented in [](#ssec-setup-hooks).
```{=include=} sections
autoconf.section.md
automake.section.md
autopatchcil.section.md
autopatchelf.section.md
aws-c-common.section.md
bmake.section.md
breakpoint.section.md
cernlib.section.md
check-phase-thread-limit-hook.section.md
cmake.section.md
desktop-file-utils.section.md
gdk-pixbuf.section.md
ghc.section.md
gnome.section.md
haredo.section.md
installShellFiles.section.md
installFonts.section.md
julec.section.md
just.section.md
libglycin.section.md
libiconv.section.md
libxml2.section.md
meson.section.md
mpi-check-hook.section.md
ninja.section.md
nodejs-install-executables.section.md
nodejs-install-manuals.section.md
npm-build-hook.section.md
npm-config-hook.section.md
npm-install-hook.section.md
patch-rc-path-hooks.section.md
perl.section.md
pkg-config.section.md
pnpm.section.md
postgresql-test-hook.section.md
premake.section.md
python.section.md
scons.section.md
tauri.section.md
tetex-tex-live.section.md
udevCheckHook.section.md
unzip.section.md
validatePkgConfig.section.md
versionCheckHook.section.md
waf.section.md
writable-tmpdir-as-home-hook.section.md
zig.section.md
xcbuild.section.md
xfce4-dev-tools.section.md
```

View File

@@ -1,47 +0,0 @@
# `installAgentSkills` {#installAgentSkills}
This hook automatically installs LLM agent skills into the proper location in `$out/share/skills/($pname|$base)/$skill/`.
Agents do not scan package outputs themselves. Expose skills via `environment.pathsToLink = [ "/share/skills" ];` and symlink the wanted `share/skills/<pname>/<skill>` directories into the agent's skill directory (e.g. `~/.claude/skills/`).
The automatic behavior of the hook can be disabled by setting the `dontInstallAgentSkills` variable to true.
Additionally, it exposes the `installSkill` function that can be used from `postInstall`
## `installSkill` {#installAgentSkills-installSkill}
The `installSkill` function takes one or two arguments: a directory to copy to the install location, and an optional base directory.
NB: passing a SKILL.md file directly as the first argument will fail as skills often contain other examples and tooling within the same directory.
### Example Usage {#installAgentSkills-installSkill-exampleusage}
```nix
{
nativeBuildInputs = [ installAgentSkills ];
postInstall = ''
installSkill skills/skill-xyz
'';
# installs to $out/share/skills/$pname/skill-xyz
# OR
postInstall = ''
installSkill skills/skill-xyz random-base
'';
# installs to $out/share/skills/random-base/skill-xyz
}
```
Where `skills/skill-xyz` may look like:
```
skills/skill-xyz:
- SKILL.md
- scripts/
- references/
- assets/
- ...
```

View File

@@ -24,10 +24,10 @@ stdenv.mkDerivation {
## Variables controlling `juce.projucerHook` {#juce-projucer-hook-variables}
### `dontUseProjucerConfigure` {#juce-dontuseprojucerconfigure}
### `dontUseProjucerConfigure`
Disables `projucerConfigurePhase`
### `dontUseProjucerInstall` {#juce-dontuseprojucerinstall}
### `dontUseProjucerInstall`
Disables `projucerInstallPhase`

View File

@@ -73,7 +73,7 @@ Controls the flags passed to `cargo tauri build`.
#### `tauriBundleType` {#tauri-bundle-type}
The [bundle type](https://tauri.app/reference/javascript/api/namespaceapp/#bundletype) to build.
The [bundle type](https://tauri.app/v1/guides/building/) to build.
#### `dontTauriBuild` {#dont-tauri-build}

View File

@@ -2,4 +2,4 @@
This setup hook provides a writable home directory for packages that require it.
To use, just add the hook to the `nativeBuildInputs` (or `nativeCheckInputs`, `nativeInstallCheckInputs`, etc.) of the package.
To use, just add the hook to the `nativeBuildInputs` of the package.

View File

@@ -1 +1,5 @@
# Interoperability Standards {#part-interoperability}
```{=include=} chapters
interoperability/cyclonedx.md
```

View File

@@ -39,7 +39,7 @@ The `nix:narinfo` properties should be accompanied by a `nix:store_path` propert
| `nix:narinfo:system` | The hardware and software platform on which this component is produced. |
| `nix:narinfo:sig` | Signatures claiming that this component is what it claims to be. |
| `nix:narinfo:ca` | Content address of this store object's file system object, used to compute its store path. |
| `nix:narinfo:references` | A whitespace-separated array of store paths that this component references. |
| `nix:narinfo:references` | A whitespace separated array of store paths that this component references. |
### `nix:fod` {#sec-interop.cylonedx-fod}

View File

@@ -116,7 +116,7 @@ options:
For each requested system image we can specify the following options:
* `systemImageTypes` specifies what kind of system images should be included.
Defaults to: `google_apis`, `google_apis_playstore`, `google_apis_ps16k` and `google_apis_playstore_ps16k`.
Defaults to: `default`.
* `abiVersions` specifies what kind of ABI version of each system image should
be included. Defaults to `armeabi-v7a` and `arm64-v8a`.

View File

@@ -34,21 +34,17 @@ Inside each package set are:
- builders: mixRelease, buildRebar3, etc
- hooks: for composing builders and packages
The package set is the only place Erlang and Elixir versions are chosen. Builders such as `mixRelease`, `fetchMixDeps` and `buildMix` take them from the set they are called from, and do not accept `erlang`, `elixir` or `hex` arguments.
To use a non-default Elixir, derive a new set with `overrideScope`. This keeps the rest of the set consistent, so every builder picks up the overridden Elixir:
To use a non-default Elixir it's important to keep the rest of the package set consistent, so it's recommended to use `.extend`. This ensures that builders like `mixRelease`, `fetchMixDeps`, and `buildMix` all pick up the overridden Elixir:
```nix
let
beamPackages = beam27Packages.overrideScope (final: prev: { elixir = final.elixir_1_18; });
beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; });
in
beamPackages.mixRelease {
# ...
}
```
`erlang` can be replaced the same way, which is useful for a patched OTP. Every member of the set is built from the set's own `erlang`, so overriding it rebuilds Elixir, Rebar3 and the rest against it.
## Build Tools {#beam-build-tools}
### Rebar3 {#beam-build-tools-rebar3}
@@ -210,7 +206,7 @@ Here is how your `default.nix` file would look for a Phoenix project.
# beam27Packages or beam29Packages is available if you need a particular version
beamPackages,
}:
beamPackages.mixRelease (finalAttrs: {
let
pname = "your_project";
version = "0.0.1";
@@ -219,6 +215,24 @@ beamPackages.mixRelease (finalAttrs: {
rev = "replace_with_your_commit";
};
# if using mix2nix you can use the mixNixDeps attribute
mixFodDeps = beamPackages.fetchMixDeps {
pname = "mix-deps-${pname}";
inherit src version;
# nix will complain and tell you the right value to replace this with
hash = lib.fakeHash;
mixEnv = ""; # default is "prod", when empty includes all dependencies, such as "dev", "test".
# if you have build time environment variables add them here
MY_ENV_VAR = "my_value";
};
in
beamPackages.mixRelease {
inherit
src
pname
version
mixFodDeps
;
# if you have build time environment variables add them here
MY_ENV_VAR = "my_value";
@@ -228,18 +242,7 @@ beamPackages.mixRelease (finalAttrs: {
mix do deps.loadpaths --no-deps-check, phx.digest
mix phx.digest --no-deps-check
'';
# if using mix2nix you can use the mixNixDeps attribute
mixFodDeps = beamPackages.fetchMixDeps {
pname = "mix-deps-${finalAttrs.pname}";
inherit (finalAttrs) src version;
# nix will complain and tell you the right value to replace this with
hash = lib.fakeHash;
mixEnv = ""; # default is "prod", when empty includes all dependencies, such as "dev", "test".
# if you have build time environment variables add them here
MY_ENV_VAR = "my_value";
};
})
}
```
Setup will require the following steps:
@@ -336,8 +339,8 @@ Usually, we need to create a `shell.nix` file and do our development inside the
with pkgs;
let
# pin OTP via beam27Packages/beam28Packages/... and Elixir via overrideScope
beamPackages = beam27Packages.overrideScope (final: prev: { elixir = final.elixir_1_18; });
# pin OTP via beam27Packages/beam28Packages/... and Elixir via .extend
beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; });
in
mkShell { buildInputs = [ beamPackages.elixir ]; }
```
@@ -372,8 +375,8 @@ Here is an example `shell.nix`.
with import <nixpkgs> { };
let
# pin OTP via beam27Packages/beam28Packages/... and Elixir via overrideScope
beamPackages = beam27Packages.overrideScope (final: prev: { elixir = final.elixir_1_18; });
# pin OTP via beam27Packages/beam28Packages/... and Elixir via .extend
beamPackages = beam27Packages.extend (self: super: { elixir = self.elixir_1_18; });
# define packages to install
basePackages = [

View File

@@ -1,32 +1,8 @@
# CHICKEN {#sec-chicken}
[CHICKEN](https://call-cc.org/) is a Scheme compiler. It includes an
interactive mode and a custom package format, "eggs". CHICKEN 5 is
[R⁵RS](https://schemers.org/Documents/Standards/R5RS/HTML/)-compliant, whereas
CHICKEN 6 targets
[R⁷RS](https://standards.scheme.org/official/r7rs.pdf).
## Package Sets {#sec-chicken-package-sets}
Each major release of CHICKEN has its own package set, because eggs are
compiled against a specific binary version of the compiler and cannot be shared
between releases:
* `chickenPackages_5` — CHICKEN 5, and the set the unversioned attributes
(`chicken`, `chickenPackages`, `eggDerivation`, `fetchegg`) point at.
* `chickenPackages_6` — CHICKEN 6.
* `chickenPackages_4` — CHICKEN 4; kept for the few packages that still need
it and maintained on a best-effort basis.
Each set provides `chicken`, `eggDerivation`, `fetchegg` and an `chickenEggs`
attrset of eggs published for that release. Note that upstream has not ported
every egg to CHICKEN 6 yet, so `chickenPackages_6.chickenEggs` is considerably
smaller than its CHICKEN 5 counterpart.
CHICKEN 5 and 6 are built from the same expressions, in
`pkgs/development/compilers/chicken/common/`; the release directories next to
it hold only what differs between releases, which is the compiler's version and
its egg set. CHICKEN 4 predates that arrangement and stands on its own.
[CHICKEN](https://call-cc.org/) is a
[R⁵RS](https://schemers.org/Documents/Standards/R5RS/HTML/)-compliant Scheme
compiler. It includes an interactive mode and a custom package format, "eggs".
## Using Eggs {#sec-chicken-using}
@@ -50,43 +26,29 @@ variables `CHICKEN_INCLUDE_PATH` and `CHICKEN_REPOSITORY_PATH`.
## Updating Eggs {#sec-chicken-updating-eggs}
For CHICKEN 5 and 6, the egg set is generated from upstream's list of latest
egg releases, so there is no list of eggs to curate in Nixpkgs: every egg
published for that release is included. The generated metadata lives in the
release directory's `deps.toml`, and is regenerated by running the shared
`update.sh` script with the release to update:
Nixpkgs only knows about a subset of all published eggs. It uses
[egg2nix](https://github.com/the-kenny/egg2nix) to generate a
package set from a list of eggs to include.
The package set is regenerated by running the following shell commands:
```
$ cd pkgs/development/compilers/chicken/common/
$ ./update.sh 6
```
This clones upstream's `eggs-<major>-latest` repository, prefetches each egg
tarball and converts the egg metadata into `deps.toml`. Do not edit
`deps.toml` by hand.
Eggs that need extra native dependencies or other fixups are patched up in
`overrides.nix` in the release directory; the entries there are keyed by egg
name and applied automatically by `eggDerivation`.
## Adding Eggs {#sec-chicken-adding-eggs}
CHICKEN 4 predates the workflow above: its much smaller set is generated with
[egg2nix](https://github.com/the-kenny/egg2nix) from a hand-written list of
eggs, so eggs have to be added explicitly. The list is
`pkgs/development/compilers/chicken/4/eggs.scm`; the first section lists eggs
which are required by `egg2nix` itself, all other eggs go into the second
section. After editing, regenerate the set:
```
$ nix-shell -p chickenPackages_4.egg2nix
$ cd pkgs/development/compilers/chicken/4/
$ nix-shell -p chickenPackages.egg2nix
$ cd pkgs/development/compilers/chicken/5/
$ egg2nix eggs.scm > eggs.nix
```
`egg2nix` resolves one collection of eggs with mutually-compatible versions, so
adding an egg may update existing ones. To keep those changes separate,
regenerate the set before adding more eggs.
## Adding Eggs {#sec-chicken-adding-eggs}
When we run `egg2nix`, we obtain one collection of eggs with
mutually-compatible versions. This means that when we add new eggs, we may
need to update existing eggs. To keep those separate, follow the procedure for
updating eggs before including more eggs.
To include more eggs, edit `pkgs/development/compilers/chicken/5/eggs.scm`.
The first section of this file lists eggs which are required by `egg2nix`
itself; all other eggs go into the second section. After editing, follow the
procedure for updating eggs.
## Override Scope {#sec-chicken-override-scope}

View File

@@ -5,7 +5,7 @@
COSMIC (Computer Operating System Main Interface Components) is a desktop environment developed by
System76, primarily for the Pop!_OS Linux distribution. Applications in the COSMIC ecosystem are
written in Rust and use libcosmic, which builds on the Iced GUI framework. This section explains
how to package and integrate COSMIC applications within Nix.
how to properly package and integrate COSMIC applications within Nix.
### libcosmicAppHook {#ssec-cosmic-libcosmic-app-hook}
@@ -17,7 +17,7 @@ and wrapping applications based on libcosmic. It handles many common requirement
- Managing Vergen environment variables for build-time information
- Setting up Rust linker flags for specific libraries
Add the hook to your package's `nativeBuildInputs`:
To use the hook, simply add it to your package's `nativeBuildInputs`:
```nix
{
@@ -61,9 +61,8 @@ rustPlatform.buildRustPackage {
}
```
> [!Note]
> `cosmic-settings` is a separate application and not a part of the libcosmic settings
> system itself. It's included by default in `libcosmicAppHook` only to provide these fallback theme
Note that `cosmic-settings` is a separate application and not a part of the libcosmic settings
system itself. It's included by default in `libcosmicAppHook` only to provide these fallback theme
settings.
### Icons {#ssec-cosmic-icons}

Some files were not shown because too many files have changed in this diff Show More