linux: new workflow for kernel changes

See https://github.com/NixOS/nixpkgs/issues/437208#issuecomment-3288623669
Depends on https://github.com/NixOS/org/pull/172

As documented below, the idea is to essentially group all changes
rebuilding all VM tests with kernel updates and merge them together into
`master` whenever the Linux kernels get updated.

This documents the workflow of updates in the nixpkgs manual. While at
it, I removed the README from the packages because

* it's horribly outdated
* I didn't even know it exists which confirms that its discoverability
  was very poor

and added the relevant portions into the nixpkgs manual as well.

(cherry picked from commit 9f95c2ace2)
This commit is contained in:
Maximilian Bosch
2025-09-21 15:37:07 +02:00
committed by github-actions[bot]
parent 7ba68e6c4e
commit 0db33960bb
6 changed files with 96 additions and 55 deletions

View File

@@ -35,6 +35,8 @@ jobs:
into: staging-next
- from: staging-next
into: staging
- from: master
into: staging-nixos
uses: ./.github/workflows/periodic-merge.yml
with:
from: ${{ matrix.pairs.from }}

View File

@@ -423,20 +423,22 @@ gitGraph
Here's an overview of the different branches:
| branch | `master` | `staging-next` | `staging` |
| --- | --- | --- | --- |
| Used for development | ✔️ | ❌ | ✔️ |
| Built by Hydra | ✔️ | ✔️ | ❌ |
| [Mass rebuilds][mass-rebuild] | ❌ | ⚠️ Only to fix Hydra builds | ✔️ |
| Critical security fixes | ✔️ for non-mass-rebuilds | ✔️ for mass-rebuilds | ❌ |
| Automatically merged into | `staging-next` | `staging` | - |
| Manually merged into | - | `master` | `staging-next` |
| branch | `master` | `staging-next` | `staging` | [`staging-nixos`](#test-driver-rebuild) |
| --- | --- | --- | --- | --- |
| Used for development | ✔️ | ❌ | ✔️ | ✔️ |
| Built by Hydra | ✔️ | ✔️ | ❌ | ❌ |
| [Mass rebuilds][mass-rebuild] | ❌ | ⚠️ Only to fix Hydra builds | ✔️ | ❌[^1] |
| Critical security fixes | ✔️ for non-mass-rebuilds | ✔️ for mass-rebuilds | ❌ | ✔️ |
| Automatically merged into | `staging-next` & `staging-nixos` | `staging` | - | - |
| Manually merged into | - | `master` | `staging-next` | `master` |
The staging workflow is used for all main branches, `master` and `release-YY.MM`, with corresponding names:
- `master`/`release-YY.MM`
- `staging`/`staging-YY.MM`
- `staging-next`/`staging-next-YY.MM`
[^1]: Except changes that cause no more rebuilds than kernel updates
# Conventions
## Branch conventions
@@ -488,6 +490,26 @@ In order to help the decision, CI automatically assigns [`rebuild` labels](https
As a rule of thumb, if the number of rebuilds is **over 500**, it can be considered a mass rebuild.
To get a sense for what changes are considered mass rebuilds, see [previously merged pull requests to the staging branches](https://github.com/NixOS/nixpkgs/issues?q=base%3Astaging+-base%3Astaging-next+is%3Amerged).
Please note that changes to the Linux kernel are an exception to this rule.
These PRs go to `staging-nixos`, see [the next section for more context](#changes-rebuilding-all-tests).
### Changes rebuilding all NixOS tests
[test-driver-rebuild]: #changes-rebuilding-all-tests
Changes causing a rebuild of all NixOS tests get a special [`10.rebuild-nixos-tests`](https://github.com/NixOS/nixpkgs/issues?q=state%3Aopen%20label%3A10.rebuild-nixos-tests) label.
These changes pose a significant impact on the build infrastructure.
Hence, these PRs should either target a `staging`-branch or `staging-nixos`, provided one of following conditions applies:
* The label `10.rebuild-nixos-tests` is set, or
* The PR is a change affecting the Linux kernel.
The branch gets merged whenever mainline kernel updates or critical security fixes land on the branch.
This usually happens on a weekly basis.
Backports are not handled by such a branch.
The relevant PRs from this branch must be backported manually.
## Commit conventions
[commit-conventions]: #commit-conventions

View File

@@ -8,6 +8,7 @@ const typeConfig = {
release: ['development', 'primary'],
staging: ['development', 'secondary'],
'staging-next': ['development', 'secondary'],
'staging-nixos': ['development', 'secondary'],
'haskell-updates': ['development', 'secondary'],
nixos: ['channel'],
nixpkgs: ['channel'],
@@ -19,6 +20,7 @@ const orderConfig = {
master: 0,
release: 1,
staging: 2,
'staging-nixos': 2,
'haskell-updates': 3,
'staging-next': 4,
}

View File

@@ -105,3 +105,56 @@ $ make -C $dev/lib/modules/*/build M=$(pwd)/drivers/net/ethernet/mellanox module
```
:::
## Maintainer information {#sec-linux-kernel-maintainer-information}
### Updating kernels {#sec-linux-updates}
Updating all kernels can be done with the following script:
```ShellSession
$ pkgs/os-specific/linux/kernel/update.sh
```
The change gets submitted like this:
* File a PR against `staging-nixos`.
* Add a `backport release-XX.XX` label for an automated backport.
We don't expect many other changes on that branch to require a backport, hence there's no such branch for stable.
By using an additional PR, we get the automatic backport against stable without manual cherry-picks.
* Merge into `staging-nixos`.
* File as PR from `staging-nixos` against `master`.
* When all status checks are green, merge.
### Add a new (major) version of the Linux kernel {#sec-linux-add-new-kernel-version}
* When running `./pkgs/os-specific/linux/kernel/update.sh`, new kernel majors get discovered automatically.
* Prepare all Nix expressions for the new kernel
* Instantiate the new kernel in `pkgs/top-level/linux-kernels.nix` in the `kernels`-section.
```nix
{
linux_X_Y = callPackage ../os-specific/linux/kernel/mainline.nix {
branch = "X.Y";
kernelPatches = [
# any new patches required (it makes to look which patches are used by its predecessor)
];
};
}
```
* Instantiate the package-set in `vanillaPackages`:
```nix
{
linux_X_Y = recurseIntoAttrs (packagesFor kernels.linux_X_Y);
}
```
* Update `linux_latest` to the new attribute.
* __SQUASH__ the changes into the `linux: init at …` commit.
* If a new hardened is available:
* Instantiate a `linux_X_Y_hardened = hardenedKernelsFor kernels.linux_X_Y { };` in `kernels` and
`linux_X_Y_hardened = hardenedKernelFor kernels.linux_X_Y { };` in the `packages`-section.
* Make sure to remove the hardened variant of the previous kernel version unless it's LTS.
We only support the latest and latest LTS version of hardened.
* If no new hardened kernel is available:
* Keep the previously latest kernel until its mainline counterpart gets removed.
After that `linux_hardened` points to the latest LTS supported by hardened.
* __SQUASH__ the changes into the `linux_X_Y_hardened: init at …` commit.

View File

@@ -4195,6 +4195,15 @@
"sec-linux-kernel": [
"index.html#sec-linux-kernel"
],
"sec-linux-kernel-maintainer-information": [
"index.html#sec-linux-kernel-maintainer-information"
],
"sec-linux-updates": [
"index.html#sec-linux-updates"
],
"sec-linux-add-new-kernel-version": [
"index.html#sec-linux-add-new-kernel-version"
],
"ex-overriding-kernel-derivation": [
"index.html#ex-overriding-kernel-derivation"
],

View File

@@ -1,47 +0,0 @@
# How to add a new (major) version of the Linux kernel to Nixpkgs:
1. Copy the old Nix expression (e.g., `linux-2.6.21.nix`) to the new one (e.g., `linux-2.6.22.nix`) and update it.
2. Add the new kernel to the `kernels` attribute set in [`linux-kernels.nix`](../../../top-level/linux-kernels.nix) (e.g., create an attribute `kernel_2_6_22`).
3. Update the kernel configuration:
1. While in the Nixpkgs repository, enter the development shell for that kernel:
```console
$ nix-shell -A linuxKernel.kernels.linux_2_6_22
```
2. Unpack the kernel:
```console
[nix-shell]$ pushd $(mktemp -d)
[nix-shell]$ unpackPhase
```
3. For each supported platform (`i686`, `x86_64`, `uml`) do the following:
1. Make a copy from the old config (e.g., `config-2.6.21-i686-smp`) to the new one (e.g., `config-2.6.22-i686-smp`).
2. Copy the config file for this platform (e.g., `config-2.6.22-i686-smp`) to `.config` in the unpacked kernel source tree.
3. Run `make oldconfig ARCH={i386,x86_64,um}` and answer all questions. (For the uml configuration, also add `SHELL=bash`.) Make sure to keep the configuration consistent between platforms (i.e., dont enable some feature on `i686` and disable it on `x86_64`).
4. If needed, you can also run `make menuconfig`:
```ShellSession
$ nix-shell -p ncurses pkg-config
$ make menuconfig ARCH=arch
```
5. Copy `.config` over the new config file (e.g., `config-2.6.22-i686-smp`).
4. Test building the kernel:
```ShellSession
nix-build -A linuxKernel.kernels.kernel_2_6_22
```
If it compiles, ship it! For extra credit, try booting NixOS with it.
5. It may be that the new kernel requires updating the external kernel modules and kernel-dependent packages listed in the `linuxPackagesFor` function in `linux-kernels.nix` (such as the NVIDIA drivers, AUFS, etc.). If the updated packages arent backwards compatible with older kernels, you may need to keep the older versions around.