ci: run the lib tests under Lix

The lib tests exercise things like `nix-instantiate` and their error
messages. Therefore, they can regress when the wording is too coupled to
a specific version or implementation.

Given that Lix is already easy to reach out in nixpkgs and
`lib/tests/test-with-nix.nix` is consumed by Lix's own CI, we propose to
run the suite under `pkgs.lix` too.

The default `lib-tests` output stays Nix-only because it is part of the
release tarball closure; `lib-tests-lix` is a separate CI job and not
part of the release tarball closure.

As far as I can tell:

- that extra test takes ~55s on my system under Lix in the sandbox. Over
  a faster builder, it takes ~20s.
- this is a new *step* in an existing job, no new checkout, no new
  install or whatever, so no extra setup cost.
- the symlink join do not cause rebuild of the nix-unit things and so
  on. pkgs.lix is substituted and not rebuilt.
- the existing lib-tests are already doing 2 runs of testWithNix, Lix is
  a 3rd.

If this proves to be a problem for CI performance, we can always revert
the long part (fileset) and decrease even further the cost of this
additional minute check.
This commit is contained in:
Raito Bezarius
2026-10-15 02:35:03 +02:00
parent 48cb369917
commit db996807d1
2 changed files with 13 additions and 2 deletions

View File

@@ -35,8 +35,8 @@ jobs:
include:
- runner: ubuntu-24.04
system: x86_64-linux
builds: [shell, manual-nixos, lib-tests, tarball]
desc: shell, docs, lib, tarball
builds: [shell, manual-nixos, lib-tests, lib-tests-lix, tarball]
desc: shell, docs, lib, lib-lix, tarball
- runner: ubuntu-24.04-arm
system: aarch64-linux
builds: [shell, manual-nixos, manual-nixpkgs]
@@ -93,6 +93,10 @@ jobs:
if: contains(matrix.builds, 'lib-tests') && !cancelled()
run: nix-build-uncached nixpkgs/untrusted/ci --arg nixpkgs ./nixpkgs/untrusted-pinned -A lib-tests
- name: Build lib tests (Lix)
if: contains(matrix.builds, 'lib-tests-lix') && !cancelled()
run: nix-build-uncached nixpkgs/untrusted/ci --arg nixpkgs ./nixpkgs/untrusted-pinned -A lib-tests-lix
- name: Build tarball
if: contains(matrix.builds, 'tarball') && !cancelled()
run: nix-build-uncached nixpkgs/untrusted/ci --arg nixpkgs ./nixpkgs/untrusted-pinned -A tarball

View File

@@ -63,6 +63,13 @@ rec {
# CI jobs
lib-tests = import ../lib/tests/release.nix { inherit pkgs; };
# The lib tests may assert on specific error messages which are
# implementation-defined. Run them under Lix too so that nixpkgs do not
# couple its tests too hard to specific wordings.
lib-tests-lix = import ../lib/tests/release.nix {
inherit pkgs;
nixVersions = [ pkgs.lix ];
};
manual-nixos = (import ../nixos/release.nix { }).manual.${system} or null;
manual-nixpkgs = (import ../doc { pkgs = docPkgs; });
nixpkgs-vet = pkgs.callPackage ./nixpkgs-vet.nix {