Portable test suite for modular services, designed to be usable with
other service managers and use cases, inside Nixpkgs and externally.
Like modular services itself, it's still quite bare-bones. New
features can crystallize around this kind of infrastructure.
Assisted-By: Claude Opus 4.7/4.8 era Claude models
Allows ignoring file metadata differences (permissions, ownership)
when comparing files.
This is especially useful on darwin, where we often run into subtle
issues like:
-Device: 1,23 Access: (0444/-r--r--r--) Uid: ( 0/ root) Gid: ( 0/ wheel)
+Device: 1,23 Access: (0444/-r--r--r--) Uid: ( 0/ root) Gid: ( 350/ nixbld)
Add an optional postFailureMessage parameter to testEqualContents that allows
users to provide additional context when tests fail. This is particularly useful
for providing instructions on how to update expected results when they change
intentionally.
The message is displayed after the standard failure output, helping maintainers
understand what to do next when a test fails.
Format all Nix files using the officially approved formatter,
making the CI check introduced in the previous commit succeed:
nix-build ci -A fmt.check
This is the next step of the of the [implementation](https://github.com/NixOS/nixfmt/issues/153)
of the accepted [RFC 166](https://github.com/NixOS/rfcs/pull/166).
This commit will lead to merge conflicts for a number of PRs,
up to an estimated ~1100 (~33%) among the PRs with activity in the past 2
months, but that should be lower than what it would be without the previous
[partial treewide format](https://github.com/NixOS/nixpkgs/pull/322537).
Merge conflicts caused by this commit can now automatically be resolved while rebasing using the
[auto-rebase script](8616af08d9/maintainers/scripts/auto-rebase).
If you run into any problems regarding any of this, please reach out to the
[formatting team](https://nixos.org/community/teams/formatting/) by
pinging @NixOS/nix-formatting.
Fixes `testers.testBuildFailure` compatibility with `__structuredAttrs`.
The two main issues are, when `structuredAttrs` is enabled:
- `$__structuredAttrs` is not set
- `$outputs` is not set
`$__structuredAttrs` not being set is fixed by checking for
`$NIX_ATTRS_SH_FILE`, as per pkgs/stdenv/generic/setup.sh
`$outputs` not being set is fixed by sourcing `$NIX_ATTRS_SH_FILE`,
as-per pkgs/stdenv/generic/default-builder.sh
This deals with some special cases, i.e. where "null" is used to avoid
replacing some values or where the syntax is not too easy to grep for
for automated replacement.
Before this change, testers.testEqualContents implementation had several
bugs (e.g. executables at different paths were not considered equal). So
we switch to diffoscope that that is designed to handle exactly these
kinds of comparisons and gives more insights into the differences in the
output.
Closes#193336Closes#261694
Related to #108984
The goal here was to get the following flake to build and run on
`aarch64-darwin`:
```nix
{ inputs.nixpkgs.url = <this branch>;
outputs = { nixpkgs, ... }: {
checks.aarch64-darwin.default =
nixpkgs.legacyPackages.aarch64-darwin.nixosTest {
name = "test";
nodes.machine = { };
testScript = "";
};
};
}
```
… and after this change it does. There's no longer a need for the
user to set `nodes.*.nixpkgs.pkgs` or
`nodes.*.virtualisation.host.pkgs` as the correct values are inferred
from the host system.
This is very useful in conjunction with meta.pkgConfigModules, as the
new tester can use the list provided by this meta attribute as a default
value for moduleNames, making its usage in passthru.tests very
convenient.
For backwards compatibility, a shim under the old name is maintained
with a warning.
When lib overrides were used, before this commit, they would not be made
available in the configuration evaluation of nixosTest's nodes.
Sample code:
``` nix
let
pkgs = import ./. {
overlays = [
(new: old: {
lib = old.lib.extend (self: super: {
sorry_dave = builtins.trace "There are no pod bay doors" "sorry dave";
});
})
];
};
in
pkgs.testers.runNixOSTest {
name = "demo lib overlay";
nodes = {
machine = { lib, ... }: {
environment.etc."got-lib-overlay".text = lib.sorry_dave;
};
};
testScript = { nodes }:
''
start_all()
machine.succeed('grep dave /etc/got-lib-overlay')
'';
}
```
See docs.
Follow-up work:
- Existing packages should be converted
- `defaultPkgConfigPackages` should assert on `meta.pkgConfigModules`
and let `tests.pkg-config` alone test the build results.
CC @sternenseemann
Co-authored-by: Robert Hensing <roberth@users.noreply.github.com>
Fixes the problem introduced by 12b3066aae
which caused nixos/release.nix to return the wrong attributes, while
intending to only affect nixos/lib's runTest.
This also removes callTest from the test options, because callTest is
only ever invoked by all-tests.nix.
Examples: support has to be added to https://github.com/nix-community/nixdoc
'nixdoc --category testers --description "nixpkgs testers" --file ./pkgs/build-support/testers/default.nix'
