Using a `tree:0` filter instead of `blob:none` reduces the checkout time
from over 3 minutes to about 45 seconds. The required trees/blobs will
then be fetched on-demand.
This on-demand fetching creates additional output for `git range-diff`
on stderr, so we hide that. This only happens the first time it's run,
so we don't need to adjust the other calls - which will still return any
real errors, should they happen.
The default is to checkout a contributors fork as "origin", thus the
NixOS/nixpkgs remote is most likely named differently. But not everybody
keeps their fork's main branches up-to-date all the time. Thus the
script would fail locally.
We recently moved the $commits variable out of a "subshell in a
herestring", let's do the same for the list of branches, where errors
would be silently swallowed as well.
Also reformat the expressions slightly, we have enough line-length.
This moves the diff of outpaths into the outpaths job, mainly as a
preparation to allow future improvements. For example, this will allow
running the purity release checks only on changed outpaths instead of
the whole eval.
This also removes the inefficiency introduced in the last commit about
uploading the intermediate paths twice. Now, only the diff is passed on.
Also, technically, the diff is now run in parallel across 4 jobs. This
should be *slightly* faster than before, where outpaths from all systems
were combined first and then diffed. It's probably only a few seconds,
though.
Everything is a result, especially when nix-build uses "result" as its
default output. This becomes confusing, when re-wiring the different
parts later.
Thus, consistently name those things after some of their properties and
avoid the term result.
We have added nixpkgs-vet as a regular package to nixpkgs a while ago,
so we can now use it from pinned nixpkgs. This avoids pulling a
platform-specific binary version from upstream.
This change also allows to run the tool easily locally, the same way as
other tools:
nix-build ci -A nixpkgs-vet
This will do a full check of the repo with the exception of
nixpkgs-vet's "ratchet" checks: Those depend on having two branches to
compare, but the default is to only look at the head branch. Those
ratchet checks will still be run in CI, though.
I think it makes sense for the `stdenv` team to handle review of
changes to these libraries that have substantial effects on the
things in their scope, like the semantics of `stdenv.mkDerivation`,
override interfaces, `meta` handling, and so on. This is already how
`lib/systems/` is handled, and it makes sense to me to extend it.
This would hopefully have helped avoid situations
like <https://github.com/NixOS/nixpkgs/pull/403678> and
<https://github.com/NixOS/nixpkgs/pull/410179>, where the `stdenv`
team was unaware of the original change for weeks.
Instead of parsing a plain text file with jq, we can make nix-env output
JSON directly, which is significantly faster.
This saves about 8 out of 10 seconds for the combine step.
compare/maintainers.nix needs to access the current checkout to check
attrpaths, but makes the mistake of using lib from that checkout as
well. All other code in ci/ uses the pinned nixpkgs instance, so
maintainers.nix should do so as well.
Previously, `eval.full` organized the results for the supported systems
in a specific layout, i.e. with a folder with one subfolder per system.
Then, `eval.combine` relied on that.
When using `eval.singleSystem` and `eval.combine` directly, the caller
was responsible to recreate the same layout. This is annoying and
error-prone to do, when downloading artifacts from CI to recreate some
steps locally.
With this change, all the artifacts can be downloaded and extracted into
the same folder - because the result from `eval.singleSystem` already
contains the <system-name>/ subfolder.
As one of the resident Nixpkgs licensing pedants and reviewer of
several recent changes to this file I think it makes sense to make
the de facto the de jure.
We don't need a separate workflow anymore, because we don't need to skip
dependent jobs on failures anymore. The biggest failure mode was
"conflict" previously, but we resolved that on the last commit. The
remaining failure modes are so rare, that it's OK to just fail the jobs
in this case instead of marking them as "skipped". Especially, because
the resolve-merge-commit job would have previously failed anyway.
By moving this to an action we avoid running separate jobs each time we
need the merge commit. This also makes the check list in PRs much
cleaner.
The reason this was a separate shell script was, that this would be
included in multiple workflows separately. But a while ago this had been
changed to a re-usable workflow, so we can just as well inline the
script.
This also allows us to use actions/github-script, which makes for a much
more readable script than the bash script before.
We really can't expect packages that are marked as broken to evaluate,
and *especially* not on unsupported platforms.
For context, we were attempting to eval them *past* the broken throw
previously, which caused fun side effects like [0].
When we set `includeBroken = true` before, this also included unfree
packages. Those would now be excluded, which is not what we want. Thus,
we explicitly enable them separately.
Commit by winterqt, message slightly reworded by wolfgangwalther.
[0]:
https://github.com/NixOS/nixpkgs/issues/355847#issuecomment-2878873137
Make teleport built by the by-name structure in package.nix.
The versioned attributes are defined in their own package.nix
in by-name using overrides.
Also change reference in ci/OWNERS
While OfBorg is still adding these, it takes a much longer time to do so
compared to the eval action. Since we're adding rebuild labels, I think
it'd be nice to just do it within the eval action.
Right now, there are some paths that don't even get exposed to certain
systems (notably Darwin, but some outliers exist for Linux such as the
Darwin-specific Hackage overlay) for one reason or another, usually
because of assertions like `stdenv.isLinux`. To catch these scenarios,
this change implements a way to specify the system to evaluate attrpaths
on, and makes it default to the system that we're evaluating outpaths
for.
This was run on .nix files only, but we recently added keep-sorted,
editorconfig-checker and actionlint to treefmt, so CI needs to check all
files instead.
Previously, the attrs step consisted of:
- 7s queue time
- 1m 15s run time
Only 25s of this were spent preparing the attr paths. A bit more than a
minute was just spent for queuing, checking out the repo, downloading
nix, downloading dependencies, uploading the artifacts - and then
downloading them again in the next step. All of that can be avoided if
we collect the attrs as part of the outpaths job.
By running the attrs step as part of each outpaths step the attrpaths
will be collected 4x, but:
- We save a minute for each eval run to complete.
- We save a full job, giving us more free runners and *possibly* less
queue times for other jobs in the repo.
- We reduce complexity in the workflow file.
This adds the minimum nix version and the latest lix version to the
matrix of parse checks. Especially the minimum nix version is relevant,
because parsing routinely breaks because of introduction of newer
syntax.
Adding lix just completes the picture.
The nix-parse workflow can now be run locally the same way as in CI.
To do this, the CI's workflow was slightly adjusted. Instead of testing
only the changed files, we're now testing all files in the repository.
This is possible in two ways:
1. By calling nix-instantiate once with all files as arguments. This
will be rather fast, but only the first error is shown before it errors
out.
2. By calling nix-instantiate once for each file. This will be much
slower, but has the advantage that we see all errors at once.
To avoid running the long variant every time, we first do a quick check
with the fast version. If that fails, we run the slower one to report
the errors. This gives us the best of both.
I added a lint-action.sh script in .github/workflows a while ago while
fixing some warnings. But I haven't run it myself ever since. This needs
to be part of CI to make any use of it.
