The conclusion of the `versions` job propagates from through `eval` to
`compare`, which meant the `compare` job was skipped. No rebuild labels,
no reviewer requests.
Also, we don't want to run eval when `versions` runs, but fails.
The additional `workflows` permissions are required to backport
Dependabot updates. The permissions had been added to the app a while
ago, but we forgot to actually use them.
With this change, we start running Eval on all available Lix and Nix
versions. Because this requires a lot of resources, this complete test
is only run when `ci/pinned.json` is updated.
The resulting outpaths are checked for consistency with the target
branch. A difference will cause the `report` job to fail, thus blocking
the merge, ensuring Eval consistency for Nixpkgs across different
versions.
This implements a kind of "ratchet style" check: Since we originally
confirmed that the versions currently in Nixpkgs at the time of this
commit match Eval behavior of Nix 2.3, we can ensure consistency with
Nix 2.3 down the road, even without testing for it explicitly.
There had been one regression in Eval consistency for Nix between 2.18
and 2.24 - two tests in `tests.devShellTools` produce different results
between Lix 2.91+ (which was forked from Nix 2.18) and Nix 2.24+. I
assume it's unlikely that such a change would be "fixed" by now, thus I
added an exception for these.
As a bonus, we also present the total time in seconds it takes for Eval
to complete for every tested version in a summary table. This allows us
to easily see performance improvements for Eval due to version updates.
At this stage, this time only includes the "outpaths" step of Eval, but
not the generation of attrpaths beforehand.
The Dependabot update change the hashes to the latest main branch commit
instead of the v5.0.0 tag - also it didn't adjust the tags in the
comments accordingly. Last but not least, one of the references used a
`@v5` reference instead of the commit hash. The latter is probably what
Dependabot tripped on.
This is slightly faster than downloading and extracting a tarball and
additionally allows a sparse checkout. No need to download docs or nixos
for our purpose.
The data is quite noisy, but suggests improvements from anywhere between
5-15 seconds for each job using the pinned nixpkgs.
None of our jobs is expected to run for 6 hours, the GitHub limit. These
limits are generous and take into accounts that some jobs need to wait
for others.
If jobs exceed these times, most likely something else is wrong and
needs investigation.
This reverts commit f2648b263b.
While the idea to never use swap was fine, in practice this meant that
when nix ran OOM, some other process was killed instead. This lead to
the job not being possible to be cancelled anymore and thus needing to
timeout, before subsequent jobs could be scheduled. This can take up to
6 hours for GitHub Actions by default.
Re-enabling the swap file to catch this case more gracefully. It's still
the goal to never actually *use* the swap file during Eval and just a
safeguard.
Keeping the changed chunkSize and not reverting it - this makes it
slightly less likely to hit the swap file when running with Lix.
Recent performance tests show that (a) swapping heavily slows down the
Eval job, while (b) lowering the chunkSize does not have an effect on
run-time. It does on memory usage, though - thus we can get rid of
swapping entirely by reducing chunkSize respectively.
Introduces a basic merge queue workflow to initially only run lints.
This will avoid accidentally merging changes which break nixfmt after
its recent update to 1.0.0.
This adds a build job for the tarball, which might help uncover eval
issues on attributes not normally touched by Eval, aka those added in
`pkgs/top-level/packages-config.nix`.
This doesn't really change the items, but shortens the writing a lot,
making them much more readable, especially while still drafting the PR.
Mentioning the release notes for the previous release is not really
important, because it doesn't apply for the big majority of pull
requests.
This *is* important, but is a niche detail which belongs into the
contributions guidelines - and not into the PR template. It's also out
of place for "what did you test?".
We expect the TODO list to be read through *on PR creation*, otherwise
the html comments would not make sense. Thus, we should make it even
only slightly readable, which was not at all the case before.
The links for release notes are removed, because the PR author has no
value from the *current* release notes. They will need to find the file
manually anyway.
While this note is important, it's also mostly invisible at this stage.
The comment only shows while creating the PR, but at this stage the
author really has other things to worry about.
If they care, they will read the contribution guidelines and will pick
up the pieces about reviewing that way. If they don't - they won't be
bothered by this notice either.
This checkbox has been used very incosistently and its meaning is not
entirely clear: What does it mean if both checkboxes are *unchecked*?
Does that mean the sandbox was disabled? Or does it mean the checkbox
was just not handled?
Also the new nixpkgs-review-gha, which is increasingly used to test
builds on darwin platforms shows this information as part of the review
- where it's in a much better place.
Most of the checks we do for cherry-picks are dismissable warnings, with
one exception: When a commit hash has been found, but this hash is not
available in any of the pickable branches, we raise this with
severity=error. This should also *block* the merge and not be
dismissable. That's because this is a fixable issue in every case.
This turns the check-cherry-pick script into a github-script based
JavaScript program. This makes it much easier to extend to check reverts
or merge commits later on.
Since all github-scripts need to be written in commonjs, we now default
to it by not setting package.json. Support from editors for .js files is
slightly better than .cjs. To still allow using module imports in the
test runner script, we trick node into loading the script itself as a
module again via `--import ./run`.
This just moves things around to use less specific naming - `labels` is
only *one* script that can potentially be run locally while still being
written in github-script. Later, we can add more.
When a PR is merged and labeled afterwards - with a non-backport label -
the following will happen:
- The first backport job is triggered on the merge.
- The second backport job is triggered on the label event.
- The second job will cancel the first one due to the concurrency group.
- The second job will cancel itself because the label event didn't
contain a backport label.
Both jobs end up cancelled and no backport happens.
We made the backport action idempotent upstream a while ago, so we don't
need to cancel those actions. Instead, we'll run all of them -
subsequent actions running through will just stay silent anyway.
Committers could get the false impression from, e.g., `PR / Build / aarch64-linux` that this workflow builds the packages changed in the current PR. Such a misunderstanding could pair poorly with the "enable auto-merge" button, once that's enabled.
