The performance comparison summary previously stated it compared against
"its pull request base branch (e.g., 'master')" regardless of the actual
base branch.
Plumb the base branch classification computed in prepare.js through the
eval workflow to the `compare` derivation so the summary names the real
base branch (e.g. "staging-25.11"). The local `eval.full` helper defaults
to "master", matching its touched-files convention.
Assisted-by: claude-code with claude-opus-4-8[1m]-high
These jobs were recently introduced into the merge queue - but I forgot
to make them actually required. They sometimes fail - but the PR is
still merged. Dumb.
This is in public preview now. These runners run in a docker container
with only a single vCPU instead of 4 like the other jobs. For most of
our jobs, this should be plenty, except for eval and linting.
Since it was previously possible, in theory, to extract the
CACHIX_AUTH_TOKEN via Nix eval on untrusted inputs, this rotates the
token and clears the cache - and while on it moves to a new cache, that
is owned by a newly created nixpkgs-ci org instead of an individual.
Evaluating untrusted code in the presence of secrets is unsafe in
general, thus we only provide the cachix auth token when these jobs run
in the merge queue. This is enough for all practical purposes, PRs will
be able to pull stuff from cachix that was built in the Merge Queue
previously.
This runs the compare step in the merge queue, to get a combined diff
against the previous queue item. This can be used as a base for multiple
different features in the future, for example preventing accidental mass
rebuilds in the merge queue, or diffing of packages' meta.
Once we start running similar jobs in the PR and Merge Queue workflows,
the Test workflow will run into a problem: Both nested workflows will
try to upload the same artifacts, which errors out.
By providing a prefix to use for artifact names, this will be a
non-issue when we later run more of Eval in the merge queue or when we
add the build jobs to the queue as well.
This change runs Lix on the target commit and Nix on the merged commit.
This does two things for us at once:
- We test both Lix and Nix.
- We ensure that both Lix and Nix produce the same output hashes.
If Lix and Nix were to produce different output hashes at some point,
this would show up as rebuilds in every PR.
Running eval in the merge queue prevents eval failures after merging
multiple PRs with semantic merge conflicts.
It's also the base for allowing more sophisticated checks about rebuild
counts in the merge queue later.
When branches are directly pushed to, such as for periodic merges, this
will not run Eval immediately. However, the next PR will run it as part
of its Eval step and will cache the results. Any further PR will then
benefit from the same caching again.
This also avoids running Eval twice when pushing to staging-next or
haskell-updates, where a PR is open at the same time. Here, the PR's
Eval still runs on the push, of course.
When the merge queue fails, the workflow currently does not post a
negative result - and GitHub Actions waits for the status check to time
out, which takes 60 minutes.
This, of course, is a waste of time and resources. By explicitly failing
the status check, we boot the PR out of the merge queue immediately.
The required status checks should depend on exactly one workflow,
triggered via pull_request_target or merge_group. Anything that is
triggered by pull_request is for testing purposes of the workflows
themselves only.
Posting the status manually allows us to avoid the strange "skipped ==
success" logic and properly skip the `unlock` job for pull_request
events in the next commit.
This should be much easier to understand than the previous logic.
Changes to the merge-group workflow should also validate that the file
is hooked up correctly and works - otherwise we risk merging CI changes
that cause the merge queue to fail consistently.
For PRs targeting a fast moving branch like master, it might not
actually help because the merge commit will still be different on each
run. But on slower moving branches or when testing in a fork, it can
speed things up.
Since each of these build results is only a few hundred bytes without
any dependencies, it has essentially no cost.
Not adding this to `treefmt`, because this looks at *all* the files in
the repo, so needs to re-run even when iterating on CI only.
Introduces a basic merge queue workflow to initially only run lints.
This will avoid accidentally merging changes which break nixfmt after
its recent update to 1.0.0.
