279 Commits

Author SHA1 Message Date
Eman Resu
93d9b98b13 fetchgit: partially apply hasInfix 2026-07-20 10:30:10 -04:00
Philip Taron
5b5c421f52 nix-prefetch-git: also disable legacy auto-gc
Commit 88cfc54552 ("nix-prefetch-git: disable maintenance mode via
environment variables") disabled maintenance.auto (git's new
auto-maintenance) via GIT_CONFIG_*, which propagates to submodules. But
the legacy auto-gc path (gc.auto plus detached gc.autoDetach) is still
enabled, and it keeps repacking .git/modules/<sub>/objects after a nested
submodule reports "checked out", racing the later `rm -rf .git`:

- rm empties objects/pack, the detached gc drops a fresh pack in, and
  rmdir fails with "Directory not empty" (exit 123); or
- removal partially completes, leaving varying .git remnants, yielding a
  non-deterministic NAR hash.

Disable gc.auto/gc.autoDetach alongside maintenance.auto, using the same
GIT_CONFIG_* mechanism so the settings reach every git invocation and its
children (including nested submodule fetches). Fold the repeated
boilerplate into a small git_config_env helper.

The successful output is unchanged (.git is removed either way), so
pinned fetchgit hashes stay valid; this only makes removal deterministic.

Fixes #524215.

Assisted-by: Claude Code (claude-opus-4-8)
2026-06-11 08:25:04 -07:00
Heitor Augusto
6c103ed7ba nix-prefetch-git: disable maintenance mode via environment variables (#525255) 2026-06-04 06:48:05 +00:00
Philip Taron
0c469218f3 build-support: remove assertMsg, throwIf, and throwIfNot usage (#524535) 2026-05-28 18:09:32 +00:00
Pratham Patel
88cfc54552 nix-prefetch-git: disable maintenance mode via environment variables
The commit 011471c7f2 ("fetchgit: disable git maintenance") disabled
git's maintenance mode, but that does not affect git repositories with
submodules.

Switch to using the GIT_CONFIG_* environment variables. This is done in
a way to make the approach more generic, but also kinda not the best
("insane") so that it works with any future change, even with
`impureEnvVars`.

Maybe `git -c maintenance.auto=false` can work but I cannot figure out
how to use that here. I would prefer disabling maintenance mode in Git's
derivation itself but did not find any build-time knob to toggle that.
2026-05-28 22:43:24 +05:30
Philip Taron
7b0f8708ea fetchgit: disable git maintenance (#524430) 2026-05-28 00:07:30 +00:00
Hans Joachim Kliemeck
8478c2384b stdenv: Package URL (PURL) metadata fields 2026-05-26 21:47:54 +02:00
Eman Resu
661fe16b08 build-support/fetchgit: remove assertMsg usage 2026-05-26 14:14:41 -04:00
Adam Dinwoodie
011471c7f2 fetchgit: disable git maintenance
By default, Git commands will trigger background processes to perform
repository maintenance.  As of Git v2.54.0, and in particular as of
452b12c2e0 (builtin/maintenance: use "geometric" strategy by default,
2026-02-24), Git has changed the strategy it uses to decide whether
maintenance needs performing, and as a result will kick off background
maintenance processes much more often.

These background processes will potentially change the contents of the
`.git` directory while `nix-prefetch-git` is deleting it.  This results in
effectively random failures, when files are added to the `.git`
directory during the `rm -rf` operation (causing it to fail) or after
the operation has completed (causing the `.git` directory to be
recreated and resulting in a FOD hash mismatch).

This also causes problems for the determinism of Git's own tests.
They've resolved this in 09505b1115 (t: fix races caused by background
maintenance, 2026-02-24) by having the maintenance processes run in the
foreground.  The discussion in that commit explains they considered just
disabling maintenance entirely, but opted for the foreground option
because disabling maintenance would mean the tests looked even less like
how most Git users work with Git.

`nix-prefetch-git` is nothing like most Git users.  In almost every
case, the `.git` directory will be deleted during or immediately after
the script is called.  The repositories certainly don't benefit from
ongoing maintenance.  To avoid the non-determinism created by the
background maintenance processes, just disable maintenance entirely.

This is done with `git config` after initialising the repository, rather
than `git config --global` earlier in the script, as `git config
--global` will fail if `fetchgit` was called with a `gitConfig`
argument.

It would be possible to set this option with
`GIT_CONFIG_[COUNT|KEY_<n>|VALUE_<n>]` environment variables; I've opted
against that because (a) they're less familiar to most people who might
want to come to edit this script in future and (b) because if someone is
using those values themselves via `impureEnvVars` or similar, making our
configuration play nicely with someone else's would require some fairly
complex logic.

I haven't made this a configurable option: any use of `nix-prefetch-git`
will run with maintenance disabled, as I cannot think of a use case
where it would be useful.  If there is such a use case, it would be
straightforward to extend `nix-prefetch-git` and `fetchgit` to allow
users to change the behaviour.

Fixes #524215.
2026-05-26 13:43:37 +01:00
Colin
348bfd5f6d tests.fetchgit.withGitConfig: fix build 2026-03-02 17:56:54 +00:00
Emily
ad90665682 tests.fetchgit: pass system and the existing config 2026-02-19 16:07:50 +00:00
HigherOrderLogic
afa98a746b fetchgit: remove unused argument 2026-02-07 02:53:27 +00:00
Yueh-Shun Li
09030960d5 nix-prefetch-git: restore the tag-related clone flags when leaving .git
When leaveDotGit == true,
restore the cloning behaviour before
commit 7e085677f9 ("nix-prefetch-git: dont't fetch tags when deep clone unless leaving .git")
to preserve the fragile hashes of .git sources.

Clean up fetchTagsCompat,
as fetchTagsCompat is no longer needed
after restoring the old tag-fetching flags.
2025-12-24 21:59:46 +08:00
Yueh-Shun Li
62b2dbef68 nix-prefetch-git: Name FETCHED_HEAD "$ref" only for refs/tags/* 2025-12-23 04:50:05 +08:00
Philip Taron
cca4ec8008 fetchgit: take postCheckout to allow gathering revision and commit metadata without leaving .git (#465497) 2025-12-17 19:41:13 +00:00
Yueh-Shun Li
89746696de tests.fetchgit: add submodule-revision-count 2025-12-17 01:19:46 +08:00
Yueh-Shun Li
7e085677f9 nix-prefetch-git: dont't fetch tags when deep clone unless leaving .git
Co-authored-by: Adam Dinwoodie <adam@dinwoodie.org>
2025-12-17 01:19:43 +08:00
Yueh-Shun Li
d4dd261ab5 nix-prefetch-git: fetch the fetching tag for NIX_PREFETCH_GIT_CHECKOUT_HOOK
Co-authored-by: Adam Dinwoodie <adam@dinwoodie.org>
2025-12-17 01:14:55 +08:00
Yueh-Shun Li
689380e029 fetchurl, fetchgit: use __structuredAttrs = true and pass curlOptsList and sparseCheckout as lists (#464475) 2025-12-11 18:25:29 +00:00
Yueh-Shun Li
4e37d03ba2 fetchgit: reference hash from finalAttrs.hash 2025-12-10 01:07:13 +08:00
Yueh-Shun Li
982cafd8c6 fetchgit: use __structuredAttrs = true 2025-12-09 12:35:42 +08:00
Yueh-Shun Li
cbd24f4a8a tests.fetchgit: add simple-tag 2025-12-09 12:27:32 +08:00
Yueh-Shun Li
fc295b1e28 fetchgit: take postCheckout to allow collecting revision without leaving .git 2025-12-09 12:27:27 +08:00
Yueh-Shun Li
ea1e4d73d8 fetchgit: adjust format for sparseCheckout default
Co-authored-by: Matt Sturgeon <matt@sturgeon.me.uk>
2025-12-08 21:52:45 +08:00
Yueh-Shun Li
5c51547b22 fetchgit: sparseCheckout: reference depending attributes from finalAttrs 2025-12-04 15:28:38 +08:00
Yueh-Shun Li
8c85578602 fetchgit: format expression after eliminating global assertions 2025-12-04 15:26:16 +08:00
Yueh-Shun Li
d1275c4ff1 fetchgit: move sparseCheckout type check down to sparseCheckoutText 2025-12-04 15:25:40 +08:00
Yueh-Shun Li
dd264228ff fetchgit: default sparseCheckout to null and handle the default internally 2025-12-04 15:25:39 +08:00
Yueh-Shun Li
83b42e34ec fetchgit: keep sparseCheckout a list and stringify as sparseCheckoutText 2025-12-04 15:25:39 +08:00
Yueh-Shun Li
ea5b3c6d5d fetchgit: nonConeMode: reference depending attributes from finalAttrs 2025-12-04 15:25:35 +08:00
Yueh-Shun Li
6c5dc5d57f fetchgit: default nonConeMode to null and manage its default internally 2025-12-04 15:22:33 +08:00
Yueh-Shun Li
9fe4e65f3c fetchgit: default argument leaveDotGit to null 2025-11-17 02:09:55 +08:00
Yueh-Shun Li
1d6c7d6650 fetchgit: move assertions down to argument values 2025-11-16 04:04:51 +08:00
Yueh-Shun Li
2946555411 fetchgit: expose getRevWithTag as fetchgit.getRevWithTag 2025-11-03 21:31:04 +08:00
Yueh-Shun Li
21677a6bd8 fetchgit: pass rev as revCustom and make overridable 2025-11-03 21:31:04 +08:00
Yueh-Shun Li
dbf76b7c1b fetchgit: pass tag directly and make overridable 2025-11-03 21:31:04 +08:00
Yueh-Shun Li
076076e946 fetchgit: abstract and simplify revWithTag 2025-11-03 21:31:04 +08:00
Yueh-Shun Li
38f61c3f10 fetchgit: take derivationArgs 2025-11-03 21:31:03 +08:00
Yueh-Shun Li
aeddd850c6 fetchurl, fetchzip, fetchgit: format Nix expression after lib.extendMkDerivation 2025-10-28 06:41:42 +08:00
Yueh-Shun Li
ae5b3bdb05 fetchgit: restructure with lib.extendMkDerivation (not formatted) 2025-10-28 06:38:46 +08:00
Yueh-Shun Li
a7023948c0 fetchgit: take passthru 2025-10-28 01:47:34 +08:00
d068328
e47455f51d Revert "stdenv: pURL implementation (#421125)"
This reverts commit 5427115670, reversing
changes made to 88ee2ac331.
2025-10-18 20:41:01 +00:00
Arian van Putten
5427115670 stdenv: pURL implementation (#421125) 2025-10-17 09:30:32 +00:00
Hans Joachim Kliemeck
028af7c17d stdenv: pURL review suggestions - replace merge 2025-10-14 17:02:51 +00:00
Hans Joachim Kliemeck
bacccc39a9 stdenv: pURL - fix chaining case (github&submodules using fetchgit) 2025-10-14 14:53:13 +00:00
h0nIg
4e2614fc07 stdenv: pURL implementation 2025-10-03 21:19:09 +00:00
Alexander Bantyev
80d9975bd4 config: add a gitConfig/gitConfigFile option
Adds a `gitConfig` option (and `gitConfigFile`), to set a default
`gitConfigFile` argument for `fetchgit`.
2025-09-29 22:34:54 +02:00
Alexander Bantyev
296399ec68 fetchgit: add a gitConfigFile argument
Passing a `gitConfigFile` argument as a string allows setting a custom
git config (as `$GIT_CONFIG_GLOBAL`). The most obvious use is to
override URLs with custom mirrors, but others are possible too.
2025-09-29 22:34:11 +02:00
Silvan Mosberger
7d75d07d44 nix-prefetch-git: Add regression test for avoiding fetching cached paths 2025-09-24 22:18:32 +04:00
Alexander Bantyev
f12f3dfb21 nix-prefetch-git: add --no-add-path argument
Adds a new argument to `nix-prefetch-git`: `--no-add-path`. It disables
adding the path to the store. It is useful when working with a read-only
store.
2025-09-24 22:18:32 +04:00