Commit Graph

118 Commits

Author SHA1 Message Date
Philip Taron
853108adbd fetchurl: small performance cleanups (#546081) 2026-07-27 19:03:17 +00:00
Eman Resu
ed8b88a0ee fetchurl: check url case before urls
This happens much more frequently via fetchFromGitHub.
2026-07-26 17:29:52 -04:00
Eman Resu
b43717e14b fetchurl: get functions ready ahead of time 2026-07-26 17:29:47 -04:00
Eman Resu
0b6bc6cb1a fetchurl: partially apply hasPrefix calls 2026-07-20 10:28:43 -04:00
Eman Resu
917bf80632 fetchurl: avoid creating nativeBuildInputs list on every call 2026-06-01 23:18:12 -04:00
Eman Resu
4645f84f9d fetchurl: avoid resolvedUrl thunk 2026-06-01 23:18:11 -04:00
Eman Resu
62972398d8 fetchurl: avoid rewriteURL logic if it's null 2026-06-01 23:18:10 -04:00
Eman Resu
87163d15c7 fetchurl: inherit lib functions to global scope 2026-06-01 23:18:07 -04:00
Eman Resu
b4a2e8626b fetchurl: avoid escaping hash 2026-06-01 23:15:30 -04:00
Eman Resu
28e6e36fa0 build-support/fetchurl: remove warnIf usage
warnIf sends our warning message through a function call, even if the
warning condition doesn't trigger. This requires a lot of thunk
allocation that can be easily avoided.
2026-05-28 00:19:37 -04:00
Eman Resu
a350fc4a03 fetchurl: use concatMap rather than filtering and mapping 2026-05-11 16:11:01 -04:00
Matt Sturgeon
1e5ca4eadb fetchurl: add doc-comment for fetchurl.resolveUrl 2025-12-18 21:00:38 +08:00
Yueh-Shun Li
ca050c53b2 fetchurl: provide fetchurl.resolveUrl 2025-12-18 17:01:40 +08:00
Yueh-Shun Li
270ee9e19b fetchurl: mirrorList: simplify expression
Co-authored-by: Matt Sturgeon <matt@sturgeon.me.uk>
2025-12-18 17:01:31 +08:00
Yueh-Shun Li
915b484681 fetchurl: move netrcPhase invokation into build.sh
Don't rely on the `postPhase` in `$stdenv/setup.sh` provided by
`stdenv/generic/make-derivation.nix`'s realBuilder hack.
2025-12-17 20:43:01 +08:00
Yueh-Shun Li
58366ffe0c Revert "fetchurl, fetchgitlab: fix curlOpts extension" 2025-12-15 01:03:34 +08:00
Matt Sturgeon
55172e9583 fetchurl, fetchgitlab: fix curlOpts extension (#470550) 2025-12-13 22:16:44 +00:00
Nicolas Benes
3db11767a9 fetchurl, fetchgitlab: fix curlOpts extension
PR #464475 enables __structuredAttrs and makes curlOpts a bash array.
Consequently, it must be extended as such to be effective.

See also https://github.com/NixOS/nixpkgs/pull/464475#issuecomment-3649765507
2025-12-13 22:47:01 +01:00
Yueh-Shun Li
303ef6a5c9 fetchurl: pass SSL_CERT_FILE as environment variable via env 2025-12-14 01:30:39 +08:00
Yueh-Shun Li
689380e029 fetchurl, fetchgit: use __structuredAttrs = true and pass curlOptsList and sparseCheckout as lists (#464475) 2025-12-11 18:25:29 +00:00
Yueh-Shun Li
ae800ea500 fetchurl: reference outputHash from finalAttrs.hash 2025-12-10 01:02:47 +08:00
Yueh-Shun Li
cf99863262 fetchurl: reference lib.fakeHash for empty hash algorithm 2025-12-10 01:02:47 +08:00
Yueh-Shun Li
cd13136f03 fetchurl: use __structuredAttrs = true and pass curlOptsList directly 2025-12-09 12:34:45 +08:00
Yueh-Shun Li
4535777468 fetchurl: sort inherited variables 2025-12-09 12:34:45 +08:00
Philip Taron
49af83567b fetchurl: make pname and version overridable via <pkg>.overrideAttrs (#458183) 2025-11-05 16:23:24 +00:00
Yueh-Shun Li
7e212bfdd5 fetchurl: fix pname and version overriding 2025-11-04 15:11:29 +08:00
Yueh-Shun Li
4ab091862a fetchurl: take derivationArgs 2025-11-03 21:31:02 +08:00
Yueh-Shun Li
1ec0227cc0 fetchurl: don't treat empty-string name special 2025-10-28 06:43:13 +08:00
Yueh-Shun Li
aeddd850c6 fetchurl, fetchzip, fetchgit: format Nix expression after lib.extendMkDerivation 2025-10-28 06:41:42 +08:00
Yueh-Shun Li
bf3e7097f3 fetchurl: restructure with lib.extendMkDerivation (not formatted) 2025-10-28 06:38:45 +08:00
Yueh-Shun Li
089fab12e3 fetchurl: move curlOpts warnings down to its value 2025-10-28 06:38:45 +08:00
Silvan Mosberger
56f680d915 fetchurl: Allow hashed mirror overriding with config.hashedMirrors
Allows having alternate hashed mirrors as fallbacks. Useful in case the
default hashed mirror is not accessible or doesn't have everything
needed.

Co-authored-by: Johan Herland <johan.herland@tweag.io>
Co-authored-by: Yuriy Taraday <yuriy.taraday@tweag.io>
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
2025-10-06 14:15:10 +02:00
Rebecca Turner
3d4fb2c3e6 fetchurl: Add missing arg to toPretty invocation
Thanks to @emilazy for the report:
https://github.com/NixOS/nixpkgs/pull/393136#issuecomment-3268711663

Fixes this crash:

```
… from call site
  at /Users/emily/Developer/nixpkgs/pkgs/build-support/fetchurl/default.nix:194:70:
   193|     else
   194|       throw "fetchurl requires a hash for fixed-output derivation: ${lib.generators.toPretty urls_}";
      |                                                                      ^
   195|

… while evaluating the value passed for the lambda argument
  at /Users/emily/Developer/nixpkgs/lib/generators.nix:524:5:
   523|   toPretty =
   524|     {
      |     ^
   525|       allowPrettyValues ? false,

error: expected a set but found a list: [ "https://cmake.org/files/v4.1/cmake-4.1.1.tar.gz" ]
```

It'll still crash, but with a nicer error message!
2025-09-09 10:06:26 -07:00
Sergei Trofimovich
425c7e46a0 fetchurl: fix resolvedUrl eval on URLs without mirror:// prefix
Without the change `resolvedUrl` fails on non-mirror:// packages as:

    $ nix eval --impure --expr 'with import ./. {}; AMB-plugins.src.resolvedUrl'
    error:
       ...
       error: expected a list but found null: null

After ther change the eval works as expected:

    $ nix eval --impure --expr 'with import ./. {}; AMB-plugins.src.resolvedUrl'
    "http://kokkinizita.linuxaudio.org/linuxaudio/downloads/AMB-plugins-0.8.1.tar.bz2"
2025-08-05 09:31:42 +01:00
Sandro Jäckel
aa9a2250c8 fetchurl: add passthru resolvedUrl
This can be used to resolve mirror:// style URLs to a full URL any http
tool understands.
2025-08-04 20:02:49 +02:00
Wolfgang Walther
5a0711127c treewide: run nixfmt 1.0.0 2025-07-24 13:55:40 +02:00
Johan Herland
abda866f17 fetchurl: Add hook for rewriting/filtering URLs
This allows on-the-fly rewriting of URLs before they are passed from
fetchurl (or fetchurlBoot) to curl.

The intended use is to allow inserting company-internal mirrors, or
working around company firewalls and similar network restrictions,
without having to extensively patch across all of nixpkgs. Instead,
users can pass a function in their nixpkgs that performs the necessary
URL rewrites.

Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
2025-06-04 10:29:08 +00:00
Winter
40d5f4e0a0 fetchurl: don't prefer hashed mirrors by default
Right now, when building any FOD that uses `fetchurl` (which is the
majority of ours), `tarballs.nixos.org` will always be contacted
before the actual URL. Given that `tarballs.nixos.org` mainly hosts
the bootstrap tools, which already explicitly set it as the host to pull
from, it doesn't make much sense to force every other `fetchurl`
invocation to first reach out to `tarballs.nixos.org`.
2025-05-08 03:23:14 -04:00
Rebecca Turner
33044b9bd8 fetchurl: Improve error messages
I recently saw this error message, which required I use the `--debugger`
to locate:

```
error:
       … while evaluating an expression to select 'drvPath' on it
         at «internal»:1:552:
       … while calling the 'derivationStrict' builtin
         at «internal»:1:208:
       (stack trace truncated; use '--show-trace' to show the full trace)

       error: multiple hashes passed to fetchurl
```

This patch improves the error message, and other error messages from
`fetchurl`:

```
error:
       … while evaluating an expression to select 'drvPath' on it
         at «internal»:1:552:
       … while calling the 'derivationStrict' builtin
         at «internal»:1:208:
       (stack trace truncated; use '--show-trace' to show the full trace)

       error: multiple hashes passed to fetchurl: [
         "https://github.com/reorg/pg_repack/archive/refs/tags/ver_1.5.0.tar.gz"
       ]
```
2025-03-25 10:11:28 -07:00
Thomas Gerbet
31ab653f7e fetchurl: revert enabling TLS verification when NIX_SSL_CERT_FILE
This reverts commit f829274128.
2024-10-26 13:58:19 +02:00
Thomas Gerbet
f829274128 fetchurl: enable TLS verification when NIX_SSL_CERT_FILE is set
This is a follow-up to a169553f7e.
In most cases it should allow the TLS verification to be enabled.
It also makes the behavior of `fetchurl` more consistent with other fetchers
like `fetchgit`.

Ideally we would always fallback on `cacert` but I am not sure how to build
`cacert` during bootstrap without making an unmaintainable mess.
2024-10-22 14:58:54 +02:00
Anderson Torres
ce21e97a1f fetchurl: nixfmt-rfc-style 2024-09-25 14:51:53 -03:00
Anderson Torres
a32c7a11dd fetchurl: fixup typo on a comment 2024-09-25 14:51:53 -03:00
Thomas Gerbet
a169553f7e fetchurl: enable TLS verification when credentials are used
This make sure the credentials cannot be leaked in a MITM attack.
Note that this change might break some existing deployments if the users
tries to fetch resources on endpoints with invalid certificates.
The impacted users will have the following choices:
* fix the endpoint providing the resource
* override SSL_CERT_FILE to either disable the verification (not
  recommended) or to set it to a path including their CA certificate.
2024-09-23 18:27:02 +02:00
nicoo
af5682b3e1 fetchurl: Correctly handle { outputHash != ""; outputHashAlgo = "" }
Previously, the error that would be thrown would have been confusing:
```
fetchurl requires a hash for fixed-output derivation
```
2023-08-30 09:51:22 +00:00
nicoo
a59f03079c fetchurl: Validate that a single hash was passed 2023-08-30 09:51:22 +00:00
Jan Malakhovski
9437e4da35 fetchurl: cleanup a bit by moving the warning into assert 2023-08-08 13:38:11 +00:00
Theodore Ni
6f30e0a906 fetchurl: clean up md5 references 2023-07-20 21:23:04 -07:00
Sandro
1fd6cf192a fixup! fetchurl: add pname+version support 2022-10-05 15:54:57 +02:00
Sandro Jäckel
7deac8b8f7 fetchurl: add pname+version support 2022-10-04 17:30:06 +02:00