warnIf sends our warning message through a function call, even if the
warning condition doesn't trigger. This requires a lot of thunk
allocation that can be easily avoided.
Clean up leftover for commit cd13136f03
("fetchurl: use __structuredAttrs = true and pass curlOptsList directly")
Continue the work of commit 23236b331d
("fetchurl: fix handling of fallback URLs"),
addressing a Bash array re-assignment quirk:
when assigning a Bash array variable as if it were a plain variable,
the value goes to the first element,
and the rest of the array stays the same.
```console
$ foo=(a b)
$ declare -p foo
declare -a foo=([0]="a" [1]="b")
$ foo="c d"
$ declare -p foo
declare -a foo=([0]="c d" [1]="b")
```
Don't rewrite the `${urls[@]}` with resolved URLs,
but hold them with `${resolvedUrls[@]}` instead.
Co-authored-by: Matt Sturgeon <matt@sturgeon.me.uk>
Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
Without the change fallback to `urls = []` does not
work as `$urls` evaluates on ony first entry. Use
`${urls[@]}` instead.
Example derivation that is fixed in `master` is
`xterm.src`:
Before:
$ nix build --no-link -f. xterm.src --rebuild -L
xterm> structuredAttrs is enabled
...
xterm> trying ftp://ftp.invisible-island.net/xterm/xterm-403.tgz
...
xterm> curl: (67) Access denied: 550
xterm> error: cannot download xterm-403.tgz from any mirror
After:
$ nix build --no-link -f. xterm.src --rebuild -L
xterm> structuredAttrs is enabled
...
xterm> trying ftp://ftp.invisible-island.net/xterm/xterm-403.tgz
...
xterm> curl: (67) Access denied: 550
...
xterm> trying https://invisible-mirror.net/archives/xterm/xterm-403.tgz
xterm> % Total % Received % Xferd Average Speed Time Time Time Current
xterm> Dload Upload Total Spent Left Speed
xterm> 100 1577k 100 1577k 0 0 703670 0 0:00:02 0:00:02 --:--:-- 703866
Allows having alternate hashed mirrors as fallbacks. Useful in case the
default hashed mirror is not accessible or doesn't have everything
needed.
Co-authored-by: Johan Herland <johan.herland@tweag.io>
Co-authored-by: Yuriy Taraday <yuriy.taraday@tweag.io>
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
fetchurl supports postFetch, which must not run when fetching from
hashedMirrors, because they provide pre-built derivation outputs.
fetchurl also supports downloadToTemp, which modifies behavior relating
to postFetch.
Before this commit, a call like
fetchurl {
url = "...";
downloadToTemp = true;
postFetch = ''
mv "$downloadedFile" "$out"
'';
}
would work in normal scenarios, but fail when fetching from
hashedMirrors, due to how internally downloadToTemp was not tied to
postFetch as closely as it should've been. This resulted in an error
like
builder [...] failed to produce output path
This commit fixes this, making hashedMirror fetching work even for
derivations that have downloadToTemp enabled.
Co-Authored-By: Silvan Mosberger <silvan.mosberger@tweag.io>
Thanks to @emilazy for the report:
https://github.com/NixOS/nixpkgs/pull/393136#issuecomment-3268711663
Fixes this crash:
```
… from call site
at /Users/emily/Developer/nixpkgs/pkgs/build-support/fetchurl/default.nix:194:70:
193| else
194| throw "fetchurl requires a hash for fixed-output derivation: ${lib.generators.toPretty urls_}";
| ^
195|
… while evaluating the value passed for the lambda argument
at /Users/emily/Developer/nixpkgs/lib/generators.nix:524:5:
523| toPretty =
524| {
| ^
525| allowPrettyValues ? false,
error: expected a set but found a list: [ "https://cmake.org/files/v4.1/cmake-4.1.1.tar.gz" ]
```
It'll still crash, but with a nicer error message!
Without the change `resolvedUrl` fails on non-mirror:// packages as:
$ nix eval --impure --expr 'with import ./. {}; AMB-plugins.src.resolvedUrl'
error:
...
error: expected a list but found null: null
After ther change the eval works as expected:
$ nix eval --impure --expr 'with import ./. {}; AMB-plugins.src.resolvedUrl'
"http://kokkinizita.linuxaudio.org/linuxaudio/downloads/AMB-plugins-0.8.1.tar.bz2"
None of these two mirrors have any of the sources these mirrors are used
for, checked with `nix-build <pkg>.src --check` while only those two are
not commented.
- ftp.belnet.be is outdated
- ftp.acc.umu.se returns a 403
- fr2.rpmfind.net is outdated
- ftp.nara.wide.ad.jp is outdated
- ftp.cse.buffalo.edu is unresponsive
This allows on-the-fly rewriting of URLs before they are passed from
fetchurl (or fetchurlBoot) to curl.
The intended use is to allow inserting company-internal mirrors, or
working around company firewalls and similar network restrictions,
without having to extensively patch across all of nixpkgs. Instead,
users can pass a function in their nixpkgs that performs the necessary
URL rewrites.
Co-authored-by: Alexander Bantyev <balsoft@balsoft.ru>
Right now, when building any FOD that uses `fetchurl` (which is the
majority of ours), `tarballs.nixos.org` will always be contacted
before the actual URL. Given that `tarballs.nixos.org` mainly hosts
the bootstrap tools, which already explicitly set it as the host to pull
from, it doesn't make much sense to force every other `fetchurl`
invocation to first reach out to `tarballs.nixos.org`.
I recently saw this error message, which required I use the `--debugger`
to locate:
```
error:
… while evaluating an expression to select 'drvPath' on it
at «internal»:1:552:
… while calling the 'derivationStrict' builtin
at «internal»:1:208:
(stack trace truncated; use '--show-trace' to show the full trace)
error: multiple hashes passed to fetchurl
```
This patch improves the error message, and other error messages from
`fetchurl`:
```
error:
… while evaluating an expression to select 'drvPath' on it
at «internal»:1:552:
… while calling the 'derivationStrict' builtin
at «internal»:1:208:
(stack trace truncated; use '--show-trace' to show the full trace)
error: multiple hashes passed to fetchurl: [
"https://github.com/reorg/pg_repack/archive/refs/tags/ver_1.5.0.tar.gz"
]
```
