Commit Graph

309 Commits

Author SHA1 Message Date
Robert Sliwinski
2bcc3b96f1 prefetch-yarn-deps: fix resolving of local packages using file:
Without this change local packages would get rewritten to resolve to
something that cannot be found at yarnpkg.com

Example from a package prior this change:

Excerpt from `yarn.lock`:
```
@fortawesome/fontawesome-pro@file:./pkg/fortawesome-fontawesome-pro-6.3.0.tgz
file_._pkg_fortawesome_fontawesome_pro_6.3.0.tgz
```

The `yarnConfigHook` results in:
```
> Executing yarnConfigHook
> yarn config v1.22.22
> success Set "yarn-offline-mirror" to "/nix/store/r02716lg5ric6rihpnh75lrmpdw9mwv4-offline".
> Done in 0.02s.
> yarn install v1.22.22
> [1/4] Resolving packages...
> [2/4] Fetching packages...
> error Can't make a request in offline mode ("https://registry.yarnpkg.com/file_._pkg_fortawesome_fontawesome_pro_6.3.0.tgz")
```
2025-03-05 15:59:19 +01:00
Rebecca Turner
f4d32e0200 nixForLinking: init
Overlays for CppNix nightly, Lix, or Tvix want to change the default Nix
implementation in Nixpkgs by overriding `pkgs.nix`. However, some
packages link against the internal/unstable CppNix APIs directly, and
these packages will break if built with different versions or
implementations of Nix.

If you want to swap out the Nix implementation in your package set, you
don't want these packages to break. Therefore, some packages will refer
to `nixForLinking` explicitly, at least until these dependencies can be
sorted out.

The addition of an explicit `nixForLinking` attribute decoupled from
`nix`, which is just "a Nix implementation", will help Nix
implementation maintainers test Nix implementations in Nixpkgs with
minimal hassle.
2025-02-27 09:44:32 -08:00
Peder Bergebakken Sundt
5aba99242e treewide: fix typos in comments
Made with

```shell
git restore .
fd '\.nix$' pkgs/ --type f -j1 -x bash -xc "$(cat <<"EOF"
    typos --no-check-filenames --write-changes "$1"
    git diff --exit-code "$1" && exit
    #( git diff "$1" | grep -qE "^\+ +[^# ]") && git restore "$1"
    count1="$( bat --language nix --diff --style changes "$1" --theme "Monokai Extended" --color always | aha --no-header | grep -E '^<span style="color:olive;">~</span> ' | wc -l )"
    count2="$( bat --language nix --diff --style changes "$1" --theme "Monokai Extended" --color always | aha --no-header | grep -E '^<span style="color:olive;">~</span> (<span style="color:#f8f8f2;"> *</span>)?<span style="color:#75715e;">.*</span>$' | wc -l )"
    [[ $count1 -ne $count2 ]] && git restore "$1"
EOF
)" -- {}
```

and filtered with `GIT_DIFF_OPTS='--unified=15' git -c interactive.singleKey=true add --patch`

I initially tried using the tree-sitter cli, python bindings and even ast-grep through various means, but this is what I ended up with.
2025-02-24 10:44:41 +01:00
K900
226cece119 buildNpmPackage: avoid using npm config (#373220) 2025-02-09 09:29:40 +03:00
K900
0fd5f209c1 Merge remote-tracking branch 'origin/master' into staging-next 2025-01-17 11:30:03 +03:00
Doron Behar
18f936b623 balena-cli: fix build by using an older nodejs (#372092) 2025-01-17 00:39:47 +02:00
github-actions[bot]
ae7e95ed1d Merge master into staging-next 2025-01-14 02:53:34 +00:00
Isaac Shapira
117929b9cc buildNpmPackage: avoid using npm config 2025-01-12 16:21:53 +01:00
Doron Behar
0fc94b12c7 npmHooks.npmInstallHook: inherit nodejs when evaluating nodejsInstallExecutables
Otherwise, using the following:

```
buildNpmPackage' = buildNpmPackage.override {
  nodejs = nodejs_20;
};
```

Doesn't override the nodejs version used by the `npmInstallHook`, that
is actually used as the nodejs executable of the wrapper created by
`makeWrapper` in the default `installPhase`.
2025-01-11 23:38:36 +02:00
phaer
e2013a5096 importNpmLock: fix check for .bin in linkNodeModulesHook
node_modules/.bin/ is expected to be (symlink to) a directory rather
than a regular file, so we check for that.

This seems to work for pre-existing directories as well as those
we link to the store (managed & unmanaged in the hooks terminology).

I am not entirely sure whether the original check was simply a typo
or intentional for some use-cases.
2025-01-10 12:37:32 +01:00
github-actions[bot]
08aa1bb06d Merge staging-next into staging 2024-12-26 12:06:04 +00:00
Austin Horstman
2e86b37d72 yarnConfigHook: fix darwin permissions 2024-12-26 00:11:34 -06:00
Arne Keller
1de0f269e9 fetch-npm-deps: handle all git+ urls the same during fixup (#280647) 2024-12-20 18:04:48 +01:00
Vladimír Čunát
b5673d9e62 Reapply "prefetch-npm-deps: check response status..." (#365658)
This reverts commit a31f38ad20, reversing
changes made to c7dcafcc23.

Moved from master to staging-next.  About 10k rebuilds in total.
2024-12-16 19:06:09 +01:00
Vladimír Čunát
59c5ef5569 Revert "prefetch-npm-deps: check response status and fail on error (#297863)"
This reverts commit 74e24a96dc.
2024-12-16 19:00:40 +01:00
Johnny Walker
74e24a96dc prefetch-npm-deps: check response status and fail on error (#297863)
Check response status and fail if error returned.
2024-12-15 18:47:22 +01:00
Silvan Mosberger
4f0dadbf38 treewide: format all inactive Nix files
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.

Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.

A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.

This commit was automatically created and can be verified using

    nix-build a08b3a4d19.tar.gz \
      --argstr baseRev b32a094368
    result/bin/apply-formatting $NIXPKGS_PATH
2024-12-10 20:26:33 +01:00
Sefa Eyeoglu
7779a86836 Fix Codeql findings in update scripts (#342541) 2024-12-01 12:17:24 +01:00
Sandro
0168c4f291 buildNpmPackage: pass down patchFlags to fetchNpmDeps (#350761) 2024-11-05 15:49:50 +01:00
Christoph Heiss
f52c685f97 buildNpmPackage: pass down patchFlags to fetchNpmDeps
`fetchNpmDeps` works from the exact same source tree, thus `patchFlags`
must be respected so that any patches get applied correctly.

Signed-off-by: Christoph Heiss <christoph@c8h4.io>
2024-10-23 21:28:12 +02:00
Johannes Kirschbauer
34b7d209bb nodejs/importNpmLock: init source overrides option 2024-10-17 12:00:45 +02:00
Doron Behar
796699f22b fetch-yarn-deps: handle packages that are locally linked (#342186) 2024-09-29 22:59:39 +03:00
Mathieu Rene
2462edca10 fetch-yarn-deps: handle packages that are locally linked 2024-09-29 14:14:45 -04:00
Artturin
e0464e4788 treewide: replace stdenv.is with stdenv.hostPlatform.is
In preparation for the deprecation of `stdenv.isX`.

These shorthands are not conducive to cross-compilation because they
hide the platforms.

Darwin might get cross-compilation for which the continued usage of `stdenv.isDarwin` will get in the way

One example of why this is bad and especially affects compiler packages
https://www.github.com/NixOS/nixpkgs/pull/343059

There are too many files to go through manually but a treewide should
get users thinking when they see a `hostPlatform.isX` in a place where it
doesn't make sense.

```
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv.is" "stdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenv'.is" "stdenv'.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "clangStdenv.is" "clangStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "gccStdenv.is" "gccStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "stdenvNoCC.is" "stdenvNoCC.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "inherit (stdenv) is" "inherit (stdenv.hostPlatform) is"
fd --type f "\.nix" | xargs sd --fixed-strings "buildStdenv.is" "buildStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "effectiveStdenv.is" "effectiveStdenv.hostPlatform.is"
fd --type f "\.nix" | xargs sd --fixed-strings "originalStdenv.is" "originalStdenv.hostPlatform.is"
```
2024-09-25 00:04:37 +03:00
lelgenio
c499cb1399 yarnInstallHook: fix temporary directory location
Currently the directory is being created in $src/yarnInstallHook.XXXXXX
Giving the directory a readable name is not particularly useful
2024-09-17 16:01:33 -03:00
Sandro Jäckel
a9c9441997 fetch-yarn-deps: fix "Incomplete URL substring sanitization"
'https://codeload.github.com' may be followed by an arbitrary host name.
2024-09-17 15:17:59 +02:00
Doron Behar
d77ecc757a fetchYarnDeps: fix by deduplicating requests (#336784) 2024-09-10 18:17:48 +00:00
Fea
207072fca3 fetchYarnDeps: fix by deduplicating requests 2024-09-09 23:39:44 +02:00
lelgenio
7783e118af yarnInstallHook: init 2024-09-04 11:15:47 -03:00
Doron Behar
3326d57216 npmInstallHook: extract nodejsInstall{Manuals,Executables} (#336644) 2024-08-30 06:06:06 +00:00
adisbladis
9c7ff7277c importNpmLock.buildNodeModules: init
`importNpmLock.buildNodeModules` returns a derivation with a pre-built `node_modules` directory, as imported by `importNpmLock`.
This is to be used together with `importNpmLock.hooks.linkNodeModulesHook` to facilitate `nix-shell`/`nix develop` based development workflows:

```nix
pkgs.mkShell {
  packages = [
    importNpmLock.hooks.linkNodeModulesHook
    nodejs
  ];

  npmDeps = importNpmLock.buildNodeModules {
    npmRoot = ./.;
    inherit nodejs;
  };
}
```
will create a development shell where a `node_modules` directory is created & packages symlinked to the Nix store when activated.

This code is adapted from https://github.com/adisbladis/buildNodeModules
2024-08-29 06:12:07 -07:00
seth
278e1bfb89 yarnConfigHook: apply as a postConfigure hook
This follows in this example of npmConfigHook and allows for using this
hook alongside others that may override configurePhase
2024-08-28 23:30:55 -04:00
lelgenio
80a2825733 npmHooks.npmInstallHook: extract npmInstall{Manuals,Executables} 2024-08-26 20:43:42 -03:00
Johannes Kirschbauer
3ed5055f0f importNpmLock: init fetcherOps per package 2024-08-13 09:38:37 +02:00
Sandro Jäckel
f4e18f55ef fetch-npm-deps: be less noisy
Before every package in node_modules was printed. Lets be a bit more
quiet and drop that.
2024-08-09 15:08:33 -04:00
github-actions[bot]
f20ed12f93 Merge master into staging-next 2024-07-25 12:01:18 +00:00
Adam Joseph
9cac7a7475 fetch-yarn-deps: improve diagnostic messages
When the hash of an url being fetched does not match the expected value, this
commit will cause fetch-yarn-deps to include the url in the error message to
assist debugging.
2024-07-25 11:37:44 +02:00
Adam Joseph
84a75e9488 fetchYarnDeps: fix broken fetching logic for github releases
When a dependency references a github *release* URL, that dependency must be
fetched using https rather than git, since github does not require that
release tarballs have any relationship whatsoever to the git history.

This commit causes them to be fetched using https, not git.

A test case (which fails prior to this commit, and passes afterwards) is included.
2024-07-25 11:37:39 +02:00
Randy Eckenrode
f9b7f4ec09 tree-wide: use top-level cctools 2024-07-17 22:36:19 -04:00
Doron Behar
2400268a38 fetch-yarn-deps: handle yarn.lock deps without a resolved url 2024-07-10 09:39:10 +03:00
Doron Behar
ee7cfec5aa doc/javascript: document yarn{Config,Build}Hook 2024-07-10 09:39:03 +03:00
Doron Behar
296556a320 yarnBuildHook: init 2024-07-10 09:39:02 +03:00
Doron Behar
a84f3ca3d8 yarnConfigHook: init 2024-07-10 09:39:02 +03:00
Doron Behar
1151b39c24 fetch-yarn-deps: format with nixfmt (RFC166) 2024-07-10 09:39:01 +03:00
Lily Foster
c678743f80 treewide: remove lilyinstarlight 2024-07-02 03:17:01 +02:00
Atemu
882cf18005 fetch-yarn-deps: reference files to be copied directly
This would cause any drv to be rebuilt when any part of the default.nix changes.
What we actually care about is the two JS files though, so simply reference them
directly.

Co-authored-by: Infinidoge <infinidoge@inx.moe>
2024-06-14 17:53:18 +02:00
Infinidoge
efff2acc5b fetchYarnDeps: properly accept src argument 2024-06-14 11:18:10 -04:00
Lily Foster
ddb94deafa prefetch-npm-deps: switch to data-encoding 2024-04-22 23:14:15 +10:00
Lily Foster
c588edaf25 prefetch-npm-deps: remove runtime dependency on nix 2024-04-22 23:14:15 +10:00
Sandro Jäckel
6181939cdf fixup-yarn-lock: split out from prefetch-yarn-deps, cleanup installing
To reduce dependencies (mainly nix-prefetch-git and through that git,
git-lfs) when we just need to fixup a lock file, eg when building electron.
This also tries to avoid needless rebuilds when eg. golang is updated.

Also this cleans up and combined the build/installPhase of both tools to
be a lot simpler.
2024-04-17 11:35:22 -04:00