Commit Graph

58 Commits

Author SHA1 Message Date
Eman Resu
ef34f35e45 lib.fileset: start by recursing into the initial element
This is guaranteed to be safe. The tree can be a simple string if the
fileset is the same as the root, but the `or` takes care of that. In the
`or` case, the tree is guaranteed to fail the isAttrs check, and so even
though the index is invalid, it won't matter.
2026-08-12 19:37:08 -04:00
Eman Resu
3b58c2719e lib.fileset: compare to baseString directly
The stringLength check is equivalent at this point, but having to
actualy create the thunk for calling stringLength is likely slower than
just letting Nix handle the equality check.
2026-08-12 14:09:05 -04:00
Eman Resu
9213fcbd18 lib.fileset: inline let variable only used once 2026-08-12 13:41:44 -04:00
Eman Resu
af732abb9b lib.fileset: partially apply substring logic 2026-08-12 13:37:51 -04:00
Eman Resu
b65b89f2de lib.fileset: move let definition higher up 2026-08-12 13:31:59 -04:00
adisbladis
b93a86f983 lib.fileset: Check the common case first in the toSource filter
The source filter runs on every path underneath the base directory, but
only on the few paths above it.

Reorder the prefix checks so the dominant below-the-base case does a
single substring + comparison instead of two, with an explicit branch
for the path being the base itself (which the previous first check also covered).
2026-08-12 16:31:50 +02:00
adisbladis
c55c59d46c lib.fileset: Don't recurse into files in _normaliseTreeFilter
_normaliseTreeFilter strictly walks the entire file set tree on every
toSource call. File entries (null or a file type string) are already
normalised, so dispatch on the subtree inside the mapAttrs closure
instead of making a recursive call that immediately returns its
argument. This also avoids allocating an unused `path + "/${name}"`
argument thunk for every file.
2026-08-12 16:31:49 +02:00
adisbladis
0d9f8e4f09 lib.fileset: Optimise _unionTrees
This function runs once per directory node per union during lazy tree evaluation.
This makes it one of the hottest paths in lib.fileset for large trees.
It was measuared at ~9% of _total evaluation_ time in a very large monorepo.

This change is a 2-in-1:

- Return singleton lists directly to avoid rebuilding subtrees

- Use foldl instead of findFirstIndex

  This is a hot path so the indexing arithmetic adds up.
2026-08-12 16:22:21 +02:00
İlkecan Bozdoğan
6acae46c84 lib: don't inherit unused functions 2026-03-04 00:10:00 +03:00
İlkecan Bozdoğan
e394a579b0 lib: update type signatures
- concrete types start with uppercase: Int, String, Bool, Derivation,
  etc.
- type variables start with lowercase: a, b, etc.
- list:
  - use `[x]` for homogeneous lists instead of `List x` or `[ x ]`
  - use `List` for heterogeneous lists (not that common in `lib`)
- attr:
  - use `AttrSet` for a generic attribute set type
  - use `{ key1 :: Type1; key2 :: Type2; ... }` for adding signatures
    for known attribute names and types
  - use `{ key1 = value1; key2 = value2; ... }` for adding attributes
    with known literals
  - end with an ellipsis (`...`) if the set can contain unknown
    attributes
  - use `{ [String] :: x }` if all the attributes has the same type `x`
- prefer `Any` over `a` if the latter is not reused
2026-03-04 00:10:00 +03:00
Dyego Aurélio
28096cc5e3 treewide: apply nixfmt 1.2.0 2026-01-22 18:37:56 -03:00
Niols
a99adc7c06 lib.fileset: add isFileset helper 2025-08-29 13:07:55 +02:00
Wolfgang Walther
9b9bef020a lib.fileset: remove conditionals for Nix 2.4
minver is now at 2.18, which means that `fetchGit` support shallow
clones and submodules for sure.
2025-08-12 15:52:40 +02:00
Johannes Kirschbauer
e0fe216f49 Format: lib/fileset/internal.nix 2025-02-12 15:01:45 +07:00
Johannes Kirschbauer
88f912da48 Docs: migrate format of comments to doc-comments 2025-02-12 15:01:43 +07:00
Silvan Mosberger
4f0dadbf38 treewide: format all inactive Nix files
After final improvements to the official formatter implementation,
this commit now performs the first treewide reformat of Nix files using it.
This is part of the implementation of RFC 166.

Only "inactive" files are reformatted, meaning only files that
aren't being touched by any PR with activity in the past 2 months.
This is to avoid conflicts for PRs that might soon be merged.
Later we can do a full treewide reformat to get the rest,
which should not cause as many conflicts.

A CI check has already been running for some time to ensure that new and
already-formatted files are formatted, so the files being reformatted here
should also stay formatted.

This commit was automatically created and can be verified using

    nix-build a08b3a4d19.tar.gz \
      --argstr baseRev b32a094368
    result/bin/apply-formatting $NIXPKGS_PATH
2024-12-10 20:26:33 +01:00
Silvan Mosberger
e3a6e38033 lib.fileset.toList: init 2024-02-26 20:21:50 +01:00
ibbem
6558e89177 lib.fileset.gitTracked: Allow clones of shallow repositories
The only reason shallow clones are not the default in
`builtins.fetchGit` is that `revCount` can't be provided when cloning a
shallow repository. However, `revCount` isn't used or exposed by
`lib.fileset`. Hence, allowing cloning shallow repositories makes
`gitTracked` more general without any drawbacks.

Co-authored-by: Silvan Mosberger <github@infinisil.com>
2024-02-01 00:01:26 +01:00
Silvan Mosberger
4a70c1e4da lib.fileset.gitTracked: Support out-of-tree builds 2023-12-19 22:08:59 +01:00
Silvan Mosberger
74f2e49543 lib.fileset.fetchGit: Refactoring 2023-12-19 01:46:15 +01:00
Silvan Mosberger
6e3be6ddb0 lib.fileset.gitTracked: Improve error when passing files 2023-12-11 23:05:51 +01:00
Silvan Mosberger
eec765f9e6 lib.fileset: Refactor gitTracked and gitTrackedWith
Introduce an internal function for them to share more behavior.
This makes future changes easier.
2023-12-11 22:57:12 +01:00
Silvan Mosberger
7486aee82e Merge pull request #266362 from tweag/fileset.fileFilter-ext
`lib.fileset.fileFilter`: Predicate attribute for file extension
2023-11-24 00:15:43 +01:00
Silvan Mosberger
6816f28c96 lib.fileset.fileFilter: Predicate attribute for file extension 2023-11-23 21:05:23 +01:00
Silvan Mosberger
1cc2c2f13d lib.fileset.maybeMissing: init 2023-11-20 19:56:45 +01:00
Silvan Mosberger
b514f0433c lib.fileset: Minor changes
- Make fromSource's missing file error message more consistent with others,
  and add a test for it
- Indent some function arguments
- Fix an internal type
2023-11-19 16:06:14 +01:00
Silvan Mosberger
d33f1a62f5 lib.fileset.gitTrackedWith: Introduce recurseSubmodules parameter 2023-11-16 01:12:18 +01:00
Silvan Mosberger
2dfb1d36cf lib.fileset.gitTracked/gitTrackedWith: init
A configuration parameter for gitTrackedWith will be introduced in the
next commit
2023-11-16 01:12:15 +01:00
Silvan Mosberger
1c3eb9eff1 lib.fileset.fileFilter: Restrict second argument to paths
While this change is backwards-incompatible, I think it's okay because:
- The `fileFilter` function is not yet in a stable NixOS release, it was only merged about [a month ago](https://github.com/NixOS/nixpkgs/pull/257356).
  - All public uses of the function on GitHub only pass a path
- Any `fileFilter pred fileset` can also be expressed as `intersection fileset (fileFilter pred path)` without loss of functionality.
  - This is furthermore pointed out in the new error message when a file set is passed
2023-11-15 01:20:36 +01:00
Silvan Mosberger
7e533bab6d Merge pull request #267381 from tweag/fileset.fileFilter-path
`fileset.fileFilter`: Don't run predicate unnecessarily
2023-11-15 01:19:36 +01:00
Silvan Mosberger
2035f8a324 lib.fileset.fileFilter: Don't run predicate unnecessarily
Before:

    nix-repl> fileset.trace (fileset.fileFilter (file: builtins.trace file.name false) ./default.nix)
    trace: README.md
    trace: benchmark.sh
    trace: default.nix
    trace: internal.nix
    trace: mock-splitRoot.nix
    trace: tests.sh

After:

    nix-repl> fileset.trace (fileset.fileFilter (file: builtins.trace file.name false) ./default.nix)
    trace: default.nix
2023-11-14 07:30:16 +01:00
Silvan Mosberger
e1d8331738 lib.fileset.fileFilter: Minor cleanups and more tests 2023-11-14 07:29:57 +01:00
Robert Hensing
cfd83c931f Merge pull request #261732 from tweag/fileset.fromSource
`lib.fileset.fromSource`: init
2023-11-10 20:32:51 +01:00
Silvan Mosberger
9cbd394aa0 lib.fileset: Remove unused bindings
Thanks nixd!
2023-11-09 00:02:43 +01:00
Silvan Mosberger
0ace383438 lib.fileset: Make error messages more uniform
Just minor changes like:
- Always using "X is a Y, but it should be Z"
- "X is a path that does not exist" rather than "X does not exist"
- Always using multi-line strings for errors
- Always quoting string-like values and not quoting path-like values
  - But do quote filesystem roots. Even though they're paths, they might
    be very small, good to have quotes to know the start/end
- Capitalise the first word
- Distinguish root vs filesystem root more
2023-11-08 23:59:04 +01:00
Silvan Mosberger
e07e80e841 lib.fileset.toSource: Mention fromSource in errors 2023-11-08 21:44:44 +01:00
Silvan Mosberger
73493584a7 lib.fileset.fromSource: init 2023-11-08 21:21:30 +01:00
Silvan Mosberger
fc28c5e5b7 Merge pull request #259065 from tweag/fileset.difference
`lib.fileset.difference`: init
2023-11-01 19:40:45 +01:00
Silvan Mosberger
50df7f9775 lib.fileset.difference: init 2023-10-30 12:51:55 +01:00
Silvan Mosberger
47c81d3286 lib.fileset.toSource: Optimise unknown file type error
Compared to the parent commit, this removes any detectable performance
cost :)
2023-10-30 12:48:13 +01:00
Silvan Mosberger
0f6cc8018c lib.fileset.toSource: Improve error for unknown file types
This does decrease performance unfortunately

Benchmarking expression toSource { root = ./.; fileset = ./.; }
Mean CPU time 0.103747 (σ = 0.012415) for 10 runs is 97.32181384964636% (σ = 16.34179537413021%) of the old value 0.106602 (σ = 0.0125571)
Statistic .envs.elements (205920) is 105.5842% (+10891) of the old value 195029
Statistic .gc.totalBytes (20247696) is 101.7495% (+348160) of the old value 19899536
Statistic .nrThunks (134824) is 108.7878% (+10891) of the old value 123933
Statistic .symbols.number (996) is 100.1005% (+1) of the old value 995
Statistic .values.number (275238) is 104.1199% (+10891) of the old value 264347
2023-10-30 12:48:11 +01:00
Silvan Mosberger
f4e0043049 lib.fileset.fileFilter: init 2023-10-24 01:44:04 +02:00
Silvan Mosberger
7247ec07ba lib.fileset: Improved error for unsupported coercion values 2023-10-16 15:50:10 +02:00
Silvan Mosberger
4ecf025814 lib.fileset.intersection: init 2023-10-11 16:17:48 +02:00
Silvan Mosberger
2541635c13 lib.fileset: Refactor for performance and future re-use 2023-10-04 23:21:45 +02:00
Silvan Mosberger
5bbe67bf67 lib.fileset: Mention trace functions in the file set evaluation error 2023-10-04 16:31:13 +02:00
Silvan Mosberger
efbcf5938f lib.fileset: Add internal helpers for pretty-printing 2023-10-04 16:28:55 +02:00
Silvan Mosberger
86802e19ce lib.fileset: _simplifyTree -> _normaliseTreeFilter 2023-10-03 21:27:42 +02:00
Silvan Mosberger
b3c2281219 lib.fileset: Order noEval last 2023-10-03 21:27:40 +02:00
Silvan Mosberger
4f35f003e6 lib.fileset: Internal representation v3, empty value without a base
`unions []` now works!

Notably the new empty value without a base is not exposed in the interface.
I don't know of any use case for it.
2023-10-03 17:05:57 +02:00