diff --git a/.mailmap b/.mailmap index d3858d78dde7..d2bf6d0e4769 100644 --- a/.mailmap +++ b/.mailmap @@ -1,3 +1,14 @@ +ajs124 +Anderson Torres Daniel Løvbrøtte Olsen +Fabian Affolter +Janne Heß +Jörg Thalheim +Martin Weinelt R. RyanTM -Sandro +Robert Hensing +Sandro Jäckel +Sandro Jäckel +superherointj <5861043+superherointj@users.noreply.github.com> +Vladimír Čunát +Vladimír Čunát diff --git a/doc/builders/images/dockertools.section.md b/doc/builders/images/dockertools.section.md index 6203b3786bfa..dfc012b80c5a 100644 --- a/doc/builders/images/dockertools.section.md +++ b/doc/builders/images/dockertools.section.md @@ -62,6 +62,8 @@ The above example will build a Docker image `redis/latest` from the given base i - `config` is used to specify the configuration of the containers that will be started off the built image in Docker. The available options are listed in the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions). +- `architecture` is _optional_ and used to specify the image architecture, this is useful for multi-architecture builds that don't need cross compiling. If not specified it will default to `hostPlatform`. + - `diskSize` is used to specify the disk size of the VM used to build the image in megabytes. By default it's 1024 MiB. - `buildVMMemorySize` is used to specify the memory size of the VM to build the image in megabytes. By default it's 512 MiB. @@ -141,6 +143,8 @@ Create a Docker image with many of the store paths being on their own layer to i `config` _optional_ +`architecture` is _optional_ and used to specify the image architecture, this is useful for multi-architecture builds that don't need cross compiling. If not specified it will default to `hostPlatform`. + : Run-time configuration of the container. A full list of the options are available at in the [Docker Image Specification v1.2.0](https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions). *Default:* `{}` diff --git a/doc/languages-frameworks/cuelang.section.md b/doc/languages-frameworks/cuelang.section.md new file mode 100644 index 000000000000..93c94027ae29 --- /dev/null +++ b/doc/languages-frameworks/cuelang.section.md @@ -0,0 +1,93 @@ +# Cue (Cuelang) {#cuelang} + +[Cuelang](https://cuelang.org/) is a language to: + +- describe schemas and validate backward-compatibility +- generate code and schemas in various formats (e.g. JSON Schema, OpenAPI) +- do configuration akin to [Dhall Lang](https://dhall-lang.org/) +- perform data validation + +## Cuelang schema quick start + +Cuelang schemas are similar to JSON, here is a quick cheatsheet: + +- Default types includes: `null`, `string`, `bool`, `bytes`, `number`, `int`, `float`, lists as `[...T]` where `T` is a type. +- All structures, defined by: `myStructName: { }` are **open** -- they accept fields which are not specified. +- Closed structures can be built by doing `myStructName: close({ })` -- they are strict in what they accept. +- `#X` are **definitions**, referenced definitions are **recursively closed**, i.e. all its children structures are **closed**. +- `&` operator is the [unification operator](https://cuelang.org/docs/references/spec/#unification) (similar to a type-level merging operator), `|` is the [disjunction operator](https://cuelang.org/docs/references/spec/#disjunction) (similar to a type-level union operator). +- Values **are** types, i.e. `myStruct: { a: 3 }` is a valid type definition that only allows `3` as value. + +- Read to learn more about the semantics. +- Read to learn about the language specification. + +## `writeCueValidator` + +Nixpkgs provides a `pkgs.writeCueValidator` helper, which will write a validation script based on the provided Cuelang schema. + +Here is an example: +``` +pkgs.writeCueValidator + (pkgs.writeText "schema.cue" '' + #Def1: { + field1: string + } + '') + { document = "#Def1"; } +``` + +- The first parameter is the Cue schema file. +- The second paramter is an options parameter, currently, only: `document` can be passed. + +`document` : match your input data against this fragment of structure or definition, e.g. you may use the same schema file but differents documents based on the data you are validating. + +Another example, given the following `validator.nix` : +``` +{ pkgs ? import {} }: +let + genericValidator = version: + pkgs.writeCueValidator + (pkgs.writeText "schema.cue" '' + #Version1: { + field1: string + } + #Version2: #Version1 & { + field1: "unused" + }'' + ) + { document = "#Version${toString version}"; }; +in +{ + validateV1 = genericValidator 1; + validateV2 = genericValidator 2; +} +``` + +The result is a script that will validate the file you pass as the first argument against the schema you provided `writeCueValidator`. + +It can be any format that `cue vet` supports, i.e. YAML or JSON for example. + +Here is an example, named `example.json`, given the following JSON: +``` +{ "field1": "abc" } +``` + +You can run the result script (named `validate`) as the following: + +```console +$ nix-build validator.nix +$ ./result example.json +$ ./result-2 example.json +field1: conflicting values "unused" and "abc": + ./example.json:1:13 + ../../../../../../nix/store/v64dzx3vr3glpk0cq4hzmh450lrwh6sg-schema.cue:5:11 +$ sed -i 's/"abc"/3/' example.json +$ ./result example.json +field1: conflicting values 3 and string (mismatched types int and string): + ./example.json:1:13 + ../../../../../../nix/store/v64dzx3vr3glpk0cq4hzmh450lrwh6sg-schema.cue:5:11 +``` + +**Known limitations** + +* The script will enforce **concrete** values and will not accept lossy transformations (strictness). You can add these options if you need them. diff --git a/doc/languages-frameworks/go.section.md b/doc/languages-frameworks/go.section.md index 42acab817b6a..523f5b26ec7f 100644 --- a/doc/languages-frameworks/go.section.md +++ b/doc/languages-frameworks/go.section.md @@ -11,7 +11,13 @@ The function `buildGoModule` builds Go programs managed with Go modules. It buil In the following is an example expression using `buildGoModule`, the following arguments are of special significance to the function: -- `vendorHash`: is the hash of the output of the intermediate fetcher derivation. `vendorHash` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorHash = null;` +- `vendorHash`: is the hash of the output of the intermediate fetcher derivation. + + `vendorHash` can also be set to `null`. + In that case, rather than fetching the dependencies and vendoring them, the dependencies vendored in the source repo will be used. + + To avoid updating this field when dependencies change, run `go mod vendor` in your source repo and set `vendorHash = null;` + To obtain the actual hash, set `vendorHash = lib.fakeSha256;` and run the build ([more details here](#sec-source-hashes)). - `proxyVendor`: Fetches (go mod download) and proxies the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build or if any dependency has case-insensitive conflicts which will produce platform dependant `vendorHash` checksums. ```nix diff --git a/doc/languages-frameworks/index.xml b/doc/languages-frameworks/index.xml index 3d5b2f738976..7df241436ff5 100644 --- a/doc/languages-frameworks/index.xml +++ b/doc/languages-frameworks/index.xml @@ -13,6 +13,7 @@ + diff --git a/lib/default.nix b/lib/default.nix index cc4bedc5869b..68e5b8dea1eb 100644 --- a/lib/default.nix +++ b/lib/default.nix @@ -101,6 +101,7 @@ let upperChars toLower toUpper addContextFrom splitString removePrefix removeSuffix versionOlder versionAtLeast getName getVersion + mesonOption mesonBool mesonEnable nameFromURL enableFeature enableFeatureAs withFeature withFeatureAs fixedWidthString fixedWidthNumber isStorePath toInt toIntBase10 readPathsFromFile fileContents; diff --git a/lib/generators.nix b/lib/generators.nix index b77cca75010f..c0fe69389e00 100644 --- a/lib/generators.nix +++ b/lib/generators.nix @@ -278,8 +278,11 @@ rec { mapAny 0; /* Pretty print a value, akin to `builtins.trace`. - * Should probably be a builtin as well. - */ + * Should probably be a builtin as well. + * The pretty-printed string should be suitable for rendering default values + * in the NixOS manual. In particular, it should be as close to a valid Nix expression + * as possible. + */ toPretty = { /* If this option is true, attrsets like { __pretty = fn; val = …; } will use fn to convert val to a pretty printed representation. @@ -294,20 +297,25 @@ rec { introSpace = if multiline then "\n${indent} " else " "; outroSpace = if multiline then "\n${indent}" else " "; in if isInt v then toString v - else if isFloat v then "~${toString v}" + # toString loses precision on floats, so we use toJSON instead. This isn't perfect + # as the resulting string may not parse back as a float (e.g. 42, 1e-06), but for + # pretty-printing purposes this is acceptable. + else if isFloat v then builtins.toJSON v else if isString v then let - # Separate a string into its lines - newlineSplits = filter (v: ! isList v) (builtins.split "\n" v); - # For a '' string terminated by a \n, which happens when the closing '' is on a new line - multilineResult = "''" + introSpace + concatStringsSep introSpace (lib.init newlineSplits) + outroSpace + "''"; - # For a '' string not terminated by a \n, which happens when the closing '' is not on a new line - multilineResult' = "''" + introSpace + concatStringsSep introSpace newlineSplits + "''"; - # For single lines, replace all newlines with their escaped representation - singlelineResult = "\"" + libStr.escape [ "\"" ] (concatStringsSep "\\n" newlineSplits) + "\""; - in if multiline && length newlineSplits > 1 then - if lib.last newlineSplits == "" then multilineResult else multilineResult' - else singlelineResult + lines = filter (v: ! isList v) (builtins.split "\n" v); + escapeSingleline = libStr.escape [ "\\" "\"" "\${" ]; + escapeMultiline = libStr.replaceStrings [ "\${" "''" ] [ "''\${" "'''" ]; + singlelineResult = "\"" + concatStringsSep "\\n" (map escapeSingleline lines) + "\""; + multilineResult = let + escapedLines = map escapeMultiline lines; + # The last line gets a special treatment: if it's empty, '' is on its own line at the "outer" + # indentation level. Otherwise, '' is appended to the last line. + lastLine = lib.last escapedLines; + in "''" + introSpace + concatStringsSep introSpace (lib.init escapedLines) + + (if lastLine == "" then outroSpace else introSpace + lastLine) + "''"; + in + if multiline && length lines > 1 then multilineResult else singlelineResult else if true == v then "true" else if false == v then "false" else if null == v then "null" @@ -326,11 +334,11 @@ rec { else "" else if isAttrs v then # apply pretty values if allowed - if attrNames v == [ "__pretty" "val" ] && allowPrettyValues + if allowPrettyValues && v ? __pretty && v ? val then v.__pretty v.val else if v == {} then "{ }" else if v ? type && v.type == "derivation" then - "" + "" else "{" + introSpace + libStr.concatStringsSep introSpace (libAttr.mapAttrsToList (name: value: diff --git a/lib/options.nix b/lib/options.nix index c80256c0e653..b13687576e81 100644 --- a/lib/options.nix +++ b/lib/options.nix @@ -218,7 +218,7 @@ rec { # the set generated with filterOptionSets. optionAttrSetToDocList = optionAttrSetToDocList' []; - optionAttrSetToDocList' = prefix: options: + optionAttrSetToDocList' = _: options: concatMap (opt: let docOption = rec { @@ -234,9 +234,8 @@ rec { readOnly = opt.readOnly or false; type = opt.type.description or "unspecified"; } - // optionalAttrs (opt ? example) { example = scrubOptionValue opt.example; } - // optionalAttrs (opt ? default) { default = scrubOptionValue opt.default; } - // optionalAttrs (opt ? defaultText) { default = opt.defaultText; } + // optionalAttrs (opt ? example) { example = renderOptionValue opt.example; } + // optionalAttrs (opt ? default) { default = renderOptionValue (opt.defaultText or opt.default); } // optionalAttrs (opt ? relatedPackages && opt.relatedPackages != null) { inherit (opt) relatedPackages; }; subOptions = @@ -256,6 +255,9 @@ rec { efficient: the XML representation of derivations is very large (on the order of megabytes) and is not actually used by the manual generator. + + This function was made obsolete by renderOptionValue and is kept for + compatibility with out-of-tree code. */ scrubOptionValue = x: if isDerivation x then @@ -265,6 +267,17 @@ rec { else x; + /* Ensures that the given option value (default or example) is a `_type`d string + by rendering Nix values to `literalExpression`s. + */ + renderOptionValue = v: + if v ? _type && v ? text then v + else literalExpression (lib.generators.toPretty { + multiline = true; + allowPrettyValues = true; + } v); + + /* For use in the `defaultText` and `example` option attributes. Causes the given string to be rendered verbatim in the documentation as Nix code. This is necessary for complex values, e.g. functions, or values that depend on diff --git a/lib/strings.nix b/lib/strings.nix index b5f5a4d9060b..9a4f29380d0d 100644 --- a/lib/strings.nix +++ b/lib/strings.nix @@ -510,7 +510,7 @@ rec { toUpper = replaceChars lowerChars upperChars; /* Appends string context from another string. This is an implementation - detail of Nix. + detail of Nix and should be used carefully. Strings in Nix carry an invisible `context` which is a list of strings representing store paths. If the string is later used in a derivation @@ -533,13 +533,11 @@ rec { splitString "/" "/usr/local/bin" => [ "" "usr" "local" "bin" ] */ - splitString = _sep: _s: + splitString = sep: s: let - sep = builtins.unsafeDiscardStringContext _sep; - s = builtins.unsafeDiscardStringContext _s; - splits = builtins.filter builtins.isString (builtins.split (escapeRegex sep) s); + splits = builtins.filter builtins.isString (builtins.split (escapeRegex (toString sep)) (toString s)); in - map (v: addContextFrom _sep (addContextFrom _s v)) splits; + map (addContextFrom s) splits; /* Return a string without the specified prefix, if the prefix matches. @@ -661,6 +659,61 @@ rec { name = head (splitString sep filename); in assert name != filename; name; + /* Create a -D= string that can be passed to typical Meson + invocations. + + Type: mesonOption :: string -> string -> string + + @param feature The feature to be set + @param value The desired value + + Example: + mesonOption "engine" "opengl" + => "-Dengine=opengl" + */ + mesonOption = feature: value: + assert (lib.isString feature); + assert (lib.isString value); + "-D${feature}=${value}"; + + /* Create a -D={true,false} string that can be passed to typical + Meson invocations. + + Type: mesonBool :: string -> bool -> string + + @param condition The condition to be made true or false + @param flag The controlling flag of the condition + + Example: + mesonBool "hardened" true + => "-Dhardened=true" + mesonBool "static" false + => "-Dstatic=false" + */ + mesonBool = condition: flag: + assert (lib.isString condition); + assert (lib.isBool flag); + mesonOption condition (lib.boolToString flag); + + /* Create a -D={enabled,disabled} string that can be passed to + typical Meson invocations. + + Type: mesonEnable :: string -> bool -> string + + @param feature The feature to be enabled or disabled + @param flag The controlling flag + + Example: + mesonEnable "docs" true + => "-Ddocs=enabled" + mesonEnable "savage" false + => "-Dsavage=disabled" + */ + mesonEnable = feature: flag: + assert (lib.isString feature); + assert (lib.isBool flag); + mesonOption feature (if flag then "enabled" else "disabled"); + /* Create an --{enable,disable}- string that can be passed to standard GNU Autoconf scripts. diff --git a/lib/tests/misc.nix b/lib/tests/misc.nix index b73da4f1010d..648c05ab3572 100644 --- a/lib/tests/misc.nix +++ b/lib/tests/misc.nix @@ -727,7 +727,7 @@ runTests { float = 0.1337; bool = true; emptystring = ""; - string = ''fno"rd''; + string = "fn\${o}\"r\\d"; newlinestring = "\n"; path = /. + "/foo"; null_ = null; @@ -735,16 +735,16 @@ runTests { functionArgs = { arg ? 4, foo }: arg; list = [ 3 4 function [ false ] ]; emptylist = []; - attrs = { foo = null; "foo bar" = "baz"; }; + attrs = { foo = null; "foo b/ar" = "baz"; }; emptyattrs = {}; drv = deriv; }; expected = rec { int = "42"; - float = "~0.133700"; + float = "0.1337"; bool = "true"; emptystring = ''""''; - string = ''"fno\"rd"''; + string = ''"fn\''${o}\"r\\d"''; newlinestring = "\"\\n\""; path = "/foo"; null_ = "null"; @@ -752,9 +752,9 @@ runTests { functionArgs = ""; list = "[ 3 4 ${function} [ false ] ]"; emptylist = "[ ]"; - attrs = "{ foo = null; \"foo bar\" = \"baz\"; }"; + attrs = "{ foo = null; \"foo b/ar\" = \"baz\"; }"; emptyattrs = "{ }"; - drv = ""; + drv = ""; }; }; @@ -799,8 +799,8 @@ runTests { newlinestring = "\n"; multilinestring = '' hello - there - test + ''${there} + te'''st ''; multilinestring' = '' hello @@ -827,8 +827,8 @@ runTests { multilinestring = '' ''' hello - there - test + '''''${there} + te''''st '''''; multilinestring' = '' ''' diff --git a/maintainers/maintainer-list.nix b/maintainers/maintainer-list.nix index a66d32294f77..21e53a2ecc94 100644 --- a/maintainers/maintainer-list.nix +++ b/maintainers/maintainer-list.nix @@ -14127,6 +14127,12 @@ githubId = 8794235; name = "Tom Siewert"; }; + tonyshkurenko = { + email = "support@twingate.com"; + github = "tonyshkurenko"; + githubId = 8597964; + name = "Anton Shkurenko"; + }; toonn = { email = "nixpkgs@toonn.io"; matrix = "@toonn:matrix.org"; diff --git a/maintainers/scripts/update-luarocks-shell.nix b/maintainers/scripts/update-luarocks-shell.nix index a58674fca8d3..346b0319b08c 100644 --- a/maintainers/scripts/update-luarocks-shell.nix +++ b/maintainers/scripts/update-luarocks-shell.nix @@ -2,7 +2,7 @@ }: with nixpkgs; let - pyEnv = python3.withPackages(ps: [ ps.GitPython ]); + pyEnv = python3.withPackages(ps: [ ps.gitpython ]); in mkShell { packages = [ diff --git a/maintainers/team-list.nix b/maintainers/team-list.nix index c75f9aa71773..5e5a61875dbf 100644 --- a/maintainers/team-list.nix +++ b/maintainers/team-list.nix @@ -506,6 +506,18 @@ with lib.maintainers; { enableFeatureFreezePing = true; }; + node = { + members = [ + lilyinstarlight + marsam + winter + yuka + ]; + scope = "Maintain Node.js runtimes and build tooling."; + shortName = "Node.js"; + enableFeatureFreezePing = true; + }; + numtide = { members = [ mic92 diff --git a/nixos/doc/manual/from_md/release-notes/rl-2111.section.xml b/nixos/doc/manual/from_md/release-notes/rl-2111.section.xml index 688f0f47676c..9b6e755fd470 100644 --- a/nixos/doc/manual/from_md/release-notes/rl-2111.section.xml +++ b/nixos/doc/manual/from_md/release-notes/rl-2111.section.xml @@ -561,6 +561,14 @@ services.prometheus.exporters.smartctl. + + + twingate, + a high performance, easy to use zero trust solution that + enables access to private resources from any device with + better security than a VPN. + +
diff --git a/nixos/doc/manual/from_md/release-notes/rl-2305.section.xml b/nixos/doc/manual/from_md/release-notes/rl-2305.section.xml index 914be23576e0..cc330e2f8870 100644 --- a/nixos/doc/manual/from_md/release-notes/rl-2305.section.xml +++ b/nixos/doc/manual/from_md/release-notes/rl-2305.section.xml @@ -22,7 +22,14 @@
New Services - + + + + blesh, + a line editor written in pure bash. Available as + programs.bash.blesh. + + fzf, @@ -168,6 +175,15 @@ sudo and sources the environment variables. + + + The dnsmasq service now takes configuration + via the services.dnsmasq.settings attribute + set. The option + services.dnsmasq.extraConfig will be + deprecated when NixOS 22.11 reaches end of life. + + A new virtualisation.rosetta module was diff --git a/nixos/doc/manual/release-notes/rl-2111.section.md b/nixos/doc/manual/release-notes/rl-2111.section.md index 1ff2e826c601..fc4b44957c36 100644 --- a/nixos/doc/manual/release-notes/rl-2111.section.md +++ b/nixos/doc/manual/release-notes/rl-2111.section.md @@ -164,6 +164,8 @@ In addition to numerous new and upgraded packages, this release has the followin - [smartctl_exporter](https://github.com/prometheus-community/smartctl_exporter), a Prometheus exporter for [S.M.A.R.T.](https://en.wikipedia.org/wiki/S.M.A.R.T.) data. Available as [services.prometheus.exporters.smartctl](options.html#opt-services.prometheus.exporters.smartctl.enable). +- [twingate](https://docs.twingate.com/docs/linux), a high performance, easy to use zero trust solution that enables access to private resources from any device with better security than a VPN. + ## Backward Incompatibilities {#sec-release-21.11-incompatibilities} - The NixOS VM test framework, `pkgs.nixosTest`/`make-test-python.nix` (`pkgs.testers.nixosTest` since 22.05), now requires detaching commands such as `succeed("foo &")` and `succeed("foo | xclip -i")` to close stdout. diff --git a/nixos/doc/manual/release-notes/rl-2305.section.md b/nixos/doc/manual/release-notes/rl-2305.section.md index 3640cf8e963e..886db43c68eb 100644 --- a/nixos/doc/manual/release-notes/rl-2305.section.md +++ b/nixos/doc/manual/release-notes/rl-2305.section.md @@ -14,6 +14,8 @@ In addition to numerous new and upgraded packages, this release has the followin +- [blesh](https://github.com/akinomyoga/ble.sh), a line editor written in pure bash. Available as [programs.bash.blesh](#opt-programs.bash.blesh.enable). + - [fzf](https://github.com/junegunn/fzf), a command line fuzzyfinder. Available as [programs.fzf](#opt-programs.fzf.fuzzyCompletion). ## Backward Incompatibilities {#sec-release-23.05-incompatibilities} @@ -51,6 +53,11 @@ In addition to numerous new and upgraded packages, this release has the followin - `services.mastodon` gained a tootctl wrapped named `mastodon-tootctl` similar to `nextcloud-occ` which can be executed from any user and switches to the configured mastodon user with sudo and sources the environment variables. +- The `dnsmasq` service now takes configuration via the + `services.dnsmasq.settings` attribute set. The option + `services.dnsmasq.extraConfig` will be deprecated when NixOS 22.11 reaches + end of life. + - A new `virtualisation.rosetta` module was added to allow running `x86_64` binaries through [Rosetta](https://developer.apple.com/documentation/apple-silicon/about-the-rosetta-translation-environment) inside virtualised NixOS guests on Apple silicon. This feature works by default with the [UTM](https://docs.getutm.app/) virtualisation [package](https://search.nixos.org/packages?channel=unstable&show=utm&from=0&size=1&sort=relevance&type=packages&query=utm). - Resilio sync secret keys can now be provided using a secrets file at runtime, preventing these secrets from ending up in the Nix store. diff --git a/nixos/lib/make-options-doc/default.nix b/nixos/lib/make-options-doc/default.nix index e097aa5eebd8..dea3eec5bd6d 100644 --- a/nixos/lib/make-options-doc/default.nix +++ b/nixos/lib/make-options-doc/default.nix @@ -26,7 +26,7 @@ # If you include more than one option list into a document, you need to # provide different ids. , variablelistId ? "configuration-variable-list" - # Strig to prefix to the option XML/HTML id attributes. + # String to prefix to the option XML/HTML id attributes. , optionIdPrefix ? "opt-" , revision ? "" # Specify revision for the options # a set of options the docs we are generating will be merged into, as if by recursiveUpdate. @@ -45,28 +45,11 @@ }: let - # Make a value safe for JSON. Functions are replaced by the string "", - # derivations are replaced with an attrset - # { _type = "derivation"; name = ; }. - # We need to handle derivations specially because consumers want to know about them, - # but we can't easily use the type,name subset of keys (since type is often used as - # a module option and might cause confusion). Use _type,name instead to the same - # effect, since _type is already used by the module system. - substSpecial = x: - if lib.isDerivation x then { _type = "derivation"; name = x.name; } - else if builtins.isAttrs x then lib.mapAttrs (name: substSpecial) x - else if builtins.isList x then map substSpecial x - else if lib.isFunction x then "" - else x; - rawOpts = lib.optionAttrSetToDocList options; transformedOpts = map transformOptions rawOpts; filteredOpts = lib.filter (opt: opt.visible && !opt.internal) transformedOpts; optionsList = lib.flip map filteredOpts (opt: opt - // lib.optionalAttrs (opt ? example) { example = substSpecial opt.example; } - // lib.optionalAttrs (opt ? default) { default = substSpecial opt.default; } - // lib.optionalAttrs (opt ? type) { type = substSpecial opt.type; } // lib.optionalAttrs (opt ? relatedPackages && opt.relatedPackages != []) { relatedPackages = genRelatedPackages opt.relatedPackages opt.name; } ); @@ -111,14 +94,16 @@ in rec { inherit optionsNix; optionsAsciiDoc = pkgs.runCommand "options.adoc" {} '' - ${pkgs.python3Minimal}/bin/python ${./generateAsciiDoc.py} \ - < ${optionsJSON}/share/doc/nixos/options.json \ + ${pkgs.python3Minimal}/bin/python ${./generateDoc.py} \ + --format asciidoc \ + ${optionsJSON}/share/doc/nixos/options.json \ > $out ''; optionsCommonMark = pkgs.runCommand "options.md" {} '' - ${pkgs.python3Minimal}/bin/python ${./generateCommonMark.py} \ - < ${optionsJSON}/share/doc/nixos/options.json \ + ${pkgs.python3Minimal}/bin/python ${./generateDoc.py} \ + --format commonmark \ + ${optionsJSON}/share/doc/nixos/options.json \ > $out ''; diff --git a/nixos/lib/make-options-doc/generateAsciiDoc.py b/nixos/lib/make-options-doc/generateAsciiDoc.py deleted file mode 100644 index 48eadd248c5a..000000000000 --- a/nixos/lib/make-options-doc/generateAsciiDoc.py +++ /dev/null @@ -1,37 +0,0 @@ -import json -import sys - -options = json.load(sys.stdin) -# TODO: declarations: link to github -for (name, value) in options.items(): - print(f'== {name}') - print() - print(value['description']) - print() - print('[discrete]') - print('=== details') - print() - print(f'Type:: {value["type"]}') - if 'default' in value: - print('Default::') - print('+') - print('----') - print(json.dumps(value['default'], ensure_ascii=False, separators=(',', ':'))) - print('----') - print() - else: - print('No Default:: {blank}') - if value['readOnly']: - print('Read Only:: {blank}') - else: - print() - if 'example' in value: - print('Example::') - print('+') - print('----') - print(json.dumps(value['example'], ensure_ascii=False, separators=(',', ':'))) - print('----') - print() - else: - print('No Example:: {blank}') - print() diff --git a/nixos/lib/make-options-doc/generateCommonMark.py b/nixos/lib/make-options-doc/generateCommonMark.py deleted file mode 100644 index bf487bd89c3f..000000000000 --- a/nixos/lib/make-options-doc/generateCommonMark.py +++ /dev/null @@ -1,27 +0,0 @@ -import json -import sys - -options = json.load(sys.stdin) -for (name, value) in options.items(): - print('##', name.replace('<', '<').replace('>', '>')) - print(value['description']) - print() - if 'type' in value: - print('*_Type_*:') - print(value['type']) - print() - print() - if 'default' in value: - print('*_Default_*') - print('```') - print(json.dumps(value['default'], ensure_ascii=False, separators=(',', ':'))) - print('```') - print() - print() - if 'example' in value: - print('*_Example_*') - print('```') - print(json.dumps(value['example'], ensure_ascii=False, separators=(',', ':'))) - print('```') - print() - print() diff --git a/nixos/lib/make-options-doc/generateDoc.py b/nixos/lib/make-options-doc/generateDoc.py new file mode 100644 index 000000000000..1fe4eb0253ad --- /dev/null +++ b/nixos/lib/make-options-doc/generateDoc.py @@ -0,0 +1,108 @@ +import argparse +import json +import sys + +formats = ['commonmark', 'asciidoc'] + +parser = argparse.ArgumentParser( + description = 'Generate documentation for a set of JSON-formatted NixOS options' +) +parser.add_argument( + 'nix_options_path', + help = 'a path to a JSON file containing the NixOS options' +) +parser.add_argument( + '-f', + '--format', + choices = formats, + required = True, + help = f'the documentation format to generate' +) + +args = parser.parse_args() + +# Pretty-print certain Nix types, like literal expressions. +def render_types(obj): + if '_type' not in obj: return obj + + _type = obj['_type'] + if _type == 'literalExpression' or _type == 'literalDocBook': + return obj['text'] + + if _type == 'derivation': + return obj['name'] + + raise Exception(f'Unexpected type `{_type}` in {json.dumps(obj)}') + +def generate_commonmark(options): + for (name, value) in options.items(): + print('##', name.replace('<', '<').replace('>', '>')) + print(value['description']) + print() + if 'type' in value: + print('*_Type_*') + print ('```') + print(value['type']) + print ('```') + print() + print() + if 'default' in value: + print('*_Default_*') + print('```') + print(json.dumps(value['default'], ensure_ascii=False, separators=(',', ':'))) + print('```') + print() + print() + if 'example' in value: + print('*_Example_*') + print('```') + print(json.dumps(value['example'], ensure_ascii=False, separators=(',', ':'))) + print('```') + print() + print() + +# TODO: declarations: link to github +def generate_asciidoc(options): + for (name, value) in options.items(): + print(f'== {name}') + print() + print(value['description']) + print() + print('[discrete]') + print('=== details') + print() + print(f'Type:: {value["type"]}') + if 'default' in value: + print('Default::') + print('+') + print('----') + print(json.dumps(value['default'], ensure_ascii=False, separators=(',', ':'))) + print('----') + print() + else: + print('No Default:: {blank}') + if value['readOnly']: + print('Read Only:: {blank}') + else: + print() + if 'example' in value: + print('Example::') + print('+') + print('----') + print(json.dumps(value['example'], ensure_ascii=False, separators=(',', ':'))) + print('----') + print() + else: + print('No Example:: {blank}') + print() + +with open(args.nix_options_path) as nix_options_json: + options = json.load(nix_options_json, object_hook=render_types) + + if args.format == 'commonmark': + generate_commonmark(options) + elif args.format == 'asciidoc': + generate_asciidoc(options) + else: + raise Exception(f'Unsupported documentation format `--format {args.format}`') + diff --git a/nixos/lib/make-options-doc/options-to-docbook.xsl b/nixos/lib/make-options-doc/options-to-docbook.xsl index 0fe14a6d2d16..ac49659c681f 100644 --- a/nixos/lib/make-options-doc/options-to-docbook.xsl +++ b/nixos/lib/make-options-doc/options-to-docbook.xsl @@ -138,82 +138,6 @@ - - - '' - - '' - - - - - - - - - - - null - - - - - - - '''' - - - "" - - - - - - - - - - - - true - - - - - false - - - - - [ - - - - - ] - - - - - - - - - - { - - - = - ; - - } - - - - - (build of ) - -