Niklas Hambüchen 51e34cde89 llama-cpp: Use runtime instruction dispatch for massive speedups
Before I added this, I made below benchmark to figure if
it's worth it:

llama-cpp with AVX-512 for faster inference and --quiet patch.
This is MUCH faster than nixpkgs's default x86_64 build,
e.g. reducing the time for a 1024x768 screenshot
(on an AMD Ryzen 7 7700X 8-Core, `llama-cpp` version 8983,
with `gemma-4-E2B-it-Q4_0.gguf` + `mmproj-gemma-4-E2B-it-F16.gguf`):

               default  AVX2  AVX-512  speedup (AVX2 / AVX-512)
    1 thread:
      CPU       360s     30s      27s            12x / 13x
      wall      293s     22s      19s            13x / 15x
    8 threads:
      CPU       405s     31s      28s            13x / 14x
      wall       51s      6s       6s             8x /  9x

where "AVX2" refers to:

    -DGGML_AVX=ON
    -DGGML_AVX2=ON
    -DGGML_FMA=ON
    -DGGML_F16C=ON
    -DGGML_AVX512=ON

and "AVX512" refers to additionally:

    -DGGML_AVX512_VBMI=ON
    -DGGML_AVX512_VNNI=ON
    -DGGML_AVX512_BF16=ON

AVX2 is available for CPUs from 2013 (Intel) / 2015 (AMD),
while AVX512 is available for CPUs from 2017 (Intel) / 2022 (AMD).

(cherry picked from commit d33227928d)
2026-05-25 18:58:19 +00:00
2026-04-07 10:41:13 -04:00
2026-01-22 23:00:52 +00:00
2025-03-30 22:46:13 -04:00
2025-07-28 07:49:23 +02:00
2025-10-29 18:41:03 +01:00

NixOS logo

Contributors badge Open Collective supporters

Nixpkgs is a collection of over 120,000 software packages that can be installed with the Nix package manager. It also implements NixOS, a purely-functional Linux distribution.

Manuals

  • NixOS Manual - how to install, configure, and maintain a purely-functional Linux distribution
  • Nixpkgs Manual - contributing to Nixpkgs and using programming-language-specific Nix expressions
  • Nix Package Manager Manual - how to write Nix expressions (programs), and how to use Nix command line tools

Community

Other Project Repositories

The sources of all official Nix-related projects are in the NixOS organization on GitHub. Here are some of the main ones:

Continuous Integration and Distribution

Nixpkgs and NixOS are built and tested by our continuous integration system, Hydra.

Artifacts successfully built with Hydra are published to cache at https://cache.nixos.org/. When successful build and test criteria are met, the Nixpkgs expressions are distributed via Nix channels.

Contributing

Nixpkgs is among the most active projects on GitHub. While thousands of open issues and pull requests might seem like a lot at first, it helps to consider it in the context of the scope of the project. Nixpkgs describes how to build tens of thousands of pieces of software and implements a Linux distribution. The GitHub Insights page gives a sense of the project activity.

Community contributions are always welcome through GitHub Issues and Pull Requests.

For more information about contributing to the project, please visit the contributing page.

Donations

The infrastructure for NixOS and related projects is maintained by a nonprofit organization, the NixOS Foundation. To ensure the continuity and expansion of the NixOS infrastructure, we are looking for donations to our organization.

You can donate to the NixOS foundation through SEPA bank transfers or by using Open Collective:

License

Nixpkgs is licensed under the MIT License.

Note

MIT license does not apply to the packages built by Nixpkgs, merely to the files in this repository (the Nix expressions, build scripts, NixOS modules, etc.). It also might not apply to patches included in Nixpkgs, which may be derivative works of the packages to which they apply. The aforementioned artifacts are all covered by the licenses of the respective packages.

Description
No description provided
Readme 4.9 GiB
Languages
Nix 95%
Shell 2%
Python 1.8%
Rust 0.3%
JavaScript 0.2%
Other 0.4%