Skip to content

roman/nixDir

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

87 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

nixDir

Note This is the v2 version of nixDir.

For information about all available versions, see the main branch README.

Project Status: Active – The project has reached a stable, usable state and is being actively developed. Github Action Tests Status

nixDir is a library that transforms a convention oriented directory structure into a nix flake.

With nixDir, you don't run into large flake.nix files, and don't have to implement the "import wiring" of multiple nix files. nixDir will use Convention over Configuration and lets you get back to your business.

Table Of Contents

Introduction

The nixDir library traverses a configured nix directory to build the flakes outputs dynamically.

The behavior is easier to explain with an example; assume you have a myproj directory with the following flake.nix:

{
  description = "myproj is here to make the world a better place";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    nixDir = {
      url = "github:roman/nixDir/v2";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = {nixDir, ...} @ inputs: nixDir.lib.buildFlake {
    systems = [ "x86_64-linux" "aarch64-darwin" ];
    root = ./.;
    dirName = "nix"; # (default)
    inputs = inputs;
  };
}

With this setup, if there is no nix subdirectory in the myproj, our flake will have no outputs.

$ nix flake show
git+file:///home/myuser/tmp/myproj?ref=refs%2fheads%2fmaster&rev=b9748c5fcb913af50bedaa8e75757b7120a6a0ba

If we want to introduce a new package hello to our project, we can add a new file in the myproj/nix/packages/hello.nix file.

Once we version this new file into the repository, the nix flake show output will have the new package available:

$ nix flake show
git+file:///home/roman/myproj?ref=refs%2fheads%2fmaster&rev=<sha>
└───packages
    ├───aarch64-darwin
    │   └───hello: package 'hello'
    └───x86_64-linux
        └───hello: package 'hello'

nixDir adds the package automatically, and it does it with the systems that we specified in the nixDir invocation in the flake.nix file.

Following are the various conventions that you can use with nixDir

Outputs

Note The examples bellow assume the configured nixDir is called nix

The packages output

To add new packages, add an entry in your nix/packages directory. The package entry may be a nix file, or a directory with a default.nix. The name of the file/directory will be the name of the exported package. For example:

# nix/packages/hello.nix

# packages receive the system, the flake inputs, and an attribute set with
# required nixpkgs packages.
inputs: { hello, writeShellScriptBin, makeWrapper, symlinkJoin }:

# We are going to do an essential wrapping of the hello package, following steps
# from: https://nixos.wiki/wiki/Nix_Cookbook#Wrapping_packages
symlinkJoin {
  name = "hello";
  paths = [ hello ];
  buildInputs = [ makeWrapper ];
  postBuild = ''
    wrapProgram $out/bin/hello --add-flags "-t"
  '';
}

Your package file must receive two arguments. The first argument is the flake's inputs, and the second argument is an attribute set with all the required dependencies for the package (e.g. callPackage convention).

Warning Packages could either be a nix file or a directory, nixDir will fail if it finds both a directory and a file with the same name.

Remove packages from a particular system platform

In some situations, you may not be able to build a package for a certain platform. nixDir will help you remove a package for a specific system platform if the package metadata's platforms attribute indicates the package is not supported by such system.

If a package doesn't configure the platform's metadata, nixDir will include the package in every specified system platform by default.

Provide packages in flake.nix

In some situations, creating a new file for a package may be a cumbersome task. You may provide the packages parameter to the buildFlake call. The packages parameter must be a function that receives the imported nixpkgs and returns an attribute-set of packages to include in the flake packages output.

Example:

{
  inputs = {}; # ...
  outputs = { nixDir, ... } @ inputs:
    nixDir.lib.buildFlake {
     # required options defined here
     packages = (pkgs: {
       my-package = pkgs.mkDerivation {
        # ...
       };
     });
    };
}

Generate the all package

Usually flake maintainers would like to be able to build and copy all the derivations from a flake. When authors enable the generateAllPackage option, an all package gets created.

The all package depends on all the packages generated by nixDir, as well as all the shells used in your project. The all package can be used to build and share the output derivations of all the flake packages with a remote nix store.

Example:

{
  inputs = {}; # ...
  outputs = { nixDir, ... } @ inputs:
    nixDir.lib.buildFlake {
     # required options defined here
     generateAllPackage = true;
    };
}

The lib output

To add a lib export to your flake, include a nix/lib.nix inside your nix directory. For example:

# nix/lib.nix

inputs: {
  sayHello = str: builtins.trace "sayHello says: ${str}" null;
}

The lib.nix file must export a function that receives the flake inputs as parameters.

Note Given that library functions should be system agnostic, the nix/lib.nix file does not receive the system argument.

The getPkgs function

The lib flake output by default will always contain a getPkgs function. This function is responsible of importing the nixpkgs input with the appropriate layouts and configuration.

Across all the flake source code, whenever you are using a given pkgs variable, it was imported using the getPkgs function. This establishes some consistency when importing nixpkgs, ensuring that:

  • You are always using the same overlays

  • You are always using the same nixpkgs.config setup

You may change its behavior depending on parameters you pass to the buildFlake invocation. For example, the nixpkgsConfig argument allows you to override the configuration of nixpkgs (e.g., allowUnfree, allowInsecure, etc.)

Following is an example:

{
  outputs = { nixDir, ... } @ inputs:
    nixDir.lib.buildFlake {
      inherit inputs;
      systems = [ "x86_64-linux" ];
      root = ./.;
      # the configuration bellow allows packages from this flake
      # to contain dependencies that are unfree.
      nixpkgsConfig = {
        allowUnfree = true;
      };
      # assuming there is a 'develop' entry inside the nix/overlays.nix file, this
      # statement will inject the overlay across all your flake source code.
      injectOverlays = [ "develop" ];
    };
}

The overlays output

To create overlays, nixDir looks for the nix/overlays.nix file. This file must receive the flake inputs as a parameter and return an attribute set with every named overlay. Following is an example:

# nix/overlays.nix

{
  self,
  nixpkgs,
  my-flake-dependency1,
  my-flake-dependency2,
  ...
}:

let
  default = final: prev:
    self.packages.${prev.system};

  develop =
    nixpkgs.lib.composeManyExtensions [
        my-flake-dependency1.overlays.default
        my-flake-dependency2.overlays.default
      ];
in
{
  inherit default develop;
}

In the example above, we are creating two overlays, the one named default includes all the packages this flake exports into the nixpkgs import. The one named develop includes the overlays of some of our flake inputs.

Using overlays in the nixpkgs import

There is an optional functionality to inject your flake overlays and use custom packages across your flake. Following is an example:

# flake.nix

{
  # inputs = {};
  outputs = { nixDir, ... } @ inputs:
    nixDir.lib.buildFlake {
      inherit inputs;
      systems = ["x86_64-linux"];
      root = ./.;
      # We want the packages injected by the `develop` overlay
      # which is defined as an entry in our `nix/overlays.nix` file.
      injectOverlays = [ "develop" ];
    };
}

In the example above, the develop overlay (which was defined on your nix/overlays.nix file and includes the overlays of some of your flake inputs) will be included in every nixpkgs import used within your flake outputs.

Note Given that flake overlays should be system agnostic, the nix/overlays.nix file does not receive the system argument.

Various modules outputs

The nix eco-system provides many different kinds of modules. The module outputs currently supported by nixDir are:

  • nixosModules -- it allow authors to write modules supported by NixOS in the nix/modules/nixos directory.

  • darwinModules -- it allow authors to write modules supported by nix-darwin in the nix/modules/darwin directory.

  • homeManagerModules -- it allow authors to write modules supported by home-manager in the nix/modules/home-manager directory.

  • devenvModules -- see devenv section for details

Following is an example of nix-darwin module:

# nix/modules/darwin/fonts.nix

inputs: {pkgs, ...}:

{
    fonts.fontDir.enable = true;
    fonts.fonts = with pkgs; [
      recursive
      (nerdfonts.override { fonts = [ "JetBrainsMono" "FiraCode" "DroidSansMono" ]; })
    ];
};

The module file must receive two arguments. The first argument contains your flake's inputs, and the second argument is the attribute set that general module systems expects (e.g. {pkgs, config, ...}). The validity of the settings may depend on which backend the module is intended for (nix-darwin, nixos or home-manager).

Passthrough keys

Sometimes it makes sense to not have a dedicated file for the configuration of some flake outputs; the most common situation where authors usually want to declare a resource in-place is for nixConfigurations, darwinConfigurations and homeManagerConfigurations. Given they must be simple declarations, you can inline them into your buildFlake call. Following is an example:

{

  inputs = {
    # ...
  };

  outputs = { nixDir, nixpkgs, nix-darwin, ... } @ inputs:
    nixDir.lib.buildFlake {
      systems = [ "x86_64-linux" "aarch64-darwin" ];

      homeManagerConfigurations = {
         # ... (your configuration here)
      };

      darwinConfigurations = {
        myMacbook =  nix-darwin.lib.darwinSystem {
          # ... (your configuration here)
        };
      };

      nixosConfigurations = {
        myLaptop = nixpkgs.lib.nixosSystem {
          # ... (your configuration here)
        };
      };
    };
}

Third-Party Integrations

nixDir can run devenv profiles (using nix flakes porcelain) automatically.

To add a new devenv, add an entry in the nix/devenvs/ folder. Following is an example, of a very basic devenv profile.

# nix/devenvs/my-devenv.nix

inputs: { config, pkgs, ... }:

{
   languages.go.enable = true;
   packages = [ inputs.self.packages.${system}.my-dance-music ];
   enterShell = ''
     echo "everybody dance now!"
   '';
}

In the same way we have it with other nixDir components, your devenv profile must add one extra parameter, the inputs of your flake.

If you invoke nix flake show, you'll notice there is a new entry in the devShells outputs called my-devenv (the name of the file containing the devenv profile)

To run your devenv profile, run the nix develop command using the name of the devenv profile.

nix develop .#my-devenv

Warning devenv modules and devShells work on the devShells namespace, nixDir will fail if there is an entry on both nix/devenvs and nix/devShells directories with the same name.

devenvModules output

Your flake is able to export devenvModule entries by adding a nix/modules/devenv directory. Following is an example:

# nix/modules/devenv/my-hello/default.nix

inputs : { config, lib, pkgs, ... }:

let
  cfg = config.services.my-hello;

  startScript = pkgs.writeShellScriptBin "start-my-hello" ''
    set -euo pipefail
    while true; do ${pkgs.hello}/bin/hello -g "my-hello enabled" && sleep 1; done
  '';
in
{
  options = {
    services.my-hello = {
      enable = lib.mkEnableOption "My Hello World app";
    };
  };

  config = lib.mkIf cfg.enable {
    processes.my-hello.exec = ''${startScript}/bin/start-my-hello'';
  };
}

Your devenv module file must receive two arguments. The first argument contains the flake's inputs, and the second argument is the attribute set that devenv modules expect (e.g. {pkgs, config, ...}).

You may inject the devenv modules on all your flake devenv configurations (e.g. nix/devenvs) by specifying the injectDevenvModules option in the nixDir.lib.buildFlake call. The argument may be a list of module names (the name of the directory or file found in nix/modules/devenv) or a boolean value true to import all devenv modules.

You can see an example bellow using a boolean for the injectDevenvModules entry:

# flake.nix

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    nixDir.url = "github:roman/nixDir";
  };

  outputs = {nixDir, ...} @ inputs:
    nixDir.lib.buildFlake {
      inputs = inputs;
      systems = ["x86_64-linux" "x86_64-darwin" "aarch64-darwin"];
      root = ./.;
      # import all devenv modules in my devenv shells
      injectDevenvModules = true;
      # ^^^^^^^^^^^^^^^^^^^^^^^^^
    };
}

When handling your flake in code, a new export called devenvModules is registered in the flake's outputs:

$ nix flake show
git+file:///home/rgonzalez/Projects/oss/nixDir?dir=example/myproj
└───devenvModules: unknown

A note on loading time

As it stands today, the devenv project requires many uncached dependencies that will take some time to build. To skip long build times, we recommend adding their cachix setup, or to include it on your flake:

{
  description = "myproj is here to make the world a better place";

  nixConfig = {
    extra-trusted-public-keys = "devenv.cachix.org-1:w1cLUi8dv3hnoSPGAuibQv+f9TZLr6cv/Hm9XgU50cw=";
    extra-trusted-substituters = "https://devenv.cachix.org";
  };

  # inputs = {};
  # outputs = {};
}

Warning We do not recommend overriding devenv flake dependencies as this would cause remote cache misses and it would produce really slow build times.

nixDir is able to integrate a single pre-commit-hooks.nix to devShells entries. This is an optional functionality; to enable it, you must have a nix/pre-commit.nix file and include a pre-commit-hooks input in your flake.

Following is an example of a pre-commit configuration.

# nix/pre-commit.nix

inputs: pkgs:

{
  # root of the project
  src = ../.;
  hooks = {
    nixfmt.enable = true;
  };
}

The file must receive two arguments, the the flake inputs and the imported nixpkgs.

You may disable the automatic injection of pre-commit hooks using the injectPreCommit option (defaults to true) in the nixDir.lib.buildFlake call.

Note As opposed to other nixDir components, the nix/pre-commit.nix receives all packages rather than relying on the callPackage convention

Accessing the pre-commit hook explicitly

Another side-effect that occurs when using the nix/pre-commit.nix is that nixDir appends a preCommitRunScript attribute to the flake's lib. This attribute contains the pre-commit script, and it may be used as a value in other places (like a docker image). Following is an example on how to add the script in a docker image package:

# nix/packages/devenv-img.nix

{self, ...}: {
  lib,
  dockerTools,
  buildEnv,
  bashInteractive
}: let

dockerTools.buildImage {
  tag = "latest";
  name = "devenv-img";
  copyToRoot = buildEnv {
    name = "devenv-img";
    paths = [
      bashInteractive
    ];
    pathsToLink = ["/bin"];
  };
  config = {
    WorkingDir = "/tmp";
    Env = [
      # Inject pre-commit script to your container environment
      "PRE_COMMIT_HOOK=${self.lib.preCommitRunScript.${system}}"
    ];
  };
}

Nixt is an attempt of unit tests for the nix programming language. When flake authors include the directory nix/tests/nixt, this utility will discover the tests and allow the nixt binary to run tests. Following is an example of a nixt test.

{ self, ... } @ inputs: { describe, it }:

let
  input = { hello = true; };
in
[
  (describe "hello world"
    (it "must not be surprising"
      # the second argument must be a boolean value, if false
      # the test is considered an assertion error.
      builtins.hasAttr "hello" input))
]

To run the tests, make sure to include the nixt input in your flake.

nix run .#nixt

You may disable the generation of the nixt application by setting the injectNixtCheck option to false in your buildFlake call.

FAQ

Should I nixDir?

If you are maintaining a project with nix flakes that has a big flake.nix file (>500 LOC) or that involves several nix files, you may benefit from this library.

About

Transform a directory into a nix flake

Topics

Resources

Stars

Watchers

Forks

Packages

No packages published