Some services will be more complicated, requiring more than the current facilities. These can be added.

let's elaborate on that so people who may not be as familiar with the history of this PR up to this point stand a chance at understanding this

the new system.services module in this PR currently does not currently have, but could require the ability to:

• create system users
• place files under /etc
• set runtime parameters of the linux kernel, as set by sysctl
• add packages to the global system environment
• open ports in the system firewall

the above is just a list of the most common/useful patterns that many nixos service oriented modules use, but anyone else should feel free to edit this comment and add to it if they feel it useful to do so - as well as check off the functionality if they contribute it to this PR

thanks for getting the ball rolling on this one @roberth 🙇‍♂️

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/pre-rfc-decouple-services-using-structured-typing/58257/16

I understand the nesting of services but I'm not sure to understand the use in practice. Would this be used to provide opinionated coupling between services? For example there would be a service grouping Nextcloud and Postgres and others to provide an out-of-the-box experience? And this would allow to split that big module into multiple ones?

This can be used for groupings of multiple "off-the-shelf" services into a common configuration, or for non-trivial services that consist of multiple executables, such as apache cassandra, or even a mix, such as pulsar.

I think the distinguishing factor is that when a sub-service is an off the shelf one, such as a database, admins may prefer to have a single db service with multiple apps that consume it. @ibizaman this is where I think the ideas you've worked out apply.
This could be a configurable thing as well - a boolean in the "parent" service that controls whether sub-services are generated to fill the requests.

I'd love to get your input

Of course! I will have a look soon-ish.

a few [...] questions

Happy to help :)

I think the distinguishing factor is that when a sub-service is an off the shelf one, such as a database, admins may prefer to have a single db service with multiple apps that consume it.

Agreed. The end user should have the choice of either an instance per service or one instance for all of them.

This can be used for groupings of multiple "off-the-shelf" services into a common configuration, or for non-trivial services that consist of multiple executables, such as apache cassandra, or even a mix, such as pulsar.

For non-trivial packages with multiple binaries, I indeed don't see how contracts can apply. But if we're talking about multiple services working together, this is where our work could be merged and become even more powerful. If one write a group of services, like you allow, and the link between those services are contracts, then we can easily swap one of those services for another one.

system.services is probably too close to systemd.services. People already get confused by toplevel services and systemd.services. Not sure what would be better though :/

Perhaps systemServices is sufficiently distinct?

Adding any extra words is probably a net negative.

{ serviceModules, ... }:
{
  systemServices.etcd = { imports = [ serviceModules.etcd ]; };
}

An intriguing thought is that "subsystem" could replace "service", which seems appropriate when these things encompass more than a process and its execution settings, while also giving space for multiple services to exist within it.
However, "subsystem" hardly gives any intuition for what the feature is for, and the "system" part of "subsystem" conflicts with the system role of these services.
It also reminds me that a current flaw is that process is required, even if you want to just compose a "subsystem" from multiple services instead of a custom process. I guess a nullOr submodule would fix that.

About naming, it seems you haven't considered changing the 'services' part 🤔

Systemd base resource names are units not services, and this terminology works for all resource types that systemd supports, like services, timer, target, socket, mount...

So in our case we could have system.units ?

Although this might easily increase the coverage support for types of manageable systemd resources, it might be too much for this proposal 🤔

So in our case we could have system.units ?

Similar problem because we do have systemd.units :P

So in our case we could have system.units ?

Similar problem because we do have systemd.units :P

systemUnits then?

units

This feature isn't really about systemd units. For instance, the process options are applicable to any init system or process supervisor, and it is possible to write modular services that support all such systems (by sticking to portable options) or supporting specific systems by feature testing (config = ... // optionalAttrs (options?systemd) ...; or other, nicer mechanisms).
Similarly, assertions and warnings are a configuration management facility that goes beyond systemd, but more significantly, so are the capabilities listed by @aanderse.

Does this take any position on configuration files (like a recommended way of integrating with NixOS/rfcs#42 for example)?

The proof of concept uses an option tree that starts with the name of the software, so you'd have options in a service like

• nginx.configFile
• nginx.settings

We could pick a fixed prefix instead, e.g. app => app.settings, app.configFile.
It would still be free-form below app, so not immediately suitable for generic code, but at least it doesn't spoil the namespace with an unbounded set of names that might impede supervisor integrations.

(like a recommended way of integrating with NixOS/rfcs#42 for example)?

42 is great in many contexts, so definitely worth a mention. As for integrating, if we go with app, then app.settings would make for a good default place to put them, certainly if the service has a single config file or a clear "main" config file.

My experience reusing NixOS modules as «modular» services on non-NixOS is that sometimes the config generation is put into a let so usage outside putting the config file in the command line of the start command is blocked. And sometimes there are files put into /etc which can be fished out from /etc generation code but not from a clear place in the module. That's why I think that maybe establishing some kind of best practices on where does the extra options go and where does the config file generation go might help with flexibility / reusability (and thus portability).

@7c6f434c We could make it a convention to put config file stuff in a separate module that doesn't have any process-related options or definitions. This module is then responsible for declaring options like settings and configFile. Definitions for settings would be minimal, and configFile defaults to the generated file containing the settings.
This way you can evalModules or imports it separately and use it basically like a function from settings to configFile.

The main module imports the config file module and adds a generic process definition and/or supervisor-specific definitions, depending which one they're loaded into using (options?some.attr.path) or something fancier.

We don't have something for symlinked config files yet ("etc/"), but that would just mean that instead of configFile it's a standard option with different semantics.

app

Having second thoughts.

• weird (it's like the early 00's idea of app?)
• not related to flake apps
• feels like it adds significant noise
• does not seem to enable generic code to use it or anything
• @7c6f434c's use case does not involve generic code that works for more than one service at a time, it seems

I like this change overall pretty much. I have the concern that services are now more difficult to discover since they have (even) more ways to be enabled now. So i hope we understand the significance that is going to be added with this PR and thinking that it really presses on the need of improved discoverability.
But it's also certainly a step in the right direction, where people can find service via search.nixos.org more easily because the package also exports the service (should be shown and searchable). We should improve the discoverability in parallel to allow for such changes. But i wouldn't block on that happening first.

Also adding those services opens a new opportunity of abstracting away from systemd and getting closer to the actual user via the options that each specific service defines. Layered approach ontop of systemd or alternatives. While systemd might be a good interface for computer systems it is not always the best language towards configuring more high level or user oriented services.

@aanderse Even without the features you've listed, modular services can be used by adding the necessary NixOS configuration by hand. We should improve on that, but I consider them to be fairly independent additions that will each require some design and implementation work, and they don't block this feature.
Nonetheless, I would be amiss not to give my thoughts on them.

the new system.services module in this PR currently does not currently have, but could require the ability to:

• create system users

Yes, this would fit. We'd have to work out some details, but I'd propose that a service module has something like

{ lib, config, user, ... }: {
  config.allocateUser = true;
}

The user module argument would then contain the user name that's generated by the service manager integration.
If the process doesn't need to know its user name, user, can just be ignored and omitted.

Other designs are possible. It depends a bit. Do we want the generic interface to be lean, or should it make it as easy as possible to deal with environment differences. For example, a supervisord integration would probably not provide this functionality.

Lean interface, worse DX:

  config = { } // optionalAttrs (options?allocateUser) {
    allocateUser = true;
  };

or we could have wantUser = true; vs requireUser = true;, or options.user = lib.mkOption { type = enum [ true "optional" false ]; }

Support for multiple users in one (leaf) service is probably not needed. I'd expect services that run as multiple uids to be composite services, where each of the leaf services can request their own user as needed.

• place files under /etc

Wouldn't be inherently modular if they can be placed in arbitrary places.
Here we could also let the service manager integration pick a location.

• set runtime parameters of the linux kernel, as set by sysctl

I'm not sure this is desirable, but we could at least add an option to enable a service to check the suitability of its host NixOS.

This means adding a new option to nixos/modules/system/service/systemd/system.nix so that a service can do:

{
  nixos.assertions = nixos@{config, ...}: 
    [ /* usual assertion format, referencing nixos and any local configs from the service */ ];
}

and this would then be picked up, with some context added to the message.
Note that to the service module itself, this is just a function. It doesn't get to read the NixOS configuration because of this.

If there's things in sysctl that we can enable safely, we could add options to cause that to happen, but all of sysctl seems too much.
Also affecting all of a NixOS configuration is tricky because of the circularity of that => infinite recursions.

• add packages to the global system environment

This is also trouble if you want to run multiple instances of a service.
We could add a shell runner that's named after the service in question perhaps?
Or only add commands that are prefixed with the service name, and then that could contain shell runner like

• [WIP] devShellTools.buildShellEnv #330822

• open ports in the system firewall

Could be done.

Would be good to have a check that no more than on service opens the same port. That's a broader set of ports than "public" ports though, so that will require differentiation in the service modules, if we want such a feature.

I think a networked service could look like:

{ ports, ... }:
{
  ports.http = { mustBe = 80; zone = "public"; };
  ports.ipc = { zone = "localhost"; type = "udp"; }; # arbitrary local port, deterministically allocated, usually the same between generations
  foo.settings.httpPort = ports.http;
  foo.settings.ipcPort = ports.ipc;
  # ...
}

Whether to automatically open ports in the "public" zone (or whatever) is something that mirrors the discussions that were had around openFirewall. A global option to auto-open all declared ports would be nice.

I am confident that we'll land on good solutions for most of these things, and it's more than ok for a service to be a non-modular NixOS service.

Let's get this going!

Backport failed for release-25.05, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin release-25.05
git worktree add -d .worktree/backport-372170-to-release-25.05 origin/release-25.05
cd .worktree/backport-372170-to-release-25.05
git switch --create backport-372170-to-release-25.05
git cherry-pick -x b915f0c5c0307c82f7834d683cf81414ce73d01b 1acabeebed483fd8a682eb6b7ef538f5b99d35e2 109a6a9d1e13013a4d3e4220d79d0f599a10ac97 53f97deb26d3defe67bbe1223efd2c5c4b9def1e b7ded190104f8b8af63d10355377b094dbc032f1 7d7b678ed6664f42223e05113e596e0947bb1a04 03c4d4bc664963b488ad575f1993890d8cd9a650 b9e4118e6dd800509784b16234f43fd8e8fb3470 af04a80c7627a83e2c0eaf71b44c4a71397a6201 d20a5749486886032aedec736311e4c3d29a48c7 b51a6c3531c669c744778eb8c986bd7e9b19dcee 1f4e47626931468baeff906a38122af052d1e53a

This should really have removed the redundant revert and squashed the nixos release change before merge.

Ok. I'll adjust my dials to capture less history in Nixpkgs.

I love this! Some notes:

Composability brings to mind infuse.nix, a library that builds, for example, .override calls neatly.

I thought of an API like this, could it work? I guess it has the problem of not being able to configure dependencies but I was thinking of having non-NixOS-managed service depenencies. Maybe DAGs or attrnames would work better?

{ config, lib, ... }:
# {  _type = "systemd"; perMachineUuid = "..."; ... exports.url = ""; }
serviceInstances.database = modules.postgres {};

# {  _type = "external"; exports.url = "redis://10.0.0.2:1234"; }
serviceInstances.cache = modules.redisExternal { host = "10.0.0.2"; port = 1234; auth = "?"; };

# produces dependency on `database`, automatically doing permissioning and provisioning secrets
serviceInstances.webapp = modules.webapp { inherit (config.serviceInstances) database cache; };


modules.webapp = { database, cache }: let
  # resolve/fix/etc; just look for it in `config`
  database = config.resolveServiceInstanceOrThrow database;
  cache = config.resolveServiceInstanceOrThrow cache;
in {
  env.DATABASE_URL = database.exports.url;
  env.CACHE_URL = cache.exports.url;

  require_units_at_startup = [
    database # resolves to systemd.Requites = [ database.exports.systemd.unitName ];
    cache # resolves to ??? (see below)
  ];
};

We really need a contract system here, as specified in the Discourse thread in the issue text. For example, how do you specify dependencies on things that aren't managed by the current NixOS instance? With systemd you can do some things like waiting for the network or mounts to be online, but could you somehow dynamically integrate external sources here?

Seeing as modifying global state is disallowed (service instances can't set environment.systemPackages), I think this is a natural step forward; adding integration for Kubernetes and similar scheduling systems, while basing the configuration on Nix/NixOS.

If the process doesn't need to know its user name, user, can just be ignored and omitted.

Could this be expanded to support systemd's DynamicUser? I.e. don't create a user.

Also let's use abstractions and FHS as much as we can: e.g. systemd's StateDirectory.

Hi @axelkar

infuse.nix

It is possible to add a translation layer like that on top, but for now I would recommend to stick to the normal module syntax, so that it's a bit easier to reason about potential changes we want to make to the implementation.

A crucial aspect of this new usage pattern of the module system is more extensive use of imports, in the context of submodules.

I didn't spot something for imports in infuse.nix just now.
Modules aren't exactly overriding, so I don't know if it's the right place, but I haven't tried it.

How do you specify dependencies on things that aren't managed by the current NixOS instance?

Currently we don't. Three possible solutions:

• Services should be robust to failure of external dependencies. Otherwise your distributed system gets weaker and harder to manage with every service you add. (Local services are just a "special" case of distributed in this context, and not special at all when it comes to this aspect.) Startup can be seen as just the resolution of the previous failure to be available. => Make the service implementation keep trying instead of giving up.
• Use more of the optional systemd options. More unit types can be forwarded by this NixOS integration (system.services etc).
• Write NixOS services and such. We could argue that some things relate between a service and the system, and are not an intrinsic part of the modular service. This would seem unsatisfying, but maybe modular services are just a small piece in the puzzle.

Seeing as modifying global state is disallowed

👍 That is definitely an important aspect of this system. I assume you mean host-global, and that would be the domain of normal NixOS modules; those model a whole host.
Something is only "global" when the host isn't considered part of the data model.

Could this be expanded to support systemd's DynamicUser? I.e. don't create a user.

We might want to make that the default setting, perhaps?

Also let's use abstractions and FHS as much as we can: e.g. systemd's StateDirectory.

SGTM. If other process manager integrations can also provide a degree of compatibility with features like that, we could even make them part of the portable interface.

Modules aren't exactly overriding, so I don't know if it's the right place, but I haven't tried it.

It's designed for derivations and overlays, but there's apparently an operating system named based on it named sixos. I couldn't find any examples earlier, but here's one.

I do agree that it's probably not a good suit here, but I found it interesting. Both the NixOS module system and overlays/Nixpkgs do (in my mind at least) use fixed points, but the NixOS module system has better syntax and looks more beginner-friendly. One downside is that there is only a number representing ordering and AFAIK you can't update a value like you would with overlays (e.g. override a package based on the old version and copy the old version into another attribute).

Eval costs and verbose error messages are another thing.

How do you specify dependencies on things that aren't managed by the current NixOS instance?

Currently we don't. Three possible solutions:

• Services should be robust to failure of external dependencies. Otherwise your distributed system gets weaker and harder to manage with every service you add. (Local services are just a "special" case of distributed in this context, and not special at all when it comes to this aspect.) Startup can be seen as just the resolution of the previous failure to be available. => Make the service implementation keep trying instead of giving up.

• Use more of the optional systemd options. More unit types can be forwarded by this NixOS integration (system.services etc).

• Write NixOS services and such. We could argue that some things relate between a service and the system, and are not an intrinsic part of the modular service. This would seem unsatisfying, but maybe modular services are just a small piece in the puzzle.

Seeing as modifying global state is disallowed

👍 That is definitely an important aspect of this system. I assume you mean host-global, and that would be the domain of normal NixOS modules; those model a whole host.
Something is only "global" when the host isn't considered part of the data model.

If I was being unclear, global state was referring to NixOS system-wide configuration. Submodules can't by themselves affect anything outside the submodules.

Modularity in services here could mean that you can run them without the static global state. See services-flake and Devenv services.

There's also wrapper-manager that takes everything even further by putting configuration into wrappers. I personally wouldn't use it with daily desktop programs because they wouldn't be able to see configuration changes. Or.. I guess I do use Nixvim with its wrapper so I have to restart Neovim when I make changes...

I think we might want to keep this reloadabilty in mind, because some services like Nginx support it too.

Could this be expanded to support systemd's DynamicUser? I.e. don't create a user.

We might want to make that the default setting, perhaps?

I agree! For most services, there are no downsides and without service users everything is even more modular.

Also let's use abstractions and FHS as much as we can: e.g. systemd's StateDirectory.

SGTM. If other process manager integrations can also provide a degree of compatibility with features like that, we could even make them part of the portable interface.

I'm thinking we could even make this the default and give modules parameters with the state, cache (if enabled), etc. directories.

Hey all, just a reminder that this is just a start!

We now have tracking issue

• Modular services tracking #428084

and a Matrix room for discussions

• https://matrix.to/#/#modular-services:nixos.org