mirror of https://github.com/nocturn9x/nimd.git
Improved README. Initial (broken!) work on dependency resolution
This commit is contained in:
parent
f2b23afe1b
commit
b12cf0f5aa
109
README.md
109
README.md
|
@ -12,39 +12,94 @@ long time. Sometimes programming only large scale software is boring, go figure.
|
||||||
|
|
||||||
## Disclaimers & Functionality
|
## Disclaimers & Functionality
|
||||||
|
|
||||||
This software is developed on a _"It works on my machine"_ basis. I don't perform any extensive testing: if this thing unmounts your
|
This software is developed on a _"It works on my (virtual) machine"_ basis. I don't perform any extensive testing: if this thing unmounts your
|
||||||
root partition while you're saving your precious family photos, I can't do much. I run an installation of Artix Linux (x86_64)
|
root partition while you're saving your precious family photos, I can't do much (although it probably won't). I currently test NimD inside a
|
||||||
using the 5.15.5-artix1-1 linux kernel.
|
minimal Alpine VM that runs the 5.10.0-9-amd64 version of the Linux kernel.
|
||||||
|
|
||||||
Also, NimD is developed **for Linux only**, as that's the kernel my OS uses: other kernels are not supported at all and NimD
|
NimD is developed **for Linux only**, as that's the kernel my OS uses: other kernels are not supported at all and NimD **will** explode with fancy
|
||||||
**will** explode with fancy fireworks if you try to run it unmodified on other kernels (although probably things like BSD and Solaris
|
fireworks if you try to run it unmodified on other kernels (although probably things like BSD and Solaris are not that hard to add support for).
|
||||||
are not that hard to add support for using some `when defined()` clauses and changing what virtual filesystems NimD expects to mount).
|
|
||||||
|
|
||||||
NimD is not particularly secure. Actually it's probably very insecure by modern standards, but basic checks like making sure regular users
|
NimD is not particularly secure (although basic checks like making sure regular users can't reboot the machine are in place), but it doesn't need to be: the only thing it does is run the services you provide to it, that's it*. No `nimd-modulenamed` madness, no `libnimd.so` libraries to link against, NimD only runs your services: if it blows up, it's your fault (or it's a bug).
|
||||||
can't reboot the machine are (_actually_, will be) at least in place, so there's that I guess.
|
|
||||||
|
|
||||||
NimD assumes that the standard file descriptors 0, 1 and 2 (stdin, stdout and stderr respectively) are properly connected to /dev/console
|
NimD expects the 3 [standard streams](https://en.wikipedia.org/wiki/Standard_streams) to be properly connected to `/dev/console` (which is something all modern versions of the Linux kernel do). I tried connecting them manually, but I was out of luck: if you happen to know how to check for (or open) them and connect them manually, please make a PR, I'd love to hear how to do that.
|
||||||
(which is something all modern versions of the Linux kernel do). I tried connecting them manually, but I was out of luck: if you happen to
|
|
||||||
know how to check for a proper set of file descriptors and connect them manually, please make a PR, I'd love to hear how to do that.
|
|
||||||
|
|
||||||
When mounting the filesystem, NimD is at least somewhat smart:
|
_*_: Well, almost. If you don't wanna write oneshot services for simple things like creating symlinks/directories (especially if you plan running BSD ports of
|
||||||
- First, it'll try to mount the standard POSIX virtual filesystems (/proc, /sys, etc) if they're not mounted already (you specify which)
|
some program on Linux) and mounting your drives then NimD can do it for you, but just because it _can_ doesn't mean it _has to_: you choose! NimD has a builtin
|
||||||
- Then, it'll parse /etc/fstab and mount all the disks from there as well (unless they are already mounted, of course).
|
fstab parser and can operate entirely independently of the `mount` command, since it directly hooks up to `mount`, `umount` and `umount2` inside `sys/mount.h`
|
||||||
Drive IDs/UUIDs, LABELs and PARTUUIDs are also supported and are automatically resolved to their respective /dev/disk/by-XXX symlink
|
|
||||||
|
|
||||||
__Note__: To check if a disk is mounted, NimD reads /proc/mounts. If said virtual file is absent (say because we just booted and haven't mounted
|
## Setup
|
||||||
/proc yet), the disk is assumed to be unmounted and is then mounted. This seems fairly reasonable to me, but in the off chance that said disk is
|
|
||||||
indeed mounted and NimD doesn't know it, it'll just log the error about the disk being already mounted and happily continue its merry way into
|
|
||||||
booting your system (_hopefully_), but failing to mount any of the POSIX virtual filesystems will cause NimD to abort with error code 131 (which
|
|
||||||
in turn will most likely cause your kernel to panic) because it's almost sure that the system would be in a broken state anyway.
|
|
||||||
|
|
||||||
The way I envision NimD being installed on a system is the following:
|
NimD expects to be installed like so:
|
||||||
- /etc/nimd -> Contains configuration files
|
- `/etc/nimd` -> Contains configuration files and utilities like `reboot` and `poweroff`
|
||||||
- /etc/nimd/runlevels -> Contains the runlevels (think openrc)
|
- `/etc/nimd/runlevels` -> Contains the runlevels (`boot`, `default`, `shutdown`)
|
||||||
- /var/run/nimd.sock -> Unix domain socket for IPC
|
- `/etc/nimd/nimd.conf` -> NimD's own configuration file
|
||||||
- /etc/runlevels -> Symlink to /etc/nimd/runlevels
|
- `/var/run/nimd.sock` -> Unix domain socket for IPC
|
||||||
- /sbin/nimd -> Actual NimD executable
|
- `/etc/runlevels` -> Symlink to `/etc/nimd/runlevels`
|
||||||
- /sbin/init -> Symlink to /sbin/nimd
|
- `/sbin/nimd` -> Actual NimD executable
|
||||||
|
- `/sbin/init` -> Symlink to `/sbin/nimd`
|
||||||
|
- `/bin/{poweroff,shutdown,reboot,halt}` -> Minimal utilities that communicate with NimD to poweroff/shutdown/reboot/halt the machine
|
||||||
|
- `/bin/nimdctl` -> Utility to interact with NimD (add/remove/start/stop services, read logs, inspect services' status, etc)
|
||||||
|
|
||||||
|
|
||||||
|
__Note__: The runlevels directory contains `*.conf` files (or they can also be symlinks, NimD doesn't care): those are NimD's own unit files.
|
||||||
|
|
||||||
|
|
||||||
|
## Unit files
|
||||||
|
|
||||||
|
Services in NimD are called _unit files_ or just _units_ (I know, __very__ original). They are configuration files that tell NimD what to do once
|
||||||
|
it has booted your system
|
||||||
|
|
||||||
|
### Dependency management
|
||||||
|
|
||||||
|
Unlike some other init systems (most notably, runit) NimD is _dependency based_: to understand this relatively simple concept disguised as a fancy term,
|
||||||
|
you have to understand NimD (like many others) relies on the concepts of _dependents_ (units that _depend_ on some others to work) and _providers_ (units
|
||||||
|
that _provide_ services to their dependents and that may in turn have dependencies themselves). For example, if you wanna start an SSH server, you probably
|
||||||
|
want to make sure your disks are mounted and that your network has been set up. To do that, you can write something like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
[Service]
|
||||||
|
|
||||||
|
name = ssh # The name of the service
|
||||||
|
description = Secure Shell Server # A short description
|
||||||
|
type = simple # Other option: oneshot (i.e. runs only once, implies supervised=false)
|
||||||
|
exec = /usr/bin/sshd <args> # Note: this is not passed trough the shell, it's executed directly
|
||||||
|
depends = net,fs # This service will be started only when these dependencies are satisfied
|
||||||
|
provides = ssh # Dependents can also be providers
|
||||||
|
restart = always # Other options are: never, onFailure
|
||||||
|
restartDelay = 10 # NimD will wait this many seconds before trying to start it again
|
||||||
|
supervised = true # This is the default. Disable it if you don't need NimD to watch for it
|
||||||
|
|
||||||
|
[Logging]
|
||||||
|
|
||||||
|
stderr = /var/log/sshd # Path of the stderr log for the service
|
||||||
|
stdout = /var/log/sshd # Path of the stdout log for the service
|
||||||
|
stdin = /dev/null # Path of the stdin fd for the service
|
||||||
|
```
|
||||||
|
|
||||||
|
A dependency name can either be the name of a unit file (without the `.conf` extension), or one of the following placeholders:
|
||||||
|
- `net` -> Stands for network connection. Services like NetworkManager and dhcpcd should be set as providers for this
|
||||||
|
- `fs` -> If you mount your disks using a oneshot service (recommended for the best experience), your service should provide this
|
||||||
|
|
||||||
|
|
||||||
|
## Configuring NimD
|
||||||
|
|
||||||
|
NimD's own configuration file is located at `/etc/nimd/nimd.conf` and its syntax is similar to those of unit files (i.e. uses an
|
||||||
|
INI-like structure), but the options are obviously different. An example config would look something like this:
|
||||||
|
|
||||||
|
```
|
||||||
|
[Logging]
|
||||||
|
|
||||||
|
level = info # Levels are: trace, debug, info, warning, error, critical, fatal
|
||||||
|
logFile = /var/log/nimd # Path to log file
|
||||||
|
|
||||||
|
[Filesystem]
|
||||||
|
|
||||||
|
autoMount = true # Automatically parses /etc/fstab and mounts disks
|
||||||
|
fstabPath = /etc/fstab # Path to your system's fstab (defaults to /etc/fstab)
|
||||||
|
createDirs = /path/to/dir1, /path/to/dir2 # Creates these directories on boot. Empty to disable
|
||||||
|
createSymlinks = /path/to/dir1, /path/to/dir2 # Creates these symlinks on boot. Empty to disable
|
||||||
|
|
||||||
|
[Misc]
|
||||||
|
|
||||||
|
controlSocket = /var/run/nimd.sock # Path to the Unix domain socket to create for IPC
|
||||||
|
```
|
|
@ -49,18 +49,59 @@ type
|
||||||
supervised: bool
|
supervised: bool
|
||||||
restart: RestartKind
|
restart: RestartKind
|
||||||
restartDelay: int
|
restartDelay: int
|
||||||
|
depends*: seq[Service]
|
||||||
|
provides*: seq[Service]
|
||||||
|
|
||||||
|
|
||||||
proc newService*(name, description: string, kind: ServiceKind, workDir: string, runlevel: RunLevel, exec: string, supervised: bool, restart: RestartKind, restartDelay: int): Service =
|
proc newService*(name, description: string, kind: ServiceKind, workDir: string, runlevel: RunLevel, exec: string, supervised: bool, restart: RestartKind,
|
||||||
|
restartDelay: int, depends, provides: seq[Service]): Service =
|
||||||
## Creates a new service object
|
## Creates a new service object
|
||||||
result = Service(name: name, description: description, kind: kind, workDir: workDir, runLevel: runLevel,
|
result = Service(name: name, description: description, kind: kind, workDir: workDir, runLevel: runLevel,
|
||||||
exec: exec, supervised: supervised, restart: restart, restartDelay: restartDelay)
|
exec: exec, supervised: supervised, restart: restart, restartDelay: restartDelay,
|
||||||
|
depends: depends, provides: provides)
|
||||||
|
result.provides.add(result)
|
||||||
|
|
||||||
|
|
||||||
|
proc extend[T](self: var seq[T], other: seq[T]) =
|
||||||
|
## Extends self with the elements of other
|
||||||
|
for el in other:
|
||||||
|
self.add(el)
|
||||||
|
|
||||||
|
|
||||||
var services: seq[Service] = @[]
|
var services: seq[Service] = @[]
|
||||||
var processIDs: TableRef[int, Service] = newTable[int, Service]()
|
var processIDs: TableRef[int, Service] = newTable[int, Service]()
|
||||||
|
|
||||||
|
|
||||||
|
proc resolveDependencies(logger: Logger, node: Service, resolved, unresolved: var seq[Service]) =
|
||||||
|
## Resolves dependencies and modifies the resolved
|
||||||
|
## parameter in place to a list that satisfies the
|
||||||
|
## dependency tree. This is basically traversing
|
||||||
|
## a directed cyclic graph, although note that cycles
|
||||||
|
## in our graph are errors and cause the dependants and
|
||||||
|
## the providers to be skipped and an error to be logged
|
||||||
|
|
||||||
|
# Note: It turns out this is an NP-hard problem (see https://stackoverflow.com/a/28102139/12159081),
|
||||||
|
# so hopefully this doesn't blow up. No wonder runit doesn't do any dependency resolution, lol.
|
||||||
|
# The algorithm comes from https://www.electricmonk.nl/log/2008/08/07/dependency-resolving-algorithm/
|
||||||
|
# and has been extended to support the dependent-provider paradigm
|
||||||
|
var ok = true
|
||||||
|
unresolved.add(node)
|
||||||
|
for dependency in node.depends:
|
||||||
|
if dependency notin resolved:
|
||||||
|
if dependency in unresolved:
|
||||||
|
logger.error(&"Could not resolve dependencies for '{node.name}' -> '{dependency.name}': cyclic dependency detected")
|
||||||
|
ok = false
|
||||||
|
continue
|
||||||
|
resolveDependencies(logger, dependency, resolved, unresolved)
|
||||||
|
for dependency in node.provides:
|
||||||
|
if dependency == node:
|
||||||
|
continue
|
||||||
|
resolveDependencies(logger, dependency, resolved, unresolved)
|
||||||
|
if ok:
|
||||||
|
resolved.add(node)
|
||||||
|
unresolved.del(unresolved.find(node))
|
||||||
|
|
||||||
|
|
||||||
proc isManagedProcess*(pid: int): bool =
|
proc isManagedProcess*(pid: int): bool =
|
||||||
## Returns true if the given process
|
## Returns true if the given process
|
||||||
## id is associated to a supervised
|
## id is associated to a supervised
|
||||||
|
@ -182,7 +223,7 @@ proc startService(logger: Logger, service: Service) =
|
||||||
let progName = arguments[0]
|
let progName = arguments[0]
|
||||||
arguments = arguments[1..^1]
|
arguments = arguments[1..^1]
|
||||||
process = startProcess(progName, workingDir=service.workDir, args=arguments)
|
process = startProcess(progName, workingDir=service.workDir, args=arguments)
|
||||||
if service.supervised:
|
if service.supervised and service.kind != Oneshot:
|
||||||
var pid = posix.fork()
|
var pid = posix.fork()
|
||||||
if pid == 0:
|
if pid == 0:
|
||||||
logger.trace(&"New child has been spawned")
|
logger.trace(&"New child has been spawned")
|
||||||
|
@ -198,6 +239,9 @@ proc startService(logger: Logger, service: Service) =
|
||||||
proc startServices*(logger: Logger, level: RunLevel, workers: int = 1) =
|
proc startServices*(logger: Logger, level: RunLevel, workers: int = 1) =
|
||||||
## Starts the registered services in the
|
## Starts the registered services in the
|
||||||
## given runlevel
|
## given runlevel
|
||||||
|
var resolved: seq[Service] = @[]
|
||||||
|
var unresolved: seq[Service] = @[]
|
||||||
|
resolveDependencies(logger, services[0], resolved, unresolved)
|
||||||
if workers > cpuinfo.countProcessors():
|
if workers > cpuinfo.countProcessors():
|
||||||
logger.warning(&"The configured number of workers ({workers}) is greater than the number of CPU cores ({cpuinfo.countProcessors()}), performance may degrade")
|
logger.warning(&"The configured number of workers ({workers}) is greater than the number of CPU cores ({cpuinfo.countProcessors()}), performance may degrade")
|
||||||
var workerCount: int = 0
|
var workerCount: int = 0
|
||||||
|
@ -223,8 +267,8 @@ proc startServices*(logger: Logger, level: RunLevel, workers: int = 1) =
|
||||||
logger.error(&"An error occurred while forking to spawn services, trying again: {posix.strerror(posix.errno)}")
|
logger.error(&"An error occurred while forking to spawn services, trying again: {posix.strerror(posix.errno)}")
|
||||||
elif pid == 0:
|
elif pid == 0:
|
||||||
logger.trace(&"New child has been spawned")
|
logger.trace(&"New child has been spawned")
|
||||||
if not servicesCopy[0].supervised:
|
if not servicesCopy[0].supervised or servicesCopy[0].kind == Oneshot:
|
||||||
logger.info(&"Starting unsupervised service '{servicesCopy[0].name}'")
|
logger.info(&"""Starting {(if servicesCopy[0].kind != Oneshot: "unsupervised" else: "oneshot")} service '{servicesCopy[0].name}'""")
|
||||||
else:
|
else:
|
||||||
logger.info(&"Starting supervised service '{servicesCopy[0].name}'")
|
logger.info(&"Starting supervised service '{servicesCopy[0].name}'")
|
||||||
startService(logger, servicesCopy[0])
|
startService(logger, servicesCopy[0])
|
||||||
|
|
40
src/main.nim
40
src/main.nim
|
@ -38,35 +38,33 @@ proc addStuff =
|
||||||
addSymlink(newSymlink(dest="/dev/std/err", source="/")) # Should say link already exists and points to /proc/self/fd/2
|
addSymlink(newSymlink(dest="/dev/std/err", source="/")) # Should say link already exists and points to /proc/self/fd/2
|
||||||
addSymlink(newSymlink(dest="/dev/std/in", source="/does/not/exist")) # Shuld say destination does not exist
|
addSymlink(newSymlink(dest="/dev/std/in", source="/does/not/exist")) # Shuld say destination does not exist
|
||||||
addSymlink(newSymlink(dest="/dev/std/in", source="/proc/self/fd/0")) # Should say link already exists
|
addSymlink(newSymlink(dest="/dev/std/in", source="/proc/self/fd/0")) # Should say link already exists
|
||||||
# Adds virtual filesystems (Update: apparently the kernel already mounts this stuff!)
|
|
||||||
#[
|
|
||||||
addFS(newFilesystem(source="proc", target="/proc", fstype="proc", mountflags=0u64, data="nosuid,noexec,nodev", dump=0u8, pass=0u8))
|
|
||||||
addFS(newFilesystem(source="sys", target="/sys", fstype="sysfs", mountflags=0u64, data="nosuid,noexec,nodev", dump=0u8, pass=0u8))
|
|
||||||
addFS(newFilesystem(source="run", target="/run", fstype="tmpfs", mountflags=0u64, data="mode=0755,nosuid,nodev", dump=0u8, pass=0u8))
|
|
||||||
addFS(newFilesystem(source="dev", target="/dev", fstype="devtmpfs", mountflags=0u64, data="mode=0755,nosuid", dump=0u8, pass=0u8))
|
|
||||||
addFS(newFilesystem(source="devpts", target="/dev/pts", fstype="devpts", mountflags=0u64, data="mode=0620,gid=5,nosuid,noexec", dump=0u8, pass=0u8))
|
|
||||||
addFS(newFilesystem(source="shm", target="/dev/shm", fstype="tmpfs", mountflags=0u64, data="mode=1777,nosuid,nodev", dump=0u8, pass=0u8))
|
|
||||||
]#
|
|
||||||
addDirectory(newDirectory("test", 777)) # Should create a directory
|
addDirectory(newDirectory("test", 777)) # Should create a directory
|
||||||
addDirectory(newDirectory("/dev/disk", 123)) # Should say directory already exists
|
addDirectory(newDirectory("/dev/disk", 123)) # Should say directory already exists
|
||||||
addDirectory(newDirectory("/dev/test/owo", 000)) # Should say path does not exist
|
addDirectory(newDirectory("/dev/test/owo", 000)) # Should say path does not exist
|
||||||
# Shutdown handler to unmount disks
|
# Shutdown handler to unmount disks
|
||||||
addShutdownHandler(newShutdownHandler(unmountAllDisks))
|
addShutdownHandler(newShutdownHandler(unmountAllDisks))
|
||||||
# Adds test services
|
# Adds test services
|
||||||
addService(newService(name="echoer", description="prints owo", exec="/bin/echo owo",
|
var echoer = newService(name="echoer", description="prints owo", exec="/bin/echo owo",
|
||||||
runlevel=Boot, kind=Oneshot, workDir=getCurrentDir(),
|
runlevel=Boot, kind=Oneshot, workDir=getCurrentDir(),
|
||||||
supervised=false, restart=Never, restartDelay=0))
|
supervised=false, restart=Never, restartDelay=0,
|
||||||
addService(newService(name="errorer", description="la mamma di gavd",
|
depends=(@[]), provides=(@[]))
|
||||||
|
var errorer = newService(name="errorer", description="la mamma di gavd",
|
||||||
exec="/bin/false", supervised=true, restart=OnFailure,
|
exec="/bin/false", supervised=true, restart=OnFailure,
|
||||||
restartDelay=5, runlevel=Boot, workDir="/", kind=Simple))
|
restartDelay=5, runlevel=Boot, workDir="/", kind=Simple,
|
||||||
addService(newService(name="exiter", description="la mamma di licenziat",
|
depends=(@[echoer]), provides=(@[]))
|
||||||
|
var test = newService(name="broken", description="", exec="/bin/echo owo",
|
||||||
|
runlevel=Boot, kind=Oneshot, workDir=getCurrentDir(),
|
||||||
|
supervised=false, restart=Never, restartDelay=0,
|
||||||
|
depends=(@[echoer]), provides=(@[]))
|
||||||
|
var exiter = newService(name="exiter", description="la mamma di licenziat",
|
||||||
exec="/bin/true", supervised=true, restart=Always,
|
exec="/bin/true", supervised=true, restart=Always,
|
||||||
restartDelay=5, runlevel=Boot, workDir="/", kind=Simple))
|
restartDelay=5, runlevel=Boot, workDir="/", kind=Simple,
|
||||||
#[
|
depends=(@[errorer]), provides=(@[]))
|
||||||
addService(newService(name="sleeper", description="la mamma di danieloz",
|
addService(errorer)
|
||||||
exec="/usr/bin/sleep", supervised=true, restart=Always,
|
addService(echoer)
|
||||||
restartDelay=5, runlevel=Boot, workDir="/", kind=Simple))
|
addService(exiter)
|
||||||
]#
|
addService(test)
|
||||||
|
echoer.depends.add(test)
|
||||||
|
|
||||||
|
|
||||||
proc main(logger: Logger, mountDisks: bool = true, fstab: string = "/etc/fstab") =
|
proc main(logger: Logger, mountDisks: bool = true, fstab: string = "/etc/fstab") =
|
||||||
|
|
Loading…
Reference in New Issue