Introducing Daemon Manager

The idea for Daemon Manager came about when I was converting a web site from Apache to Nginx. Nginx doesn’t launch FastCGI programs itself—it only connects to FastCGI sockets and so it requires that you manage the FastCGI server yourself.

For a simple web site it might be OK to manually create an /etc/init.d script, but even for our relatively simple solitaire site we ended having about 5 separate FastCGI servers (between the blog, the forum, and our test servers). At home I host virtual servers for various members of my family and so there’s a ton of accounts and blogs and forums and other random stuff. I just can’t abide copy and pasting 20 /etc/init.d scripts and then managing them all as they slowly fork away from each other over time. Not to mention that ordinary users can’t manage /etc/init.d scripts themselves (without compromising system security) and if the script does any sort of setuid() calls then they can’t even restart their FastCGI servers without there being some sort of arcane sudo configuration.

I also ran into a problem with PHP. I wanted to run WordPress on Nginx which meant I had to run the “php-cgi” FastCGI server. The problem is that “php-cgi” seems to just die randomly which means I needed some sort of watchdog that starts it back up when it fails.

What I really wanted is a program that lets non-privileged users launch and respawn their own daemons securely and very simply.

Daemon Manager is the result.

Design

The main principles of Daemon Manager’s design were:

1. It must be possible for a user to set up and configure their daemons—no root access must be required for a user to create a new daemon or restart one of their daemons.
2. It must be secure—users should not be allowed to control other users’ daemons (unless they are given explicit permission).
3. It should allow for good security practices—users should be allowed to launch a daemon as a user other than themselves if root has explicitly allowed it. This is so you can run your FastCGI server as a “nobody” style user.¹
4. It should restart the daemons if they crash (I’m looking at you, php-cgi).
5. It should be easy to use—1 config file per daemon and a simple command line interface to interact with the running daemons.

There are programs out there that do parts of that list, but none that do everything:

• daemon tools: I’ve used it before and I really like its philosophy of being small and simple. But it seems to really want to run as root which means you have to be root to control it. Also, setting up new daemons is kind of a pain.
• Upstart: It’s very similar and it makes setting up a new daemon pretty easy but since it’s an “init” replacement it doesn’t seems very adept at running programs meant for non-root users. I’ve done it before but there was a lot of “sudo” configuration and it wasn’t easy. Also the config files are stored in /etc/init and only root can write new ones.
• Systemd: I really love systemd. Or the idea of it, really—some day it will make it into Debian and I’ll start actually using it. But its philosophy is great. But again, being an “init” replacement gives it most of the same downsides as upstart.

Implementation

From those ideas Daemon Manager was born. I prototyped it in Perl in about 8 hours. The idea seemed sound but I was unhappy with the memory requirements of Perl. I wanted it to be something really small and lean, without any external dependencies. So I rewrote it in C++. It now takes up hardly any RAM which makes it suitable for smaller or embedded environments. The main loop just poll()s on the control sockets and calls wait() when necessary, occasionally restarting a crashed daemon.

I tried to think about security because if it’s insecure then nobody is going to want to use it seriously. Daemon Manager will refuse to read config files that don’t have the right permissions. It uses Unix domain sockets to communicate with users (1 socket for each user in ~/.daemon-manager/command.sock) and makes sure that each socket has the correct permissions when it starts—this way user authentication is handled by the operating system’s filesystem permissions layer. By default users of the system are not allowed to run anything—root must authorize each user and specify which users their daemons can be run as.

A Quick Tutorial Through Examples

Master config: /etc/daemon-manager.conf

[runs_as]
david: www-data
michaelc: www-data
amy: www-data
joann: www-data
jim:
greenfelt: www-data

[manages]
david: michaelc,amy,joann,greenfelt
michaelc: amy, joann
bill: joann
jim: greenfelt

The main section is the “runs_as” section. This section tells Daemon Manager which users are allowed to start daemons and which users the daemons can be run as. In the above example, “david”, “michaelc”, “amy”, “joann”, and “greenfelt” can launch daemons as themselves and also “www-data”. “jim” is only allowed to launch daemons as himself. No other users on the system are allowed to launch daemons at all because they weren’t listed.

The “manages” section is a little experimental at this point, but the idea is that “david” is allowed to manage (start, stop, or restart) the daemons of “michaelc”, “amy”, “joann”, and “greenfelt” in addition to his own daemons. This is so you can have help desk type users who can stop or restart other users’ daemons even though they may not have read or write access to the users’ home directories.

Daemon: deluged.conf

dir=/home/david
start=exec deluged -d

This is a simple Daemon Manager config file that launches the deluge bittorrent client daemon. “dir” and “start” are the only required entries in the file. “dir” is the working directory and “start” is a one line shell script to run. Because it is a shell script it needs to “exec”, otherwise Daemon Manager can’t manage it properly.²

Daemon: wordpress.conf

dir=/home/user/wordpress
user=www-data
start=exec php-cgi -q -b wordpress.socket

This starts a PHP FastCGI daemon in a WordPress directory and starts it running with as the “www-data” user.

Daemon: greenfelt.conf

dir=/var/www/greenfelt.net
user=www-data
start=exec ./script/greenfelt_fastcgi.pl -l greenfelt.socket -n 1
output=log

This is a paraphrase of the config file that runs greenfelt.net which is a Catalyst app. The “output” parameter tells Daemon Manager to collect the stdout and stderr of the daemon and save it to a log file in the “~/.daemon-manager/logs/” directory.

Controlling Daemon Manager

Daemon Manager is controlled using the “dmctl” command. It is relatively simple at this point, allowing you to start, stop and restart daemons. It also lets you scan for new conf files and query for daemon statistics. Here’s an example output the “status” command:

$ dmctl status
daemon-id                      state              pid respawns cooldown   uptime    total
david/deluge-web               running           2948        0       0s     3w3d     3w3d
david/deluged                  running           2950        0       0s     3w3d     3w3d
david/greenfelt                stopped              0        0       0s       0s       0s
david/minecraft                stopped              0        0       0s       0s       0s
david/moviefile                running           2951        0       0s     3w3d     3w3d
david/pytivo                   running          22905        0       0s     4d7h     4d7h
david/streamium                running           2958        0       0s     3w3d     3w3d
david/wordpress                running          27012       33       0s   12h57m     3w3d

Notice that the wordpress server (good old php-cgi) has crashed 33 times (and has been automatically respawned by Daemon Manager). Also notice that despite FastCGI being the impetus for creating Daemon Manager, most of my running daemons are not actually FastCGI servers.

Download and Use

Daemon Manager is licensed under the GPL and can be downloaded here (the source is also available on github). If you use Debian there’s a Debian branch available for building a package. The version as of this writing is 0.9 which means that there are some obvious things that need to be fixed³ and that I’m not sure it’s 100% secure or bug free yet, though I have boldly started using it on a couple sites and so far it’s been working great.