From /etc/init to systemd

A journey through the years ofA journey through the years of

UNIX andand Linuxservice managementservice management

Lubomir Rintel <lkundrak@v3.sk>BTC: 15wvWxN5QMpreKR37pYb7VBu8xLu4TiNR2

Sixth Edition UNIX (1976)● /etc/init (219 lines)

● Single user shell● gettys from /etc/ttys● utmp & wtmp

● /etc/rc (2 lines)

rm -f /etc/mtab/etc/update rm -f /etc/mtab/etc/update

UNIX System III (1980)● /etc/init (486 lines)

● /etc/inittab "states" controlling gettys on terminals

● /etc/rc (67 lines)● State aware, single user mode● acct● errdemon● cron

● "edit to add umounts"

2.9 BSD UNIX (1983)● /sbin/init (593 lines)

● Single user shell● gettys from /etc/ttys

● /etc/rc (53 lines)● fsck, quotacheck● mount -a, from /etc/fstab● savecore, ex/vi recovery● clear /tmp, locks in /usr/spool● update, cron, acct● hostname

UNIX System V Release 4 (1983)● /etc/init

● /etc/inittab with runlevels, respawns● /etc/rc?

● Modular init system, /etc/rc.d● Per-daemon init scripts (8-56 lines, avg. 26)

– start & stop arguments

– pid from ps, stop = kill -TERM

● Enablement/disablement by linking into level dir

● Ordered by numbers

#ident "@(#)/etc/init.d/cron.sl 1.1 4.0 10/15/90 8479 AT&T-SF"

# cron controlpid=`/usr/bin/ps -e | /usr/bin/grep cron |

/usr/bin/sed -e 's/^ *//' -e 's/ .*//'`case $1 in'start') if [ "${pid}" = "" ] then /usr/bin/rm -f /etc/cron.d/FIFO if [ -x /usr/bin/cron ] then /usr/bin/cron elif [ -x /usr/sbin/cron ] then /usr/sbin/cron fi fi ;;'stop') if [ "${pid}" != "" ] then /usr/bin/kill ${pid} fi ;;*) echo "usage: /etc/init.d/cron {start|stop}" ;;esac

2.11 BSD UNIX (1986)● /sbin/init (782 lines)

● Single user● gettys from /etc/gettytab

● /etc/rc (165 lines)● Everything 2.9BSD had● Network, inetd, routed, named, lpd, rwhod

● /etc/rc.local

● Editable for starting local daemons

Red Hat Enterprise Linux 5 (2007)● SVR4-like init

● Some BSD elements: /etc/rc, /etc/rc.local● Init scripts LSB compliant

● 61-584 lines, avg. 128● start, stop, status, restart, condrestart● Pidfiles in /var/run● Subsystem locks in /var/lock/subsys● /etc/rc.d/functions library● /etc/sysconfig init script configuration

if [ -x /usr/bin/make -a -f /etc/mail/Makefile ]; then make all -C /etc/mail -s > /dev/nullelse for i in virtusertable access domaintable mailertable ; do if [ -f /etc/mail/$i ] ; then

makemap hash /etc/mail/$i < /etc/mail/$i fi donefidaemon /usr/sbin/sendmail $([ "x$DAEMON" = xyes ] && echo -bd) \ $([ -n "$QUEUE" ] && echo -q$QUEUE)RETVAL=$?killproc sendmail -HUPRETVAL=$?echoif [ $RETVAL -eq 0 -a -f /var/run/sm-client.pid ]; then

echo -n $"reloading sm-client: "killproc sm-client -HUPRETVAL=$?echo

fireturn $RETVAL

}

stop() {# Stop daemons.if test -f /var/run/sm-client.pid ; then

echo -n $"Shutting down sm-client: "killproc sm-clientRETVAL=$?echo[ $RETVAL -eq 0 ] && rm -f /var/run/sm-client.pid[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/sm-client

fiecho -n $"Shutting down $prog: "killproc sendmailRETVAL=$?echo[ $RETVAL -eq 0 ] && rm -f /var/lock/subsys/sendmailreturn $RETVAL

}

# See how we were called.case "$1" in start)

start;;

stop)stop;;

reload)reloadRETVAL=$?;;

restart)stopstartRETVAL=$?;;

condrestart)if [ -f /var/lock/subsys/sendmail ]; then stop start RETVAL=$?fi;;

status)status sendmailRETVAL=$?;;

*)echo $"Usage: $0 {start|stop|restart|condrestart|status}"exit 1

esac

exit $RETVAL

#!/bin/bash## sendmail This shell script takes care of starting and stopping# sendmail.## chkconfig: 2345 80 30# description: Sendmail is a Mail Transport Agent, which is the program \# that moves mail from one machine to another.# processname: sendmail# config: /etc/mail/sendmail.cf# pidfile: /var/run/sendmail.pid

# Source function library.. /etc/rc.d/init.d/functions

# Source networking configuration.[ -f /etc/sysconfig/network ] && . /etc/sysconfig/network

# Source sendmail configureation.if [ -f /etc/sysconfig/sendmail ] ; then

. /etc/sysconfig/sendmailelse

DAEMON=noQUEUE=1h

fi[ -z "$SMQUEUE" ] && SMQUEUE="$QUEUE"[ -z "$SMQUEUE" ] && SMQUEUE=1h

# Check that networking is up.[ "${NETWORKING}" = "no" ] && exit 0

[ -f /usr/sbin/sendmail ] || exit 0

RETVAL=0prog="sendmail"

start() {# Start daemons.

echo -n $"Starting $prog: "if test -x /usr/bin/make -a -f /etc/mail/Makefile ; then make all -C /etc/mail -s > /dev/nullelse for i in virtusertable access domaintable mailertable ; do if [ -f /etc/mail/$i ] ; then

makemap hash /etc/mail/$i < /etc/mail/$i fi donefi/usr/bin/newaliases > /dev/null 2>&1daemon /usr/sbin/sendmail $([ "x$DAEMON" = xyes ] && echo -bd) \

$([ -n "$QUEUE" ] && echo -q$QUEUE) $SENDMAIL_OPTARGRETVAL=$?echo[ $RETVAL -eq 0 ] && touch /var/lock/subsys/sendmail

if ! test -f /var/run/sm-client.pid ; thenecho -n $"Starting sm-client: "touch /var/run/sm-client.pidchown smmsp:smmsp /var/run/sm-client.pidif [ -x /usr/sbin/selinuxenabled ] && /usr/sbin/selinuxenabled; then /sbin/restorecon /var/run/sm-client.pidfidaemon --check sm-client /usr/sbin/sendmail -L sm-msp-queue -Ac \

-q$SMQUEUE $SENDMAIL_OPTARGRETVAL=$?

echo [ $RETVAL -eq 0 ] && touch /var/lock/subsys/sm-client fi

return $RETVAL

reload() {# Stop daemons.}

echo -n $"reloading $prog: "/usr/bin/newaliases > /dev/null 2>&1

far∙pot∙shket (Yiddish):

pronoun

1. completely ruined as a result

of attempts to fix a minor imperfection

··

Issues● inittab service control inflexible

● Not actually used very much● No way to monitor services● Error output is lost● Inflexible startup ordering● Single-threaded startup

● Slow● Malfunctional startup script can lock up boot

● Limited to single instance of a service

Issues (cont.)● No way to track processes belonging to a service

● They double-fork to daemonize● No way to reliably terminate a service● No way to tell whether the service is operational● Resource hogging

● Inconsistent● Who drops privileges?● Who writes PID files?● Who chroots?

SVR4 & LSB init scripts● Unbelievably shitty

● Unreliably work around issues mentioned● Very long and ugly● Lots of code duplication● Races, improper

subsystem locks● Insecure PID

determination

How did UNIX address this● Sun Solaris 10: SMF

● Multiple service instances

● Apple Darwin: launchd● System and User sessions● On demand startup of network daemons (inetd)● Job scheduler

● Common:● Parallel startup● Dependency management, service ordering● Monitoring/restarts of services

● Various others: upstart, daemontools, Monit, ...

Linux: systemd● Heavily inspired by concepts from launchd● Already used in most modern Linux distributions● Solves all of the mentioned issues!● Leverages Linux inovations -- a lot more powerful

● Control groups, Namespaces● Seccomp, Capabilities, SELinux● Auditing● Automounter● DBus API● Structured kernel messaging

systemd Unit typesUnit type Description/equivalent

service A daemon (SVR4 init script)

socket A network or UNIX socket (inetd)

device UDev device instance

mount Mount point (fstab)

target Runlevel

swap Swap space (fstab)

automount Autofs

path Inotify watch

timer Crond

snapshot Dynamically created target

httpd.service├─system.slice└─basic.target ├─fedora-loadmodules.service ├─paths.target ├─sockets.target │ ├─cups.socket │ ├─rpcbind.socket │ ├─systemd-initctl.socket │ └─systemd-udevd-kernel.socket ├─sysinit.target │ ├─kmod-static-nodes.service │ ├─systemd-udev-trigger.service │ ├─local-fs.target │ │ ├─-.mount │ │ ├─fedora-import-state.service │ │ ├─home.mount │ │ ├─systemd-fsck-root.service │ │ └─tmp.mount │ └─swap.target │ └─dev-disk-by\x2dlabel-yolo.swap └─timers.target └─systemd-tmpfiles-clean.timer

● Defined from unit files● /lib/systemd● /etc/systemd

● Generated automatically● Compatibility or dynamic changes● device unit appears as device appears in

udev● mount units generated from /etc/fstab

systemd Units

Service units● Service runs in its own control group● Isolated from the rest of system to some extent● A process can't escape● Freezer control group assures reliable shutdown● Service should not double-fork (launchd)● systemd-journald takes care of logging● Can depend on socket units for activation (inetd)● DBus activation also possible

User sessions● Manages processes for a user session (e.g. tty

or GNOME desktop)● Session runs in separate control group● systemd-logind replaces ConsoleKit● Multiseat● Reliable log-off

Essential tools

systemctl --allsystemctl stop sshd.servicesystemctl status sshdsystemctl disable sshdsystemd-cgtopsystemd-cglsjournalctl -fsystemd-analyze blamesystemd-analyze critical-chain

What else● Takes care of system-wide events

● Pinging watchdog● Laptop lid close● Shutdown, kexec

● Documentation● Well written manual pages for everything

● Lightweight virtualization (LXC)● clone()s all namespaces

Found this useful? My Bitcoin address: 15wvWxN5QMpreKR37pYb7VBu8xLu4TiNR2

Thanks for listening!