Back in 2014 I suggested a solution using make, in order to address
dependencies. I didn't closely read your scripts or the make file, but
the only way I can imagine this new system would speed boot is if it
ran things in the background and found ways to see when dependencies
finished. Otherwise, a make-based solution would be no slower or faster
than a parallel-start process manager.
What I like about this system is that it makes a complete mockery of
the original purpose of systemd. You did that with a couple
shellscripts and a makefile, to compete with Redhat and their
megadollar per year programmers' salary.
Be aware, however, that Redhat's original poetterclaim was that the
only choices were systemd, sysvinit and upstart. False then, false now.
There are several other great init systems.
Your system appears to keep the 50 to 300 line sysvinit init scripts,
which to me are the real problem with sysvinit. And I think it was
those monster scripts that the "debian devs" and the "upstreams" hated
so much. Right now, both s6 and runit execute in parallel, with
dependencies, using run scripts (equivalent of an init script) that are
between 2 and 15 lines.
One other thing: Parallel instantiation is significantly faster only if
one of the daemons takes a long time to return. In my experience, this
is usually caused by a misconfiguration of the slow daemon, often a
malfunctioning reverse DNS. If sysvinit had had a decent reporting
system on time taken per daemon, and the daemon's return code, the
admin would have fixed the daemon and sped up boot. Parallel
instantiation just sweeps the misconfiguration problem under the rug.
Back during the init wars, a guy brought out a very nice init system
called Epoch, which was serial instantiation but in fact came up faster
than runit. Systemd came up faster than Epoch, but in the wild, without
special tweaks, I've heard systemd comes up slowly.
I like the fact that you've made software that makes systemd look
silly, but please keep in mind that there are other init alternatives,
or sysvinit + process supervisor configurations, that solve all the
objections to sysvinit: The supposedly slow bootup, the hundred line
init scripts backed by hundreds of lines of shellscript helper files,
the absolute requirement that a daemon background itself, the commented
out numbers that actually are read and mean something (eeeeeuuuu,
meaningful comments), and all the rest. Here's how I rate init systems,
based on a 0 to 10 rating, where 10 is perfect and 0 is useless:
s6: 10
runit: 10
sysvinit plus s6, runit or daemontools-encore process supervisor: 10
OpenRC: 6
Epoch: 5
Sysvinit both PID1 and process manager: 2
Systemd: -15
NOTE: A negative number denotes "worse than useless".
SteveT
Steve Litt
Spring 2021 featured book: Troubleshooting Techniques of the Successful
Technologist
http://www.troubleshooters.com/techniques
Alejandro Colomar \(man-pages\) via Dng said on Wed, 5 Jan 2022
03:03:53 +0100
>Hi all,
>
>Most of you I added you to this email because I found you on the
>maintainers list for the Debian sysv-rc package (now dead for a long
>time). I also CCd Devuan, since I hope you'll be interested in this
>little project of mine.
>I also CCd linux-man@, since there's not many people listening there,
>and not much traffic these days so I hope you won't be angry for this
>little bit of spam; and I hope some good guys reading that list may
>have some comments on this. Sorry again for the bit of spam.
>I also CCd help-make@. I (ab)used your software in a way that it was
>never designed to; not sorry for that; I'll say it was the original
>Unix authors' fault for designing such (ab)usable tools :). Maybe
>you're interested in this thread, maybe you don't care, but I'll CC
>you in case some of you may be interested in this.
>And finally, Randy, as you asked, I'll CC you for news on this.
>Anyone not interested in follow-ups, please email me, and I'll try to
>remove from CC.
>
>So, last friday (yes, that's New Year's Eve), I was reading something,
>and got this idea... the main valid claim for systemd is that it blows
>away competition in terms of performance? Full parallelization?
>Knows about dependencies? I don't know too much of systems; I know
>some basic stuff, but I'm mostly a C programmer, so I don't know
>init(1)... okay. But since I program, I sure know the good ol'
>make(1). It's old, and it's good at fully parallelizing _everything_.
> Dependencies? sure; fast? sure; parallel? sure; simple? sure; a bit
> bloated? GNU make maybe,
>especially for init(1), but far from systemd(1):
>
>$ ls -lh $(realpath $(which systemd make bash sh 2>/dev/null))
>-rwxr-xr-x 1 root root 1.2M Dec 15 00:43 /usr/bin/bash
>-rwxr-xr-x 1 root root 123K Nov 3 11:51 /usr/bin/dash
>-rwxr-xr-x 1 root root 235K Apr 10 2021 /usr/bin/make
>-rwxr-xr-x 1 root root 1.8M Nov 19 21:11 /usr/lib/systemd/systemd
>
>So, if the problem is that the rc scripts don't run parallel and don't
>know about exact dependencies from each-other, let's rewrite that part
>and only that part of the system with make(1); that was the idea.
>Don't touch init, don't touch the scripts themselves... only the part
>that decides on which order to run them. That's the idea.
>
>
>I decided to start with a clean install of Devuan (since I'm a Debian
>user, and I'm used to it; it might have been easier to start with a
>smaller system, but I might have found other issues in the middle; I
>know that the only difference in Devuan is sysvinit, which is what I
>want). The install has XFCE and sysvinit as options. I wanted the
>system to have GUI, so that if I can boot XFCE with it, I can boot
>anything (except maybe for the bad boy, GNOME, but I don't even like
>it).
>
>I wrote a couple of scripts to port the existing rc system to my
>make-based rc system. The script didn't touch the old one, in case I
>needed it to boot if I screwed something, of course. One script
>creates the makefiles without declaring the dependencies between them,
>and then another makefile creates the dependencies to match the
>current boot order (allowing for the same level of parallelization
>that the current semi-parallel rc script uses). From there, one could
>analyze the dependencies to remove some that are incorrect, and make
>it faster, but since I don't know the real dependencies, I didn't want
>to break stuff.
>
>So, here go the scripts:
>
>====================== script1_basicmk.sh ========================
>#!/bin/bash
>
>find /etc/rc[0123456S].d/[KS]* \
>|sed s,/etc/rc..d/...,, \
>|sort \
>|uniq \
>|while read x; do\
> mk="/etc/rc.mk.d/$x.mk";
>
> test -e $mk \
> && continue;
>
> >$mk cat <<EOF
>.PHONY: K$x S$x
>
>: K$x
>
>: S$x
>
>K$x:
> /etc/init.d/$x stop
>
>S$x:
> /etc/init.d/$x start
>EOF
>
> find /etc/rc[0123456S].d/K*$x 2>/dev/null \
> |sed s,/etc/rc,, \
> |sed s,.d/K.*,, \
> |sort \
> |tac \
> |while read n; do \
> sed -i "/: K$x$/s/^/$n /" $mk;
> done;
>
> find /etc/rc[0123456S].d/S*$x 2>/dev/null \
> |sed s,/etc/rc,, \
> |sed s,.d/S.*,, \
> |sort \
> |tac \
> |while read n; do \
> sed -i "/: S$x$/s/^/$n /" $mk;
> done;
>
> sed -i "s/ *:/:/" $mk;
> sed -i "/^:/d" $mk;
>done;
>=========================================================
>
>===================== script2_deps.sh ===================
>#!/bin/bash
>
>find /etc/rc.mk.d/*.mk \
>|sed s,/etc/rc.mk.d/,, \
>|sed s,.mk$,, \
>|sort \
>|while read x; do\
> mk="/etc/rc.mk.d/$x.mk";
>
> for n in 0 1 2 3 4 5 6 S; do \
> K=$(find /etc/rc$n.d/K??$x 2>/dev/null \
> |grep -o /K.. \
> |sed s,/K0*,,);
> test -z "$K" || test $K -eq 1 \
> && continue;
> k=$(printf %02d $(($K - 1)));
>
> find /etc/rc$n.d/K$k* \
> |sed s,/etc/rc$n.d/K$k,,;
> done \
> |sort \
> |uniq \
> |while read dep; do \
> echo K$dep;
> done \
> |xargs echo \
> |while read Kdeps; do
> test -z "$Kdeps" \
> && break;
>
> >>$mk cat <<-EOF
>
> K$x: $Kdeps
> EOF
> done;
>
> for n in 0 1 2 3 4 5 6 S; do \
> S=$(find /etc/rc$n.d/S??$x 2>/dev/null \
> |grep -o /S.. \
> |sed s,/S0*,,);
> test -z "$S" || test $S -eq 1 \
> && continue;
> s=$(printf %02d $(($S - 1)));
>
> find /etc/rc$n.d/S$s* \
> |sed s,/etc/rc$n.d/S$s,,;
> done \
> |sort \
> |uniq \
> |while read dep; do \
> echo S$dep;
> done \
> |xargs echo \
> |while read Sdeps; do
> test -z "$Sdeps" \
> && break;
>
> >>$mk cat <<-EOF
>
> S$x: $Sdeps
> EOF
> done;
>
> for n in 0 1 2 3 4 5 6 S; do \
> test -e /etc/rc$n.d/S01$x \
> || continue;
> K=$(find /etc/rc$n.d/K* 2>/dev/null \
> |grep -o /K.. \
> |sed s,/K,, \
> |sort \
> |uniq \
> |tac \
> |head -n1)
> test -z "$K" \
> && continue;
> k=$(printf %02d $K);
>
> find /etc/rc$n.d/K$k* \
> |sed s,/etc/rc$n.d/K$k,,;
> done \
> |sort \
> |uniq \
> |while read dep; do \
> echo K$dep;
> done \
> |xargs echo \
> |while read Kdeps; do
> test -z "$Kdeps" \
> && break;
>
> >>$mk cat <<-EOF
>
> S$x: $Kdeps
> EOF
> done;
>done;
>===============================================
>
>
>I know some stuff could go into functions to make them shorter, and
>probably some comments could also help, but I wrote them fast, just to
>have some proof of concept, and later I could makes them nicer.
>Special thanks to Doug for creating pipes, I love them :)
>
>The One Makefile To Rule Them All is the following:
>
>================ /etc/rc.mk.d/Makefile ================
>MAKEFLAGS += --no-builtin-rules
>MAKEFLAGS += --no-builtin-variables
>
>.PHONY: all 0 1 2 3 4 5 6 S
>
>all: 5
>
>0 1 2 3 4 5 6 S:
>
>include /etc/rc.mk.d/*.mk
>=======================================================
>
>And then a sample file (created by the above scripts) is:
>
>================ /etc/rc.mk.d/vboxadd.mk ===============
>.PHONY: Kvboxadd Svboxadd
>
>0 1 6: Kvboxadd
>
>2 3 4 5: Svboxadd
>
>Kvboxadd:
> /etc/init.d/vboxadd stop
> sleep 1
>
>Svboxadd:
> /etc/init.d/vboxadd start
> sleep 1
>
>Kvboxadd: Kalsa-utils Kbrightness Kelogind Knetwork-manager
>Kpulseaudio-enable-autospawn Ksaned Kslim Kspeech-dispatcher Kurandom
>Kvboxadd-service
>
>Svboxadd: Sconsole-setup.sh
>========================================================
>
>I added the sleep 1 for debug; the rest is the output of the scripts.
>
>
>As you can see I kept the K... and S... syntax of rc scripts.
>However, instead of a symlink, now there's a makefile declaring
>dependencies and calls directly the script with no symlink. Now all
>makefiles are in a single directory (I found it simpler this way), but
>we could think of a different structure if it's better. These
>makefiles (or more technically, parts of a makefile), resemble the
>systemd service files, where each one declares what should have
>already run before start and before kill, and also contains the info
>about which runlevels want this (both S and K).
>
>Because of the script, and because some scripts used .sh extension and
>some others didn't, there are some traces of that in the names of the
>targets; nothing really problematic, only ugly. I could have removed
>it, but it was more work, and I noticed late.
>
>I only had to manipulate a single line from only one of these
>makefiles after running the scripts, but that I'll keep if for later;
>it was about dependencies, because there was a missing number in one
>of the runlevel dirs (no jump from S02 to S04 in one of the dirs, no
>S03)
>
>
>And finally, to run all this madness, I had to tweak /lib/init/rc a
>little bit:
>
>diff --git a/rc b/rc
>index 5e7f3a4..e1ffa31 100755
>--- a/rc
>+++ b/rc
>@@ -60,7 +60,7 @@ export VERBOSE
> # insserv package to be enabled. Boot concurrency also requires
> # startpar to be installed.
> #
>-CONCURRENCY=makefile
>+CONCURRENCY=makefile2
> test -s /etc/init.d/.depend.boot || CONCURRENCY="none"
> test -s /etc/init.d/.depend.start || CONCURRENCY="none"
> test -s /etc/init.d/.depend.stop || CONCURRENCY="none"
>@@ -168,7 +168,10 @@ then
> done
> fi
>
>- if [ makefile = "$CONCURRENCY" ]
>+ if [ makefile2 = "$CONCURRENCY" ]
>+ then
>+ make -C "/etc/rc.mk.d/" "$runlevel"
>+ elif [ makefile = "$CONCURRENCY" ]
> then
> if [ S = "$runlevel" ]
> then
>
>
>
>I'd like to know your thoughts about this. If it seems like it could
>be a good replacement for sysv-rc (for Devuan; I guess for Debian it's
>too late) I'd like to know. Or if you think it can be a good thing
>but needs some polish, I'd like to hear it too. It's just a proof of
>concept after 2 days of having had fun with it :)
>
>I'd like it to be able to compete in terms of performance with
>systemd(1), so that we could possibly take over it, but hey, that's
>only a hope.
>
>And now I'll tell you a bit about the outcome I had: I booted with
>this configuration, and I first got into single-user mode, which I
>don't yet understand why it happened (there's probably something I
>didn't get right, but no real blocking problem, I guess). But then,
>after the typical type password or ^D to continue, I pressed ^D, and I
>got a shiny XFCE GUI completely wirking. Not bad for this prototype,
>I'd say. I'd like some help to polish this, if you may.
>
>Happy new year!
>
>Alex
>
>
>--
>Alejandro Colomar
>Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
>http://www.alejandro-colomar.es/
>_______________________________________________
>Dng mailing list
>Dng@???
>https://mailinglists.dyne.org/cgi-bin/mailman/listinfo/dng
::
Re: [DNG] make-rc: A parallel (as i…
