Hi Isaac,

[Snip]

>
> Theoretically, it *should* work if /etc/{passwd,group} are present in the
> initramfs, with those paths. It's possible to mistakenly copy them to /
> instead of /etc, but I assume that you've already checked that.
>
> Detail that shouldn't make a difference but might:
> static or non-static busybox?
>

Not sure; will find out and let you know if it makes a difference.


>
> A couple shots in the dark:
> -check that you're using properly spelled usernames/groups, and that
> $IFS isn't something weird.
>

Did the former, but not yet the latter.


> -double-check that you have a non-empty passwd/groups file, containing
> the same users/groups and world-readable.
>

Does it have to be world-readable even if the chown(2) happens from a
process with uid=0? I verified that they're non-empty, and verified that I
was unable to (as root) use busybox chown from the initramfs shell.


>
> To debug it:
> -Add ltrace (with dependencies) and a *non-static* busybox to your
> initramfs and try running
> ltrace busybox chown USER:GROUP FILE
>
> This shows you the functions that are called, so you can check if they
> return success.
>

I will add ltrace (and ldd) to my initramfs and try this out :)


>
> > * As Anto's experience shows, getting a working initramfs is tedious to
> do
> > and easy to get wrong. This is not only because there are multiple init
> > systems that vdevd will need to work with (e.g. sysv-rc, file-rc, openrc,
> > etc.), but also init scripts and initramfs scripts that come from
> packages
> > that depend on udev are (unsurprisingly) tightly coupled to udev. This
> > makes it difficult to install vdevd on a running Debian system without
> > removing udev (which is in itself undesirable, since it will take a large
> > swath of packages out with it), since I have to pretty much fork both
> sets
> > of scripts and try to mash them up into a custom initramfs without
> altering
> > /usr/share/initramfs-tools. This is effectively what my (extremely
> kludgy)
> > initramfs Makefile does.
>
> Check what I did with mdev (in hooks/ directory of github.com/idunham/mdev
> );
> I did *not* find that initramfs-tools was at all tightly coupled with udev,
> to the point that dropping the correct scripts into
> /usr/share/initramfs-tools and convincing dpkg that it didn't need udev
> was quite adequate.


Thanks for the link! It's good to get feedback from someone who's packaged
a device manager before :)

Regarding setting up the initramfs, my experience is a bit more involved.
I had to patch initramfs-tool's init script so it wouldn't try to mount
devtmpfs, and wouldn't try to pull in udev config files or migrate udev's
info from /run. This is because to the best of my knowledge, the only
reason devtmpfs needs to be mounted at all is because udev no longer
creates device files on its own (it only changes ownership and adds
symlinks). However, using devtmpfs will be problematic for libudev-compat,
which relies on inotify(2) to detect device changes (i.e. a device file
that shows up in devtmpfs from the kernel does not generate an inotify
event).


> However, you *will* need to take care that there is only one script moving
> /dev to the target filesystem, or you will almost certainly get a failed
> boot.
>
> Also, I cannot comment on whether LVM et al. are tightly coupled with udev.
>

I'm still looking into this, but so far it seems that the dmsetup and
cryptsetup hooks both pull udev-specific files in for their own
consumption. Also, the initramfs functions library, as well as the nfs,
cryptoroot, and lvm2 local-* scripts explicitly depend on udevadm to run
what is basically a poll loop to wait for udev to find the relevant
devices. It's not clear to me yet whether or not this is a consequence of
udev daemonizing before it finishes processing coldplugged devices
(introducing the need to run "udevadm settle" over and over), or a
consequence of some interplay between the udev-isms that have worked their
way into lvm and dmsetup. Either way, some refactoring and encapsulation
may be in order.


>
> > What I'm going to make my next priority is making a .deb for vdevd that
> > automatically generates the initramfs in a "safe" manner. I'd need to do
> > this in a way that disables, but does not remove the udev scripts. This
> is
> > easier said than done due to tight integration with udev, but I don't
> think
> > it's insurmountable.
> >
> > What we'll need is a well-written .deb post-install script that:
> > * sets up config files needed to generate persistent device names
>
> Install config files, marked as such.
>

It's more involved than that--the /etc/vdev/ifnames.conf file needs to be
generated from the host's network interfaces, for example (like the
/etc/udev/rules.d directory). But obviously surmountable :)


> > * installs our versions of the initramfs scripts for things like lvm,
> > suspend/resume, etc.
> > * disables the udev versions, but does not uninstall them
>
> It will be non-trivial to disable another package's scripts while leaving
> them installed, unless you divert them.
> What I did was install scripts that would become a no-op if udev was still
> installed.
>

That sounds reasonable. I'll take that approach too :)


>
> > * safely builds the initramfs from our scripts.
>
> See my mdev repo, especially debian/postinst and hooks/.
>
>
This is exactly what I'm looking for. Thanks!


>
> > * (long-term) does the above in a way that is agnostic to the device
> > manager, since there are real-world use-cases for selecting mdev or a
> > static /dev/ over udev or vdev.
> >
> > (That last point is why I'd like to work on making it possible to select
> a
> > device manager via the alternatives system--the user should be able to
> pick
> > whatever device manager they need, but without having to jump through all
> > these hoops).
>
> I doubt that you could make the device manager safely selectable via
> alternatives, unless you install the interchangeable files in
> /lib/<hotplugger>/, have that selectable via alternatives, and have
> everything that can conflict symlink into /etc/alternatives/device-manager/
> or a directory that symlinks to that.
>
> This is needed because otherwise, you are inviting an inconsistent state
> that can render a system unbootable.
>
> For example, this might be udev/mdev/initramfs-tools:
> /lib/udev-rd:
> devmanager-hook
> devmanager-top
> devmanager-bottom
>
> /lib/mdev-rd:
> devmanager-hook
> devmanager-top
> devmanager-bottom
>
> /etc/init.d:
> udev
> mdev
>

> /usr/share/initramfs-tools/hooks/devmanager ->
>         /etc/alternatives/rd-dev-manager/devmanger-hook
> /usr/share/initramfs-tools/scripts/init-top/devmanager ->
>         /etc/alternatives/rd-dev-manager/devmanager-top
> /usr/share/initramfs-tools/scripts/init-bottom/devmanager ->
>         /etc/alternatives/rd-dev-manager/devmanager-bottom

>
> An alternative is just have udev/vdev/mdev all Provide:/Conflict:
> device-manager,
> and initramfs-tools can depend on that; but this is clumsy.
>

I'm considering the former option in the long-term (/lib/<hotplugger>
etc.). Obviously this is way down the road (after libudev-compat), but
it's helpful for me to keep the overall architecture of Devuan's device
management components in mind :) I wouldn't want people who prefer
different device managers (or none at all) to have to deal with the same
integration challenges I'm dealing with.

Thanks,
Jude