Allow bootctl and kernel-install to be called without /etc/machine-id present - #16687

This reverts 341890d, which is fine by me. @keszybz

@daandemeyer can you mention in the commit msg, that the first commit is pretty much a revert of that commit?

@daandemeyer if the presumption is that the Linux will always exist on the efi partition why not stop using machined-id altogether ( as opposed to fallback on it )? If the presumption is that the Linux directory does not exist in all cases then you are down the same spiral path as machined ID is, The UUID/PartUUID of the efi partion will always exist regardless of any content or directory structure that may reside efi partition thus the only thing that should be relied upon and used ( and probably would be better for some form of "rescue environment" ) and does not required multiple check for kernel-install to create if it does not exist.

@poettering you mentioned something about some convenience in your comment on #16658 convenience for what exactly? what is it improving or making more convenient?

@daandemeyer if the presumption is that the Linux will always exist on the efi partition why not stop using machined-id altogether ( as opposed to fallback on it )?

The machine ID is a good thing because it allows multiple Linux installations to conflict-freely use the same ESP/boot partition, because each is dropping their stuff into a private dir, named after the identifier that sets each installation apart: the machine ID. In some context where multi-boot scenarios are not relevant the machine ID doesn't make sense for this though, and my presumption is @daandemeyer is in such a situation, i.e. where images are built that only consist of one OS and one OS only. Hence, machine ID is preferable in most contexts, except when it is not available.

PR looks good to me, but @keszybz should acknowledge this

@poettering Maybe something to consider is that I've read in a few places that you'd like to support A/B style partitioning setups with repart and I assume with mkosi as well in the future. Maybe that is a good argument to use the partition UUID instead of a fixed fallback since it keeps the bootloader/initramfs stuff of both partitions separate as well without requiring a machine-id to be available (assuming you'd want those to use different kernels and initrds in each partition which doesn't seem an unreasonable use case to me).

@poettering Maybe something to consider is that I've read in a few places that you'd like to support A/B style partitioning setups with repart and I assume with mkosi as well in the future. Maybe that is a good argument to use the partition UUID instead of a fixed fallback since it keeps the bootloader/initramfs stuff of both partitions separate as well without requiring a machine-id to be available (assuming you'd want those to use different kernels and initrds in each partition which doesn't seem an unreasonable use case to me).

The model I am going for would use boot loader spec type #2 kernels only for A/B style setups. I think type #2 kernels are a lot nicer if you care about robustness, since you can drop them in and remove them atomically, because they are just one file. My thinking is that if you want A/B you want robustness and immutability, and hence also type #2 kernels.

Type #2 kernels already are placed in a single, fixed directory, without any machine ID logic. so the whole problem goes away there.

The way I see type #1 vs. type #2 boot entries is that type #1 means: hacker stuff, locally generated drop-ins, support for locally changed cmdlines, pet not cattle, multiboot supported, local identity. And type #2 means: cattle not pet, robustness, same systems everywhere, limited multiboot at best, immutability, generic identity.

And thus in type #1 we need the machine ID because things are locally generated/configured, and we need some key to discern local installations. But in type #2 we don't really need that, since there's no local config for any of this, every system gets the same immutable, robust stuff, and thus keying by local identity, i.e. local machine ID is not necessary.

Or to say this differently: in a general purpose distro like classic Fedora, you take a pre-compiled fedora kernel, attach a locally built initrd to it, a locally configured kernel cmdline and bind it all together under the local machine ID. But for the A/B capable systems I have in mind none of that happens locally. instead the vendor builds a unified kernel that boots exactly one verity root disk, and there's no local configuration any of that, and no local generation, it's the same, robust, simple thing everywhere.

Now the lines are blurry between the two cases, and I am sure there are people who want to mix things differently, and that's OK, but I just want to give some background where I want to go with the A/B stuff. And for that I don't want any local IDs at all basically...

I know this is kind-of off-topic but related to the work being done, and didn't want to create a separate issue for it needlessly.

May i add an extra suggestion to this wip?
Moved my suggestion into #16759

I just realized that even if we add the fallback logic to kernel-install, installation will work but subsequent removal when upgrading the kernel will break since by then there will be a machine id so kernel-install won't look in the correct location anymore when removing a kernel.

I still wonder if we shouldn't just replace the machine id usage in kernel-install with the PARTUUID of the root partition. That should be unique inside an image/block device and doesn't change before and after an image is booted.

I still wonder if we shouldn't just replace the machine id usage in kernel-install with the PARTUUID of the root partition. That should be unique inside an image/block device and doesn't change before and after an image is booted.

Why not just replace the machine id with a GUID specially created for the loader-entry config? (each loader-entry config it's own GUID)
That way the loader-entry config and the kernel/initrd will be bound to each-other without any possibility of pollution, it will also be easy to remove that directory because the same GUID will be used by the loader-entry config using it.

Now that i think about it a bit more, maybe you should use separate GUID's per kernel and initrd files, because most of the time only the initrd is changed and the kernel can be re-used...
That would allow to make use of entries using the previous initrd ala "old kernel" distros used, but this time we get "current", "old" and now "previous" (see here) 😉

I still wonder if we shouldn't just replace the machine id usage in kernel-install with the PARTUUID of the root partition. That should be unique inside an image/block device and doesn't change before and after an image is booted.

Note that one thing systemd-repart does besides creating new partitions and growing them is "give partitions an identity". And that means you can ship a disk image with the partition uuid set to all zeroes, and repart will then fill them in for you with something better.

We used to have a TODO list item somewhere that was about initializing /etc/machine-id from the fs uuid of the root fs by default if /etc/machine-id is missing. I removed that TODO item eventually, realizing that on environments where the rootfs is an immutable image this made little sense, since in that case you want that each invocation of the same immutable image should get its own uuid...

hmm, maybe a tweak to this should be that if $ESP/Linux exists we only use that dir, and not the machine id.

That basically would mean if you have it once we use it forever. And means that in setups where you have no machine ID initialized then you don#t get multiboot, ever. but that should be ok

We used to have a TODO list item somewhere that was about initializing /etc/machine-id from the fs uuid of the root fs by default if /etc/machine-id is missing. I removed that TODO item eventually, realizing that on environments where the rootfs is an immutable image this made little sense, since in that case you want that each invocation of the same immutable image should get its own uuid...

@poettering kernel-install needs to be able to repair itself which requires the tool to be able to recreate identical functional bootable setup based on the environment it resides in which one of the reason the PARTUUID is an better engineer choice and arguably should be used and even be the string that exists and is used in /etc/machine-id instead of the generated one. ( then it wont matter if /etc/machine-id exist or not )