Skip to content

rebase - #3

Merged
taskset merged 2752 commits into
taskset:upstreamfrom
systemd:master
Aug 19, 2020
Merged

taskset merged 2752 commits into
taskset:upstreamfrom
systemd:master

Conversation

@taskset

@taskset taskset commented Aug 19, 2020

Copy link
Copy Markdown
Owner

No description provided.

yuwata and others added 30 commits July 22, 2020 19:55
time-util: clock mapping improvements
The Address objects in the set generated by ndisc_router_generate_addresses()
have the equivalent prefixlen, flags, prefered lifetime.
This commit makes ndisc_router_generate_addresses() return Set of
in6_addr.
This reverts commit c7220ca.

The removal was done as a reaction to the messages from systemd:
initrd-root-fs.target: Requested dependency OnFailure=emergency.target ignored (target units cannot fail).
initrd.target: Requested dependency OnFailure=emergency.target ignored (target units cannot fail).
initrd-root-device.target: Requested dependency OnFailure=emergency.target ignored (target units cannot fail).
initrd-fs.target: Requested dependency OnFailure=emergency.target ignored (target units cannot fail).
local-fs.target: Requested dependency OnFailure=emergency.target ignored (target units cannot fail).
...
But it seems that the messages themselves are wrong, and the units were OK.
Fixes #16401.

c80a9a3 introduced the .can_fail field,
but didn't set it on .targets. Targets can fail through dependencies.
This leaves .slice and .device units as the types that cannot fail.

$ systemctl cat bad.service bad.target bad-fallback.service
[Service]
Type=oneshot
ExecStart=false

[Unit]
OnFailure=bad-fallback.service

[Service]
Type=oneshot
ExecStart=echo Fixing everythign!

$ sudo systemctl start bad.target
systemd[1]: Starting bad.service...
systemd[1]: bad.service: Main process exited, code=exited, status=1/FAILURE
systemd[1]: bad.service: Failed with result 'exit-code'.
systemd[1]: Failed to start bad.service.
systemd[1]: Dependency failed for bad.target.
systemd[1]: bad.target: Job bad.target/start failed with result 'dependency'.
systemd[1]: bad.target: Triggering OnFailure= dependencies.
systemd[1]: Starting bad-fallback.service...
echo[46901]: Fixing everythign!
systemd[1]: bad-fallback.service: Succeeded.
systemd[1]: Finished bad-fallback.service.
There are a lot of edge cases that the current implementation
doesn't handle, especially in cases where one of passwd/shadow
exists and the other doesn't exist. For example, if
--root-password is specified, we will write /etc/shadow but
won't add a root entry to /etc/passwd if there is none.

To fix some of these issues, we constrain systemd-firstboot to
only modify /etc/passwd and /etc/shadow if both do not exist
already (or --force) is specified. On top of that, we calculate
all necessary information for both passwd and shadow upfront so
we can take it all into account when writing the actual files.

If no root password options are given --force is specified or both
files do not exist, we lock the root account for security purposes.
Let's simplify things a bit, and take ownership of more fields in
ExecParameters, so that they are automatically freed when the structure
is released.
Currently, each change to NEWS triggers a meson reconfigure that
changes SOURCE_EPOCH which causes a full rebuild. Since NEWS changes
relatively often, we have a full rebuild each time we pull from
master even if we pull semi-regularly. This is further compounded
when using branches since NEWS has a relatively high chance to
differ between branches which causes git to update the modification
time, leading to a full rebuild when switching between branches.

We fix this by using the creation time of the latest git tag instead.
firstboot: Add --root-shell option and tighten up passwd/shadow handling
semaphoreci was failing with:
Can't exec "tree": No such file or directory at /tmp/autopkgtest-lxc.v9oand4g/downtmp/build.TIm/src/test/udev-test.pl line 1752.

https://semaphoreci.com/systemd/systemd/branches/pull-request-16551/builds/1
github ci was failing with:

meson.build:685:16: ERROR: String '' cannot be converted to int
We would add and remove definitions based on which colors were used by other
code. Let's just define all of them to simplify tests and allow easy comparisons
which colors look good.
We need to do better here, but for now let's at least not trick
users into nuking their graphical environment. Inspired by #16548.
This API is a complete mess. We forgot to do a hashed comparison for duplicate
entries and we use a direct pointer comparison. For trivial_hash_ops the result
is the same. For all other case, it's not. Fixing this properly will require
auditing all the uses of set_put() and ordered_set_put(). For now, let's just
acknowledge the breakage.
network: sync link state file on dbus call, and ndisc cleanups
[zjs: Looking at https://packages.ubuntu.com/bionic/iptables-dev, iptables-dev
was a transitional package that was pulling in libxtables-dev, libip4tc-dev,
and libip6tc-dev (as listed by @GiedriusS). iptables-dev is gone in focal, so
replace it by the expanded list.]
mscherer and others added 27 commits August 16, 2020 15:10
All paragraphs are indented to 8 spaces, but one. Fix that.
Unprivileged test-fs-util fails on my system since /sys/dev/block is
inaccessible for unprivileged users, so let's skip encrypted path test if we
get EACCES or similar.
I am pretty sure /etc/hosts (i.e. an explicitly configured, local,
trusted database) should be useful for overriding the automatic
myhostname logic.

resolved's internal logic handles it that way and hence we should
suggest it in the NSS fallback line, too.

Let's also bring the factory file back into sync with what the docs say.

And update the prose a bit too, to actually match what we recommend.
This information was already available in the debug output, but I think it
is good to include it in the message in the table. This makes it easier to wrap
one's head around the allowlist/denylist filtering.
This comes up occasionally with new users. The phrase "Logs begin ..." is
ambiguous because it can be taken to mean the logs being displayed or all logs
(the intended meaning). Let's rephrase this as "Journal begins ..." to make
this clearer.
Every time I was using this function I had to check whether "newline"
means that newlines are good or bad.
…vity

We do it in other cases, we should here too.

Fixup for b070c7c.
Also order the syscalls by syscall number for easier comparisons with the
kernel headers.

Fixup for 5f152f4.
This was done for all replacements back in 5187dd2,
but some newer stuff didn't do this.
glibc 2.26 lifted restrictions on search domains count or length to
unlimited. This has also been backported to 2.17 in some distributions (RHEL 7
and derivatives). Other softwares may have their own limits for search domains,
but we should not restrict what is written out any more.

https://sourceware.org/legacy-ml/libc-announce/2017/msg00001.html
This was needed before commit 16e4fd8 added a
mode that opens the log fds for every single log message. This mode is used in
execute.c since then making the explicit call to log_open unnecessary.

This basically reverts ea89a11.
Allow bootctl and kernel-install to be called without /etc/machine-id present
Fix a presumed copy&paste error. SetLockedHint corresponds to the "locked hint", not the "idle hint".
load-fragment: fix grammar in error messages
Was trying to run src/partition/test-repart.sh on CentOS 8 and the first
resize call kept failing with ERANGE. Turned out that CentOS 8 comes
with libfdisk-devel-2.32.1 which is missing
util-linux/util-linux@2f35c1e
(in libfdisk 2.33 and up).
Improve systemd-analyze security a bit and other assorted bits
@taskset
taskset merged commit f8b4008 into taskset:upstream Aug 19, 2020
taskset added a commit that referenced this pull request Sep 15, 2020
The stack is as follows:
(gdb) bt
    #0  0x00007f644f19e8c7 in kill () from /lib64/libc.so.6
    #1  0x00005556566edcdd in crash (sig=6) at src/core/main.c:206
    #2  <signal handler called>
    #3  0x00007f644f19e5f7 in raise () from /lib64/libc.so.6
    systemd#4  0x00007f644f19fce8 in abort () from /lib64/libc.so.6
    systemd#5  0x0000555656756882 in log_assert_failed (text=text@entry=0x5556567fc545 "dev_autofs_fd >= 0",
    file=file@entry=0x5556567fc3b4 "src/core/automount.c", line=line@entry=370,
    func=func@entry=0x5556567fd0b4 <__PRETTY_FUNCTION__.17397> "open_ioctl_fd") at src/shared/log.c:754
    systemd#6  0x00005556567b064a in open_ioctl_fd (dev_autofs_fd=-1, where=<optimized out>, devid=<optimized out>) at src/core/automount.c:370
    systemd#7  0x00005556567b10f6 in automount_send_ready (a=a@entry=0x555656b79110, tokens=0x555656c8b560, status=status@entry=0)
    at src/core/automount.c:469
    systemd#8  0x00005556567b360e in automount_update_mount (a=0x555656b79110, old_state=old_state@entry=MOUNT_DEAD, state=state@entry=MOUNT_MOUNTED)
    at src/core/automount.c:509
    systemd#9  0x00005556567ac9e8 in mount_notify_automount (state=MOUNT_MOUNTED, old_state=MOUNT_DEAD, m=0x555656b77000) at src/core/mount.c:588
    systemd#10 mount_set_state (m=m@entry=0x555656b77000, state=MOUNT_MOUNTED) at src/core/mount.c:619
    systemd#11 0x00005556567ad068 in mount_coldplug (u=0x555656b77000, deferred_work=<optimized out>) at src/core/mount.c:671
    systemd#12 0x000055565679c589 in unit_coldplug (u=0x555656b77000, deferred_work=deferred_work@entry=0x555656d3e070) at src/core/unit.c:2886
    systemd#13 0x00005556566f031e in manager_coldplug (m=m@entry=0x555656ac5980) at src/core/manager.c:1125
    systemd#14 0x00005556566f4a7a in manager_startup (m=0x555656ac5980, serialization=0x555656ac5230, fds=<optimized out>) at src/core/manager.c:1288
    systemd#15 0x00005556566ea4e3 in main (argc=4, argv=0x7ffe78ac9848) at src/core/main.c:1798

(gdb) frame  7
    at src/core/automount.c:469
469         ioctl_fd = open_ioctl_fd(UNIT(a)->manager->dev_autofs_fd, a->where, a->dev_id);

(gdb) p *a
$1 = {meta = {manager = 0x555656ac5980, type = UNIT_AUTOMOUNT, load_state = UNIT_LOADED, merged_into = 0x0,
    id = 0x555656b29ce0 "proc-sys-fs-binfmt_misc.automount", instance = 0x0, names = 0x555656b79450, dependencies = {0x555656b78500, 0x0, 0x0,
      0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x555656b794f0, 0x0, 0x0, 0x0, 0x0, 0x555656b76980, 0x555656b784c0, 0x0, 0x555656b76710, 0x0, 0x0, 0x0, 0x0,
      0x555656b769f0, 0x555656b79530}, requires_mounts_for = 0x555656b76750,
    description = 0x555656b76eb0 "Arbitrary Executable File Formats File System Automount Point", documentation = 0x555656b76960,
    fragment_path = 0x555656b6e540 "/usr/lib/systemd/system/proc-sys-fs-binfmt_misc.automount", source_path = 0x0, dropin_paths = 0x0,
    fragment_mtime = 1595213181000000, source_mtime = 0, dropin_mtime = 0, job = 0x0, nop_job = 0x0, job_timeout = 0,
    job_timeout_action = EMERGENCY_ACTION_NONE, job_timeout_reboot_arg = 0x0, refs_by_target = 0x0, conditions = 0x555656b769c0, asserts = 0x0,
    condition_timestamp = {realtime = 1591608832758220, monotonic = 70060990954163}, assert_timestamp = {realtime = 1591608832758232,
      monotonic = 70060990954175}, inactive_exit_timestamp = {realtime = 1591608832758666, monotonic = 70060990954609}, active_enter_timestamp = {
      realtime = 1591608832758666, monotonic = 70060990954609}, active_exit_timestamp = {realtime = 1591608832758141,
      monotonic = 70060990954084}, inactive_enter_timestamp = {realtime = 1591608832758141, monotonic = 70060990954084}, slice = {source = 0x0,
      target = 0x0, refs_by_target_next = 0x0, refs_by_target_prev = 0x0}, units_by_type_next = 0x0, units_by_type_prev = 0x0,
    has_requires_mounts_for_next = 0x0, has_requires_mounts_for_prev = 0x0, load_queue_next = 0x0, load_queue_prev = 0x0, dbus_queue_next = 0x0,
    dbus_queue_prev = 0x0, cleanup_queue_next = 0x0, cleanup_queue_prev = 0x0, gc_queue_next = 0x555656b78840, gc_queue_prev = 0x555656b796c0,
    cgroup_queue_next = 0x0, cgroup_queue_prev = 0x0, target_deps_queue_next = 0x0, target_deps_queue_prev = 0x0, pids = 0x0, sigchldgen = 0,
    gc_marker = 0, auto_stop_ratelimit = {interval = 10000000, begin = 0, burst = 16, num = 0}, deserialized_job = -1, load_error = 0,
    unit_file_state = _UNIT_FILE_STATE_INVALID, unit_file_preset = -1, cgroup_path = 0x0, cgroup_realized_mask = 0, cgroup_subtree_mask = 0,
    cgroup_members_mask = 0, on_failure_job_mode = JOB_REPLACE, stop_when_unneeded = false, default_dependencies = false,
    refuse_manual_start = false, refuse_manual_stop = false, allow_isolate = false, ignore_on_isolate = true, ignore_on_snapshot = false,
    condition_result = true, assert_result = true, transient = false, in_load_queue = false, in_dbus_queue = false, in_cleanup_queue = false,
    in_gc_queue = true, in_cgroup_queue = false, in_target_deps_queue = false, sent_dbus_new_signal = true, no_gc = false, in_audit = false,
    cgroup_realized = false, cgroup_members_mask_valid = true, cgroup_subtree_mask_valid = true}, state = AUTOMOUNT_DEAD,
  deserialized_state = AUTOMOUNT_RUNNING, where = 0x555656b76fd0 "/proc/sys/fs/binfmt_misc", timeout_idle_usec = 0, pipe_fd = 24,
  pipe_event_source = 0x0, directory_mode = 493, dev_id = 1048609, tokens = 0x555656c8b560, expire_tokens = 0x0, expire_event_source = 0x0,
  result = AUTOMOUNT_SUCCESS}

Fix it by processing manager->dev_autofs_fd before calling open_ioctl_fd.
taskset pushed a commit that referenced this pull request Nov 6, 2020
This lets the libc/xcrypt allocate as much storage area as it needs.
Should fix systemd#16965:

testsuite-46.sh[74]: ==74==ERROR: AddressSanitizer: stack-buffer-overflow on address 0x7f3e972e1080 at pc 0x7f3e9be8deed bp 0x7ffce4f28530 sp 0x7ffce4f27ce0
testsuite-46.sh[74]: WRITE of size 131232 at 0x7f3e972e1080 thread T0
testsuite-46.sh[74]:     #0 0x7f3e9be8deec  (/usr/lib/clang/10.0.1/lib/linux/libclang_rt.asan-x86_64.so+0x9feec)
testsuite-46.sh[74]:     #1 0x559cd05a6412 in user_record_make_hashed_password /systemd-meson-build/../build/src/home/user-record-util.c:818:21
testsuite-46.sh[74]:     #2 0x559cd058fb03 in create_home /systemd-meson-build/../build/src/home/homectl.c:1112:29
testsuite-46.sh[74]:     #3 0x7f3e9b5b3058 in dispatch_verb /systemd-meson-build/../build/src/shared/verbs.c:103:24
testsuite-46.sh[74]:     systemd#4 0x559cd058c101 in run /systemd-meson-build/../build/src/home/homectl.c:3325:16
testsuite-46.sh[74]:     systemd#5 0x559cd058c00a in main /systemd-meson-build/../build/src/home/homectl.c:3328:1
testsuite-46.sh[74]:     systemd#6 0x7f3e9a88b151 in __libc_start_main (/usr/lib/libc.so.6+0x28151)
testsuite-46.sh[74]:     systemd#7 0x559cd0583e7d in _start (/usr/bin/homectl+0x24e7d)
testsuite-46.sh[74]: Address 0x7f3e972e1080 is located in stack of thread T0 at offset 32896 in frame
testsuite-46.sh[74]:     #0 0x559cd05a60df in user_record_make_hashed_password /systemd-meson-build/../build/src/home/user-record-util.c:789
testsuite-46.sh[74]:   This frame has 6 object(s):
testsuite-46.sh[74]:     [32, 40) 'priv' (line 790)
testsuite-46.sh[74]:     [64, 72) 'np' (line 791)
testsuite-46.sh[74]:     [96, 104) 'salt' (line 809)
testsuite-46.sh[74]:     [128, 32896) 'cd' (line 810)
testsuite-46.sh[74]:     [33152, 33168) '.compoundliteral' <== Memory access at offset 32896 partially underflows this variable
testsuite-46.sh[74]:     [33184, 33192) 'new_array' (line 832) <== Memory access at offset 32896 partially underflows this variable
testsuite-46.sh[74]: HINT: this may be a false positive if your program uses some custom stack unwind mechanism, swapcontext or vfork
testsuite-46.sh[74]:       (longjmp and C++ exceptions *are* supported)
testsuite-46.sh[74]: SUMMARY: AddressSanitizer: stack-buffer-overflow (/usr/lib/clang/10.0.1/lib/linux/libclang_rt.asan-x86_64.so+0x9feec)

It seems 'struct crypt_data' is 32896 bytes, but libclang_rt wants more, at least 33168?
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.