Skip to content

[Bug]: Log pipes open even when no writer will attach, leading to deadlock during initdb #11436

Open

Description

@willmurnane

Is there an existing issue already for this bug?

  • I have searched for an existing issue, and could not find anything. I believe this is a new bug.

I have read the troubleshooting guide

  • I have read the troubleshooting guide and I think this is a new bug.

I am running a supported version of CloudNativePG

  • I have read the troubleshooting guide and I think this is a new bug.

Contact Details

[email protected]

Version

1.30.0

What version of Kubernetes are you using?

1.36

What is your Kubernetes environment?

Self-managed: kind (evaluation)

How did you install the operator?

Helm

What happened?

A bug in PR #10043 causes a leaked OS thread during initdb. This does not have a major impact on Linux systems where CNPG generally runs, but it's still a bug.

In that PR, jsonPipe := logpipe.NewJSONLineLogPipe(...) was added to WithActiveInstance() in pkg/management/postgres/instance.go.

1. `jsonPipe.Start()` creates `/controller/log/postgres.json` as a named pipe (FIFO) and calls `fileutils.OpenFileAsync(ctx, p.fileName, os.O_RDONLY, 0o600)`.
2. In Linux/POSIX, calling `open(fifo, O_RDONLY)` blocks in kernel `wait_for_partner` indefinitely until an external writer connects.
3. During `initdb` bootstrap, Postgres only opens `/controller/log/postgres.csv` (`log_destination: csvlog`). No Barman or external tools are running yet, so nothing ever opens `/controller/log/postgres.json` for write.
4. When `WithActiveInstance` finishes and calls `ctxCancel()`, `OpenFileAsync` returns on `ctx.Done()`, but the background goroutine running `os.OpenFile` **remains permanently stuck in the blocking kernel syscall on the OS thread**.
5. When `WithActiveInstance` runs `defer jsonPipe.GetExitedCondition().Wait()`, or when the Go runtime attempts process exit/thread synchronization, the blocked thread causes a deadlock.
6. Writing any byte to `/controller/log/postgres.json` from the host instantly unblocks the kernel `open()` and allows `initdb` to complete immediately.

As a consequence, under amd64 emulation using Rosetta on macOS, the initdb processes occasionally (about 5% of the time, in my testing) deadlock. Actually running the database is fine, because during that state the postgres.json named pipe does get open, unblocking the thread.

I have a suggested fix, which I think is unambiguous, and will submit a PR shortly.

Cluster resource

Any Cluster object

Relevant log output

{"level":"info","ts":"2026-08-22T14:14:08.290455636Z","msg":"Configuring new PostgreSQL instance","logging_pod":"grafana-db-1-initdb"}
    {"level":"info","ts":"2026-08-22T14:14:08.729123974Z","msg":"Executing post-init SQL instructions","logging_pod":"grafana-db-1-initdb"}
    {"level":"info","ts":"2026-08-22T14:14:08.76115752Z","msg":"Executing post-init template SQL instructions","logging_pod":"grafana-db-1-initdb"}
    {"level":"info","ts":"2026-08-22T14:14:09.271973743Z","msg":"executing Application instructions","logging_pod":"grafana-db-1-initdb"}
    <process hangs here indefinitely>

  #### Kernel process call stacks (/proc/<pid>/task/*/stack):

    === TID 8362 (goroutine opening postgres.json) ===
    [<0>] wait_for_partner+0x84/0x100
    [<0>] fifo_open+0x1b8/0x2a8
    [<0>] vfs_open+0x118/0x4a8
    [<0>] path_openat+0x41c/0x1db0
    [<0>] do_file_open+0x9c/0x160
    [<0>] __arm64_sys_openat+0x120/0x1f8

Code of Conduct

  • I agree to follow this project's Code of Conduct

Activity

  1. added a commit that references this issue on Sep 5, 2026
    b915923
  2. changed the title [-][Bug]:[/-] [+][Bug]: Log pipes open even when no writer will attach, leading to deadlock during initdb[/+] on Sep 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bug 🐛Something isn't workingtriagePending triage

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions