fix: make memory opt-in and preserve pressure compaction - #580

What & why

Fixes #561.

CompozyOS is a control plane. Starting a daemon or completing an ordinary turn should not opt an operator into persistent memory extraction or background dreaming. memory.enabled and the existing independent roles.dream.enabled now default to false. To opt in, set [memory] enabled = true in the daemon configuration and restart; enable [roles.dream] enabled = true separately for dreaming. Existing explicit settings retain their meaning, and no user configuration or stored memory is rewritten or deleted.

The opt-in path is also repaired. Previously, the extractor discarded the entire JSONL batch on its first invalid line, treated common no-memory answers as parse failures, and stopped the child with a misleading completion detail before parsing was accounted for. The timeout reported in the issue is a separate execution deadline, not a JSON-parser consequence.

• The prompt requests {"no_candidates":true}. Conventional empty answers, an empty array, and JSONL fences are handled deliberately. Unknown prose and malformed candidates remain line-numbered failures.
• Valid candidates survive a mixed batch and reach the existing inbox/controller. The batch records memory.extractor.failed, not a false completed event.
• A successful semantic terminal is required before candidate admission. Interrupted/error output remains in redacted diagnostics; cancellation, truncation, or missing terminal is a failure.
• Child stop causes distinguish parsed output, extraction failure, and timeout. Extraction-stage failures enter the existing DLQ and failure listing with parent session/workspace/agent identity and redacted diagnostics. Raw model output cannot be replayed as a normalized candidate inbox.
• Session pressure compaction remains under session.compaction.enabled, independently of persistent memory. Factory boot retains private checkpoint WAL storage and resume-only coverage. Only actual pressure can launch its summary child; no public memory provider or session-end worker is registered. An explicit checkpoint-role opt-out or failed summary leaves uncovered events unarchived.
• Public background-role status and invocation use the same daemon/workspace memory gate. Effective memory roles honor the memory switch. Dream eligibility is checked before consolidation gates or state-changing work; enabling memory alone leaves dreaming off, and enabled runtime instances retain live/workspace dream routing.

How you verified it

Implemented and reviewed with Codex, GPT-6-Astra, high reasoning. Checked head: 5e90cd6.

• Reproduced the original parser defect using a Go source overlay and the same canonical output-contract test: invalid N/( responses and loss of valid candidates around malformed output. The fixed cases passed with -race.
• Full config, extractor, and consolidation suites passed. Capped daemon reruns covered changed extractor, role, and lifecycle behavior. An unchanged boot-barrier timeout under shared host load passed its standalone retry without changing the deadline or assertions.
• Canonical checkpoint/compaction suites passed, including factory-role suppression, rejection of uncovered disabled-summary requests, session hook/archive behavior, and crash-safe compaction integration. Scope detail is in the QA report below.
• Real daemon/ACP-subprocess E2E verified zero default extractor/dream sessions and extraction events, opted-in no-candidate success, persistence of a valid candidate beside malformed output, HTTP failure identity, CLI extractor status, UDS drain/trigger behavior, deliberate built-in/workspace dreaming, live model application, catalog parity, and next-session recall.
• Factory boot + pressure + ACP disconnect/recovery passed (9.770 s; package 11.925 s): one pressure summary child, covered archive, retained history, checkpoint-backed replay without raw duplication, and no background memory children.
• Existing skills-only daemon boot integration verified current shared schema-stream heads while memory remains disabled.
• Review remediation race checks passed: daemon roles/extractor/checkpoints (6.990 s), core/HTTP/UDS health and settings. Active-memory health fixtures explicitly opt in, preserving all original assertions.
• The owner generator go run ./cmd/compozy-codegen all completed without generated drift.
• The initial local make gate found a trailing blank line, which was corrected. The owner explicitly requested publication with the full gate delegated to CI; a clean full local gate is not claimed. All required CI checks remain delivery requirements.

Final heavy checks used GOMAXPROCS=4, COMPOZY_GO_TEST_P=1, and COMPOZY_GO_LINT_CONCURRENCY=2 through mise exec. Manual Go tests retained -race -p=1 -parallel=4; integration also used the repository's modernc checkptr setting. The E2E runner acquired the repository's machine-wide verification lock. The short isolated lab used 57-byte daemon and at-most-88-byte harness sockets. Both labs completed targeted teardown with clean: true. The strict QA audit has one remaining criterion, C14 (full local gate), delegated to CI by the owner; it found zero other blockers or warnings.

QA evidence and cross-surface audit.

Impact

Existing keys, CLI verbs, HTTP/UDS routes, native tool IDs, and response shapes remain. Global/profile/workspace configuration precedence is retained under the daemon memory master switch. No SQLite shape or migration changes are required; disabled boot still applies memory schema migrations. This preserves SD-013 user state and public-surface compatibility while intentionally changing omitted defaults.

Site configuration/memory references, the official Compozy skill reference, affected QA scenarios, and a release note co-ship. Web uses the existing controls and contracts. Retrieval algorithms (#562) and startup tool-manual policy (#563) are unchanged.

Limits: deterministic ACP fixtures prove daemon/process/protocol behavior, not live-provider reliability or timeout frequency. Provider-native compaction was not exercised; the factory pressure/resume check exercises Compozy checkpoint coverage and degraded replay.

• Full local gate delegated to CI at the owner's explicit request; required current-head CI checks must pass before delivery
• New or changed behavior is covered by tests, or I explained above why not
• If an agent wrote or co-wrote this, I named it above and verified the result myself