Skip to content

Update writ receipts to v4 graph format - #7

Merged
David-Noble-at-work merged 2 commits into
developfrom
feature/writ-v4-graph-receipt
Jan 24, 2026
Merged

David-Noble-at-work merged 2 commits into
developfrom
feature/writ-v4-graph-receipt

Conversation

@David-Noble-at-work

Copy link
Copy Markdown
Contributor

Summary

  • Replace flat entries array with graph-based nodes+edges structure (v4)
  • Add legacy.go with LegacyReceipt type and ToGraph() converter for v2/v3 backward compat
  • Version-detecting Load() auto-converts legacy receipts in memory
  • ToDependencyGraph() projects execution graph to coarse dependency graph
  • Updated builder API: AddNode, AddNodeWithChecksums, AddDelegated(node)
  • Updated state.go, commands.go, status.go to consume new types

Test plan

  • go build ./... compiles
  • go test ./internal/writ/... all pass
  • Pre-commit hooks pass
  • Manual: writ add produces v4 graph-format receipt
  • Manual: writ status reads v4 receipt correctly
  • Backward compat: place a v2/v3 receipt, confirm Load() converts it

David Noble added 2 commits January 23, 2026 19:36
Replace flat entries array with graph-based nodes+edges structure.
Add version-detecting Load() with legacy v2/v3 conversion, dependency
graph projection, and updated builder API (AddNode, AddDelegated).
@David-Noble-at-work
David-Noble-at-work merged commit 8b3369a into develop Jan 24, 2026
1 check passed
@David-Noble-at-work
David-Noble-at-work deleted the feature/writ-v4-graph-receipt branch January 31, 2026 03:57
David-Noble-at-work pushed a commit that referenced this pull request May 6, 2026
…gen multi-Resource

Closes 13.0(i). Restructures the status / result packages around a shared
[pkg/sink.Sink] byte-out contract; renames status.Console to status.Narrator
to reflect the progress-arc semantic; concretizes both wrapper types on
[op.RuntimeEnvironment]; deletes the loud-failure-default sinks; and fixes
codegen to detect Resource types by constructor signature so packages with
multiple Resources (mem.Resource + mem.Function) get one gen file each.

1. pkg/sink (new package).

   pkg/sink/sink.go defines the byte-out contract used by both pkg/status
   (categorized narration) and pkg/result (structured emission). One Sink
   interface (io.WriteCloser + IsTTY() bool); one canonical impl `*impl`
   parameterized internally by writer + close + isTTY; six convenience
   constructors (Stderr / Stdout / Capture / Discard / File / Tee) and
   one generic (New). Capture returns (Sink, *bytes.Buffer) so tests can
   assert on captured bytes. File owns its file's lifecycle; Tee cascades
   Close. The other four no-op on Close (don't own their writers).

   Naming dropped `New` prefix from convenience constructors — sink.Console
   wasn't constructing a Console type, it was producing a configured Sink.
   Generic constructor keeps `sink.New(writer)` since it does construct the
   package's primary type. Stderr/Stdout chosen over Console as the file-
   descriptor names are self-documenting and avoid the "Console wraps non-
   console writer" oddity.

2. pkg/status restructure.

   Console/Capture/Discard/Sink-interface deleted. New status.Narrator
   concrete type wraps a [sink.Sink] with the six categorized methods
   (Note / Warn / Error / Succeed / Fail / Print) plus program-name prefix
   and color-via-IsTTY. NewNarrator(programName, sink) is the single
   constructor.

   Test fixture status.Capture deleted entirely. Audit of 75 use sites
   showed every test only ever inspected `.Notes` — no test compared
   counts across categories or asserted on multi-category state. Tests now
   construct a Narrator wrapping sink.Capture() and assert on the buffer's
   bytes. Single mechanism, clean test code.

   Narrator named for the progress-arc semantic — emissions build context
   for a terminal Succeed (positive resolution) or Fail (negative). Distinct
   from a discrete event-stream "Reporter" — the stream has shape (open /
   body / terminal). Package doc says "narration"; the type name now
   matches.

   Silence is selected at construction by passing sink.Discard(), not by
   toggling a Narrator field. The cli boundary's --silent flag forks
   construction: silent → sink.Discard(), default → sink.Stderr().

3. pkg/result restructure.

   Sink interface deleted. UnconfiguredSink (loud-failure default) deleted.
   result.Pipeline keeps its name (composition-shaped, not role-shaped) but
   now consumes a [sink.Sink] instead of an io.Writer; Pipeline.Emit feeds
   the filtered value to formatter.Format(value, sink) where sink is the
   [io.Writer] surface of the Sink. Formatters unchanged — they already
   took io.Writer; sink.Sink IS an io.Writer so signatures still work.

4. env.Status / env.Result concrete types.

   pkg/op/RuntimeEnvironment.Status and .Result are now concrete
   `*status.Narrator` and `*result.Pipeline` instead of the deleted
   interfaces. Same change in pkg/op/RuntimeEnvironmentSpec. WithStatus /
   WithResult builder methods take the concrete pointer types. Defaults
   in NewRuntimeEnvironmentSpec construct a Narrator wrapping sink.Discard
   and a Pipeline wrapping sink.Discard with JSONFormatter; defaults in
   NewRuntimeEnvironment (when Spec.Build hits zero values) construct a
   Narrator wrapping sink.Stderr and a Pipeline wrapping sink.Stdout with
   JSONFormatter.

   Catalog initialization: NewRuntimeEnvironment now sets `Catalog:
   NewResourceCatalog()` unconditionally. This was always missing for the
   starlark-bridge dispatch path (only the executor and graph initialized
   it), and codegen's file.write_text calls panicked on catalog.GetOrCreate
   with a nil receiver. The fix puts catalog ownership at env construction.

5. pkg/process Runner.

   pkg/process/Runner.NewRunner takes `pipeline *result.Pipeline,
   narrator *status.Narrator` instead of the deleted interfaces. Field
   types updated to match. No behavior change.

6. internal/cli output layer.

   internal/cli/output.go: BuildSink renamed BuildPipeline; returns
   *result.Pipeline instead of result.Sink. Wraps caller-supplied
   io.Writer in sink.New() internally so callers can keep passing
   bytes.Buffer / os.File / etc. Package-global narrator default
   constructed via status.NewNarrator("", sink.Discard()) — replaces
   "var statusUI status.Sink = status.Discard{}". SetUI / UI pair takes
   *status.Narrator. The cli facade functions (Note/Warn/Error/Failure/
   Success/Print) forward unchanged.

   internal/cli/root.go: --silent fork at construction. Was previously
   passing `silent` as a parameter to status.NewConsole; now picks
   sink.Discard() vs sink.Stderr() based on the flag and constructs a
   Narrator from the chosen sink. Same shape applied in
   cmd/devlore-test/devloretest/root.go and cmd/star/main.go.

7. Codegen multi-Resource detection.

   star/extensions/com.noblefactor.devlore.Actions/commands/generate.star:
   detect_resource (single, hardcoded "Resource" struct name, stale
   *op.ExecutionContext signature) replaced with detect_resources (plural)
   that walks all functions in the package and identifies any whose
   signature matches `func(*op.RuntimeEnvironment, any) (*T, error)`.
   Returns a list of (struct_name, constructor_name) pairs. Constructor
   signature IS the public contract — embedding chains can be transitive
   (mem.Function embeds mem.Resource which embeds op.ResourceBase) and
   structural-only detection misses those.

   detect_resource_params now takes a struct_name argument so it can pull
   methods on the right receiver type per Resource.

   Downstream emit loop produces one gen file per detected Resource type,
   named after the type via to_snake (Resource → resource.gen.go,
   Function → function.gen.go).

   resource.gen.go.template parameterized: {{.struct_name}} and
   {{.constructor_name}} replace the previously-hardcoded `Resource` and
   `NewResource`. Same template, multiple outputs per package.

8. mem package — both Resource and Function now have gen files.

   pkg/op/provider/mem/gen/resource.gen.go (regenerated) and
   pkg/op/provider/mem/gen/function.gen.go (new) announce mem.Resource and
   mem.Function respectively. The hand-rolled init() in mem/function.go
   that previously did the AnnounceResource for Function is deleted —
   the gen file is the canonical registration.

   Makefile rule for mem updated to declare both outputs as a grouped
   target (&:); function.gen.go added to NEW_OP_INVENTORY.

9. Codegen template updates.

   star/extensions/com.noblefactor.devlore.Actions/templates/
   receiver_type.gen_test.go.template: newCtx returns *op.RuntimeEnvironment
   wrapping a Narrator over sink.Discard (silent — for tests that don't
   inspect narration); dryRunCtx returns (*op.RuntimeEnvironment,
   *bytes.Buffer) so dry-run tests can assert on captured bytes via
   substring matching. Imports bytes + sink + status.

   action.gen_test.go.template: dry-run test pattern simplified — was
   `notes := ctx.Status.(*status.Capture).Notes; for ... strings.Contains
   ...`; now `ctx, buf := dryRunCtx(t); ... strings.Contains(buf.String(),
   wantSubstring)`. Single substring check per test, no slice iteration.
   status import dropped (no longer references status.Capture).

   resource.gen.go.template: parameterized struct/constructor (see #7).

10. Test-site updates.

    pkg/op/provider/service/provider_test.go: newTestProvider env gains
    Status: status.NewNarrator("test", sink.Discard()) so provider methods
    that emit through env.Status don't panic on nil. Imports sink + status.

    pkg/op/provider/ui/provider_test.go: captureUI test fixture deleted;
    forwarding tests now construct a real Narrator over sink.Capture()
    and assert via strings.Contains on the buffer. Imports bytes + sink +
    status.

    cmd/star/provider/starcode/provider_test.go: env constructions gain
    Catalog: op.NewResourceCatalog() so file.Provider methods that call
    catalog.GetOrCreate don't panic on nil.

    internal/cli/output_test.go: BuildSink → BuildPipeline; assertion
    relaxed from interface type-assertion to non-nil check. result import
    dropped.

    cmd/devlore-test/devloretest/data/*.star (~25 scripts): file method
    calls renamed mode= → chmod= to match the file-provider parameter
    rename from earlier in the branch.

11. Regenerated gen files (~75 across pkg/op/provider/* and
    cmd/star/provider/*).

    Every provider's receiver_type.gen_test.go and action.gen_test.go
    regenerated to pick up the new test-fixture pattern (newCtx /
    dryRunCtx with Narrator + sink). Module gen tests also touched where
    the receiver type signatures changed. Provider gen files unchanged
    in structure; the regen confirms idempotence under the new templates.

Verified: go build ./... clean. go test ./... — three packages still
fail with pre-existing branch debt unrelated to this refactor:
cmd/devlore-test (CLI) and cmd/devlore-test/devloretest (~43 tests fail
with node_count=0 because the runner harness doesn't materialize plan
invocations into graph nodes — Phase 8 D5 design change without harness
update); cmd/star/provider/goast (TestConfigSchemas_ProviderPicksUpConfig,
config-merge-vs-replace bug). Independent concerns; documented as
follow-ups.

13.0(i) is complete.
David-Noble-at-work pushed a commit that referenced this pull request May 16, 2026
Capture two related architectural decisions worked out in conversation:

(1) ExecutableUnit.Parameters() is asymmetric by type. Node returns the
full Go-method signature (current Bind behavior, no code change).
Subgraph returns the bubble-up variable surface — the deduped,
type-checked set of VariableValue references walked across children's
slots, recursing into nested subgraphs. The topological-root union
framing is gone; the graph-walk over slot fills is the simpler primitive.
No separate UnboundSlots() accessor — variables ARE the bubble-up; that
abstraction collapsed once the binding layer existed.

(2) Execution model commitment: stack frames + subgraph ownership +
per-type executors. Every subgraph invocation pushes a frame carrying
its own Variables overlay (scope-chain to parent), Results map,
Attempts, and Status. Concurrent invocations of the same Subgraph
template (e.g., gather iterating 16 items) are isolated by frames, not
by cloning templates. Subgraphs own children via an explicit `parent
*Subgraph` pointer; per-type executors (root / gather / choose /
wait_until / plain) dispatch their own children's frames. The current
Subgraph.Branch boolean is deprecated by this model.

Doc changes:

  - D3 rewritten with the asymmetric Parameters() semantics. Note added
    about Gather's use of Subgraph.Parameters() as the discovery surface
    for per-iteration variable binding.

  - D11 inserted (forward-looking). Documents the Frame struct, what
    frames solve (per-iteration scoping, scope chain, concurrent
    isolation, compensation), what stays per-executor (retry policy,
    status emission, dispatch logic), the ownership pointer, and the
    Branch deprecation.

  - Phase 3 rewritten. Scope: Subgraph.Parameters() bubble-up via
    graph-walk + parent pointer + Subgraph.AddChild method + migration
    of the two direct Children-append sites in pkg/op/graph.go. Full
    frame plumbing and per-type executors stay downstream (Step
    13/14/15) — Phase 3 plants only the forward-compat scaffolding.

  - Locked decisions extended with #6 (execution model) and #7
    (Parameters() asymmetry).

D7 / D9 / D10 unchanged — they describe the current execution model
(env.variables-based, executor stores variable map on receiver) which
remains accurate for 13.0(n) Phases 1-6. D11 forecasts the model shift.

Issue: #283
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant