Round-012 probe of the let()/ref()/call() DAG surface — never probed before this campaign. ~75 case-shapes × both engines, oracles written to ORACLES.md before execution, every finding verified twice on two fixtures. 4 correctness findings (3 silent-wrong), 4 typed-error gaps. The Cypher GRAPH x = … / USE x surface came back exemplary and is the fix precedent for several of these.
F1 — SILENT-WRONG: a let() DAG's default output is NONDETERMINISTIC across processes
topological_sort iterates dependents.get(current, set()) — a set of strings (chain_let.py:157). Python randomizes string hashing per process, so sibling ordering, and therefore which binding is "last" when output= is omitted, changes run to run.
b = {'root': n({}), 'q1': ref('root',[n({'k':'A'})]),
'q2': ref('root',[n({'k':'B'})]), 'q3': ref('root',[n({'k':'C'})])}
g.gfql(let(b)) # no output=
Across 14 process runs (identical on both engines):
| PYTHONHASHSEED |
0 |
1 |
2 |
3 |
4 |
5 |
10 |
11 |
13 |
14 |
| returned |
[] |
a |
b |
a |
d |
b |
w,x |
y,z |
u,v |
w,x |
Deterministic within a process — which is exactly what test_chain_let.py::test_execution_order_deterministic checks, so the existing pin structurally cannot see it. Individual binding values are correct; only the choice of returned binding is wrong. Fix: sort the dependents iteration.
F2 — SILENT-WRONG: an ASTCall binding silently consumes the previous binding's output
execute_node passes __original_graph__ to ASTNode/ASTEdge/Chain/nested-ASTLet but passes accumulated_result to ASTCall (chain_let.py:352-355 vs :275-284, :364). Independent siblings leak into each other, and reordering two unrelated dict keys changes the answer — violating the contract pinned by test_let_matchers.py::test_matchers_operate_on_root_graph.
P = let({'flt': n({'type':'person'}), 'deg': call('get_degrees')})
Q = let({'deg': call('get_degrees'), 'flt': n({'type':'person'})})
Oracle (root-graph degrees) 5 nodes {a:3,b:2,c:3,d:1,e:1}. Actual, both engines: P → 3 nodes with every person reported as degree 0; Q → correct. Reproduced with get_degrees, get_indegrees, hop, count_table, and call-after-call; a name= flag from one binding leaks a column into a sibling.
F3 — SILENT-WRONG: a user binding named __original_graph__ hijacks the DAG root
The root graph is stashed in the same namespace as user bindings (chain_let.py:423) and every result is stored via set_binding (:374). let({'__original_graph__': n({'type':'company'}), 'x': n({})}) returns 2 nodes for x instead of all 5 — every later binding silently re-roots on that binding. output='__original_graph__' is also addressable.
F4 — Cross-engine divergence + a bare assert
Row-pipeline call() results have _edges but unbound _source/_destination. Downstream ref(x, [n()]): pandas silently returns a result; polars raises RuntimeError: … AssertionError: from a bare assert with an empty message at polars/chain.py:898. Verified on limit, skip, distinct, drop_cols, rows, count_table. The chain surface declines this shape cleanly (NotImplementedError: polars chain engine does not yet support call() before a traversal); the DAG surface has no such guard.
Typed-error gaps
F5 nested let dependencies are invisible to the scheduler (extract_dependencies, chain_let.py:37-40), so lexical read-through resolves by declaration order — reordering turns a working DAG into a RuntimeError. F6 DAG bindings downgrade GFQL*Error to bare RuntimeError (:586-592); the same failing op gives GFQLSchemaError on the chain surface. F7 single-binding DAGs skip dependency validation entirely (:191-192), so let({'x': ref('x',[])}) misses the dedicated self-reference message a 2-binding DAG produces. F8 structural DAG errors (missing ref, cycle, self-ref, unknown output=) are bare ValueError — while the Cypher GRAPH/USE surface raises properly coded GFQLValidationError for the identical classes, a ready-made template.
NIE-tier
Schema effects under-declared (hypergraph adds undeclared EventID/category; tree_layout adds level); collapse leaks the internal __gfql_node_collapse_0__ column into user output, in the namespace validate_column_name forbids users from creating. ASTLet/chain_let docstrings advertise an idiom that is pinned to return empty (test_let_matchers.py::test_ref_uses_binding_graph). circle_layout fails with a raw KeyError: 'x'.
QUIET (auditable)
call() frame immutability — 32 safelist ops × 2 engines, zero mutations (the DAG/call surface was NOT in the earlier no-mutation sweep, so this closes a real hole). #1913 stale-index class clean on DAGs. Evaluate-once confirmed via hook count. Var-length inside a binding value-identical to the same chain. Mid-DAG decline raises unwrapped with no partial result. JSON round-trip, aggregates in bindings, empty/zero-row bindings, diamond independence, inner-binding non-leakage. Cypher GRAPH/USE: 9 shapes × 2 engines fully correct and fully typed.
Note F1-F3 are all cases where both engines agree and both are wrong — differential testing could not have found them.
Priority: F2 (reordering changes answers) → F1 (nondeterminism) → F3 (namespace hijack) → F4 → the typed-error gaps, several of which can copy the GRAPH/USE implementation.
Round-012 probe of the
let()/ref()/call()DAG surface — never probed before this campaign. ~75 case-shapes × both engines, oracles written toORACLES.mdbefore execution, every finding verified twice on two fixtures. 4 correctness findings (3 silent-wrong), 4 typed-error gaps. The CypherGRAPH x = … / USE xsurface came back exemplary and is the fix precedent for several of these.F1 — SILENT-WRONG: a
let()DAG's default output is NONDETERMINISTIC across processestopological_sortiteratesdependents.get(current, set())— a set of strings (chain_let.py:157). Python randomizes string hashing per process, so sibling ordering, and therefore which binding is "last" whenoutput=is omitted, changes run to run.Across 14 process runs (identical on both engines):
[]abadbw,xy,zu,vw,xDeterministic within a process — which is exactly what
test_chain_let.py::test_execution_order_deterministicchecks, so the existing pin structurally cannot see it. Individual binding values are correct; only the choice of returned binding is wrong. Fix: sort the dependents iteration.F2 — SILENT-WRONG: an
ASTCallbinding silently consumes the previous binding's outputexecute_nodepasses__original_graph__to ASTNode/ASTEdge/Chain/nested-ASTLet but passesaccumulated_resulttoASTCall(chain_let.py:352-355vs:275-284,:364). Independent siblings leak into each other, and reordering two unrelated dict keys changes the answer — violating the contract pinned bytest_let_matchers.py::test_matchers_operate_on_root_graph.Oracle (root-graph degrees) 5 nodes
{a:3,b:2,c:3,d:1,e:1}. Actual, both engines:P→ 3 nodes with every person reported as degree 0;Q→ correct. Reproduced withget_degrees,get_indegrees,hop,count_table, and call-after-call; aname=flag from one binding leaks a column into a sibling.F3 — SILENT-WRONG: a user binding named
__original_graph__hijacks the DAG rootThe root graph is stashed in the same namespace as user bindings (
chain_let.py:423) and every result is stored viaset_binding(:374).let({'__original_graph__': n({'type':'company'}), 'x': n({})})returns 2 nodes forxinstead of all 5 — every later binding silently re-roots on that binding.output='__original_graph__'is also addressable.F4 — Cross-engine divergence + a bare assert
Row-pipeline
call()results have_edgesbut unbound_source/_destination. Downstreamref(x, [n()]): pandas silently returns a result; polars raisesRuntimeError: … AssertionError:from a bareassertwith an empty message atpolars/chain.py:898. Verified onlimit,skip,distinct,drop_cols,rows,count_table. The chain surface declines this shape cleanly (NotImplementedError: polars chain engine does not yet support call() before a traversal); the DAG surface has no such guard.Typed-error gaps
F5 nested
letdependencies are invisible to the scheduler (extract_dependencies,chain_let.py:37-40), so lexical read-through resolves by declaration order — reordering turns a working DAG into aRuntimeError. F6 DAG bindings downgradeGFQL*Errorto bareRuntimeError(:586-592); the same failing op givesGFQLSchemaErroron the chain surface. F7 single-binding DAGs skip dependency validation entirely (:191-192), solet({'x': ref('x',[])})misses the dedicated self-reference message a 2-binding DAG produces. F8 structural DAG errors (missing ref, cycle, self-ref, unknownoutput=) are bareValueError— while the Cypher GRAPH/USE surface raises properly codedGFQLValidationErrorfor the identical classes, a ready-made template.NIE-tier
Schema effects under-declared (
hypergraphadds undeclaredEventID/category;tree_layoutaddslevel);collapseleaks the internal__gfql_node_collapse_0__column into user output, in the namespacevalidate_column_nameforbids users from creating.ASTLet/chain_letdocstrings advertise an idiom that is pinned to return empty (test_let_matchers.py::test_ref_uses_binding_graph).circle_layoutfails with a rawKeyError: 'x'.QUIET (auditable)
call()frame immutability — 32 safelist ops × 2 engines, zero mutations (the DAG/call surface was NOT in the earlier no-mutation sweep, so this closes a real hole). #1913 stale-index class clean on DAGs. Evaluate-once confirmed via hook count. Var-length inside a binding value-identical to the same chain. Mid-DAG decline raises unwrapped with no partial result. JSON round-trip, aggregates in bindings, empty/zero-row bindings, diamond independence, inner-binding non-leakage. Cypher GRAPH/USE: 9 shapes × 2 engines fully correct and fully typed.Note F1-F3 are all cases where both engines agree and both are wrong — differential testing could not have found them.
Priority: F2 (reordering changes answers) → F1 (nondeterminism) → F3 (namespace hijack) → F4 → the typed-error gaps, several of which can copy the GRAPH/USE implementation.