You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Build N epic: CD target surface + base rebuild bundle (quartet, assets, QRF fix, feed regen, gate hardening) #449
Max greenlit (2026-07-17) the congressional-district target surface as the next build after the Build M certification: "yes do it."
What
Turn on --include-congressional-district-targets for the next certified US build (Build N), expanding the calibration surface from the national+state class (~5,667 compiled targets in Build M) to the full geographic feed surface: the v8 Ledger feed carries 23,968 congressional-district facts alongside 11,191 state and 2,101 national — a ~30k-target-class surface.
Mechanics already in place
The builder flag exists and is opt-in-off; the compile path translates CD facts via --congressional-district-vintage-crosswalk (source vintage → current districts) and hard-requires household.congressional_district_geoid on the support frame — which the Build M frame already carries (export parity tracks it at +2.3% drift).
gate_congressional_district_targets currently records False in build manifests; Build N flips it to a gated surface.
Solve-cost calibration: ~5.4× more constraint rows; measure a bounded-epoch dense probe before committing the full ladder. Loss-weighting review for the CD family (11k state rows already dominate counts; 24k CD rows will need family-level weighting the same way state rows got it).
Gate adjudication pass: within-10% and zero-support baselines will move on a 30k surface; set expectations from the probe, not Build M's 90.2%.
Scorecard: report Build N vs Build M on the shared national+state subset AND the CD surface separately, so the flip decision compares like with like.
Sequencing
After the Build M cert (policyengine.py#475) lands. Interacts with the geography-ladder/ACS lane (#288, Build L plan): the CD vintage story should be settled once and shared between calibration targets and the geography assignment.
Max (2026-07-17): time the CD surface's cost, and "if not too much we'll remove the flag and make it default behavior." So #449's deliverable includes a decision-grade timing comparison, and the flag's fate is explicit:
Timing protocol (decision evidence)
Run the same commit, same base, same machine, no other compute, and record wall-clock per stage from the builder's own stage telemetry plus /usr/bin/time -l peak RSS:
stage
CD-off (Build M baseline)
CD-on
registry compile
~2 min (measured tonight)
?
target-frame materialization (cold)
?
? (per-target cache shared where applicable)
constraint-matrix build
?
?
sparse refit (1500 epochs, frozen selection)
?
?
dense solve
?
?
gate battery + export + smoke
?
? (should be ~unchanged)
total sparse arm
~4.5 h under contention / ~3 h clean
?
Also record: peak RSS per stage (jetsam headroom on 128 GB is a live constraint), and loss/within-10% on the shared national+state subset so quality regression is visible next to the cost.
Decision rule (Max's)
If the CD-on delta is acceptable ("not too much"), remove --include-congressional-district-targets entirely and make CD targets default behavior — no permanent flag, no config fork; the surface becomes the surface (consistent with the no-codified-tiers stance). The vintage crosswalk then becomes a required pinned input of every US build rather than an optional pairing.
If the delta is material, keep the flag and record the measured cost here for the next revisit.
changed the title [-]Build N: turn on the congressional-district target surface (~30k targets; 23,968 CD facts in feed)[/-][+]Build N epic: CD target surface + base rebuild bundle (quartet, assets, QRF fix, feed regen, gate hardening)[/+]on Jul 17, 2026
Scope ratified by Max (2026-07-17). Build N's theme: one base rebuild + one feed regeneration + the CD surface + gate hardening. Build O's theme: the ACS multispine merge.
Architecture/executor: #378–#382, #326, #388. UK/BE/NZ lanes run independently.
Hygiene
Verify-and-close sweep of superseded issues (#377, #386, #359, #356, #434; #367 stays open — unexercised, not fixed) running now with per-issue evidence.
SIPP+SCF blend + SSI-aware refit: land the asset magnitudes for the Build J re-certification #374 acceptance sharpened by the certified-Build-M measurement: $10k/$20k currently +$5.73B/+793k (unchanged from Build J — the asset stage is still SCF-only: 64.3% zeros, mean $119k); dense-class acceptance ~+$1.60B (OACT $1.0B / CBPP 224k). The $2k–$20k band mass is the lever; full-removal is already robust.
Acceptance harness available as a build gate: ~/PolicyEngine/ssi-asset-limits/.local-results/populace-rerun-scripts/ (pass_c_scenario.py + pass_f_combine.py, ~5 min/scenario on any 57k artifact; pass_h_* = the 40k-chunked variant for dense frames). Anchors: baseline SSI by age vs SSA 3.99M/2.42M/0.98M; $10k/$20k vs OACT $1.0B / CBPP 224k (23k children / 154k adults / ~47k elderly).
Scope addition from the IARIW OBBBA paper loop: #454 (ESI employer premium contributions) joins the Build N bundle, same workstream as the #451 quartet and the #340/#361 residuals — it is that exact class at ~100× the mass: dense f0af251 imputes ~$1.13T (right scale vs KFF), sparse exports drop the column, nothing anchors it, and the frozen eCPS computed $0 (documented as a levels-omission in the IARIW paper's CBO-style resource measure). Fix ladder is the standard motion: input-coverage requirement across release classes, Ledger fact for aggregate employer contributions (NHEA employer share or KFF EHBS × covered lives), calibration target + release gate against the anchor. The per-column disposition pass over the #361 94-column inventory should treat ESI as its highest-mass member.
CD crosswalk revival done: PR #288 rebased onto main tip (86871cf, post-#455), conflicts resolved (builder import list; CD test now asserts the packaged-default semantics), 14 crosswalk + 116 builder tests green locally, CI green on 3.13/3.14. Marked ready for review — this unblocks the CD target surface (#314) and the timed CD A/B.
Scope addition: #463 (stage the release builder — the #406 pattern applied to build_us_fiscal_refresh_release.py) joins as the dense-lane front item, replacing the closed #456. Tonight's full-scale dense measurements are in the issue: #461 fixed the leak class (RSS returns between families), and the remaining legitimate working set crests at ~429GB / ~536GB (eligibility high-water, SSI reconcile passes) — past physical RAM it swap-crawls (6h of zero log writes at ~1 effective core), which is a throughput wall, not a correctness one. Acceptance: dense full-pool on n2-highmem-32 with the #406-style bit-equivalence proof. The #406 checkpoint machinery is directly reusable; prior art codex/jct-materialization-memory-20260617.
Sequencing consequence of #464 (merged 9243675): the CGD route legs are now hard-required export columns with no reviewed exclusion — red on any build from the current base by design. That folds #462 fix (3) (the puf_tax_detail QRF zero-inflation/donor-support guard + a base re-run through the staged #406 pipeline, frozen selection reused) into the N-lite scope: N-lite cannot certify without it. Revised N-lite contents: quartet release-side (tips target + protection, probes — #465 merged; v9 feed splice pending), CGD fix (3) + base re-run, CD target surface behind its flag for the timed A/B. The two Table 1.4 returns-count misses #464 observed on M (+495.9% / +328.1%, same #457 family) join the N gate-adjudication pass.
Timed A/B verdict: CD-on fails on quality, not wall-clock — per the decision rule, the flag stays
First CD-on run at full scale (Build N sparse attempt 2, commit fdbcf07, base 3573cdcd…, v9.2 feed, frozen rmloss100 selection, same machine as the M baseline, no contention):
measure
CD-off (Build M sparse, certified)
CD-on (N attempt 2)
compiled targets
5,667
32,820 (+27,148 CD rows, 4.8× the national+state surface)
total sparse arm wall-clock
~1h46m warm / ~3h clean
37 min to full terminal battery (registry+materialization+solve+gates)
final loss
0.02666
0.06095
within-10%
90.2%
54.5%
SS benefits national ($1.4712T, critical 5%)
pass
−14.7% ($1.2546T)
Wall-clock and memory are non-issues (peak RSS fine, run faster than M's warm baseline thanks to the deleted reconcile loop). Quality is the cost: at naive equal footing, 27k CD rows drown the ~5.7k national/state rows — within-10% halves and a critical national aggregate misses by 3× its tolerance. The concept-budget grouping (_fiscal_target_concept_budget_weights) as shipped does not protect the national surface at this scale.
Decision per Max's pre-agreed rule ("if the CD-on delta is acceptable, remove the flag; if material, keep the flag and record the cost"): the delta is material → the flag stays, and Build N ships CD-off. The CD surface needs a loss-architecture rework before any future on-arm — e.g., CD families sharing a fixed fraction of their parent national row's budget rather than competing row-by-row — which is its own design piece with this run's diagnostics (32,820-row calibration_diagnostics.json retained at populace-us-2024-buildn-sparse-rmloss100-fdbcf07-20260721T194425Z) as the baseline evidence.
Operational follow-through: #484's one-surface parity coupling gets flipped back to the CD-off regime (with a fail-closed regime constant so the next flip is deliberate), and the sparse launcher drops --include-congressional-district-targets.
Campaign index cross-post: the UK dense/local arm is now scoped and claimed as #495 — rowwise constituency-grain dataset by cloning the national staging pool (the UK analog of this build's dense/CD arm; the FRS pool is already 10× cloned, so K=1 is at US-dense per-area scale). First two increments (local-target census tool; rowwise fence-carriage + dry-run) are in progress on that issue.
UK dense/local arm update: both scoping increments of #495 are now merged — #498 (local-target census: drift-gated inventory of the local surface with six reviewed binding fences) and #501 (rowwise fence carriage, exact mass conservation, pool-lineage layer, and a dry-run whose realized support runs the real sampler at the build seed). Both went through sol cross-family review with all confirmed findings fixed pre-merge. Remaining epic increments (ladder S/NI, solve doctrine, K=1 pilot, calibrated candidate) are open for claiming on #495.
Build O dense battery (evidence arm) — populace-us-2024-buildo-dense-22bd902-20260722T175010Z, rc=1
Chain: base-O built in 21.5 min at main 22bd902f (23 stages incl. #500's adult_care_post_clone + SE-health attribution; sha 356cf501…; new columns verified: 14,462 persons incapable-of-self-care, 579 care-expense carriers with sane levels). Dense battery on feed v9.3 (eab45a2b…), 1h15m.
First production past-cap census (#494, new in this run): 5,698 targets — init-past 484, final-past 459, escaped 31, frozen 453, pushed out 6 (two EITC-by-AGI cells 0.35→2.60 and 0.63→2.14; OK/NV small interest bins; SC real-estate taxes). Contrast with sparse N's hand census (929/377/569/17): the dense arm's clamped weights barely rescue dead rows (31 escapes vs 569) — a clean quantitative statement of what the 5× anchor trades away, and the dense baseline #492's experiment wanted.
Quality: within-10% 86.52% on 5,698 (N dense 86.59% — flat, as the bound predicts).
Disposition: this artifact is the intended first evidence-tier publish once #506's machinery lands (H5 + manifest carrying these exact failures with owners). Sparse-O holds for the #492 loss-shape verdict. Evidence on disk: _buildo-runtime/out/buildo-run/dense/releases/…175010Z/.
Sparse-O attempt 1: rc=0 — GREEN ON THE FIRST ATTEMPT (Build N took six). populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z, published immutable (latest.json deliberately untouched — still N; flip/cert = maintainer, and the recommendation is to hold for attempt 2, below).
Scorecard vs certified N: within-10% 89.56% (N 89.49%), final loss 0.02813 (N 0.02824), 5,672 targets; CGD −0.03%, net CG −2.40%, SS benefits −0.26%, tips dollars −0.03%, medical +20.85% (inside the #490 0.25 bound); census 930/378/569/361/17 (same flat-cap equilibrium as N, as expected pre-#492 adoption). Coverage 163/163 including the #500 adult-care leaves — the quartet input stages are now on a gate-green sparse export.
Carried defects, both owned: the #507 SSI aged-band collapse (inherited by construction — this is why attempt 1 is NOT the certification candidate) and the tips return-count carrier deficit (−50.25%).
Why first-attempt green matters as system evidence: the six N attempts were spent discovering gate/register misalignments (#462 arc) — with #491's shared register, the measured tail register, the truthful surface, and the adjudicated bounds, the pipeline went straight through. The alignment work is what bought this.
Next: #508's SSI fix (universe adjudication posted — bands bind, national = Σ bands) → attempt 2 = the certification candidate. Equal-budget dense-3000 running now for the #493 bound-vs-convergence confound.
Dense-3000 battery complete (buildo-dense3000-22bd902-20260723T020906Z, rc=1, same two gate classes): the equal-budget arm for the #493 confound — netCG −29.59% (vs −30.18% at 1500 epochs; sparse reaches −2.40%), within-10% 87.10%, census 484/455/34/450/5. Verdict on #493: dense's CG gap is structural (optimization geometry, not budget). Build O battery day complete: base ✅, sparse-1 rc=0 published, dense-1500 ✅ evidence, dense-3000 ✅ evidence. Next certification motion = attempt 2 on the #508 SSI fix.
US local-artifact lane claimed → #512. The only local-area product in circulation (populace-us-2024-buildl-acs-local-36de5d9a-20260712T104640Z) predates the entire M/N/O target-surface repair campaign — pre-truthful CG/interest chains, old SSI machinery, no adult-care/SE-health inputs — and pe.py's dataset_overlays.us.populace_us_2024_acs_local still pins it. Rebuilding it on the certified O lineage: base-O (sha 356cf501…) donor + the O-1 surface vintage (feed v9.3), Build L calibration doctrine verbatim (no new knobs, #492), donor rows keeping base-O's certified block-ladder geography. Publish will implement #398's dataset-role contract rather than routing around the tracked path again; immutable tag, latest.json untouched; pe.py repin PR to follow. The #507 SSI aged defect rides this artifact by construction — declared in the release, with the committed one-command chain making the O-2 refresh mechanical once #508 certifies. Receipts on #512.
Feed v9.4 cut (owner): b3c0835631a446eb…, 37,405 rows = v9.3 with the jct.tax_expenditures.cy2024 family superseded 5→11 (ledger#118; values of the incumbent five byte-identical, six new #514 anchors added with concept caveats). Preflight vs base-O green-in-substance (standing 4-column advisory). Build O attempt 2 (post-#508) should launch on v9_4; the sparse launcher's FACTS pin updates accordingly. Pre-existing duplicate-srid wart filed as ledger#119.
#518 merged — Build O attempt 2 gains the truthful interest surface for free: the certified equilibrium already sits at $340.4B, so the ~470 past-cap interest rows flip near-truth at recompile without the solve moving. Attempt-2 launch config: base-O + feed v9_4 (b3c08356…; ledger#117's Table 1.4 interest facts NOT in this cut — #518's lattice auto-prefers them whenever a future feed carries them, test-pinned) + the #508 fix when its lane delivers. Expect a material within-10% jump on attempt 2 from the interest family alone.
CD-on decomposition (owner, answering 'how did CD targets affect the non-CD surface') — attempt 2 (fdbcf07, CD-on, 32,820 targets) split by the shared CD classifier:
vs ~90% CD-off same era ⇒ the true national-surface cost was ~6–7pp, not the pooled 36pp — but the verdict-driver was critical-row breakage, which decomposition confirms: SS benefits −14.72% (hard fail of the 5% critical gate; CD-off runs: −0.25% to −2.9%), itemized total −11.24%, while income tax held at −1.09%.
Two caveats for any future re-test: (1) attempt 2 ran the PRE-repair surface (its own CGD row sits at +490%; the ~470 stale interest rows were then-impossible) — post-#486/#488/#518 a CD-on rerun should cost less; (2) per #492, the flat cap made small-area dumping free, which flattered CD-on's containment — a live-tail loss changes the dynamics both ways. The honest magnitude estimate therefore needs a post-repair A/B if national CD-on is ever wanted again — which the SLD/local lane may obviate (district credibility incl. CD in the local product, where small-area targets structurally cannot touch the national surface).
Corollary to today's decomposition, per the owner's architecture ruling: #449's verdict re-reads as support starvation (27,148 CD rows on a 57,240-record support = ~2.1 hh/row; base has NO ACS spine — two channels only, asec+puf_tax_detail 168,852 each), not small-area-target toxicity. The retest that would actually answer the CD question: CD (and SLD) targets on, at ACS-spine record count (#512's expansion), post-#518 surface — one dataset, one target system, record count as the only variable.
US local-area lane (#512): PUBLISHED.populace-us-2024-buildo-acs-local-77e2061-20260724T110908Z — the ACS local artifact rebuilt on the certified O lineage (donor = the certified O-1 sparse release, sha-verified; base-O geography preserved). 1.589M households, 4,461-target state surface, within-10% 92.83% (buildl 87.87%), ESS 3× buildl, all gates green, consumer-loadable bytes, immutable tag with latest.json untouched, published through the tracked path under the #398 role contract. The #403 weight-composition skew is essentially resolved (persons/hh 2.72 vs 2.66; was 1.50 vs 2.93). Full receipts + known boundaries on #512; pe.py repin PR to follow (owner-gated).
The adequate-support CD retest (Max: 'lets try') — measured.buildo-densecd-f70bb4b-20260724T115701Z: full 338k pool, v9.4, post-#518 surface, 3000 epochs, CD-on via a provenance-committed evidence branch (regime + feed pin + regenerated manifest, f70bb4b). 32,846 targets; rc=1 (3 critical fails).
attempt 2 (sparse, stale surface)
retest (full pool, repaired)
CD-off control (dense-3000)
CD rows within-10
48.46%
65.10%
—
non-CD within-10
83.37%
83.56%
87.10%
SS benefits
−14.72%
−8.52% (fails 5%)
−3.2%-class
itemized (1_2 / ht2)
−11.2%
−15.0% / −12.4% (1_2 fails)
≈−2%
medical ht2
—
−1.72% (!)
+20%-class
net CG
—
−28.2%
−29.6%
pushed-out (census)
—
414
5–6
Verdict: support starvation was real but not the whole story. At 6× support the CD rows gain 17pp and the national cost shrinks from ~6-7pp to ~3.5pp — but criticals still break, and the census shows HOW: 414 in-cap rows dumped past the flat cap (vs 5-6 CD-off) — the #492 dumping dynamic is the mechanism by which the solve pays for CD rows. Notable inversion: CD-on FIXES medical (−1.7% vs +20.8%) — the district income-bin rows discipline exactly the weight mass the six-state problem lived in.
Composition hypothesis for the definitive A/B, after #492's sequence (donor tails → adoption): CD-on × rational tail — the rational arm independently killed the dumping (pushed-out→6) and fixed medical; combined with adequate support it is the plausible configuration where one dataset carries all geography levels. Until then the certified default stays CD-off. Evidence branch cd-retest-449-evidence (f70bb4b, not for merge); artifacts under _buildo-runtime/out/buildo-run/dense/releases/.
Run: populace-us-2024-buildo-densecdrat-61980d8-20260724T223254Z (full 338k pool, v9.4 feed, CD-on parity declaration, POPULACE_LOSS_SHAPE=rational c=3, 3000 epochs, rc=1 on gates). Compared against the two flat-cap arms.
Important caveat discovered during decomposition: this evidence branch forked before #518 (1c568c0) landed, so it still carries the 519 stale HT2 taxable-interest rows (register value $149.1B national vs the Pub 1304 actual $313.8B). That turns out to matter more than the composition question itself.
run
CD rows w10
non-CD w10
non-CD w10 excl. stale-interest†
SS benefits (5% gate)
itemized 1.2 (15% gate)
medical
net CG
final past-cap / pushed-out
dense-3000 · CD-off · cap
—
87.10%
93.86%
+0.04%
−6.76%
−0.28%
−29.59%
455 / 5
densecd · CD-on · cap
65.10%
83.56%
90.33%
−8.52%
−15.02%
−1.72%
−28.23%
1,940 / 414
densecdrat · CD-on · rational
64.25%
86.00%
90.69%
−7.11%
−15.03%
−2.16%
−34.45%
953 / 392
† symmetric exclusion of the 519 historic_table_2.*.taxable_interest_amount rows in all three runs (9 CD-side interest rows also excluded from the CD column; CD w10 moves ≤0.02pp).
Findings
The apparent rational-tail recovery is mostly an artifact of the stale interest register. Naively, rational recovers +2.44pp of the 3.54pp non-CD composition cost (83.56 → 86.00 toward 87.10). Excluding the stale rows symmetrically, the recovery is +0.36pp (90.33 → 90.69, vs the 93.86 CD-off baseline). 120 additional stale rows moved inside 10% under rational — by hitting wrong targets.
The rational tail makes register truth load-bearing; the flat cap was accidentally shielding the frame from bad targets. National HT2 taxable interest (stale target $149.1B): both cap runs sit past-cap at truth (estimates $351.1B / $371.3B — error >100%, gradient dead, mass preserved). The rational run kept gradient alive and ground the frame to $158.7B — destroying ~$200B of real interest mass to satisfy a wrong target. The input-mass parity gate caught it: export $158.7B vs the Build G reference $320.2B (−50.4%, beyond ±50%, terminal fail). This is the empirical case for the Loss shape: flat cap at 1.0 makes past-cap rows free dumping grounds (17 rows pushed out in Build N); bounded-tail experiment + past-cap census diagnostic #492 adoption sequence: register re-verification (Rebase stale HT2 taxable-interest rows onto the Pub 1304 national actual (#489 adjudication) #518-class) strictly precedes any loss-shape change. With cap, a wrong register row costs you one abandoned target; with rational, it costs you the frame.
The CD composition cost on the clean surface is ~3.5pp regardless of tail shape (93.86 → 90.3–90.7), and the CD rows themselves are indifferent to it (65.1 vs 64.3). The loss tail is not the CD lever; support is where the remaining CD misses live — consistent with the calibration doctrine (misses are generally support).
Binding gates for a CD-on dense are unchanged: SS benefits and itemized. SS improves under rational (−8.52% → −7.11%) but still fails the 5% gate; itemized 1.2/2.1 sits at the 15% boundary in both CD-on runs. Medical stays healed under CD-on in both shapes (−1.7% / −2.2%). Net CG worsens under rational (−34.45% vs −28.23%, fails the SOI 1.4 25% gate) — plausibly capacity reallocating toward large-error rows the cap had abandoned, but that attribution is unverified.
Census: rational thins the boundary population. Final past-cap 1,940 → 953; frozen 1,526 → 561; pushed-out 414 → 392. Direction favorable, but partly confounded by the interest chase pulling those carriers off the boundary.
Run diagnose_us_target_support (c192b63) on SS benefits and itemized 1.2 under the CD-on frame to classify the two binding misses as support-starvation vs capacity competition.
CD targets on the SHIP arm, clean register: the composition answer
populace-us-2024-buildo-sparsecd-rmloss100-a8ac960-20260725T132040Z — sparse (the certified arm's configuration), CD surface ON, post-#518 register, post-#548 code, 3,000 epochs, 60 min, rc=1. This run exists because #548 landed: the identical configuration this morning failed the same SSI gate and left only us_ssi_take_up.json, destroying the surface it was built to measure. This one kept calibration_diagnostics.json, input_coverage.json, and input_mass_parity.json through a four-group gate failure.
Composition cost
certified O-1 (CD-off)
this run (CD-on)
targets
5,672
32,823 (27,148 CD)
CD rows within 10%
—
47.4%
non-CD within 10%
89.56%
89.20%
non-CD, excluding the stale-HT2-interest rows
95.95%
90.40%
final loss
0.02813
0.05602
past-cap final / pushed-out
378 / 17
1,832 / 737
The headline non-CD comparison (89.56 → 89.20) is misleading and should not be quoted: O-1's population still contained 509 stale HT2 interest rows it was missing, which drag its own number down. On the comparable surface the cost of turning CD on is ≈5.5pp (95.95 → 90.40).
Criticals: SS benefits −14.1% (fails the 5% gate — the same break seen at −14.7% sparse and −8.5% dense under CD-on), itemized 1.2 −13.0% (inside the 15% bar), medical +2.8% (healed from O-1's +20.9%), net CG +1.6% (from −2.4%). The medical and CG improvements are confounded — #518's register rebase and the CD surface both changed here — so do not attribute them to CD.
The finding that matters: sparse cannot carry the CD surface
CD rows land at 47.4% within 10% here, against 65.1% on the full 338k-record dense pool with the same surface. The distribution is the story: median |error| 11.8%, p75 41.9%, p90 97.0%; only 36.7% inside 5%, and 21.4% of CD rows are off by more than half.
That is a support result, not a loss-architecture result. Same targets, same loss, same code — the only variable is record count, and the sparse frame does not have enough records per district to carry 27,148 small-area rows. It is consistent with the one-dataset architecture rather than an argument against it: CD rows bind where support suffices, and 57k records is not where it suffices. The path to CD-on shipping is the record count (full pool, and ultimately the ACS spine), not a CD-specific loss treatment.
Also visible: the pushed-out census goes 17 → 737 under CD-on, the flat cap paying for CD rows exactly as the dense arms showed.
Register recompile, quantified on the ship arm
Interest-amount rows within 10%: 25.0% → 44.2% (128/511 → 445/1,007). #518's rebase is delivering on the sparse arm, and the previously stale national HT2 row is gone from the miss list.
Residual found by the run
QRF tail concentration failed: exclusion register carries entries the checked surface did not use: ['partnership_self_employment_net_earnings'] — the per-run tail register is stale under the CD-on surface. Fail-closed working as designed; the register needs a per-surface refresh before any CD-on ship candidate.
Activity
MaxGhenis commentedon Jul 17, 2026
Max (2026-07-17): time the CD surface's cost, and "if not too much we'll remove the flag and make it default behavior." So #449's deliverable includes a decision-grade timing comparison, and the flag's fate is explicit:
Timing protocol (decision evidence)
Run the same commit, same base, same machine, no other compute, and record wall-clock per stage from the builder's own stage telemetry plus
/usr/bin/time -lpeak RSS:Also record: peak RSS per stage (jetsam headroom on 128 GB is a live constraint), and loss/within-10% on the shared national+state subset so quality regression is visible next to the cost.
Decision rule (Max's)
If the CD-on delta is acceptable ("not too much"), remove
--include-congressional-district-targetsentirely and make CD targets default behavior — no permanent flag, no config fork; the surface becomes the surface (consistent with the no-codified-tiers stance). The vintage crosswalk then becomes a required pinned input of every US build rather than an optional pairing.If the delta is material, keep the flag and record the measured cost here for the next revisit.
[-]Build N: turn on the congressional-district target surface (~30k targets; 23,968 CD facts in feed)[/-][+]Build N epic: CD target surface + base rebuild bundle (quartet, assets, QRF fix, feed regen, gate hardening)[/+]MaxGhenis commentedon Jul 17, 2026
Scope ratified by Max (2026-07-17). Build N's theme: one base rebuild + one feed regeneration + the CD surface + gate hardening. Build O's theme: the ACS multispine merge.
In (Build N)
Conditional: #424 (SSI candidate undercount) jumps in if the dense #447 trajectory adjudication points at the candidate universe.
Out (Build O)
Multispine program (#390, #395, #403, #346) and modeling-layer items: #372, #341, #321, #327, #417, #320, #360, #416, #412.
Parallel tracks (not build-gated)
Architecture/executor: #378–#382, #326, #388. UK/BE/NZ lanes run independently.
Hygiene
Verify-and-close sweep of superseded issues (#377, #386, #359, #356, #434; #367 stays open — unexercised, not fixed) running now with per-issue evidence.
MaxGhenis commentedon Jul 18, 2026
Scope addition from the SSI asset-limit analysis loop (cross-checked on the certified Build M):
is_disabled ≡ 0for ages 0–14 (CPS battery starts at 15); SIPP pu2023 carries verified-populated parent-reported child disability items (ECOGNIT 100% answered / 9.2% yes for 5–14, plus hearing/seeing/ambulatory/self-care) and RSSI_YRYN as a take-up anchor, with 0–4 banded to the SSA age curve.~/PolicyEngine/ssi-asset-limits/.local-results/populace-rerun-scripts/(pass_c_scenario.py+pass_f_combine.py, ~5 min/scenario on any 57k artifact;pass_h_*= the 40k-chunked variant for dense frames). Anchors: baseline SSI by age vs SSA 3.99M/2.42M/0.98M; $10k/$20k vs OACT $1.0B / CBPP 224k (23k children / 154k adults / ~47k elderly).MaxGhenis commentedon Jul 18, 2026
Scope addition from the IARIW OBBBA paper loop: #454 (ESI employer premium contributions) joins the Build N bundle, same workstream as the #451 quartet and the #340/#361 residuals — it is that exact class at ~100× the mass: dense f0af251 imputes ~$1.13T (right scale vs KFF), sparse exports drop the column, nothing anchors it, and the frozen eCPS computed $0 (documented as a levels-omission in the IARIW paper's CBO-style resource measure). Fix ladder is the standard motion: input-coverage requirement across release classes, Ledger fact for aggregate employer contributions (NHEA employer share or KFF EHBS × covered lives), calibration target + release gate against the anchor. The per-column disposition pass over the #361 94-column inventory should treat ESI as its highest-mass member.
MaxGhenis commentedon Jul 19, 2026
CD crosswalk revival done: PR #288 rebased onto main tip (86871cf, post-#455), conflicts resolved (builder import list; CD test now asserts the packaged-default semantics), 14 crosswalk + 116 builder tests green locally, CI green on 3.13/3.14. Marked ready for review — this unblocks the CD target surface (#314) and the timed CD A/B.
MaxGhenis commentedon Jul 20, 2026
Scope addition: #463 (stage the release builder — the #406 pattern applied to
build_us_fiscal_refresh_release.py) joins as the dense-lane front item, replacing the closed #456. Tonight's full-scale dense measurements are in the issue: #461 fixed the leak class (RSS returns between families), and the remaining legitimate working set crests at ~429GB / ~536GB (eligibility high-water, SSI reconcile passes) — past physical RAM it swap-crawls (6h of zero log writes at ~1 effective core), which is a throughput wall, not a correctness one. Acceptance: dense full-pool on n2-highmem-32 with the #406-style bit-equivalence proof. The #406 checkpoint machinery is directly reusable; prior artcodex/jct-materialization-memory-20260617.Blocking Table 1.4 dollar-row fit + QRF tail-concentration gates; exp…
MaxGhenis commentedon Jul 21, 2026
Sequencing consequence of #464 (merged 9243675): the CGD route legs are now hard-required export columns with no reviewed exclusion — red on any build from the current base by design. That folds #462 fix (3) (the
puf_tax_detailQRF zero-inflation/donor-support guard + a base re-run through the staged #406 pipeline, frozen selection reused) into the N-lite scope: N-lite cannot certify without it. Revised N-lite contents: quartet release-side (tips target + protection, probes — #465 merged; v9 feed splice pending), CGD fix (3) + base re-run, CD target surface behind its flag for the timed A/B. The two Table 1.4 returns-count misses #464 observed on M (+495.9% / +328.1%, same #457 family) join the N gate-adjudication pass.MaxGhenis commentedon Jul 21, 2026
Timed A/B verdict: CD-on fails on quality, not wall-clock — per the decision rule, the flag stays
First CD-on run at full scale (Build N sparse attempt 2, commit fdbcf07, base 3573cdcd…, v9.2 feed, frozen rmloss100 selection, same machine as the M baseline, no contention):
Wall-clock and memory are non-issues (peak RSS fine, run faster than M's warm baseline thanks to the deleted reconcile loop). Quality is the cost: at naive equal footing, 27k CD rows drown the ~5.7k national/state rows — within-10% halves and a critical national aggregate misses by 3× its tolerance. The concept-budget grouping (
_fiscal_target_concept_budget_weights) as shipped does not protect the national surface at this scale.Decision per Max's pre-agreed rule ("if the CD-on delta is acceptable, remove the flag; if material, keep the flag and record the cost"): the delta is material → the flag stays, and Build N ships CD-off. The CD surface needs a loss-architecture rework before any future on-arm — e.g., CD families sharing a fixed fraction of their parent national row's budget rather than competing row-by-row — which is its own design piece with this run's diagnostics (32,820-row
calibration_diagnostics.jsonretained atpopulace-us-2024-buildn-sparse-rmloss100-fdbcf07-20260721T194425Z) as the baseline evidence.Operational follow-through: #484's one-surface parity coupling gets flipped back to the CD-off regime (with a fail-closed regime constant so the next flip is deliberate), and the sparse launcher drops
--include-congressional-district-targets.Parity CD regime: explicit constant, fail-closed both ways, surface b…
MaxGhenis commentedon Jul 22, 2026
Campaign index cross-post: the UK dense/local arm is now scoped and claimed as #495 — rowwise constituency-grain dataset by cloning the national staging pool (the UK analog of this build's dense/CD arm; the FRS pool is already 10× cloned, so K=1 is at US-dense per-area scale). First two increments (local-target census tool; rowwise fence-carriage + dry-run) are in progress on that issue.
MaxGhenis commentedon Jul 22, 2026
UK dense/local arm update: both scoping increments of #495 are now merged — #498 (local-target census: drift-gated inventory of the local surface with six reviewed binding fences) and #501 (rowwise fence carriage, exact mass conservation, pool-lineage layer, and a dry-run whose realized support runs the real sampler at the build seed). Both went through sol cross-family review with all confirmed findings fixed pre-merge. Remaining epic increments (ladder S/NI, solve doctrine, K=1 pilot, calibrated candidate) are open for claiming on #495.
MaxGhenis commentedon Jul 22, 2026
Build O dense battery (evidence arm) —
populace-us-2024-buildo-dense-22bd902-20260722T175010Z, rc=1Chain: base-O built in 21.5 min at main
22bd902f(23 stages incl. #500'sadult_care_post_clone+ SE-health attribution; sha356cf501…; new columns verified: 14,462 persons incapable-of-self-care, 579 care-expense carriers with sane levels). Dense battery on feed v9.3 (eab45a2b…), 1h15m.What passed:
Terminal gate failures (same two classes as N dense, now with tighter evidence):
net_capital_gainsnational: −30.18% ($887.35B vs $1,270.86B) — byte-close to N dense's −30.20%, confirming the miss is structural to the 5× stretch bound on this base, not run noise. Owner: PUF donor uprating overstates mean-reverting distribution series: non_sch_d at 2.39x the SOI concept #487 (donor uprating / base CG mass). max_weight_ratio anchors differently per arm: 5x vs design (dense) but ~25x effective (sparse refit re-anchors) — declare one stretch contract #493's declared-stretch-contract question stands.First production past-cap census (#494, new in this run): 5,698 targets — init-past 484, final-past 459, escaped 31, frozen 453, pushed out 6 (two EITC-by-AGI cells 0.35→2.60 and 0.63→2.14; OK/NV small interest bins; SC real-estate taxes). Contrast with sparse N's hand census (929/377/569/17): the dense arm's clamped weights barely rescue dead rows (31 escapes vs 569) — a clean quantitative statement of what the 5× anchor trades away, and the dense baseline #492's experiment wanted.
Quality: within-10% 86.52% on 5,698 (N dense 86.59% — flat, as the bound predicts).
Disposition: this artifact is the intended first evidence-tier publish once #506's machinery lands (H5 + manifest carrying these exact failures with owners). Sparse-O holds for the #492 loss-shape verdict. Evidence on disk:
_buildo-runtime/out/buildo-run/dense/releases/…175010Z/.MaxGhenis commentedon Jul 23, 2026
Sparse-O attempt 1: rc=0 — GREEN ON THE FIRST ATTEMPT (Build N took six).
populace-us-2024-buildo-sparse-rmloss100-22bd902-20260722T232627Z, published immutable (latest.json deliberately untouched — still N; flip/cert = maintainer, and the recommendation is to hold for attempt 2, below).Scorecard vs certified N: within-10% 89.56% (N 89.49%), final loss 0.02813 (N 0.02824), 5,672 targets; CGD −0.03%, net CG −2.40%, SS benefits −0.26%, tips dollars −0.03%, medical +20.85% (inside the #490 0.25 bound); census 930/378/569/361/17 (same flat-cap equilibrium as N, as expected pre-#492 adoption). Coverage 163/163 including the #500 adult-care leaves — the quartet input stages are now on a gate-green sparse export.
Carried defects, both owned: the #507 SSI aged-band collapse (inherited by construction — this is why attempt 1 is NOT the certification candidate) and the tips return-count carrier deficit (−50.25%).
Why first-attempt green matters as system evidence: the six N attempts were spent discovering gate/register misalignments (#462 arc) — with #491's shared register, the measured tail register, the truthful surface, and the adjudicated bounds, the pipeline went straight through. The alignment work is what bought this.
Next: #508's SSI fix (universe adjudication posted — bands bind, national = Σ bands) → attempt 2 = the certification candidate. Equal-budget dense-3000 running now for the #493 bound-vs-convergence confound.
MaxGhenis commentedon Jul 23, 2026
Dense-3000 battery complete (
buildo-dense3000-22bd902-20260723T020906Z, rc=1, same two gate classes): the equal-budget arm for the #493 confound — netCG −29.59% (vs −30.18% at 1500 epochs; sparse reaches −2.40%), within-10% 87.10%, census 484/455/34/450/5. Verdict on #493: dense's CG gap is structural (optimization geometry, not budget). Build O battery day complete: base ✅, sparse-1 rc=0 published, dense-1500 ✅ evidence, dense-3000 ✅ evidence. Next certification motion = attempt 2 on the #508 SSI fix.MaxGhenis commentedon Jul 23, 2026
US local-artifact lane claimed → #512. The only local-area product in circulation (
populace-us-2024-buildl-acs-local-36de5d9a-20260712T104640Z) predates the entire M/N/O target-surface repair campaign — pre-truthful CG/interest chains, old SSI machinery, no adult-care/SE-health inputs — and pe.py'sdataset_overlays.us.populace_us_2024_acs_localstill pins it. Rebuilding it on the certified O lineage: base-O (sha356cf501…) donor + the O-1 surface vintage (feed v9.3), Build L calibration doctrine verbatim (no new knobs, #492), donor rows keeping base-O's certified block-ladder geography. Publish will implement #398's dataset-role contract rather than routing around the tracked path again; immutable tag, latest.json untouched; pe.py repin PR to follow. The #507 SSI aged defect rides this artifact by construction — declared in the release, with the committed one-command chain making the O-2 refresh mechanical once #508 certifies. Receipts on #512.MaxGhenis commentedon Jul 23, 2026
Feed v9.4 cut (owner):
b3c0835631a446eb…, 37,405 rows = v9.3 with thejct.tax_expenditures.cy2024family superseded 5→11 (ledger#118; values of the incumbent five byte-identical, six new #514 anchors added with concept caveats). Preflight vs base-O green-in-substance (standing 4-column advisory). Build O attempt 2 (post-#508) should launch on v9_4; the sparse launcher's FACTS pin updates accordingly. Pre-existing duplicate-srid wart filed as ledger#119.MaxGhenis commentedon Jul 24, 2026
#518 merged — Build O attempt 2 gains the truthful interest surface for free: the certified equilibrium already sits at $340.4B, so the ~470 past-cap interest rows flip near-truth at recompile without the solve moving. Attempt-2 launch config: base-O + feed v9_4 (
b3c08356…; ledger#117's Table 1.4 interest facts NOT in this cut — #518's lattice auto-prefers them whenever a future feed carries them, test-pinned) + the #508 fix when its lane delivers. Expect a material within-10% jump on attempt 2 from the interest family alone.MaxGhenis commentedon Jul 24, 2026
CD-on decomposition (owner, answering 'how did CD targets affect the non-CD surface') — attempt 2 (
fdbcf07, CD-on, 32,820 targets) split by the shared CD classifier:vs ~90% CD-off same era ⇒ the true national-surface cost was ~6–7pp, not the pooled 36pp — but the verdict-driver was critical-row breakage, which decomposition confirms: SS benefits −14.72% (hard fail of the 5% critical gate; CD-off runs: −0.25% to −2.9%), itemized total −11.24%, while income tax held at −1.09%.
Two caveats for any future re-test: (1) attempt 2 ran the PRE-repair surface (its own CGD row sits at +490%; the ~470 stale interest rows were then-impossible) — post-#486/#488/#518 a CD-on rerun should cost less; (2) per #492, the flat cap made small-area dumping free, which flattered CD-on's containment — a live-tail loss changes the dynamics both ways. The honest magnitude estimate therefore needs a post-repair A/B if national CD-on is ever wanted again — which the SLD/local lane may obviate (district credibility incl. CD in the local product, where small-area targets structurally cannot touch the national surface).
MaxGhenis commentedon Jul 24, 2026
Corollary to today's decomposition, per the owner's architecture ruling: #449's verdict re-reads as support starvation (27,148 CD rows on a 57,240-record support = ~2.1 hh/row; base has NO ACS spine — two channels only, asec+puf_tax_detail 168,852 each), not small-area-target toxicity. The retest that would actually answer the CD question: CD (and SLD) targets on, at ACS-spine record count (#512's expansion), post-#518 surface — one dataset, one target system, record count as the only variable.
MaxGhenis commentedon Jul 24, 2026
US local-area lane (#512): PUBLISHED.
populace-us-2024-buildo-acs-local-77e2061-20260724T110908Z— the ACS local artifact rebuilt on the certified O lineage (donor = the certified O-1 sparse release, sha-verified; base-O geography preserved). 1.589M households, 4,461-target state surface, within-10% 92.83% (buildl 87.87%), ESS 3× buildl, all gates green, consumer-loadable bytes, immutable tag withlatest.jsonuntouched, published through the tracked path under the #398 role contract. The #403 weight-composition skew is essentially resolved (persons/hh 2.72 vs 2.66; was 1.50 vs 2.93). Full receipts + known boundaries on #512; pe.py repin PR to follow (owner-gated).MaxGhenis commentedon Jul 24, 2026
The adequate-support CD retest (Max: 'lets try') — measured.
buildo-densecd-f70bb4b-20260724T115701Z: full 338k pool, v9.4, post-#518 surface, 3000 epochs, CD-on via a provenance-committed evidence branch (regime + feed pin + regenerated manifest, f70bb4b). 32,846 targets; rc=1 (3 critical fails).Verdict: support starvation was real but not the whole story. At 6× support the CD rows gain 17pp and the national cost shrinks from ~6-7pp to ~3.5pp — but criticals still break, and the census shows HOW: 414 in-cap rows dumped past the flat cap (vs 5-6 CD-off) — the #492 dumping dynamic is the mechanism by which the solve pays for CD rows. Notable inversion: CD-on FIXES medical (−1.7% vs +20.8%) — the district income-bin rows discipline exactly the weight mass the six-state problem lived in.
Composition hypothesis for the definitive A/B, after #492's sequence (donor tails → adoption): CD-on × rational tail — the rational arm independently killed the dumping (pushed-out→6) and fixed medical; combined with adequate support it is the plausible configuration where one dataset carries all geography levels. Until then the certified default stays CD-off. Evidence branch cd-retest-449-evidence (f70bb4b, not for merge); artifacts under _buildo-runtime/out/buildo-run/dense/releases/.
MaxGhenis commentedon Jul 25, 2026
CD-on × rational-tail composition run: verdict
Run:
populace-us-2024-buildo-densecdrat-61980d8-20260724T223254Z(full 338k pool, v9.4 feed, CD-on parity declaration,POPULACE_LOSS_SHAPE=rationalc=3, 3000 epochs, rc=1 on gates). Compared against the two flat-cap arms.Important caveat discovered during decomposition: this evidence branch forked before #518 (1c568c0) landed, so it still carries the 519 stale HT2 taxable-interest rows (register value $149.1B national vs the Pub 1304 actual $313.8B). That turns out to matter more than the composition question itself.
† symmetric exclusion of the 519
historic_table_2.*.taxable_interest_amountrows in all three runs (9 CD-side interest rows also excluded from the CD column; CD w10 moves ≤0.02pp).Findings
The apparent rational-tail recovery is mostly an artifact of the stale interest register. Naively, rational recovers +2.44pp of the 3.54pp non-CD composition cost (83.56 → 86.00 toward 87.10). Excluding the stale rows symmetrically, the recovery is +0.36pp (90.33 → 90.69, vs the 93.86 CD-off baseline). 120 additional stale rows moved inside 10% under rational — by hitting wrong targets.
The rational tail makes register truth load-bearing; the flat cap was accidentally shielding the frame from bad targets. National HT2 taxable interest (stale target $149.1B): both cap runs sit past-cap at truth (estimates $351.1B / $371.3B — error >100%, gradient dead, mass preserved). The rational run kept gradient alive and ground the frame to $158.7B — destroying ~$200B of real interest mass to satisfy a wrong target. The input-mass parity gate caught it: export $158.7B vs the Build G reference $320.2B (−50.4%, beyond ±50%, terminal fail). This is the empirical case for the Loss shape: flat cap at 1.0 makes past-cap rows free dumping grounds (17 rows pushed out in Build N); bounded-tail experiment + past-cap census diagnostic #492 adoption sequence: register re-verification (Rebase stale HT2 taxable-interest rows onto the Pub 1304 national actual (#489 adjudication) #518-class) strictly precedes any loss-shape change. With cap, a wrong register row costs you one abandoned target; with rational, it costs you the frame.
The CD composition cost on the clean surface is ~3.5pp regardless of tail shape (93.86 → 90.3–90.7), and the CD rows themselves are indifferent to it (65.1 vs 64.3). The loss tail is not the CD lever; support is where the remaining CD misses live — consistent with the calibration doctrine (misses are generally support).
Binding gates for a CD-on dense are unchanged: SS benefits and itemized. SS improves under rational (−8.52% → −7.11%) but still fails the 5% gate; itemized 1.2/2.1 sits at the 15% boundary in both CD-on runs. Medical stays healed under CD-on in both shapes (−1.7% / −2.2%). Net CG worsens under rational (−34.45% vs −28.23%, fails the SOI 1.4 25% gate) — plausibly capacity reallocating toward large-error rows the cap had abandoned, but that attribution is unverified.
Census: rational thins the boundary population. Final past-cap 1,940 → 953; frozen 1,526 → 561; pushed-out 414 → 392. Direction favorable, but partly confounded by the interest chase pulling those carriers off the boundary.
The six QRF donor-tail concentration failures (alimony_expense, alimony_income, bond_assets, estate_income, qualified_bdc_income, unrecaptured_section_1250_gain) fired identically on the flat-cap CD run — a dense-arm imputation property (the Build M live default: non_sch_d_capital_gains lands at $74.6B vs its $10.2B SOI target (+635%); Schedule-D CGD route is empty #462 signature), not a rational-tail effect. Already on the Loss shape: flat cap at 1.0 makes past-cap rows free dumping grounds (17 rows pushed out in Build N); bounded-tail experiment + past-cap census diagnostic #492 pre-adoption list.
Next
diagnose_us_target_support(c192b63) on SS benefits and itemized 1.2 under the CD-on frame to classify the two binding misses as support-starvation vs capacity competition.MaxGhenis commentedon Jul 25, 2026
CD targets on the SHIP arm, clean register: the composition answer
populace-us-2024-buildo-sparsecd-rmloss100-a8ac960-20260725T132040Z— sparse (the certified arm's configuration), CD surface ON, post-#518 register, post-#548 code, 3,000 epochs, 60 min, rc=1. This run exists because #548 landed: the identical configuration this morning failed the same SSI gate and left onlyus_ssi_take_up.json, destroying the surface it was built to measure. This one keptcalibration_diagnostics.json,input_coverage.json, andinput_mass_parity.jsonthrough a four-group gate failure.Composition cost
The headline non-CD comparison (89.56 → 89.20) is misleading and should not be quoted: O-1's population still contained 509 stale HT2 interest rows it was missing, which drag its own number down. On the comparable surface the cost of turning CD on is ≈5.5pp (95.95 → 90.40).
Criticals: SS benefits −14.1% (fails the 5% gate — the same break seen at −14.7% sparse and −8.5% dense under CD-on), itemized 1.2 −13.0% (inside the 15% bar), medical +2.8% (healed from O-1's +20.9%), net CG +1.6% (from −2.4%). The medical and CG improvements are confounded — #518's register rebase and the CD surface both changed here — so do not attribute them to CD.
The finding that matters: sparse cannot carry the CD surface
CD rows land at 47.4% within 10% here, against 65.1% on the full 338k-record dense pool with the same surface. The distribution is the story: median |error| 11.8%, p75 41.9%, p90 97.0%; only 36.7% inside 5%, and 21.4% of CD rows are off by more than half.
That is a support result, not a loss-architecture result. Same targets, same loss, same code — the only variable is record count, and the sparse frame does not have enough records per district to carry 27,148 small-area rows. It is consistent with the one-dataset architecture rather than an argument against it: CD rows bind where support suffices, and 57k records is not where it suffices. The path to CD-on shipping is the record count (full pool, and ultimately the ACS spine), not a CD-specific loss treatment.
Also visible: the pushed-out census goes 17 → 737 under CD-on, the flat cap paying for CD rows exactly as the dense arms showed.
Register recompile, quantified on the ship arm
Interest-amount rows within 10%: 25.0% → 44.2% (128/511 → 445/1,007). #518's rebase is delivering on the sparse arm, and the previously stale national HT2 row is gone from the miss list.
Residual found by the run
QRF tail concentration failed: exclusion register carries entries the checked surface did not use: ['partnership_self_employment_net_earnings']— the per-run tail register is stale under the CD-on surface. Fail-closed working as designed; the register needs a per-surface refresh before any CD-on ship candidate.