Repository navigation
Exploration proposes an alignment per table #554
Description
Exploration today reads a schema and produces entities of two classes it inserted itself. 0036 decisions 1, 2 and 6: it produces instead an alignment proposal per table — the class the table is a table of, the attributes its columns carry, the relations its foreign keys are, and for each aligned column an expression that turns the column into the attribute's value — and a person adopts it.
What a proposal holds
For dw.dwd_ord_dtl on the wide corpus:
| column | is |
|---|---|
amt_pay, qty, ord_st, dt_crt |
attributes of Order |
buyer_id, shop_id |
relations Order → Customer, Order → Shop |
shop_nm, prov, buyer_lvl |
attributes of Shop, Address, Customer, flattened in |
ver, etl_dt, rmk, price_old |
nothing |
Exploration finds existing classes and attributes first (the ontology, with a pack from 0008, usually has Order) and proposes new ones only where none fits — 0003's loop with a schema in place of a corpus. On a base with no ontology it proposes the class (decision 6); the difference from today is that a proposal waits and ensure_concept_types did not.
Where it goes
ontology_proposals already has the four sections this needs — entity_types, attribute_types, relation_types, map_to — and the attribute payload already carries datatype and unit. What it does not have is a way to say "these forty rows are one table's alignment, adopt or reject them together", and decide_proposal today only marks a row; creating the ontology rows is the separate adopt endpoints' job. So this needs:
- an alignment table: table → class, column → attribute (or relation) with the conversion tree from A column becomes an attribute through the tree a rule computes with #555;
- a proposal that groups a table's rows, adopted as one thing — adopting
Orderwhile rejecting every attribute of it is not a decision anyone means to make; - an adopt path that writes the ontology rows and the alignment rows in one transaction.
What exploration does not propose
Definitions. GMV, refund rate, valid orders — 0 of 18 on the wide corpus, and what it would propose is sum(amt_pay): runs, looks right, off by a hundred and includes the test orders. A definition is a rule a person writes over aligned attributes (decision 4, #556). #507 is the other way a rule can arrive — stated in a document, with provenance — and is complementary, not this.
Measured by
scripts/bench/mappings.mjs scores today's proposals by number. An alignment is scored differently: did amt_pay land on Order.paid_amount, did buyer_id become a relation, did ver stay out. The truth file for wide has keys, traps and dimensions already; it gains an alignment section.
Depends on #553 having landed, or every exploration on a fresh base re-creates the column-name entities it is trying to replace.
Activity