You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
xorq stats: a stat taking min/max as float fails on int columns since D8, with no DAG warning #1051
Since dac617b (ADR-001 D8, in #1040), xorq min and max return ColumnValue and keep the column's type (xorq_stats_v2.py:164, :169). Before, they were cast to float64. A stat that declares them as float now gets an int on int columns, and the input check at stat_pipeline.py:121 rejects it because isinstance(1, float) is False. Float columns still work.
The DAG normally warns when a float consumer reads a ColumnValue provider. Here it doesn't: min and max are in XorqStatPipeline.EXTERNAL_KEYS (xorq_stat_pipeline.py:178), and build_typed_dag skips the type check for external keys (typed_dag.py:58). The first sign is a runtime error on each int column.
No stat in buckaroo, tallyman or pydata-app declares min/max as float. Tallyman project stats can't take them at all, since the loader builds a one-parameter compute(col). Only a stat written against the Python API is affected.
Suggested fix
Any of:
A CHANGELOG note that min/max keep the column's type, so a stat reading them should declare ColumnValue or Any.
Let the input check accept an int (not a bool) where float is declared, as type checkers do under PEP 484's numeric tower.
Type-check an external key in build_typed_dag when a stat also provides it, so the mismatch warns at build time.
Context
Found during review of #1040. Line numbers are from the PR branch at a8df9b9.
Activity