Follow up to #569 (text in issue generated by llm for aid in the future fix)
Background
The multi-database PR established src/reader/registry.rs as the single source of per-backend knowledge (schemes, detection, dialect constructors, ADBC driver details, ODBC quirks). One category of knowledge deliberately stayed behind in the readers: driver-specific runtime workarounds. They are currently hardcoded at their call sites with string matching on errors, DBMS names, or connection strings. This issue tracks moving them behind registry hooks, as deferred during the reader-layer cleanup.
Current state
| Workaround |
Location |
Mechanism |
BigQuery DDL: "no destination table to read" → return empty frame |
reader/adbc.rs execute_sql |
error-substring match |
Databricks DDL: "schema bytes are empty" → drop statement, retry via execute_update, return empty frame |
reader/adbc.rs execute_sql |
error-substring match |
Snowflake Workbench credentials (connections.toml discovery, token detection/injection) |
reader/odbc/snowflake.rs (~250 lines) + 2 call sites in reader/odbc/mod.rs |
is_snowflake() string match on the connection string |
Oracle fetch batch_size = 1 (block cursors rejected with HY090) |
reader/odbc/mod.rs connect path |
DBMS-name string match |
| DataFusion ingest schema-alignment retry (Utf8View vs Utf8) |
reader/adbc.rs register |
unconditional (worth gating per driver while here) |
Proposal
Add three fields to DatabaseEntry and store the resolved entry on both readers:
adbc_no_result: Option<NoResultHandling> — enum: EmptyFrame(&'static str) (BigQuery) or RetryUpdate(&'static str) (Databricks), carrying the error substring. The two msg.contains(...) checks in execute_sql become one lookup on the entry.odbc_fetch_batch_size: Option<usize> — Oracle sets 1. The ODBC connect path already runs DBMS detection for dialect resolution, so the entry is in hand and the string match disappears.odbc_credential_provider: Option<fn(&mut String)> — fn-pointer hook (precedented by the dialect-constructor field). The Snowflake module stays as the implementation; its two call sites collapse into one hook invocation.- Optionally gate the DataFusion ingest retry behind a bool.
Wiring: AdbcReader and OdbcReader store Option<&'static DatabaseEntry>. The entry is already resolved at construction in both paths (ADBC's from_connection_string resolves it for the dialect and currently discards it; ODBC detects the DBMS on connect) — it just needs to be kept. Direct-construction paths (with_dialect, tests) get None, i.e. no workarounds.
Behavior changes (beyond cleanup)
- Tightening: today the Databricks
"schema bytes are empty" retry fires for any ADBC driver whose error contains that substring. Registry gating fires it only for Databricks. Strictly more correct; flag in review.
- Loss on generic paths: readers constructed directly (tests, jupyter) currently get error-text matching for free; with
None entry they lose it. Acceptable — document it.
adbc://adbc_driver_bigquery-style URIs resolve the entry via detection, so workarounds now apply there too (an improvement).
Caveats
- Backend ≠ driver: the quirks are driver-specific, but entries model backends. Fine at 1:1 today (one Foundry driver per backend); if entries ever support multiple drivers, these fields need to move one level down. Naming them
adbc_*/odbc_* accepts this.
- Error-text fragility remains: substrings are driver-version-sensitive. Centralizing makes them discoverable and unit-testable; only the live CI legs pin the actual strings.
- Touches reader hot paths — land after the multi-database PR, once its ODBC legs (postgres, monetdb, oracle) have validated the current runtime changes on CI.
Follow up to #569 (text in issue generated by llm for aid in the future fix)
Background
The multi-database PR established
src/reader/registry.rsas the single source of per-backend knowledge (schemes, detection, dialect constructors, ADBC driver details, ODBC quirks). One category of knowledge deliberately stayed behind in the readers: driver-specific runtime workarounds. They are currently hardcoded at their call sites with string matching on errors, DBMS names, or connection strings. This issue tracks moving them behind registry hooks, as deferred during the reader-layer cleanup.Current state
"no destination table to read"→ return empty framereader/adbc.rsexecute_sql"schema bytes are empty"→ drop statement, retry viaexecute_update, return empty framereader/adbc.rsexecute_sqlconnections.tomldiscovery, token detection/injection)reader/odbc/snowflake.rs(~250 lines) + 2 call sites inreader/odbc/mod.rsis_snowflake()string match on the connection stringbatch_size = 1(block cursors rejected with HY090)reader/odbc/mod.rsconnect pathreader/adbc.rsregisterProposal
Add three fields to
DatabaseEntryand store the resolved entry on both readers:adbc_no_result: Option<NoResultHandling>— enum:EmptyFrame(&'static str)(BigQuery) orRetryUpdate(&'static str)(Databricks), carrying the error substring. The twomsg.contains(...)checks inexecute_sqlbecome one lookup on the entry.odbc_fetch_batch_size: Option<usize>— Oracle sets 1. The ODBC connect path already runs DBMS detection for dialect resolution, so the entry is in hand and the string match disappears.odbc_credential_provider: Option<fn(&mut String)>— fn-pointer hook (precedented by the dialect-constructor field). The Snowflake module stays as the implementation; its two call sites collapse into one hook invocation.Wiring:
AdbcReaderandOdbcReaderstoreOption<&'static DatabaseEntry>. The entry is already resolved at construction in both paths (ADBC'sfrom_connection_stringresolves it for the dialect and currently discards it; ODBC detects the DBMS on connect) — it just needs to be kept. Direct-construction paths (with_dialect, tests) getNone, i.e. no workarounds.Behavior changes (beyond cleanup)
"schema bytes are empty"retry fires for any ADBC driver whose error contains that substring. Registry gating fires it only for Databricks. Strictly more correct; flag in review.Noneentry they lose it. Acceptable — document it.adbc://adbc_driver_bigquery-style URIs resolve the entry via detection, so workarounds now apply there too (an improvement).Caveats
adbc_*/odbc_*accepts this.