Repository navigation
Add Overture Maps building data integration - #886
Conversation
Complement OSM building data with Overture Maps ML-derived footprints to fill coverage gaps in areas with sparse OSM mapping (e.g., rural Africa, parts of Asia). Key implementation details: - STAC collections.parquet catalog for fast partition discovery (1 req instead of 512) with geographic pre-filtering - Sparse HTTP Range requests: only download matching Parquet row groups (~4-5 MB each) instead of full partition files (~580 MB each) - Source-based dedup: skip buildings whose primary source is OSM - Spatial dedup: grid-accelerated centroid-in-bbox check against existing OSM buildings - Extract height, min_height, num_floors, roof_shape, roof_material, roof_orientation, facade_color, roof_color, building class/subtype - Smart height filtering: omit low Overture heights so buildings use the pipeline's default, keeping visual consistency with OSM buildings - New --overture CLI flag (default on) with GUI toggle Co-Authored-By: Claude Opus 4.6 <[email protected]>
Resolve conflicts: keep both rotation (from main) and no_overture (from this branch) in args.rs and gui.rs. Co-Authored-By: Claude Opus 4.6 <[email protected]>
There was a problem hiding this comment.
Pull request overview
Integrates Overture Maps GeoParquet building footprints as a supplemental data source to improve building coverage where OpenStreetMap is sparse, and merges/deduplicates them into the existing OSM-based generation pipeline.
Changes:
- Added
src/overture.rsimplementing STAC partition discovery, sparse HTTP Range downloads of Parquet row groups, OSM-source filtering, and conversion toProcessedElements. - Wired Overture fetching + OSM-dedup into both CLI and GUI generation flows, gated by a new CLI arg (
--no-overture). - Added new dependencies (
parquet,bytes) and updated the lockfile accordingly.
Reviewed changes
Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| src/overture.rs | New Overture integration module: fetch, parse, filter, dedup, and convert buildings into existing element format. |
| src/main.rs | CLI: fetches Overture buildings (unless disabled) and merges them after OSM parsing. |
| src/gui.rs | GUI: fetches Overture buildings (currently hardcoded enabled) and merges them after OSM parsing. |
| src/args.rs | Adds --no-overture CLI switch to disable supplementation. |
| Cargo.toml | Adds bytes and parquet dependencies required for sparse Parquet reading. |
| Cargo.lock | Locks new transitive dependencies introduced by Parquet/Bytes. |
Comments suppressed due to low confidence (1)
src/gui.rs:907
- GUI generation hardcodes
no_overture: false, and there’s no other in-repo reference that lets the UI set/override this option. Since the PR description calls out a GUI toggle, this looks incomplete: please plumb a GUI setting through toArgs.no_overtureso users can disable Overture supplementation from the GUI.
// Create generation options
let generation_options = GenerationOptions {
path: generation_path.clone(),
format: world_format,
level_name,
spawn_point: mc_spawn_point,
};
// Create an Args instance with the chosen bounding box
// Note: path is used for Java-specific features like spawn point update
let args: Args = Args {
bbox,
file: None,
save_json_file: None,
path: Some(if world_format == WorldFormat::JavaAnvil {
generation_path
} else {
world_path
}),
bedrock: world_format == WorldFormat::BedrockMcWorld,
downloader: "requests".to_string(),
scale: world_scale,
ground_level,
terrain: terrain_enabled,
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Track non-OSM building count for early-stop limit instead of total count (prevents premature partition processing halt in OSM-heavy areas) - Reject rows with missing id field instead of falling back to "unknown" (prevents ProcessedWay ID collisions from hash convergence) - Reject invalid WKB byte order values in parse_wkb_polygon (only 0/1 are valid per WKB spec; other values indicate corrupt geometry)
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace vertex-in-bbox containment check with polygon bbox overlap test in building_to_processed_way. The old check dropped buildings that straddled the bbox boundary (edges crossing but all vertices outside). The new check matches the standard overlap test used by parse_overture_row, correctly including buildings whose footprint overlaps the target area regardless of vertex positions. - Fix GUI progress bar regression: Overture fetch emitted progress value 4.0 after OSM parsing had already reached 14.0, causing the progress indicator to jump backwards. Now emits 14.5, which slots correctly between parse completion (14.0) and elevation fetch (14.0).
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 6 changed files in this pull request and generated 5 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Remove --overture/--no-overture CLI flag and conditionals — Overture Maps data is now always fetched as a supplemental building source. The error boundary in fetch_overture_buildings already degrades gracefully to an empty Vec on any failure. - Clip Overture building polygons to xzbbox via clip_way_to_bbox, matching how OSM ways are clipped in osm_parser. Previously Overture buildings straddling the bbox boundary could produce out-of-bounds XZ coordinates. - Fix panic in process_partition_file: add length check on the Parquet tail bytes returned by fetch_range before slicing. A truncated HTTP 206 response would have caused an unrecoverable index-out-of-bounds panic that bypassed the Result error boundary. - Remove redundant Bytes allocation in list_partition_files: pass response.bytes()? directly to SerializedFileReader instead of round-tripping through Vec<u8>. - Filter out OSM-sourced buildings at collection time rather than at the final conversion step, reducing peak memory usage.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 8 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace DefaultHasher with FNV-1a in gers_id_to_u64. Rust does not guarantee DefaultHasher's algorithm across compiler versions, so Overture building IDs (and deterministic RNG seeded from them) could silently change on toolchain upgrades. FNV-1a is a well-known, stable hash with good distribution and zero dependencies. - Guard fetch_range against length==0: the subtraction end = start + length - 1 would underflow to u64::MAX, producing an invalid HTTP Range header. Now returns an explicit error.
|
retrigger-benchmark |
|
⏱️ Benchmark run finished in 0m 36s 📈 Compared against baseline: 30s 📅 Last benchmark: 2026-04-05 22:39:18 UTC You can retrigger the benchmark by commenting |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 5 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
When the user draws a new rectangle, edits/resizes the existing one, deletes it, navigates via city search, or types new coordinates in the bbox input field, the rotation angle is now reset to 0. This prevents stale rotation values from carrying over to a completely different area. Rotation reset flows through two funnel points: - displayBboxInfoText: all map-originated bbox changes (draw, edit, delete, search, paste) - handleBboxInput: manual text input of new coordinates
The local_boost cap controls how many stair layers are placed before the flat ridge section at the peak. At 0.75, many small-medium houses had a noticeably wide flat strip at the top, missing just one more stair layer. Bumping to 0.85 narrows the ridge for most building sizes while the wall_cap (60% of wall height) still prevents disproportionately tall roofs. Applied to both gabled and hipped roofs.
Complement OSM building data with Overture Maps ML-derived footprints to fill coverage gaps in areas with sparse OSM mapping (e.g., rural Africa, parts of Asia).
Key implementation details: