Skip to content

Telemetry cleanup: fix UTF-8 panic, drop noise, sharpen signal - #977

Merged
louis-e merged 4 commits into
mainfrom
chore/telemetry-fixes
Apr 24, 2026
Merged

louis-e merged 4 commits into
mainfrom
chore/telemetry-fixes

Conversation

@louis-e

@louis-e louis-e commented Apr 23, 2026

Copy link
Copy Markdown
Owner

Reported crash 80612 (progress.rs:53):

  • emit_gui_error truncated with &message[..35], panicking when byte 35 landed inside a multi-byte UTF-8 character. Triggered in production by a Russian Windows OS error 1450 message. Switch to .chars().take(35).collect() which is UTF-8 safe.

Telemetry noise reduction — ~88% of a week's logs were three non-actionable categories; remove them from telemetry while keeping the stderr prints for developer debugging:

  • retrieve_data.rs: per-attempt "Request error in download_with_reqwest" fired ~58% of volume. The outer retry loop expects individual provider failures and handles them transparently, so the per-attempt log was pure noise. Replaced with a single session-level summary emitted only when the entire provider chain fails — containing the attempted host list and the truncated last error. Added a tiny url_host helper to strip the scheme/path/query so we log "overpass.private.coffee" instead of a 1000-character URL that includes the full bbox query (also stops leaking user coordinates).
  • land_cover.rs: per-tile "Failed to fetch some land cover data" (~20% of volume) was graceful degradation. Demoted to stderr-only. Added a new one-per-session warning for the stronger condition "ESA WorldCover returned nothing for the bbox" — that one IS actionable (the generation proceeds without land cover and we want to know).
  • elevation/postprocess.rs: "Terrain peak Y=N exceeds data pack max" (10 entries) fires legitimately for Everest/Aconcagua-scale bboxes. Informational; removed from telemetry, kept as stderr.

Telemetry signal sharpening — make the remaining warnings more useful when they do fire:

  • ground.rs: "Elevation unavailable, using flat ground" now includes the truncated error text, so we can distinguish "all providers down" from "bbox outside all provider coverage" without repro.
  • world_editor/bedrock.rs: NBT serialization failures now prefix their error with which compound failed ("level.dat: ...", "block-entity/entity compound: ...", "block palette state (minecraft:foo): ..."). Replaces the opaque "Serializing Options is not supported" messages where we couldn't tell which struct to fix.

Expected effect on next release: telemetry volume down ~88%, each remaining log actionable, and the UTF-8 crash retired.

Copilot AI review requested due to automatic review settings April 23, 2026 23:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens GUI/telemetry logging by fixing a UTF-8 truncation panic, reducing high-volume non-actionable telemetry, and improving the diagnostic value of remaining warnings/errors.

Changes:

  • Fix GUI error truncation to be UTF-8 safe by truncating by characters instead of bytes.
  • Reduce telemetry volume by removing per-attempt/per-tile noise and adding a single session-level Overpass failure summary with host-only reporting.
  • Improve log usefulness by adding contextual prefixes to Bedrock NBT serialization errors and including elevation fetch failure details in the “flat ground” fallback warning.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
src/world_editor/bedrock.rs Adds context prefixes to NBT serialization failures to pinpoint which compound/state failed.
src/retrieve_data.rs Removes per-attempt telemetry noise and adds a session-level Overpass failure summary; introduces url_host.
src/progress.rs Fixes UTF-8 panic by truncating error messages by character count.
src/land_cover.rs Demotes per-tile failures to stderr-only; adds a stronger session-level warning when no land cover data is returned.
src/ground.rs Includes the (intended-to-be) truncated elevation error in the GUI warning when falling back to flat ground.
src/elevation/postprocess.rs Removes non-actionable “peak exceeds max” telemetry while keeping stderr output.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/retrieve_data.rs Outdated
Comment thread src/retrieve_data.rs
Comment thread src/land_cover.rs Outdated
Comment thread src/ground.rs Outdated
louis-e and others added 2 commits April 24, 2026 01:24
Reported crash 80612 (progress.rs:53):
- `emit_gui_error` truncated with `&message[..35]`, panicking when
  byte 35 landed inside a multi-byte UTF-8 character. Triggered in
  production by a Russian Windows OS error 1450 message. Switch to
  `.chars().take(35).collect()` which is UTF-8 safe.

Telemetry noise reduction — ~88% of a week's logs were three
non-actionable categories; remove them from telemetry while keeping
the stderr prints for developer debugging:

- retrieve_data.rs: per-attempt "Request error in download_with_reqwest"
  fired ~58% of volume. The outer retry loop expects individual
  provider failures and handles them transparently, so the per-attempt
  log was pure noise. Replaced with a single session-level summary
  emitted only when the *entire* provider chain fails — containing
  the attempted host list and the truncated last error. Added a tiny
  `url_host` helper to strip the scheme/path/query so we log
  "overpass.private.coffee" instead of a 1000-character URL that
  includes the full bbox query (also stops leaking user coordinates).
- land_cover.rs: per-tile "Failed to fetch some land cover data"
  (~20% of volume) was graceful degradation. Demoted to stderr-only.
  Added a new one-per-session warning for the stronger condition
  "ESA WorldCover returned nothing for the bbox" — that one IS
  actionable (the generation proceeds without land cover and we
  want to know).
- elevation/postprocess.rs: "Terrain peak Y=N exceeds data pack max"
  (10 entries) fires legitimately for Everest/Aconcagua-scale bboxes.
  Informational; removed from telemetry, kept as stderr.

Telemetry signal sharpening — make the remaining warnings more
useful when they do fire:

- ground.rs: "Elevation unavailable, using flat ground" now includes
  the truncated error text, so we can distinguish "all providers
  down" from "bbox outside all provider coverage" without repro.
- world_editor/bedrock.rs: NBT serialization failures now prefix
  their error with which compound failed ("level.dat: ...",
  "block-entity/entity compound: ...", "block palette state (minecraft:foo): ...").
  Replaces the opaque "Serializing Options is not supported"
  messages where we couldn't tell which struct to fix.

Expected effect on next release: telemetry volume down ~88%, each
remaining log actionable, and the UTF-8 crash retired.

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@louis-e
louis-e force-pushed the chore/telemetry-fixes branch from 1b2881e to b9ee6f7 Compare April 23, 2026 23:26
Copilot review: format!("{e:.N}") relies on the error type's Display
impl honoring f.precision(), which reqwest::Error and boxed errors do
not do — verified with rustc that a 100-char Box<dyn Error> with
{e:.10} yields a 106-char string. That meant the Overpass stderr
lines, the per-session fetch-failure summary, and the elevation
fallback log could all leak full URLs / queries / unbounded error
text to telemetry.

Switch to explicit .chars().take(N).collect::<String>() at the three
sites. Cosmetic: inline the land_cover no-data string as a single
literal (the \-continuation was already stripping whitespace
correctly, but the one-liner is cleaner).

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/retrieve_data.rs
Comment thread src/ground.rs
Copilot flagged that truncated error strings (120/200 chars) still
leak bbox coordinates, since reqwest errors format as
'error sending request for url (https://host?data=...%5Bbbox%3A...)'
— the query starts ~70 chars in, so the bbox is well within the
truncation window. Elevation provider errors embed URLs too
(regional.rs:276 formats 'Invalid payload from {url}').

Fix centrally in telemetry.rs:

- New redact_url_queries helper: find '://', scan to next whitespace
  or ')', and drop everything from '?' onward in that span. Preserves
  scheme/host/path (still tells us which provider failed). Preserves
  non-URL '?' (English punctuation) by anchoring on the scheme.
  Terminator set is {whitespace, ')'}; ',' is deliberately excluded
  because raw commas are valid inside URL queries (bbox tile-service
  format 'bbox=50.1,13.7') — caught by a unit test that initially
  failed with the naive implementation.
- Applied automatically inside send_log so every current and future
  call site is protected, not just the two I added this round.
- Also applied in the panic-hook crash path so panic messages
  carrying request URLs don't leak bbox either.
- Four unit tests cover: Overpass-style percent-encoded query,
  non-URL '?' preservation, multiple URLs, and truncated/unterminated
  URL at end of string.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/retrieve_data.rs
Comment thread src/telemetry.rs
Comment thread src/retrieve_data.rs
@louis-e
louis-e merged commit 0bad32d into main Apr 24, 2026
4 of 6 checks passed
@louis-e
louis-e deleted the chore/telemetry-fixes branch April 24, 2026 00:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants