Repository navigation
Fix spatial Hilbert to use offset of the minimum range key - #2586
Merged
echeipesh merged 1 commit intoMar 29, 2018
Merged
Conversation
Contributor
Author
|
Because this is basically a glaring bug and evidence is that this is not popular index strategy the intention is to back-port this to |
echeipesh
force-pushed
the
fix/spatial-hilbert-offset
branch
from
March 29, 2018 15:52
7efbf22 to
da11948
Compare
echeipesh
force-pushed
the
fix/spatial-hilbert-offset
branch
from
March 29, 2018 16:00
da11948 to
3e39aa5
Compare
lossyrob
approved these changes
Mar 29, 2018
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
Spatial Hilbert index incorrectly did not offset the keys being encoded by the minimum key in the
KeyBoundsof the index. Because Hilbert will attempt to figure out the number of bits required to encode values in the given key range it the following exception can be thrown whenKeyBoundsprovided to the HilbertSpatialKeyIndex constructor does not containminKey = SptialKey(0, 0)java.lang.IllegalArgumentException: value doesn't fitHowever, it is possible that a layer indexed by
HilbertSpatialKeyIndexwithminKey>SpatialKey(0,0)would not produce this error when writing if the minimum key was large enough to require the same number of bits as the maximum key. This PR will break reading those layers by changing the mapping from stored record index toSpatialKeyinstance.It seems unlikely that this bug would have gone unreported if this is was a prevailing case, however there is a way to fix such layers without re-ingesting them.
IF this PR breaks your layer indexed by
HilbertSpatialKeyIndexyou can fix it by changing the layer metadatakeyIndex.properties.keyBounds.minKeyto havecol = 0androw = 0.This value can be find in the JSON file
metadata__<layer_name>__<zoom_level>.jsonin S3, File, and Hadoop attribute stores.Before:

After:

This will not effect the layer
KeyBoundsas shown in theTileLayerMetadata[K]but only adjust theKeyBoundsused by theHilbertSpatialKeyIndexinstance, adjusting it to match new reality.docs/CHANGELOG.rstupdated, if necessary- [ ]docsguides update, if necessary- [ ] New user API has useful Scaladoc strings