Skip to content

Removed inefficient LayerFilter check - #2324

Merged
lossyrob merged 2 commits into
locationtech:masterfrom
lossyrob:fix/remove-slow-check
Aug 9, 2017
Merged

lossyrob merged 2 commits into
locationtech:masterfrom
lossyrob:fix/remove-slow-check

Conversation

@lossyrob

@lossyrob lossyrob commented Aug 9, 2017

Copy link
Copy Markdown
Member

The non-overlapping check took upwards of 2 minutes in a case where I was querying a zoom 12 512x512 GlobalLayout for the state of Nevada. This is a nice thing to ensure, but not at that price. Make a note in the scaladocs for the implementor's responsibility to ensure non-overlapping KeyBounds.

@lossyrob

lossyrob commented Aug 9, 2017

Copy link
Copy Markdown
Member Author

@echeipesh need advice here. That check really slows things down unacceptably in the workshop. But the failing test reminds me why it’s there (hint, it’s for the or that I doubt has been used in anger once). Is there some clever way we can get around having to do the costly check without allowing users to potentially do invalid queries (which I believe pulled back duplicate tiles)?

@echeipesh

Copy link
Copy Markdown
Contributor

Actually this should be fine now. I think this check existed before we started using MergeQueue on the RDDReaders: https://github.com/locationtech/geotrellis/blob/master/s3/src/main/scala/geotrellis/spark/io/s3/S3RDDReader.scala#L58

Any intersecting ranges should be taken care of there.

@lossyrob
lossyrob merged commit ac4a6c2 into locationtech:master Aug 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants