Skip to content

Estimate partitions number basing on a desired partition size - #2289

Closed
pomadchin wants to merge 1 commit into
locationtech:masterfrom
pomadchin:feature/s3geotiffrdd-opt
Closed

pomadchin wants to merge 1 commit into
locationtech:masterfrom
pomadchin:feature/s3geotiffrdd-opt

Conversation

@pomadchin

@pomadchin pomadchin commented Jul 19, 2017 •

Copy link
Copy Markdown
Member

This PR also introduces a new default S3GeoTiffRDD behaviour.
The question is: should we change the default reader behaviour? Or it's still should be an optional feature.

@pomadchin
pomadchin requested a review from echeipesh July 19, 2017 14:18
@pomadchin
pomadchin force-pushed the feature/s3geotiffrdd-opt branch 2 times, most recently from 26b5618 to c7a012c Compare July 19, 2017 14:35
…sired partition size in bytes.

Signed-off-by: Grisha Pomadchin <[email protected]>
@pomadchin
pomadchin force-pushed the feature/s3geotiffrdd-opt branch from c7a012c to a44f158 Compare July 19, 2017 14:40
@echeipesh

Copy link
Copy Markdown
Contributor

To answer the question: The default behavior on principle should "just work" as much as possible.

That is not the case currently, we pretty much require partition and window size tuning before we can expect good performance. So in such a case the defaults can be changed to provide the correct behavior.

@pomadchin

Copy link
Copy Markdown
Member Author

Fixed in #2296

@pomadchin pomadchin closed this Aug 2, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants