Skip to content

Relax safeguards on delete methods - #2039

Merged
lossyrob merged 1 commit into
locationtech:masterfrom
moradology:feature/quiet-deleter
Mar 8, 2017
Merged

lossyrob merged 1 commit into
locationtech:masterfrom
moradology:feature/quiet-deleter

Conversation

@moradology

@moradology moradology commented Mar 2, 2017 •

Copy link
Copy Markdown
Contributor

Prior to this change, delete methods threw in any case where the reference to a layer happened to be faulty. This is based on metadata which, though always present in case of tile data (and necessary for its deletion), might not be present. Lacking this metadata, we should log information about the failure and delete any _attribute data we can.

@moradology moradology changed the title Relax safeguards on delete methods Relax safeguards on s3 delete methods Mar 2, 2017
} catch {
case e: AttributeNotFoundError => throw new LayerDeleteError(id).initCause(e)
case e: AttributeNotFoundError =>
logger.info(s"Metadata for $id was not found. Any associated layer data (if any) will require manual deletion")

@pomadchin pomadchin Mar 2, 2017 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we already had similar way of wrapping errors (SomeException.initcause() ...), probably you remember it; should consider changing the behaviour of all backends? As currently all layer deleters throw LayerNotFoundError in case of missing layer, and native s3 errors in other cases. Don't forget that all layer managements functions change should be consistent across all backends.

@pomadchin pomadchin Mar 2, 2017 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Btw, can this change be done as? A good thing would be add to all backends.

@pomadchin

pomadchin commented Mar 2, 2017 •

Copy link
Copy Markdown
Member

Eh i didn't understand immediately that you want to improve logging and md deletion, my bad :D

case e: AttributeNotFoundError =>
logger.info(s"Metadata for $id was not found. Any associated layer data (if any) will require manual deletion")
case e: Exception =>
throw e

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can be omited

@moradology moradology changed the title Relax safeguards on s3 delete methods Relax safeguards on s3 delete methods (add clobbering writes?): WIP Mar 2, 2017
@pomadchin pomadchin added this to the 1.0.1 milestone Mar 2, 2017
@moradology
moradology force-pushed the feature/quiet-deleter branch from 02a84a5 to 528c385 Compare March 7, 2017 17:50

sourceLayerPath.delete
}
class FileLayerDeleter(val attributeStore: FileAttributeStore) extends LazyLogging with LayerDeleter[LayerId] {

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is worth taking a close look at - I don't think newing a trait inside an apply is a good idea for readability.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you mean?

@moradology moradology changed the title Relax safeguards on s3 delete methods (add clobbering writes?): WIP Relax safeguards on delete methods Mar 7, 2017
@moradology
moradology force-pushed the feature/quiet-deleter branch from aa1acc8 to 582a07e Compare March 7, 2017 19:41
import scala.collection.JavaConversions._

class CassandraLayerDeleter(val attributeStore: AttributeStore, instance: CassandraInstance) extends LayerDeleter[LayerId] {
class CassandraLayerDeleter(val attributeStore: CassandraAttributeStore, instance: CassandraInstance) extends LazyLogging with LayerDeleter[LayerId] {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here can be used a generic AttributeStore, only FileLayerDeleter has a restricted attributeStore type here (but it's a local for-tests backend).

@moradology
moradology force-pushed the feature/quiet-deleter branch from 582a07e to 4980646 Compare March 8, 2017 17:54
Prior to this change, delete methods threw in any case where the
reference to a layer happened to be faulty. This is based on metadata
which, though always present in case of tile data (and necessary for
its deletion), might not be present. Lacking this metadata, we should
log information about the failure and delete any _attribute data we can.
@lossyrob
lossyrob merged commit 147c6a2 into locationtech:master Mar 8, 2017
@lossyrob lossyrob modified the milestones: 1.1, 1.0.1 Mar 12, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants