Skip to content

Add Doc.toSnapshot - #508

Merged
alecgibson merged 12 commits into
share:masterfrom
curran:doc-snapshot-method
Oct 5, 2021
Merged

alecgibson merged 12 commits into
share:masterfrom
curran:doc-snapshot-method

Conversation

@curran

@curran curran commented Aug 22, 2021

Copy link
Copy Markdown
Contributor

Fix #500

@curran

curran commented Aug 22, 2021

Copy link
Copy Markdown
Contributor Author

CI is failing for what looks like a silly reason unrelated to the changes here.

image

Any ideas on how to turn CI green?

Comment thread lib/client/doc.js Outdated
Comment thread lib/client/doc.js Outdated
Comment thread test/client/doc.js Outdated
Comment thread lib/client/doc.js Outdated
@curran curran changed the title Add Doc.createSnapshot Add Doc.toSnapshot Aug 24, 2021
@curran

curran commented Aug 31, 2021

Copy link
Copy Markdown
Contributor Author

What is the suggested approach for doing the cloning? I see there are two existing implementations of clone in the codebase:

function clone(obj) {

function clone(obj) {
  return (obj === undefined) ? undefined : JSON.parse(JSON.stringify(obj));
}

This is not exported.

The same implementation is found in https://github.com/share/sharedb/blob/master/test/projections.js#L389

My instinct is to refactor clone into its own module, and import that into both of these existing places where it's used, and also import it into Doc where I can use it to clone for the snapshot generation.

Alternatively, I could copy-paste the implementation into Doc.js. That would make 3 copies of the same implementation.

Thoughts?

@curran

curran commented Aug 31, 2021

Copy link
Copy Markdown
Contributor Author

CI is failing due to something seemingly unrelated to these changes.

image

https://github.com/share/sharedb/pull/508/checks?check_run_id=3475145150

@curran

curran commented Sep 7, 2021

Copy link
Copy Markdown
Contributor Author

Actually, I'm not sold on the clone operation. I would favor calling out in the docs "Do not mutate the returned value", as cloning documents like this, every time a snapshot is created, is a huge waste of CPU resources.

@curran

curran commented Sep 7, 2021

Copy link
Copy Markdown
Contributor Author

Notes from call:

  • Viable option: add an optional flag to opt out of cloning step
  • We can put clone in the utils module

@curran

curran commented Sep 8, 2021

Copy link
Copy Markdown
Contributor Author

Requesting re-review @alecgibson @ericyhwang . Thanks!

CI is failing on ShareDB Mongo, not sure why (and not sure why this would not be the case in master).

Comment thread lib/client/doc.js Outdated
// to guard against mutations. You can opt out of cloning by passing `true`
// as the optional `doNotClone` argument.
//
// @param doNotClone (optional) If `true`, the document data is not cloned. This can be used

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure we should let people do this. I know cloning is a performance overhead, but the failure mode is bad if consumers accidentally mutate the snapshot (because they'll mutate doc.data, which shouldn't happen outside of OT).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main use case for this is to serialize the snapshot and send it over the network. I'm of the opinion that consumers should be allowed to opt out of cloning if they know for sure that their code will never mutate the data.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still personally think that the risks of weird side-effects massively outweigh the relatively small performance impact of cloning. Cloning is a perfectly normal, defensive thing to do. json0 itself uses clone in many places through its lifecycle.

(As a side-note, json0 is just using a JSON approach to cloning, although I believe the comment about it being fast is out-of-date).

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@curran I discussed with @ericyhwang and we think we shouldn't include this option for now. Let's keep the API footprint small and fool-proof where possible.

If you find that this does impact performance with some benchmarks, then we can consider adding an override later.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. That simplifies things. Will do!

Comment thread lib/util.js
Comment on lines +100 to +102
exports.clone = function(obj) {
return (obj === undefined) ? undefined : JSON.parse(JSON.stringify(obj));
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code has only hitherto been used in tests, and I wonder if we should use something more performant in production code?

If we're okay with adding a dependency, we've been using rfdc and it's worked well.

On the other hand, the performance gain is (relatively) small (~3x speed-up), so I don't mind just sticking with this.

@ericyhwang thoughts?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code has only hitherto been used in tests

It was also used in lib/db/memory.js.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The MemoryDB shouldn't be used in production, so this clone method still hasn't touched production code before now.

@alecgibson

Copy link
Copy Markdown
Collaborator

@curran we've dropped Node.js v10 support now. You should be able to rebase and greenify the build.

@curran

curran commented Sep 28, 2021

Copy link
Copy Markdown
Contributor Author

Addressed feedback and merged latest from master. Requesting re-review please @alecgibson @ericyhwang . Thanks!

@coveralls

coveralls commented Sep 28, 2021 •

Copy link
Copy Markdown

Coverage Status

Coverage increased (+0.004%) to 97.377% when pulling a104fcf on curran:doc-snapshot-method into 46c4aba on share:master.

@alecgibson alecgibson left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!

@alecgibson
alecgibson merged commit 1c5aa08 into share:master Oct 5, 2021
@curran

curran commented Oct 5, 2021

Copy link
Copy Markdown
Contributor Author

Hooray! Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Doc utility method for creating a snapshot

4 participants