Evaluate text app HPB sync options #2669

Initial Experiments

• Try plugging in the new collaboration extension into nextcloud text -> works if we drop user colors
• Check how easily we can use https://tiptap.dev/hocuspocus/ as a drop in replacement for our polling based sync backend
  • First implementation uses y.js steps but sends them over the existing sync backend and still uses client side logic to save the file

Drop in Replacement

#2847

Use the yjs based collaboration extension with the existing server API (+small modifications).
Turn the sync service provider experiment into a working solution.

• send and store steps encoded as base64
• combine multiple yjs changes into a single change before they are send to the server
• remove the DB lock when pushing changes to the server
• write tests for joining an existing editing session and collaborating
• write tests for opening an existing md file and collaborating on it
• test performance under load - maybe during contributor week.

Explore combining with other providers

During company week:

• See if using notify_push just for the message passing is an option that may improve the situation Using notify_push for syncing #1805
• Try combination of sync service provider with webrtc
• Try combination of sync service provide with hocuspocus
• Prioritize / pick combination to work on (first)

Combine with Hocuspocus

Use Hocuspocus to send steps and awareness messages between clients more efficiently.

• Allow configuring a hocuspocus url
• Define a mapping from a text file to a hocuspocus document
• Implement access control like tokens or JWT to validate that clients are allowed to connect to the web socket of a specific document.
• We need to clarify how this could be handled from a license/release perspective as hocus-pocus is currently in private beta, repos is private, npm package is public already

Awareness (user presence and authorship)

• Use yjs awareness instead of our current user session tracking
• explore implementing user colors based on yjs awareness
• plug in user cursor extension

Fully integrate

Combine the various providers seamlessly and make use of their strength.

• Serialize markdown and store it to the server from within hocuspocus
• Detect available providers in the client and use them accordingly
• Compress document history