Skip to content

feat(muya): restore clipboard image paste + imageAction routing + copyAsRich (PG5/PG6/PG9) - #4411

Merged
Jocs merged 4 commits into
developfrom
feat/muya-clipboard-parity-pg5-6-9
Jun 8, 2026
Merged

Jocs merged 4 commits into
developfrom
feat/muya-clipboard-parity-pg5-6-9

Conversation

@Jocs

@Jocs Jocs commented Jun 8, 2026

Copy link
Copy Markdown
Member

What

Engine-only follow-up to #4406 (desktop migrated packages/muyajs →
@muyajs/core). Closes three clipboard parity gaps from the #4407 scoreboard,
all in packages/muya/src/clipboard:

  • PG5 — binary/bitmap clipboard image paste (screenshot, browser "Copy
    Image") was lost. The new engine never read clipboardData.files / ran a
    FileReader, so a bitmap-only clipboard inserted nothing.
  • PG6 — a pasted image FILE bypassed imageAction: the raw absolute path
    was written verbatim, ignoring the copy-to-assets / upload / relative
    insert preference and leaving documents non-portable.
  • PG9 / PG-COPYRICH — "Copy as Rich Text" pasted HTML source as literal
    text. Desktop maps copyAsRich → copyAsHtml, whose branch blanks
    text/html and drops markup into text/plain. There was no copyAsRich
    path that puts rendered HTML in text/html.

How

  • PG9: new public Muya.copyAsRich() + Clipboard.copyAsRich() backed by
    a 'copyAsRich' branch in copyHandler that mirrors the normal branch
    (text/html = renderedHtml, text/plain = text).
  • PG6/PG5: on paste, route both a resolved clipboard file path and an
    in-memory bitmap through options.imageAction({ src, alt, title }) and
    insert the returned (persisted) src. Added IMuyaOptions.imageAction /
    IImageActionState (ported from legacy @muyajs), plus
    getClipboardImageFile + readFileAsDataURL helpers in utils/paste.ts.
    The image File is snapshotted synchronously before the first await so the
    detached-DataTransfer hazard is avoided. readFileAsDataURL prefers the
    native FileReader (legacy parity + chrome70 target) and falls back to
    Blob.arrayBuffer() + btoa.

Tests

Flips the failing scoreboard specs from it.fails → it (now passing):

  • packages/muya/src/clipboard/__tests__/parityImagePaste.spec.ts — PG5 (×1),
    PG6 (×2).
  • packages/muya/src/clipboard/__tests__/parityCopyAsRich.spec.ts — PG9 (×2).

Scoreboard PG5/PG6/PG9 marked green (12/15 remaining); PARITY_QA.md § PG5
updated — the engine half is implemented, only the OS-clipboard delivery
(real bitmap, macOS screencapture) stays manual.

Gates: lint (0 errors), lint:types, check-circular, test
(517 passed / 15 expected-fail), test:spec (1347 passed) all green.
(lint:css has one pre-existing no-descending-specificity error in
blockSyntax.css on develop, untouched here.)

Wave 2 (desktop, not in this PR)

Point COPY_PASTE_METHOD_MAP.copyAsRich in editor.vue at the new
muya.copyAsRich() instead of copyAsHtml. The desktop imageAction
adapter (muyaImageAction) already accepts { src, alt, title }, so PG5/PG6
need no desktop change beyond confirming clipboardFilePath is wired.

Refs #4406, #4407.

🤖 Generated with Claude Code

Jocs and others added 3 commits June 9, 2026 01:28
Expose a public Muya.copyAsRich() backed by a new 'copyAsRich' copyType
branch in the clipboard copyHandler. Unlike copyAsHtml (which blanks
text/html and drops markup into text/plain as literal source), copyAsRich
mirrors the 'normal' branch: rendered HTML in the text/html slot so a
rich-text target (Word, email, contenteditable) renders formatting, with
the markdown source in text/plain.

Restores the legacy @muyajs 'Copy as Rich Text' behaviour (PG-COPYRICH).
The desktop COPY_PASTE_METHOD_MAP.copyAsRich remap is wave 2.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
…aste (PG5/PG6)

On paste, both the resolved clipboard FILE path (PG06) and an in-memory
bitmap (PG05, screenshots / browser 'Copy Image') now flow through the
embedder's options.imageAction so the insert preference (copy-to-assets /
upload / keep-path) applies and a portable src is written — instead of
linking the raw on-disk path verbatim (PG06) or inserting nothing (PG05).

- types.ts: add IMuyaOptions.imageAction({ src, alt, title }) => Promise<string>
  and the IImageActionState shape, ported from legacy @muyajs.
- utils/paste.ts: add getClipboardImageFile (reads clipboardData.files /
  items for an image File) and readFileAsDataURL (FileReader.readAsDataURL,
  falling back to Blob.arrayBuffer + btoa for the chrome70 target / Node tests).
- clipboard: snapshot the image File synchronously before the first await,
  add tryPasteImage (file path then binary) + insertImageSrc (imageAction
  routing); split the raw markdown splice out as insertImageText.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
The clipboard image-paste + copyAsRich fixes land the engine behaviour, so
convert the parity specs from it.fails to it:
- parityImagePaste.spec.ts: PG5 (binary paste -> imageAction) + PG6 (x2,
  resolved path -> imageAction).
- parityCopyAsRich.spec.ts: PG9 (x2, copyAsRich sets text/html=html and
  text/plain=text).

Mark PG5/PG6/PG9 green on PARITY_SCOREBOARD.md (gaps remaining 12/15) and
update PARITY_QA.md § PG5: the engine half is implemented; only the
OS-clipboard delivery (real bitmap, macOS screencapture) stays manual.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
Copilot AI review requested due to automatic review settings June 8, 2026 17:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR closes three engine-side clipboard parity gaps in @muyajs/core by restoring image paste support (both file-path and bitmap clipboard cases), routing pasted images through the embedder’s imageAction, and adding a true “Copy as Rich Text” path that writes rendered HTML to text/html.

Changes:

  • Add bitmap clipboard image extraction + File→data: URL decoding utilities, and use them during paste.
  • Route both resolved clipboard image paths and in-memory bitmap images through options.imageAction before insertion.
  • Introduce copyAsRich end-to-end (Muya.copyAsRich() + Clipboard.copyAsRich() + copyHandler branch) and flip parity specs from it.fails to passing it.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
packages/muya/src/utils/paste.ts Adds helpers to snapshot an image File from DataTransfer and read it as a data: URL for bitmap clipboard paste.
packages/muya/src/types.ts Introduces IMuyaOptions.imageAction and IImageActionState to formalize image persistence/routing.
packages/muya/src/muya.ts Exposes copyAsRich() on the public Muya API.
packages/muya/src/clipboard/index.ts Implements copyAsRich handling and restores image paste via clipboardFilePath + bitmap File, routing through imageAction.
packages/muya/src/clipboard/tests/parityImagePaste.spec.ts Flips PG5/PG6 parity tests to passing.
packages/muya/src/clipboard/tests/parityCopyAsRich.spec.ts Flips PG9 parity tests to passing.
packages/desktop/test/PARITY_SCOREBOARD.md Marks PG5/PG6/PG9 engine-side as fixed and updates remaining gap count.
packages/desktop/test/PARITY_QA.md Updates PG5 manual QA notes to reflect engine-side completion.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +214 to +224
return file
.arrayBuffer()
.then((buffer) => {
const bytes = new Uint8Array(buffer);
let binary = '';
for (let i = 0; i < bytes.length; i++)
binary += String.fromCharCode(bytes[i]);
const base64 = btoa(binary);
return `data:${file.type};base64,${base64}`;
})
.catch(() => '');

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 942c441. The fallback now guards typeof file.arrayBuffer !== 'function' || typeof btoa !== 'function' and resolves '' instead of throwing out of the Promise<string>, and base64-encodes in 0x8000-byte chunks (bufferToDataURL) so large blobs don't build one huge per-byte string. I kept btoa rather than a Node Buffer path on purpose: this is a browser-targeted engine package, so pulling in the Node-only Buffer global would be off-side for lint/build, and btoa works identically under Node 20 and the browser.

Comment on lines +45 to +46
| **PG5** | major | binary/bitmap clipboard image paste lost (screenshot, browser "Copy Image") | `packages/muya/src/clipboard/__tests__/parityImagePaste.spec.ts` (`PG5:`) · `packages/desktop/test/PARITY_QA.md` § PG5 | `it.fails` + manual-QA | ✅ engine fixed (OS-clipboard manual-QA remains) |
| **PG6** | major | pasted image FILE bypasses `imageAction` (copy-to-assets / upload preference ignored) | `packages/muya/src/clipboard/__tests__/parityImagePaste.spec.ts` (`PG6:` ×2) | `it.fails` | ✅ fixed |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 942c441 — the PG5/PG6 Mechanism column now reads passing it (+ manual-QA for PG5).

| **PG7** | major | export loads core CSS from CDN instead of inlining it (unstyled offline) | `packages/muya/src/state/__tests__/parityExportHtml.spec.ts` (`PG7:` ×2) | `it.fails` | ❌ xfail |
| **PG8** | major | exported headings carry no `id` (dead TOC / `[TOC]` anchors) | `packages/muya/src/state/__tests__/parityExportHtml.spec.ts` (`PG8:` ×2) | `it.fails` | ❌ xfail |
| **PG9** | major | "Copy as Rich Text" pastes HTML *source* not rich text (no `copyAsRich` path) | `packages/muya/src/clipboard/__tests__/parityCopyAsRich.spec.ts` (`PG9:` ×2) | `it.fails` | ❌ xfail |
| **PG9** | major | "Copy as Rich Text" pastes HTML *source* not rich text (no `copyAsRich` path) | `packages/muya/src/clipboard/__tests__/parityCopyAsRich.spec.ts` (`PG9:` ×2) | `it.fails` | ✅ engine fixed (desktop `copyAsRich` map = wave 2) |

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 942c441 — the PG9 Mechanism column now reads passing it.

…m (review)

Address Copilot review on PR #4411:
- readFileAsDataURL: guard the non-FileReader fallback for missing
  arrayBuffer/btoa (resolve '' instead of throwing out of Promise<string>)
  and base64-encode the bytes in 0x8000 chunks so a large blob avoids a huge
  per-byte intermediate string.
- PARITY_SCOREBOARD: update the Mechanism column for PG5/PG6/PG9 from
  it.fails to 'passing it' now that those parity specs are flipped.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
@Jocs
Jocs merged commit e61cf0e into develop Jun 8, 2026
21 checks passed
@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

Build artifacts for PR #4411:

Run: https://github.com/marktext/marktext/actions/runs/27155634215

Artifact Size Link
marktext-macos-x64 256.7 MB Download
marktext-windows-arm64 256.2 MB Download
marktext-linux 556.3 MB Download
marktext-macos-arm64 246.5 MB Download
marktext-windows-x64 257.5 MB Download

Jocs added a commit that referenced this pull request Jun 8, 2026
… heading-link, source cursor, saved indicator) (#4415)

* fix(desktop): map copyAsRich to the engine copyAsRich method (PG9)

The legacy "Copy as Rich Text" command was remapped to copyAsHtml, which
blanks text/html and puts the HTML source into text/plain, so pasting into
Word/email yielded raw HTML markup as literal text. #4411 added a real
Muya.copyAsRich() that writes rendered HTML to text/html and plain text to
text/plain; point COPY_PASTE_METHOD_MAP.copyAsRich at it.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* feat(desktop): wire heading-copy-link to copyGithubSlug (PG11)

#4414 made the engine attach a hover-to-copy affordance to every heading and
emit heading-copy-link { key } (key == the heading's stable slug) on click.
Re-subscribe in editor.vue and forward the key to editorStore.copyGithubSlug,
which copies `#<githubSlug>` to the clipboard — restoring the heading-anchor
copy affordance that was a documented gap after the @muyajs/core migration.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(desktop): match exported-TOC anchors to engine heading ids (PG8)

editor.vue now exports via @muyajs/core (#4406), and #4412 injects
github-compatible heading ids onto exported headings (deduped in document
order with a `-N` suffix). getHtmlToc still slugged via the legacy muyajs
Slugger, so `href="#slug"` targets no longer matched the injected ids and TOC
/ [TOC] links were dead. Swap to @muyajs/core's generateGithubSlug and
replicate the engine's whole-document `-N` dedup so the anchors resolve.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(desktop): consume selection-change affiliation for menu state (PG1)

adaptSelectionChange hardcoded affiliation:[] and copied changes.type
('Caret'/'Range') into start.type/end.type, so createApplicationMenuState's
`start.type === 'span'` and functionType guards never fired — the native
Paragraph-menu check marks, loose/task-list toggles, table/code-fence
detection, and Format-disable-in-code all went dead after the @muyajs/core
migration.

#4410 added an `affiliation` chain (outermost-first) plus per-endpoint
`anchorBlockInfo`/`focusBlockInfo` (`type: 'span'` + `functionType`) to the
selection-change payload. Map them onto the legacy shape:
  - start/end `.type` and `.block.functionType` from the leaf block info,
  - affiliation passed through, with a derived `functionType` surfaced on
    `pre`/`figure` containers so table / code-fence detection lights up.

Also fix the consumer's loose-list read: the engine affiliation entry carries
`isLooseListItem` on the list block directly, not via a `children` chain.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(desktop): restore saved indicator on undo-to-disk (PG15)

makeSyntheticHistory minted an ever-incrementing editSeq id on every
json-change (including undo/redo), so after edit-then-undo-to-saved the id
never matched lastSavedHistoryId and the tab stayed marked dirty even when its
content matched disk.

Derive the synthetic id from the engine undo-stack DEPTH instead — a stable
position marker that returns to its saved value when an edit is undone back to
the baseline. Seed lastSavedHistoryId to 0 (the engine's post-setContent
baseline depth, since setContent clears history) so a freshly-loaded,
never-saved document clears its dirty indicator when undone back to disk
content, mirroring the legacy history-index behaviour.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* feat(muya): add setCursorByOffset for source-mode cursor restore (PG2)

The source-code -> WYSIWYG handoff carries only a CodeMirror {line, ch} index
cursor; @muyajs/core had no index-offset -> block-path conversion, so the
WYSIWYG caret was lost. Add Muya#setCursorByOffset, reproducing the legacy
muyajs approach: inject sentinel strings into the current markdown at the
line/ch offsets, rebuild the tree (sentinels embed as literal text), find the
content blocks they landed in, then rebuild the clean document and set the
cursor by the resolved block paths + offsets. Both setContent calls run
synchronously so no intermediate paint occurs, and the method is a no-op for
stale/unresolvable cursors.

Engine helper only — desktop wiring lands in a separate commit.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix(desktop): restore WYSIWYG caret after source-mode edit (PG2)

handleFileChange dropped the saved muyaIndexCursor on the source-code ->
WYSIWYG handoff, so the caret was lost. Consume the new engine
Muya#setCursorByOffset: when the tab has no key-based cursor but carries a
CodeMirror {line, ch} index cursor, map it onto a block-key cursor so the
caret lands where the source-mode cursor was. Restore the per-tab engine
history afterwards (setCursorByOffset re-runs setContent internally, which
clears history).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* test(desktop): flip parity e2e for PG1/PG2/PG15, defer PG14

Remove the `test.fail()` markers on the PG1 (Paragraph-menu check mark), PG2
(source-mode caret restore), and PG15 (saved indicator on undo-to-disk) parity
e2e tests now that the desktop wiring lands — all three genuinely pass.

PG14 (first undo after source mode reverts the bulk edit in one step) stays
`test.fail()`: recording the source-mode change as a single undo boundary needs
a general whole-document json1 diff through Editor.updateContents' pick/drop
walker, which only handles specific op shapes and would risk corrupting the
document. Deferred with an explanatory note here and in handleFileChange.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* docs(test): reconcile parity scoreboard after wave-2 (14/15 fixed)

Update the Status column to the true post-merge state: all seven Wave-1 engine
PRs (#4408-#4414) plus the Wave-2 desktop wiring close PG1-PG13 and PG15. Fix
the "Gaps remaining" count (11 -> 1), add the PG2 setCursorByOffset engine spec,
and document why PG14 is accept-deferred (single-undo-boundary across the
source-mode handoff needs a whole-document json1 diff the op walker can't safely
apply).

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

* fix: address Copilot review on parity wave-2

- editor.vue adaptSelectionChange: restore start/end block.text from the live
  anchorBlock/focusBlock so SELECTION_CHANGE can still slice the selected text
  (search prefill); the previous {functionType}-only block dropped it.
- editor.vue isIndexCursor: validate both line AND ch are numbers (factored out
  isIndexPosition) so a missing ch no longer silently clamps to column 0.
- muya setCursorByOffset: snapshot getHistory() and restore it after the
  internal setContent rebuild so the public API is caret-only and does not clear
  the undo stack; document the behaviour and cover it with a test.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <[email protected]>
@Jocs
Jocs deleted the feat/muya-clipboard-parity-pg5-6-9 branch June 10, 2026 07:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants