Add persistent multi-session R Interactive windows - #1805

Add native VS Code Interactive windows whose R processes survive editor reloads, application exit, and disconnection. Users can run independent background R or headless arf sessions, attach to existing arf sessions in tmux, and switch between them while retaining their environments and executed history. The implementation uses VS Code's Interactive/notebook APIs, a bundled sess bridge, languageserver, and JGD; it does not require Jupyter.

Sessions and execution

• Separate agent policy and persistence from runtime integration through a Node-only SessionBackend contract. The shared sess backend composes plain-R and arf adapters, with private console/RPC and graphics helpers. Backend-specific preparation runs for both creation and restart; the independent agent bundle no longer shares the sess installation cache. Legacy config/protocol compatibility and current version requirements are preserved.
• Keep dispatch acknowledgement separate from execution completion, never retry ambiguous submissions, and distinguish IPC disconnection from confirmed process exit. Adopted-session disposal restores the original frontend without killing R; explicit Stop also works during startup. Cancel pending arf transport requests on disposal.
• Start fresh windows with empty input and history even when VS Code recycles native models, reject saved URI associations owned by another session, and consistently report observer execution errors from native input, cells and source commands.
• Supervise session agents independently of the extension host, with Linux auto preferring tmux when available and falling back to an independent detached process when tmux is absent or not executable; macOS uses detached processes. The actual supervisor appears in session details and the fallback is logged. Durable execution journals, reconnection, and restoration of open windows preserve session identity and prevent duplicate submissions.
• Execute cells serially within each session while allowing separate sessions to run concurrently. Support streamed output, readable R errors, input prompts, debugger continuation, interruption, queued-cell cancellation, and explicit control transfer between editor windows.
• Restart managed R/arf sessions in the same Interactive window, retaining prior code, output, drafts, and a restart boundary. Stop adds a session notice; disconnect leaves R running. Stop Selected and Stop All support bulk management with one confirmation and skip sessions controlled by another window. Restart honors the current supervision setting; missing explicitly selected tmux/systemd executables are rejected before stopping the live R process. Installed-supervisor launch failures retain their diagnostics and do not launch a second agent.
• Show live state, connection/control status, PID, working directory, and session age in the session tree, pickers, kernel controls, and tooltips. Validate saved endpoints so unavailable sessions do not become dead picker entries.
• Keep the provider picker compact: R and arf, each with its executable path. Missing arf offers configuration while plain R remains available. Executable/runtime checks happen before creation or before stopping a process for restart.
• Integrate with existing R execution commands. When no target is available, Run Selection offers a new Interactive window, an existing session, or a traditional R terminal. Source bindings and explicit terminal routing remain supported.

Plots, tables, widgets, and history

• Render base/grid/ggplot graphics through JGD as retained SVG, with a standard-graphics fallback. Preserve multi-panel titles and axes, raster interpolation/rotation/reflection, zero-count bar borders, and font sizing; keep incremental and resized plots associated with their originating cells.
• The standard-graphics fallback retains actual plot pages, replaces partial updates, ignores layout-only changes, and leaves explicitly opened file devices out of the transcript. Both backends use native R page hooks, including calls imported by stats/lattice, preserving three time-series pages in one expression and two pages for six model diagnostics in a 2×2 layout.
• Group multiple plots from one cell into one paged viewer with one toolbar. For example, eight plots with mfrow = c(2, 2) produce two four-panel pages. The selected page survives updates and reconnection; Open, Save, and Fit target that page. Save opens a native SVG/PNG format picker without resizing the cell. Standard graphics images support Open and Save without offering live device resizing.
• Provide lightweight inline tables with first/last navigation, arbitrary page jumps, 20/50/100-row pages, column sorting, typed filters, drag-to-reorder columns, and Reset. Browsing beyond the saved preview automatically retrieves live rows from the full object; sorting/filtering use the full dataset. Filters run on Apply, never on each keystroke. Reset locally restores the original preview and column order, clears queries/drafts, and works after stop. The expanded data viewer remains available. Match R's numeric precision, align headers/values, show R type tooltips, distinguish missing values from literal strings, and format nested JSON readably. Matrix/array columns retain complete composite cells with scalar sorting/filtering disabled, avoiding bogus rows from flattened indices.
• Preserve analytical reproducibility: opening a table never draws random numbers; retained data.table results keep their original values/schema after later edits by reference; := and set() stay quiet unless explicitly displayed. Respect options(warn) and suppressWarnings() during analysis. Cell snapshots retain at most 1,000 rows / 100,000 cells, subsetting before class printing or serialization. Large tables use a separate full-data handle without a deep copy: the saved row limit is not an inline browsing limit. The cell distinguishes Saved preview from Live data; exports preserve the original preview/scope. Live query caches expire after reference edits. New inline queries refresh column metadata and clear incompatible filters/sorting after schema changes; schema changes during ordinary navigation require Reset/reopen. Synthetic row-name representation differences do not clear valid queries.
• Avoid additional full-data allocations during inspection: matrix paging subsets rows before column extraction, row-name trimming is paged, sort/filter queries reuse original vectors rather than an identity-index copy, automatic workspace names are bounded, and large nested list/model cells and long strings have compact previews. Full-data filtering still uses complete strings. Inline navigation requests at most 100 rows and retains one current page per cached output (up to 32 outputs), without accumulating visited pages. The toolbar exposes Data viewer, Text, Filters and Reset; column reordering uses header dragging. Page/query/order choices survive output updates in that window but remain temporary; reloads and exports use the saved output.
• Add a per-output Text / Table switch and the r.interactive.tableView default setting. Text uses the class-specific R printout captured at execution time, including data.table type labels and R print options. Switching does not rerun R. Snapshots are bounded to 256 KiB, preserve Unicode and portable line endings, and fail gracefully when a custom printer errors. Explicit print(x) remains console output.
• Keep local table/text switching and plot pagination usable while R is busy, after stop/restart, and in saved notebooks. Retained plot export remains available while the agent is connected; live table queries and device resizing are disabled when their R process is unavailable.
• Support HTML widgets, local dependencies, explicit MIME output, and live viewers in sandboxed frames. Add durable history search, insert/copy/run-again/source navigation, plot browsing, and R/RNB/IPYNB/HTML export. RNB retains offline plot pagination; Jupyter and HTML exports include every plot page. Saved RNB output excludes transient connection URLs and credentials.
• Compress and deduplicate generated SVG/JSON assets, coalesce drawing updates, and reclaim superseded or unreferenced assets while protecting retained output and widget dependencies. The default asset quota is 4 GiB per session, measured in stored bytes; manual cleanup and quota changes are available without restarting R. Quota failures are reported once per execution rather than flooding the cell.

Workspace, language services, and R libraries

• Keep Workspace, source bindings, completion, hover, and parameter hints aligned with the owning session when switching tabs or windows. Discard stale workspace replies and preserve action ownership across dialogs. Live function hover and signatures work in both source documents and Interactive inputs.
• Serve virtual cells and inputs through languageserver without filesystem/cache errors. Suppress diagnostics on blank prompts and ignore stale lint replies after the input is cleared. Rebind their language servers when session ownership becomes known or changes, including native inputs created before their owner is registered; use that session's R, libraries, and working directory, and coalesce retained-cell rebinding into one restart per owner. Custom .lintr linter lists require languageserver #782: released 0.3.20 forces default linters for pathless documents. The extension does not patch R package namespaces.
• If bundled sess installation fails, try a compatible R-universe binary (or a published pure-R release) without compiling repository packages. Select macOS/Windows binaries through R, and Linux binaries only for the matching Ubuntu codename, architecture and R version. Check package version, the Interactive compatibility marker, exports and native routine signatures before accepting a private runtime. The currently published sess 3.0.1 is pure R and predates Interactive: ordinary terminals can use it now; compiler-free Interactive requires a new published build containing this bridge.
• Preserve the library paths and package installation destination established by ordinary R or renv startup. Load the private sess bridge explicitly without making its library the default installation target or adding it to renv snapshots. Project dependency versions take precedence, unrelated user packages remain isolated, and arf startup accepts profile/renv banners.

Connection, output, and cancellation robustness

• Deliver replayed history before live events received in the same socket read, so reconnecting during streaming output does not skip cells or output.

• Bound rich events by their encoded JSON byte size before native IPC transmission. Oversized HTML, tables, and escaped strings produce a retained truncation notice while R keeps running.

• Keep forked R workers off the parent console socket. Parallel output uses worker stdout/stderr instead of interleaving protocol frames or consuming parent input.

• Interrupt slow inspections, including queries whose request has timed out. Return an interrupted RPC response while preserving the R polling loop, so subsequent code and inspections still work.

Documentation

The R Interactive wiki page contains the experimental user guide, including setup, session controls, rich output, storage limits, and troubleshooting. The current architecture/backend contract, contributor test instructions, and analysis fixtures remain beside the source. The original plans and dated reports are preserved in the PR's history; report links below refer to that fixed revision.

Validation

The expanded public-code review added fourteen sourced R-manual examples and fixed five output problems: raster rotation/reflection/interpolation, phantom rows from matrix-column queries, missing pages through imported graphics calls, duplicate old plots after layout changes, and missing zero-count bar outlines. All 108 public/research executions passed across three configurations. Twelve new JGD plots were rendered and visually compared with ordinary-R PNGs; the runner now saves every retained page. The public-example report records sources, adaptations, reproductions and checks.

Connection/output/cancellation regressions also cover replay/live events sharing a socket read, oversized Unicode and JSON-escaped displays, concurrent mclapply output, and inspection cancellation both before and after timeout.

The installer was tested with the compiler deliberately disabled: it installed a compatible locally built native binary, handled Linux-style binary repositories without compiling, and rejected an incompatible same-version build without replacing an existing installation. A live R-universe test installed and loaded the public sess binary successfully for ordinary use and correctly rejected it for Interactive. renv and user-library isolation checks pass with both R and arf.

The large-table review records the copy audit and reproducible allocation benchmark. On a 2-million-row × 23-column data.table, the old copy alone allocated 351 MiB; the complete warmed preview now allocates 0.63 MiB in 7 ms. An exact-dimension 832,976,871 × 23 compact ALTREP data.frame previews in 52 ms and fetches its final page in 1 ms. These are local R-side measurements, not timings on the user’s remote server. The allocation regression was rerun successfully. New tests fetch the final 11 rows through the inline helper and actual notebook renderer message path, combine date/logical/text filters with descending sorting, recover from schema edits, preserve unfinished filter drafts, and verify Reset/offline/stale-response behavior.

The research review maps PR capabilities to coverage and records six findings fixed during analysis work. Reproducible workflows cover import/cleaning, joins/reshape, grouped summaries, hypothesis tests, linear/logistic/survival models, bootstrap reproducibility, diagnostic pages, faceted visualization, CSV/gzip/RDS and plot-file export, and 100,000-row previews. Native sessions retained independent models and data after reload; historical tables were paged/sorted/filtered after by-reference edits. Standard graphics was checked natively, and bulk stop retained all three disposable transcripts.

New supervisor regressions launch real plain R and arf with no tmux on PATH, verify that the agent leaves the editor process tree, terminate the launcher, and reconnect to the same R PID with retained objects. They also cover explicit-supervisor preflight, restart recovery, and launch failures without duplicate agents.

Native testing covers multiple sessions, real reload/application exit, workspace switching, restart/bulk stop, execution/input/language features, missing-arf recovery, and export. The latest pass verifies the two-page plot example, exact data.table printing, selected-page SVG save, default-view changes, and view-choice restoration after reload. Regressions also cover stopped controls, stale replies, save readiness, output lifecycle ordering, compressed-asset quotas, and Windows print-snapshot line endings.

The original ten public examples include R Graph Gallery plots, dplyr summaries, DT, and Plotly. Seven JGD plots were visually compared with independent R PNG renders; DT search/sort/paging and Plotly hover/zoom/reset were exercised through the actual asset service and iframe sandbox. See the public-example report and live-testing report for sources, reproducible fixtures, and coverage.

Backend-refactor validation used R 4.6.1, arf 0.5.3, and VS Code 1.110.0. The new tests also cover a backend with no sess library/bootstrap, dispatch/completion ordering, transport ambiguity, stale events, input/request ownership, request cancellation, early Stop, and the original arf process remaining usable after disposal. The production VSIX was rebuilt and its bundled agent, extension, sess handler and manifest verified against the workspace before pushing. Older R/OS combinations were not rerun for this refactor; their requirements are unchanged.

Requirements and limits

Requires VS Code 1.110+. The persistent Interactive runtime supports Linux and macOS and requires R, the documented R dependencies, and either compatible prebuilt sess/dependency packages or a package compiler; tmux and arf are optional. The agent uses VS Code's extension host runtime by default: Electron in Node mode on desktop, or VS Code Server's Node remotely. The optional experimental r.interactive.nodePath setting selects standalone Node.js 18+; no separate Node installation is required by default. macOS uses writable Application Support storage by default.

Persistence preserves a running R process through editor reload/exit; host logout policies may still terminate detached processes. It does not checkpoint the R heap across process termination or host reboot. Actual Remote SSH transport and server-specific systemd policies have not been exercised locally. Windows CI verifies the compatible extension/package paths; it does not imply Windows support for the native persistent runtime.

Existing agents retain their original runtime. Start a new session to use the updated bootstrap, graphics handling, storage capabilities, printed table snapshots, reproducible data inspection, corrected fallback plot pages, and bounded large-table previews, or restart a managed session after preserving needed R objects. Editor-side plot grouping, table alignment and inline controls also apply to retained output. Sessions already providing bounded snapshots and full-data handles can keep running after the extension update/reload; older output without a full handle cannot expand a truncated snapshot. Setup, lifecycle semantics, and other known limits are in the user guide; the implementation review records the workflow comparison and regression coverage.

Screenshots