Skip to content

DevFlow skills: prefer unified maui CLI over standalone tools + improve JSON/error guidance #197

Closed
Closed
DevFlow skills: prefer unified maui CLI over standalone tools + improve JSON/error guidance#197
#203 (+3)

Description

@rmarinho
rmarinho
opened on Apr 30, 2026

Summary

The maui-devflow-debug and maui-devflow-onboard agent skills (under plugins/dotnet-maui/skills/) currently steer AI agents toward a mix of clients to prepare and handle devices:

  • androidsdk.tool (android sdk|avd|device|jdk …)
  • appledev.tools (apple xcode|runtime|simulator …)
  • Raw adb … and xcrun simctl …
  • Plus lsof, pgrep, etc.

Most of those operations are already covered by the unified maui CLI (Microsoft.Maui.Cli). Mixing tools forces agents to install three global tools, learn three command surfaces, and parse three different output shapes. JSON / error guidance in the skills is also weak — agents are not told to use --json, parse error.code, or follow remediation.command hints that the CLI already emits via MauiToolException.

This issue tracks (a) retargeting the active DevFlow skills to the maui CLI everywhere it supports the operation today, (b) improving JSON and error-troubleshooting guidance inline, and (c) capturing the real maui CLI gaps as follow-up work.

Audit — maui CLI coverage matrix

Verified against src/Cli/Microsoft.Maui.Cli/Commands/.

Skill operation Old reference maui equivalent Status
List devices/emulators adb devices / xcrun simctl list devices booted / apple simulator list / android device list maui device list --platform android|apple ✅ replace
List AVDs android avd list maui android emulator list ✅ replace
Create / start / stop / delete AVD android avd … maui android emulator create|start|stop|delete ✅ replace
Android SDK install / list / info / accept-licenses android sdk … maui android sdk … (+ maui android install for guided setup) ✅ replace
JDK install / check android jdk … maui android jdk … ✅ replace
Xcode list apple xcode list maui apple xcode list ✅ replace
Simulator runtime list (n/a in skills today) maui apple runtime list ✅ added as a discovery hint
List sims apple simulator list / xcrun simctl list devices maui apple simulator list ✅ replace
Boot / shutdown / delete sim apple simulator boot|shutdown|delete / xcrun simctl … maui apple simulator start|stop|delete ✅ replace
Environment health check (n/a) maui doctor ✅ added
Connectivity diagnostic mixed maui devflow diagnose ✅ surfaced earlier
Create / erase simulator xcrun simctl create|erase ❌ not in maui ⚠️ kept raw fallback
Install / launch app on sim xcrun simctl install|launch ❌ not in maui ⚠️ kept raw fallback
Sim privacy / appearance / openurl / push / location / addmedia xcrun simctl … ❌ not in maui ⚠️ kept raw fallback
Sim screenshot (non-MAUI app) xcrun simctl io booted screenshot ❌ not in maui (use maui devflow ui screenshot for a running MAUI app) ⚠️ kept raw fallback
Android adb reverse 19223 (broker) adb reverse tcp:19223 tcp:19223 ❌ not in maui ⚠️ kept raw; flagged as gap
Android adb forward <agent port> adb forward tcp:<port> tcp:<port> ❌ not in maui ⚠️ kept raw; flagged as gap
APK install / uninstall / launch / stop adb install / am start / am force-stop ❌ not in maui ⚠️ kept raw
logcat / app logs (pre-agent) adb logcat maui devflow ui logs works once the agent is up; adb logcat still required pre-agent ⚠️ partial replace
Screencap / screenrecord (Android, pre-agent) adb shell screencap|screenrecord maui devflow ui screenshot once agent is up ⚠️ partial replace
Port-in-use diagnosis lsof -i :<port> ❌ not in maui ⚠️ kept raw
Build / run app dotnet build -t:Run / dotnet run (intentionally out of maui CLI scope) ✅ keep dotnet

What changed in the skills (this PR)

All edits are under plugins/dotnet-maui/skills/:

  • maui-devflow-debug/SKILL.md — INVOKES line rewritten to lead with the maui CLI; new "Reading machine-readable output" section covering --json, error code categories (E1xxx–E5xxx), remediation.type (AutoFixable / UserAction / Terminal), and --ci for non-interactive failure-fast use.
  • references/android.md — fully rewritten to be maui-first. Raw adb ops (port forwarding, install, logcat, screencap, push/pull/shell) grouped under a clearly labeled "Raw fallbacks not yet in maui CLI" section.
  • references/ios-and-mac.md — restructured around maui apple xcode|runtime|simulator …. Raw xcrun simctl ops (create/erase/install/launch/privacy/appearance/openurl/push/location/addmedia) grouped under the same labeled fallback section. Permission and dark-mode subsections explicitly flag the raw fallback.
  • references/connectivity.md — leads with maui devflow diagnose + maui device list; new "Reading errors from the CLI" section with the structured error envelope and code-prefix table.
  • references/troubleshooting.md — new top section "Reading machine-readable output and errors" with the full envelope JSON, code taxonomy, remediation types, and a worked example using E2106 (AndroidEmulatorNotFound). Existing items now reference maui commands and concrete codes (E2101, E2103, E2402).
  • references/setup.md — Microsoft.Maui.Cli is the only required global tool; androidsdk.tool and appledev.tools moved to an "Optional extras" subsection. Raw adb port-forwarding step labeled. maui doctor added to verification.
  • maui-devflow-onboard/SKILL.md — added maui doctor to the verify block, plus a one-line note: "Use --json and read error.code/remediation.command for programmatic recovery."
  • references/macos.md, linux.md, batch.md — verified clean (no changes needed).

Verification

rg -n '^\s*(android |apple |xcrun simctl |adb )' \
  plugins/dotnet-maui/skills/maui-devflow-debug \
  plugins/dotnet-maui/skills/maui-devflow-onboard

Every remaining match is inside a clearly labeled "Raw fallbacks not yet in maui CLI" section or is preceded by a # Not yet wrapped by 'maui' CLI comment.

JSON / error troubleshooting improvements

Skills now teach agents to:

  1. Pass --json to any command an agent will parse.
  2. Branch on error.code (categories from src/Cli/Microsoft.Maui.Cli/Errors/ErrorCodes.cs):
    • E1xxx — tool/internal
    • E2xxx — platform/SDK (E20xx JDK, E21xx Android, E22xx Apple, E23xx Windows, E24xx .NET)
    • E3xxx — user action
    • E4xxx — network
    • E5xxx — permission
  3. When error.remediation.type == AutoFixable, run error.remediation.command and retry.
  4. When error.remediation.type == UserAction, follow error.remediation.manualSteps.
  5. Use --ci for non-interactive failure-fast runs.

Worked example (E2106) is included in references/troubleshooting.md.

Follow-up backlog — real maui CLI gaps

These are out of scope for this PR but should drive future maui CLI work so the skills can drop more raw fallbacks:

  1. maui apple simulator create|erase|install|launch|screenshot|openurl|push|location|addmedia — today only list|start|stop|delete.
  2. maui apple simulator privacy grant|revoke|reset — for permission management.
  3. maui apple simulator appearance light|dark — for theme toggling in tests.
  4. maui android port reverse|forward|list|clear — wrap adb reverse/adb forward, with the broker port 19223 as a default and an --agent-port shortcut for DevFlow.
  5. maui android device install|uninstall|launch|stop — wrap adb install -r, pm uninstall, am start, am force-stop.
  6. maui android device logcat [--tag …] [--pid …] — wrap adb logcat with sensible MAUI defaults (MauiDevFlow, DOTNET).
  7. maui android device screenshot|screenrecord — wrap adb shell screencap/screenrecord.
  8. maui port check <port> — wrap lsof -i :<port> cross-platform (also useful for broker port conflicts).
  9. Structured JSON for errors — confirm and document that --json writes a stable { "error": { "code", "message", "remediation": { "type", "command", "manualSteps" }, "context", "nativeError" } } shape from MauiToolException. Add it to skills + CLI README.
  10. maui devflow diagnose --json — ensure the connectivity diagnostic emits a structured per-check result so agents can branch on it instead of grepping prose.

Out of scope

  • The legacy maui-ai-debugging skill — it already redirects new clients to maui-devflow-debug.
  • The devflow-automation skill — it's about [DevFlowAction], not device handling.
  • No new maui CLI commands are added in this PR; only skill markdown is changed.

References

  • Skills: plugins/dotnet-maui/skills/maui-devflow-debug/, plugins/dotnet-maui/skills/maui-devflow-onboard/
  • CLI command surface: src/Cli/Microsoft.Maui.Cli/Commands/
  • Error taxonomy: src/Cli/Microsoft.Maui.Cli/Errors/ErrorCodes.cs
  • Error envelope: src/Cli/Microsoft.Maui.Cli/Errors/MauiToolException.cs

Activity

  1. rmarinho commented on Apr 30, 2026

    @rmarinho
    rmarinho
    on Apr 30, 2026
    MemberAuthor

    Implementation plan + upstream issues filed

    Following up on this audit, I traced the gap list against the actual upstream repos to see what was already there, what was already filed, and what we still needed to file. Three findings of note:

    1. Xamarin.MacDev.SimulatorService already ships Create and Erase — they're just not exposed in maui apple simulator. So that pair is CLI-only work, no upstream blocker.
    2. Xamarin.Android.Tools.AdbRunner reverse-port API is merged upstream (dotnet/android-tools#305), but not in our currently-consumed 1.0.143-preview.10 (verified by reflection on the cached package — only ListDevices/WaitForDevice/StopEmulator/GetShellProperty/RunShellCommand are public there). So maui android port reverse is gated on a package bump, not on new upstream API work.
    3. AdbRunner.RunShellCommandAsync is already public in preview.10 — we don't need a separate "generic shell" upstream issue.

    Upstream issues filed today

    # Repo Title
    macios-devtools#174 macios SimulatorService app lifecycle (install/uninstall/launch/terminate/get_app_container)
    macios-devtools#175 macios SimulatorService extras (privacy/appearance/status_bar/openurl/push/location/addmedia/screen capture)
    android-tools#347 android AdbRunner app management (install/uninstall/launch/force-stop/list-packages)
    android-tools#348 android AdbRunner structured logcat streaming
    android-tools#349 android AdbRunner screencap and screenrecord
    android-tools#350 android AdbRunner forward port management (mirror of dotnet/android-tools#305 reverse)

    Already-filed upstream issues we'll consume

    android-tools: dotnet/android#12071 (parent AdbRunner), dotnet/android#12069 (AvdManager), dotnet/android#12070 (EmulatorRunner), dotnet/android#12074 (push/pull), dotnet/android-tools#321/#322 (device profiles + skins), dotnet/android#12073/#327/#323 (track-devices), dotnet/android#12083/#340 (emulator launch result), dotnet/android#12080/#331/#332 (SDK channel/version pinning/--all-versions), dotnet/android#12075/#272 (license), dotnet/android#12076 (GPU accel), dotnet/android#12077 (sysimg compat), dotnet/android#12078 (getprop fallback).

    macios-devtools: dotnet/android-tools#149/#150/#151/#152/#153/#154 (closed — already shipped APIs we're starting to consume).

    Phases (full plan saved internally)

    1. Now (no upstream blocker) — maui apple simulator create|erase, maui port check <port>, verify maui devflow diagnose --json is structured per-check. Each lands as its own focused PR.
    2. Filed upstream, awaiting — the 6 issues above, plus the existing android-tools roadmap.
    3. Track + consume — as upstream lands, bump XamarinAndroidToolsAndroidSdkVersion / XamarinAppleToolsMaciOSVersion in eng/Versions.props (via Maestro/DARC) and expose the corresponding maui subcommand. Refactor Providers/Android/Adb.cs to wrap upstream AdbRunner. Refactor maui devflow ui permission to consume SimulatorService.PrivacyService once macios#175 part 1 ships.
    4. Skill cleanup — per shipped command, remove the matching "Raw fallbacks not yet in maui CLI" entry in references/android.md / references/ios-and-mac.md.
    5. JSON / error hardening — document the ErrorResult envelope in src/Cli/README.md, add maui errors list --json to enumerate codes, ensure diagnose --json emits structured per-check shape. (The skill-side documentation work landed in Add reference to System.Security.Cryptography.Xml android-tools#198.)

    This issue stays open as the tracker until all six upstream issues are merged and exposed in the CLI.

  2. rmarinho commented on Apr 30, 2026

    @rmarinho
    rmarinho
    on Apr 30, 2026
    MemberAuthor

    Correction on the upstream filing list: closed android-tools#350 (forward port) as a duplicate. The AdbPortSpec / AdbPortRule types from PR dotnet/android-tools#305 already cover the design surface, so forward is a small 4-method follow-up rather than a separate design issue. Moved the request to a comment on dotnet/android-tools#305 so the design context stays together.

  3. rmarinho commented on Apr 30, 2026

    @rmarinho
    rmarinho
    on Apr 30, 2026
    MemberAuthor

    Filed forward-port follow-up upstream as a draft PR: dotnet/android-tools#351 (instead of a separate issue, given the design context already lives on PR dotnet/android-tools#305).

  4. rmarinho
    added 3 commits that reference this issue on Apr 30, 2026

    Expose maui apple simulator create|erase

    2cda2a6

    Expose maui apple simulator create|erase

    be8ea33

    docs: document --json error envelope contract in CLI README

    ebb4253
  5. jfversluis
    closed this as completedin #203on May 4, 2026
  6. jfversluis
    added a commit that references this issue on May 4, 2026

    Expose maui apple simulator create|erase (#203)

    5b260c4
  7. rmarinho
    added 2 commits that reference this issue on May 4, 2026

    feat: add maui port check command

    7c50a8e

    docs: document --json error envelope contract in CLI README

    e57c879

  8. rmarinho commented on May 5, 2026

    @rmarinho
    rmarinho
    on May 5, 2026
    MemberAuthor

    Upstream update — macios-devtools#176 merged ✅

    macios-devtools#176 — Add simulator app lifecycle APIs — was merged at 2026-05-05 11:18 UTC (commit 87eb6ab). This implements the design from macios-devtools#174.

    SimulatorService now ships:

    • Install(udid, appPath)
    • Uninstall(udid, bundleId)
    • Launch(udid, bundleId, args?, env?) (returns PID)
    • Terminate(udid, bundleId)
    • GetAppContainer(udid, bundleId, container?)

    What unblocks here

    Once the next XamarinAppleToolsMaciOSVersion bump (via Maestro/DARC) reaches eng/Versions.props, we can scope a single follow-up PR to expose:

    • maui apple simulator install <udid> <app>
    • maui apple simulator uninstall <udid> <bundleId>
    • maui apple simulator launch <udid> <bundleId> (emit PID in --json)
    • maui apple simulator stop-app <udid> <bundleId> (terminate; distinct from existing simulator stop which currently means shutdown)

    Naming caveat: today maui apple simulator stop shuts down the device. If we surface Terminate, we should pick a non-overloaded verb (stop-app, kill, or rename stop → shutdown and free up stop for the app — would be a small breaking change worth flagging).

    Still waiting

    macios-devtools#177 (privacy/appearance/status_bar/openurl/push/location/addmedia/screenshot/recordvideo, implementing #175) is still draft + review-required. That one also unblocks the maui devflow ui permission refactor away from shelling out to xcrun simctl privacy.

  9. rmarinho
    added 3 commits that reference this issue on May 5, 2026

    [AdbRunner] Add ADB forward port management (follow-up to dotnet#305)

    40c20d0

    feat: add maui port check command

    7ffad57

    feat: add maui port check command

    ad08afc
  10. jfversluis
    added a commit that references this issue on May 12, 2026

    docs: document --json error envelope contract in CLI README (#199)

    19387c2
  11. Copilot
    added a commit that references this issue on May 12, 2026

    docs: document --json error envelope contract in CLI README (#199)

    0274823
  12. rmarinho
    reopened this on May 13, 2026

  13. rmarinho commented on May 13, 2026

    @rmarinho
    rmarinho
    on May 13, 2026
    MemberAuthor

    Status update — 2026-05-13

    PR dotnet/android-tools#198 status

    PR dotnet/android-tools#198 is being rebased and updated to address all 4 code review findings:

    Finding Severity Fix
    Error envelope shape ("error": {} wrapper doesn't match CLI output) 🔴 Critical Remove wrapper, use flat { "code": ... }
    Remediation type casing (AutoFixable → autofixable) 🔴 Critical Lowercase all values
    Property name (manualSteps → manual_steps) 🟡 Moderate Match [JsonPropertyName]
    Nonexistent commands (sdk info/jdk info) 🟡 Moderate Replace with sdk check/jdk check

    Cross-repo upstream tracker

    dotnet/macios-devtools

    PR Title Status Gaps covered
    #176 Simulator app lifecycle (install/launch/terminate) ✅ Merged Gap 1 partial
    #177 Simulator extras (privacy, appearance, statusbar, openurl, push, location, addmedia, screencap) 📝 Draft, CI all green, mergeable Gaps 1-3
    #179 Fix ArgumentNullException in AppleSdkSettings ✅ Merged —

    Open issue: #175 — tracking issue for simulator extras

    dotnet/android-tools

    PR Title Status Gaps covered
    #351 ADB forward port management Open, mergeable, ready-to-review Gap 4 (adb reverse/forward)
    #361 Signal-isolated emulator launch (Ctrl+C fix) Open, mergeable, ready-to-review Bug fix (#360)
    #340 EmulatorLaunchResult (serial/ports/log) ❌ Changes requested, conflicts Emulator metadata
    #327 ADB device tracking Open, conflicts Device monitoring
    #326 AVD skin enumeration Open, conflicts AVD creation UX

    Open issues (no PR yet):

    • #347 — ADB app management (install/uninstall/launch/stop) → Gap 5
    • #348 — ADB logcat streaming → Gap 6
    • #349 — ADB screencap/screenrecord → Gap 7

    Gap closure progress (from original follow-up backlog)

    # Gap Upstream status Can drop raw fallback?
    1 simctl create/erase/install/launch macios-devtools dotnet/android-tools#176 ✅ + dotnet/android-tools#177 (draft) Once dotnet/android-tools#177 merges + consumed
    2 simctl privacy grant/revoke/reset macios-devtools dotnet/android-tools#177 (draft, CI green) Once dotnet/android-tools#177 merges + consumed
    3 simctl appearance light/dark macios-devtools dotnet/android-tools#177 (draft, CI green) Once dotnet/android-tools#177 merges + consumed
    4 adb reverse/forward android-tools dotnet/android-tools#351 (mergeable) Once dotnet/android-tools#351 merges + consumed
    5 adb install/uninstall/launch/stop android-tools dotnet/android#12084 (issue only) ❌ Needs PR
    6 adb logcat android-tools dotnet/android#12085 (issue only) ❌ Needs PR
    7 adb screencap/screenrecord android-tools dotnet/android#12086 (issue only) ❌ Needs PR
    8 port check <port> No upstream work ❌ Needs design
    9 Structured JSON error envelope docs PR dotnet/android-tools#198 (in progress) ✅ Being fixed now
    10 maui devflow diagnose --json No upstream work ❌ Needs design

    Summary: 5/10 gaps have active upstream PRs (2 already merged). The critical path is merging android-tools dotnet/android-tools#351 and macios-devtools dotnet/android-tools#177 then consuming them in maui-labs.

  14. rmarinho
    added a commit that references this issue on May 13, 2026

    Skills: retarget DevFlow skills to the unified `maui` CLI

    3c8e018
  15. rmarinho
    added 2 commits that reference this issue on Jun 2, 2026

    feat: add maui port check command

    fb3c4a8

    Skills: retarget DevFlow skills to the unified `maui` CLI

    84ab130
  16. jonathanpeppers
    added a commit that references this issue on Jun 2, 2026

    [AdbRunner] Add ADB forward port management (#351)

    5165523
  17. jfversluis
    added a commit that references this issue on Jun 16, 2026

    feat: add maui port check command (#200)

    4654df8
  18. jfversluis
    closed this as completedin #200on Jun 16, 2026
  19. rmarinho
    added a commit that references this issue on Jul 7, 2026

    Refactor AndroidDevFlowPortForwarder to use AdbRunner

    85d96b5
  20. jfversluis
    added a commit that references this issue on Jul 8, 2026

    Refactor AndroidDevFlowPortForwarder to use AdbRunner (#363)

    e75e2a4
  21. jonathanpeppers
    added a commit that references this issue on Jul 13, 2026

    [AdbRunner] Add ADB forward port management (#351)

    61deb75
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions