[AI] Contextual skill loading — progressive disclosure for agent abilities #37

Goal

Implement a skill-based progressive disclosure system for the ReAct agent. Instead of sending all tool descriptions to the LLM on every request, the agent starts with a lightweight skill index and loads full instructions + tools on demand when it detects a relevant task.

This is critical for our 4096-token context window (Qwen 3 1.7B) — as the ability count grows beyond the current 13, the system prompt will consume an unsustainable share of available tokens.

Why: The current architecture eagerly loads all abilities into every LLM request (react-agent.js:787-790). The system prompt grows linearly with tool count. Progressive disclosure solves this — the agent sees a compact index of what it can do, and only loads the full details when it needs them. This is the same pattern used by Claude Code (deferred tools + skill loading), MCP tool discovery, and the laravel-ai-sdk-skills package.

How It Works

The Pattern (Progressive Disclosure)

The core idea: don't tell the agent everything upfront — let it discover and load capabilities on demand.

This pattern is well-established in AI tooling:

• Claude Code uses deferred tools (agent sees tool names only, fetches full schemas on demand) and skills (markdown files with instructions loaded contextually)
• MCP separates tool discovery (list available tools) from tool invocation (call a specific tool)
• Laravel AI SDK Skills splits capabilities into markdown files with YAML frontmatter; agent gets lite tags and loads full content via a skill tool call

For our small local model with a 4096-token budget, this matters even more than for cloud LLMs.

1. Lite mode (default): Agent sees only skill names + one-line descriptions as compact tags:

   <skill name="site-health" description="Check plugins, themes, updates, and disk usage" />
   <skill name="content-management" description="List, search, and manage posts and comments" />
   <skill name="security" description="Security scans, error logs, and user auditing" />
   

2. On-demand loading: When the agent detects a task matches a skill, it calls a load_skill tool to get the full instructions and tool definitions for that skill only.

3. Skill definition files: Each skill is a Markdown file with YAML frontmatter:

   ---
   name: site-health
   description: Check plugins, themes, updates, and disk usage
   abilities:
     - plugin-list
     - theme-list
     - update-check
     - disk-usage
   ---
   
   # Site Health
   You are checking the health of a WordPress site. Start with an overview...

New Tools for the Agent

• list_skills — returns the lite index of all available skills
• load_skill — loads full instructions + tool definitions for a specific skill
• skill_read (stretch) — read reference files bundled with a skill (e.g., checklists, templates)

Requirements

• Skills are defined as Markdown files (with YAML frontmatter) in a skills directory
• Skill definitions group existing abilities by domain and include contextual instructions
• Agent system prompt includes only skill tags in lite mode, not full tool definitions
• Agent can load a skill mid-conversation via a load_skill tool call
• Loaded skill tools become available for subsequent ReAct iterations
• Support both "lite" (default) and "full" discovery modes
• Skills should be extensible by third-party plugins (like abilities are today via wp.agenticAdmin.registerAbility())

Key Files

Reference Pattern

The progressive disclosure pattern for AI agents follows this general flow:

Agent Init → Register all abilities internally
           → Build lite skill index (name + description only)
           → System prompt includes skill tags, NOT full tool list

User Request → Agent sees skill tags
             → Recognizes relevant skill (e.g., "list plugins" → site-health)
             → Calls load_skill("site-health")
             → Gets full instructions + tool definitions for that skill
             → Calls the actual ability tool (e.g., plugin-list)
             → Returns result to user

Key design principles:

• Skill = grouping layer on top of existing abilities, not a replacement
• Markdown + YAML frontmatter for skill definitions (easy for devs to write, digestible for LLMs)
• Three built-in tools (list_skills, load_skill, skill_read) give the agent self-serve access
• Discoverable vs active — skills listed in the prompt are discoverable; skills become active only when loaded

Technical Notes

• Context budget: With 4096 tokens, the current system prompt (~300 tokens for 13 tools) leaves ~3700 for conversation. At 30+ tools this becomes untenable. Skill tags are ~15 tokens each vs ~25 per full tool definition.
• Relates to [AI] Tool selection at scale — pre-filter tools before LLM #20 (tool selection at scale) — this is the architectural approach to that problem. [AI] Tool selection at scale — pre-filter tools before LLM #20 discussed pre-filtering; skills are the modular unit for that filtering.
• Relates to Vector Database: Semantic ability selection + NLP test framework #3 (vector database for semantic selection) — skills could be selected semantically as a future enhancement, but keyword matching is sufficient for v1.
• Backward compatibility: Abilities registered via registerAbility() should continue to work. Skills are a grouping layer on top of abilities, not a replacement.

Skills Needed

• AI/JS Dev: Skill registry, system prompt refactor, new tools
• Writers: Define skill groupings and contextual instructions for each skill
• LLM Tester: Validate that skill loading works reliably with Qwen 3 1.7B