Context
After landing #209 Fix 1 (PR #210), the doc-search path correctly bypasses tool selection. But on an empty / unindexed knowledge base, the model answers from training data alone and confidently fabricates specifics.
Repro
- Book button on
- Knowledge base not indexed (or query has no hits)
- Ask: "Whats the hook that allows me to inject content in the footer?"
- Model answers: "The hook that allows you to inject content in the footer is `footer_init`…"
`footer_init` is not a real WordPress hook. The correct answer is `wp_footer`.
Root cause
- Default model is Qwen 3 1.7B — knowledge gaps on specifics.
- Doc-search path's no-hits system prompt currently says "answer from general WordPress knowledge and tell them no local docs were found." That permits free-form answering, doesn't strongly enforce "say you're unsure when uncertain."
- No factual grounding mechanism when the KB is empty.
Proposed fixes (any subset)
Out of scope
- Improving model accuracy on small Qwen — that's an external-provider migration concern, separate from this UX gap.
Related
Context
After landing #209 Fix 1 (PR #210), the doc-search path correctly bypasses tool selection. But on an empty / unindexed knowledge base, the model answers from training data alone and confidently fabricates specifics.
Repro
`footer_init` is not a real WordPress hook. The correct answer is `wp_footer`.
Root cause
Proposed fixes (any subset)
Out of scope
Related