System prompt keyword rules (“when user mentions refund, use process_refund”) override description-based tool selection. Removing them and relying on tool descriptions improved routing accuracy by 16 points. Keyword rules are the most common cause of systematic misrouting in agents with otherwise-good tool descriptions.
The data: keyword rules hurt
| Configuration | Routing accuracy | Error pattern |
|---|---|---|
| With keyword rules | 78% | Keyword triggers on ambiguous queries |
| Without (description only) | 94% | Only genuinely ambiguous queries |
Same tools, same descriptions. The 16-point improvement came entirely from removing rules like “when customer mentions refund → process_refund.” The model’s semantic understanding of intent is more accurate than surface-level keyword matching.
Three keyword rule failures
Keyword trigger override: “When the customer mentions refund, always use process_refund.” Customer asks “Where is my refund?” → routed to process_refund (initiates new refund) instead of check_refund_status (checks existing). The keyword “refund” triggers the hardcoded rule before the model evaluates descriptions.
Blanket tool preference: “For any extraction task, use extract_table first.” Plain text documents get forced through table extraction → garbled output. The rule overrides the model’s ability to match text documents to extract_text based on description.
Keyword association with familiar tools: “When searching for code, use the available search tools.” This creates an association between “search” and the built-in Grep tool (more familiar from training). An MCP tool code_search with semantic search capabilities is bypassed — Grep used in 85% of code searches despite code_search being better suited.
Accumulated rules create conflicts
Over 8 months, a CI agent’s system prompt accumulated 12 keyword routing rules (each added to fix a specific incident). Some now conflict: “deploy” → deploy_tool AND “deploy” → verify_deployment. New queries containing “deploy” have unpredictable routing.
The fix: remove rules, improve descriptions
- Audit the system prompt for keyword-to-tool mapping rules
- Remove all keyword routing rules
- Expand tool descriptions with “when to use” guidance that the model can match semantically
- Replace blanket preferences with intent-based guidance: instead of “use extract_table first for all extractions,” use description-level guidance in each tool
Adding more rules makes it worse
“Fix misrouting by adding more keyword rules” is a common but counterproductive response. Each new rule creates new failure modes: queries containing multiple trigger keywords, natural language variations not covered by rules, and rule-to-rule conflicts. The maintenance burden grows while accuracy declines.
When keyword rules are appropriate
Almost never for tool routing. The model’s description-based selection handles nuance better than keyword matching. Keyword rules are appropriate only for absolute safety overrides — not for routine tool selection.
One-liner: Removing keyword routing rules from the system prompt improved accuracy from 78% to 94% — keyword rules override description-based selection, create conflicts when accumulated, and handle nuance worse than the model’s semantic understanding.