System prompts and user messages look like they do the same thing — both are text that influences the model. But they serve fundamentally different purposes, live in different places in the API, and have different temporal scopes.
Different purposes
System prompt — sets the agent’s role, behavioral rules, and persistent instructions. “You are a research assistant. Only cite verified sources. Maintain a professional tone.” This frames EVERY response the agent gives.
User message — provides the specific task or question for the current turn. “Summarize this document.” “What does the processPayment function do?” This is per-turn input that changes with each interaction.
Persistent behavioral requirements (tone, policies, role) go in the system prompt. Per-turn content (questions, documents to analyze) goes in user messages.
Different locations in the API
System instructions use the top-level system parameter — outside the messages array. User questions go in the messages array with role: "user". They are structurally separate.
There is no role: "system" in the messages array. There is no combined instructions parameter. System content is not in HTTP headers. The system prompt has its own dedicated parameter.
The API is stateless
A critical fact: the Claude API is stateless. System prompts are not stored server-side after the first request. Every request must include the system prompt — there’s no “the server remembers from last time.” If you omit the system prompt from a request, the model has no behavioral framing for that call.
This means the system prompt is included in every API call, adding to token usage. But this is by design — each request is self-contained, and the system prompt provides consistent behavioral framing.
The right split
For a customer support agent:
- System prompt: professional tone, company policies, escalation rules → applies to every response
- User message: “Customer asks about return policy for order #12345” → varies per turn
Don’t put persistent rules in user messages (wasteful duplication each turn). Don’t put per-turn questions in the system prompt (it doesn’t change). Each has a natural home.
One-liner: System prompts provide persistent behavioral framing (role, rules, tone) in the top-level system parameter; user messages provide per-turn input in the messages array — the API is stateless, so include the system prompt in every request.