agentic_rag_cost
Multi-turn loops are powerful, and expensive
Every loop re-sends conversation history plus retrieved docs, so input tokens grow with loop depth. These three real profiles — support bot, research agent, document analyzer — are priced across every model, with the multiplier over single-pass search made explicit.
why_loops_cost_more
Single-turn search. Query → retrieve docs → LLM ranks and formats. One LLM pass.
Multi-turn loop. Query → retrieve → analyze → refine query → retrieve again → re-rank. Each turn re-sends the conversation history, so input tokens grow with loop depth.
A 5-turn research agent might cost 3–5× as much as single-pass search, even on the same model. The multiplier is driven by loop depth plus retrieval size per turn.
loop_profiles
Multi-Turn Support Bot
Customer asks question → retrieve ticket history + KB → answer or escalate → optionally refine answer based on feedback. Typical 2-3 turns.
cost_across_models · 1k_req_day
| Model | Per request | Per month | vs single-turn |
|---|---|---|---|
| Claude Haiku Anthropic | $0.02 | $497 | 3.6× |
| Claude Sonnet Anthropic | $0.06 | $1,863 | 3.6× |
| Claude Opus Anthropic | $0.31 | $9,315 | 3.6× |
| GPT-4o mini OpenAI | $0.0029 | $85.53 | 3.6× |
| GPT-4o OpenAI | $0.05 | $1,418 | 3.6× |
| Gemini 1.5 Flash Google (Vertex) | $0.0014 | $43.00 | 3.5× |
| Gemini 1.5 Pro Google (Vertex) | $0.02 | $709 | 3.6× |
This agent costs about 3.6× more than single-pass search on the same model, driven by 2 turns and 2 tools/turn. Trimming loop depth or batching retrievals cuts cost sharply.
Multi-Step Research Agent
Research question → web search → fetch pages → extract facts → refine search → synthesize. Typical 4-6 turns with high token cost per turn.
cost_across_models · 1k_req_day
| Model | Per request | Per month | vs single-turn |
|---|---|---|---|
| Claude Haiku Anthropic | $0.16 | $4,752 | 14.1× |
| Claude Sonnet Anthropic | $0.59 | $17,820 | 14.1× |
| Claude Opus Anthropic | $2.97 | $89,100 | 14.1× |
| GPT-4o mini OpenAI | $0.03 | $831 | 14.7× |
| GPT-4o OpenAI | $0.46 | $13,838 | 14.8× |
| Gemini 1.5 Flash Google (Vertex) | $0.01 | $415 | 14.6× |
| Gemini 1.5 Pro Google (Vertex) | $0.23 | $6,919 | 14.8× |
This agent costs about 14.1× more than single-pass search on the same model, driven by 5 turns and 2 tools/turn. Trimming loop depth or batching retrievals cuts cost sharply.
Document Analysis Agent
Process document → extract sections → classify → generate summary → answer follow-up questions. Typical 3-4 turns.
cost_across_models · 1k_req_day
| Model | Per request | Per month | vs single-turn |
|---|---|---|---|
| Claude Haiku Anthropic | $0.07 | $2,043 | 6.0× |
| Claude Sonnet Anthropic | $0.26 | $7,653 | 6.1× |
| Claude Opus Anthropic | $1.28 | $38,253 | 6.1× |
| GPT-4o mini OpenAI | $0.01 | $363 | 5.9× |
| GPT-4o OpenAI | $0.20 | $6,003 | 6.1× |
| Gemini 1.5 Flash Google (Vertex) | $0.0061 | $183 | 5.7× |
| Gemini 1.5 Pro Google (Vertex) | $0.10 | $3,003 | 6.1× |
This agent costs about 6.0× more than single-pass search on the same model, driven by 3 turns and 2 tools/turn. Trimming loop depth or batching retrievals cuts cost sharply.
cost_multiplier
The cost multiplier
A 5-turn research agent at $17,820/month costs roughly 4× more than single-turn search on the same model. The difference is loop depth plus retrieval size per turn, not the per-token price.
optimization_playbook
Five ways to cut agentic cost
Reduce Loop Depth
Can you solve it in 2 turns instead of 5? Each turn re-sends context, so fewer loops = exponential savings.
example Research agent: Instead of refine + re-search, limit to search + synthesize = 2 turns, 40% cost savings.
Batch Retrievals
Fetch all docs upfront instead of iterating per turn. Single retrieval call + synthesize is cheaper than multi-turn gradual refinement.
example Support bot: Fetch full KB + ticket history upfront = single retrieval vs fetching KB then fetching ticket separately.
Use Cheaper Models for Retrieval
First turn can use a cheap model to rank candidates; later turns use premium model only for final synthesis.
example Research agent: GPT-4o mini for search ranking + Claude Opus for synthesis = 30% cheaper than Opus all the way.
Cache Retrieved Context
If the KB or document doesn't change, use prompt caching for retrieval results. Pay once per day, amortize across all requests.
example Document analyzer: Cache the full document + classification results, pay embedding cost once per 24h instead of per request.
Limit Tool Calls
Each tool call adds an LLM turn. Can you combine tools (e.g., web search + page fetch in single tool) to reduce turns?
example Research agent: 2 tools/turn × 5 turns = 10 tool calls. Batch tools per turn or use a single 'search and fetch' tool.
prompt_caching
Prompt caching: the game changer
If your agent’s KB or document does not change often, use prompt caching. Pay the embedding cost once per 24 hours, then amortize it across every request that day.
Example: a research agent at 1000 req/day. Without caching, 1000 retrieval calls cost $0.01. With caching, one call per 24h costs about $0.00.
next_steps
Model these costs: the AI Agent Cost Calculator lets you adjust loop depth and tool calls for your exact workflow.
Understand failure modes: agentic loops can also fail. See function-calling reliability and MCP tool failure modes for how tool-calling errors cascade cost.
Read related: LLM cost by use case and the agentic context tax glossary term.
Want your agent's loop cost pressure-tested?
The calculator models it; an audit finds the loop depth and retrieval bloat quietly multiplying your bill. Book a call, or leave your email and I'll reach out.
Prefer proof first? See how this plays out in real case studies →