Now in Beta · v0.14.2

Diagnose and fix your AI agent.
Without the code.

Add two lines of code. Agent Doctor auto-captures every step in your pipeline: LLM calls, vector DB lookups, tool executions, API calls, all of it. Override any prompt or model from the dashboard and re-run on your infrastructure. Your data never leaves your servers.

Zero code changes
Auto-captures everything
Runs on your infra
Your data never leaves
customer_support_agent11 spans3.21s
support_agent
3.21sauto
classify_intent
210msllm
fetch_customer
85msapi
search_knowledge
340msvector_db
embed_query
62msllm
pinecone_query
278msvector_db
check_refund_policy
45mstool
draft_response
1.24sllm
safety_check
310msllm
log_to_crm
52msapi
Auto-instruments the tools you already use. No config.
Python SDK available todayNode.js SDK coming soonBuilt on OpenTelemetry
Faster debugging vs.
manual log inspection
+10%
Avg. accuracy lift from
optimized prompts
0
Bytes of your data that
leave your servers
2 lines
To capture every step
of your pipeline
Builder × Domain Expert

Code alone doesn't make a great agent.
Expertise does.

An engineer can wire up the pipeline, but they can't judge a sound medical diagnosis, a well-written paragraph, or a legally airtight clause. Agent Doctor hands those controls to the experts who can. Pair a builder with a domain expert and agent quality multiplies.

Engineers
Build the pipeline & ship it to production
×
Domain experts
Doctors, engineers, artists, writers. No code needed.
=
Agents, multiplied
Real-world expertise baked into every prompt
How it works
Two lines. Zero changes to your agent.

Initialize the SDK, register a webhook. Agent Doctor auto-instruments every LLM call, vector DB lookup, tool execution, and API request. No decorators, no wrappers, no code changes.

1

Add two lines

Install the SDK and call init(). Every LLM call, vector DB query, tool execution, and API request is auto-captured. No decorators. No wrappers.

import agent_doctor
agent_doctor.init(api_key="sk-...")

# That's it. Every LLM call
# is now auto-captured.
2

Override from the dashboard

Domain experts open the UI, pick any node, swap the prompt or model, and hit run. The experiment fires on your servers via webhook.

# Dashboard sends to your webhook:
{
  "node": "reformulate",
  "model": "gpt-4o",
  "temperature": 0.1
}
3

Compare & ship

Traces stream back via OpenTelemetry. Compare runs side-by-side. Deploy the winning prompt. No code deploy needed.

+7.2% accuracy
-34% tokens
-0.3s latency

# Ship with one click
The Bridge
Developers build it.
Domain experts perfect it.

The biggest bottleneck in AI agents isn't the code. It's that the people who understand the domain can't touch the prompts. Agent Doctor gives domain experts a dashboard to iterate on prompts, models, and parameters without ever opening the codebase.

Developer

Builds the pipeline

Writes the agent code, connects to databases, sets up tools, deploys to production.

def diagnose_patient(symptoms):
  history = ehr_db.lookup(patient_id)
  research = pubmed.search(symptoms)
  diagnosis = llm.generate(...)
  return treatment_plan(diagnosis)

Adds two lines & a webhook

Drops in the SDK. Done. Back to building features.

import agent_doctor
agent_doctor.init(api_key="sk-...")

Reviews & ships

When the domain expert finds a winning prompt, the dev reviews and deploys, or it ships automatically via the SDK.

traces
overrides
results
Domain Expert

Sees every step visually

Opens the dashboard, sees the full pipeline trace. Every prompt, every output, every decision the agent made.

Tweaks prompts & parameters

Spots a bad prompt, rewrites it in plain English. Changes the model or temperature. Hits "Run". No code, no deploys.

- "Provide a general diagnosis"
+ "You are a board-certified cardiologist.
  Prioritize differential diagnosis.
  Cite relevant clinical guidelines."

Iterates until it's right

Compares runs side-by-side, uses Dr. Pinpoint to debug, runs flakiness tests. All from the dashboard.

Example: Medical Triage Agent
Dev built
Doctor improved
💻
Sarah · Backend Engineer
Built the 6-node triage pipeline
Pipeline she builtintake symptom_analysis medical_db
differential_dx treatment_plan safety_review
Deployed & SDK initialized
🩹
Dr. Patel · Cardiologist
Iterates on prompts via dashboard
What she changed (no code)Node: differential_dx
Prompt: "Apply AHA guidelines for
 cardiac symptom evaluation..."

Model: gpt-4o claude-sonnet
Temp: 0.7 0.2
+23% diagnostic accuracy
The SDK
Your existing code. Unchanged.

Agent Doctor hooks into LLM SDKs, vector databases, and tool calls at the library level. Your pipeline code stays exactly as-is. Every step is auto-captured.

support_agent.py
# Add these two lines, that's it
import agent_doctor
agent_doctor.init(api_key="sk-proj-...")

# Your existing agent code, zero changes needed
# LLM calls, vector DB queries and tool calls are auto-captured

def handle_ticket(query, customer_id):
    # LLM call → auto-captured
    intent = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "user", "content": query}]
    )

    # Vector DB lookup → auto-captured
    docs = pinecone_index.query(vector=embed(query), top_k=5)

    # Tool call → auto-captured
    policy = check_refund_policy(customer_id)

    # LLM call → auto-captured
    response = openai.chat.completions.create(
        model="gpt-4o",
        messages=[{"role": "system", "content": f"Docs: {docs} Policy: {policy}"},
                  {"role": "user", "content": query}]
    )
    return response

# Webhook handler. Overrides auto-injected by the SDK
with agent_doctor.run_context(run_id=payload.run_id, overrides=payload.overrides):
    result = handle_ticket(payload.query, payload.customer_id)
    # Every step auto-captured. Overrides auto-injected. Done.
Platform
Debug, stabilize, and optimize.

Four tools to take your agent from broken to production-ready.

Pipeline Capture & Override

Auto-capture every step: LLM calls, vector DB queries, tool executions. Override any prompt, model, or temperature from the UI.

Dr. Pinpoint

Chat with an AI debugger: "my agent should say X but it says Y." It asks 3 to 4 questions and pinpoints the exact broken node.

Flakiness Detection

Run your pipeline 10× and see which node is non-deterministic, via a stability heatmap and AI root cause report.

Prompt Optimizer

Automatically generate, test, and rank improved prompt variants for any node. Strategies include GEPA (genetic evolution), grid search, and more. Optimize for accuracy, consistency, cost, or latency.

See it in action
Override any prompt. Re-run the full pipeline.

Watch Dr. Patel, a cardiologist with zero coding skills, diagnose a broken AI agent, override the faulty prompt, and get better results. All from the dashboard.

🩹
Dr. Patel
Cardiologist · Domain Expert
1Baseline Run (Auto-Captured)
patient_triage
2.8sroot
classify_symptoms
180msllm
search_medical_db
340msvec
differential_dx
1.1sllm
treatment_plan
520msllm
safety_review
140msllm
Final Output (Incorrect)
"Patient likely has general anxiety. Recommend lifestyle changes and follow-up in 2 weeks."
⚠ Dr. Patel: "This is wrong. These symptoms point to unstable angina, not anxiety. The prompt has no examples to guide it."
2Dr. Patel Overrides from Dashboard
Selected: differential_dx
System Prompt
"Analyze the symptoms and provide a diagnosis."
"Analyze the symptoms and provide a differential diagnosis. Use the examples below as guidance.

+ POSITIVE EXAMPLE:
Input: chest pressure, diaphoresis, jaw pain
Output: Unstable angina (ACS). Order troponin, 12-lead ECG.

- NEGATIVE EXAMPLE:
Input: chest pressure, diaphoresis, jaw pain
Output: General anxiety. Lifestyle changes. ← wrong"
Model
gpt-4o-mini  →  gpt-4o
Temperature
0.7  →  0.2
3Compare Results
Baseline Run
DiagnosisGeneral anxiety
Accuracy34%
Modelgpt-4o-mini
Temperature0.7
Tokens286
Latency1.1s
Override Run Better
DiagnosisUnstable angina (ACS)
Accuracy97%+63%
Modelgpt-4o
Temperature0.2
Tokens412+44%
Latency1.8s+0.7s
New Output (with few-shot examples)
"Presenting symptoms (chest pressure, diaphoresis, jaw pain radiating to left arm) are consistent with unstable angina. Recommend immediate troponin levels, 12-lead ECG, and cardiology consult."
✓ Dr. Patel: "Adding those two examples fixed it. The model needed to see what right and wrong look like."
Step 1 of 3: Baseline captured
Dr. Pinpoint
"My agent should say X
but it says Y."

Describe the expected behavior and the actual result. Dr. Pinpoint uses binary search to ask 3 to 4 plain-English questions, then pinpoints the exact node that caused the downstream failure. No logs, no code, no debugging skills needed.

  • Chat interface for domain experts, not developers
  • O(log n) questions to find root cause
  • Highlights the broken node in the pipeline
  • "Open Rx" hands off to prompt optimizer
Dr. Pinpoint
Step 2 of ~3
My agent should return a refund confirmation, but instead it's asking the customer to call support.
Got it. Let me trace through the pipeline. The router node received the customer's request and routed it to "refund_handler." Its output was: "Customer requests refund for order #4821." Does this routing look correct?
Yes, that's right.
Good. The refund_handler then looked up order #4821 and produced: "Order found. Status: delivered. Refund policy: eligible within 30 days." Does this look correct?
Yes, that's correct too.
Root cause found: response_composer

All upstream nodes are healthy. The response_composer node received correct refund eligibility data but generated a "call support" response instead of processing the refund.

Suggestion: The prompt lacks explicit instruction to confirm eligible refunds directly.

Open Rx → Fix this prompt
Stability Analysis
Find the flaky node.
Fix it in one click.

Run your pipeline 10× with identical input. Per-node consistency scores reveal exactly which step is non-deterministic, and an AI report tells you why.

  • Find the flaky node in one run
  • Per-node consistency scores & decision divergence
  • AI root cause report with fix suggestions
  • One-click fix → re-test → verify
Stability Analysis
10 / 10 runs
user_query0.99
router0.54
rag0.78
synthesize0.72
validate0.98
response0.95
FLAKY router | Decision Divergence: 30%
  7/10 → Knowledge Base  |  3/10 → Account Tools
MODERATE synthesize | Consistency: 0.72
STABLE validate | Consistency: 0.98

Root cause: Router prompt lacks explicit routing rules for password queries.
Prompt Optimizer
Generate, test, and rank
better prompts. Automatically.

Pick a node, define what "good" looks like, and let the optimizer find the best prompt. Multiple strategies available, starting with GEPA (Genetic Evolutionary Prompt Algorithm), with more coming soon.

  • GEPA: evolve prompts across generations
  • Mutation operators: rephrase, expand, compress…
  • Crossover: recombine winning traits
  • Leaderboard ranks variants by fitness score
Prompt Optimizer
GEPA · Gen 7/7
GEPA
Grid Search
Bayesian
+ More
1.0
0.65
0.3
#VariantFitnessAcc.Tok.
1G7-V30.9496.1%134
2G6-V10.9294.8%151
3G5-V50.8992.3%168
BBaseline0.7187.1%185
Role: Elite
Task: Rephrased
Format: Crossover
Constraints: Negated
Examples: Elite
Why Agent Doctor
Why not just use a regular observability tool?

Most LLM tracing platforms run experiments on their own cloud, so your agent can't reach your private databases, RAG, or internal tools during a test. They also keep prompt iteration locked behind code, leaving your domain experts out.

Capability
Other tools
Agent Doctor
Runs experiments on your infra (private DBs, RAG, tools)
SaaS-hosted only
Remote-triggered
Your data & payloads never leave your servers
Sent to their cloud
Zero-trust payload
Non-technical domain experts can tune prompts
Python scripts only
Visual dashboard
Root-cause the exact node behind a bad output
Manual trace reading
Dr. Pinpoint AI
Automatically evolve better prompts (genetic optimizer)
Manual iteration
GEPA built-in
Setup effort
SDK wrappers & decorators
2 lines, zero changes
FAQ
Questions, answered.
Does my data or model ever leave my servers?
No. Agent Doctor uses remote-triggered execution: experiments run entirely inside your own infrastructure, where your agent already has access to private databases, vector stores, and internal tools. We only orchestrate the prompt override and visualize the telemetry trace your SDK streams back. We never see or execute your underlying payloads.
How much code do I have to change?
Two lines: import the SDK and call init(). It auto-instruments your LLM calls, vector DB lookups, tool executions, and API requests at the library level, with no decorators, wrappers, or refactors required.
Which languages and frameworks are supported?
Python and JavaScript/TypeScript SDKs today, with auto-instrumentation for OpenAI, Anthropic, LangChain, LlamaIndex, and major vector databases. Traces are emitted over OpenTelemetry, so anything OTel-compatible works.
Can non-technical teammates actually use it?
Yes. Domain experts get a visual dashboard to inspect every step, rewrite prompts in plain English, change models or temperature, and re-run, all without opening the codebase. Developers stay focused on building.
What does it cost?
There's a free tier to get started, with no credit card required. Beyond that, you hold credits and each remote experiment run deducts a small fraction, so you only pay for what you actually test. Reach out for team and enterprise plans.

Ready to diagnose and fix?

Two lines of code. Zero changes to your agent. Every step captured automatically.

No credit card Free tier forever 5-minute setup