Agent Judge
AgentJudgeReasoningA judge for outputs you already have. Pass the output to evaluate as the task; the judge identifies the domain, applies criteria that fit it, and returns a reasoned assessment of strengths, weaknesses, factual problems and concrete improvements. With max_loops above 1, each pass sees its earlier verdicts and sharpens them.
Reasoning agents need a Pro or Premium plan. Requests from a free-tier key return 403. See plans
Architecture
How one run moves through the agent
- 1You send the output to evaluate, with any context, as the task.
- 2The judge identifies the domain and picks evaluation criteria for it.
- 3It returns a verdict: strengths, weaknesses, accuracy issues and recommendations.
- 4With max_loops above 1, the next pass reads the earlier verdicts and refines them.
- 5The response contains the full evaluation.
At a glance
AgentJudgePOST /v1/reasoning-agent/completionsBest for
- Grading LLM or agent outputs in evaluation pipelines
- A quality gate before an agent’s answer reaches users
- Comparing candidate answers and explaining which is better
- Reviewing code, writing or analysis for accuracy and gaps
Parameters
Fields that shape this agent, alongside task, agent_name and description
max_loopsEvaluation passes. Each pass builds on the earlier verdicts.model_nameModel used as the judge. Defaults to claude-sonnet-5.system_promptOptional rubric or criteria. Leave empty to use the built-in evaluation protocol.Quick start
Get a key, list the reasoning agent types, then run this one. Code examples in cURL, Python, TypeScript, Go and JSON.
Get your Swarms API key
Create a key, then export it as SWARMS_API_KEY in your environment.
List the reasoning agent types
Returns every supported swarm_type.
List reasoning agent types
curl -X GET 'https://api.swarms.world/v1/reasoning-agent/types' \ -H 'x-api-key: '"$SWARMS_API_KEY"Run the Agent Judge
Send a task with swarm_type: "AgentJudge" to the reasoning agent endpoint.
Run Agent Judge
curl -X POST 'https://api.swarms.world/v1/reasoning-agent/completions' \ -H 'x-api-key: '"$SWARMS_API_KEY" \ -H 'Content-Type: application/json' \ -d '{ "agent_name": "output-judge", "description": "Evaluates an answer for accuracy and completeness.", "swarm_type": "AgentJudge", "model_name": "gpt-5.5", "max_loops": 1, "task": "Evaluate this answer for accuracy and completeness.\n\nQuestion: What causes the seasons on Earth?\nAnswer: The seasons happen because Earth is closer to the Sun in summer and farther away in winter."}'Read the response
outputs holds the agent's result, and usage reports tokens and the cost of the run.
{ "job_id": "reasoning-agent-…", "status": "success", "outputs": "…", "timestamp": "2026-09-25T12:00:00+00:00", "agent_name": "output-judge", "agent_type": "AgentJudge", "agent_id": "…", "usage": { "input_tokens": 64, "output_tokens": 1120, "total_tokens": 1184, "total_cost": 0.0213 }}Other reasoning agents
FAQ
What is the Agent Judge?
A judge for outputs you already have. Pass the output to evaluate as the task; the judge identifies the domain, applies criteria that fit it, and returns a reasoned assessment of strengths, weaknesses, factual problems and concrete improvements. With max_loops above 1, each pass sees its earlier verdicts and sharpens them.
How do I run the Agent Judge with the Swarms API?
POST to https://api.swarms.world/v1/reasoning-agent/completions with "swarm_type" set to "AgentJudge" and a task, authenticated with your x-api-key header.
When should I use the Agent Judge?
It is best for: Grading LLM or agent outputs in evaluation pipelines; A quality gate before an agent’s answer reaches users; Comparing candidate answers and explaining which is better; Reviewing code, writing or analysis for accuracy and gaps.
Which plans can use reasoning agents?
Reasoning agents are a premium endpoint: they need a Pro or Premium plan. A request from a free-tier key returns 403.
How is a reasoning agent run billed?
By tokens, like any other completion. Every sample, loop and evaluation step is a model call, so raising num_samples or max_loops raises the cost. The response reports input, output and total tokens and the total cost.