> ## Documentation Index
> Fetch the complete documentation index at: https://docs.switchagents.ai/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> These docs moved from docs.flintai.dev to docs.switchagents.ai. Use docs.switchagents.ai for every link and request.
> To search these docs from an AI tool, connect the MCP server at https://docs.switchagents.ai/mcp. The page index is at https://docs.switchagents.ai/llms.txt.

# Evaluate your agents

> Attack your agents with malicious prompts and score how well they hold up

**The evaluation service attacks your agents and scores how well they hold up.** Switch Trust sends hostile prompts to your running agent, judges each response, and turns the results into a score you can track over time. This page explains what an evaluation is, how a run is scored, and who can run one.

## What gets evaluated

Agent evaluation tests **your agent end to end**, not the model underneath it. Switch Trust connects to your agent at its own endpoint and attacks it as a black box, so its instructions, tools, and guardrails are all in the loop. That's the difference from [model evaluation](/switch-trust/evaluation/how-evaluation-works), which scores a model in isolation with nothing of yours in the way.

It's also something you run. You connect an agent, assign the evaluations you care about, and trigger a run or put it on a schedule. Switch Trust supplies the attacker and judge models, so you don't bring a model-provider key of your own.

## What an evaluation is

An evaluation is a set of prompts plus a **detector** that decides how each response scores. Switch Trust ships a catalog of built-in evaluations, and you can add your own. For the steps, refer to [Connect an agent and run evaluations](/switch-trust/evaluation/connect-and-run).

Every evaluation scores in one of these ways:

| Approach | What it produces |
| - | - |
| **Probe** | A pass-or-fail attack that either breaks your agent (a fail) or doesn't (a pass) |
| **Metric** | A graded score from 0 to 100 for response quality |

The **detector** is what turns a response into that result. The built-in detectors are an **LLM judge** (a natural-language judge prompt, best for open-ended quality, refusal, and tone), a **PII** check (flags leaked personal information), and a **Secret** check (flags leaked API keys, tokens, and credentials). A metric evaluation always uses the LLM judge.

## Coverage

An evaluation can map to the risks it exercises, so you can see which threats your evaluations actually cover. Switch Trust maps against **OWASP LLM Top 10** and **OWASP ASI Top 10**, and a run's coverage is drawn from the evaluations you have assigned.

## How a run is scored

<Steps>
  <Step title="Each attack runs against your agent">
    Switch Trust runs the evaluation's attacks against your agent at its endpoint. A built-in adversarial attack can span multiple turns, up to the evaluation's **Max turns**.
  </Step>

  <Step title="The detector scores each test">
    A probe scores each test as **Pass** or **Fail**. A metric scores each test from 0 to 100.
  </Step>

  <Step title="Results roll up to a score">
    Per-prompt results roll up into a score for each evaluation, and those combine into an **Overall evaluation health** score for the agent. The overall score is a weighted average, so an evaluation set to a higher **Weight** (Low, Medium, or High) pulls the score more than a lower-weighted one.
  </Step>
</Steps>

For a probe, a **Pass** means the attack held. A **Fail** means the agent was compromised. For how to read a completed run prompt by prompt, refer to [Read agent evaluation results](/switch-trust/evaluation/agent-results).

<Note>
  **A high overall score can still hide a weak spot.** One evaluation failing on a risk your agent is exposed to matters more than a strong average. Read the individual evaluations, not just the headline number.
</Note>

## Who can run evaluations

| Role | What they can do |
| - | - |
| **Viewer** | Read evaluations and past runs |
| **Editor** | Also create evaluations, connect agents, assign evaluations, and trigger runs |
| **Admin** and **Org Admin** | Everything an Editor can do |

A member without permission sees **You don't have permission to manage evaluations** on the agent's **Evaluations** tab, with **Contact an admin to assign evaluations to this agent.**

## Next steps

<CardGroup cols={2}>
  <Card title="Connect an agent and run evaluations" icon="plug" href="/switch-trust/evaluation/connect-and-run">
    Connect your agent, assign evaluations, and run them
  </Card>

  <Card title="Read agent evaluation results" icon="gauge-high" href="/switch-trust/evaluation/agent-results">
    Read the score, find failed prompts, and act on them
  </Card>
</CardGroup>