Connecting an agent and assigning evaluations needs the Editor role or higher. A Viewer can read runs but can’t set them up.
Connect your agent
Switch Trust needs to know how to reach your agent before it can attack it.1
Open the agent's Evaluations tab
In Agents, select the agent you want to test, then select the Evaluations tab. You can also start from the Evaluations section under Connections & data sources on the agent’s Overview tab.
2
Describe how to reach the agent
Under Connect this agent, set the Agent type. It’s Generic HTTP by default, with options for common agent frameworks such as ADK, OpenAI Agent, and Anthropic Agent. Enter the Endpoint where your agent accepts requests, and a Model name if the type asks for one.
3
Add authentication if your agent needs it
Set Authentication to match your endpoint: None, Bearer token, API key, or Custom, then enter the Credential. Add any request Headers your agent expects with Add header.Switch Trust supplies the attacker and judge models itself, so you don’t provide a model-provider key. The credential here is only for reaching your own agent.
4
Test and save the connection
Select Test connection to check that Switch Trust can reach your agent with the settings you entered. Once it succeeds, select Save changes. Switch Trust can now run evaluations against the agent.
Assign evaluations
With the agent connected, choose which evaluations to run against it.1
Open the assign panel
On the agent’s Evaluations tab, select Assign evaluations.
2
Pick the evaluations
Search the catalog and select the built-in and custom evaluations you want. To build your own first, refer to Create a custom evaluation below.
3
Set a schedule
Choose how often the evaluations run: Manual, Daily, Weekly, or Monthly. Manual runs only when you trigger it.The Evaluation schedule toggle controls whether one schedule covers everything. Leave it on to run all evaluations together, or turn it off to schedule each evaluation on its own.
4
Set each evaluation's weight
Set each evaluation’s Weight to Low, Medium, or High. The overall evaluation health is a weighted average of your assigned evaluations, so a higher weight gives that evaluation more pull on the score.
Run an evaluation
A scheduled evaluation runs on its own. To run one now, select Run now on the agent’s Evaluations tab. A run moves through Queued and Running, and lands on Done or Failed. The scores appear on the tab as each run finishes. Refer to Read agent evaluation results.Manage assigned evaluations
To change how Switch Trust reaches the agent, select Connection settings on the Evaluations tab, then Edit connection, and save your changes. Editing a connection clears the saved credential and headers, so re-enter them. To stop running one evaluation against the agent, open its … menu in the evaluations table, select Remove, and confirm. Removing an evaluation stops its scheduled runs and can’t be undone, but the past run history is kept. To disconnect the agent, select Connection settings, then Disconnect agent. Switch Trust no longer evaluates the agent and detaches its active evaluations. The run history is kept, and you can reconnect at any time.Create a custom evaluation
Beyond the built-in catalog, you can build an evaluation from your own prompts.1
Open the evaluations catalog
Open Evaluations in Settings and select New evaluation.
2
Choose how it scores
Under Approach, pick Probe for a pass-or-fail attack, or Metric for a graded 0 to 100 quality score.
3
Map it to a risk framework
Under Risk mapping, set OWASP LLM Top 10 and OWASP ASI Top 10 so the evaluation counts toward each framework’s coverage. Both are optional, so leave a framework on None if it doesn’t apply. Add your own Tags if you want labels the frameworks don’t cover.
4
Name it
Under Details, give the evaluation a name and description.
5
Upload your prompts
Upload a CSV of prompts. The file needs a prompt column, and other columns are ignored.
6
Choose a detector
Pick one detector to score the responses: LLM judge, PII, or Secret. For an LLM judge, start from a template or write your own judge prompt. A Metric evaluation always uses the LLM judge.
7
Create the evaluation
Save the evaluation. It joins the catalog, ready to assign to any connected agent.
Evaluate an external agent
To test an agent that isn’t in your inventory, select Add agents on the Agents page, then Set up evaluations under Evaluate agents. Name the agent and select Create agent. From there, connect it and assign evaluations just like any other agent.Next steps
Read agent evaluation results
Read the score, find failed prompts, and act on them
Evaluate your agents
What an evaluation is and how a run is scored