Skip to main content
Release notes and version history for Switch Trust CLI, formerly Flint AI CLI.
​
October 2, 2026
Maintenance release.Fixed:
  • Corrected project links - The package’s homepage and issue-tracker links point to the switch-trust-cli repository, and the README uses the current Switch Trust branding
​
October 1, 2026
Flint AI CLI is now Switch Trust CLI, with steadier evaluation against live agents.Changed:
  • Flint AI CLI is now Switch Trust CLI - Breaking: The package is now switch-trust-cli (pip install switch-trust-cli) and the command is switch-trust, so flintai scan becomes switch-trust scan and flintai eval becomes switch-trust eval. The config directory moves from ~/.flintai to ~/.switch-trust — re-run switch-trust init or move the directory, as there is no automatic fallback. The FLINTAI_TELEMETRY_CONSENT and FLINTAI_CLIENT_ID environment variables become SWITCH_TRUST_TELEMETRY_CONSENT and SWITCH_TRUST_CLIENT_ID; the old names still work for now and print a one-time deprecation notice. See environment variables
  • Steadier evaluation against live agents - Evaluation limits how many requests it sends to a single agent endpoint at once and spreads out the opening burst, so a run stays within the agent’s rate limit. This covers the HTTP-based model types — ADK, OpenAI agent, Anthropic agent, generic HTTP, LangServe, and Vertex AI Agent Runtime; SDK-based models are unaffected
  • Smarter retries - Evaluation honors a server’s Retry-After response after a rate limit, and no longer retries authentication or not-found errors (401, 403, 404), so a bad key or wrong endpoint fails fast instead of looping
Fixed:
  • Evaluation says why a run failed - A failed run now reports the specific reason — no prompts to run, every prompt failed, or too few prompts succeeded to score — instead of a generic empty result
  • Clearer errors from agent endpoints - When an agent’s HTTP endpoint returns an error, the CLI surfaces the useful detail from the response without logging your content
​
September 25, 2026
Scan output accuracy fixes.Fixed:
  • Scan findings point to the right file in SARIF - A scan finding now uses the actual evidence file path as its location in SARIF output, instead of a free-text label that varied between runs for the same issue. Component names are kept under properties.affectedComponents. See scan results
  • More accurate finding categories - A finding’s category is derived from its subcategory and mapped to the canonical taxonomy, so a correct finding is no longer mis-scored when the model names the wrong category; an unrecognized category is flagged rather than dropped
​
September 23, 2026
A new evaluation target and steadier eval runs.Added:
  • Evaluate agents on Google Vertex AI Agent Runtime - Point flintai eval at an agent deployed to Vertex AI Agent Runtime (Reasoning Engine), authenticating with either a service-account key or an OAuth access token
  • Custom headers for ADK agents - The ADK model type accepts custom HTTP headers, so you can reach a deployment behind an app-level token such as an Authorization bearer
Changed:
  • An unreliable run is flagged, not scored - When fewer than 90% of an evaluation’s prompts succeed, the run is reported as an error instead of returning a score computed from a fraction of the prompts
  • The judge resists prompt injection - The response under evaluation is fenced off and the LLM judge is instructed to treat it strictly as data, so content in a model’s response can’t steer the judge
  • Deeper scan reasoning - The scanner reasons for longer before concluding, improving coverage on larger agents
  • Faster garak evaluations - Garak detector datasets load once per process instead of reloading for every job
​
September 21, 2026
Maintenance release.Changed:
  • Dependency update - The bundled transformers library, used by the optional full extra, was updated to a newer release
​
September 4, 2026
A reasoning-based scanner and more reliable evaluation.Changed:
  • Reasoning-based scanning - flintai scan now investigates an agent the way an application security researcher would — following untrusted input into the agent, tool output fed back to the model, unbounded loops, and missing authorization or guardrails — instead of running a fixed checklist, which broadens what a scan finds. See scan results
  • Reliable structured output from eval models - Evaluation asks each provider for structured JSON directly (OpenAI, Anthropic, Gemini, Ollama, LiteLLM, and vLLM), so judge and detector results parse reliably, falling back to text only when a provider can’t
  • Judges explain their score - An LLM-judge detector returns a reason alongside its score, so you can see why a response was marked the way it was
  • Errored prompts no longer drag down a score - A prompt that errors during evaluation is excluded from the average instead of counting as a zero
  • Adversarial content isn’t blocked mid-eval - The Gemini generator and judge no longer apply safety filters to red-team content, so a blocked judge no longer surfaces as a spurious error. This affects only the generator and judge, not the model under test
  • Updated default models for new setups - flintai init writes current model defaults into a new config; existing configs keep the models they already pin
Fixed:
  • Clearer failures when a key or model is wrong - Scan and evaluation classify a failure as authentication, model, provider, or input and give a precise message; a missing API key skips the LLM step with a warning instead of crashing
​
August 26, 2026
Per-project configuration and a quicker start.Added:
  • Per-project configuration - A .env in your working directory now configures that project, layered over the global ~/.flintai/.env. Anything already set in your shell or CI still wins. See environment variables
Changed:
  • Quicker to start - flintai --version, flintai --help, and a mistyped argument return without loading the evaluation and telemetry stack, and flintai scan loads the scanner only once a scan runs
Fixed:
  • Scans complete on GPT-5 reasoning models - Tool calls reach the interface that supports them, and a parameter the model declines is dropped rather than ending the scan
​
August 13, 2026
A smaller, faster default install.Changed:
  • Leaner default install - Breaking: The standard install is roughly 2 GB smaller and no longer pulls CUDA on Linux. Scanning and the built-in OWASP LLM evaluations work as before. Garak evaluations, toxicity detection, and locally run HuggingFace models now need the optional full extra. See installation
​
August 12, 2026
Optional usage analytics, a clear warning when a scan runs short, and changes to machine-readable output.Added:
  • Pseudonymous usage analytics - The CLI asks on first run whether to share pseudonymous usage data, and pressing Enter turns it on. You can change your answer at any time, CI runs never send, and code, prompts, and keys are never collected. See data privacy
  • Incomplete scans now say so - When an analysis tool is unavailable, flintai scan warns that the findings are partial and names what’s missing, instead of quietly returning a shorter list. See scan results
  • --log on flintai scan - Set the log file path, matching the flag already available on flintai eval
Changed:
  • Default scan model - New setups scan with gemini-3.6-flash. Existing setups keep the model they already pin. See environment variables
  • Eval results name the schema version in snake_case - Breaking: the schemaVersion field in eval JSON output is now schema_version, and the format is version 2.0. Update automation that reads the old key. See eval results
  • Scan JSON drops two empty fields - Breaking: agents_found and agent_profiles are gone from flintai scan JSON output. Both were always empty. See scan results
  • Predictable dependency resolution - The OpenTelemetry and protobuf version ranges are pinned so pip install resolves without backtracking
Fixed:
  • Scanning a single file returns findings - flintai scan path/to/agent.py previously completed and reported nothing. A single file is now analyzed the same way it would be inside a directory. See the command reference
Security:
  • Dependency hardening - Cryptography and networking libraries updated to current maintained releases
​
July 27, 2026
Changed:
  • Scan finding impact text - The impact field on a scan finding now carries the description from the issue taxonomy instead of a generated Exploitable via ... string. This text appears in both JSON and SARIF output, so update any automation that matches on the old wording
  • Safety refusals during eval - A model refusal for a disallowed prompt now reports as blocked for safety rather than as a model error, so an eval run distinguishes a refusal from a failure
Fixed:
  • Eval no longer stops on a malformed provider error - An error response with an unexpected body is handled instead of ending the run
  • Corrected documentation links - Links in the project README now point to migrated documentation paths
​
July 15, 2026
Machine-readable output and broader Python support.Added:
  • --format {json,sarif} - Choose the output format for flintai scan and flintai eval run, with -f as the short form. json stays the default, so existing commands behave as before
  • SARIF 2.1.0 output - Feed scan and eval results directly into GitHub code scanning and other SARIF-aware workflows
  • Versioned eval output - Machine-readable eval results carry a schema version, so downstream automation stays stable across releases
Changed:
  • Python 3.11 or later - Lowered from 3.13, so the CLI installs in more development environments
  • Default output filenames follow the chosen format - Breaking: Scan writes scan_<timestamp>.<format>, renamed from results_<timestamp>.json, and eval writes eval_<timestamp>.<format>. Update any automation that looks for the old results_*.json name
​
June 30, 2026
Evaluation reliability and setup fixes.Changed:
  • More reliable adversarial evaluation - Built-in detector behavior tightened, with structured JSON judge output now required for key adversarial checks
  • Smoother getting started - Clearer virtual environment setup and example-agent instructions, plus corrected command-reference links
​
June 18, 2026
Initial public release of Flint AI CLI.Added:
  • flintai scan - AI-powered static analysis of Python agent code, with findings mapped to the OWASP Top 10 for Agentic Applications and scored with CVSS severity
  • flintai eval - Runtime behavioral evaluation of running agents using adversarial and functional tests
  • Built-in evaluation content - Evaluations for OWASP LLM risks, plus the detector and model configuration needed to run them locally
  • Framework detection - Auto-detect supported frameworks