Draft · working estimates
Agent platform security benchmark (draft)
10 commercial AI-agent platforms, estimated against the ten risk families of the OWASP Top 10 for LLM Applications (2025).
Working estimates from public documentation and agent-audit-kit telemetry. Not yet audited, and no repro recipes are published yet.
- Status
- Draft
- Drafted
- 2026-05-23
- Platforms
- 10
- Risk families
- 10
- Score range
- 0-3
- Max total
- 30
Draft: working estimates, not audited results
The ten families follow the OWASP Top 10 for LLM Applications (2025). Mapping them to the OWASP Top 10 for Agentic Applications (2026) is pending.
Scorecard
Each cell: 0-3. Total per platform: out of 30. Hover a cell to see the one-line justification. Sorted by total descending.
| Platform | A1 | A2 | A3 | A4 | A5 | A6 | A7 | A8 | A9 | A10 | Total |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Managed Agents Anthropic · Managed Agent | 2 | 3 | 2 | 2 | 2 | 2 | 2 | 2 | 2 | 3 | 22/30 |
| Amp Sourcegraph (spinout) · Agent IDE | 2 | 2 | 2 | 1 | 2 | 2 | 2 | 2 | 1 | 2 | 18/30 |
| Cursor Agent Anysphere · Agent IDE | 2 | 2 | 1 | 1 | 2 | 2 | 2 | 1 | 1 | 2 | 16/30 |
| Windsurf Codeium · Agent IDE | 2 | 2 | 1 | 1 | 2 | 2 | 1 | 2 | 1 | 2 | 16/30 |
| Augment Code Augment Computing · Agent IDE | 2 | 2 | 1 | 1 | 2 | 2 | 1 | 2 | 1 | 2 | 16/30 |
| Devin Cognition Labs · Managed Agent | 1 | 2 | 1 | 1 | 1 | 1 | 2 | 1 | 1 | 2 | 13/30 |
| Continue.dev Continue · Agent IDE | 1 | 2 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 2 | 13/30 |
| Cline Cline · Agent IDE | 1 | 2 | 1 | 1 | 2 | 1 | 1 | 1 | 1 | 1 | 12/30 |
| Bolt.new StackBlitz · Managed Agent | 1 | 2 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 2 | 10/30 |
| Replit Agent Replit · Managed Agent | 1 | 1 | 1 | 0 | 1 | 1 | 1 | 1 | 0 | 2 | 9/30 |
Platform commentary
Ranked by total score. The bar shows each platform's share of the 30-point maximum.
Strongest overall posture today. Investments in tool-call governance and audit-trail surfaces are visible. Per-call capability leases are the gap.
Amp
Multi-agent decomposition is structurally the closest to a per-call capability-lease pattern in the IDE class. Weakest on grounding enforcement.
Cursor Agent
IDE-class agent. Human-in-the-loop diff review is the load-bearing safety surface. Excellent for code; capability-lease story underdeveloped.
Windsurf
Enterprise-flavoured IDE class. Strong defaults on disclosure + identity; capability-lease story still per-feature, not per-call.
Augment Code
Enterprise-tier IDE assistant. Strong on identity + isolation. Capability-lease pattern still implicit.
Devin
Sandbox is the load-bearing control. Strong on isolation, weaker on per-call capability leasing and indirect-injection.
Continue.dev
Open-source posture pushes responsibility to the operator. Excellent transparency. Weakest where corporate procurement wants default-on controls.
Cline
MCP-native open-source agent. Power-user-friendly. Lowest default-on protection - by design.
Bolt.new
Different category - generation, not execution. Lower-stakes surface, lower scores on disclosure-class risks.
Replit Agent
Optimized for ship-fast prototyping. Lower-stakes surface; lower default-on protection. Capability lease basically absent.
Risk families (OWASP Top 10 for LLM Applications, 2025)
The ten risk families each platform is estimated against. Mapping them to the OWASP Top 10 for Agentic Applications (2026) is pending.
- A1 Prompt Injection
- Direct + indirect injection across user-controlled inputs, retrieved context, and tool outputs.
- A2 Sensitive Info Disclosure
- Agent leaks secrets, PII, or training-set artifacts through outputs or tool calls.
- A3 Supply Chain
- Compromised models, MCP servers, plugins, or dependency graphs.
- A4 Data + Model Poisoning
- Adversarial inputs into fine-tuning, RAG corpus, eval set, or memory store.
- A5 Improper Output Handling
- Downstream system trusts agent output without validation (SQL, shell, browser, network).
- A6 Excessive Agency
- Overbroad tool inventories, long-lived keys, no allow-listing, no human-in-the-loop on irreversible ops.
- A7 System Prompt Leakage
- System prompt exfiltration via user input, function-calling, or memory injection.
- A8 Vector + Embedding Weakness
- Embedding-store poisoning, retrieval-based prompt injection, multi-tenant leakage.
- A9 Misinformation
- Confident hallucinations propagated into downstream systems without grounding or citation.
- A10 Unbounded Consumption
- Cost / latency / token attacks; agent loops; budget-router absence.
Families A1-A10 are LLM01-LLM10 of the OWASP Top 10 for LLM Applications 2025 (opens in a new tab).
Methodology
How scores are assigned, and what does not move them.
Planned inputs for the audited version: (1) agent-audit-kit scans of each platform's public SDK / OSS components. (2) Vendor documentation, blog posts, and security pages. (3) Reproducible behavioural probes (small adversarial inputs that test the documented protection), published with the scores. (4) Conversations with vendor security engineers (when willing). This draft draws on public documentation and agent-audit-kit telemetry only.
What I do NOT score on: private red-team results, non-reproducible anecdotes, or marketing slides. If a claim can't be reproduced, it doesn't move the score.
Scoring band rationale: bands (0/1/2/3) not decimals. Decimals invite false precision and rank-quibbling. Bands force "is this default-on or not."
Status: draft since 2026-05-23. The audited version replaces these estimates and ships with a changelog showing which scores moved and why.
Disputes
If you work on one of these platforms and disagree with a score, here's the channel:
- 1Email me at sattyamjain96@gmail.com with the platform, the risk family, and your evidence.
- 2If your evidence changes the band, the score moves in the next revision, with attribution.
- 3Vendor-marketed claims don't move scores. Reproducible evidence does.