Interactive scorecard

AI-Sec Tool Scorecard Builder

Pick 1–3 reviewed tools, then tune what matters for your context. Every dimension score comes from the documented review on this site — vendor documentation, source code and published evaluations — with the exact evidence sentence shown inline, so you can see why a tool ranks where it does, not just that it does.

Each dimension is scored 0–5 from the linked review, on published evidence rather than first-hand testing. 5 = strongest documented; 0 = effectively absent or a hard weakness. Higher is always better, including for cost/effort dimensions (5 = lowest cost / least effort). Scores reviewed 2026-08.

1. Pick 1–3 tools to compare

Garak NVIDIA · LLM vulnerability scanner Lakera Guard Lakera · Prompt-injection detection API Guardrails AI Guardrails AI · Output validation framework PyRIT Microsoft · AI red-teaming framework Rebuff ProtectAI · Self-hosted prompt-injection defense Arize Phoenix Arize · LLM observability platform

2. Apply my context (optional)

Auto-sets dimension weights for a common usage pattern. You can still hand-tune below.

3. Weight each dimension (0 = ignore, 5 = critical)

Detection rate 3

How well it catches the clear, in-scope attacks it is built to catch, on real traffic or standard test sets.

Novel-attack resilience 3

Holds up against adversarially-optimized, encoded, or previously-unseen attacks rather than only fixed/known patterns.

Low false-positive cost 3

How safe it is to act on individual decisions. 5 = very low false positives; low score = noisy / sampling-only.

Latency fit 3

Suitability for the synchronous, latency-bound path. 5 = sub-10ms-ish; low = seconds / hours.

Integration effort 3

How little work it takes to wire into a pipeline or CI. 5 = drop-in API/CLI; low = heavy configuration.

Deployment flexibility 3

Range of deployment models, especially self-host / data-residency-friendly options.

Maintenance signal 3

Health of upstream maintenance and how little ongoing care the operator must invest.

All tools & raw review scores

Tool	Detection rate	Novel-attack resilience	Low false-positive cost	Latency fit	Integration effort	Deployment flexibility	Maintenance signal
Garak Apache 2.0 (open source)	4	2	3	1	2	4	4
Lakera Guard Commercial (SaaS; enterprise self-host)	4	3	4	3	5	4	4
Guardrails AI Apache 2.0 (open source)	3	2	3	3	4	5	4
PyRIT MIT (open source)	4	3	3	3	4	4	5
Rebuff Apache 2.0 (open source)	4	2	3	3	3	5	3
Arize Phoenix Apache 2.0 (open source)	3	2	2	2	4	5	4

All tools & raw review scores

Related tools in this network