Public benchmark · 5 tools · 28 corpora · reproducible

audytx vs Checkov, Trivy, KICS & Terrascan

Five IaC security tools, 28 real-world AWS Terraform corpora, one deterministic scorer — every number below is measured and reproducible from a public repo. audytx, a reasoning engine with no AI anywhere in the pipeline, reaches full recall on all 31 documented IAM privilege-escalation paths at the highest precision of any tool that also reaches it, and fires ~36× fewer false positives than Checkov on the clean modules.

The scorecard

Three numbers, measured

Full IAM-privesc recall, the lowest false-positive count of any tool tested, and double Checkov's precision at the same recall.

100%

recall on 31 documented IAM privilege-escalation paths

full recall, matched only by Checkov · KICS 3% · Trivy 0%

36×

fewer false positives than Checkov on 21 clean modules

33 vs 1,193 · fewest of all five (KICS 34) · 14× fewer than Terrascan

2×

IAM-privesc precision vs Checkov at the same 100% recall

23% vs 12% · half the alert volume (135 vs 269)

Provenance. Competitor tables pinned to the re-run at audytx engine v0.14.8 (the engine's check/axis counts at that date; live counts on /status) on 2026-06-16, against Checkov 3.2.520 · Trivy 0.71.0 · KICS 2.1.20 · Terrascan 1.19.9 (all pinned). Terrascan has since been archived by Tenable (November 2025); its pinned results are retained for reference and remain reproducible. The audytx column is pulled live from GitHub Code Scanning per bench branch and accepted only if its SARIF driver.version equals the live engine, so no stale data can slip in. Scored with scripts/score.py (Python-3 stdlib, byte-deterministic). We re-run the full suite each milestone; reproduce any row yourself from the steps at the bottom.

Table 1 — IAM privilege-escalation: precision & recall

Corpus: BishopFox iam-vulnerable — 31 documented AWS IAM privilege-escalation paths, one Terraform file per path. A tool "detects" a path if it fires at least one HIGH/CRITICAL finding on the file implementing it. audytx and Checkov are scored on HIGH; KICS and Trivy on HIGH+CRITICAL.

Tool	HIGH findings	TP	FP	FN	Precision	Recall
audytx	135	31	104	0	23%	100%
Checkov	269 ¹	31	238	0	12%	100%
KICS	9	1	8	30	11%	3%
Terrascan	DNF ²	—	—	—	—	—
Trivy	7	0	7	31	0%	0%

¹ Checkov run offline (no Bridgecrew API) emits no per-check severity, so this is total failed checks, not HIGH-only — generous for Checkov's recall, conservative for its precision.
² Terrascan exceeded the 5-minute timeout on this corpus (large module graph).
audytx detects all 31 paths at the highest precision of any tool with full recall: roughly twice Checkov's precision at half its alert volume. Most of audytx's 104 "FP" here are legitimate secondary detections (e.g. AWS_OPS_038 firing on the same privesc file the primary check already claimed) plus the corpus's own intentional FP-test fixtures. TP matching counts only one finding per path, which structurally undercounts audytx precision.

Table 2 — False positives on 21 clean production modules

This is the wedge. Each corpus is a well-regarded, actively-maintained AWS community Terraform module with an expected high-severity count of 0. Lower is better — every HIGH finding here is noise a reviewer has to triage. Raw counts shown as measured (nothing subtracted).

Corpus (clean module)	audytx	Checkov	Trivy	KICS	Terrascan
cloudposse-s3-bucket	1	36	1	0	2
terraform-aws-alb	4	54	13	2	6
terraform-aws-apigateway-v2	0	20	7	1	5
terraform-aws-autoscaling	0	11	6	0	1
terraform-aws-cloudfront	0	24	8	0	6
terraform-aws-ecr	0	5	1	2	0
terraform-aws-ecs	0	86	16	2	16
terraform-aws-eks	6	88	38	1	1
terraform-aws-eventbridge	7	57	18	12	24
terraform-aws-iam	3	287	1	0	4
terraform-aws-kms	0	1	0	0	11
terraform-aws-lambda	6	112	23	7	9
terraform-aws-rds	2	124	7	1	10
terraform-aws-s3-bucket	3	129	18	4	348 ³
terraform-aws-secure-baseline	1	107	8	2	8
terraform-aws-security-group	0	10	2	0	0
terraform-aws-sns	0	4	0	0	0
terraform-aws-sqs	0	1	0	0	0
terraform-aws-step-functions	0	6	1	0	1
terraform-aws-vpc	0	25	3	0	1
trussworks-s3-private	0	6	4	0	1
Total	33	1,193	175	34	454

³ Terrascan fires 348 HIGH alerts on a single module (terraform-aws-s3-bucket) — a mass-rule blowup that alone accounts for 77% of its total.
audytx's 33 = ~36× fewer than Checkov (1,193), 14× fewer than Terrascan (454), and 5× fewer than Trivy (175). That's the fewest raw false positives of all five. KICS is closest at 34, and reaches it only by detecting 3% of the privesc paths (Table 1) to audytx's 100%. Several of audytx's 33 trace to documented justified exceptions (real issues in the modules' own example code, tracked in clean-modules.yaml); the rest are newer checks (EKS secrets-encryption, deprecated Lambda runtimes) correctly flagging issues in the modules' examples/ — details in the trade-offs section below.

Table 3 — Recall corpora (raw HIGH counts)

Six additional corpora with deliberately insecure configurations. We have no path-level ground truth here beyond iam-vulnerable, so these are raw HIGH counts, not scored precision/recall — more is not automatically better, since a chunk of any tool's count is noise. Shown for completeness.

Corpus	audytx	Checkov	Trivy	KICS	Terrascan
KaiMonkey	40	109	112	0	21
iam-role-chain	4	9	0	1	0
learn-terraform-provision-eks-cluster	2	3	3	0	0
sadcloud	47	201	26	53	58
terraform-aws-eks-blueprints	29	210	DNF ⁴	13	7
terragoat	52	466	93	70	35

⁴ Trivy timed out on eks-blueprints (5-min limit); Terrascan DNF on iam-vulnerable (Table 1).

How the precision gap happens — a worked example

The Table 2 gap doesn't come from running fewer checks. It comes from cross-resource reasoning: audytx pre-computes relationship graphs and suppresses findings that context proves benign, showing the rationale instead of dropping them silently. Here's the mechanism on one fixture (testbed #11).

Serverless messaging — SQS DLQ chain, sync + polled-async Lambdas, TTL'd DynamoDB

Single-resource scanners flag each resource against a checklist. audytx reads how the resources connect first.

Single-resource scanner

aws_lambda_function.chirp_api
Lambda DLQ missing

aws_lambda_function.chirp_outbox_worker
Lambda DLQ missing

aws_dynamodb_table.chirp_request_log
point-in-time recovery not enabled

aws_sqs_queue.chirp_outbox_dlq
queue has no DLQ of its own

fires on every resource that fails a pattern 4 noise findings

audytx — same resources, with context

aws_lambda_function.chirp_api
suppressed — sync via API Gateway; a Lambda DLQ only fires on async invokes, so it would never receive an event

aws_lambda_function.chirp_outbox_worker
suppressed — polled-async via SQS event-source mapping; failures handled by the queue's redrive_policy, not a function DLQ

aws_dynamodb_table.chirp_request_log
suppressed — TTL configured for ephemeral request logs; PITR is mismatched for data that self-expires

aws_sqs_queue.chirp_outbox_dlq
suppressed — this queue is the dead-letter queue; requiring it to have its own DLQ is infinite regress

each suppression carries its reasoning 0 noise findings

Multiply this across DLQ identity, Lambda invocation graphs, encryption variants, data lifetime, network exposure, IAM trust/policy reachability, tag environment and IMDSv2 inheritance (the context-reasoning axes in the live engine; live count on /status), and you get the 33-vs-1,193 gap in Table 2.

Open beta

Start free during the open beta

Install the GitHub App and audytx runs this exact engine on your own pull requests — free on every repo, public or private, no feature gates.

Install the GitHub App Read the docs

Methodology

Everything needed to reproduce the run, exactly as it was performed.

Tool	Version	How it was run
audytx	0.14.8	Live GitHub App scan → Code Scanning SARIF (version-verified)
Checkov	3.2.520	pip install · `checkov -d <dir> --framework terraform -o json`
Trivy	0.71.0	`trivy config <dir> --severity HIGH,CRITICAL`
KICS	2.1.20	`kics scan -p <dir> -t Terraform`
Terrascan	1.19.9 (final release — archived Nov 2025)	`terrascan scan -i terraform -d <dir>`

Corpus: 28 AWS-Terraform repos/modules in the public audytx-testbed, each on a bench/<name> branch. 5-minute timeout per tool per corpus (timeouts = DNF). No suppression files for any tool. Scoring is scripts/score.py (Python-3 stdlib only) — given the same inputs it produces a byte-identical scorecard every run. TP matching: a finding counts once per ground-truth path if its file and resource/rule reference that path; extra findings on the same path count as FP (conservative for audytx).

The trade-offs, measured

This is a benchmark, not a sales sheet. Here's exactly what audytx trades away:

Checkov has more raw coverage; here's the exact accounting. We mapped every one of Checkov's 432 AWS Terraform checks (pinned snapshot, Checkov 2e0ab64) against audytx's actual checks. audytx covers 138 of them (about a third) and deliberately doesn't cover 294. If you want breadth-first "tell me everything potentially wrong," Checkov has more checks. audytx's catalog is a curated set focused on patterns it understands deeply enough to reason about — depth over breadth, by design.
- What audytx genuinely doesn't flag (the 294): mostly breadth-tail services and one-off encryption variants — Amazon MQ, DocumentDB, DMS, AppSync, MemoryDB, Timestream, QLDB, Kendra, Connect, Network Firewall, snapshot-copy CMK checks, and similar. Real concerns, just not yet in a catalog we can reason about with precision.
- Where audytx covers the same check but suppresses Checkov's false positive (five checks, the whole point of the tool): the missing-DLQ check (suppressed on synchronous-only Lambdas), Redshift encryption (suppressed when the provider v6 default already encrypts), Secrets Manager rotation, DynamoDB point-in-time recovery (suppressed on TTL-expiring tables), and load-balancer deletion protection (suppressed on non-production, env-tagged resources). Checkov flags all of these unconditionally; audytx flags them only where they are real.
The full per-check mapping is maintained in-repo and regenerable against a newer Checkov snapshot.
Read the IAM precision number in context. On the deliberately-vulnerable iam-vulnerable corpus, audytx fires 135 HIGH findings for 31 paths. Even accounting for legitimate secondary detections and the corpus's own FP-test fixtures, that's a lot of alerts. Appropriate for a corpus that is wall-to-wall privesc, but the low-volume story is Table 2.
Clean-module false positives: 27 (v0.4.1) → 33 (v0.14.8). Investigating that rise surfaced a real bug: AWS_OPS_010 (public Lambda Function URL) had inverted match-logic and fired on phantom module-synthesized URLs. Fixed in v0.14.8, which removed 15 of them. The remaining handful are newer checks (EKS secrets-encryption, deprecated Lambda runtimes) correctly flagging real issues in the modules' own examples/ code. Net, audytx is again the lowest-false-positive tool of the five (33), ~36× below Checkov.
The ground truth was authored by us. The iam-vulnerable path list follows directly from BishopFox's upstream docs and audytx checks weren't tuned against it, but it's our scoring file. The unmatched-findings audit is published for independent checking.
Single run, AWS-only. Each tool was scanned once (Terrascan in particular shows timeout variance on large corpora), and the whole corpus is AWS Terraform — this says nothing about multi-cloud, which audytx deliberately does not cover, nor about audytx's CloudFormation support, which shipped after this benchmark was pinned and has no measured numbers yet.

Reproduce it yourself

The corpus, the ground truth, and the scorer are public. You don't have to take our numbers on faith.

# 1. Clone the public benchmark corpus
git clone https://github.com/victorsinha/audytx-testbed
cd audytx-testbed

# 2. Run any competitor on a corpus (example: Checkov on a clean module)
checkov -d corpus/terraform-aws-rds --framework terraform -o json | jq '.summary'

# 3. audytx numbers come from the live Code Scanning SARIF on each bench branch
gh api "repos/victorsinha/audytx-testbed/code-scanning/analyses?ref=refs/heads/bench/terraform-aws-rds" \
  --jq '[.[] | select(.tool.name=="audytx")] | sort_by(.created_at) | last | .id'

# 4. Re-score everything deterministically
python3 scripts/score.py results ground-truth

audytx vs Checkov

audytx vs Checkov: AWS Terraform security scanner comparison

The short version: both tools deliver full recall on the 31 documented IAM privilege-escalation paths. audytx does it at the highest precision of any tool with full recall, and fires 36× fewer false positives than Checkov on clean production modules (33 vs 1,193). Checkov has more raw check coverage. If your team muted its scanner because of the noise, audytx is the Checkov alternative built for exactly that problem; if you want maximum breadth and can absorb the triage work, Checkov delivers more checks.

Choose audytx when

Your team has turned off or started ignoring another scanner due to alert fatigue
You need 100% IAM privesc recall AND low noise (in this benchmark, audytx is the only tool that delivers both)
You want the reasoning behind each suppressed finding, not a bare pass/fail
You use AI coding agents and need an MCP server for pre-PR checks
You want free PR comments without a Bridgecrew account or API key

Choose Checkov when

You need maximum check breadth: X-Ray tracing, code signing, function-in-VPC, TLS version pinning, reserved concurrency — Checkov has these, audytx doesn't yet
You're already on the Bridgecrew/Prisma Cloud platform and want native integration
You run multi-cloud (audytx is AWS-only by design; its CloudFormation support is live on the MCP server, with the GitHub App surface next)
You want a broad "tell me everything possibly wrong" sweep rather than a high-precision review

Key numbers: IAM privesc recall — audytx 100%, Checkov 100%. False positives on 21 clean modules — audytx 33, Checkov 1,193 (36×). IAM precision — audytx 23%, Checkov 12% (2× at the same recall). Alert volume on iam-vulnerable — audytx 135, Checkov 269 (half). Full data in Tables 1–2 above. Dedicated page: audytx vs Checkov →

audytx vs Trivy

audytx vs Trivy: Terraform IaC scanner comparison

Trivy is a multi-purpose security scanner (containers, images, SBOMs, IaC), and that breadth is real. Its Terraform coverage focuses on common misconfigurations and has 0% recall on IAM privilege-escalation paths: Trivy fires 7 HIGH findings on iam-vulnerable, none of them correct privilege-escalation detections. audytx has 5× fewer false positives on clean modules (33 vs 175) while detecting all 31 IAM privesc paths Trivy misses. For teams scanning AWS Terraform specifically, that makes audytx the deeper Trivy alternative.

Choose audytx when

IAM security is a priority — Trivy has no IAM attack-path detection
You want cross-resource reasoning and context-aware suppression
You're Terraform-on-AWS focused and want depth over breadth
You need MCP server integration for AI coding agents

Choose Trivy when

You need a single tool covering containers, images, SBOMs, and IaC together
You scan multiple cloud providers (audytx is AWS-only; on CloudFormation both tools now scan it — audytx via its MCP server)
You want Kubernetes manifest and Helm chart scanning alongside Terraform
You need offline / airgapped scanning with self-contained binaries

Key numbers: IAM privesc recall — audytx 100%, Trivy 0%. False positives on 21 clean modules — audytx 33, Trivy 175 (5×). Trivy fires 7 HIGH findings on iam-vulnerable; all 7 are false positives (0 true positives). Dedicated page: audytx vs Trivy →

audytx vs KICS

audytx vs KICS: Terraform security tool comparison

KICS (Keeping Infrastructure as Code Secure, by Checkmarx) scores close to audytx on clean-module false positives (34 vs 33) — but reaches that only by detecting 3% of IAM privilege-escalation paths (1 of 31) versus audytx's 100%. KICS trades recall for precision. audytx achieves both: the lowest false-positive count and full IAM attack-path coverage.

Choose audytx when

You need both low false positives AND comprehensive IAM privesc detection
You want each suppressed finding shown with its rationale, not just fewer alerts
You want a GitHub App (install in 60s, no CI step) instead of a CLI tool
MCP server support for AI coding agents matters

Choose KICS when

You need multi-cloud support: KICS covers Azure, GCP, Kubernetes, Docker, Ansible, CloudFormation
You're already on the Checkmarx platform for SAST and want a unified tool
IAM attack-path detection is not a priority and low alert volume is
You prefer a self-hosted CLI with no external calls

Key numbers: IAM privesc recall — audytx 100%, KICS 3% (1 of 31 paths). False positives on 21 clean modules — audytx 33, KICS 34 (audytx the fewest of any tool). KICS achieves low noise by missing nearly all IAM attack paths; audytx achieves both.

audytx vs Terrascan

audytx vs Terrascan: Terraform static analysis comparison

Status first: Tenable archived the Terrascan repository on 2025-11-20: "This project is no longer maintained. The repository is archived and no further updates, issues, or pull requests will be accepted." Its final release is the 1.19.9 we benchmarked; the results below are retained for reference and remain reproducible, but Terrascan is no longer a tool to adopt. In the benchmark, Terrascan timed out on the iam-vulnerable corpus (5-minute limit exceeded) and produced 454 false positives on 21 clean modules — 14× more than audytx. A single module (terraform-aws-s3-bucket) triggered 348 HIGH alerts in a mass-rule blowup, 77% of Terrascan's total clean-module count.

Choose audytx when

Scan time reliability matters — Terrascan timed out on large module graphs
You need IAM privilege-escalation path detection (Terrascan DNF'd on this corpus)
14× lower false-positive volume is a meaningful team-productivity gain
You want a GitHub App with PR comments and SARIF upload, not a local CLI

If you're on Terrascan today

The project is archived — no further updates, rules, or security fixes will ship
Its former strengths (multi-cloud breadth, OPA/Rego custom policies, airgapped scanning) are covered today by the maintained tools above — Checkov, Trivy, KICS
For the AWS-Terraform slice specifically, this page is the migration comparison: the same corpus, scored the same way

Key numbers (final release, 1.19.9): IAM privesc recall — audytx 100%, Terrascan DNF (timeout on iam-vulnerable corpus). False positives on 21 clean modules — audytx 33, Terrascan 454 (14×). Terrascan's 348 alerts on a single module suggest a rule-volume issue on larger module graphs. These numbers are now frozen: the project's archive means they can no longer change. Migration guide: audytx vs Terrascan →

What changed in the market

A benchmark page that pretends the field is frozen isn't worth much. Here's the status of the IaC security tools on this page, and of three capabilities announced since the benchmark ran.

Terrascan is archived. Tenable archived the repository on 2025-11-20: "This project is no longer maintained. The repository is archived and no further updates, issues, or pull requests will be accepted." The 1.19.9 we benchmarked is its final release. We keep its rows because the pinned results stay reproducible — but nothing on this page should be read as a recommendation to adopt it.

tfsec is in maintenance mode. tfsec never appeared in this benchmark, but prospects ask about it: per its own repository, "Tfsec is now part of Trivy" and Aqua's "engineering attention will be directed at Trivy going forward." Its Terraform engine lives on inside Trivy — which is the row we benchmark. The two migration paths for tfsec users are laid out on the tfsec page.

Checkov's graph is detection, not suppression. Checkov builds an in-memory graph of resources and connections, and its composite policies use it well — to find more cross-resource misconfigurations. What the graph doesn't do is the inverse: use context to prove a fired finding benign and suppress it with a written rationale. Suppression in Checkov remains manual (checkov:skip comments, skip-lists). The 33-vs-1,193 clean-module gap in Table 2 is that difference, measured.

AI triage is probabilistic; context suppression is deterministic. Orca's AppSec Triage Agent (announced March 2026) uses AI to validate findings and surfaces confidence levels with its dismissals. That's a real advance for alert fatigue, and an approach we respect. It's also a different contract: a probabilistic model assessing likelihood vs a relationship graph proving a property. When audytx suppresses a finding, the rationale is a verifiable statement about your own HCL ("nothing invokes this Lambda asynchronously, so no DLQ is needed"), never a confidence score. Deterministic means the same input produces the same verdict, every run, and you can check the reasoning yourself.

Trivy has an MCP server too. trivy-mcp (v0.0.x) is a plugin you install into a local Trivy CLI; it exposes scanning to VS Code, Cursor, JetBrains, and Claude Desktop. It's a sensible design for local scans. The audytx MCP server is a hosted endpoint (nothing to install or keep updated) and goes past scan-and-report: an autofix loop with soundness gates, dry-run previews, a context graph the agent can query, and baseline parity with your repo's PR checks. If your agent writes Terraform, the difference is whether it gets a scanner or a reviewer.

Competitor status last verified 2026-07-06 against each project's own repository or announcement. Benchmark numbers are pinned to the versions and dates in the Methodology section and are re-run each milestone, not silently.

Frequently asked questions

The short answers, drawn from the benchmark numbers above.

Is audytx a good Checkov alternative?

Both reach full recall on the 31 documented IAM privilege-escalation paths; audytx does it at the highest precision of any tool with full recall and about 36× fewer false positives than Checkov on clean production modules.

Is audytx a Trivy alternative?

Yes. Trivy is a broad multi-purpose scanner but has 0% recall on IAM privilege-escalation paths, firing 7 HIGH findings on the iam-vulnerable corpus with no correct privilege-escalation detections. audytx detects all 31 paths and logs 5× fewer false positives on clean modules (33 vs 175), making it the deeper Trivy alternative for AWS Terraform.

How is audytx different from Checkov?

Checkov runs more raw checks but suppresses false positives only manually (checkov:skip comments). audytx pre-computes cross-resource relationship graphs and suppresses findings that context proves benign, showing the rationale — which is why it fires 33 clean-module false positives to Checkov's 1,193 (about 36× fewer) at the same 100% IAM privilege-escalation recall.

See your own numbers

One click to install. Free on every repo, public or private — no feature gates.

Install audytx →