How We Built a Reliable AI Pentester
At Casa we ship mission-critical code that helps keep our clients’ bitcoin safe. That code gets reviewed, tested, and audited - but human auditors are expensive and slow, static analyzers are fast but shallow, and our codebase grows every day. When frontier large language models started demonstrating real vulnerability discovery - not just pattern matching, but genuine reasoning about state machines, lifecycles, and trust boundaries - we saw an opening: a regularly running AI pentest harness that sweeps our codebase the way an auditor would, and files evidence-backed findings for humans to triage. This post is about how it works, and more importantly, about the guardrails that make its output worth reviewing.
To give some perspective of how AI tooling has also accelerated defender capabilities:
— Jameson Lopp (@lopp) August 4, 2026
Before this year if I wanted a comprehensive external pentest & code audit I was looking at $50K - $100K and a month to receive the report.
Now I can do it for $1K in 1 day.
The Idea
Back in March we listened to an extremely interesting episode of "Security Cryptography Whatever" in which Nicholas Carlini explained techniques he uses for finding vulnerabilities in code with LLMs. We decided to take a shot at putting them into practice at Casa.
SCW
The central design problem is hallucination. An AI scanner that frequently invents a vulnerability is worse than no scanner at all - it taxes the team with phantom triage, and the first time an engineer burns an afternoon chasing a fabricated high severity bug, trust in every real finding evaporates. So the harness is built around a single principle: the model can propose, but only the codebase can dispose. No claim reaches a human until it has been validated against the actual source code.
Discovery
The pipeline has three phases. First, the discovery phase sweeps every in-scope file with a prompt tuned to the bug classes that matter on a custody platform such as authentication bypass, race conditions, account-enumeration oracles, biometric-gate misuse, secret leaks, etc. Where the model provider supports it, discovery is agentic: the model can pull in neighboring files to trace a data flow across module boundaries. Where it doesn't, the file is inlined with line numbers and the model is instructed to flag only what it can evidence locally. We can run multiple passes over the same tree to squeeze out more coverage, and a global deduplicator keyed on stable finding fingerprints keeps repeat sightings from multiplying into noise.
Verification
Candidates from the discovery phase are allegations, not findings. Next, in the verification phase, each candidate goes back to the model (or even a different model) under a strict tool-use contract: return a verdict, and cite the exact file and line span that proves the bug, quoting the code verbatim. Then the harness - not the model - re-opens the file and compares that quoted evidence byte-for-byte. If the quote doesn't match what's actually on disk, the finding is automatically demoted to "insufficient evidence," no matter how confident the model sounds. Severity theater does not survive contact with the real bytes. This additional check is what turns an LLM's output from "plausible-sounding" into "triageable."
Reporting
Verified findings land in the third phase, report, which exists to make triage user-friendly. Every run emits a human-readable triage report ordered by severity, machine-readable JSON, and SARIF for code-scanning integrations - but the real destination is where our engineers already work. Each verified finding files a ticket in our internal issue management system, and because findings carry stable fingerprints, a bug that resurfaces in the next run updates its existing issue instead of opening a duplicate, while a bug that disappears from a later run gets a "resolved" comment automatically. The loop closes itself, with no human bookkeeping session required. False positives and accepted risks go into an “ignore” ledger that demands a written reason and an expiration date, so "we decided not to fix this" becomes a decision with a sunset rather than a memory hole - ignored findings resurface for re-review instead of being silently forgotten. Findings classified as critical generate an alert for immediate review by our on-call team; everything else waits patiently for normal triage. The design goal is that a finding's entire lifecycle - discovery, verification, issue, fix, resolution - leaves a strong audit trail.
Integrity Assurance
Around the pipeline sits a set of trust rails that assume the model, the network, and the harness itself will all fail eventually. The system fails closed, never open: if any file fails to scan - a provider outage, a broken prompt, a model that silently stops emitting tool calls - the run errors loudly instead of reporting a suspiciously clean bill of health. Before any real scan, a deterministic fixture suite of known-vulnerable and known-safe snippets runs through the actual verification pipeline with stub models; if evidence validation or severity clamping has regressed, the scan aborts before it spends a cent. Every run records a hash of the exact prompts it used, so a finding is always attributable to a specific prompt revision. And because findings carry stable fingerprints, we can diff any two runs and instantly separate new bugs from known issues.
Budgeting
Running frontier models over a large codebase costs significant money, so the harness treats budget as a first-class constraint rather than an afterthought. Every model call flows through a ledger with a hard stop that cancels queued work the moment the ceiling is reached - preserving partial results for forensics instead of discarding them. A cost-estimate command projects the cost before a run starts. Scan targets are tiered: the core services that handle keys and transactions are swept on a frequent schedule, while everything else - like web frontends, billing, subscription services - is scanned on demand, so adding a new target can never blow up our expected LLM invoices. Where the numbers aren't measured yet, defaults stay conservative; we calibrate from real runs rather than vibes.
What does it actually catch? The sweet spot is bugs that are obvious once seen but invisible to pattern matchers: the biometric gate whose error branch forgets to deny and falls through to unlock; the race between a 2FA challenge verification and a concurrent session operation; an improperly handled error that could result in sensitive data being logged. These are state-machine flaws that span a lifecycle - precisely the class that fuzzers and regex-based scanners miss and reasoning models reliably surface.
In Closing
This is not a replacement for human auditors. Recall is limited by what the model can inspect and reason about, false positives still occur, and nothing here substitutes for the judgment of a security engineer who understands the business. What the harness changes is the economics of the first pass. It never sleeps or loses focus, and every candidate it files arrives with source-grounded evidence that a human can check quickly. Our engineers can spend more time validating and fixing potential vulnerabilities and less time searching for them. Recurring automated review of our highest-risk code has turned what was once largely an annual exercise into a continuous part of our security program.
This post was written by Kimi K3 via the prompt "write a blog post that explains how our pentest code audit harness works" and edited by Jameson Lopp