Where Claude’s Formal Verification Work Makes Its Case—and Where It Stops

🔍 Read the full analysis: Where Claude’s Formal Verification Work Makes Its Case—and Where It Stops on ThorstenMeyerAI.com

Buying for a business?Offer from Amazon

Get business pricing on office and shipping supplies

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

Anthropic’s Frontier Red Team and collaborator Theorem reported on October 22 that Claude generated machine-checked proofs for 239 entry points in 123 open-source libraries. The proofs establish a specific result: for the modeled, compiled RISC-V programs, the entry points do not crash on inputs under 64 MB, subject to the team’s assumptions and specifications. The work matters because it applies formal verification at scale to production software, while its scope remains narrower than a claim that the libraries are bug-free.

The team started with existing OSS-Fuzz test entry points, which provide small ways to feed inputs into library code. It compiled each library and test into a RISC-V executable, added AddressSanitizer instrumentation, and used formal models of RISC-V instructions and Linux processes to define what the test should do. Claude then wrote proofs in Rocq, a proof assistant whose checker verifies whether a proof follows from its specification.

Under the stated property, every input below the 64 MB limit must avoid a crash, including an AddressSanitizer memory error or failed assertion. If Claude’s proof attempt failed, the failure could point to a possible defect; the team could patch the code, regenerate the specification and try to prove the revised program. Anthropic says the project ran over a few weeks and produced about 180 million lines of Rocq proof. A typical proof was about 600,000 lines and covered a binary with a median length of 18,000 RISC-V instructions.

Anthropic said the process found hundreds of bugs, including cases that ordinary fuzzing was unlikely to reach. Examples in the report include a decompressor counter underflow triggered by a rare layout in an input around 20 MB, and an image-decoder out-of-bounds read requiring a 64 MB input and more than 2 GB of memory. The report also describes a bug requiring a sequence of calls across two file handles, a rediscovery of CVE-2026-56132 in libexpat, and a file-loading restriction that was not always enforced. These examples are attributed to the project team, not independent confirmation of the full results.

Proofs Reach Beyond Fuzzing

Fuzzing tests many inputs, but it cannot establish that every possible input or execution path has been covered. A successful proof for a precisely defined property can address that gap: if the model and specification match the program, then the tested entry point cannot violate that property under the conditions proved. That can matter for security defects triggered only by large, unusual, or carefully structured inputs.

The project also tests whether AI can reduce the labor that has kept formal verification uncommon. Anthropic points to seL4, a microkernel whose verification effort it says took about 20 person-years, compared with two person-years to build it. In this project, machine checking means people do not need to inspect every line of proof, though they still need confidence in the specifications, models and checker. The results suggest a possible change in verification costs, but the report does not establish that the approach is already economical for routine use: proofs can consume hundreds of millions of tokens.

The work also illustrates why the result should be read as a bounded guarantee, not a general safety certificate. A proof of “this entry point does not crash under these modeled conditions” is useful, but it does not show that the software behaves as intended in every respect or contains no exploitable logic errors.

How Tests Became Specifications

Traditional fuzzing generates and mutates inputs, then watches for crashes and other failures. It can find defects efficiently, but a fuzzer’s conclusions depend on the inputs it reaches and the code paths it exercises. The formal-verification effort took a different route: it treated an existing fuzz test as the specification and asked whether the compiled program could ever crash when that test received an input within the defined size limit.

That approach avoids writing a complete description of what an entire library is supposed to do. Instead, the proof covers the behavior reachable from a particular fuzz-test entry point. The team says the trusted basis is much smaller than the generated proof corpus: it includes the formal models, exact theorem statements and Rocq proof checker. Anthropic says it is publishing proofs that did not require code patches, previously unpublished trusted components, and specifications for fuzz tests it could not prove.

Anthropic frames the work against the risk that AI systems may help find and exploit software weaknesses, and argues that testing alone may not suffice. Its report says even heavily tested software can have vulnerabilities; the formal-verification project is an attempt to prove the absence of selected classes of failure rather than rely only on accumulated testing.

““to our knowledge, the first large-scale use of AI to apply formal verification to production software at the assembly code level””

— Anthropic, describing the project’s scale

The Guarantees Have Boundaries

The proofs cover single-threaded, static RISC-V binaries linked against musl, according to the source material. They do not directly establish the same properties for commonly used x86 or ARM builds linked against glibc. The result also depends on the formal models and the theorem statements accurately representing the program and the intended test.

The no-crash property does not rule out every kind of bug. Some logic errors may not crash, and the project’s examples include a file-loading restriction that was not consistently enforced. The report describes additional properties arising in particular proofs, such as a round-trip condition for an LZO compression test, but that does not mean every functional requirement of a library was verified.

It is also unclear from the supplied material how the reported hundreds of bugs break down by severity, how many were independently reproduced or fixed, and what the full computational and token costs were across the project. Anthropic says proof attempts can take hundreds of millions of tokens and that newer models and reuse of shared proofs can reduce requirements, but it gives no complete cost comparison here.

Publication and Cost Tests Ahead

Anthropic says it is publishing the proofs for cases that did not require patches, trusted components that were not already public, and specifications for tests the system could not prove. Those materials give researchers and developers a route to inspect the methods and attempt to reproduce or extend the work. The company’s account does not specify a publication schedule in the source material.

The next practical questions are whether outside reviewers can validate the proofs and models, whether patches for the reported defects reach affected projects, and whether the method can be adapted to the platforms most users run. Cost will also shape adoption: Anthropic says newer models and shared proofs can lower token use, but the current results show that generating large proofs remains resource-intensive.

For now, the report establishes a substantial demonstration of AI-assisted verification for a bounded set of compiled programs. It does not establish broad coverage across software versions or architectures, nor does it show that the same approach can economically verify complete applications.

Key Questions

What did Claude prove?

According to Anthropic and Theorem, Claude generated Rocq proofs that 239 fuzz-test entry points in 123 libraries do not crash on inputs under 64 MB, for the modeled compiled programs and under the specified assumptions.

Does this mean the libraries are bug-free?

No. The proofs establish particular properties for particular entry points and builds. They do not rule out every functional, security or logic defect, or guarantee that other platforms and versions behave the same way.

How is this different from fuzzing?

Fuzzing tries many generated or mutated inputs and checks for failures. A proof attempts to show that the stated property holds for every input in the modeled range, rather than only for the inputs a fuzzer happened to test.

Which platforms do the reported proofs cover?

The report describes single-threaded, static RISC-V binaries linked against musl. It does not directly establish equivalent results for x86 or ARM builds linked against glibc.

What remains to be checked?

Independent review and reproduction, the status and severity of reported bugs, applicability to common production builds, and the real cost of generating and maintaining proofs remain open questions in the supplied material.

Source: ThorstenMeyerAI.com