Anthropic FlashProve: cryptographic provable inference for large AI

Anthropic researchers, alongside academic partners from EPFL, MIT, and other institutions, have shared significant progress in verifiable AI inference. The newly introduced system, FlashProve, addresses critical trust issues by allowing users to cryptographically verify that AI responses come from specific models.

• FlashProve is an open-source performant prototype for provable inference using zero-knowledge succinct arguments.
• Evaluated on a 400B-parameter mixture-of-experts model (Llama 4 Maverick) with 4-bit quantized weights.
• A single H100 GPU generates proofs at roughly 110 tokens/second with ~19 MB proof size per 1,024-token request.
• Source code, Lean formal proofs, and documentation are released under the Apache 2.0 license.

The Need for Provable Inference in AI

When interacting with an AI model today, users have no definitive way to know if the response truly originated from the claimed model or was silently swapped by a third-party router or malicious actor. Cryptographic proofs of inference solve this by letting providers commit to a model and mathematically prove that each response matches.

Using zkSNARK technology, a provider can attach a concise cryptographic certificate to the model's output. A user with limited computing power can quickly verify the correctness of the computation without ever gaining access to the private model weights, preserving intellectual property while ensuring absolute integrity.

FlashProve Performance and Technical Specs

Generating cryptographic proofs for massive neural networks is notoriously compute-intensive, but FlashProve demonstrates practical efficiency. For a 400-billion-parameter Llama 4 Maverick model, a single H100 GPU generates proofs at over 100 tokens per second.

The proof size remains manageable at approximately 19 megabytes per 1,024-token request using integer arithmetic during prefill. While this requires 70–95x the GPU time of standard non-verified execution via vLLM, it drastically outperforms previous open-source cryptographic verification frameworks.

Formal Verification and Open Source Release

To counter the challenge of auditing complex optimization decisions, the team provided top-level auditable statements written in Lean, a formal verification language. The release includes executable code, a fast C++ verifier, a provably correct Lean verifier, and interactive visualizers.

By releasing all materials under the Apache 2.0 license, Anthropic aims to accelerate community development. Provable inference promises powerful new tooling for independent AI evaluators, regulators, and businesses requiring rigorous compliance and security guarantees.