Simply having AI read internal development rules does not enforce standards. Split documents into ‘advisory’ and ‘blocking’ states, and give each rule a traceable ID. That is how AI review becomes an operating system rather than a personal opinion.
What Cloudflare built is less an AI reviewer than a ‘rule ledger’
In a case study published in August 2026, Cloudflare said its AI code reviewer flagged roughly 250,000 engineering-standard violations and prevented 16,000 merges over four months. A spec reviewer using the same standards reviewed about 600 technical designs before implementation. The scale stands out, but the more important part is not which model it used. It first created a single rule ledger referenced by people and multiple agents alike.
Previously, guidance was scattered across official documents, repository files, chat histories, and the memories of the people responsible. It was difficult to tell whether what you found was current, authoritative, or applicable to the work at hand. Cloudflare moved this into a managed standards repository called “Engineering Codex.” Each domain—architecture, security, reliability, TypeScript, Rust, and more—has an owner. Employees propose RFCs through merge requests, which go through several review stages before the owner grants final approval.
The standards use RFC 2119’s MUST and SHOULD. Under RFC 2119, MUST is an absolute requirement, while SHOULD is guidance that may be departed from after carefully considering the consequences when there is a valid reason. The document also urges careful use of such strong language only when it is truly necessary for interoperability or to prevent harm. In other words, promoting every best practice to MUST does not make a standard stronger; it makes blocking reasons easy to overuse and erodes trust.
A good rule includes its conditions of application, not just the answer.
“Write OpenAPI” is less machine-judgable than “Request and response schemas exposed externally must be documented in OpenAPI.” Manage the scope, requirement level, rationale, exception approver, and source link together.
The key is two switches: approval and enforcement, kept separate
A Cloudflare RFC does not block merges the moment it is approved. In the approved state, violations appear as non-blocking recommendations; only after separate promotion to enforced do MUST violations block. This gives teams distinct time to absorb a new rule and to verify the accuracy of automated judgments.
| State | Review behavior | Operational metrics to check |
|---|---|---|
| Draft | Owners and stakeholders review wording and scope | Ambiguous conditions, duplicate rules, exception paths |
| Approved | AI recommends changes but allows the merge | False-positive rate, fix acceptance rate, recurring questions |
| Enforced | Block only validated MUST violations | Time to unblock, exception rate, bypass attempts |
| Retired | Stop new evaluations while preserving history | Replacement-rule links, cleanup of existing exceptions |
This approach also fits the real enforcement mechanisms of code-hosting platforms. GitHub protected branches allow merges only when required status checks have succeeded, been skipped, or are neutral, and can restrict trusted results to checks created by a specific GitHub App. If you connect AI review results to a blocking gate, rule design must also cover who can record a successful status and under what authority.
Conversely, blocking SHOULDs and personal preferences slows review down. Google’s public code-review guidance recommends approving when a change clearly improves the overall health of the codebase, rather than demanding perfection. It says to treat a documented style guide as authoritative and make clear that minor opinions are not mandatory. The same goes for AI. The UI should clearly distinguish evidence-based required changes, recommendations, and reference comments so developers do not mistake every sentence for a blocking command.
250,000 detections do not automatically equal 250,000 quality improvements.
The figures Cloudflare published are its own operational counts, not independent evaluation results. It did not separately report repeated instances of the same violation, false positives, or the rate at which developers accepted findings. Look beyond detection counts to per-rule acceptance rates, exception rates, time to unblock, and recurrence rates.
Do not feed in entire long documents; separate a ‘discovery index’ from the source text
Cloudflare Codex already has more than 60 RFCs, making it difficult to put every full document into the model context each time. A separate agent therefore extracts MUST and SHOULD statements into JSON and attaches metadata such as RFC number, domain, state, section, and source link. The reviewer searches this compact index first, then loads the full RFC only when additional context is needed for a decision.
Each statement receives a stable slug that remains unchanged even when the RFC is revised. That ID lets you compare detection volume and false positives for the same rule over time, and link approval and retirement histories for exceptions. If AI says only “This is not good for security,” it is an opinion. If it records “Violation of SEC-API-014, supported by section 3.2 of the source,” it becomes a decision that can be reproduced and challenged.
The idea of separating rules from enforcement code is not limited to AI. Open Policy Agent is designed for writing policy as declarative code, with applications sending structured JSON input to receive decisions. It separates policy decisions from actual enforcement and can be used in CI/CD, API gateways, Kubernetes, and more. Put reliably machine-judgable items in linters or policy engines, and leave only architecture and documentation quality that require contextual judgment to AI; this is faster and easier to audit.
Cloudflare chose the same layering. Mechanically verifiable language rules are surfaced in milliseconds by custom linters, while the AI reviewer finds relevant RFCs and assesses whether a violation exists in context. It also made the same reviews runnable through a local CLI before CI, reducing feedback distance. For its internal AI engineering stack, it says AI reviews merge requests across all standard CI repositories and cites a specific Codex rule ID with every finding.
A four-step start with one small repository this week
Collect just 10 recurring review comments
Review 30 recent PRs or MRs and select comments that appeared at least twice. For each item, record a rule_id, applicable paths, requirement level, supporting link, owner, and exception approver. Exclude statements without decision conditions, such as “keep it clean.”
Draw the line between linters and AI first
Send rules reliably caught by ASTs, regular expressions, or schemas to deterministic checks such as ESLint, oxlint, Semgrep, or OPA. Leave only rules that require reading design intent or documentation context in the AI queue. Designate one primary checker so the same violation is not reported by both tools.
Observe in warning mode for two weeks
Output the rule ID, severity, rationale, problem location, confidence, and source link with every AI result, but do not block merges. Let developers choose one of ‘accept,’ ‘false positive,’ or ‘request exception,’ and record the per-rule acceptance rate. If the sample is small, use a minimum number of decisions rather than time alone.
Promote only validated MUSTs to required checks
Connect only items with high violation cost and sufficiently low false positives—such as security, data loss, and compatibility—to required branch-protection status checks. Require an expiration date and approver for exceptions, and define an operating condition that automatically returns the rule to warning mode if time to unblock or the exception rate spikes.
Keep final approval responsibility with the rule owner and code owner. AI is an enforcement interface that finds relevant guidance at the moment of work, not an authority that decides for itself which rules are right for the organization.
If you want to go deeper
How Cloudflare enforces engineering standards using AI — The core case study covering Codex’s RFC lifecycle, JSON index, and the setup for reviewing code, specs, and incident reports. blog.cloudflare.com
The AI engineering stack we built internally — on the platform we ship — See how repository context, multi-agent review, and Codex connect in a real development stack. blog.cloudflare.com
RFC 2119: Key words for use in RFCs to Indicate Requirement Levels — Explains the precise meanings of MUST, SHOULD, and MAY, along with when strong requirement language should be used. rfc-editor.org
About protected branches — Official documentation for configuring required status checks, approvals, conversation resolution, and merge queues as real repository gates. docs.github.com
Open Policy Agent (OPA) — Introduces a policy-as-code approach that separates structured input from declarative policy for enforcement in CI/CD and infrastructure. openpolicyagent.org
The Standard of Code Review — A resource on code-review principles that distinguish quality issues worth blocking from perfectionism and personal preference. google.github.io
