The tests are green, but merging still feels wrong
You gave a small change to an AI coding agent, and it changed more files than expected. Every test and lint check passed, but on closer inspection, it bypassed an existing data flow or even changed business rules you never asked it to touch.
In that situation, standard automated checks may catch syntax or type problems but fail to answer “Are we still building the product our team agreed on?” Code that runs and code that matches product intent are two different things to review.
Prelint focuses on that second question. It is a product-consistency review layer that compares product specifications, ADRs, and business rules in your repository with PR changes to find scope drift and violated decisions. It is not a replacement for linters, type checkers, or security scanners.
The difference is not bug detection, but the review standard
What Prelint evaluates is not the shape of the code, but the decisions the code was supposed to follow. According to its official documentation, it reviews product-spec violations, ADR compliance, business logic, and task-scope drift, while leaving code style and general security-vulnerability checks to dedicated tools.
| Review layer | Question it asks | Typical evidence |
|---|---|---|
| CI · static analysis | Does it build and pass tests? | Tests, types, lint rules |
| Security scanning | Are there known vulnerabilities or risky patterns? | Security rules, dependency data |
| Product-consistency review | Does it follow the agreed requirements and design? | Product specs, ADRs, business rules |
For example, imagine the documentation says, “The default payment term for every invoice is 30 days from the issue date,” but a PR changes the default to 60 days. The code may run perfectly, yet it conflicts with a documented business rule. That is exactly the kind of change Prelint aims to catch.
Do not start with installation; test one rule and one PR
The goal of your first review is not to document the entire repository. Add one rule that is currently valid, then confirm that a small PR violating it produces an evidence-linked finding. The currently public general onboarding path is a GitHub App, and you need permission for the organization or repository where you approve the installation.
- Authenticate with GitHub in the Prelint app. When installing the GitHub App, select only the repository you want to test rather than the entire organization. Prelint reads the code and PR data needed for review, but does not directly execute repository code.
- Add one currently valid rule to the repository in Markdown. For example, record in
specs/billing.mdthat “The default payment term for every invoice is 30 days from the issue date.” A separate configuration file is not required for the first review. - Open a small PR that changes the default payment term to 60 days. Make the change on a separate branch and convert it into a non-draft PR. Draft PRs, PRs created by bots excluded by default, and PRs with no relevant file changes may be skipped.
- Check the PR’s check-run summary and inline comments on the diff. The success criterion is that, when the change conflicts with the documentation, an evidence-linked finding is posted on the relevant line. For a change without a conflict, the result indicates there are no active findings for the current PR version.
- Fix the flagged value and push a new commit. Check whether the existing finding disappears in the updated result. Creating or reopening a PR, marking it ready for review, and pushing a new commit can trigger a review.
- Check Billing before expanding the trial. Each completed review costs $1, while failed, canceled, and timed-out reviews are not billed. You receive $10 in credits when signing up, and public-repository reviews are free, but a completed re-review after a new commit also counts as a separate review. Check auto-reload status and your monthly spending limit first.
Make the first test an intentionally clear conflict.
A case where a person can immediately judge the conflict between the documentation and the diff—such as changing 30 days to 60—is better than an ambiguous design principle. When a result seems odd, it is easier to tell whether the issue is the tool’s judgment or the original document’s ambiguity.
More documentation does not automatically mean better review
Prelint automatically indexes Markdown specifications and ADRs in the repository, but the context assembled for a single review is capped at 80K characters. It builds context according to priorities such as organization, project, and repository documents, and lower-priority material may be cut off when the limit is exceeded.
So rather than adding every past meeting note, it is better to make the status of decisions clear. State briefly and clearly what was decided, where it applies, whether it is still valid, and what the exceptions are. Documents that conflict with one another or do not indicate whether they have been retired confuse human reviewers as well as AI.
Review the documents and code that will be included in the review first.
Prelint does not execute repository code, but it reads code and PR data for review and processes them in an AWS environment. If you have sensitive files, check access scope, exclusion settings, and your organization’s data-handling standards before connecting the repository.
36.6% is not an effect size; it signals that the review scope changed
Prelint’s own study reviewed 56,706 PRs across 331 public repositories both with and without documentation. The flag rate was 13.3% without documentation and 36.6% with documentation included. It reported 979 items found only because context was available, while context suppressed 232 otherwise unnecessary warnings.
These results support the hypothesis that documentation can change what gets reviewed. But a higher flag rate does not mean actual defects or incidents fell by the same amount. The study was conducted by the product vendor, used Opus 4.6 for model grading, and did not publish its repository sampling, human-review process, or raw results in enough detail for independent reproduction.
When your team evaluates it, look at three things rather than the number of findings: whether a finding was actually valid, whether it identified a product decision a person might have missed, and whether the same violation disappeared after the fix. Only then does product-consistency review become a layer that helps merge decisions instead of merely generating more comments.
You can decide whether to adopt it after the first five cases
Prelint’s value is not in eliminating existing code review. It lies in adding one more question that compares documented intent while keeping tests and security checks in place.
Do not roll it out across the whole repository from day one. Start with one clear rule and five small PRs with unambiguous outcomes. Classify each finding as a valid violation, ambiguous documentation, or false positive, and the next action becomes clear. If valid violations recur, broaden the scope; if documentation issues are common, clean up the ADRs first; if false positives are frequent, it is safer not to use it as an automatic merge gate.
If you want to dig deeper
Documentation - Prelint You can find the official onboarding steps for connecting the GitHub App, indexing documents, and running your first review. prelint.com
How reviews work - Prelint Explains the review pipeline from PR events and context assembly through secondary verification and posting inline comments. prelint.com
AI Code Pulse — 56,706 PRs Analyzed Read the original source for flag rates by documentation condition and the study’s scope. Keep in mind as you read that this is vendor-run research. prelint.com
