How to cut false positives in AI code security review

Key takeaways

• Developer trust is fragile: if an AI security tool fills a report with unverified guesses, engineers stop reading the output altogether.
• Most AI false positives happen because a model flags a local function without checking whether upstream routers, middleware or type invariants already neutralise the input.
• Dokima separates vulnerability discovery from skeptical peer review, automatically gathering missing upstream files to disprove weak candidates before reporting.
• Separating severity (impact if true) from confidence (strength of evidence) lets teams focus immediately on proven, high-impact bugs.

Why do AI security tools generate so many false positives?

Traditional static analysis (SAST) is notorious for false positives because rigid syntax rules cannot see runtime context. Many engineering teams hoped large language models would immediately fix that problem. Instead, first-generation AI security bots simply replaced rigid rule noise with probabilistic speculation.

When you ask an LLM to find security bugs in a file or pull request, the model is helpful by default. If it does not have enough context to know whether a parameter is sanitised upstream, it tends to report a “potential vulnerability” anyway, hedged with phrases like “if user input reaches this function without validation…”

To a busy developer, a “potential vulnerability” that turns out to be impossible because of router middleware five files away is indistinguishable from a false positive. Once a tool sends an engineer on three wild-goose chases in a row, the tool gets muted.

The four main sources of false positives in AI code review

Understanding how to eliminate false positives starts with recognising why models make wrong calls on real repositories:

1. Missing upstream guards: The model inspects an internal helper or database store method in isolation and misses the authentication middleware, path-confinement helper or schema validator that every caller runs first.
2. Misclassified trust boundaries: The model treats local admin CLI flags, build scripts, test fixtures or trusted startup configuration files as if they were unauthenticated HTTP inputs from the public internet.
3. Framework and language defaults: The model flags SQL injection on a query builder that automatically parameterises inputs, or warns about XSS in a template engine that auto-escapes variables by default.
4. Prompt pressure to produce output: When a prompt asks a model to “find vulnerabilities in this module”, the model feels pressure to return a non-empty list even when the module is clean.

How adversarial peer review filters out weak candidates

The core architectural principle in Dokima is that the stage that hunts for candidate bugs is never allowed to publish directly to your report.

Instead, Dokima separates discovery from falsification across multiple dedicated stages:

1. Grounded discovery

Before any hunter stage runs, Dokima builds an architectural baseline of the repository: mapping real external entrypoints, breaking the codebase into components, and verifying explicit security assumptions. Hunters are given that context so they know the difference between a public HTTP handler and an internal test helper.

2. Skeptical Candidate Finding Peer Review

When a hunting stage spots a potential issue, it emits a structured candidate. A separate Candidate Finding Peer Reviewer stage examines the claim with an explicitly skeptical mandate:

• Does the cited code actually behave the way the candidate claims?
• Is there an upstream check, type constraint or configuration gate that blocks exploitation?
• Did the hunter trace a complete path from an untrusted entrypoint to the vulnerable operation?

3. Targeted context expansion (instead of guessing)

If the peer reviewer sees that a crucial piece of evidence is missing (for example, where a service struct is instantiated or how a route group is mounted), it does not guess or pass the burden to the human developer. It triggers an Information Gatherer or Red Flag Investigator stage to retrieve the relevant files from the repository and re-evaluates the candidate with the complete call chain.

4. Final validation

Candidates that survive peer review pass through the Finding Validator, which confirms concrete exploitability and real-world impact. Candidates that fail validation are discarded before dokima-report/ is generated.

Instead of telling the developer "this might be a bug if X is not checked upstream", Dokima goes and checks upstream itself.

Why separating severity from confidence matters

Many security tools conflate how bad a bug would be with how sure the tool is that the bug exists. A speculative remote-code-execution guess gets marked “Critical”, triggering false alarms, or gets downgraded to “Medium”, hiding a potentially catastrophic flaw.

Dokima tracks severity (Critical, High, Medium, Low) and confidence as distinct dimensions on every finding:

• Severity measures the security impact if an attacker exploits the flaw.
• Confidence reflects the completeness of the verified code trace from entrypoint to sink.

This separation allows engineering leads to filter immediately for High/Critical severity + High confidence findings that warrant immediate patching, while reviewing lower-confidence architectural edge cases when time permits.

Teaching the workspace once with persistent dispositions

Even when a code path is genuinely reachable, your team may decide that the behaviour is intentional for your deployment model (By Design) or handled by an external control such as a reverse proxy or network policy (Mitigated).

In Dokima, you can record a disposition (Confirmed, False Positive, By Design, or Mitigated) in the interactive terminal dashboard or CLI. Because Dokima anchors each finding to a deterministic fingerprint in .dokima/, that decision sticks across future sweeps and commits. Your team never has to triage the same false alarm twice.

Judge the signal-to-noise ratio for yourself in our unretouched 57-finding sample report, or start a 21-day free trial and run Dokima on your own codebase.