inkdex: an index that states facts, a scanner that judges them

TL;DR

• The engine states facts; one place turns them into verdicts. inkdex indexes a repository into a light code property graph and composes data flows across functions and files. inkdex-vuln reads that index and decides what is a finding.
• Fourteen releases, every benchmark re-run each time. OWASP Java went from 9.5 to 99.2 and Python from 11.9 to 90.2, and a release gate refuses any drop without a written reason.
• Precision first. A finding is proven only when the index shows the whole path. On RealVuln, 73.5 % of what we report on Python is a real vulnerability.
• The limits are real and we name them. RealVuln recall is 28.9 %, TypeScript resolution trails, and prototype pollution and ReDoS are young. See Limits.

Facts, never verdicts

inkdex is a local code index: a light code property graph stored in SQLite under .inkdex/ in the repository. It parses a codebase, resolves calls across files, records security facts (sources, sinks, sanitizers, validators, entry points, secrets, crypto) and composes data flows across functions and files. What it never does is decide that something is a vulnerability.

That decision lives in inkdex-vuln, which reads the index and turns facts into findings, each with a CWE, a severity and a verdict: proven when the index shows the whole path, verify when a human should look. Both are written in Rust, ship as single binaries and analyse code entirely on the user's machine.

Four rules that show in the numbers

1. Facts in the engine, verdicts in one place. A new finding class is a change to inkdex-vuln, not to the index, and every fact stays queryable for whatever comes next.
2. Fix the model, never tune a rule to a benchmark. When OWASP Python stores values in a plain dictionary used as a session, weak randomness stays at 0 rather than gaining a rule that only exists for that suite.
3. Every rule ships with a test. 244 taxonomy tests live in the rule files themselves and are checked when the rules load.
4. Precision first. proven means the index shows the whole path, from the source to the sink. Anything less is verify.

What it covers

The 33 frameworks run from Flask, FastAPI, Django and DRF to Express, Fastify, NestJS, Next.js and Remix, axum, actix-web, Rocket, Salvo and Poem, net/http, gin, echo, chi and fiber, Phoenix, LiveView and Absinthe, Spring MVC, JAX-RS and servlets, and job runners such as Celery and BullMQ. An entry point is where outside input enters: a route handler, a job, a socket.

On top of those facts, inkdex-vuln has about fifty finding rules: injections (SQL, NoSQL, command, argument, code, template, LDAP, XPath, XSS, header, log, prototype pollution, deserialization, XXE), path traversal and zip slip, SSRF, open redirect, authorization and trust boundary, denial of service (ReDoS, allocation, decompression, atom table, body limit, timeouts), ten secrets and crypto rules, settings (TLS, debug, cookies, CORS, WebSocket origin, CSRF, cleartext) and dependencies. SARIF output carries CWE tags, security-severity, code flows and stable fingerprints.

Resolving calls without a compiler

A data flow is only as good as the call graph under it. We measure call resolution against compiler-grade indexes of real repositories: of the calls such an index can resolve, how many inkdex resolves (the rate), and how many of those point to the same target (the precision).

Precision is 100 % everywhere but flask, and that is a choice: calls are typed by declared return types, never inferred ones. A wrong edge invents a path that does not exist, and a proven finding built on it is worse than a miss. The price shows on zod, a library built on heavy generic types: 72.8 %.

Flows composed at query time

The security pass computes one taint summary per function: which parameters reach which sinks, which reach the return value, which are cleaned on the way. Flows are not precomputed. They are composed when queried, with a call stack, so a value returned by a helper goes back to the call site that asked for it rather than to every caller of the helper. The same composition follows instance fields, closures, callbacks and constructors.

Composition is where a subtle bug would hide, so it has a reference: on every security fixture, the composer is checked against a brute-force walk of the whole graph, at depths 0, 1, 2 and 6. The results are identical at every depth tested.

Accuracy, on five benchmarks

No single benchmark says much on its own. Synthetic suites measure whether each class is modelled at all; deliberately vulnerable repositories measure precision on code written by people; real CVE fixes measure how far a data-flow engine reaches into the bugs that actually ship. We run all of them on every release.

OWASP Benchmark

Synthetic test cases, scored by the benchmark's own formula: true positive rate minus false positive rate.

On Java, ten categories reach 100 %: command injection, weak crypto, weak hash, LDAP injection, path traversal, insecure cookie, SQL injection, trust boundary, XPath injection and XSS. Weak randomness reaches 91.7 %, and no category has a single false positive. On Python, eight categories reach 100 % (command injection, code injection, deserialization, weak hash, open redirect, insecure cookie, SQL injection, trust boundary), then XSS 95.1, LDAP 93.8, path traversal 90.8 and XPath 88.2. Weak randomness is 0, on purpose: the suite keeps its values in a plain dictionary used as a session, and we left it alone rather than tune a rule to it.

RealVuln

140 real, deliberately vulnerable repositories, scored by RealVuln 3.1.0's own dashboard. Its headline metric is F3, which weighs recall three times as much as precision.¹

The grey part is large, and most of it is out of reach for any analyser that reads only code: the ground truth includes missing authentication, mass assignment, session invalidation and error disclosure. The red part is where we still have work, mostly in TypeScript and JavaScript.

SecBench.js

573 real vulnerabilities in npm packages, scanned in library mode: the parameters of exported functions are treated as input, since a library has no route handler.

Real CVE fixes

Reviewed advisories with a fix commit, scanned in library mode. A hit counts when the vulnerable version has a finding of the advisory's class in the lines the fix changed; a false alarm counts when the fixed version still has one there.

These scores are low, and they are expected to be low for every static tool: most of these advisories are logic, memory or protocol bugs, not data flows. We keep the benchmark precisely because it measures the reach of the approach rather than flattering it.

Internal bench and traps

Deliberately vulnerable apps plus one hardened real application, with every documented flow in scope and a set of safe pairs: code that looks like a vulnerability but is not. A safe pair that produces a finding is a trap hit.

Fourteen releases, one gate

Every release re-runs every benchmark. A gate refuses any headline number below the previous release unless a written, version-pinned reason accepts it, and the reasons are shown on that release's benchmark page. The chart below is what that discipline looks like over time.

1. 0.6.x, the OWASP plan. One release per vulnerability category (command injection, code injection, crypto, deserialization, hash, LDAP, path traversal, redirect, cookies, SQL, trust boundary, weak randomness, XPath, XSS, XXE), each taken as far as it goes in every language.
2. 0.6.6, interface dispatch. A call to an interface method now enters its implementations. That alone took OWASP Java from 81.6 to 89.6, with no new false positive.
3. 0.7.x, every language measured. SecBench.js, real CVE fixes for Go, Rust and Elixir, and the internal bench. Library mode arrives: the parameters of exported functions count as input.
4. 0.8.2, new classes. Prototype pollution, ReDoS (the pattern's worst case read from its text, with no regex engine dependency), Go and Rust denial-of-service facts, Elixir and Java expression sinks. SecBench.js went from 40.0 % to 53.8 % overall. RealVuln precision dipped, because its ground truth does not model those classes.
5. 0.8.3, application context. Browser code, admin routes, scripts, log formats, template escaping, flow and guard states. RealVuln precision went from 50.6 % to 54.4 %, with no true positive lost on any benchmark.

Making it fast without changing a finding

By 0.7 the engine was right more often than it was fast: the largest Go target took 35 minutes end to end. The 0.8 work had one constraint, that every finding stays byte-identical, and the benchmark suite checked it on every change.

None of it was a new algorithm. Each fix removed work the engine was doing for nothing:

• A name-driven alias lookup. The planner scanned every resolved symbol for each typed parameter. Looking aliases up by name instead took summaries on the largest target from 102 s to 1.3 s.
• One load per file. A file's resolved calls are loaded once, instead of one query per call.
• Memoized validator wrappers, and per-unit variable indexes, which turned a quadratic step linear.
• Flows composed once per scan, not twice.

0.8.1 then made fact extraction parallel. The security pass alone went from 16.7 s to 5.0 s on the large JavaScript framework (3.3×) and from 13.1 s to 4.1 s on the Kubernetes-ecosystem Go project (3.2×), with three changes: one read-only connection per thread, on 4 threads, since more doubled the CPU for no gain; glob patterns compiled once per thread (entry points: 10.4 → 1.3 CPU-seconds); and every tree-sitter query rule of a grammar compiled into a single multi-pattern query, so each file is walked once.

The engine, one operation at a time

criterion benchmarks, release build, on an Apple M1 Pro. The numbers that matter most for daily use are the incremental ones: nothing to do costs 3.5 ms, and editing one function body costs 58 ms, security facts included.

End to end on 10,000 files, a cold inkdex index takes 2.1 s, and a no-op one 30 to 40 ms, process start included, thanks to the stat cache.

History for almost nothing

inkdex keeps one snapshot per indexed commit. Stored naively, history is expensive; sharing each file's pass rows between the snapshots that contain the same file makes it nearly free. On 50 commits of a real Python project plus the working tree:

A hardened application, read twice

The benchmarks above are mostly code written to be vulnerable. The hardest test is the opposite: a production-grade Rust service on actix, already well defended, with sandboxed subprocesses, storage keys validated against traversal after every percent-decoding, and no advisory suppressions.

The fix was not a suppression. The model learned the check(x)? idiom, judged Result-returning validators by their body, and learned three more things the service did correctly: Debug-escaped log values, linear-time regex engines, and alphabets that are not secrets. Then the application went into the internal bench, with its 8 former false flows kept as traps. None has been hit since.

A false proven finding on a careful codebase is a bug in the model, not in the code. The trap stays in the bench so the bug cannot come back.

How it is built and tested

• Incremental equals full. Property tests check that an incremental update gives exactly what a full rebuild gives, for resolution and for the security pass.
• A reference for the hard part. The flow composer is checked against a brute-force walk on every security fixture, at depths 0, 1, 2 and 6.
• Stable identities. Finding fingerprints carry no line numbers, so a finding keeps its id when the code around it moves.
• Local by design. Code is analysed on the user's machine and never leaves it. Runs report counts, dependencies and findings (rule, place, fingerprint), never source code.
• Plain distribution. Two binaries of about 32 MB each, packaged as .deb and .rpm for amd64 on signed repositories, apt.mlab.sh and pkg.mlab.sh, glibc 2.34 or later.

Limits

What we take from it

1. Separate facts from verdicts. Progress comes from changes to the model (interface dispatch, a Rust idiom, application context), not from rules written for one suite, and every fact stays available to the next verdict.
2. Make the measure cheap, then make it mandatory. The 0.8 speed work turned 2.5 hours of benchmarks into 20 minutes, and the release gate turned them into a rule.
3. Prefer a miss to a false proof. Declared types, proven only on a whole path, traps kept forever: precision is a set of choices, each with a known cost.

1. F_β with β = 3. The benchmark also reports F2 (31.9 on all repositories); we quote its own headline metric. ↩