Test run · July 2026 · not a client deliverable

CADi’s first end-to-end run against a public synthetic benchmark — 6,362,620 PaySim rows on SCM 3.0.0. It demonstrates the mechanism: a decision made at the authorization moment, with a named reason, and the fraud label never in scope. It is not a client deliverable and not a fraud-accuracy claim. It reports recall without the operating point that decides whether the screen could actually be run — the flag alerts on 23.9% of all transactions, roughly 190 alerts for every fraud found — states no baseline comparison, and recorded no provenance stamp while it ran. That re-instrumented run has since landed (SCM 3.2.0, 1 August 2026). It reproduced this run’s confusion matrix exactly - the same 8,012 catches and 201 misses - and added the transaction amount this run never recorded. On that measure recall is 87.5% by value against the 97.6% by count quoted below, because the 201 missed frauds average 5.7× larger than the 8,012 caught. Every recall figure in this document is by count unless it says otherwise.

Executive summary

CADi benchmark test run · PaySim (Kaggle paysim1) — synthetic mobile-money fraud benchmark

On 6,362,620 transactions, CADi flagged 97.6% of genuine fraud by count at decision time, and 87.5% of it weighted by the amount at stake, before any outcome existed and without ever seeing the fraud label.

97.6%

fraud recalled by count (of 8,213)

87.5%

fraud recalled by value

0.53%

precision @ 0.129% base rate

6,362,620

transactions screened

fraud labels seen by the engine

The finding

BALANCE_EMPTYING_RISK operates as a high-recall, low-precision first-line screen: it surfaces the large majority of genuine fraud for review, at the cost of flagging a share of legitimate balance-emptying activity. A review-triage signal — not an automated decline — raised live, with a named reason a conventional dashboard cannot produce.

What this is — and is not

Synthetic benchmark. PaySim is a synthetic dataset that reproduces the statistical structure of real mobile-money transactions. These results demonstrate the engine's mechanism, not a production fraud-detection accuracy figure.
The engine never saw the label. The fraud label travels out-of-band and is applied only after each decision, to score it. CADi decided at the authorization moment on the information visible then — no outcome leakage.
A first-line screen, not a verdict. The BALANCE_EMPTYING_RISK flag is recall-oriented: it is designed to surface risk for review, and flags a share of legitimate balance-emptying activity too - precision is 0.53% at the true base rate.
Recall by count flatters the screen. Weighted by amount rather than transaction count, recall is 87.5%, not 97.6%. The 201 frauds it missed average 5.7× larger than the 8,012 it caught, with a median sitting exactly on PaySim’s 10,000,000 per-transfer cap: the predicate fires on an account drained to zero, so a capped fraud against a larger balance leaves a residue behind and escapes. That is structural rather than statistical - the screen systematically misses the largest frauds. Measured on the re-instrumented run described above; amounts are notional simulator units, not money.
Routing is pinned to AUGMENT for this payment workload by design; the demonstrable signal here is the explainable risk flag, not route variation.
Computed, not hand-entered — but not reproducible. Every figure was computed from results.json over all 6,362,620 transactions at render time. That run recorded neither its engine commit nor its input hash, so these figures can be read and cited but cannot be re-derived. Its artefacts have since been sealed into runs/paysim-3.0.0/ and are under version control, which makes the numbers hash-checkable — a different property from reproducibility.