Test run · July 2026 · not a client deliverable
CADi’s first end-to-end run against a public synthetic benchmark — 6,362,620 PaySim rows on SCM 3.0.0. It demonstrates the mechanism: a decision made at the authorization moment, with a named reason, and the fraud label never in scope. It is not a client deliverable and not a fraud-accuracy claim. It reports recall without the operating point that decides whether the screen could actually be run — the flag alerts on 23.9% of all transactions, roughly 190 alerts for every fraud found — states no baseline comparison, and recorded no provenance stamp while it ran. That re-instrumented run has since landed (SCM 3.2.0, 1 August 2026). It reproduced this run’s confusion matrix exactly - the same 8,012 catches and 201 misses - and added the transaction amount this run never recorded. On that measure recall is 87.5% by value against the 97.6% by count quoted below, because the 201 missed frauds average 5.7× larger than the 8,012 caught. Every recall figure in this document is by count unless it says otherwise.
CADi™ Proof · Internal test run
Methodology, leakage controls and measured performance · PaySim (Kaggle paysim1) — synthetic mobile-money fraud benchmark
This pack documents how CADi's decision-time routing was evaluated against ground-truth labels on a public synthetic benchmark, and states plainly what the result does and does not establish.
All 6,362,620 transactions were mapped through CADi's foreign-source adapter onto the frozen internal event contract and run through the same decision spine as the live stream. Each transaction was decided in isolation at its authorization anchor (a genuine decision-time cut; no post-decision events in scope). The engine returned a routing decision plus the detector flags behind it.
The evaluator-only fraud label (isFraud) is never placed on the event or its payload — the adapter omits it by construction — so the engine cannot see the outcome it is later scored against. The label is applied only after each decision, to populate the confusion matrix below.
BALANCE_EMPTYING_RISK as a fraud signalBase rate: 8,213 fraud in 6,362,620 transactions (0.129%). Metrics are computed over the full population at the true base rate — no re-weighting or sampling.
The same matrix weighted by amount. Recall by transaction count is 97.6%; weighted by the amount at stake it is 87.5%. The 201 false negatives carry 1,508,357,150 of the 12,056,415,428 total fraud value, so 2.4% of the fraud by count is 12.5% of it by value. The cause is structural: the predicate fires on an account drained to zero, so a fraud capped below the balance it targets leaves a residue behind and escapes - the misses have a median of exactly 10,000,000, PaySim’s per-transfer cap. This split comes from the re-instrumented run identified in the notice above, which reproduced this matrix exactly and recorded the amount field this run did not. Amounts are notional simulator units, not money.
| Transaction type | Count | Flag-fire rate |
|---|---|---|
| CASH_OUT | 2,237,500 | 42.9% |
| PAYMENT | 2,151,495 | 15.2% |
| CASH_IN | 1,399,284 | 0.0% |
| TRANSFER | 532,909 | 42.9% |
| DEBIT | 41,432 | 13.6% |
A standard fraud analysis is retrospective: it fits a model on historical rows including the fraud label and reports a score after the outcome is known. This report documents something a conventional analysis structurally cannot produce.
| Dimension | Conventional fraud analysis / dashboard | This CADi run |
|---|---|---|
| When the call is made | After the fact, on outcomes that already happened | At the authorization moment, before the outcome exists |
| Needs the fraud label? | Yes — trains on it | No — the label was never in scope; the pattern was caught structurally |
| What it outputs | A probability / score | A live routing action + a named reason (BALANCE_EMPTYING_RISK) an operator can act on |
| Auditability | Model weights; reason often opaque | Every decision carries its detector flags and an attached intervention (100% of decisions here) |
| Coverage on this run | — | Recalled 97.6% of 8,213 genuine fraud cases across 6,362,620 transactions, label-free - 87.5% of that fraud by value |
The distinction in one line: a dashboard can tell you next month that accounts drained to zero turned out to be fraud; CADi raises a named flag this second, on a transaction whose outcome does not yet exist — without ever having been shown a fraud label.
The flag operates as a high-recall, low-precision first-line screen: it surfaces the large majority of genuine fraud, at the cost of also flagging legitimate balance-emptying activity. This is appropriate for a review-triage signal, not an automated decline. Precision at a 0.129% base rate is bounded by that base rate and should be read accordingly.
BALANCE_EMPTYING_RISK flag is recall-oriented: it is designed to surface risk for review, and flags a share of legitimate balance-emptying activity too - precision is 0.53% at the true base rate.results.json over all 6,362,620 transactions at render time. That run recorded neither its engine commit nor its input hash, so these figures can be read and cited but cannot be re-derived. Its artefacts have since been sealed into runs/paysim-3.0.0/ and are under version control, which makes the numbers hash-checkable — a different property from reproducibility.Every figure above was read from a results.json produced by run-paysim-full.ts over the PaySim CSV. That run recorded neither its engine commit nor its input hash, so this pack cannot be re-derived from the repository and a fresh run would produce a new results file, not a reproduction of this one. Its artefacts have since been sealed into runs/paysim-3.0.0/ and are under version control, which makes the numbers citable and hash-checkable — a different property from reproducibility.