Risk Data engAnalyticsMLAgentic paymentslending Loss rate −31%, queue −40%

Fraud review assistant

6 people · 11 months Two data engineers · Two ML engineers · Risk analyst · Me — architecture and delivery

the engagement client data — not shown

heard

"My analysts clear four hundred cases a day and the good ones are all making the same decision twice."

real problem

Rules had accreted for years with no measurement of which ones still earned their false positives. The review queue was first-in-first-out, so high-loss cases waited behind trivial ones.

system built

A feature pipeline over transactions and device signals, a scoring model with per-rule attribution, and an agent that prepares each case: the evidence, the comparable past decisions, and a recommended action. Analysts decide; the agent never auto-declines.

what broke

Recommendations anchored the analysts — agreement was suspiciously high. We hid the recommendation until the analyst had logged their own read.

the open rebuild same architecture · public data

dataset

IEEE-CIS Fraud Detection (590k transactions)

what the rebuild covers

Full pipeline, the model with rule attribution, and the case-preparation agent. The anchoring experiment is reproduced with a simulated analyst.

artifacts

NotebookCodeDatasetWrite-upDemo

Open the code Open notebook