A fraud detection model that blocks 99% of fraud sounds like a success. Then you notice it also blocks 5% of legitimate transactions. You've built a system that punishes your best customers to catch your worst ones. Because the fraud losses are visible in a report while the false positive costs are scattered across declined sales, support tickets, and customer churn, the system will probably keep running that way for longer than it should.
The precision-recall tradeoff
Two metrics define the quality of a fraud classifier. Precision is the fraction of flagged transactions that are actually fraudulent. If you flag 100 transactions and 70 of them are fraud, your precision is 70%. Recall is the fraction of all fraudulent transactions that you caught. If there were 200 fraudulent transactions in total and you caught 140 of them, your recall is 70%.
These metrics move in opposite directions. To increase recall (catch more fraud), you lower your threshold for flagging, which means flagging more transactions, including more legitimate ones, which decreases precision. To increase precision (fewer false alarms), you raise your threshold, which means you miss more fraud, decreasing recall. There is no model configuration that optimizes both simultaneously for a real-world dataset. A model that claims perfect precision and perfect recall should be treated with the same suspicion as a perpetual motion machine.
The right precision-recall tradeoff is not a technical question. It depends entirely on how expensive each type of error is in your specific business context.
Why false positives are expensive in fintech
Fraud losses are easy to quantify and visible to everyone. A fraudulent transaction shows up as a loss on the income statement. Someone owns it, someone is measured against it. False positive costs are the opposite: diffuse, hard to attribute, and often not measured at all.
When you decline a legitimate transaction, the customer experience goes something like this: the payment fails with a generic error. The customer doesn't know if their card is broken, if their bank declined it, or if the merchant is having a technical problem. They call support, which costs you money and their time. If it's a high-value purchase, they may give up and not retry. If they retry and it fails again, they're probably done with you. They start using a different card, or a different merchant, for future purchases. None of this shows up as "fraud system false positive cost" on any report.
For high-value customers (the ones spending the most, most reliably) a false decline is a relationship-ender. These are exactly the customers whose spending patterns sometimes look anomalous: large purchases, travel, unusual hours. A fraud model that treats "unusual" as synonymous with "suspicious" will disproportionately decline your best customers while more consistently approving low-value routine purchases. That's the worst possible tradeoff.
Threshold tuning
A fraud model outputs a risk score between 0 and 1. It does not output "block" or "approve." The decision to block or approve comes from a threshold you set. A transaction scoring above 0.7 might be blocked, below 0.7 approved. You chose 0.7. You could have chosen 0.5 or 0.9. The choice is entirely yours, and it determines the precision-recall tradeoff in practice.
Setting the threshold is a business decision that should be made by people who know the unit economics. You need your fraud loss rate (what fraction of transaction volume is fraudulent, and what's the average fraudulent transaction size), your false positive rate at candidate thresholds (how many good transactions get blocked), and your customer lifetime value (how much does a false decline cost you in the long run, accounting for churn). With those numbers, you can calculate the expected value of each threshold choice and pick the one that maximizes it.
In practice, the calculation is messy. Customer churn from false declines is hard to measure, fraud losses are lumpy and seasonal, and model performance drifts over time. But doing the rough calculation beats setting a threshold by instinct or optimizing a single aggregate metric like overall accuracy.
Features over model complexity
In fraud detection, what signals you use matters more than which model architecture you choose. A logistic regression with well-engineered features frequently outperforms a deep neural network trained on raw transaction data. The fraud signal is often behavioral and contextual, and encoding that context explicitly is more reliable than hoping the model learns it.
The most predictive features are typically velocity checks (three purchases in ten minutes from different cities is not a single person), device fingerprinting (is the device associated with this account, or is it new?), behavioral biometrics (typing rhythm, mouse movement, time spent on the checkout page), and address mismatches (billing address and shipping address that have never matched before on this account). IP geolocation against card issuing country. Transaction amount relative to account history. Time since account creation. How the account logged in.
Each of these features encodes a specific fraud pattern. Each one was probably in someone's head before it was in the model. Encoding domain knowledge explicitly is efficient: you don't need thousands of examples to teach a model something you can just write down as a rule. The model's job is to integrate these signals in a way that accounts for correlations and varying base rates across merchant categories and geographies. That's where machine learning earns its keep.
The adversarial problem
Fraud patterns change because fraudsters adapt. A fraud ring that gets blocked by your model will change their behavior until they find what gets through. A model trained on last month's fraud is already partially obsolete before it ships. It's not a problem you solve once.
Static rule-based systems decay faster than learned models, because a specific rule ("block all transactions from IP range X") can be bypassed by anyone who knows the rule. A learned model is harder to reverse-engineer but not impossible: a determined adversary can probe it with test transactions and infer the boundaries. Neither is impervious.
Model performance on recent data should be a first-class metric that someone owns and monitors, not just a number you compute at training time and forget. Track precision and recall on a rolling window of recent transactions, plot them over time, and set up alerts when they degrade beyond a threshold. The degradation is your signal to retrain or update your rules. Retraining cadence depends on how fast your fraud patterns evolve: weekly or monthly for most businesses, potentially more frequently for high-volume targets.
Layered defenses
Real production fraud systems are not a single model. They're a pipeline, each layer with different latency and accuracy characteristics.
The first layer is real-time rules: simple, fast, low-latency checks that catch obvious fraud. Things like "block if the card is on the known-stolen list," "block if there have been more than five failed attempts in the last minute," "block if the transaction amount exceeds twice the account's historical maximum." These run in under a millisecond and catch the easy cases.
The second layer is a real-time ML model. This runs in the same request, adding perhaps 20 to 100ms. It integrates the behavioral signals, account history, and contextual features into a risk score. This is where most of the discrimination happens.
A third layer, running asynchronously after authorization, handles complex cases that take too long to resolve in real time: network graph analysis (is this account connected to known fraud accounts through shared device identifiers?), deep behavioral analysis, cross-merchant correlation. The payment has already been authorized or declined by the time this runs. What it informs is future decisions: whether to flag the account for review, lower the trust score for next time, or trigger a manual review queue.
Human review sits at the top of the stack for high-value edge cases. Automated systems are cheaper per decision but worse at novel patterns. A human reviewer can look at an account holistically, call the customer, and make a judgment that no model would make. The queue should be sized to the volume where human time is worth spending, typically transactions above a dollar threshold where the expected fraud savings exceeds the cost of review.
The objective
Fraud detection is not an accuracy problem. Optimizing for accuracy on an imbalanced dataset (where fraud is 0.1% of transactions) gives you a model that approves everything and achieves 99.9% accuracy while catching zero fraud. That's a toy example of a real failure mode: optimizing for a metric that doesn't reflect the actual cost structure.
The objective is to maximize revenue minus fraud losses minus friction costs. Fraud losses are what you pay when fraud gets through. Friction costs are what you pay when legitimate transactions get blocked: declined revenue, support costs, and customer lifetime value destroyed. Both are real. Both vary with your threshold choice. The right threshold is the one where the marginal cost of catching one more dollar of fraud (in friction imposed on legitimate customers) equals the marginal benefit.
Every decision your fraud system makes is a bet. The goal is to make bets with positive expected value, measured in terms you actually care about. A system that blocks every suspicious transaction has won the battle and lost the war.
Back to writing