The fraud scorecard problem: why we still can't tell a good bank from a lucky one

Every regulator, vendor, and bank in this industry can point to a number that makes them look good. That should worry you more than it reassures you.

The UK's Payment Systems Regulator publishes reimbursement league tables. Zelle's parent company says 99.95% of transactions have no reported fraud. Monzo says it prevented 2.9 times as much fraud value in 2025 as the year before. Bank of Ireland says attempted payment fraud is down 30% since it switched on a new detection system.

All these numbers can be true. None, on their own, tells us whether a bank is getting better at stopping fraud. And the gap between true and meaningful is where the story gets interesting - because almost everyone in the fraud ecosystem has a reason to prefer a metric that shows progress.

While looking for data and perspectives for this series, I kept coming back to a conversation with Tony Fish. He has previously contributed to The Paypers with a deliberately provocative argument that our traditional approach to security starts in the wrong place. Rather than waiting for threats to manifest and then trying to contain them, he argues for building what he calls security immunity: creating the conditions in which an organisation is better able to absorb and respond to threats before they become damaging. His analogy is close to something we all understand intuitively. You don't wait until you are seriously ill to start thinking about your health. You eat the broccoli, take the vitamins, and build the habits that make the organism more resilient before the disease arrives.

That idea stayed with me because it raises a much less comfortable question for fraud prevention: what if we have become too focused on treating fraud once it materialises, rather than understanding and disrupting the conditions that allow it to happen in the first place?

And that question became the starting point for this article. If prevention is the real objective, then we need to look beyond the numbers that describe what happened after the fraud. We need to ask whether the industry's favourite scorecards are capable of showing us which institutions are preventing fraud - and which may simply be getting lucky.

When the scorecard becomes the story

The UK's PSR data seemed like an obvious place to start. But Tony Fish immediately put a useful warning label around it: 'Be careful with the PSR data as it's a scorecard, not a league table.' That distinction matters.

The dataset brings together several different dimensions - reimbursement outcomes, fraud sent from customer accounts, and fraud received into accounts used by money mules. Those measures can point in different directions. A bank can look strong on reimbursement while telling a different story when you examine the fraud entering its ecosystem. Nationwide and TSB, for example, look strong on reimbursement, while Barclays, NatWest, and Santander perform better across some of the broader measures.

The temptation is to turn these columns into a ranking: this bank is good, that bank is bad. But that is precisely where the scorecard becomes misleading. The numbers describe different parts of the problem, rather than one simple measure of fraud-prevention effectiveness. And because the dataset predates mandatory reimbursement, even the strongest figures represent a snapshot of a system whose incentives have since changed.

The question, then, is not whether the numbers are wrong. It is whether we are asking them to answer a question they were never designed to answer.

The leaky bucket

That problem came up again in a conversation with Mike Nathan, Vice President, Professional Services International at LexisNexis Risk Solutions. He offered a metaphor that captures the difficulty better than another league table ever could: the industry is dealing with a leaky bucket. You can plug a few holes, but leakage will appear somewhere else. The metaphor is useful because it changes the question. Regulation can force a bank to plug one particular hole. Reimbursement rules can change who ultimately pays when money disappears. New detection technology can close another vulnerability. But none of those interventions necessarily tell us whether the bucket itself is becoming less leaky.

In fact, regulation can produce an important unintended effect in how we perceive progress. It can change behaviour without necessarily changing the underlying volume of fraud. The industry may therefore become very good at measuring what happens after the leak - who pays, how quickly they pay, and how much is reimbursed - while remaining less certain about what caused the leak in the first place. And that brings us to an even more fundamental problem: when exactly are we measuring fraud?

The rear-view mirror problem

The answer, surprisingly often, is after the money has already moved. Alloy's 2026 State of Fraud Report puts some hard numbers behind this. Only 24% of organisations report catching fraud at onboarding, down from the previous year, while one in five do not identify fraud until money has already left the institution.

In other words, much of the industry's understanding of fraud is still retrospective. We see the loss, investigate the transaction, identify the pattern, and then try to work backwards to understand what happened. That creates a peculiar paradox. We increasingly have access to signals throughout the customer's financial journey, yet we often evaluate our performance at the very end of that journey.

The signal might have been there when the account was opened. It might have appeared when a new beneficiary was added. It might have emerged from an unusual change in transaction behaviour, a newly created recipient account, a suspicious inbound payment, or a pattern that becomes visible only when activity is viewed across a wider network. But if the organisation only asks, 'Did this transaction turn out to be fraud?', those earlier signals can disappear into the background.

And this is where the problem of measurement becomes a problem of prevention. If we measure fraud only when the money is gone, we are effectively measuring the last visible symptom of a much longer process. Zelle provides a particularly striking illustration.

Case study: Zelle and the problem of two truths

The dispute surrounding Zelle and Early Warning Services shows how the same system can produce radically different stories depending on where you choose to put the measuring line. By Early Warning's own numbers, more than USD 1 trillion moved through Zelle in 2024, and 99.95% of transactions carried no reported fraud across participating institutions.

Yet the New York Attorney General's case presents a very different picture: more than USD 1 billion lost to fraud, allegations that stronger safeguards were delayed despite internal warnings, and marketing that allegedly gave customers a greater sense of protection than the system provided.

Both narratives rely on numbers. Neither necessarily requires the other to be false. That is precisely what makes the example so useful. An institution can be overwhelmingly safe in aggregate and still fail a significant number of people in precisely the situations where its controls matter most. A 99.95% fraud-free rate sounds extraordinarily reassuring. But it does not tell us where the remaining 0.05% sits, how vulnerable those customers were, whether the fraud could have been prevented earlier, or whether warning signals were available before the payment was completed.

The lesson is not that Zelle is uniquely bad. It is that almost any large financial system can look reassuring or alarming depending on which part of the journey its metrics capture.

Stop measuring the end of the journey

Perhaps the better benchmark is not who reimburses fastest after harm, or even who reports the lowest percentage of fraudulent transactions. It is who most consistently stops the scam journey from completing at all - and who gives customers the clearest, earliest opportunity to recognise that something is wrong.

That requires a different way of looking at fraud. Instead of starting at the point where money disappears and working backwards, we need to follow the journey forwards. Because a fraudulent payment rarely begins with the click on 'send'.

By the time a customer is sitting in front of their banking app preparing to make a payment, the scam may already have been developing for days, weeks, or even months. The victim may have encountered an advertisement on social media, received a message, spoken to someone pretending to represent a bank or investment company, been persuaded to open an account, moved money into it, added a new beneficiary, and only then reached the payment screen. For the bank, the payment is the most visible event. For the criminal, it may be the final stage of a much longer operation.

So where, exactly, should we say the fraud began?

Before the payment: spotting compromise early

The first opportunity may arrive long before a suspicious transaction appears. According to LexisNexis Risk Solutions' Cybercrime Report, login attack rates nearly doubled year-on-year, rising 89%, while new-account creation now sees roughly one in 11 attempts flagged as attacks. At the same time, mobile app attack rates fell sharply as fraudsters shifted back towards desktop browsers. The numbers illustrate an important characteristic of modern fraud: the threat moves when the defence moves. Make one channel harder to attack, and criminals look for another route in. That makes early detection less about finding one perfect signal and more about building a picture of what is happening around the customer.

Mike Nathan describes five areas that banks increasingly need to bring together: beneficiary risk, mule detection within the bank's own customer base, inbound payment monitoring, real-time signals about the customer's situation, and transaction context. Each addresses a different part of the journey.

Start with the beneficiary. A new recipient is not automatically suspicious, but it creates a moment when the bank can ask useful questions. Is this an account the customer has paid before? Is it newly opened? Is there unusual payment velocity? Does intelligence from payment networks or industry consortiums suggest that other institutions have already seen something concerning about this account?

Then there is the receiving side of the equation. A bank cannot protect the payment journey simply by watching what its customers send. It also needs to understand what is arriving into its own accounts. That matters because the account receiving the money may be part of the fraud journey too. Some mule accounts are opened deliberately for criminal purposes. Others begin as ordinary customer accounts and are later recruited, sold or taken over. Some customers may even be exploited mules who do not understand that they are participating in criminal activity.

The problem becomes even more acute once money starts moving. Mike describes cases where scam proceeds pass through seven or eight generations of mule accounts in a very short period. By then, detecting the original scam at the payment endpoint is already a race against the clock.

Which is why the most interesting signals may not be about the transaction at all. They may be about the customer's situation at the moment the transaction is being prepared. Is the customer simultaneously receiving a phone call? Is remote-access software active? Is somebody else controlling the device? Has the customer's behaviour suddenly changed? Is there evidence of unusual hesitation or interaction? For APP fraud, these questions matter because the customer is technically authorising the payment. The challenge is not simply to determine whether the transaction looks fraudulent; it is to determine whether the person authorising it is acting freely and knowingly.

Only after those signals are considered does the transaction itself come into focus. Is the amount unusual? Was the beneficiary added moments ago? Does the payment fit the customer's normal behaviour? Does the wider context suggest that this particular payment is different from the customer's usual activity?

The point is not that one of these signals will reveal the scam. It is that the earlier stages can give the bank more context with which to interpret the final payment. And that leads to a crucial distinction: detecting risk is not the same as stopping it.

At the point of payment: creating a moment of doubt

The payment screen may be the last opportunity to interrupt the scam before the money leaves. This is where Verification of Payee (or Confirmation of Payee in markets where that terminology is used) becomes important. The principle is deceptively simple: before the payment goes through, the bank checks whether the account details and the intended recipient correspond. For a customer being told by a fraudster to move money to a supposed 'safe account', that check can create something the scammer has spent considerable effort removing: a moment of doubt.

The bank identifies a potential problem. The customer receives a warning. The customer then has to decide whether to continue. That changes the role of the bank. It is no longer merely trying to classify the transaction as fraudulent or legitimate in the background. It is becoming part of the customer's decision-making process at the precise moment when the scam can still be interrupted.

It also creates a clearer allocation of responsibility. As Freddy Arthur of NICE Actimize explained to me, where the bank has properly applied the verification process and warned the customer, the consequences can differ if the customer chooses to proceed anyway. The regulatory framework therefore turns the warning itself into part of the fraud-control mechanism.

There are still relatively few examples that give us a clean before-and-after picture of what these interventions achieve. One often-cited case is Bank of Ireland, which reported a 30% reduction in attempted payment fraud and a 25% reduction in customer losses after deploying AI-based detection with Featurespace in 2024. It is an encouraging result - but it is important not to overstate what it proves. The published figures describe payment-fraud detection broadly; they do not, by themselves, establish that the reduction specifically came from detecting APP scams or another particular scam category. The more useful point is that genuine before-and-after measurements remain unusually rare in an industry otherwise full of isolated percentages.

And that brings us to another problem.

More signals do not automatically mean more prevention

Financial institutions have spent years expanding the number of signals available to fraud teams: device intelligence, behavioural data, identity information, transaction patterns, consortium intelligence, biometrics, and increasingly AI-generated risk scores. Yet more data has not necessarily produced better outcomes.

As BioCatch argues in its discussion of fraud signals, the challenge is increasingly about the quality, context, and durability of those signals, rather than simply accumulating more of them. Furthermore, Uri Rivner, CEO of Refine Intelligence, argues that banks are not necessarily short of detection signals; they are short of the capacity to resolve the alerts those signals generate. In other words, the problem may no longer be 'Can we see something suspicious?' but 'Can we do something useful about it before the opportunity disappears?'

That distinction is crucial. A bank can have an impressive fraud engine generating thousands of alerts and still fail to protect customers if those alerts cannot be investigated, contextualised, and resolved quickly enough.

The moment of doubt: when the human becomes the signal

There are fraud situations where no model can fully understand what is happening because the critical information is sitting inside the customer's head. The customer may genuinely believe the person on the other end of the phone is a police officer, a bank employee, an investment adviser, or a romantic partner. The transaction may look perfectly consistent with what the customer believes they are doing. At that point, the bank may need to stop analysing the transaction and start talking to the person.

Santander UK's Break the Spell team is an unusually tangible example. According to Chris Ainsley, head of fraud risk management at Santander UK, the team helps customers understand that they have been manipulated by scammers. In extreme cases, the intervention can continue for months, with conversations taking place by phone or face-to-face in a branch. In 2025 alone, the team prevented customers from handing over GBP 6.8 million to scammers, bringing the total saved since its launch in 2021 to GBP 24.4 million. There is something important hidden in those numbers. The intervention is not really 'fraud detection' in the conventional sense.

The bank is not proving that the transaction is fraudulent. It is helping the customer recognise the fraud themselves.

After the money moves: recovery depends on how fast information travels

If the payment still goes through, the fraud journey enters a different phase - but prevention has not necessarily ended. Money that leaves one account may enter another bank, then another account, and potentially another mule before anyone has fully understood what has happened. At this stage, speed becomes critical.

A bank can investigate its own customer and identify a suspicious recipient. But if that information remains inside the institution, the receiving bank may be starting from zero. This is why inter-bank intelligence can matter as much as better internal detection. Data from LexisNexis Risk Solutions suggests that when banks share real-time alerts about suspected mule accounts, 50–70% of those alerts identify accounts that the receiving bank had not yet flagged independently.

The implication is significant: the value of fraud intelligence does not stop at the boundary of the institution that generated it. The faster that information moves, the more opportunities another bank has to intervene before the money is moved again.

Learning: closing the loop

But even recovery is not the end of the journey. Every fraud event leaves information behind: how the customer was approached, which device was involved, how the account behaved, which beneficiary received the money, which signals were visible, when an alert fired, what intervention worked - or failed - and how quickly the money moved afterwards. The question is whether that information disappears into a case-management system once the investigation closes, or whether it changes what the institution does next time.

That is where the idea of a fraud feedback loop becomes important. Alloy's 2026 State of Fraud Report frames this as a different mindset: fraud is not simply something institutions defend against; it is something they can learn from. Tommy Nicholas, Alloy's CEO, argues that institutions that treat fraud as a source of learning are better positioned to let their fraud strategy evolve alongside their growth strategy, rather than allowing it to lag behind.

The important shift is therefore not from one fraud technology to another. It is from thinking about fraud as an event to thinking about fraud as a journey.

But there is one complication that runs through every stage of that map. The customer is present at every point - and yet, in much of the fraud technology stack, the customer is still represented primarily as a collection of signals: a device fingerprint, a login anomaly, a transaction pattern, a velocity score. That works reasonably well when the customer is the victim of a stolen credential or an unauthorised transaction. It becomes much harder when the customer is the person being manipulated.

In APP scams, romance fraud, and investment scams, the customer may be actively participating in the transaction while simultaneously being deceived. The fraud can develop over weeks or months, through conversations and relationships that never appear in the bank's transaction data. Which raises the next question in the journey:

What happens when the most important fraud signal isn't the transaction, the device, or the account - but the person?

That's where the next article goes (to be published on Sept. 17).

About author

Mirela Ciobanu is Lead Editor, Banking and Fintech at The Paypers, focusing on the latest trends and developments in fraud, cybersecurity, and technology (generative AI, blockchain analytics, data, etc.). Mirela is particularly passionate about the importance of having interoperable digital identity solutions that help not only to secure payments but also transactions in other areas of life (travel, health, education). She is a strong advocate for online data privacy and protection.

As a skilled writer, she strives to deliver accurate and informative insights to her readers, always in pursuit of the most compelling version of the truth. To share more ideas and get inspired, connect with Mirela on LinkedIn or reach out via email at mirelac@thepaypers.com.