Perspective

How Should a Firm Assess Transaction Monitoring Performance?

Alert volume alone cannot show whether transaction monitoring is working. A more useful assessment follows the control from the risk it is meant to address, through data and scenarios, to investigation decisions and feedback.

Begin with the risk and the population covered

First ask what activity the monitoring is intended to identify and which customers, products, channels and transactions are in scope. Each scenario should connect to a defined risk or behaviour, explain why its data fields and parameters are relevant, and state known limits. A low alert count could reflect low exposure; it could also reflect missing data, narrow coverage or a threshold that rarely triggers.

Check the information reaching the system before interpreting its output. Are expected feeds, transaction types and customer attributes present, accurate and timely? Have system changes or new business activity altered what is captured? In a 2021 enforcement case, the FCA identified weaknesses in scenario risk assessment, parameter testing and data accuracy and completeness. That example shows why performance review needs to cover inputs as well as results; it is not a benchmark for another firm.

Test whether scenarios behave as intended

A scenario test should start with its purpose: what behaviour should it detect, under what conditions, and what can it miss? Use controlled examples that should trigger and examples that should not, then review whether the data and configured rules produce the expected result. Check that thresholds have a documented rationale and are retested when the risk profile, data, rules or customer mix changes.

Results need interpretation. A rule can produce many alerts because it is too broad, because data is noisy, or because the underlying exposure warrants attention. A rule with few alerts may be well targeted or may fail to detect relevant variations. Record test conditions, assumptions and limitations so teams can distinguish a design issue from a data or operational issue.

Learn from investigation quality and customer feedback

Alert dispositions are useful only when the reasoning is clear enough to review. Examine a sample of closed and escalated alerts: did investigators see the customer's known profile and relevant history, ask appropriate follow-up questions, and document why the activity was or was not concerning? Review whether material findings lead to escalation, updated customer information or a reassessment of risk.

The FCA's Financial Crime Guide asks firms to consider how unusual transactions are reviewed, how monitoring findings feed into customer risk profiles, and whether rules and typologies are reviewed for effectiveness. It also stresses understanding automated system capabilities and limitations. Investigator feedback can therefore reveal more than an alert-to-report ratio: it may uncover unclear scenario purpose, missing context, recurring data gaps or a need for further testing.

Choose measures that answer a question

A balanced view might track data coverage and feed exceptions; test outcomes by scenario; alert volumes and ageing by rule or risk segment; quality assurance findings on investigation rationales; and repeat reasons for escalation, dismissal or correction. Define each measure, its denominator and the decision it informs. Compare trends within the firm's own context and investigate material changes rather than treating any one number as proof of effectiveness.

There is no universal alert-conversion rate or volume target that establishes good monitoring across firms. A 2017 NICE Actimize webinar series, co-led by Trevor Barritt for one installment, highlighted data lineage completeness amid fragmented AML systems. That dated discussion remains a useful prompt to test what enters monitoring, but it does not set a current performance standard. The assessment should make limitations visible and lead to specific questions about coverage, scenario design, investigation practice and control changes.

Common questions

Is the number of alerts a good measure of transaction monitoring effectiveness?

It is useful operational information, but not a stand-alone effectiveness measure. Alert counts need context about risk coverage, source data, scenario behaviour, investigation quality and known limitations.

Are there standard alert-conversion benchmarks for every firm?

No universal ratio establishes effective monitoring across different firms and risk profiles. Use defined measures to test the firm's own control objectives and investigate changes in context.