8 Areas that Must be Part of Your Evaluation of a Screening Provider & Why False Positives Alone are Not a Good Metric for Screening Success
Every vendor claims they can reduce your false positives. With all these promises, it’s a wonder any false positives exist at all. Sure, with enough work and tuning, even the oldest of systems can probably reduce some false positives. But at what cost?
This guide from the experts at Sigma360 highlights why focusing solely on false positive reduction misses many other critical factors essential for selecting the right vendor and may open your firm to significant gaps and regulatory action if not addressed properly.
The Bigger Picture
Reducing false positives should be the result of a suite of advanced capabilities working together. An optimized solution not only manages false positives but also minimizes the risk of false negatives, a component that is often overlooked. Moreover, the system should be a future-proofed risk and compliance technology and data operation that meets regulatory expectations and puts you in control from a testing and configuration perspective.
At a minimum, in our experience, a successful and meaningful effort to modernize screening operations should consider the following key areas:
1. Matching Algorithms and Model Explainability
Ensure the vendor’s matching capabilities are configurable by risk type, robust, tested, and documented. Poor matching algorithms can lead to missed risks and misleadingly low false positives. This can lead to regulatory exposure and painful audit inquiries that are easily avoidable. For example, in recent months, we have seen several vendors’ matching engines failing to identify key names and variations that should have been detected, while also choosing to not update or upgrade their algorithms.
2. Data Quality and Testing
Poor or incomplete data can lead to missed risk and inadequate screening. Data that doesn’t update fast enough or that is automatically generated (e.g., political exposure lists) can lead to significant risk and data-related oversights. Vendor solutions need to have robust data governance processes including data quality and testing, as well as a review of what is and is not included. Moreover, systems that integrate data and software are better positioned to deliver more efficiency than those that operate in isolation (e.g., content only or software only). This allows the algorithms and data to work more seamlessly and minimize the effort, implementation and testing required by banks.
3. Fully Integrated and Linked Risk Intelligence
It is hard to deliver true false positive reduction, if multi-source data is not integrated and resolved at scale. Enriching alerts with other internal and external data sources not only assists in reducing false positives but also provides additional risk intelligence and context to close alerts that are generated.
4. System Self Configuration
One reason many systems generate so many false positives is a result of “one size fits all types” of approaches and the difficulty of configuring screening solutions for different use cases easily. Firms simply ‘elect to live with the noise’ which is bad for business, efficiency, and focusing your best people on higher level tasks. And several leading solutions require teams of vendor and internal-based technical resources for the initial implementation and on an ongoing basis, which exposes an organization to unnecessary hold-up risk and cost. Just keep in mind, if you cannot tune it yourself (e.g., low-code, no-code), you will likely always be on the backfoot regarding system tuning and optimization and reliant on others unnecessarily.
5. Workflow and Data Enrichment
You will have alerts, even if you minimize false positives. Having a modern, easy to navigate platform with workflow capabilities is important to hitting productivity and efficiency goals, a point lost on many providers. Legacy systems, as well as those that are not designed with the end-user first, will slow down operations and typically disappoint from an ROI perspective. Evaluate not only the data and tech to process it, but also the ease and time it takes for an analyst to clear an alert.
6. System Uptime, Scalability, and Speed
A system that falls over and struggles to perform is a hard pass, regardless of its false positive reduction claims. Older systems and those that are not continually upgraded with new R&D and technology present significant execution risk. In many larger organizations, these systems require over eight hours to screen clients, which can lead to backlogs and other operational efficiency issues and risks. In other words, system reliability at scale is just as important – if not more – than false positive reduction.
7. Robust, Evidenced Security Posture
Organizations that lack SOC 2, Type II, a CISO, regular penetration testing and other critical security elements present delivery, execution, and security risk for your organization and client data.
8. Vendor Approach and Team Experience
Evaluating your vendor as a partner (vs. simply a vendor) will go a long way toward a successful outcome and reliability. Ensure you are speaking to the team who will deliver and support you in maximizing efficiency and addressing questions from regulators. In addition, you want to look at the total cost of ownership over a number of years including your dependency on the vendor and consulting firms.
Conclusion
In sum, it is critical to fully evaluate your objectives before selecting a vendor and content provider to improve your screening operations. While false positive reduction is critical, it is but one of many extremely important criteria a buyer should consider when purchasing screening software.
