Communications surveillance came under focus in two sessions at the recent A-Team Group Trading Technology Summit in London in February.
During a morning Keynote session, Andrew Delaney, President and Chief Content Officer at A-team sat down with Paul Liesching, Chief Revenue Officer at 1Global to discuss “Navigating the Complexity of Communications Surveillance.”
An afternoon panel focused on “The Evolution of Communications Surveillance – How to Manage Risk, Improve Oversight and Ensure Compliance in Today’s Complex environment.” This panel comprised: Moderator: Anish Amin, Senior Manager, Surveillance, Macquarie; Justin Nathan, Head of Surveillance, Glencore; Simon Abbott, Global Head of Capital Markets Compliance, Northern Trust; Simon Hand, Global Head of Sales, 1GLOBAL Compliance; Phil Fry, VP Financial Compliance, Verint.
Five core themes emerged form these two sessions and are presented along with anecdotal evidence gathered during background for this story.
The Proliferation of Devices, Apps, Channels
An audience poll indicated that one of the key challenges was “too many channels” with MS Teams and Zoom being the most prevalent followed by WeChat and WhatsApp. A follow-up poll asking about the key challenges came back with misuse of prohibited platforms, loss of data, lack of consistency across platforms topping the list.
“BYOD seemed like a good idea at the time.”
Mobile devices usage was proliferating across the business and the cost of managing and maintaining an inventory of mobile devices was already becoming “noticeable”.
The increase in demand created a backlog in the approvals process which In some cases could take several months to complete. The economic case for BYOD was fine except it didn’t fully examine the compliance impacts of losing control of a single supervised and controlled comms channel and having it replaced with a plethora of unsupervised channels.
Off channel communications use was already gathering steam but the COVID lockdown triggered a surge in digital mobile business communications and use of apps WeChat and WhatsApp among others. Apps like WhatsApp being offered through multiple service providers leading to different versions and another layer of complexity.
Some observations from the panels – “Up until 2020, everyone was happy with a very basic set of things. So, voice and SMS, nine countries was fine. And that suited everybody. In the last four years there’s been a proliferation of new services that needs to be captured, triggered purely by what people get used to working from home during lockdown.”
“We definitely saw an upsurge in network traffic during the lockdown but today, it’s higher than it ever was.” This observation really underscores the proliferation challenge, it’s here to stay.
But it’s a problem that must be addressed, and soon. Regulators have shown they’re out of patience. Anecdotal evidence suggests that enforcement actions are having an effect and spurring firms to take action.
AI/ML and GenAI in Comms Surveillance
It’s critically important to understand the limitations of LLMs like ChatGPT. At the end of the day, they are no more than sophisticated prediction engines trained to generate an “expected output” based on hundreds of millions of training examples.
The output can look very convincing but still be factually incorrect. An LLM trained on a corpus of highly curated and polished compliance case histories can still generate non-compliant output.
Transformer based engines – the GPTs – can do a much better job than traditional lexicon-based models but they require carefully designed prompting and a high-quality training data set.
As one of the panels noted: “GPTs trained on trusted corpus of actual case histories and legal filings can deliver recall rates of 70% whilst lexicon-based models average 5%.”
The question of explainability versus efficiency was put to an audience poll which asked to select the more important of explainability versus efficiency. The result showed 76% of the audience voting them equally important which came as a surprise. The need for explainability when undergoing a regulatory examination makes it a must-have. High false positive rates are the result of overly sensitive models designed to avoid a false negative “at any cost.”
Regulators have made it clear; they expect that firms will keep systems updated and models calibrated to reflect the latest version of the rules. Equally important is ensuring their changes in their underlying risk profile due to changes in the business are also reflected.
Anecdotal evidence suggests customers are resisting – “This is the model that got us through the last inspection – we really don’t want to touch it!”
To which the regulators advise: “Don’t expect the next inspection to go well if we find that you haven’t kept things up to date!”
The consensus from the afternoon panel was, given the current levels of false positives most firms are experiencing in their existing models, a meaningful improvement could be achieved with minimal risk by taking some careful first steps. The advice to firms looking to take a first step is, above all else, get the skills and the training you need before taking on anything meaningful.
Getting the Requirements Correct
A common theme from the vendor community was a polite ask of their clients, “Please tell us exactly what you want, and ask us the right questions.”
Again, the regulators make it clear that firms who include market abuse and comms surveillance as part of a comprehensive risk assessment fair a lot better under regulatory examinations. Drill down and understand how the risk profile in your firm changes across asset classes, markets and jurisdictions.
Ask less about how the model handles false positives and more about the detailed use case and what the model needs to deliver. It’s important to be able to clearly define what a true positive is. It’s not the same as a false negative!
Integrated Comms and Trade Surveillance?
Analytics powered comms surveillance is still developing relative to trade surveillance which is a mature market with a lot of excellent/proven technology available. Attempts to bring trade and comms surveillance under a single control framework sound like a worthwhile goal. To be able to respond to an alert and visualize, in near real-time, all of the related antecedent comms and trade activity.
This has proven a challenge in the past due to the differences in maturity and there have been several initiatives that ultimately went nowhere. But the capabilities of the new LLMs and GPTs might be the catalyst that finally makes this achievable.
What’s on the Horizon
Major changes are taking place in the mobile communications industry with GSM in retirement phase rapidly being replaced by 4G LTE and 5G. This mens a lot more bandwidth and capacity for larger apps. With GSM probably gone within 5 years, the possibility of a single messaging platform (Teams? WhatsApp?) emerging through consolidation. “Are we heading for a single personal messaging platform with a single number.”
Generative AI has the potential to make major improvements in model efficiency and explainability and support better visualization and status of complex risk profiles.
But there remains a lot of work to do to get clarity around the compliance of AI itself – particularly explainability – down to applying it to improve the overall performance of the comms surveillance function.
Those are the hot topics on the comms surveillance agenda for Q1 of 2024.
It’s going to be a busy year with EMIR Reft in April, T+1 in May and UK-Refit in September.
Subscribe to our newsletter
