To realise the potential of AI in finance, users need to grasp the scale of the problem

In Douglas Adams' sci-fi novel The Hitchhikers Guide to the Galaxy, an alien invasion fleet arrives on Earth intent on ousting humanity.

But, owing to a vast underestimation of scale, the invader's entire fleet of mighty attack ships is swallowed … by a small dog.

I was reminded of this failure to grasp the size of the issue at hand when hearing some of the inflated, over-ambitious claims being made for artificial intelligence (AI) in the banking and finance sector. Because, just as in Adams' story, some of those promising the Earth lack the technology and understanding to attack the problem on the scale the problem demands.

Growing datasets

Here's why. AI is delivering two chief things in the finance sector: it's automating many middle and back-office functions, and it's also letting firms train machine-learning algorithms to recognise patterns in massive datasets - so they can discover actionable, profitable insights that banks and finance houses can process, respond and react to. But the datasets needed to do this need careful acquisition, delivery, storage and curation - on both the technical and legal fronts.

Technically, where once we may have set analytical software working on a 3-gigabyte dataset - and we thought that was big - it could now be at least 100,000 times larger, with a 300 terabyte volume, say. Or even bigger, moving into petabyte and exabyte territory. But banks and finance houses can only push AI into working on such high data volumes if they have scaled their technology to the size of the task in hand. They need trustworthy, hyperscale cloud storage, plus reliable, secure compute power to be able to process this data, as well as the networking flexibility to get that data to where it is needed, when it is needed.

Push-me-pull-you

And legally, scaling up the amount of raw data being captured is going to lead to ever more reporting to regulators, and more abiding by complex – and possibly conflicting - rules on access to personal data. For instance, the Markets in Financial Instruments Directive (MIFID II) asks investment service companies to hold more data about customers and the deals they strike (such as recording voice calls in which deals are discussed), whilst the General Data Protection Regulation (GDPR), the right-of-access data privacy act, urges restraint on collecting personally identifiable information.

This emerging, push-me-pull-you legislative seesaw is just another sign that scaling up is not for those vendors who fight shy of complexity. At BT's R&D lab, engineers are developing the AI capabilities that can cope with this, with technologies spanning everything from AI-optimised software defined networks, to criminal anomaly detection and the intelligent acquisition of the trading and recorded data that MIFID II demands.

Cross-pollination

Data storage in the cloud will be an essential part of the AI equation. That's not only because MIFID II will send storage demands sky high, but also because real-time, always-on, always-available storage will be needed to serve the massive datasets that machine intelligence APIs need. If that's okay with the GDPR, that is.

It's becoming a complex world - and for good reason. We all want legal, decent, honest and trustful investment dealings, and we also want our personal data treated fairly, with unbiased algorithms working on it. To make that happen some clever cross pollination of storage, compute power and networking is going to be needed if AI and machine learning systems are to flourish in the finance space.

Architecturally, the network serves an essential role in enabling AI solutions, either because of the distributed compute, storage and processing involved, the aggregation of data at hyper scale sites, or the volumes and diversity of data sources. Telecommunications vendors such as BT therefore have a significant part to play in enabling this.

Scale, flexibility, agility and security are watch words in an age where transforming legacy infrastructures should really become a must do for financial services sector firms looking to make the most of AI or other innovations against an ever-changing regulatory framework.

Like those aliens in the Hitchhiker's Guide, the solution vendors and market participants that can't deliver on the white heat of technology will have to, er, beware of the dog.

You need a more agile, flexible network that takes full advantage of the cloud. Find out how we can help you can balance the risk and reward of the cloud.