The role of screen scraping and the Open Banking journey


Open Banking has already had a significant impact in terms of unlocking the market to make way for new blood. Where traditional banks historically held the monopoly over customer account information, the regulatory drive towards a (consumer consent based) data-sharing ecosystem has enabled a plethora of start-ups to innovate the delivery of financial services.

While the primary aim of PSD2 legislation may have been to foster innovation and level the playing field, it comes at a time where customer expectations of technology are higher than ever. Deep in the digital age, the tone has been set by trend-setters across all industries that if we can use tech to make consumers’ lives easier, we should.  In fact, according to research from the Open Data Institute (ODI), 64% of UK consumers would share more personal data in return for new benefits that are more convenient. 

The new wave of financial services products we’re seeing is underpinned by data aggregation and the categorisation of all transactions within it, whereby FinTech firms are granted access to transactional data across their users’ bank accounts in order to broaden the scope of their digital offering. Until recently, a technique known as ‘screen scraping’ has been the primary method of assembling this data. Screen scraping has undoubtedly delivered a valuable service to many, but the possibilities now available through Open Banking are truly game changing. 

The UK Competition and Markets Authority (CMA) compelled all major UK banks to make customer current account transaction data available via API in January 2018 with the customer’s permission.  On September 14th a further milestone will be reached with all transactional (payment) accounts being subject to this change through PSD2 legislation for the EU taking place. In addition, the API made available by law must use secure customer authentication (SCA) during login. This change will quite simply make screen scraping for all payment accounts impossible. At the same time, PSD2 legislation makes it illegal to access payment transaction data via screen scraping even if still technically possible.  

The impact of Open Banking for the end consumer?

Open Banking was the next logical step towards finally breaking the banks’ monopoly on customer data. Complimenting GDPR and breaking the stronghold of incumbent banks, the initiative aims to level the playing field for new entrants and create an ecosystem in which consumer financial information flows seamlessly between different institutions with authorisation from and control directly by the customer.

For the consumer, Open Banking marks the start of a new relationship with the financial services industry based on trust, transparency and consent. By enabling them to share their data securely, Open Banking means they can easily evaluate personalised financial products and manage their own finances across several accounts without having to go through their bank, truly empowering the individual.

Does screen scraping still have a place?

With roughly 75% of organisations developing both internal and public-facing APIs and legislation encouraging this route as the industry standard, it’s clear API integration is the future. However, until we achieve full Open Data via API's, screen scraping is still required to view a customer’s full financial picture, as not all financial accounts are yet required to offer API access. We should also note, that the process of screen scraping (in a read-only format) is widely acknowledged as a ‘safe’ process, when done by an organisation that takes data security seriously and has the customer’s best interests at heart. It has been common practice since the 80’s by many financial institutions from credit bureaus, news organisations to the banks themselves in order to streamline processes involving legacy systems and to minimise the need for re-keying data.

Markets such as pensions and investments have no obligation to make data available by API anytime soon. Whilst some providers are embracing change, and proactively implementing API’s anyway, others are not. In the short term at least, offering a combination of direct open banking API’s and screen scraping for accounts where this is not an option, gives customers the most optimal solution to see all their finances in one place. 

Moneyhub is, at its core, an Open Banking platform. Not only was our API the first live Open Banking integration, but our technology features the most data links of any aggregation provider in the UK.

We have invested considerable time and money in direct API Open Banking integration to deliver innovative, secure banking services to our users, and benefits beyond just data such as auto-categorisation and insights via personalised smart nudges. Certainly at this stage, however, coverage of all accounts depends on both API integration and screen scraping to deliver this seamless, holistic service. In light of this, we have established a triage system to find the best source of data when an account is connected.  Starting with direct Open Banking API’s, then bespoke API’s that are available and implemented through to Screen Scraping when necessary. In addition to our account triage system we also have a set of migration tools in place to support the move from screen scraping to Open Banking API’s and more broadly Open Data API’s as and when these become available. 

In the short-term, screen scraping will continue to be a workaround in the absence of available APIs – but it won’t be long until this method is a thing of the past.

With the end goal of opening a closed market and ultimately empowering customers the importance of any system has got to be about Trust, Transparency and Consent.  The days of arguing about who owns the customers data are well behind us. The customer owns their data - every last transaction of it.