- Anders Granerød
DNB processes billions of payment transactions every year
When I started working in DNB ten years ago, it was not long before I heard the phrase “Data is our new gold”. This statement was rooted in the belief that the massive amounts of data in DNB’s IT systems could constitute actual monetary value. However, time went by, and the data continued to mostly support internal and existing processes with little or no tangible value associated with its contribution. In retrospect it has become clear to me that the reason for the challenging start to create monetary value from data was simply because going from raw banking data to datasets that can be packaged and sold as products is actually very complex and more challenging than first imagined.
The Shift Towards Cloud Computing
Raw data sitting in operational systems built for specific banking processes isn’t doing much good except for the processes which they are meant to support. This was well-known to the people in the data environment in DNB all those years ago, but doing something about it was at the time considered an insurmountable task in terms of financing and manpower. However, when the Norwegian financial authorities in 2017 approved cloud computing for financial institutions, significant barriers began to diminish. Several strategic initiatives started to pull in the same direction, and the foundational building blocks for creating commercial insight from payments were suddenly seeing the light of day. Transactional data that had previously been thrown away to save valuable storage and processing power was now being stored on cheaper cloud storage, and a group-wide strategy on data insight sparked both the creation of a self-serviced data platform and initiatives on data commercialization and insight creation.
The DNB Insight Factory
DNB has ambitions to “build the world’s best bank for Norway”, and to do that we need to put our data to work. The DNB Insight Factory is a part of this ambition, developed on the building blocks started some seven years ago, made possible by embracing cloud computing. The Insight Factory is a solution for large-scale automatic insight creation from payment transactions. It serves insights to customers, media, customer advisers, and internal decision-makers to name a few. Customer demographics and transactions, together with (largely) open external data are being processed on a regular basis by different computing clusters on the DNB self-served data platform named Insight Platform for Analytics (IPA) to produce a great number of different data products for both specialized and generic use. The data products are currently being distributed on the internal DNB Data Market, but our goal is to also create value through external data markets in the time to come.
Multidisciplinary Approach and Challenges
Building something like the Insight Factory requires a multi-disciplinary approach. Market desirability research is needed to build the business case for investing resources. Legal expertise is required to determine the boundaries to operate within and the necessary safeguards that are required. Reputational risk assessment is required when the largest bank of Norway is planning to use its data in new ways. Piloting with real customers is required to validate hypotheses and fine-tune business logic. From a data and technology point of view the challenge has been to convert payment transactions meant to support payment processes into rich data fit for statistical analysis with a minimum need of manual maintenance while at the same time restraining use computing power and processing time. Accomplishing this with only a three-person team would not have been possible had we not stood on the shoulders of all those people who have worked on our foundational building blocks and are still working on them every day to support DNB with knowledge, tooling and data. It would also not have been possible without having sponsors in the organization that believed in the idea and the future promise that it holds.
DNB-internal dashboard with insight on Swedish commerce from the Insight Factory
The Insight Factory has been a bottom-up initiative within DNB, an idea that has been given the opportunity to prove its feasibility through iterative development and continuous improvement to its core models and production pipelines. It is now capable of estimating and categorizing the spending of the entire Norwegian population, it can estimate travel patterns internally in Norway as well as boarder commerce in Sweden, it can tell us about customer demographics and economic trends in C2B merchants across Norway, it can estimate economic behavior in the most fine granular of C2B industries, it can enrich the payment user experience in our digital channels, and the list goes on…
Technical Composition
Technically, the Insight Factory consists of two main components - the Transaction Refinery and the factory “floor” with its Insight Pipelines. The refinery is responsible for distilling, cleaning, identifying and enriching merchants from the transactions it is being fed. It achieves this by running a large number of in-memory python-models that have been specifically adapted to the type of transactions they are meant to process. All the different payment transaction types have their own data quality issues that must be dealt with uniquely. The Insight Pipelines then use the enriched data from the refinery together with customer data and SQL-based specialized models to produce all the different variants of insight that the Insight Factory delivers. The entire process is being monitored for quality and the quality state of each time period processed is available for later inspection. The result is that we finally have a scalable solution for automated insight production.
Future expansion
This year the factory enters its next phase of development, rolling out insight to a much larger audience internally in DNB and externally to the Norwegian market. It will be thrilling to witness the journey from here, as the technical solution built by three techie guys in DNB take on its own life and becomes a potential game-changer in understanding market behavior - and potentially starts the transformation of much of Norway’s traditional businesses into data-driven businesses.
Written by Anders Granerød, Product Manager for the DNB Insight Factory.
Shoutout to my core-team member Nils Håkon Delphin and Jon-Mikkel Ryen Korsvik!