In the banking, financial services, and insurance (BFSI) industry, data plays a critical role in designing innovative products, enhancing customer experience, improving risk management, and increasing operational efficiency.
Banks and insurers must ensure flexible data platforms to ingest and access the right data at the right time. As highlighted in an earlier paper, embracing data mesh architecture can help address key challenges of legacy data platforms and help BFSI firms move toward a data-as-a-product mindset, leading to data democratization.
Data mesh architecture is gaining traction in the BFSI industry given its ability to address the challenges of legacy data platforms, rapidly implement innovative business use cases through domain-oriented federated data ownership and drive an organization-wide culture of innovation. A decentralized architecture approach like data mesh can infuse the agility to unlock increased business value from data and enable a business domain-centric platform to leverage quality data to rapidly develop innovative data products. Leveraging data originating from within and beyond organizational boundaries, improving data quality to facilitate superior business decision-making, and enhancing customer experience also become easier with a data mesh architecture. Transforming to a data mesh architecture, however, will require a fresh thought process on how decentralization and product thinking can address key data challenges. Before embarking on an implementation journey, BFSI firms will need to resolve questions around how the data mesh architecture should be implemented, the challenges, the roadmap, and critical steps for successful execution.
While the business case for embracing data mesh architecture is clear, as with many other emerging and evolving concepts, it comes with some pitfalls. The core principles of the architecture are well understood at a broad level, but the actual implementation poses some key challenges and risks.
In our experience, the benefits of embracing data mesh architecture far offset the challenges.
A leading European bank wanted to become a digitally-savvy, data-driven, and innovation-friendly organization. The bank had earlier simplified its enterprise data warehouse into smaller but focused domain data marts. It now wanted to leverage cloud technologies and the data mesh architecture to transform its data platform into an agile innovation lab in order to develop customer-focused products as well as for regulatory reporting and analytics.
The bank implemented the data mesh architecture and realized multiple benefits:
While it is clear that BFSI firms must embrace data mesh architecture to enable business-centric data platforms, successful execution will necessitate focus on some key aspects.
In BFSI firms, business and IT ownerships have evolved over a period of time. Aligning the data mesh architecture with business demands consensus on ownership and management of domain-centric data platforms.
The other important aspect is to enable self-service to create, consume, and manage data products, which will demand robust data governance and metadata management across the enterprise. Creating data products on domain-centric data platforms will require insights on customers, products, sales and marketing, and other business and corporate functions. In addition, data scientists will need logical sets of data for cross domain leverage as well as to identify use cases and create targeted and specific business products for end users and business applications. This will help end users in domains such as payments, lending, risk management and so on to easily access quality data for quick business decision-making.
Another key aspect to be considered is visualizing the target state data mesh architecture. In our view, the architecture must include certain key components:
Data-infrastructure-as-a-platform
Domain-agnostic infrastructure that can be leveraged through the as-a-platform model by domain teams for data ingestion, storage, compute, monitoring, alerting, logging and sharing is an imperative. The initial responsibility to develop this infrastructure, comprising universal ingestion framework, data transformation framework, coding standards, testing framework, best practices and so on, will lie with the central team. It is also important to ensure that domain teams such as insurance policy administration, claims processing, and finance can access the common data infrastructure to create a standardized data platform to meet domain data requirements.
Domain-oriented data architecture and organization
BFSI firms must develop domain oriented data architecture, rather than technology-driven architecture, by leveraging the data-infrastructure-as-a-platform. In terms of organization, domain teams must include data owners, data stewards, system owners, data architects, data engineers, DevSecOps engineers, and business data analysts equipped with the requisite functional and technical knowledge to work independently of other domains. Such domain teams must also develop the capabilities to serve data as-a-product to other domains, business intelligence users, and data scientists as well as to analytics and operational teams. Data product owners for BFSI domains like customer, products, finance and risk should be an integral part of the teams.
Central data marketplace
A central data marketplace can help ease the process of exchanging data products between producers and consumers as well as across domains and external parties, thereby improving data currency. The marketplace model can also help streamline product cataloguing, discovery, and provisioning. In addition, adopting the marketplace based data-as-a-product concept can facilitate the smooth exchange of data with external ecosystem players such as fintechs and insurtechs as well as cross-industry players from retail, travel, hospitality and so on. Data products come with different dimensions (see Figure 1).
Self-serve data platform
Defining the governance tools for data catalogue, data lineage, and data quality will help lay the foundation for a successful self-serve data platform. Implementing a data marketplace solution can help in managing agreements between data producers and consumers across sourcing and consumption of data products. A flexible meta model is the key to interoperability of data products, which in turn will infuse the agility needed to develop innovative offerings such as personalized products with dynamic pricing and fraud and anti-money laundering (AML) offerings.
Critical questions now arise around how BFSI organizations should start on the data mesh journey and the key steps that must be taken to transform legacy data platforms to the data mesh architecture leveraging modern technologies like cloud (see Figure 2).
As with any other IT system or architecture, laying down an efficient governance structure is crucial to the successful functioning of the data mesh architecture.
Having a federated model with a team at each domain level can help handle all data governance activities across areas like business glossary and technical and operational metadata resulting in enhanced data quality. Data owners, stewards, data governance experts along with data engineering teams must be a core part of the governance team defined at the domain level. Roles and responsibilities must be divided between different actors both at the domain as well as the central level (see Figure 3). Ownership of data products needs to be established along with data ownership within the realm of data domains. Data product owners must be held responsible for conceptualizing, developing, enhancing, and measuring the usefulness of data products along with improving the quality and richness of the data for internal as well as external ecosystems.
BFSI firms face a plethora of data challenges—legacy and monolithic data platforms, huge volumes of data but limited insights, rigid infrastructure, and a complex regulatory landscape.
The data mesh architecture can help firms overcome many obstructions and drive innovation by leveraging contextual data depending on domain requirements. In addition, embracing the data-as-a-product principle can help unlock value by facilitating monetization of data assets internally or even externally in the future. While implementation is complex and poses tough problems, in our view, the benefits of moving to a data mesh architecture far outweigh the difficulties. It is well worth the effort—banks and insurers that move quickly will gain a lead by realizing the benefits of an efficient and futuristic data platform.