Driving Retail Digital Transformation for Pandora
The company needed to ensure seamless integration between multiple diverse systems, making data exchange between them fast, efficient, and accurate.
We helped our partner to build a customized Kafka integration landscape with a precise focus on Pandora’s business needs.
The new integration platform helped our partner increase scalability, efficiency, and data integrity. The product created a digital foundation for integrating all business processes and unlocking the value of real-time data analytics.
Managed Delivery, Team Extension
Event-driven architecture, microservices, Cloud, DevOps, Data Analytics, .Net, Java, Kafka, ServiceNow, Business Analysis
02Success story in detail: About our client
Pandora is a famous European Jewelry manufacturer and retailer selling its products in over 60 countries on 6 continents with more than 7,000 points of sale. The jewelry giant constantly expands its global presence.
03Business challenge: Seamless integration between multiple diverse systems
When expanding to new markets, which had specific regulatory needs and called for different functionality, Pandora implemented numerous third-party vendor systems supported by external contractors and using different technologies.
All of these systems needed to exchange data with each other. Integrations were often developed as batch-based data exchanges with a lot of non-reusable data, per market. As a consequence, each point-to-point integration became a long and resource-consuming project, which complicated the integration landscape further with each new market introduced.
So Pandora needed to find a solution that would help to:
- Reduce latency and increase data accuracy
- Increase scalability
- Eliminate data duplication and usage of extra memory and processing resources
- Reduce maintenance costs and complexity
- Make the efficiency and correctness of data exchange more trackable
- Eliminate operational and analytical data discrepancies
To solve these problems and build a solid foundation for digital transformation, Pandora decided to develop a scalable integration hub based on Event-Driven Architecture and Kafka.
- Ensure seamless integration between many different systems, making data exchange between them fast, efficient, and accurate
- Integrate new and old markets, unify functionality, and standardize data objects
- Shift from many-to-many to many-to-one
- Provide customers with personalized, omnichannel user experience (e.g., order online - pick up in store)
- Enable omnichannel inventory management and more efficient logistics and transportation management
- Provide real-time visibility into all business processes (via dashboards and BI reports)
- Ensure future-proof scaling
- Reduce development and maintenance costs
- Optimize performance
04Zoolatech Approach: From in-depth analysis to implementation
The Zoolatech Team took a product centric approach with building the EDA integration platform and this has allowed us to quickly deliver value in a few months with the MVP product. Over many releases they have built an impressive track record of quality work on time and within budget.
They also leveraged their past experience and helped us accelerate adoption of the platform among stakeholders. We're in the second year of a five year digital transformation ; I'm thrilled and grateful to have Zoolatech as a strategic partner in this journey.
Product Discovery Phase
We analyzed the legacy system and its limitations as well as investigated the current and upcoming needs of Pandora. With a precise focus on Pandora’s business needs, we helped our client choose the most suitable tech stack and architecture for the platform to reduce maintenance costs, withstand high load, and mitigate risks.
PoC, MVP, and Further Development
We provided Pandora with a strong engineering team. The team built a PoC and MVP and proceeded with further product development. Also, the Zoolatech team prepared a set of clear engineering standards to drive operational excellence and optimize costs.
Best DevOps Practices:
We helped our client to streamline, unify, and standardize all DevOps processes as well as apply best DevOps practices.
- Security: Secretless approach and workload identity, secret key rotation, private endpoints.
- Automation: CI/CD pipeline (running a pipeline after each merge request), Infrastructure as a Code; building templates for deploying new services.
Site Reliability Engineering
Our site reliability engineers analyze Pandora’s IT ecosystem, monitor their software reliability in the production environment and help make Pandora’s software systems more reliable and scalable.
Omnichannel Inventory Management
We help our partner integrate their ERP, Transportation management, and Warehouse management systems and make the data exchange between these diverse systems fast, efficient, and consistent. It helps to provide real-time or near real-time visibility into all inventory levels and optimize logistics and transportation management.
Also, we help accelerate adoption of the platform among other teams by building templates, writing documentation, and transferring knowledge. What’s more, it helps to considerably reduce development time and costs.
05Implementation: Custom Kafka integration landscape
Having analyzed the main technological challenges and keeping a new vision in mind, the Zoolatech team developed a platform called Nexus. Nexus is a custom cloud-based integration hub based on EDA and Apache Kafka. It connects producers and consumers of real time data and facilitates the data exchange between them (e.g producer - ERP system; consumers - Warehouse management system, Accounting system, Reporting system).
The conceptual approach of the data exchange between different systems is based on business events. A business event comprises a data object and a real-time notification about something important that has happened (e.g., B2B order created: the notification about the order creation and the details of the order). Any interested consumer can get these notifications for operational processing and analytical purposes. Nexus allows the reuse of business event notifications streamed by their owners, helping to standardize and unify them.
Nexus utilizes Apache Kafka open-source distributed event streaming technology with extended infrastructure around it, allowing seamless connectivity to stream and to consume data.
The key challenge was to integrate Nexus with Pandora’s third-party vendor systems, which are large, have different architecture, functionality and data structures.
To solve the challenge and facilitate integration between Nexus and different business systems, our team has been implementing facades. A facade is a middleware layer that helps to address compatibility issues, modify and unify data, and do error-handling.
To make the platform more efficient, scalable, and secure, our team has extended the Apache Kafka integration hub with additional custom services and systems, such as:
Nexus Management System
It comprises an internal storage for history tracking and UI to search and debug single events. The system helps to improve the user experience for all end-users of the platform.
It helps to eliminate excessive copies of data and significantly decrease storage capacity requirements.
It allows replacing sensitive data with tokens, improving data security, and remaining compliant with regulatory requirements across markets.
What’s more, the messages streamed through Kafka are defined in the Central Schema Registry, which allows transparent data definition, standardization, reusage and data quality control.
06What we have achieved together: Real-time data streaming platform
Together with the client, we redesigned Pandora’s analytics and integration platform from a batch point to point solution to near real time events that can be reused. We’ve built a scalable integration hub based on Event-Driven Architecture, with extended infrastructure around it to meet Pandora's specific needs.
The platform is characterized by high scalability, low cost of maintenance, and high fault-tolerance. The new Event-Driven Architecture builds on a number of principles:
- Clear engineering standards in place for all integrations to ensure reusability, reduce costs, and drive operational excellence
- Making agile product teams responsible for data ingestion and consumption
- Exposing reusable data objects on the integration platform and business logic moved to the consumption and production layers
- Consistency and reusability of data objects
- Accessibility and reusability of integration modules
- Logging, failed message storage, and retry mechanisms embedded on all integration points to avoid data loss
- Event notification management for production maintenance and support
- Accessibility of the platform functionality for facades both within and outside the used network zones
- First-stream data ownership approach, which allows to eliminate discrepancies in data usage
- Shared-code client library diversity, which corresponds to the client’s engineering standards and personnel hiring priorities
In addition, the new custom platform, which is based on the open-source data streaming tool, fully belongs to Pandora and its maintenance is more cost-efficient compared to other third-party vendor solutions.
07Value delivered: Increased scalability, reliability, and efficiency
The product created a digital foundation for integrating all business processes and operating in a digital manner end-to end. It helps our partner to:
- Ensure seamless integration between multiple Pandora’s systems, making data exchange between them fast, efficient, and accurate
- Decrease latency from 36 hours to milliseconds and improve data accuracy
- Unlock the value of real-time data analytics and provide visibility into all business process
- Build a solid foundation for Advanced Data Analytics and AI
- Provide customers with omnichannel user experience
- Increase fault-tolerance
- Enable more efficient logistics and transportation management
- Ensure future-proof scalability
- Reduce maintenance costs and complexity
In 2021 Pandora started its Digital transformation to improve its consumer personalization capabilities and develop a scalable platform to support the company's Phoenix growth strategy. We needed a digital foundation that would bring together all of the various business processes which would allow us to operate in a digital manner end to end. This required us to redesign our analytics and integration platform from a batch point to point solution to near real time events that can be re-used. This will also be critical to our ERP and Business transformation initiative that we started this year.
We needed a partner with a track record of technical excellence and delivery in Event Driven Architectures and Kafka in particular. After an extensive assessment of several major service providers we selected Zoolatech due to their extensive track record in EDA and experience working with retailers.