As the business was growing and scaling exponentially, the data flow was increasing daily, and new challenges were arising:

• Scale the system to support the exponential SaaS customer base growth, speed up the onboarding process for new customers.
• Ensure that increased load or data quality issue for some customers is not affecting others.
• Ensure bulletproof data privacy and client experience compartmentalization.
• Ensure that the complexity of the solution and maintenance cost would both remain under control as the quantity of customers increased, and collected data and customizations grew exponentially.

The team created a unique architecture for collecting and storing petabytes of data which allowed analyzing it to generate meaningful insights and statistics. Customers were isolated from each other while the code and core functions were shared. Whenever a new feature was added for one customer, it was possible to reuse it for others. At the same time, the design guarantees data privacy and security for each customer.

Many efforts were spent to create a so-called “Normalizer” which helps to use data that might not be complete or could be broken. It was even possible to make a generic data model for all customers regardless of the data that is received. Finally, the newly-created architecture allowed smooth onboarding of new customers regardless of the data structures and business processes they are using.

By creating a solution which ensures an extreme level of data privacy and security, the team leveraged the platform’s anonymized knowledge for the mutual benefit of all the platform clients.