Netlight partnered with a leading e-commerce price-comparison platform to enhance its analytics capabilities on AWS – building a scalable cloud-native event collection system with real-time validation and efficient ETL processing – slashing data latency from 24-48 hours to just 15 minutes, enforcing data quality and privacy, and delivering a cost-transparent, high-traffic-ready architecture.
The client offers a prominent E-Commerce platform that empowers users to compare prices and products across various industries. With a reputation for providing comprehensive and unbiased information to consumers, the platform has become a go-to platform for informed shopping decisions.
The client needed to enhance its analytics capabilities for deeper user insights without compromising privacy. Failure to do so risked lagging in market analysis, overlooking trends, and making uninformed decisions that could harm user satisfaction and business growth. They enlisted Netlight to create a cloud event collection system and an analytics framework that balanced centralization and decentralization for efficient data handling across teams.
Netlight leveraged its extensive AWS expertise to build a scalable cloud-native solution capable of easily handling high-traffic events like Black Friday. Specifically, the solution featured a central event processing layer for data validation against predefined schemas, with storage in team-specific data lake buckets powered by AWS Elastic Container Service (ECS) for resilient and scalable event collection. A two-step collection-validation architecture and AWS Kinesis ensured data accuracy and stability. Further, we implemented efficient ETL processes via AWS Glue and Amazon Kinesis Firehose to transform raw data into actionable insights. Custom recovery tools are provided to minimize data loss. We used AWS Lambda functions to optimize file sizes, improving query times by 10-100x, and CloudWatch provided monitoring and transparency with detailed alerting.
What are your ambitions? We want to help you achieve them. Drop us an email and we will get back to you as soon as possible.
