Implementing Industrial Machine Streaming Data Analytics Solution and BI Platform
ManufacturingBusiness Impacts
Unified monitoring for 7 factory tiers in the UK and Hungary
Cut data processing time from hours to 10-15 minutes
Automated pipelines and QuickSight dashboards
Customer Key Facts
- Country : United States
- Industry : Manufacturing
Problem Context
The client, a top global machinery provider for two-piece can and end-making, received IT and business teams’ data from machine sensors individually, making the creation of unified data views difficult. Their IT team resorted to utilizing MS Excel to process this data and create ad-hoc reports, which was time-consuming. This made it difficult to derive insights from machine data causing delays in business planning and execution, as well as hindered the identification of opportunities to enhance machine performance. Along with that, no automation or standard governance framework could identify issues with the data collection and management systems.
Challenges
- Lack of automated processes and standardized governance frameworks to detect issues in the data collection and management systems.
- Real-time monitoring of logs, prompt actions, and the seamless onboarding of new sources with minimal adjustments.
Technologies Used
Amazon MSK
Amazon Redshift
AWS Glue
Amazon QuickSight
Amazon OpenSearch
Solution
Quantiphi created a serverless, fully-managed, streaming ELT pipeline and a Lake House solution to provide near real-time analytics on manufacturing factory data. Using Amazon QuickSight, a near real time dashboard reporting solution was established, allowing the client to get insights, identify bottlenecks, inform operators of approaching failures, and make recommendations for the next best actions. To monitor and manage the entire near real-time data ingestion and processing pipeline a centralized logs monitoring dashboard was created using Amazon Opensearch.
Results
- 200 records processed per minute from factory sensors
- ~6 GB of data processed for two factories in a year
- 3 QuickSight dashboards were created (covering 10-12 industry standard KPIs for users)
- Provided ingestion pipeline logs monitoring framework using Amazon Cloudwatch, AWS Lambda and Amazon Opensearch that triggered alarms to admins in case of issues/failures in the pipelines
