Real-Time Supply-Chain Data Platform
Event-driven open lakehouse for end-to-end supply-chain visibility.
Problem
Enterprise supply chains stitch together ERPs, warehouses, and logistics systems whose state changes constantly. Batch ETL leaves operators looking at stale snapshots, so stockouts, late shipments, and demand swings are noticed hours after they happen.
Solution
An event-driven open lakehouse that captures changes at the source, streams them through real-time processing, and lands them in an open table format queryable by analysts and ML alike. Decisions are made on fresh data, with a RAG layer that lets users ask questions of the platform in natural language.
Architecture
Operational databases emit row-level changes via Debezium CDC into Kafka. Apache Flink performs streaming transforms and enrichment, writing to Apache Iceberg tables backed by MinIO object storage. Trino serves federated SQL, dbt models the analytics layer, and Dagster orchestrates pipelines and table maintenance. MLflow tracks models, and a FastAPI service exposes a RAG interface over the curated data.