MLOps

ML System Design (Feature Store, Model Serving)

An ML system in production is mostly a data system with a model in the middle. The model is the smallest, most-discussed, and least-troublesome part. The hard parts are training data pipelines, feature freshness and parity between training and serving, the feature store that enforces that parity, model deployment and rollback, online and offline evaluation, and the operational concern that the model silently degrades as the world drifts. This lesson covers the canonical reference architecture: training pipeline, feature store with online and offline halves, model registry, serving infrastructure, monitoring, and the feedback loop. It is the senior-level mental model for designing 'add ML to product X' without falling into the standard traps.

Design time-aware evaluation and drift response

Design rolling-origin backtests, capacity-bound metrics, label-delay handling, and drift monitoring for a synthetic card-fraud model whose chargebacks arrive weeks late, then decide whether to promote, shadow, or hold a retrained candidate.

Design a model specialization program

Design a specialization program for a fictional hint model within GPU-hour and expert-review budgets: choose learning signals and parameterization, protect splits and evaluation, and plan a release with rollback. Then investigate a candidate that won a preference comparison while experts found wrong explanations.