NYC Taxi Demand Forecasting: MLOps Lifecycle

Welcome to the MLOps Lab! This project is a showcase of a production-quality, end-to-end Machine Learning Operations (MLOps) lifecycle.

We implement the principles of Continuous Delivery for Machine Learning (CD4ML) to build a reproducible, automated, and audited system using industry-standard tools:

  • Feast as our Feature Store.
  • MLflow as our Model Registry & Experiment Tracking store.
  • DuckDB and SQLite as our local data stores.
  • FastAPI for real-time online inference serving.

MLOps Lifecycle Stages

DiagramMlops Lifecycle
  1. Problem Framing: Framing the business objective and prediction task.
  2. Data Collection: Gathering raw Yellow Taxi TLC trip records.
  3. Exploratory Data Analysis: Exploring raw dataset patterns and distributions.
  4. Data Preprocessing: Cleaning transaction logs and grouping into hourly demand.
  5. Feature Engineering: Managing features with Feast entities and views.
  6. Train: Fitting a Ridge Regressor on historical features.
  7. Tune: Optimizing parameters via MLflow sweeps.
  8. Evaluate: Enforcing quality gates on the candidate models.
  9. Deploy: Serving models in real-time or via batch scoring.
  10. Monitoring: Detecting drift and observing metrics.

How to run the project locally

Each stage of the MLOps lifecycle is executable via make targets. To run the entire pipeline end-to-end:

make collect
make preprocess
make features
make tune
make train
make evaluate
make deploy
make monitor