MLOps: Model Registry, CI/CD for Models and Reproducibility
MLOps explained: experiment tracking, model registries, versioned data and code, CI/CD for models, approvals, rollback and reproducible training.
MLOps is the set of practices that make machine learning systems reproducible, testable and safe to change, in the way DevOps does for software. Its core components are experiment tracking (what was tried and how it performed), a model registry (which model versions exist, what produced them, and which is in production) and CI/CD for models (automated training, validation, approval and deployment). The goal is that any production prediction can be traced back to the code, data and configuration that produced the model.
Why ML needs more than normal CI/CD
In ordinary software, behaviour is determined by code. In ML, behaviour is determined by code and data and training configuration, and a model can be retrained with identical code and still behave differently. So you must version all three, test the data as well as the code, and validate the model's quality, not just that it builds.
The model lifecycle
Experiment tracking
Every training run records its parameters, code version, data snapshot ID, environment, metrics and output artifacts. That turns "the model from three weeks ago was better, what did we change?" from archaeology into a query, and it's the foundation for reproducing any model later.
What a model registry holds
| Field | Why |
|---|---|
| Model name and version | Unique, immutable identity for each artifact |
| Artifact location and format | What serving loads |
| Lineage: training run, code commit, data snapshot, features used | Reproducibility and debugging |
| Evaluation results, including slices | Promotion decisions and audits |
| Stage or alias (candidate, staging, production, archived) | Which version each environment should serve |
| Owner, approvals and change notes | Accountability, especially in regulated domains |
| Serving signature: input schema and output format | Prevents deploying a model the service can't call |
CI/CD for models
- Code checks: unit tests for feature transformations and training code, as in any CI pipeline.
- Data checks: schema, volume and distribution validation of the training data before training starts. See data quality.
- Model checks: the candidate must beat or match the production model on a fixed benchmark, on every important slice, within latency and size limits.
- Approval: automated for low-risk models; a human sign-off for high-risk ones such as credit decisions.
- Deployment: shadow or canary rollout with automatic rollback on regressions, and an A/B test when business impact is uncertain.
- Continuous training: a scheduled or triggered pipeline retrains on fresh data and runs the same gates, so retraining is routine rather than heroic.
Make rollback boring: keep the previous production version in the registry with its serving configuration, and make switching back a one-line change of an alias, not a retraining job.
Maturity levels
| Level | What it looks like |
|---|---|
| Manual | Notebooks, hand-copied model files, deployment by an engineer; hard to reproduce |
| Automated training | Training is a pipeline with tracked runs and a registry; deployment still manual |
| Automated delivery | Validated models are promoted and deployed through CI/CD with rollback |
| Continuous training | Data or schedule triggers retraining; monitoring closes the loop automatically |
Where it’s used
Every credit model version has full lineage, documented validation and a recorded approval before it serves decisions.
Daily automated retraining promotes a new model only if it beats production on offline benchmarks and passes a canary.
A shared registry and pipeline templates let many product teams deploy models the same way.
In an interview
- Version code, data and configuration together, and record lineage in the registry.
- Gate promotion on benchmark metrics, slice checks and latency, not just training success.
- Deploy with shadow or canary and make rollback a registry alias switch.
- Close the loop: monitoring triggers retraining through the same gates.
Further reading
- MLOps: Continuous delivery and automation pipelines in machine learningGoogle Cloud Architecture Center
- MLflow Model RegistryMLflow documentation