MLOps basics: from a notebook to production

MLOps is the engineering practice of getting machine learning models into production and keeping them working there. The MLOps basics are four things: version everything that produced the model, automate testing and release, serve predictions in a way that fits how they are used, and monitor the model after launch because its quality can fall without anyone touching the code. A notebook proves a model can work once. MLOps makes it work every day, for other people, in a way you can repair.

These skills are core for the AI platform and MLOps role and for ML engineers, and they come after evaluation on the AI engineer roadmap because you cannot automate a quality check you have not defined.

What is MLOps?

MLOps is DevOps extended to systems whose behaviour depends on data as well as code. In ordinary software, the same code gives the same behaviour. In machine learning, behaviour is set by code, training data and the resulting model artefact together, and the world the model predicts keeps changing.

That creates problems ordinary software does not have: a model that cannot be rebuilt because nobody recorded which data it used; features computed one way in training and another in production; a model whose accuracy quietly decays as customer behaviour shifts. MLOps is the set of habits and tools that prevent those.

What are the steps from a notebook to production?

The steps from a notebook to production turn exploratory code into a pipeline that can run without you. Each step removes one way the system depends on a person's laptop or memory.

The first step is plain Python engineering, covered in Python for AI engineers.

What needs to be versioned in machine learning?

Machine learning needs four things versioned together: code, data, configuration and the model artefact. The test is reproducibility. Given a model running in production, you should be able to say exactly which code, which data and which settings produced it, and rebuild it.

Experiment tracking ties these together. Each training run records its code version, data version, parameters and metrics, so the question "why is this model better than last month's?" has an answer.

How is CI/CD different for machine learning?

CI/CD for machine learning tests data and model quality in addition to code, and it deploys a pipeline as well as a service. A code change, a data change or a scheduled retrain can each trigger a release, and each needs checks.

Three kinds of test run before a model ships:

• Code tests. Unit tests for feature functions, preprocessing and prediction logic, as in any software project.
• Data tests. Checks that incoming data matches expectations: schema, ranges, missing-value rates, no duplicated keys. Bad data is the most frequent cause of bad models.
• Model tests. The candidate must meet a minimum score on held-out data and must not be worse than the current production model, overall and on important segments.

Release itself is gradual. A shadow deployment runs the new model alongside the old one without using its predictions, so the two can be compared on real traffic. A canary release sends a small share of traffic to the new model first. Both depend on a fast rollback, which the model registry makes simple: redeploy the previous version.

How are models served in production?

Models are served in one of three patterns, chosen by how fresh a prediction must be and how quickly it is needed.

Batch is the right default when it meets the need, because it avoids most operational difficulty. Online serving introduces the classic failure of training-serving skew: a feature is calculated differently in the live service than it was in training, so the model receives inputs it has never seen. The fix is one shared implementation of each feature, used in both places.

What is model drift and how do you monitor it?

Model drift is the decline in a model's usefulness as the data it sees in production moves away from the data it was trained on. Data drift means the inputs have changed, for example a new customer segment. Concept drift means the relationship between inputs and outcome has changed, for example fraud tactics evolving. Either can happen with no change to code.

Monitoring works at three levels: the service (latency, errors, throughput), the data (input and prediction distributions) and the outcome (accuracy once true labels arrive). True labels are often delayed by days or weeks, so drift in inputs is the early warning. A common measure is the population stability index, which compares a feature's distribution in production with its distribution at training time:

import numpy as np


def psi(expected: np.ndarray, actual: np.ndarray, bins: int = 10) -> float:
    """Population stability index between a training and a production sample."""
    edges = np.quantile(expected, np.linspace(0, 1, bins + 1))
    edges[0], edges[-1] = -np.inf, np.inf
    e = np.histogram(expected, edges)[0] / len(expected)
    a = np.histogram(actual, edges)[0] / len(actual)
    e, a = np.clip(e, 1e-6, None), np.clip(a, 1e-6, None)
    return float(np.sum((a - e) * np.log(a / e)))

A value near zero means the two distributions match; larger values mean a bigger shift. Alert thresholds should be set from how the feature has varied historically, and an alert should lead to a decision: investigate, retrain or roll back.

What changes for LLM applications?

LLM applications keep the same principles and change what is versioned and monitored. The artefact is often a prompt, a retrieval index and a reference to a hosted model instead of weights you trained.

The release gate for an LLM application is an evaluation run, described in how to evaluate an LLM application.

How do interviews test MLOps?

Interviews test MLOps mostly through design and incident questions. Typical prompts are to take a described model to production, to explain what you would monitor, or to diagnose a model whose accuracy fell after launch. Good answers name what is versioned, what gates a release, which serving pattern fits and why, and what triggers a rollback.

These questions form a large part of the ML system design interview. The day-to-day work is described in the MLOps and AI platform engineer role, and a deployed, monitored model is one of the strongest AI portfolio projects. To compare how much operations depth each track expects, use the role explorer.