Machine learning fundamentals every AI engineer needs
Machine learning fundamentals that interviews return to: generalisation, bias and variance, data splits, leakage, baselines and choosing the right metric.
On this page
Machine learning fundamentals are a short list of ideas about trust: whether a model will work on data it has not seen, and how you know. They are generalisation, the bias-variance trade-off, honest data splits, leakage, baselines and metric choice. Algorithms change with each project, but these ideas apply to a logistic regression and to an LLM application alike, which is why interviewers return to them at every level of seniority.
In the AI engineer roadmap, ML fundamentals and model evaluation are the stage that most later skills depend on, from experiment tracking to ML system design.
What are the fundamentals of machine learning?
The fundamentals of machine learning are the concepts that hold for any model that learns from examples. A model is fitted to training data, but its job is to perform on new data, and every fundamental idea is about closing or measuring the gap between the two.
| Idea | The question it answers | What goes wrong without it |
|---|---|---|
| Generalisation | Will the model work on unseen data? | A model that memorises its training set |
| Bias and variance | Is the model too simple or too flexible? | Tuning the wrong thing for weeks |
| Data splits | How do I estimate real performance? | A test score that means nothing |
| Leakage | Is the model using information it will not have? | Excellent offline results, failure in production |
| Baselines | Is the model better than something trivial? | Complexity with no measured benefit |
| Metric choice | What does "better" mean for this problem? | Optimising a number the business does not care about |
What is the bias-variance trade-off?
The bias-variance trade-off is the tension between a model too simple to capture the pattern and a model so flexible that it fits noise. Bias is error from wrong assumptions: a straight line fitted to a curved relationship. Variance is error from sensitivity to the particular training sample: a deep tree that changes shape when a few rows change. Making a model more flexible lowers bias and raises variance.
In practice you diagnose which problem you have by comparing training and validation error.
Underfitting (high bias)
- Training error is high
- Validation error is close to training error
- More data does not help
- Fix: richer features, a more flexible model, less regularisation
Overfitting (high variance)
- Training error is low
- Validation error is much higher
- More data usually helps
- Fix: regularisation, a simpler model, early stopping, more data
This diagnosis is the standard interview follow-up to any project description: "your validation score was lower than your training score, what did you do?" A good answer names the gap, the cause you suspected and the change you made, in that order.
Why do you need train, validation and test sets?
You need three sets because each decision you make using a dataset uses up that dataset's honesty. The training set fits the model. The validation set is for choices: features, hyperparameters, which model to keep. The test set is touched once, at the end, to estimate performance on new data. If you tune against the test set, the score becomes optimistic, because you have selected the model that happened to suit those particular rows.
How you split matters as much as whether you split:
- Time-ordered data must be split by time: train on the past, validate on the future. A random split lets the model see the future.
- Grouped data, such as many rows per customer or per patient, must keep each group entirely on one side. Otherwise the model recognises the customer instead of learning the pattern.
- Imbalanced classes need stratified splits so that rare cases appear in every set.
What is data leakage?
Data leakage is when information that would not be available at prediction time reaches the model during training. The model appears to perform very well, and the performance vanishes in production. Leakage takes three main forms:
- Target leakage. A feature is a consequence of the outcome. A "refund issued" flag in a model predicting complaints is the answer in disguise.
- Train-test contamination. Preprocessing is fitted on all the data before splitting, so statistics from the test rows shape the training features. Duplicated rows across the split do the same thing.
- Temporal leakage. Features are computed using events after the prediction moment. This is usually a query problem, covered in SQL for AI and ML roles.
The fix for contamination is mechanical: put every fitted preprocessing step inside a pipeline, so that each cross-validation fold fits it on its own training portion only.
from sklearn.datasets import make_classification
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=2000, weights=[0.9], random_state=0)
# The scaler is refitted inside each fold, so no validation rows leak into it.
model = make_pipeline(StandardScaler(), LogisticRegression(max_iter=1000))
scores = cross_val_score(model, X, y, cv=5, scoring="average_precision")
print(scores.mean())Why should you always start with a baseline?
You should start with a baseline because a score has no meaning on its own. A baseline is the simplest thing that could work: predicting the majority class, predicting last week's value, a rule the business already uses, or a linear model on a few obvious features. Every later model is judged by how much it improves on that.
Baselines protect you in two ways. They expose problems early, since a complex model that cannot beat the majority class points to a bug or a hopeless feature set. And they give the cost-benefit argument that a team needs: if gradient-boosted trees beat a simple rule by a small margin, the extra training, serving and monitoring effort may not be worth it. Interviewers ask "what was your baseline?" precisely because candidates who skipped it cannot say whether their model was good.
How do you choose the right evaluation metric?
You choose the evaluation metric by working out what each type of mistake costs and picking the metric that penalises the expensive ones. Accuracy is the default and is often wrong: when one class is rare, a model that never predicts it can still be highly accurate and completely useless.
| Situation | Metric to consider | Reason |
|---|---|---|
| Missing a positive is costly, such as fraud or disease | Recall, at a fixed precision | You care about catching cases |
| A false alarm is costly, such as blocking a customer | Precision, at a fixed recall | You care about not acting wrongly |
| Rare positive class, ranking quality matters | Precision-recall curve and its area | It ignores the mass of easy negatives |
| Predicted probabilities drive decisions | Calibration and log loss | The number itself must be trustworthy |
| Predicting a quantity | Mean absolute or squared error | Choose by how much large errors should hurt |
Two further points come up in interviews. A classifier's threshold is a business decision, not a fixed constant, and moving it trades precision against recall. And an offline metric is a proxy: the real test is whether the model improves the outcome it was built for, which needs an online experiment.
How do interviews test machine learning fundamentals?
Interviews test machine learning fundamentals by asking you to reason about a situation, not to recite definitions. Typical prompts are a model that performs well offline and badly in production, a dataset with a rare positive class, or one of your own projects examined decision by decision.
Frame the problem
What is predicted, when, and what each kind of error costs.
Describe the data and split
Where labels come from, and how the split respects time and groups.
Name the baseline
The simple reference every model was compared against.
Justify the metric
Why this metric matches the cost of errors.
Diagnose and iterate
What the training and validation gap showed, and what you changed.
The same structure carries into machine learning interview questions and scales up into the ML system design interview. The theory behind these ideas is covered in how much maths machine learning needs, and the LLM version of the same discipline in how to evaluate an LLM application.
These fundamentals are core for the ML engineer role and expected of every other AI role at working level. To check which of them you can already explain under questioning, take the free readiness check for your target role.
Key takeaways
- The goal of machine learning is performance on unseen data, so everything depends on measuring that honestly.
- High training error suggests underfitting; a large gap between training and validation error suggests overfitting.
- Data leakage is the most common reason a model looks excellent offline and fails in production.
- A simple baseline tells you whether a complex model has earned its complexity.
- The right metric follows from the cost of each mistake, not from habit; accuracy misleads when classes are imbalanced.
Frequently asked questions
Do GenAI engineers need classical machine learning fundamentals?
Yes. Evaluating an LLM application uses the same ideas: a held-out set of examples, a baseline, metrics matched to the cost of errors and care about leakage between the examples you tune on and the ones you test on. GenAI engineers need less breadth across classical algorithms, but the fundamentals are treated as core.
What is the difference between a parameter and a hyperparameter?
A parameter is a value the model learns from data during training, such as the weights of a linear model or neural network. A hyperparameter is a setting you choose before training, such as tree depth, learning rate or regularisation strength. Hyperparameters are tuned on validation data, never on the test set.
How much data do you need to train a machine learning model?
There is no fixed number. The amount depends on how complex the pattern is, how noisy the labels are and how flexible the model is. A learning curve gives a practical answer: train on increasing fractions of the data and see whether validation performance is still improving. If it has flattened, more data of the same kind will not help much.
When should you use deep learning instead of a classical model?
Use deep learning when the input is unstructured, such as text, images or audio, or when there is enough data for learned representations to beat hand-built features. For tabular business data, gradient-boosted trees and linear models are strong, cheaper to train and easier to explain, so they are the sensible first choice.
What is cross-validation and when is it worth using?
Cross-validation splits the training data into several folds, trains on all but one and validates on the remaining fold, rotating through them. Averaging the scores gives a steadier estimate than a single split. It is most useful on small datasets; for time-ordered data, use splits that always train on the past and validate on the future.