The Machine Learning Workflow

Imagine a model that predicts which customers will cancel their phone contract with 99% accuracy — and then discovering it learned from a column called cancellation_fee_paid. Of course it was accurate: it was reading the answer. Nothing in the code was wrong; the workflow was. This page teaches the workflow — the sequence of steps and checks that turns “the computer printed a high number” into “this model will work on customers it has never met”.

Framing the problem

If you have read How Machines Learn, you know the idea: instead of writing rules, you show a program many examples with answers and let it find the pattern. In practice that is called supervised learning, and before writing any code you need to pin down three things.

• What is one example? A patient, a house listing, a loan application, a review. Each becomes one row.
• What are the features? The facts you will know at the moment you need a prediction — floor area, location, number of rooms. By convention the table of features is called X.
• What is the label? The answer you want predicted — the sale price, or “spam/not spam”. The column of labels is called y.

The type of label decides the type of problem.

from sklearn.datasets import load_diabetes
from sklearn.dummy import DummyRegressor
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error
from sklearn.model_selection import train_test_split

X, y = load_diabetes(return_X_y=True, as_frame=True)   # y = disease progression score
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=42)

for model in [DummyRegressor(strategy="mean"), LinearRegression()]:
    model.fit(X_train, y_train)
    error = mean_absolute_error(y_test, model.predict(X_test))
    print(f"{type(model).__name__:16} average error: {error:.1f}")

DummyRegressor   average error: 65.5
LinearRegression average error: 41.5

A useful test of a framing: could a well-trained human, given only the features, make a decent guess at the label? If not, a model probably cannot either.

The loop at a glance

Here is the whole workflow as a cycle. Step round it once, then we will do each step in code with scikit-learn, the standard Python library for classic machine learning.

Split, then set a baseline

We will use the Breast Cancer Wisconsin dataset that ships with scikit-learn: 569 tumour samples, each described by 30 measurements computed from a microscope image, and labelled malignant or benign. It is small, clean and real — ideal for learning the workflow. (A teaching model on this dataset is not a medical tool; real diagnostic systems need clinical validation and regulatory approval.)

Examples on this page were run with Python 3.13 and scikit-learn 1.9 As of 2026, with fixed random seeds, so you should see the same numbers.

from sklearn.datasets import load_breast_cancer

data = load_breast_cancer(as_frame=True)
X, y = data.data, data.target          # X = features (a table), y = labels

print(X.shape)
print(list(X.columns[:4]))
print(data.target_names)               # label 0 = malignant, 1 = benign
print(y.value_counts())

(569, 30)
['mean radius', 'mean texture', 'mean perimeter', 'mean area']
['malignant' 'benign']
target
1    357
0    212
Name: count, dtype: int64

as_frame=True gives us pandas objects, so X is a DataFrame of 569 rows and 30 feature columns, and y is a column of 0s and 1s. The classes are unbalanced: 357 benign, 212 malignant. Remember that — it matters in a moment.

Now the most important line in this page: the train-test split.

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True, as_frame=True)

X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)
print(X_train.shape, X_test.shape)
print(y_train.mean().round(3), y_test.mean().round(3))   # share of benign in each part

(426, 30) (143, 30)
0.627 0.629

• test_size=0.25 keeps a quarter of the rows (143) aside as the test set. The other 426 are for training.
• stratify=y makes sure both parts have the same mix of classes — 62.7% and 62.9% benign — so the test is fair.
• random_state=42 fixes the shuffle so you get exactly these rows every time. Change it and every number below shifts a little, which is itself a lesson: small test sets give noisy scores.

Before any real model, score a baseline.

from sklearn.datasets import load_breast_cancer
from sklearn.dummy import DummyClassifier
from sklearn.model_selection import train_test_split

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

baseline = DummyClassifier(strategy="most_frequent")
baseline.fit(X_train, y_train)          # "learns" only which label is commonest
print("Baseline accuracy:", round(baseline.score(X_test, y_test), 3))

Baseline accuracy: 0.629

DummyClassifier(strategy="most_frequent") ignores the features entirely and always predicts the commonest label — here “benign”. It scores 62.9%, simply because 62.9% of the test set is benign. That number is now your floor. A model that reports “63% accuracy” on this data has learned nothing, however sophisticated its name. On heavily unbalanced problems — fraud, rare diseases — the baseline can be 99%, which is why accuracy alone can mislead badly.

Fit, predict, evaluate

Every scikit-learn model has the same two core methods: fit learns from training data, and predict makes predictions for new rows. Let us try a decision tree.

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeClassifier

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

tree = DecisionTreeClassifier(random_state=42)
tree.fit(X_train, y_train)                 # learn from the training set only
predictions = tree.predict(X_test)         # predict for patients it has never seen

print("First ten predictions:", predictions[:10].tolist())
print("First ten true labels:", y_test[:10].tolist())
print("Train accuracy:", round(tree.score(X_train, y_train), 3))
print("Test accuracy: ", round(tree.score(X_test, y_test), 3))

First ten predictions: [1, 0, 0, 1, 1, 0, 0, 0, 0, 1]
First ten true labels: [1, 0, 1, 1, 0, 0, 0, 0, 0, 1]
Train accuracy: 1.0
Test accuracy:  0.923

The tree gets eight of the first ten test patients right and 92.3% of the whole test set — far above the 62.9% baseline. But look at the training accuracy: a perfect 1.0. The tree has grown until it fits every training example exactly, including their quirks and noise. That gap between training and test performance is overfitting, and it is why we never judge a model on the data it learned from. A student who memorised last year’s exam paper also scores 100% on last year’s paper.

Pipelines and honest iteration

Now we iterate: try a different model. Logistic regression (explained on Classic Models) works best when features are on similar scales, but our features are not — “mean area” runs into the hundreds and thousands while “mean smoothness” is around 0.1. The fix is feature scaling: StandardScaler shifts and stretches each feature to have average 0 and spread 1.

The scaler has to learn each feature’s average and spread from somewhere — and that somewhere must be the training data only. A scikit-learn pipeline handles this automatically: it chains the steps into one object, and whenever the pipeline is fitted, every step is fitted on the same training rows.

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score, train_test_split
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler

X, y = load_breast_cancer(return_X_y=True, as_frame=True)
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.25, stratify=y, random_state=42
)

model = make_pipeline(StandardScaler(), LogisticRegression())

# iterate using the TRAINING data only: 5-fold cross-validation
scores = cross_val_score(model, X_train, y_train, cv=5)
print("Cross-validation:", scores.round(3), "mean", scores.mean().round(3))

# when you are done choosing, check the test set once
model.fit(X_train, y_train)
print("Test accuracy:", round(model.score(X_test, y_test), 3))

Cross-validation: [0.977 0.976 0.953 0.965 0.976] mean 0.969
Test accuracy: 0.986

Two new ideas here. make_pipeline(StandardScaler(), LogisticRegression()) builds a model that scales and then classifies; you call fit and score on it exactly as on a single model. And cross_val_score runs cross-validation: it splits the training data into five folds, trains five times on four folds and scores on the fifth, and reports all five scores. Their average, 96.9%, is a steadier estimate than any single split, and — crucially — we got it without touching the test set.

That is how honest iteration works. Compare models, features and settings with cross-validation on the training data. When you have made your final choice, look at the test set once and report that number (here 98.6%). If you keep checking the test set after every tweak, you slowly tune your choices to those 143 particular rows, and the test score stops being a fair preview of the future. The spread of the five fold scores (95.3% to 97.7%) is also worth noticing: differences between models smaller than that spread may just be luck.

Data leakage: the brilliant model that isn’t

Data leakage is when information that would not be available at prediction time — from the test rows, or from the answer itself — slips into training. The contract-cancellation example at the top is the blatant kind: a feature that is really a consequence of the label. The subtle kind happens in preprocessing. Here is a deliberately extreme demonstration. We create 5,000 columns of random noise and labels from a coin toss. There is nothing to learn; an honest workflow should score about 50%.

import numpy as np
from sklearn.feature_selection import SelectKBest, f_classif
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import make_pipeline

rng = np.random.default_rng(42)
X = rng.normal(size=(100, 5000))      # 5,000 columns of pure noise
y = rng.integers(0, 2, size=100)      # labels from a coin toss

# WRONG: choose the 20 "best" columns using ALL rows, then test
X_picked = SelectKBest(f_classif, k=20).fit_transform(X, y)
leaky = cross_val_score(LogisticRegression(), X_picked, y, cv=5)
print("Leaky accuracy:", leaky.mean().round(2))

# RIGHT: put the selection inside the pipeline, so each fold picks
# columns from its own training rows only
honest_model = make_pipeline(SelectKBest(f_classif, k=20), LogisticRegression())
honest = cross_val_score(honest_model, X, y, cv=5)
print("Honest accuracy:", honest.mean().round(2))

Leaky accuracy: 0.89
Honest accuracy: 0.52

The leaky version picks the 20 columns most correlated with the labels using every row, and only then cross-validates. With 5,000 random columns, some will happen to line up with the coin tosses — including on the rows later used for testing — so the model looks 89% accurate on pure noise. The honest version puts the column selection inside the pipeline, so each fold chooses columns using its own training rows only, and the score falls to a coin toss, which is the truth. The same trap appears, more quietly, when you scale, fill missing values or remove outliers using the whole dataset before splitting.

• Split first. Before exploring, cleaning statistics or feature selection.
• Put every learned preprocessing step in a pipeline. Scalers, imputers, selectors, encoders.
• Ask of every feature: “would I really know this at the moment I need the prediction?”
• Respect time. If you predict the future, train on the past and test on later data, not on a random shuffle.
• Be suspicious of great results. A score far above what experts expect is more often a leak than a breakthrough.

Consensus Leakage is one of the most common reasons published machine-learning results fail to reproduce; a 2023 review by Kapoor and Narayanan found leakage errors affecting hundreds of papers across many scientific fields.

Key terms on this page

Sources & further reading

• scikit-learn developers, Getting started and Cross-validation: evaluating estimator performance (scikit-learn documentation).
• scikit-learn developers, Common pitfalls and recommended practices — includes data leakage and inconsistent preprocessing.
• F. Pedregosa et al., “Scikit-learn: Machine Learning in Python”, Journal of Machine Learning Research 12 (2011).
• W. N. Street, W. H. Wolberg & O. L. Mangasarian, “Nuclear feature extraction for breast tumor diagnosis”, IS&T/SPIE Biomedical Image Processing (1993) — origin of the Breast Cancer Wisconsin dataset.
• Sayash Kapoor & Arvind Narayanan, “Leakage and the reproducibility crisis in machine-learning-based science”, Patterns 4(9) (2023).
• Shachar Kaufman, Saharon Rosset & Claudia Perlich, “Leakage in data mining: formulation, detection, and avoidance”, KDD (2011).