Introduction to Machine Learning¶

1. Why this matters¶

If you're an engineer in 2026, ML literacy is table stakes. You don't need a PhD — you need to know:

When ML is the right tool (vs rules vs an LLM).
How to frame a business problem as a learning problem.
How to evaluate a model honestly (no leakage, right metric).
How to ship one without it falling over.

This first topic gives you the map. The rest of the chapters drill into each piece.

2. Mental model¶

A model is a function with parameters:

y = f(X; θ)

X — features (inputs).
y — target (output).
θ — parameters the model learns from data.
Training = pick θ that makes predictions close to truth on training data.
Inference = use the trained f on new X.

Three categories, decided by what y is:

Category	What `y` looks like	Example
Supervised — Regression	Continuous number	House price, temperature, salary
Supervised — Classification	Discrete label	Spam/not, churn yes/no, image class
Unsupervised	No `y` — find patterns in `X`	Customer segments, anomalies
Reinforcement	Reward signal	Game playing, recommendation policies

3. Architecture / Flow¶

The end-to-end pipeline — every project follows this:

flowchart LR
    A[1. Frame problem] --> B[2. Collect data]
    B --> C[3. EDA<br/>understand the data]
    C --> D[4. Clean<br/>missing, outliers]
    D --> E[5. Feature engineering<br/>scaling, encoding, new features]
    E --> F[6. Train/test split]
    F --> G[7. Train model]
    G --> H[8. Evaluate<br/>right metric, no leakage]
    H --> I{Good enough?}
    I -->|no| C
    I -->|yes| J[9. Deploy + monitor]

The loop on step 8→3 is the iteration that takes most of the time.

4. Core concepts¶

Features (X) — input variables, shape (n_samples, n_features).
Target (y) — what you're predicting, shape (n_samples,).
Training set — data used to fit the model.
Validation set — data used to tune hyperparameters.
Test set — data used ONCE at the end to estimate generalization. Never touch during training.
Overfitting — model memorizes training data, fails on new data.
Underfitting — model is too simple; bad on both train and test.
Bias-variance tradeoff — simple model = high bias / low variance. Complex model = low bias / high variance. Sweet spot in between.
Feature engineering — transforming raw data into features the model can use. Usually the highest-leverage step.
Hyperparameters — knobs you set before training (learning rate, tree depth). Tuned on validation set.
Parameters — what the model learns during training.
Data leakage — information from the test set leaks into training. Inflates scores, kills production.

5. Code — minimal working example¶

End-to-end in one snippet — load, split, train, evaluate:

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report

# 1. Load data
X, y = load_iris(return_X_y=True)

# 2. Split — stratified for classification
X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

# 3. Train
model = LogisticRegression(max_iter=200).fit(X_tr, y_tr)

# 4. Evaluate
y_pred = model.predict(X_te)
print("Accuracy:", accuracy_score(y_te, y_pred))
print(classification_report(y_te, y_pred))

That's the skeleton of every supervised ML project.

6. Code — real-world pattern¶

Production-shape: a Pipeline that bundles preprocessing + model into one object — prevents leakage and is trivially deployable:

import pandas as pd
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from sklearn.impute import SimpleImputer
from sklearn.linear_model import LogisticRegression

df = pd.read_csv("customers.csv")
y  = df.pop("churned")
X  = df

num_cols = ["age", "tenure_months", "monthly_charges"]
cat_cols = ["plan_type", "region"]

X_tr, X_te, y_tr, y_te = train_test_split(X, y, test_size=0.2,
                                          stratify=y, random_state=42)

# Per-column preprocessing
preprocess = ColumnTransformer([
    ("num", Pipeline([
        ("impute", SimpleImputer(strategy="median")),
        ("scale",  StandardScaler()),
    ]), num_cols),
    ("cat", Pipeline([
        ("impute", SimpleImputer(strategy="most_frequent")),
        ("ohe",    OneHotEncoder(handle_unknown="ignore")),
    ]), cat_cols),
])

pipe = Pipeline([
    ("prep",  preprocess),
    ("clf",   LogisticRegression(max_iter=500, class_weight="balanced")),
])

# Cross-validated score (no leakage — preprocess fits inside each fold)
print("CV accuracy:", cross_val_score(pipe, X_tr, y_tr, cv=5).mean())

# Final fit + holdout test
pipe.fit(X_tr, y_tr)
print("Test accuracy:", pipe.score(X_te, y_te))

# Save the whole pipe for deployment
import joblib
joblib.dump(pipe, "churn_model.joblib")

Pipeline is the single biggest hygiene tool in sklearn. Use it from day one.

7. Common pitfalls¶

❗ Data leakage — fitting a scaler / imputer on the FULL dataset before splitting. Always split first, fit on train only. Pipeline makes this automatic.
❗ Touching the test set during development. It's a one-shot final exam. Use a validation set or CV for tuning.
❗ Picking the wrong metric. Accuracy on a 95/5 class imbalance is meaningless. See the classification metrics chapter.
❗ Skipping EDA. You'll miss data quality problems, duplicates, label noise, distribution shifts.
❗ Tuning forever to gain 0.2% accuracy. Almost always, better data > better model.
❗ No baseline. Always compare against a stupid baseline (predict the majority class, predict the mean). If a fancy model can't beat it, something is wrong.
❗ Ignoring class imbalance. Stratify the split; consider class_weight="balanced"; pick metrics that handle imbalance.

8. When to use vs not use ML¶

Use ML when	Don't use ML when
Rules are too many or fuzzy	A simple rule works
You have lots of labeled examples	You have < a few hundred examples
Patterns are stable over time	The world changes faster than you can retrain
Mistakes are tolerable (or correctable)	One bad prediction is catastrophic and irreversible
You can measure success objectively	Subjective output (use an LLM with eval)

ML vs deep learning vs LLM	When
Classical ML (this topic)	Tabular data, structured features, < millions of rows
Deep learning	Images, audio, large unstructured text, > 1M examples
LLM + prompting	Text understanding, generation, few-shot tasks, no labeled data
LLM + fine-tune	When prompting isn't enough AND you have specialized data

9. Cheatsheet¶

# The 80% of sklearn imports you'll use
from sklearn.model_selection import (
    train_test_split, cross_val_score, GridSearchCV, KFold, StratifiedKFold,
)
from sklearn.pipeline import Pipeline
from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import (
    StandardScaler, MinMaxScaler, OneHotEncoder, OrdinalEncoder,
    FunctionTransformer, PowerTransformer,
)
from sklearn.impute import SimpleImputer, KNNImputer
from sklearn.decomposition import PCA

# Models
from sklearn.linear_model import (
    LinearRegression, LogisticRegression, Ridge, Lasso, ElasticNet,
)
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
from sklearn.ensemble import (
    RandomForestClassifier, GradientBoostingClassifier,
    AdaBoostClassifier, StackingClassifier,
)
from sklearn.cluster import KMeans

# Metrics
from sklearn.metrics import (
    accuracy_score, precision_score, recall_score, f1_score,
    roc_auc_score, confusion_matrix, classification_report,
    mean_squared_error, mean_absolute_error, r2_score,
)

Quick decision tree:

Task	First-try model
Regression baseline	`Ridge()`
Classification baseline	`LogisticRegression(class_weight="balanced")`
Tabular performance	`RandomForestClassifier()` / `XGBoost` / `LightGBM`
Need interpretability	Linear / Tree (small depth)
Need probability calibration	`LogisticRegression`, or wrap with `CalibratedClassifierCV`
Clustering	`KMeans(n_clusters=...)`
Dimensionality reduction	`PCA(n_components=...)`

10. Q&A — recall test¶

Q: Three types of ML? A: Supervised (labeled), unsupervised (no labels), reinforcement (reward signal). The first two are 99% of practical work.
Q: Why split data into train/val/test? A: Train fits the model. Validation tunes hyperparameters without leaking into final scoring. Test is touched once to get an unbiased generalization estimate.
Q: Most common leakage mistake? A: Fitting a scaler or imputer on the full dataset (including test) before splitting. Always split first; use Pipeline to fit preprocessing only on train.
Q: Bias vs variance? A: Bias = error from wrong assumptions (underfit). Variance = error from sensitivity to training data (overfit). Total error has a sweet spot in between.
Q: First thing to do when starting a new ML problem? A: Build a stupid baseline (majority class, mean prediction). Then EDA. Then a simple model in a Pipeline. Improve from there.

Practice¶

What does this print?

Expected: 1.0

from sklearn.linear_model import LogisticRegression
from sklearn.datasets import load_iris
X, y = load_iris(return_X_y=True)
model = LogisticRegression(max_iter=200).fit(X, y)
print(model.score(X, y))    # ~1.0 — overfits the same data it was trained on

Score the model on UNSEEN test data, not the training data

Expected: True

from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
X, y = load_iris(return_X_y=True)
X_tr, X_te, y_tr, y_te = train_test_split(X, y, random_state=42)
model = LogisticRegression(max_iter=200).fit(X_tr, y_tr)
print(model.score(X_tr, y_tr) >= 0.9)   # bug: should score on X_te, not X_tr

Quiz — Quick check¶

What you remember

Q1. What's data leakage?

Loading too much data into memory
Information from the test set (or future) influencing the trained model — inflates scores and breaks production
Missing values
Outliers

Why: The classic case: fitting a scaler/imputer on the whole dataset before splitting. The scaler "sees" test statistics and bakes them into preprocessing. Pipeline prevents this by fitting only on the training fold.

Q2. When does a model underfit?

When it's too simple to capture the pattern — bad on both training and test data
When it memorizes the training data
When the dataset is too big
When you use scikit-learn

Why: Underfitting = high bias = wrong assumptions. Overfitting = high variance = sensitive to noise. The bias-variance tradeoff means you're tuning for both.

Q3. Why use a Pipeline from day one?

It's faster
It bundles preprocessing with the model, prevents data leakage, and makes the whole thing deployable as a single object
Required by sklearn
Compresses the model

Why: Without a Pipeline, you'd have to manually keep the scaler/encoder in sync between training and inference. Pipeline does it for you and ensures preprocessing is fit only on training folds during cross-validation.

Common doubts¶

When should I use classical ML vs deep learning vs an LLM?

Classical ML for structured tabular data (rows × features, 1M examples. LLM + prompting for natural-language understanding/generation with no labeled data. Start with classical ML — it's faster, cheaper, more interpretable.

Is sklearn still relevant in 2026?

Absolutely. Tabular data still dominates business use cases (churn, fraud, pricing, ranking), and sklearn is the standard. Deep learning is for unstructured data. Even AI startups end up using sklearn for the structured-data parts.

How much data do I need to train a model?

Depends on the problem complexity. Rule of thumb: ~100 samples per feature for linear models, ~10,000+ for deep learning. If you have <500 rows, classical ML or transfer learning. If <50 rows, consider rules or human review instead.