Skip to content

Linear Regression Tutorial

Linear regression is the simplest supervised learning algorithm for predicting a continuous number (price, temperature, score) from one or more input features.

It assumes the relationship between the inputs X and the output y is roughly a straight line (in higher dimensions: a flat plane).

y = β₀ + β₁·x₁ + β₂·x₂ + … + βₚ·xₚ
  • β₀ — the intercept (where the line crosses the y-axis).
  • β₁ … βₚ — the slopes (one per input feature).
  • The algorithm finds the values of β that make the line "best fit" the data.

What you'll learn

# Page What it covers
1 Simple Linear Regression One feature, one target. The 2D line.
2 Multiple Linear Regression Multiple features. The hyperplane.
3 Gradient Descent How the algorithm finds the best β.
4 Types of Gradient Descent Batch vs Stochastic vs Mini-batch.
5 Regression Metrics MAE, MSE, RMSE, R².
6 Worked Example California housing — full pipeline.
7 Try It Yourself Build a model. Run it in the browser.

Prerequisites

How to use this tutorial

Each page is short. Read top to bottom in order. Every code example has a ▶ Run button — you can edit the code and re-run it without leaving your browser.

Note: The first time you press Run, Python loads in your browser (~10 MB download, one-time). After that it's instant.

Ready? → Next: Simple Linear Regression

Practice

What does this print?

Expected: True

from sklearn.linear_model import LinearRegression
print(hasattr(LinearRegression(), "fit"))

Provide features (X) as a 2D array, not 1D

Expected: True

from sklearn.linear_model import LinearRegression
import numpy as np
X = np.array([1, 2, 3, 4])           # bug: 1D — sklearn expects shape (n_samples, n_features)
y = np.array([2, 4, 6, 8])
try:
    LinearRegression().fit(X, y)
    ok = True
except ValueError:
    ok = False
print(not ok)   # we expect the fit to FAIL, hence "not ok" should be True

Quiz — Quick check

What you remember

Q1. Linear regression predicts a…

  • Class label
  • Continuous numeric value
  • Probability between 0 and 1
  • Cluster ID

Why: Regression = predicting a number (price, temperature). For probabilities or classes, use logistic regression.

Q2. What's the shape sklearn expects for the feature matrix X?

  • 2D: (n_samples, n_features) — rows are observations, columns are features
  • 1D: (n_samples,)
  • Always (n, 1)
  • Depends on the model

Why: Every sklearn estimator expects X to be 2D, even for a single feature. Use X.reshape(-1, 1) if you have a 1D array of one feature.

Q3. The output target y should be…

  • 2D
  • 1D: (n_samples,)
  • A DataFrame
  • A scalar

Why: For single-output models, y is 1D. For multi-output regression (predicting multiple targets), y would be 2D — but that's the exception.

Common doubts

Why is linear regression a starting point even for fancy problems?

It's fast, interpretable, and works surprisingly often. Use it as a baseline — if your fancy XGBoost model beats linear regression by only 1%, the extra complexity isn't worth it. If it beats by 20%, you've learned the problem has non-linear structure.

Can linear regression handle categorical features?

Not directly — you must encode them first (one-hot encoding) so they become numeric. See Encoding chapter for how.

How is linear regression related to neural networks?

A neural network with one neuron and no activation function is linear regression. Add more layers and activations to model non-linearities. Linear regression is the simplest possible neural net.