Simple Linear Regression¶

Simple linear regression has one input and one output:

y = β₀ + β₁·x

The algorithm finds the best line through your data points by minimizing the sum of squared distances from each point to the line.

The intuition¶

   y |          •    
     |       •  •  /
     |   •  •  /•           ← fitted line
     |  •  /•          
     | / •
     |/
     +--------------- x

β₁ (slope) tells you "for every 1-unit increase in x, how much does y change?" β₀ (intercept) tells you "what is y when x = 0?"

Example¶

We have 5 (x, y) pairs that lie on y = 2x plus a tiny bit of noise. Let's fit a line.

from sklearn.linear_model import LinearRegression
import numpy as np

X = np.array([[1], [2], [3], [4], [5]])      # shape (5, 1) — sklearn needs 2D input
y = np.array([2.1, 4.0, 6.1, 8.2, 10.0])

model = LinearRegression().fit(X, y)

print("slope    (β₁):", round(model.coef_[0], 3))
print("intercept (β₀):", round(model.intercept_, 3))
print("predict x=6:", model.predict([[6]])[0])

Expected output:

slope    (β₁): 2.005
intercept (β₀): 0.04
predict x=6: 12.07

Edit the data in the editor and press Run again. Try changing it to a clearly non-linear shape (e.g. y = x²) and see how poorly the linear model fits.

How does it find the best line?¶

For one variable, there's a clean closed-form formula:

β₁ = Σ(xᵢ − x̄)(yᵢ − ȳ) / Σ(xᵢ − x̄)²
β₀ = ȳ − β₁ · x̄

scikit-learn computes this for you automatically when you call .fit().

For multiple variables and big datasets, we use gradient descent — covered in two pages.

What you learned¶

Simple linear regression fits a straight line y = β₀ + β₁·x.
LinearRegression() from sklearn does the heavy lifting.
The trained model has .coef_ (slope) and .intercept_ (β₀) attributes.
.predict() works on new inputs.

Practice¶

What does this print?

Expected: [20.]

import numpy as np
from sklearn.linear_model import LinearRegression
X = np.array([[1], [2], [3], [4], [5]])
y = np.array([2, 4, 6, 8, 10])
model = LinearRegression().fit(X, y)
print(model.predict([[10]]).round())   # y = 2x, so x=10 → 20

Reshape the 1D array X into 2D before fitting

Expected: True

import numpy as np
from sklearn.linear_model import LinearRegression
X = np.array([1, 2, 3])              # bug: 1D
y = np.array([2, 4, 6])
try:
    LinearRegression().fit(X, y)
    ok = True
except Exception:
    ok = False
print(not ok)

Quiz — Quick check¶

What you remember

Q1. In y = mx + b, what does m (.coef_) tell you?

How much y changes per unit change in x (the slope)
The starting value of y
The squared error
The number of samples

Why: The coefficient is the slope. The intercept (.intercept_) is the value of y when x = 0.

Q2. When does linear regression fail to fit a relationship well?

When the data is too clean
When the relationship between x and y is non-linear (curved)
When there are negative values
When the dataset is balanced

Why: Linear regression assumes a straight-line relationship. For curves, use polynomial features, transformations (log), or a non-linear model.

Q3. What's the loss function linear regression minimizes?

Absolute error
Mean Squared Error (MSE)
Cross-entropy
Hinge loss

Why: MSE = average of (y_true - y_predicted)². Squared so positive and negative errors don't cancel; differentiable for gradient descent. Cross-entropy is for classification.

Common doubts¶

What if X and y are not perfectly linear?

Linear regression finds the best-fit line anyway — just with a worse R². The residuals (actual − predicted) reveal the misfit. Plot them; if you see a curve, you need polynomial features or a non-linear model.

Why can't I use model.fit(X, y) when X is 1D?

sklearn standardized on (n_samples, n_features) shape so it works uniformly for 1 or 1000 features. Reshape with X.reshape(-1, 1) (the -1 means "figure out").

What does r² mean intuitively?

"Fraction of variance in y explained by the model." R² = 0.85 means the model explains 85% of the variability in y. R² = 0 means it's no better than predicting the mean. R² can be negative on test data when the model is worse than the mean baseline.

→ Next: Multiple Linear Regression