The principle of linear regression

A scatter plot, a line that fits it

Machine learning often starts with a very simple problem: you have a table of data and you want to predict one value from another.

Take a concrete example. Here is the weight (in thousands of pounds) and fuel efficiency (in miles per gallon, MPG) of seven cars:

There is a clear trend: the heavier the car, the fewer miles per gallon. A linear regression model looks for a line that summarizes this trend, of the form:

y' = b + w1 * x1

where y' is the predicted value, x1 is the input (here, weight), w1 is the weight learned by the model (the slope), and b is the bias (the intercept).

A model trained on this data

Fitting a line to this scatter plot can give, for example:

y' = 34 + (-4.6) * x1

Here b = 34 and w1 = -4.6. The negative weight matches exactly what we observe: every extra thousand pounds lowers the predicted MPG by 4.6.

Take a 4,000-pound car, so x1 = 4:

y' = 34 + (-4.6 * 4)
y' = 34 - 18.4
y' = 15.6

The model predicts about 15.6 MPG for this car. You can redo this calculation with a plain pencil: that is all a linear regression model does, at every prediction.

Why a line, not a single number?

A line captures a relationship, not one isolated case. Instead of memorizing seven pairs of values, the model learns a general rule (a slope and an intercept) that can then predict the fuel efficiency of a car that was not in the original table. That is the core promise of supervised machine learning: generalizing from examples.

The open question, covered in the next lesson, is: how do we know b = 34 and w1 = -4.6 are good values, rather than b = 20 and w1 = -1?

Try it yourself

With the model y' = 34 + (-4.6) * x1, what fuel efficiency does it predict for a 3,000-pound car (x1 = 3)?

Answer: y' = 34 + (-4.6 * 3) = 34 - 13.8 = 20.2 MPG.

Keep in mind: a simple linear regression model comes down to two numbers, a weight and a bias, applied through a formula you can compute by hand.