Lesson 01 · Part I · Why learning works at all
A function with knobs
Can a program find its own numbers?
Five taxi rides
Here are five taxi receipts from one city: how far each ride went, and what it cost.
| distance (km) | fare |
|---|---|
| 1 | 5.60 |
| 2 | 7.90 |
| 3 | 10.40 |
| 5 | 15.60 |
| 8 | 22.90 |
The fares grow with distance, but not by a round amount per kilometre. The second kilometre adds 2.30 to the first ride's 5.60, and the third adds 2.50.
Nobody gave you the city's pricing rule. Can a program work out what a 10 km ride will cost from these five alone?
A program reads these five receipts and afterwards quotes a fare for a 10 km ride. Nobody typed in the pricing rule. What does the program need?
The program brings a shape, a straight line, with two adjustable numbers, and lets the receipts set them. Setting a function's numbers from examples is what this course means by learning.
Before any program tries, try it yourself. You have two sliders and your eye, and nothing else.
Drag w and b until the line runs through the receipts. There is no score: judge by eye.
| distance (km) | fare |
|---|---|
| 1 | 5.6 |
| 2 | 7.9 |
| 3 | 10.4 |
| 5 | 15.6 |
| 8 | 22.9 |
Two people fit the line by eye. Ana settles on w = 2.4, b = 3.4. Ben settles on w = 2.6, b = 2.6. Both lines pass close to every receipt. Whose line is better?
By eye both lines are good and neither is better. Ben's misses are smaller on average, and Ana's biggest miss is smaller. To choose, a program needs a rule that turns a line's misses into one number.
Two knobs
Each slider turned a knob. w is how much the fare grows for each extra kilometre: the line's slope. b is the fare before the first kilometre: where the line crosses the fare axis, at a distance of 0. The course's word for a knob is a parameter: w is also called a weight, and b a bias. The fares on the receipts are the targets the line aims at.
A straight line has two knobs: With w = 2.5 and b = 3, a 4 km ride costs 2.5 · 4 + 3 = 13.
A model is a function with knobs, and learning means setting the knobs from examples.
With w = 2.5 and b = 3, what does the line quote for a 10 km ride?
2.5 · 10 + 3 = 28. Each kilometre adds w = 2.5, and b = 3 is there before the first one.
What: write predict_line(x, w, b), which returns the line's prediction for one input.
Why: without it there is no prediction to score. Lessons 02 and 03 call it: lesson 02 scores its predictions, and lesson 03 turns its knobs by walking downhill. The Linear layer of lesson 22 is the same idea for many inputs at once.
Your first piece of ember: a function with two knobs.
- The line's height at x is the slope times x, plus its height at x = 0. Which argument is the slope, and which is the height at x = 0?
- Multiply the input by w, then add b. It is one return statement.
- Nearly the code: return w * x + ...
- predict_line(x, w, b) is w times x, plus b.
- At x = 0 the prediction is the intercept b, whatever the slope.
- Moving x by 1 moves the prediction by exactly w.
Put the knobs you chose with the sliders into the w, b line and run it.
from ember.basics import predict_line
km = [1, 2, 3, 5, 8]
fare = [5.6, 7.9, 10.4, 15.6, 22.9]
w, b = 2.5, 3.0 # put the knobs you chose with the sliders here
for x, y in zip(km, fare):
guess = predict_line(x, w, b)
print(f"{x} km: receipt {y:5.2f}, line {guess:5.2f}, off by {y - guess:+.2f}")
print(f"10 km: the line predicts {predict_line(10, w, b):.2f}")1 km: receipt 5.60, line 5.50, off by +0.10 2 km: receipt 7.90, line 8.00, off by -0.10 3 km: receipt 10.40, line 10.50, off by -0.10 5 km: receipt 15.60, line 15.50, off by +0.10 8 km: receipt 22.90, line 23.00, off by -0.10 10 km: the line predicts 28.00
If you kept the cell's knobs, w = 2.5 and b = 3, every receipt is off by 0.10. That beats Ana's line and Ben's, on the average miss and on the biggest miss. The printed misses showed it; the sliders alone could not. Ana's and Ben's lines are the usual case: one wins on average, the other at its worst, and only a rule can choose between them.
A thermometer reads 0 °C when water freezes and 100 °C when it boils. In °F those are 32 and 212. A line turns °C (x) into °F (y). What is w?
Between the two readings x grows by 100 and y grows by 180, so each °C adds 1.8 °F: w = 1.8, and b = 32, the reading at 0. Two exact examples fix both knobs, and nobody told the line "9/5".
In one or two sentences: how could a program decide which slider position is best?
What would the program need to compute for each position of the sliders, so it could compare them?
It needs one number per position that says how far off the line is on the receipts. Then "best" means "smallest number", and a program can compare positions it has never seen.
Common answers, and what each one misses
- "The line through the most receipts." On real receipts no line goes through them all, and hitting two exactly can cost a big miss elsewhere, as Ben's did.
- "The average miss." A good start. Lesson 02 shows which misses to average, and why their signs matter.
Where this goes
A program can only choose between lines if it can score them. Lesson 02 turns a line's misses into one number, then finds the best line by trying hundreds.
When you finish this lesson
Your ember can now: turn two knobs, w and b, into a prediction
Next How wrong is wrong? · Why square the error instead of adding it up? · ~35 min