Notation

Every symbol the course uses, in the order the lessons introduce them. A lesson never uses a symbol before its row here.

TeX	Say it	Meaning	Worked example	Lesson
`x`	x	an input to the model	a trip of 4 km	01
`y`	y	the target: the answer an example says the model should give	a fare of 13.0	01
`w`	w	the weight knob: the slope of a line, later any weight	2.5 per km	01
`b`	b	the bias knob: where the line crosses x = 0; from lesson 08 also a second input Value, and from lesson 09 also a small expression's output, as in b = a + a	3.0	01
`\cdot`	times	multiplication	2.5 · 4 + 3 = 13	01
`\sum`	the sum of	add up a list: $\sum_{i=1}^{n}$ adds the terms for i = 1 up to n	$\sum_{i=1}^{4} i = 1 + 2 + 3 + 4 = 10$	02
`\frac`	a over b	a fraction	$\frac{1}{4} \cdot 20 = 5$	02
`\lvert`	the size of	$\lvert a \rvert$ : a number without its sign (the left bar)	$\lvert -3 \rvert = 3$	02
`\rvert`	(closes a size)	the right bar of $\lvert a \rvert$	$\lvert -3 \rvert = 3$	02
`n`	n	how many examples	4 points	02
`i`	i	which example (a subscript: $x_i$ is the i-th input)	$x_3 = 3$	02
`L`	L	the loss: one number for how wrong the model is	MSE 5.0 for the flat line	02
`\partial`	partial	$\frac{\partial L}{\partial w}$ : how fast L changes when w alone is nudged, other knobs held still	$\frac{7.35075 - 7.5}{0.01} = -14.925$	03
`\text`	(words)	ordinary words inside a formula, such as $\text{lr}$	lr = 0.01	03
`\approx`	is about	approximately equal	−14.925 ≈ −15	03
`h`	h	the size of a nudge	h = 0.01	03
`e`	e	Euler's number, about 2.718; $e^{x}$ is the function exp	$e^{-1} \approx 0.368$	04
`z`	z	a neuron's weighted sum before its activation	2·1 + (−1)·0 − 0.5 = 1.5	04
`\mathrm{sigmoid}`	sigmoid	$\frac{1}{1 + e^{-z}}$ , a smooth step from 0 to 1	sigmoid(0) = 0.5	04
`\nabla`	grad	$\nabla L$ : the list of every partial slope of L	$\nabla L = [3, -2]$ for $L = 3 w_1 - 2 w_2 + 5$	06
`\ldots`	and so on	an elided run of a list	w₁, w₂, …, w₁₀₀₀₀	06
`a`	a	an input Value of an operation, as in $a \cdot b$	a = 2	08
`c`	c	a third input Value, or from lesson 09 a small expression's output, as in c = a · b + a	c = 4	08
`f`	f	the output of a small expression, or a function, as in f(x)	$f = (a + b) \cdot c = -9$ at a = 1, b = 2, c = −3	08
`g`	g	a second expression's output	$g = (a \cdot b) \cdot c = 24$ at a = 2, b = 3, c = 4	08
`d`	d	a fourth input Value, or a small expression's output, as in d = a · b + b	d = −6	09
`\ln`	natural log	the inverse of exp: $\ln e^{x} = x$ ; ember's method is `log`	slope of ln at 2 is 1/2	11
`\tanh`	tanh	the hyperbolic tangent, a smooth step from −1 to 1 (a row of lesson 11's derivative table)	tanh(1) ≈ 0.7616	11
`\mathrm{relu}`	relu	$\max(0, x)$ : negative in, 0 out (a row of lesson 11's derivative table)	relu(−1.5) = 0	11
`\max`	max	the larger of the values	max(0.001, 0.002) = 0.002	11
`(64, 10)`	shape 64 by 10	a shape tuple: the size of each axis	64 rows of 10 features	13
`\mathsf{T}`	transpose	$A^{\mathsf{T}}$ : A with its last two axes swapped	(3, 5)ᵀ has shape (5, 3)	17
`\mathrm{Var}`	variance	the mean squared distance from the mean	Var(1, 3, 5, 7) = 5	22
`\sigma`	sigma	the standard deviation, $\sqrt{\mathrm{Var}}$	σ(1, 3, 5, 7) ≈ 2.236	22
`\sqrt`	square root	square root	√100 = 10	22
`p`	p	a probability (a list of them sums to 1)	softmax([2, 1, 0]) ≈ [0.665, 0.245, 0.090]	24
`\infty`	infinity	an overflowed or masked value	float32 exp(89) = inf	24
`\mathbf{y}`	one-hot y	the target as a vector with a 1 at the true class	class 2 of 3: [0, 0, 1]	25
`\beta`	beta	an EMA's keep factor	β = 0.9 keeps about the last 10 steps	26
`\mathbb{E}`	expected value	the average over the randomness	E[kept·2] = 0.5·2 = 1 for p = 0.5	29
`\varepsilon`	epsilon	a small constant that keeps a division safe	1e−5	33
`\lVert`	norm of	$\lVert x \rVert$ : the length of a vector	‖[3, 4]‖ = 5	37
`\rVert`	(closes a norm)	the right bar of $\lVert x \rVert$	‖[3, 4]‖ = 5	37