0x60Lesson 7 of 12

Neural networks: the basics

Build an artificial neuron from a weighted sum and an activation, then stack neurons into layers.

22 min 6-question quiz 2 code exercises

By the end of this lesson you can

Compute a neuron’s output from its inputs, weights and bias
Explain why activation functions must be nonlinear
Describe how layers turn simple neurons into a deep network

Your brain has about 86 billion neurons. Each one receives signals from others, and if the combined signal is strong enough, it fires. Artificial neural networks borrow that idea in a very simplified, mathematical form.

An artificial neuron does two things:

Takes a weighted sum of its inputs plus a bias: $z = w_1 x_1 + w_2 x_2 + \dots + w_n x_n + b$ - a dot product again.
Passes it through an activation function that decides how strongly it “fires”: $a = f(z)$ .

The weights say how much each input matters (and whether it pushes up or down); the bias shifts how easily the neuron fires.

A single artificial neuron: multiply each input by a weight, add a bias, then apply an activation function.

A popular activation is the sigmoid, which squashes any number into the range 0 to 1, so the output can be read as a probability:

$\sigma(z) = \frac{1}{1 + e^{-z}}$

Large positive $z$ gives nearly 1, large negative gives nearly 0, and $z = 0$ gives exactly 0.5.

Try it

Tune a neuron by hand

This neuron has two inputs and a sigmoid activation. Its output above 0.5 means “blue”, below means “orange”, and the line is where it’s exactly 0.5. Move the weight and bias sliders until every point is classified correctly (no red circles). Then try to break it: what does making w₁ negative do?

w_1

1.0

w_2

1.0bias0.0

z = 1.0x_1 + 1.0x_2 + 0.0

y = \text{sigmoid}(z)

Accuracy: 38% (3/8 correctly classified — points circled in red are wrong)

neuron.py

1import math
2
3def neuron(inputs, weights, bias):
4    z = sum(x * w for x, w in zip(inputs, weights)) + bias
5    return 1 / (1 + math.exp(-z))
6
7# Should I go outside? inputs: sunny (0 or 1), temperature in °C / 10, raining (0 or 1)
8weights = [2.0, 0.5, -4.0]
9bias = -1.5
10print(f"{neuron([1, 2.4, 0], weights, bias):.3f}")   # sunny, 24 °C, dry
11print(f"{neuron([0, 1.0, 1], weights, bias):.3f}")   # cloudy, 10 °C, raining

Output

0.846
0.007

From one neuron to a network

One neuron can only draw one straight boundary. Some problems need more. The famous example is XOR: output 1 when exactly one of two inputs is 1. No single straight line separates those cases.

The fix is to connect neurons in layers: an input layer holds the features, one or more hidden layers compute intermediate features, and an output layer gives the answer. Every neuron in one layer feeds every neuron in the next.

A deep neural network is just neurons like the one above, arranged in connected layers.

xor_network.py

1def step(z):
2    return 1 if z > 0 else 0
3
4def xor_network(x1, x2):
5    either = step(x1 + x2 - 0.5)       # hidden neuron 1: OR
6    both = step(x1 + x2 - 1.5)         # hidden neuron 2: AND
7    return step(either - both - 0.5)   # output: OR but not AND
8
9for x1 in (0, 1):
10    for x2 in (0, 1):
11        print(x1, x2, "->", xor_network(x1, x2))

Output

0 0 -> 0
0 1 -> 1
1 0 -> 1
1 1 -> 0

Deep learning simply means networks with many hidden layers - from a handful to hundreds. Each layer builds on the one before, so early layers learn simple patterns and later layers combine them into complex ones. Here we chose the XOR weights by hand; real networks have millions or billions of weights, far too many to set by hand. The next lesson shows how they’re learned.

Key takeaways

A neuron computes $z = w \cdot x + b$ , then an activation $a = f(z)$ .
The sigmoid $\sigma(z) = 1 / (1 + e^{-z})$ squashes outputs into 0-1; ReLU is $\max(0, z)$ .
Layers of neurons can learn shapes no single neuron can, like XOR - but only with nonlinear activations.
Deep learning = many hidden layers, each building on the last.

Lesson quiz

6 questions · pass with 5 correct · up to 50 XP

Passing this quiz completes the lesson and keeps your streak going. Questions you miss come back in review sessions later.

Practice: write Python

Write Python in the editor and run it against sample inputs. Python runs locally in your browser using a WebAssembly runtime.

Exercise 1

Run a neuron

+25 XP

Line 1: the inputs. Line 2: the weights. Line 3: the bias. Line 4: the activation, sigmoid or relu. Print z = Z and output = A, both with 3 decimal places.

Sigmoid
ReLU, negative
ReLU, positive

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Exercise 2

Run a two-layer network

+25 XP

Line 1 holds two inputs. The network has two hidden ReLU neurons and one output neuron with no activation:

hidden 1: weights (1, 1), bias 0
hidden 2: weights (1, −1), bias −1
output: weights (2, −3) applied to the two hidden values, bias 1

Print hidden: H1 H2 and output: Y, formatting each number with :g.

Small inputs
Second neuron fires
Negative inputs

main.py

Loading editor…

Python runs in a sandboxed browser worker with a 60 second time limit. Its runtime loads from the Pyodide CDN; your code stays in this browser.

Questions about this lesson

Stuck? Ask. Figured something out? Share it. Explaining is one of the best ways to learn.

Loading posts…