Perceptron — The Single Artificial Neuron

The perceptron (Rosenblatt, 1958) is the simplest neural network: one artificial neuron that learns to classify inputs into two classes by finding a linear decision boundary.

It computes a weighted sum of inputs plus a bias, then applies a step function:

$\hat{y} = \text{sign}(w_1 x_1 + w_2 x_2 + b) \in \{+1, -1\}$

The decision boundary is the line $w_1 x_1 + w_2 x_2 + b = 0$.

Learning rule: on each misclassified point (predicted $\hat{y} \neq$ true $y$), update weights:

$w_i \leftarrow w_i + \eta \cdot y \cdot x_i, \quad b \leftarrow b + \eta \cdot y$

The Perceptron Convergence Theorem proves this terminates in finite steps if — and only if — the data is linearly separable. It fails on XOR because XOR is not linearly separable; that limitation drove the invention of multi-layer networks.

$\hat{y} = \text{sign}\!\left(\mathbf{w} \cdot \mathbf{x} + b\right) = \text{sign}\!\left(\sum_{i=1}^{n} w_i x_i + b\right)$

Weight update rule (on misclassification):

$\mathbf{w} \leftarrow \mathbf{w} + \eta \cdot y \cdot \mathbf{x}$

Weights $w_1 = 1,\ w_2 = 1,\ b = -1.5$ implement a logical AND gate:

For OR, use $b = -0.5$: output $+1$ whenever $z = x_1 + x_2 - 0.5 > 0$.

The perceptron is the direct ancestor of modern deep learning. Stack thousands of perceptrons in layers, swap the step function for smooth activations (ReLU, sigmoid), and train with backpropagation — that is a deep neural network. Today's large language models, image classifiers, and recommendation engines all descend from this 1958 idea.

1. A perceptron has $w_1 = 1,\ w_2 = 1,\ b = -0.5$. What does it output for inputs $(0, 1)$? Is this closer to AND or OR behaviour?

2. Why does the perceptron fail to learn XOR, even with many training iterations and any choice of weights?