Neural Network
A computing architecture loosely inspired by the brain - layers of interconnected nodes that transform inputs into outputs through learned mathematical operations.
Added May 21, 2026 · 2 min read
Neural networks are the foundation of deep learning, and deep learning is the foundation of modern AI. Understanding what a neural network is - and what it is not - provides the conceptual base for understanding transformers, large language models, image generators, and nearly every other AI system in the news today.
The name "neural network" comes from a loose analogy with biological neurons - the cells in the brain that fire electrical signals to each other. The analogy is rough. Artificial neural networks share some mathematical structure with neurons, but they are not models of how brains actually work. They are mathematical functions that turned out to be extremely useful for machine learning.
The basic unit is a node (sometimes called a neuron or perceptron). Each node receives one or more numerical inputs, multiplies each by a learned weight, sums the results, and passes the sum through an activation function - a mathematical operation that introduces non-linearity. Non-linearity is what allows neural networks to represent complex, curved relationships in data. Without it, a network of any depth would behave identically to a single linear operation.
Nodes are organised into layers. The first layer takes the raw input - pixel values in an image, numerical features in a dataset, or token identifiers in text. Each subsequent layer transforms its inputs into a new representation, learning increasingly abstract features. An image classifier's early layers might detect edges and textures; deeper layers detect shapes and object parts; the final layers assemble these into object categories.
The process of training a neural network is gradient descent: compute how wrong the network's output is, calculate how much each weight contributed to that error (using backpropagation), and nudge every weight slightly in the direction that reduces the error. Repeat this for millions of examples until the network's predictions are accurate.
The power of neural networks is their generality. The same architecture - layers of nodes with learned weights - can be applied to images, text, audio, tabular data, and more. The specific patterns the network learns depend entirely on the training data, not on any hard-coded assumptions about the problem domain. This flexibility is why neural networks became the dominant paradigm in machine learning.
Analogy
An assembly line with multiple inspection stations. Each station looks for specific things, passes what it has found to the next station, and the final station delivers a verdict. Raw material enters and a finished product emerges. Similarly, raw data enters a neural network and passes through layers that progressively extract and combine features until the final layer produces a prediction.
Real-world example
A neural network for image recognition processes a photo by first detecting edges, then shapes from those edges, then objects from those shapes, and finally identifying what is in the photo. The network did not have these stages described to it - it learned this hierarchical decomposition by training on millions of labelled images.
Why it matters
Neural networks are the foundation of deep learning, and deep learning is the foundation of modern AI. Understanding what a neural network is - and what it is not - provides the conceptual base for understanding transformers, large language models, image generators, and nearly every other AI system in the news today.
In the news
AI Breakthrough: New Method Estimates Neural Network Outputs Without Sampling
AI Alignment Forum · 5d ago
AI Breakthrough in Decoding EEG Signals for Better Clinical Trust
arXiv CS.LG · 6d ago
AI Training Breakthrough: Correlated Noise Mechanisms Improve Privacy and Utility
arXiv CS.LG · 1w ago
New AI Framework Boosts Accuracy in Complex Networks
arXiv CS.LG · 1w ago
AI Boosts Graph Analysis with New Path-Based Method
arXiv CS.LG · 1w ago
Related concepts
Artificial Intelligence
The field of building computer systems that can perform tasks normally requiring human intelligence - from recognising speech to writing code.
Deep Learning
Machine learning using neural networks with many layers - the approach behind almost every significant AI breakthrough of the past decade.
Gradient Descent
The algorithm that trains neural networks - iteratively adjusting parameters in the direction that reduces the model's error.
Machine Learning
A way of teaching computers by showing them examples, rather than writing explicit rules - the engine behind almost everything labelled AI today.
Parameters
The numbers inside a neural network that get adjusted during training and define everything the model knows and can do.