From Neural Networks to LLMs: The Mental Model I Was Missing

Neural networks are machine learning models that learn patterns from data rather than relying on manually programmed rules. A concrete starting point is handwritten digit recognition. A network is shown thousands of images of handwritten digits; each image flows through the network as numerical pixel values, is transformed layer by layer, and produces a probability distribution over digit classes as the prediction. No developer writes rules such as “if the image has this curve and this line, it must be a 3.” Instead, the network discovers the relevant patterns from labeled examples during training. The distinction between a 4 and a 9 is captured implicitly through adjusted weights, not through an explicit rule.

Everything the network learns is stored in its parameters, primarily weights and biases. These numerical values determine how strongly one unit influences another and how much a unit’s threshold is offset. Training adjusts weights and biases to reduce the difference between predicted and actual outputs. When the number of layers increases, the model enters deep learning. The relationship can be summarized as: machine learning → neural networks → deep learning.

Different neural network architectures became useful for different problem types:

• CNNs (convolutional neural networks) became common for image-related tasks.
• RNNs and LSTMs were commonly used for sequential data, including language.
• Transformers later became central for language and many other sequence-based tasks.

Language poses a special challenge: meaning depends on context. In the sentence “The animal didn’t cross the road because it was tired,” resolving what “it” refers to requires understanding the grammatical and semantic relationship between words across the sentence. This contextual ambiguity is a key reason attention mechanisms became important, and it eventually led to Transformer architectures.

Recurrent neural networks (RNNs) and long short-term memory networks (LSTMs) were the standard architecture for language tasks before the Transformer. These models process a sequence one token at a time, maintaining a hidden state updated at each step. For a sentence like "I love machine learning," the network reads "I," then "love," then "machine," then "learning," and each step depends on all previous steps. This sequential dependency creates two practical problems for enterprise-scale training:

• Limited parallelization: each step depends on the previous hidden state, so training examples cannot be fully processed in parallel. Training becomes slower and more expensive as sequence lengths grow.
• Weak long-range dependencies: information from early tokens must survive many recurrent steps to influence later tokens. Even with LSTM gating, relationships between distant tokens are difficult to learn reliably in long documents, source code, or conversation histories.

The Transformer architecture, introduced in the 2017 paper "Attention Is All You Need," was designed to address both limitations. The original model was built for machine translation, mapping English input to French output. The flow is:

1. Input sequence (English) passes through an encoder.
2. The encoder produces a contextual representation of the entire sequence.
3. The decoder consumes that representation.
4. The decoder generates the output sequence (French) token by token.

For example, the English sentence "I love cats" enters the encoder, and the decoder produces "J'aime les chats."

The key mechanism is attention. Attention computes a weighted relevance between every pair of tokens, allowing the model to determine which tokens should influence the representation of any given token. In "The animal didn't cross the road because it was tired," the token "it" is directly related to "animal" even though several tokens separate them. Because attention scores are computed for all token pairs simultaneously, the architecture removes the sequential bottleneck: training is parallelized across the full sequence length while still modeling long-range relationships explicitly.

When a Transformer processes a token, it does not treat that token in isolation. The model decides which other tokens in the sequence are relevant and assigns each one a different level of importance for the current step. This allows the representation of the current token to be influenced by the context around it.

Consider the sentence: “The animal didn’t cross the road because it was tired.” To understand what “it” refers to, the model must determine which other tokens matter. Conceptually, when processing “it,” the model can look back at “animal,” “road,” “cross,” and the surrounding words, then assign different weight to each. The result is a contextual representation of “it” that carries information from the tokens that matter most.

This is called self-attention because tokens attend to other tokens in the same sequence. The sequence itself provides the context; there is no external database or separate retrieval step.

• Attention is not a separate add-on to Transformers.
• Attention is one of the core mechanisms inside a Transformer.
• Transformers use attention to capture relationships between tokens, including tokens that are far apart.
• Unlike step-by-step recurrent models, attention allows relationships to be considered directly across the whole sequence, which also enables more parallel processing during training.

A practical mental model for a single token looks like this:

1. Start with the current token.
2. Look at other relevant tokens in the same sequence.
3. Combine useful information from those tokens according to their assigned importance.
4. Create a contextual representation of the current token that reflects that combined information.

This simplified flow is enough to reason about how Transformers process language. The mathematical details — Query, Key, Value vectors, attention scores, and matrix multiplication — are a separate deep dive. What matters here is the conceptual mechanism: every token’s meaning is built from the tokens around it, with different levels of attention paid to each.

The original Transformer, introduced in the paper "Attention Is All You Need," is an encoder-decoder architecture. Its defining mechanism is self-attention: when processing a token, the model weighs which other tokens in the sequence are relevant and combines that information into a contextual representation. The mental model is straightforward: the encoder processes and represents the input, and the decoder generates the output. This design targets a natural problem — transforming one sequence into another. Practical sequence-to-sequence examples include English→French translation, article→summary summarization, and question→answer generation.

Researchers later realized that both components are not always needed. Transformer is an architecture, not a single model, so different models can use different parts. This yields three families.

• Encoder-only models (e.g., BERT) use the encoder to build bidirectional contextual representations. BERT is trained with masked language modeling: it predicts a [MASK]ed token using context from both directions. These models are suited to understanding and representing text for classification, sentiment analysis, named entity recognition, search understanding, and information extraction.
• Decoder-only models (e.g., GPT) use the decoder with causal language modeling: predict the next token from previous tokens and generate one token at a time. This makes them naturally suited to text generation.
• Encoder-decoder models (e.g., T5, BART) retain both parts. The encoder represents the input; the decoder generates the output. They are suited to transforming one sequence into another, such as summarization and translation.

For engineering decisions: choose encoder-only when the goal is to understand or represent text, decoder-only when generating text, and encoder-decoder when the output is a transformed version of the input.

A Large Language Model (LLM) is a neural network trained to model language. The name is a direct description of its structure:

• Large — Refers to the enormous number of learned parameters, primarily weights and biases. Modern models can contain billions of these parameters, and those values encode the patterns the network has learned.
• Language — The model is trained on large amounts of text data and learns statistical patterns, relationships, and structures within that language.
• Model — It is fundamentally a neural network whose behavior is determined by its learned parameters, not by hand-coded linguistic rules.

GPT (Generative Pre-trained Transformer) is a specific class of LLM built as a decoder-only Transformer. Unlike encoder-only models such as BERT, which build bidirectional representations, GPT is trained primarily with causal language modeling: given a sequence of previous tokens, predict the next token. Because it can only attend to prior context, it generates text one token at a time.

The training objective is deceptively simple. Using the training text "The sky is blue", the process works as follows:

• Input context: Feed the model the tokens "The sky is".
• Predict next token: The model produces a probability distribution over its vocabulary for the next token.
• Compare with actual token: The target is the known next token, "blue".
• Calculate error: A loss function measures the difference between the predicted distribution and the actual token.
• Update parameters: Backpropagation adjusts the weights and biases to reduce the error.
• Repeat: Slide the context forward — "The sky is blue" becomes input for predicting the following token — and repeat across the entire corpus.

This single objective, applied across massive datasets containing billions of tokens, is what enables the model to learn grammar, reasoning patterns, and factual associations. The Transformer's self-attention mechanism is what allows the model to relate each token to relevant earlier tokens in the context, making next-token prediction effective over long sequences.

The central puzzle is that GPT is trained only to predict the next token, yet it answers questions, summarizes, and generates code. The resolution lies in the chain from neural network parameters to the Transformer architecture.

A neural network learns patterns from data and stores them in parameters—primarily weights and biases. During training, the model takes input context, predicts the next token, compares it to the actual token, calculates error, and updates parameters. A decoder-only Transformer trained this way, with causal language modeling, accumulates statistical knowledge about language across billions of parameters and vast corpora. There is no separate mechanism for question answering or summarization; those capabilities emerge from the richness of patterns encoded in the parameters.

Attention makes this accumulation possible. Instead of processing sequences strictly step by step, self-attention lets each token attend to other relevant tokens in the same sequence and combine their information into a contextual representation. In "The animal didn't cross the road because it was tired," processing "it" assigns relevance to "animal" and "road," resolving reference. The contextual representation is what the model conditions on for the next token.

Scale converts next-token prediction into broad capability. Parameters capture grammar, factual associations, instruction patterns, and code idioms. Answering a question is next-token prediction conditioned on a prompt statistically resembling question-answer pairs in training data. Summarization is next-token prediction conditioned on an article, generating tokens that follow the distribution of summaries.

• Encoder-only models (e.g., BERT) build bidirectional contextual representations; they understand and represent text for classification, sentiment analysis, and named entity recognition.
• Decoder-only models (e.g., GPT) generate text one token at a time; they are naturally suited to generation.
• Encoder-decoder models (e.g., T5, BART) transform sequences, processing input into a representation and generating an output.

These are different uses of the same attention-based Transformer foundation, not fundamentally different mechanisms.