What is Epoch?

Definition

An epoch represents one full cycle through every example in the training dataset. If a dataset contains 5,000 examples and the model processes all 5,000 during training, that constitutes one epoch. Fine-tuning typically runs for multiple epochs — commonly between 1 and 5 — so the model sees each example several times, progressively refining its weights to better fit the training distribution.

The number of epochs is a critical hyperparameter that directly affects model quality. Too few epochs and the model may not fully absorb the patterns in the training data (underfitting). Too many epochs and the model begins to memorize specific examples rather than learning generalizable patterns (overfitting). The sweet spot depends on dataset size, model size, learning rate, and the complexity of the task. For most LLM fine-tuning tasks with a few thousand examples, 2–4 epochs is a common starting point.

Within each epoch, the dataset is typically shuffled and divided into batches (determined by the batch size hyperparameter). The model processes one batch at a time, computing the loss and updating weights after each batch via backpropagation. Monitoring the training loss and validation loss across epochs provides the primary signal for deciding when to stop training — ideally when validation loss plateaus or begins to increase.

Why It Matters

Getting the epoch count right is essential for producing a useful fine-tuned model. In practice, most fine-tuning failures can be traced to either too few epochs (the model hasn't learned the task) or too many (the model has overfit to the training data). Understanding epochs also helps practitioners estimate training time and cost: doubling the number of epochs roughly doubles the GPU hours required. For teams operating on limited budgets, this makes epoch selection a key lever for balancing quality against compute costs.

How It Works

At the start of each epoch, the training examples are shuffled to prevent the model from learning spurious patterns based on data order. The shuffled dataset is then split into mini-batches of size determined by the batch-size hyperparameter. For each mini-batch, the model performs a forward pass (generating predictions), computes the loss (measuring how far predictions are from targets), performs a backward pass (computing gradients), and updates the adapter or model weights. After all mini-batches have been processed, the epoch is complete. The training loop then evaluates the model on the validation set to track generalization performance before starting the next epoch.

Example Use Case

A team fine-tunes a 7B model on 3,000 customer support examples. After 1 epoch, the model shows improvement but still misses nuanced responses. After 3 epochs, validation accuracy peaks at 87%. At 5 epochs, validation loss starts climbing — a clear sign of overfitting. They select the 3-epoch checkpoint as their production model, balancing learning completeness against generalization.

Key Takeaways

• One epoch equals one complete pass through all training examples.
• Most LLM fine-tuning tasks use 1–5 epochs, with 2–4 being the common range.
• Too few epochs leads to underfitting; too many leads to overfitting.
• Monitoring validation loss across epochs is the primary signal for when to stop training.
• Epoch count directly scales training time and compute cost.