skip to content

Fine-tuning and Trainer

Trainer runs the loop from TrainingArguments, and TRL's SFTTrainer adds chat-data handling, packing and a LoRA adapter through peft_config. Interviewers want a real fine-tuning run wired end to end.

on this pageshow

questions

5

In Hugging Face Transformers v5, what does a minimal Trainer fine-tuning run need, and which TrainingArguments do you actually set?

level: juniorimportance: must knowfreq 38%

answer

  1. five objects and one call
  2. output_dir and the two batch arguments
  3. an eval set alone never evaluates
  4. trackers are off unless named
  5. v5 renamed the tokenizer argument

basics

~10 s

Trainer needs a model, a TrainingArguments object, a tokenized train_dataset and, in Transformers v5, the tokenizer passed as processing_class. First set output_dir, per_device_train_batch_size, gradient_accumulation_steps, eval_strategy, save_strategy, bf16 and report_to.

solid answer

~30 s

`Trainer(model=..., args=TrainingArguments(...), train_dataset=..., eval_dataset=..., processing_class=tokenizer)` and then `trainer.train()`. In Transformers v5 the `tokenizer=` argument is gone; `processing_class` is how the tokenizer reaches the default collator and gets saved with each checkpoint. For a causal LM I also pass `DataCollatorForLanguageModeling(tokenizer, mlm=False)` so labels are built from the inputs. In `TrainingArguments` I set `output_dir`, `per_device_train_batch_size` and `gradient_accumulation_steps` (together with the device count they give the effective batch), `eval_strategy` (default `"no"`, so an eval set alone never evaluates), `save_strategy` / `save_steps` / `save_total_limit`, `bf16=True` on hardware that supports it, and `report_to`, which defaults to `"none"` in v5.18.

code

python · 26 lines
python
from datasets import load_dataset
from transformers import (AutoModelForCausalLM, AutoTokenizer,
                          DataCollatorForLanguageModeling, Trainer, TrainingArguments)

model_id = "your-org/your-base-model"
tokenizer = AutoTokenizer.from_pretrained(model_id)
tokenizer.pad_token = tokenizer.pad_token or tokenizer.eos_token
model = AutoModelForCausalLM.from_pretrained(model_id)

ds = load_dataset("your-org/your-corpus", split="train")
ds = ds.map(lambda b: tokenizer(b["text"], truncation=True, max_length=1024),
            batched=True, remove_columns=ds.column_names)
splits = ds.train_test_split(test_size=0.02, seed=42)

args = TrainingArguments(
    output_dir="runs/base-ft",
    per_device_train_batch_size=4,
    gradient_accumulation_steps=8,
    eval_strategy="steps", eval_steps=200,
    save_strategy="steps", save_steps=200, save_total_limit=3,
    bf16=True,
    report_to="tensorboard",
)
trainer = Trainer(model=model, args=args, train_dataset=splits["train"], eval_dataset=splits["test"],
                  processing_class=tokenizer, data_collator=DataCollatorForLanguageModeling(tokenizer, mlm=False))
trainer.train()

go deeper

for a junior

Recall the five pieces: model, tokenized dataset, TrainingArguments, processing_class for the tokenizer, and a collator that builds labels. Name output_dir and the batch-size argument without hesitation.

for a middle

Explain the defaults that bite: eval_strategy is no, report_to is none, save_total_limit keeps everything, and steps count optimizer updates rather than micro-batches.

for a senior

Show you can port a v4 script: processing_class instead of tokenizer, warmup_steps as a float instead of warmup_ratio, no overwrite_output_dir, and per_device rather than per_gpu names.

for a principal

Argue which arguments a team should pin in a shared training template - output layout, checkpoint retention, tracker, precision - so runs stay comparable and disks do not fill.

## What Trainer is for `Trainer` is the ready-made training loop in Hugging Face Transformers. You hand it a model, data and a configuration object; it builds the data loader, runs the forward and backward passes, steps the optimizer and the learning-rate scheduler, evaluates, logs, writes checkpoints and can upload the result to the Hugging Face Hub. Writing that loop by hand is a separate skill. The point of `Trainer` is that a fine-tune becomes mostly configuration, which is why interviewers ask which arguments a candidate actually touches and what happens to the ones left alone. ## The pieces a run needs 1. **A model** - usually `AutoModelForCausalLM.from_pretrained(...)` or another `AutoModelFor*` head. In Transformers v5, `from_pretrained` loads with `dtype="auto"` unless told otherwise. 2. **A tokenized dataset** - typically a Datasets `Dataset` whose rows hold `input_ids` and `attention_mask`. Columns that the model's `forward()` does not accept are dropped for you, because `remove_unused_columns` defaults to `True`. 3. **A `TrainingArguments` object** - every knob of the run lives here. 4. **`processing_class`** - the tokenizer (or processor). Trainer uses it to choose a default collator and saves it next to the weights, so each checkpoint can be loaded on its own. 5. **A data collator, when the default is wrong.** With a tokenizer as `processing_class`, the default is `DataCollatorWithPadding`, which pads but builds no labels. A causal language model needs `DataCollatorForLanguageModeling(tokenizer, mlm=False)`, which copies the inputs into `labels` and sets padded positions to -100. Without labels the model returns no loss, and Trainer stops with "The model did not return a loss from the inputs". ## TrainingArguments that matter first | Argument | Default at v5.18 | What you set and why | |---|---|---| | `output_dir` | `"trainer_output"` when omitted | One path per run; checkpoints and the final model land here | | `per_device_train_batch_size` | `8` | What fits in memory on one device | | `gradient_accumulation_steps` | `1` | Micro-batches per optimizer update; the effective batch is this times the per-device batch times the device count | | `eval_strategy` | `"no"` | `"steps"` or `"epoch"`; otherwise an `eval_dataset` is never used during training | | `save_strategy` / `save_steps` / `save_total_limit` | `"steps"` / `500` / unset | How often to checkpoint, and how many checkpoints to keep (unset keeps all of them) | | `bf16` | `False` | `True` on GPUs that support bfloat16 mixed precision | | `report_to` | `"none"` | Name a tracker such as `"tensorboard"`, `"trackio"` or `"wandb"` to get curves | | `push_to_hub` / `hub_model_id` | `False` / derived from `output_dir` | Upload saves to a Hub repository | Learning rate, warmup and epoch count are carried by the same object, but choosing their values is a hyperparameter question rather than a wiring one. ## What changed in Transformers v5 - **`tokenizer=` is gone from `Trainer`.** The argument is `processing_class`; passing `tokenizer=` now fails with a `TypeError`, because the constructor has no such parameter. - **`warmup_ratio` was removed.** `warmup_steps` accepts a float below 1, read as a fraction of total steps. - **`overwrite_output_dir` was removed.** Resuming is controlled by `resume_from_checkpoint` on `train()`. - **Old spellings were deleted**: `per_gpu_train_batch_size` became `per_device_train_batch_size`, `no_cuda` became `use_cpu`, and `push_to_hub_model_id` / `push_to_hub_token` became `hub_model_id` / `hub_token`. - **`use_cache` is switched off for training** in the model config; `TrainingArguments` has a `use_cache` argument if you need it back. ## Mistakes that cost a run - **An eval set that never runs** - `eval_dataset` passed, `eval_strategy` left at `"no"`. - **No curves anywhere** - `report_to` left at `"none"` in v5.18, so nothing reaches a tracker even when one is installed. - **A full disk** - an unset `save_total_limit` keeps every checkpoint, and each holds the weights plus optimizer state. - **A missing pad token** - many causal-LM tokenizers define none, and the collator cannot pad without one; set `tokenizer.pad_token` (commonly to the EOS token) first. - **Steps that are not batches** - `save_steps`, `eval_steps` and `logging_steps` count optimizer updates, so with `gradient_accumulation_steps=8` each step consumed eight micro-batches. A minimal run is therefore five objects and one call: model, tokenizer, tokenized data, `TrainingArguments`, `Trainer`, then `trainer.train()`. Everything an interviewer probes after that - resuming, adapters, chat data - is a change to one of those five.

  • You passed an eval_dataset to Trainer, but no eval loss ever appears in the logs. Why?
    `eval_strategy` defaults to `"no"`, so Trainer never evaluates during training no matter what `eval_dataset` holds. Set `eval_strategy="steps"` with `eval_steps`, or `"epoch"`. If `eval_steps` is left unset it falls back to `logging_steps`. You can still call `trainer.evaluate()` by hand after training.
  • With gradient_accumulation_steps=8 and save_steps=200, after how many micro-batches does the first checkpoint appear?
    After 1,600 per device. Trainer's step counter counts optimizer updates, and each update consumes `gradient_accumulation_steps` micro-batches of `per_device_train_batch_size` rows. `save_steps`, `eval_steps` and `logging_steps` all count those updates, so raising accumulation silently stretches every interval measured in data.
  • Why does a causal-LM run with only processing_class=tokenizer fail on its first step?
    With a tokenizer as `processing_class`, the default collator is `DataCollatorWithPadding`, which pads but adds no `labels`. With no labels the model returns no loss and Trainer raises "The model did not return a loss from the inputs". `DataCollatorForLanguageModeling(tokenizer, mlm=False)` copies the inputs into labels and masks padding with -100.

  • Passing tokenizer=tokenizer to Trainer still works in Transformers v5.
  • Supplying eval_dataset is enough to make Trainer evaluate during training.
  • per_device_train_batch_size alone is the batch each optimizer update sees.
  • Trainer reports to every installed tracker unless told otherwise.
  • Trainer keeps only the latest checkpoint unless you configure it.

Assumes Transformers v5.18: v5 removed Trainer's tokenizer argument in favour of processing_class and dropped warmup_ratio and overwrite_output_dir from TrainingArguments.

open as a page

How do you attach a LoRA adapter to a TRL SFTTrainer run through peft_config, and what do its checkpoints and Hub push contain?

level: middleimportance: should knowfreq 22%

basics

~10 s

Pass peft_config=LoraConfig(...) to SFTTrainer; TRL wraps the model with get_peft_model so only adapter weights train. Checkpoints, save_model and push_to_hub then hold just adapter_model.safetensors and adapter_config.json, which records the base model.

open as a page

A Hugging Face Trainer fine-tune on a preemptible GPU is killed at step 3,200 - how do you make it resume where it stopped instead of restarting?

level: seniorimportance: should knowfreq 15%

basics

~10 s

Save full checkpoints to durable storage with save_strategy, save_steps and save_total_limit, then call trainer.train(resume_from_checkpoint=True) to reload the newest checkpoint's weights, optimizer, scheduler, RNG and step. enable_jit_checkpoint=True adds a save on SIGTERM.

open as a page

In TRL's SFTTrainer, how do conversational and prompt-completion datasets differ, and which SFTConfig flags decide which tokens the loss trains on?

level: seniorimportance: should knowfreq 14%

basics

~10 s

In TRL 1.14 a conversational row holds a messages list; a prompt-completion row holds separate prompt and completion. completion_only_loss=None trains only completions on prompt-completion data, while messages data trains the whole conversation unless assistant_only_loss=True.

open as a page

In TRL's SFTConfig, how do you switch on packing, which strategy does it use by default, and which attention implementation does it expect?

level: middleimportance: nice to knowfreq 9%

basics

~20 s

Set packing=True in SFTConfig; in TRL 1.14 it is off by default, packs to max_length (default 1024) and uses the 'bfd' strategy, which also turns on padding-free batching. Load the model with a FlashAttention implementation, or TRL only warns.

open as a page