Fine-tuning and LoRA: train, save and evaluate an adapter

A model repeatedly returns the wrong output format even though the necessary facts are already in the request. Fine-tuning may help teach that recurring behavior. If the problem is missing current information, retrieval is often the more direct fix: training yesterday's procedure into a checkpoint does not keep it current tomorrow.

This guide follows a small LoRA adaptation from examples to a saved adapter, then explains how to decide whether it deserves use. The recommended first run is a smoke test in an isolated Python environment. Its purpose is to inspect the training and reload mechanics. Four synthetic examples cannot establish that a classifier works.

What LoRA changes

Fine-tuning continues training from an existing model. Full fine-tuning can update all its parameters. LoRA, or low-rank adaptation, instead trains smaller update matrices while keeping the original weights frozen. The update participates in the selected layers' computation, so the model's behavior can change without saving another complete copy of all base weights.

An adapter is therefore not a standalone model. It depends on a compatible base checkpoint and configuration. A small adapter file also does not imply that training needs only that much memory: the base model, activations and optimizer work still have to fit. The weights guide explains the underlying training distinction.

Prepare a bounded experiment

Use a machine with a supported Python and PyTorch installation, room to download the base checkpoint and enough available memory for training. This example uses CPU computation and a small model to avoid GPU-specific setup; it can still take time and consume substantial memory. Use an isolated practice environment, not a serving process, and stop on resource exhaustion rather than changing shared limits.

The SmolLM2 model card documents loading the selected checkpoint. PEFT supplies adapter configuration and saving. This is sourced technical instruction with synthetic data, not Nate's original training recipe or a claim of measured model improvement.

python3 -m venv .venv
. .venv/bin/activate
python -m pip install torch transformers peft
python -m pip freeze > requirements-used.txt

Save the following script as lora-smoke.py and run python lora-smoke.py. It downloads the model, performs four training steps, saves the adapter and reloads it against the recorded base revision. The software and model versions in your run matter, so retain the dependency list and generated run record with the adapter.

import json
from pathlib import Path
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import LoraConfig, get_peft_model, PeftModel

base = 'HuggingFaceTB/SmolLM2-135M-Instruct'
torch.manual_seed(7)
tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(base)
revision = model.config._commit_hash
model = get_peft_model(model, LoraConfig(
    task_type='CAUSAL_LM', r=4, lora_alpha=8,
    target_modules=['q_proj', 'v_proj'], lora_dropout=0.0,
))
model.print_trainable_parameters()
# Synthetic smoke data, not a representative training collection.
train = [
    ('Certificate expired on the public endpoint.', 'CERTIFICATE'),
    ('The disk cannot create files because no inodes remain.', 'STORAGE'),
    ('TLS negotiation reports an expired certificate.', 'CERTIFICATE'),
    ('The volume has exhausted available space.', 'STORAGE'),
]

def prompt(text):
    return ('Classify the report as CERTIFICATE, STORAGE or UNKNOWN.\n'
            'Report: ' + text + '\nClass:')

optimizer = torch.optim.AdamW(
    [p for p in model.parameters() if p.requires_grad], lr=1e-4)
model.train()
for text, answer in train:
    prefix = tokenizer(prompt(text), add_special_tokens=False)['input_ids']
    suffix = tokenizer(' ' + answer + tokenizer.eos_token,
                       add_special_tokens=False)['input_ids']
    ids = torch.tensor([prefix + suffix])
    labels = ids.clone()
    labels[:, :len(prefix)] = -100
    optimizer.zero_grad()
    loss = model(input_ids=ids, attention_mask=torch.ones_like(ids),
                 labels=labels).loss
    loss.backward()
    optimizer.step()
    print('training loss', float(loss.detach()))

out = Path('certificate-adapter-smoke')
model.save_pretrained(out)
tokenizer.save_pretrained(out)
(out / 'run.json').write_text(json.dumps({
    'base': base, 'revision': revision, 'seed': 7,
    'purpose': 'synthetic four-step smoke test, no efficacy claim',
}, indent=2))
# Reload the exact base revision with the saved adapter.
reloaded_base = AutoModelForCausalLM.from_pretrained(base, revision=revision)
reloaded = PeftModel.from_pretrained(reloaded_base, out)
reloaded.eval()
inputs = tokenizer(prompt('The certificate validity period has ended.'),
                   return_tensors='pt', add_special_tokens=False)
with torch.no_grad():
    output = reloaded.generate(**inputs, max_new_tokens=8, do_sample=False,
                               pad_token_id=tokenizer.eos_token_id)
print(tokenizer.decode(output[0, inputs['input_ids'].shape[1]:],
                       skip_special_tokens=True))

The script trains only the selected query and value projection adapters. It masks the prompt tokens with -100 in the labels, so the loss concerns the answer continuation rather than learning to reproduce the prompt. This is plain completion formatting, kept identical during training and generation. It is not the model's chat-template path; changing to chat messages requires consistent preparation and a fresh evaluation.

Inspect the actual output and trainable-parameter report. A falling loss on these four examples shows optimization on those examples, not generalization. The final generated label may be wrong or include extra text. That is an outcome to record, not a reason to manufacture an expected accuracy result.

Replace the smoke data with a real evaluation

For a hypothetical support classifier, collect permitted examples with reviewed labels and a useful UNKNOWN category. Separate training, development and final test collections by the underlying case, keeping paraphrases and related records together. Otherwise a near-duplicate can leak the answer into a supposedly unseen evaluation.

Before training, run the unchanged base setup on the held-out questions and retain its answers. Define what counts as useful: correct class, valid output format, appropriate uncertainty and behavior on unrelated requests. Select thresholds that reflect the task's consequences before inspecting the final test. A routing error and a harmless formatting error need not carry the same cost.

Use development results to choose training settings. Keep the final test sealed until the chosen candidate is ready; repeatedly tuning against it turns it into another development set. Retain failures as well as successes. Training labels can encode inconsistencies or private information, and a checkpoint does not make that material safe to distribute.

Decide whether to keep the adapter

Compare useful completed tasks within the same data and hardware constraints, rather than adopting the adapter because training finished. Hold the prompt, evaluation inputs, decoding, runtime and supplied evidence constant when you want to assess the adaptation. If you change those too, report a comparison of complete setups; do not assign every improvement to LoRA.

If representative held-out work does not improve, retain the working setup. Return to the original base checkpoint and prompt without loading the adapter; keep both artifacts distinct so that reversal is explicit. Do not merge the adapter into the only copy of your baseline.

When the smoke test succeeds, the next achievement is a trustworthy comparison, not a production deployment. If you cannot evaluate the intended behavior on representative unseen work, you do not yet have evidence to replace the existing system. That boundary is what keeps a technically successful training run from becoming an unsupported operating decision.