> ## Documentation Index
> Fetch the complete documentation index at: https://facetai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Fine-tuning Guide

> Complete guide to configuring and launching training jobs in Facet

This comprehensive guide walks you through every aspect of training configuration, from selecting the right model to monitoring your training progress.

## Selecting Your Base Model

### Model Size Considerations

<Tabs>
  <Tab title="Small Models (270M-1B)">
    **Best for**: Simple tasks, quick prototyping, resource-constrained environments

    * **Gemma 3 270M**: Fastest, good for basic text generation
    * **Gemma 3 1B**: Balanced performance, good for most tasks

    <Tip>
      Start with 1B for most applications. It provides good performance while being resource-efficient.
      These models do not support vision fine-tuning!
    </Tip>
  </Tab>

  <Tab title="Medium Models (4B-12B)">
    **Best for**: Complex reasoning, high-quality responses, production use -
    **Gemma 3 4B**: High-quality responses, good for most production tasks -
    **Gemma 3 12B**: Advanced reasoning, complex problem-solving

    <Note>
      These models require more compute resources but offer significantly better
      performance.
    </Note>
  </Tab>

  <Tab title="Large Models (27B)">
    **Best for**: State-of-the-art performance, research, complex domains

    * **Gemma 3 27B**: Maximum performance, best for complex tasks

    <Warning>
      Large models require significant compute resources and longer training times.
    </Warning>
  </Tab>
</Tabs>

## Choosing Your Training Method

### Supervised Fine-Tuning (SFT)

**When to use**: General conversation, instruction following, domain adaptation

**Data format**: Language modeling

<AccordionGroup>
  <Accordion title="Best Practices">
    * Use high-quality conversation data
    * Include diverse examples
    * Ensure consistent formatting
    * Balance different types of interactions
  </Accordion>

  <Accordion title="Example Use Cases">
    * Customer service chatbots
    * Code assistants
    * Educational tutors
    * Domain-specific Q\&A systems
  </Accordion>
</AccordionGroup>

### Direct Preference Optimization (DPO)

**When to use**: Aligning models with human preferences, safety, helpfulness

**Data format**: Preference tuning (chosen vs rejected responses)

<AccordionGroup>
  <Accordion title="Data Requirements">
    * Paired examples (chosen vs rejected)
    * Clear preference signals
    * Diverse preference types
    * High-quality annotations
  </Accordion>

  <Accordion title="Example Use Cases">
    * Safety alignment
    * Helpfulness optimization
    * Style preference learning
    * Response quality improvement
  </Accordion>
</AccordionGroup>

### Odds Ratio Preference Optimization (ORPO)

**When to use**: Alternative to DPO, preference learning

**Data format**: Same as DPO (chosen vs rejected responses)

<Note>Unlike DPO, you do not need to perform SFT prior to ORPO training.</Note>

### Group-Related Policy Optimization (GRPO)

**When to use**: Reasoning tasks, math problems, structured thinking

**Data format**: Prompt-only (no assistant responses)

<AccordionGroup>
  <Accordion title="Reward Functions">
    * **Built-in**: Expression accuracy, numerical accuracy, format checking
    * **Reference-based**: String comparison, text similarity
    * **Model-based**: LLM scoring, classification, relative ranking
  </Accordion>

  <Accordion title="Example Use Cases">
    * Math problem solving
    * Logical reasoning
    * Code generation
    * Scientific problem solving
  </Accordion>
</AccordionGroup>

## PEFT Configuration

### Full Fine-tuning

**When to use**: Large datasets, major domain changes, maximum performance

**Pros**:

* Updates all parameters
* Maximum learning capacity
* Best for major domain shifts

**Cons**:

* Requires more memory
* Longer training time
* Higher compute costs

### LoRA (Low-Rank Adaptation)

**When to use**: Most fine-tuning tasks, good balance of performance and efficiency

**Pros**:

* Memory efficient
* Faster training
* Good performance
* Easy to merge

**Cons**:

* Slightly lower capacity than full fine-tuning
* May need higher rank for complex tasks

## Quantization Configuration

### QLoRA (Quantized LoRA)

**When to use**: Resource-constrained environments, most use cases

**Pros**:

* Most memory efficient
* Good performance
* Fast training
* Recommended default

**Cons**:

* Slight quantization overhead
* May need fine-tuning for optimal results

<Tip>
  Start with QLoRA for most tasks. It provides excellent results while using
  minimal resources.
</Tip>

Note that we currently only support 4 bit quantization for QLoRA. We will be moving away from bitsandbytes in the future to more robust methods, stay tuned.

## Hyperparameter Configuration

### Essential Parameters

<Tabs>
  <Tab title="Learning Rate">
    **Purpose**: Controls how much the model updates during training

    **Recommended values**:

    * Full fine-tuning: 0.00005
    * LoRA: 0.0001
    * QLoRA: 0.0002

    <Note>
      Lower learning rates are more stable but may require more epochs.
    </Note>
  </Tab>

  <Tab title="Batch Size">
    **Purpose**: Number of examples processed together **Recommended values**: -
    Small models: 4-8 - Medium models: 2-4 - Large models: 1-2

    <Warning>
      Larger batch sizes require more memory but may improve stability.
    </Warning>
  </Tab>

  <Tab title="Epochs">
    **Purpose**: Number of complete passes through the dataset

    **Recommended values**:

    * 1-3 epochs for most tasks

    <Tip>
      Monitor validation loss to avoid overfitting. Stop early if validation loss starts increasing.
    </Tip>
  </Tab>
</Tabs>

### Advanced Parameters

<AccordionGroup>
  <Accordion title="Gradient Accumulation">
    **Purpose**: Simulate larger batch sizes without using more memory

    **Formula**: Effective batch size = batch\_size × gradient\_accumulation\_steps

    **Example**: batch\_size=2, gradient\_accumulation\_steps=4 → effective batch\_size=8
  </Accordion>

  <Accordion title="Learning Rate Scheduler">
    **Purpose**: Adjusts learning rate during training **Options**: - `linear`:
    Linear decay from initial to 0 - `cosine`: Cosine annealing - `constant`:
    Fixed learning rate **Recommendation**: Use `linear` for most cases
  </Accordion>

  <Accordion title="Sequence Length">
    **Purpose**: Maximum length of input sequences

    **Considerations**:

    * Longer sequences use more memory
    * Should match your data's typical length
    * Common values: 1024, 2048, 4096

    **Recommendation**: Start with 2048, adjust based on your data
  </Accordion>
</AccordionGroup>

## Evaluation Configuration

Note that this evaluation is different from the inference/evaluation service used post-training. This calculates token-level metrics and raw eval loss.
For most use cases, you can disable this to speed up training and use the inference service for evaluation after training.

### Evaluation Strategies

* **`no`**: No evaluation (fastest training)
* **`steps`**: Evaluate every N steps
* **`epoch`**: Evaluate at the end of each epoch

<Tip>
  Use `epoch` evaluation for most cases. It provides regular feedback without
  slowing training significantly.
</Tip>

### Evaluation Metrics

The system automatically computes:

* **Accuracy**: Token-level accuracy
* **Perplexity**: Model's confidence in predictions

<Note>
  For task-specific metrics, use the inference service's evaluation endpoint
  after training.
</Note>

## Monitoring Training Progress

### Weights & Biases Integration

Track your training with W\&B for detailed monitoring by providing your API key.

## Trackio

We are working on this :)

## Export Configuration

<Tabs>
  <Tab title="Adapter Format">
    **Best for**: LoRA/QLoRA models, easy to merge later

    * Smaller file size
    * Requires base model to run
    * Easy to combine with other adapters
  </Tab>

  <Tab title="Merged Format">
    **Best for**: Standalone deployment, vLLM inference

    * Self-contained model
    * Larger file size
    * Ready for deployment
  </Tab>

  <Tab title="GGUF Format">
    **Best for**: CPU inference, llama.cpp, ollama

    * Quantized for efficiency
    * CPU-optimized
    * Good for local deployment
  </Tab>
</Tabs>

Learn more about export options on the [Inference, Evaluation, and Export](/evaluation/inference_export.mdx) page.

## Troubleshooting

### Common Issues

<AccordionGroup>
  <Accordion title="Out of Memory (OOM)">
    **Solutions**:

    * Reduce batch size
    * Increase gradient accumulation steps
    * Use QLoRA instead of LoRA
    * Reduce sequence length
    * Use a smaller model
  </Accordion>

  <Accordion title="Training Loss Not Decreasing">
    **Possible causes**: - Learning rate too high or too low - Poor data quality -
    Incorrect data format - Model too small for task **Solutions**: - Adjust
    learning rate - Check data quality - Verify data format - Try larger model
  </Accordion>

  <Accordion title="Overfitting">
    **Signs**:

    * Training loss much lower than validation loss
    * Validation loss increasing

    **Solutions**:

    * Reduce epochs
    * Increase dropout
    * Use more diverse data
    * Early stopping
  </Accordion>
</AccordionGroup>

## What's Next?

After your training completes:

1. **Evaluate Performance**: Use the evaluation tools to assess your model
2. **Export Your Model**: Download in your preferred format
3. **Deploy**: Set up your model for production use
4. **Iterate**: Use feedback to improve with additional training

Ready to evaluate your model? Head to the [Evaluation guide](/evaluation/introduction) to learn how to test your fine-tuned model.