Post-trains transformer language models with the TRL CLI - SFT, DPO, GRPO, KTO, RLOO, and reward modeling - covering dataset preparation, configuration, and the right command per method.
---
name: trl-training
description: Train and fine-tune transformer language models with TRL (Transformers Reinforcement Learning) CLI commands - SFT, DPO, GRPO, KTO, RLOO, and reward model training. Use for post-training/alignment workflows with TRL.
---
# TRL Training
TRL provides CLI commands for post-training foundation models with state-of-the-art techniques.
## Methods
- `trl sft` - supervised fine-tuning on instruction/conversational datasets.
- `trl dpo` - Direct Preference Optimization to align from preference data.
- `trl grpo` - Group Relative Policy Optimization (rank multiple sampled outputs by comparative reward).
- `trl rloo` - REINFORCE Leave-One-Out online RL with generation-based rewards.
- `trl kto` - Kahneman-Tversky preference optimization.
- `trl reward` - train reward models for RLHF.
## Workflow
1. Pick the method that matches your data (instructions, preference pairs, or reward signals).
2. Prepare and format the dataset for that method.
3. Run the matching `trl <method>` command with model, dataset, and training-config arguments.
4. Evaluate and push the resulting model to the Hub; see the TRL docs (huggingface.co/docs/trl/en/clis) for per-method options.
Full skill & source: https://github.com/huggingface/skills/tree/ea9a24f38a98baf2430dab3bcb31a89753ae9e6d/skills/trl-training