PyTorch distributed training library for LLMs/VLMs with out-of-the-box Hugging Face support
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale
Build a ChatGPT-like LLM in PyTorch from the ground up, step by step
Collaborative speedrun to train a 124M GPT-2 model in under 90 seconds on 8xH100s
