Fast state-of-the-art tokenizers optimized for AI research and production
Minimalist, high-performance ML framework for Rust with GPU support and LLM inference