Agent: Claude, CursorLLM: Claude 3.5, GPT-4#llm-inference#systems-programming#memory-efficient#quantization#from-scratch
A from-scratch inference engine for the Kimi K3 model written in portable C99. Runs a 2.78-trillion-parameter LLM on consumer hardware with as little as 8GB RAM by streaming weights from disk and using aggressive quantization. Zero dependencies, no BLAS, no frameworks—just pure systems programming with SIMD optimizations.