<cite index="56-1,56-2">Meta released Llama 4 Scout and Maverick models with open weights; as of mid-2026, Behemoth is still in preview/internal training with early benchmark numbers showing it outperforms GPT-4.5, Claude 3.7 Sonnet, and Gemini 2.0 Pro on STEM reasoning tasks</cite>. <cite index="60-2">Llama 4 Scout features 17 billion active parameters, 16 experts, and 109 billion total parameters, while Maverick has 17 billion active parameters, 128 experts, and 400 billion total parameters</cite>. <cite index="60-3">Scout achieves an unprecedented 10-million token context window, far exceeding the 128K context window of Llama 3, enabling processing of entire books or codebases in a single prompt</cite>.
<cite index="51-2">Llama 4 uses alternating dense and mixture-of-experts (MoE) layers for inference efficiency, with 128 routed experts and a shared expert, allowing only a subset of total parameters to activate while serving; Maverick can run on a single NVIDIA H100 DGX host for easy deployment or distributed inference</cite>. <cite index="60-4">Meta trained using FP8 precision without sacrificing quality, achieving 390 TFLOPs/GPU while pre-training the Behemoth model on 32K GPUs, and developed MetaP, a new training technique that reliably sets critical hyperparameters such as per-layer learning rates</cite>.
<cite index="60-1">Llama 4 Maverick offers approximately 9-23x better price-performance ratio compared to GPT-4o while maintaining comparable or better performance on most benchmarks</cite>. However, <cite index="58-2">open-source communities were not entirely satisfied with the cost and memory intensity of sparse MoE models, which price out participants on limited GPUs compared to smaller-scale dense alternatives like Qwen 2.5</cite>.