Reflection Releases Beam 501B Open MoE Model on Hugging Face

Reflection Beam 501B open MoE model release

A fresh U.S. lab named Reflection just dropped its debut model on Hugging Face: Beam, a 501-billion-parameter mixture-of-experts system released under Apache 2.0. That license is the headline for developers-it means you can download, tweak, and run this frontier-scale model on your own iron without asking anyone for permission. Beam is text-only, laser-focused on coding and agentic workflows, and ships with a one-million-token context window. That’s leagues beyond what most open models offer today.

Under the hood, there are 52 decoder layers with a hidden size of 5,120. Only the first three layers use a dense SwiGLU feed-forward network; the other 49 layers each pack 256 experts-also SwiGLU blocks with an intermediate dimension of 2,560. A router based on the group-limited scheme from DeepSeek-V3 picks eight experts per token, so the active parameter count per forward pass lands around 23 billion. That design lets the model keep the representational capacity of a half-trillion-parameter network while computing at the speed of a much smaller one. The BF16 checkpoint weighs roughly a terabyte in safetensors format, while FP8 and NVFP4 quantized versions target Hopper, Ampere, and Blackwell GPUs respectively.

Beam 501B hybrid attention architecture details

Attention is hybrid: 43 layers use sliding-window attention over the most recent 512 tokens with rotary position embeddings, and every sixth layer plus the final layer attend globally without positional encodings. Instead, global layers apply a learnable query-scaling factor that grows slowly with sequence length, preserving softmax precision as context expands. Grouped-query attention with 48 query heads and eight key-value heads, plus RMSNorm on queries and keys, rounds out the block. The residual stream adopts an unusual normalization scheme-each sublayer output is added with weight 1 over the square root of the step index, while the existing stream is retained with the complementary weight, keeping the overall scale constant from the first layer to the last. A tanh softcap at ±30, computed in FP32, bounds the final logits.

For deployers, the KV cache grows at about 36 kilobytes per token in BF16, roughly one-sixth the rate a fully global 52-layer model would need. At the full one-million-token context, that translates to 36 gigabytes per sequence versus more than 200 gigabytes for an all-global equivalent. Day-one support exists in Transformers, vLLM, and SGLang, and Reflection provides a dedicated Rust-based inference harness they say extracts the best throughput. The tokenizer includes a response template and a dedicated grammar for tool calling, while a speculative decoding drafter called DFlash2 ships alongside the main weights.

Reflection supplied benchmark numbers across BF16, FP8, and NVFP4 precisions. On agentic coding suites, the model scores 81.6 percent on SWE-Bench Verified in BF16, 80.2 in FP8, and 80.7 in NVFP4. Terminal Bench v2.1 comes in at 80.1, 78.9, and 78.2 percent respectively. Reasoning benchmarks show 97.7 percent on AIME 2026 at BF16, 90.6 percent on GPQA Diamond, and 88.3 percent on LiveCodeBench. Tool-use performance on the public split of AutomationBench reaches 39 percent. The lab notes that quantization preserves most scores within a point or two, though a few tasks such as CriPT AA and SciCode show slightly larger gaps.

Beam 501B Apache 2.0 license commercial use

The Apache 2.0 license means teams can fine-tune, distill, or embed Beam in commercial products without royalty obligations. Hugging Face lists supervised fine-tuning recipes using TRL, full-parameter reinforcement learning with the Miles framework, and LoRA adapters for lighter adaptation. Because all 256 experts must reside in memory, deployment still demands the RAM footprint of a 501B model-roughly a terabyte in BF16-but the compute per token matches a 23B dense model. That trade-off makes Beam practical for organizations that already operate multi-GPU nodes or rent equivalent cloud instances.

To encourage experimentation, Reflection and Hugging Face are offering ten thousand dollars in Hub credits for the best community projects built on Beam. Submissions can take the form of a Space demo, a quantized checkpoint for a specific hardware target, a synthetic dataset, an RL environment, a benchmark report, or any other creative use. Participants publish their work as a repo on the Hub and link it from the Beam model page; winners receive credits directly to their accounts.

How to run Beam 501B on Hugging Face

For engineers evaluating whether to adopt Beam, the immediate steps are straightforward: pull the desired precision checkpoint from the Hub, test inference through the Transformers, vLLM, or SGLang integration, and measure latency and memory on your target hardware. Teams with coding-heavy workloads or long-context agent pipelines should compare the published benchmarks against their internal evals. Groups that need a fully open, customizable model at this scale and have the GPU memory to host it will find few alternatives today. The contest deadline and selection criteria haven't been announced, so early submissions might get more visibility.

Source: Hugging Face