The MoE Models Arrive Locally

Tools

vLLM Adds Model Support for Llama 4 and Qwen 3 in Latest Release

vLLM 0.8.5 is a stability-focused release, but the new support for the Llama4 and Qwen3 model families is the real story for builders. It means you can now run these larger, more capable models locally with vLLM's optimized serving stack, which was previously a major pain point.

llama.cpp PR #12435: Official Gemma Sampling Logic for Higher Quality Output

This PR to llama.cpp is a massive win for inference accuracy with Gemma models. It implements the official sampling logic, which means the output you get locally will be much closer to what Google intended. For anyone building with Gemma, this is a must-have upgrade.

MLX Community Hub Hits 1,000+ Optimized Models for Apple Silicon

The MLX ecosystem for Apple Silicon keeps getting more serious. The mlx community hub now has over 1000 models, optimized and ready to go. This removes the biggest friction for developers on Mac: finding and converting models for the hardware.

News

Qwen3-30B-A3B: A Potent 30B MoE Model Runs on Just 3B Active Parameters

Qwen3-30B-A3B is a big deal. It's a Mixture-of-Experts model that runs with only 3B active parameters, making it one of the most capable models you can actually run on consumer hardware. This is the kind of model that lets you build features that were previously server-only.

Microsoft's Phi-4-Reasoning-Plus: A New Small Model for Logical Tasks

Phi-4-reasoning-plus is Microsoft's play for the small, fast, reasoning-focused model space. It's designed for tasks that need step-by-step logic, not just pattern matching. This gives builders another strong option for on-device AI that needs to think, not just autocomplete.