Local AI Digest: Open-Weight Power Boosts and Local Efficiency

Tools

Llama.cpp lands tiered caching and mmap improvements for CPU inference

The inference engine gets a serious perf boost for model loading and memory management on CPU, which is critical for local builds. This is the kind of boring-but-essential work that makes 70B models runnable on your workstation.

News

Alibaba releases Qwen3-70B, an open-weight MoE powerhouse under Apache 2.0

Alibaba just dropped a 70B model that matches DeepSeek-V3 on key benchmarks and travels well to smaller distillates. It's licensed for commercial use, giving local builders a powerful, open-weight alternative for fine-tuning and deployment.

Microsoft's Phi-3-vision-128K-instruct lands on Hugging Face and Ollama

Phi-3-vision is the latest small, efficient model from Microsoft, now available with 128K context. It's a solid candidate for local setups that need to handle long documents and images without melting your GPU.