Gemma 4 Runs on Raspberry Pi, Jetson, and Your Browser: The Edge Deployment Breakdown

Tools

LiteRT-LM Runs Gemma 4 E2B Under 1.5GB on Mobile and Raspberry Pi

LiteRT-LM uses 2-bit and 4-bit quantization plus memory-mapped embeddings to fit Gemma 4 E2B into under 1.5GB RAM, and hits 133 prefill tokens/s on a Raspberry Pi 5 CPU. On Qualcomm Dragonwing IQ8 with NPU acceleration that jumps to 3,700 prefill tokens/s, which is a real signal for edge deployment viability. The new litert-lm CLI also supports tool calling out of the box, so you can prototype agentic pipelines on a Pi without writing any code.

Gemma 4 Ships Day-0 on llama.cpp, MLX, WebGPU, and Rust with Full Multimodal Support

Hugging Face coordinated day-0 support across transformers, llama.cpp, MLX, WebGPU, and mistral.rs, meaning you can run Gemma 4 with audio, image, and text inputs locally today across basically every popular inference stack. Google also released Multi-Token Prediction drafters that claim up to 3x inference speedups via speculative decoding with no quality loss. If you were waiting for the ecosystem to catch up before testing Gemma 4 locally, that moment is now.

Gemma 4 31B and 26B Run on RTX PCs and DGX Spark for Local Agentic Dev Workflows

The 31B dense and 26B MoE variants run efficiently on NVIDIA RTX GPUs and DGX Spark, which makes them viable for local coding assistants and agentic workflows without a cloud bill. NVIDIA collaborated with Ollama and llama.cpp to optimize the experience, so the setup friction is low. OpenClaw compatibility means you can build always-on local agents that pull context from personal files right now.

News

NVIDIA Jetson Orin Nano Runs Gemma 4 E2B and E4B for Physical AI Agents

NVIDIA confirmed Gemma 4 E2B and E4B run on Jetson Orin Nano via llama.cpp and vLLM, with the model family scaling up to Jetson Thor for more demanding robotics workloads. This makes Gemma 4 a practical option for embedded agents that need audio, vision, and reasoning without cloud connectivity. Unsloth also ships day-one quantized checkpoints for local fine-tuning if you need to adapt the model to a specific domain.