Agent Evaluation Gets Testable, Structured Output Unlocks Production, and The Observability Gap Closes

Tools

Claude 3.5 Sonnet Adds Structured Output Natively

Anthropic's latest model update bakes structured JSON output directly into the API, eliminating the fragile prompt-engineering gymnastics builders have been doing with tool-use schemas. This is table stakes for production agents that need to reliably parse results without retry loops.

Anthropic Cookbook Updates Tool-Use Patterns for Multi-Turn Agent Flows

The refreshed cookbook examples now show tool-use patterns that handle error recovery and multi-turn context, not just happy-path single calls. Worth studying if you're building agents that need to gracefully degrade when tools fail mid-execution.

Analysis

Why Agent Evaluation Needs Decomposition, Not End-to-End Benchmarks

BAIR's research on compound AI systems argues that evaluating entire agent loops tells you almost nothing about where failures happen. The actionable insight: instrument each step independently and benchmark components, not just final outputs.

The Batch: Agent Reliability Becomes the New Bottleneck

Andrew Ng's latest newsletter zeroes in on agent reliability as the constraint holding back production adoption, with practical notes on where builders should focus testing efforts. The signal is clear: reliability beats capability for shipping.

News

LangGraph Platform GA Brings Production Observability to Agent Workflows

LangGraph's GA release ships with built-in tracing and state inspection for agent loops, finally giving builders visibility into what's actually happening inside multi-step workflows. The gap between prototype and production agents narrows when you can actually see where things break.