Benchmarking: An Introduction

“When a measure becomes a target, it ceases to be a good measure” – Goodhart’s law Current AI research, especially the frontier LLM research, is dominated by benchmarks. It is the first thing we look at when a new model comes out, it is the headline of each release, and they dominate the discourse when […]

Guide · AI · Benchmarks · LLMs · Research

Running local LLMs on your Mac: what fits, what's free, and what's overkill

A practical guide to running language models on Apple Silicon: which sizes fit common Macs, free options that work well, and when bigger local models are overkill.

Guide · LLMs · Hardware · Developer Tools · AI

Frequently Asked Questions (And Answers) About AI Evals

Hamel Husain and Shreya Shankar’s sharp FAQ on AI/LLM product evals: start with error analysis on real traces, build targeted evaluators, validate LLM judges with TPR/TNR, and avoid generic off-the-shelf metrics.

Guide · AI · LLMs · Engineering · Machine Learning

Prompting Claude Opus 5.5

Anthropic’s guide to prompting Claude Opus 5.5: how the model behaves, patterns that work for complex agentic and coding tasks, and practical prompt-engineering advice for builders.

Guide · AI · LLMs · AI Agents · Tutorials