Publisher
The leading Composable CDP and AI Decisioning platform
LLM Judges for Identity Resolution at 4x Lower Cost
We wanted to test whether LLMs and decision models, specifically Jev and Luna, could solve a hard problem that we encounter regularly (grading identity resolution results). We found that while both models performed okay out of the box (88.8–97.3% accuracy), implementing a hybrid model that combines Jev + Opus achieves frontier accuracy at > 4x lower costs. - Luna performs better than Jev out of the box, but the gap disappears on harder cases. - Jev's confidence is better calibrated. Most of its mistakes come with low confidence its right. Most of Luna's mistakes come with…
11 min · 2,468 words