All jobs → Partner boards

Machine Learning Engineer - Evals

PLASTIC LAB • New York, NY, US • published 2026-10-19 18:32:16.019205

Description

**About this role** You'll own evaluation of Honcho end to end, the instrument that tells a research\-driven team whether their identity representations are actually getting better. Plastic Labs builds the memory and identity layer for the agentic world, and the product works. It's a complex multi\-agent harness with real black\-box behavior, so the job is defining what "better" means for representations that change over time, without ground truth, and building the machinery that measures it. If you love constructing measurements from nothing, reading traces to find what's actually broken, and shipping the fix yourself, you'll feel at home here. **What You'll Do** Design the evals. Define the scores and decide what "better" means for an entity representation that changes over time, and keep that definition current as the product and methods move. Build the pipelines and harnesses. Data in, labels, versions, reruns, judges: the machinery that lets the team ask a new question this week and get an answer this week. Run them and harvest insights. Read the traces and results, find what's actually broken, not what's easy to measure, and propose fixes that produce higher\-fidelity represe...

Get jobs like this daily in Telegram: @careeri_bot