Benchmarks We Actually Believe

Proposed by Diego Ferreira · Cartograph AI

Draft

Decision

Currently draft. You can change your mind at any time.

Proposal

Draft: building an internal benchmark suite that correlates with user-reported quality instead of leaderboard position.

AI EngineeringTalkIntermediate

Evals

Speaker

People attached to this proposal. Accepting copies them onto the session and assigns their onboarding tasks.

Diego FerreiraSpeakerPrimary contact

Staff ML Engineer, Cartograph AIdiego.ferreira@example.com

Diego works on evaluation tooling and spends most of his week arguing that a benchmark you cannot reproduce is a rumour.

Application answers

Everything the speaker filled in on the call-for-papers form.

Talk title: Benchmarks We Actually Believe
Abstract: Draft: building an internal benchmark suite that correlates with user-reported quality instead of leaderboard position.
Track(s): AI Engineering
Session format: Talk (25 min)
Audience level: Intermediate
Have you presented this talk before?: No
Do you need travel support to attend?: No
Speaker biography: Diego works on evaluation tooling and spends most of his week arguing that a benchmark you cannot reproduce is a rumour.

Reviewers

Assign this proposal to a specific reviewer, on top of whoever already gets it through their tracks.

Reviews

0 reviews from the programme committee.

No reviews yet

Once reviewers score this proposal their comments show up here. You can still make a decision without them.

Details

FormatTalk

Audience levelIntermediate

SubmittedNot submitted yet

Decision recorded—

FormDevFlow Conf 2027 — Call for Speakers

Recent emails

Sent to the people on this proposal.

Your talk was accepted for DemoConf 2026diego.ferreira@example.com · 30 Jul 2026 · delivered