DevFlow Conf 2027

A rapid-fire Q&A on running AI agents in production: what breaks, what monitoring actually catches problems, and the guardrails that matter most when agents touch real systems.

Code generation is easy; trusting it is hard. This session covers verification patterns for AI-generated code — property tests, mutation coverage, snapshot judges, and CI gates — with data from 18 months of running them on a 200-engineer codebase. Includes what we stopped doing because it didn't catch anything.

Longer context is not free. A cost model for context spend, with the break-even points where retrieval beats stuffing the window.

Unglamorous optimisations with outsized effects, benchmarked on production traffic rather than a synthetic loop.

A 10-minute tour of turning a static docs site into one that answers questions with citations, stays honest when it doesn't know, and costs under $50/month to run. Live demo, real failure cases, and a checklist you can apply to your own docs this week.

Our monorepo CI took 40 minutes on a good day. This talk walks through how we cut it to 6 minutes with content-addressed caching, remote execution, and a test-selection model — including the two migrations that failed first. You'll leave with a decision framework for which incremental-build investments pay off at which repo sizes, and the graphs to convince your platform team. This session now includes a live demo of remote build caching. Attendees should bring a laptop.

A migration story across four teams: the ownership model, the CI that made it safe, and the two rewrites we would skip if we did it again.

Review queues, escalation and undo, designed so the humans in the loop stay willing to be in it. Patterns, anti-patterns and measured throughput.

A field report on running long-lived agents in production: timeouts, retries, poison inputs, and the runbook that finally made the pager quiet. We started with one agent that could file a support ticket and ended up with a fleet of them touching billing, inventory and customer email. Everything that broke, broke at night — a tool call that hung for nineteen minutes, a retry loop that re-sent the same refund four times, and a prompt change that quietly doubled token spend on the busiest queue of the week. This keynote walks through the three incidents that reshaped our architecture and the guardrails we shipped after each one: hard per-step deadlines, idempotency keys on every side effect, a spend ceiling enforced outside the model, and a replay view that lets an on-call engineer see exactly what the agent believed at the moment it went wrong. You will leave with the runbook we now hand to every team before their agent gets production credentials, the four dashboards we page on, and an honest account of which safeguards were worth the engineering time and which ones we deleted six weeks later.

Evaluations are usually treated as an internal chore. We shipped ours to customers instead, and it changed how the whole team argued about quality.

Three years of retrieval architectures, what survived, what did not, and why the boring parts (chunking, freshness, permissions) still decide the outcome.

A twenty-minute look at how Kestrel Compute schedules mixed CPU/GPU workloads, presented by our platinum sponsor.

Caching, batching, routing and quantisation, measured end to end on a real workload. Includes the changes that looked clever and made things worse. Our inference bill grew faster than our traffic for two quarters straight, and the obvious answers — a smaller model, a cheaper provider — were off the table for quality reasons. So we went looking for the money everywhere else: in the cache we were not reusing, in the requests we were sending one at a time, and in the long tail of prompts nobody had read since launch. In this hands-on session we rebuild that programme from scratch against a live workload. You will add a prompt cache and measure the real hit rate rather than the hoped-for one, batch a stream of requests and watch the latency percentiles you just traded away, route by difficulty so the expensive path only handles the requests that need it, and trim context that has been dead weight since the second sprint. We finish with the cost model we now review monthly — cost per resolved request rather than cost per token — plus the two optimisations we rolled back and the graphs that told us to. Bring a laptop; every exercise ships with a workload you can run locally.

Schema drift, partial responses and retries: why validating model output belongs in the same mental model as any other unreliable network call.

Live-coded from an empty file: a minimal, debuggable tool-calling loop with cancellation, tracing and a sandbox, and what each abstraction buys you.

Traces, spans and sampling when the same input legitimately produces different output. What to log, what to redact, and how to make it searchable.