Track: AI EngineeringTrack: Infrastructure & ScaleFormat: TalkLevel: Intermediate
The Economics of Context Windows
Tuesday, May 11: 9:00 AM – 9:30 AM
Presidio Room
Longer context is not free. A cost model for context spend, with the break-even points where retrieval beats stuffing the window.
Speakers (1)
- JWJonas Weber
Infrastructure Lead, Kestrel Compute
Jonas has been on call for GPU fleets since before it was fashionable. He writes about scheduling, spot capacity and the true cost of a token.