Bursty demand and the shape of a team's year

Plot a team's compute demand across a year and it rarely looks like a line. It looks like a handful of spikes with long quiet stretches between them: a retraining when the data shifts, a backlog cleared before a deadline, a sweep of simulations before a review. The shape is bursty, and the shape is the reason E31 Network is being designed the way it is.

What a bursty year looks like

A product team ships a new model, then spends weeks collecting what the model gets wrong, then retrains. An operations team accumulates camera footage for a season and processes it when the season ends. A research group runs a simulation sweep before each design review and nothing between. Each of these teams has a year made of a few heavy weeks and many light ones.

In each case the heavy weeks are predictable in kind but not in timing. The team knows there will be a retraining; it does not know which week the data will justify it. It knows the backlog will need clearing; it does not know how large it will have grown. Planning capacity for this means planning for a spike whose height and date are both uncertain.

Why fixed capacity fits a bursty year badly

Owned compute is a horizontal line across the year. Wherever the demand curve rises above it, work queues on the team's own machines and deadlines stretch. Wherever the demand curve falls below it, the hardware idles and the cost continues. There is no height for the line that avoids both. Set it high and pay for idle months. Set it low and pay in late work.

The usual compromise is to set the line somewhere in the middle and accept both costs in smaller doses. That is a reasonable thing to do when there is no alternative. It is still paying for a shape the hardware cannot follow.

What the network is being designed to do with the shape

The network treats each spike as a job rather than as a reason to resize anything. The team submits the retraining, the backlog pass, or the simulation sweep as a job that declares its work, its data, its request, and its output. Compute is held for the job, the job runs, the output comes back, and the compute is released. The team's year becomes a series of jobs with bills attached, and a series of quiet stretches with none.

The queue is what lets this work. A spike from one team lands in the pool alongside spikes from others, and because most batch work can wait, jobs are scheduled as compute frees up rather than all at once. The team does not need to hold anything in reserve, and the meter does not run while a job waits.

What the network cannot do is flatten a spike that must happen now. Real-time demand, the kind that comes from a machine on a floor or a person waiting for an answer, has no quiet stretch to borrow from and cannot wait for a queue. That work stays on dedicated systems the team owns.

Planning a year around jobs instead of machines

The practical shift is to inventory the spikes rather than estimate a peak. Each spike becomes a job description: what runs, what data it reads, roughly how much compute and for how long, and what should come back. The illustrative requests on this site are examples of that form: compute tonight to retrain a model, eight GPUs for four hours to process a dataset, forty hours of warehouse footage that needs processing, ten thousand simulations to run.

A team that can write its year as a list of such requests has already done most of the planning the network would ask of it. The remaining question is cost per hour, and that figure has not been published. It will be published when access opens, and not before.

The examples page shows what each of those requests might look like as a job. By the hour, not by the month covers why the hourly unit fits a bursty year better than a monthly commitment, and what a published rate would have to cover covers what the missing figure would have to account for.