Idle capacity is the cost nobody budgets

Every compute budget has a line for the hardware. Almost none has a line for the hours the hardware spends waiting. Idle capacity is paid for in full, shows up nowhere as a cost, and is the largest reason that owned compute is more expensive than it looks for a team whose demand comes in bursts.

Where the idle hours hide

A GPU that is not running a job still draws power, still occupies a rack slot, still needs cooling, and still depreciates. None of that stops when the job queue is empty. The expense continues at close to the full rate, but the budget records it as compute, not as compute that is mostly idle.

The idle hours also hide inside the schedule. When a team owns a fixed amount of compute, the busiest week is the one that sets the size. Every week that is less busy than that one leaves a gap between what is owned and what is used. Added together over a year, the gaps are usually far larger than the work.

A team rarely sees this because nobody is asked to measure it. Utilization is a figure that gets looked at when someone is justifying a purchase, and not often afterwards. The cost of what was not used is real, but it is invisible by default.

Why it does not appear in the budget

Budgets are built around purchases and contracts, which are events. Idle capacity is not an event. It is the absence of one, spread thinly across every day the hardware is on and the queue is empty. There is no invoice for it, so there is no line.

There is also an incentive not to look. A team that bought hardware for a peak has already made the case for the peak. Reporting that the hardware sat idle most of the quarter undercuts that case, even though it is the expected outcome and not a failure of anyone's judgment. The result is that the cost is known in principle and ignored in practice.

What the network is being designed to remove

E31 Network is being built so that idle capacity is not the team's problem. Compute is held for a job while the job runs, and released the moment the job is done. There is no hardware sitting in the team's rack between jobs, because there is no hardware in the team's rack at all.

The meter follows the same rule. It starts when compute is held for a job and stops when the compute is released. A job waiting in a queue is not metered. A team with no jobs this week pays nothing this week. The hours that would have been idle under an owned model are hours that simply do not appear on the bill.

This moves the problem of idle capacity rather than making it disappear. Somewhere, a pool of compute is still waiting between jobs, and keeping that pool busy across many teams' bursts is one of the scheduling problems the program has not yet solved. The difference is who carries the cost. Under the network's design, the pool carries it, and the team pays for the hours it uses.

Budgeting for the job instead of the hardware

The practical change for a team is that the unit of budgeting shifts from the machine to the job. A retraining run is a request for compute for some hours. A footage backlog is a request for compute until the backlog is clear. The budget question becomes how many hours this job needs, not how many machines the year needs.

Rates have not been published, so the hourly figure that would make this calculation concrete does not exist yet. What exists is the shape: a cost attached to a job, for the hours the compute is held, and nothing attached to the hours in between.

For a team that wants to see what a job request looks like in practice, the examples page walks through a dataset pass, a retrain, a footage backlog, and a simulation sweep. For the question of when owning compute is still the right call, see when owning compute makes sense.