When owning compute makes sense, and when renting does

Owning compute makes sense when the compute is busy most of the time, or when the work it does cannot wait. Renting makes sense when the compute would be idle most of the time and the work can wait for a queue. Most teams have some of each, and the honest answer is that both arrangements have a place.

This article is about where the line falls. E31 Network is being designed for the renting side of it, and Element 31 builds a separate product for the owning side, so the program has no reason to pretend that one answer fits every job.

The utilization test

Start with how busy the compute would be. A team that runs training, inference, or processing jobs around the clock, every day, has a flat demand curve. For that team, owned hardware is used close to its capacity and there is little idle time to pay for. Renting by the hour would cost more, not less, because every hour is a used hour either way and the owned hardware has no intermediary.

A team whose demand is heavy some weeks and light most weeks has the opposite curve. Owned hardware sized for the heavy weeks sits mostly idle. Owned hardware sized for the light weeks turns every heavy week into a bottleneck. Renting lets the team pay for the heavy weeks as they happen and pay nothing in between.

The test is not precise, and the program does not publish a threshold, because no rate has been published to compute one against. But the direction is clear: the flatter the curve, the stronger the case for owning. The spikier the curve, the stronger the case for renting.

The latency test

The second test is whether the work can wait. A job on the network goes into a queue when compute is not free, and runs when it is. That is fine for a retraining run that needs to finish by morning, a footage backlog that needs to be clear by the end of the week, or a simulation sweep ahead of a design review.

It is not fine for anything that must answer now. Robot control, live perception for a moving machine, safety functions, live operator tooling, control loops, and interactive assistants that must respond in the moment all need compute that is already there and already theirs. That compute has to be owned, or at least dedicated, because a queue of any length is a failure.

The network is not being designed for that work and will not be. Anything real-time stays on the team's own infrastructure, on edge devices, or on the machines themselves. A team with a mix of both kinds of work would own compute for the real-time part and rent compute for the batch part, and the two would not compete for the same hardware.

Where Czar fits

For the owning side, Element 31 builds Czar, a sealed appliance for a team that wants AI compute in its own rack on its own subscription, or joined into a Czar Grid for a facility. This site sells neither product and describes no pricing for Czar; it is mentioned here because it is the reason the network does not need to stretch to cover work it is not suited for.

A practical way to split the work

List the jobs the team runs over a year. For each, ask whether it could run tonight instead of now, and whether it runs every day or only sometimes. The jobs that could run tonight and run only sometimes are network jobs: the retrain, the backlog, the sweep, the one-time dataset pass. The jobs that must run now, or run constantly, are owned-compute jobs.

Most teams will find the list splits cleanly. The retraining that produces a model is a network job; serving that model to a machine on the floor is not. Processing a season of footage is a network job; the live camera feed that produced it is not. The network is being designed to take the first half of each pair and leave the second where it belongs.

The split also clarifies the budget. The owned half is a fixed cost that should be sized for the real-time and constant work only, not inflated to absorb the occasional spike. The rented half is a cost that appears when a job runs and not otherwise. Keeping them separate makes each one easier to justify and easier to measure.

See the workloads page for the full list of what belongs on the network, and paying for the peak at the trough for the cost of owning hardware sized for a spike.