The queue is part of the design

Most systems treat a queue as something to apologise for. E31 Network is being designed around one. When a job is submitted and no compute matching its request is free, the job waits. While it waits, the meter does not run. When compute is released by another job, the next waiting job that fits can take it. That loop, repeated, is what a shared pool of hourly compute is.

This article explains why the queue is a feature of the design, what it is meant to cost the submitter, and which parts of the scheduling problem the preview is honest about not having solved.

What the queue is for

A team whose compute need is heavy some of the time and light most of the time has two options. Own enough for the peak and pay for it at the trough, or rent at the peak and own nothing. Renting only works if the rented compute is doing something for someone else when you are not using it. The queue is what makes that true.

Compute that is released by one job becomes free. A job that has been waiting for compute of that shape takes it. The pool stays occupied, and the cost of keeping it occupied is that jobs sometimes wait. For batch work, waiting is cheap. A retraining run that starts at midnight instead of ten in the evening still has its model ready by morning.

The meter does not run in the queue

The design draws a hard line between a job that is waiting and a job that is renting. A queued job holds no compute. Because it holds nothing, nothing is metered. The meter is designed to start at the moment compute is held for the job and not a moment earlier.

This is the detail that makes waiting tolerable. If the meter ran from submission, a team would be paying for the queue, and a long queue would be a cost rather than an inconvenience. The article on when the meter starts goes through the three states a job can be in and which of them are paid.

The queue as a filter

The queue also does a second job. It is the test the network applies to decide whether a workload belongs here at all. Can it wait for a queue. If the answer is yes, it is a batch or asynchronous job and the network is designed for it. If the answer is no, it must answer now, and it stays on dedicated local infrastructure, an edge device, a robot, or the customer's own systems.

That filter is not a limitation the program plans to remove. A job that cannot wait is not a network job. A control loop, live perception for a moving machine, a safety function, an interactive assistant that must respond in the moment: these have no place in a queue, and the network makes no attempt to serve them. The workloads page lists what does belong.

Fair scheduling is unsolved

Here is the part the preview is candid about. A queue with one kind of job is simple. A queue holding a job that wants eight GPUs for four hours beside a job that wants a few minutes on one is not. Serve the small jobs first and the large job may wait indefinitely. Serve in strict arrival order and the small jobs pile up behind the large one.

The program lists scheduling mixed job sizes fairly as an open problem. The related question of what a job must declare for the scheduler to place it well is open too. The design intent is that a job's request, including its deadline, gives the scheduler enough to make a reasonable choice, but the policy that turns requests into an order has not been settled or published.

The reason to say this plainly is that a scheduler's fairness policy is something a design partner would want to understand before trusting a backlog to it. Publishing the question before the answer is how the preview intends to work. The design partners page describes the kind of team the program is looking for when intake opens, and how it works shows where the queue sits in the sequence of a job.