The life of a job, in five steps

Every job on E31 Network is meant to pass through the same five steps, in the same order: submit, rent, run, output, release. The steps are the whole product. There is no account to keep warm between jobs, no machine to maintain after the job ends, and no step where compute sits allocated with nothing to do. This article walks through each step as it is being designed, and notes where the design is still open.

The network is a technical preview. No job has been submitted yet, and the sequence below describes the intent rather than something you can try today.

Submit

A job begins as a description. The submitter states the work (a training run, a batch of inference, a processing pass over footage, a set of simulations), the data the work reads, the request for compute, and the output that should come back. Those four parts are what the scheduler would use to decide when and where the job runs. A thin description makes a job hard to place; a precise one makes it easy. The companion article on what a job declares goes through each part in detail.

At this step, no compute is held and the meter has not started. A submitted job is a request, not a rental.

Rent

Once compute is free that matches the request, it is held for the job. This is the moment the meter starts. Holding compute means it is set aside for this job and no other, for as long as the job needs it or until the limit in the request is reached.

If nothing suitable is free, the job waits in a queue. Waiting is not renting. The meter does not run while a job is queued, which is why the design treats the queue as a feature rather than a failure.

Run

The job runs without anyone watching it. The staged data is read, the work proceeds, and the job is kept apart from other jobs sharing the pool while it runs. The submitter does not need to remain connected. The design assumes the common case is a job that starts in the evening and is finished by morning.

The meter continues through this step. Rent and run are the two steps you pay for. The piece on running unattended covers what the job is expected to do on its own and what the open questions are when it does not go to plan.

Output

When the work completes, the result is returned to the submitter. The result is whatever the job description said it would be: a model, a set of results, processed files, a report. The design intent is that the output arrives as a defined artifact, not as a machine you have to log in to and copy from.

How the output is packaged and how long anything is retained are not yet published. The article on how output comes back separates what is decided from what is not.

Release

With the output returned, the compute is released back to the pool and the meter stops. Nothing is owned. Nothing is left running. What was held is accounted for, so the submitter can see what was rented and for how long.

Release is also the step that makes the economics work. Compute that is released is available to the next job, which is what lets a team rent heavily for one night without paying for the machine the rest of the month. The article on release treats this step on its own.

The sequence is settled. The details inside each step are not. Getting large datasets to where the compute is remains an open problem for the submit and rent steps. Scheduling mixed job sizes fairly, so that a long job needing many GPUs and a short job needing few both get a reasonable turn, is open for rent. Exactly what a job must declare to be scheduled well cuts across all of it.

The preview publishes these as unsolved on purpose. If the job model above fits the way your team already thinks about its work, how it works lays out the same sequence in brief and the examples show it applied to four illustrative requests.