Batch workload
A body of compute work submitted as a whole, run without anyone interacting with it, and finished with a result at the end. The natural fit for rented compute.
· 2 min read
IN SHORT
A batch workload is compute work submitted as a whole, run unattended, and completed with a result at the end. It needs no person in the loop and no answer within a fixed moment, which is what lets it wait for a queue and run on shared compute. E31 Network is being built for batch and asynchronous AI work such as training, footage processing, and simulation.
A batch workload is a body of compute work that is submitted as a whole, runs without anyone interacting with it, and produces its result at the end. It is defined by what it does not need: a person waiting on each step, or an answer within a fixed moment.
Why it matters
Batch work is the natural fit for rented compute. Because nothing is waiting on an immediate response, a batch job can be scheduled when compute is free, run without pauses for input, and finish whenever it finishes. That flexibility is what lets a shared pool serve many teams without each team owning capacity.
It is also where most of the heavy compute in applied AI lives. Training, fine-tuning, and retraining are batch by nature. So are processing a backlog of footage, running an evaluation suite, generating synthetic data, transforming a large dataset, and sweeping a large set of simulations. These jobs are large, occasional, and tolerant of a wait, which is the opposite profile from the live systems they feed.
How it works on E31 Network
E31 Network is being built for batch and asynchronous workloads specifically. The workloads that belong on it include batch inference, model training, fine-tuning, retraining, video processing, image processing, large dataset processing, simulation, rendering, synthetic data generation, evaluation jobs, asynchronous large-model inference, data transformation, and analytics, along with other GPU-heavy jobs that can run unattended.
A batch workload would be submitted as a job: the team describes the work, the data it reads, the compute it needs, and the output it expects. If compute is free, the job starts; if not, it waits in a queue, and the meter does not run while it waits. Once running, the job is isolated from other jobs, and when it completes, its output is returned and the compute is released.
The design assumes the submitter is not watching. A batch job on the network should be describable in a sentence of the form "I have forty hours of warehouse camera footage that needs processing" and then left alone until the output arrives. Anything that needs a person in the loop while it runs, or an answer the moment it is asked, is not a batch workload and does not belong here. The workloads page lists the categories in more detail.
Related terms
Questions this raises
- Is batch inference really batch if the model is a chat model?
- Yes, if the prompts are submitted together and the completions come back as a file. What makes it batch is that nobody is waiting on each reply. The same model serving a person in a live conversation is a real-time workload and stays on the team's own systems.
- How large does a batch have to be to be worth submitting?
- The design does not set a minimum, and no sizing rules are published. The practical test is whether the work can be described as a job with a defined output and left alone until it finishes. A single retraining run or a single pass over a dataset qualifies.
- Can a batch workload be split into several jobs?
- Nothing in the design prevents it, and a sweep of simulations or a backlog of footage might be submitted as one job or several. How mixed job sizes are scheduled fairly is one of the open problems the program lists, so this is an area where the design is still being worked out.
KEEP READING