Isolation while a job runs

Of the five steps in the life of a job, RUN is the one the team is least present for. The job has been submitted. Compute has been held and the meter has started. Now the work runs, unattended, until it finishes. The property that makes unattended running acceptable is that the job runs isolated from every other job in the pool.

Isolation is a word that carries a lot of weight in infrastructure, and it is worth being precise about what E31 Network means by it, what the design is meant to provide, and where the program is deliberately not making claims.

The three things that are isolated

In this design, isolation has three parts, and they line up with three of the four things a job declares.

The first is the data. A job reads the data staged for it and nothing else. Another job running at the same time reads its own staged data. The staged inputs of one job are not something another job can read.

The second is the compute. The rent step holds compute for the job. While the job runs, that held compute is the job's. Another job does not share it; another job's work does not run on it. When the job finishes, the compute is released, and only then is it available to be held for something else.

The third is the output. What the job produces is returned to the submitter. It is not pooled with the results of other jobs or visible to them. The output of the retraining run goes to the team that submitted the retraining run.

Put together, a job is a closed boundary while it runs: its data in, its compute working, its output out, with no other job inside that boundary.

Why the design needs it

Unattended running is the whole reason to rent compute for batch work. A team that had to supervise a job for its entire duration would gain little over running it on its own hardware. The point is to describe the job, let it run tonight, and collect the output in the morning.

That only works if the team can trust the boundary without watching it. Isolation is the property that stands in for supervision. If a job can only read its own data, only use its own compute, and only produce its own output, then nothing another team does with the pool at the same time affects the job, and nothing the job does leaks into another team's work.

Isolation also keeps the accounting simple. If held compute belongs to exactly one job at a time, then what was held for that job is a clean quantity. The meter runs against it from the rent step to the release step, and the record at the end describes one job's hold, not a share of something tangled.

What isolation does not mean here

The program is careful about what it claims, and this is a place where care matters.

Isolation is design intent. The network is a technical preview, and nothing has shipped. Saying the design keeps jobs isolated describes how it is meant to work, not how it has been shown to work under load with real jobs.

Isolation is not a security certification or a compliance claim. The program makes none. An engineer can read the design and judge it, and a record of what was held is being designed to be something an auditor could read, but no external attestation is being asserted.

Isolation is not a statement about the physical layout of the compute. Whether a held allocation is a whole machine, several machines, or a partition is an engineering question the preview is working through. The design describes the boundary around the job, not the hardware under it.

And isolation does not extend past the job. When the job ends, the compute is released into the pool and will be held for someone else. What happens to the staged data and the returned output after that point is a retention question, and the retention specifics are not yet published.

Isolation in the life of a job

Trace it through. At SUBMIT, the team declares the work, the data, the request, and the output. At RENT, compute is held for the job and the meter starts; from this point the held compute is the job's alone. At RUN, the job reads its staged data on its held compute, and no other job crosses that line. At OUTPUT, the results are returned to the submitter and to no one else. At RELEASE, the compute goes back to the pool, the meter stops, and what was held is accounted for.

Isolation lives in the middle of that sequence. It begins when compute is held and ends when it is released, and it is the reason the team can be absent for everything in between. The how it works page lays out the five steps; the article on what is accounted for when a job ends picks up where isolation leaves off.