What happens to my data after the job?
As designed, data is staged for the job, the job runs isolated, the output is returned, and the compute is released. Retention and deletion specifics are not yet published.
· 3 min read
IN SHORT
The design says this much: data is staged for the job, the job runs isolated from other jobs, the output is returned to the submitter, the compute is released, and what was held is accounted for. What it does not yet say is how long staged data persists after release or how deletion is confirmed. Those specifics are not published, and the program says so rather than guessing.
The honest answer has two halves. The first half is what the design states: your data is staged for the job, the job runs isolated from other jobs while it runs, the output is returned to you, the compute is released back to the pool, and what was held is accounted for. The second half is what the design does not yet state: how long staged data persists after release, how deletion happens, and how you would confirm it. Those specifics are not published. The program says so directly rather than filling the gap with reassurance.
What the design says
Follow the life of a job. At submit, you describe the data the job will read. At rent, compute is held and the data is staged for the job, meaning it is placed where that job can read it. At run, the job executes unattended, isolated from whatever else is running in the pool. At output, the results, whether a model, a set of processed files, or a report, are returned to the submitter. At release, the compute goes back to the pool, the meter stops, and what was held is written down.
That last clause is the accounting. The intent is a record of which compute was held for which job and for how long. It is the kind of thing an auditor could read, and it is also the basis of the bill, since you pay for rent and run only. The how it works page lays out the five steps.
What the design does not say yet
Retention. Staged data exists somewhere while the job runs. Whether it is removed at release, after a window, or on request is not published. Deletion. How removal would be performed and how a team would confirm it is not published. Output copies. Whether anything remains after the output is returned is not published.
The program is also candid that getting large datasets to where the compute is remains unsolved. A team with a large dataset to process once, which is one of the kinds of team the network is for, would want to know how it gets there and how it leaves. Both ends of that path are open.
There are no certification or compliance claims attached to any of this. A preview that has not shipped cannot honestly make them.
How to read the gap
A gap stated plainly is more useful than a policy written in advance of the system it governs. The retention and deletion rules, when published, would have to match what the system actually does, and the system is being built. Teams that need those answers before they can consider the network are exactly the teams whose questions would shape them; the design partners page describes that kind of engagement, with intake not yet open.
If your data cannot leave your own systems at all, the question shifts from the network to owning compute. That comparison is at how is E31 Network different from Czar, and the broader question of whether you need hardware is at do I need to own any hardware.
Questions this raises
- Can another job read my staged data?
- The design intent is that a job runs isolated from other jobs while it runs, and that staged data is staged for that job. How isolation is implemented is not published, and the program makes no security promise about it during the preview.
- Is the output stored on the network after it is returned?
- The design describes the output being returned to the submitter at the output step. Whether a copy persists afterwards, and for how long, falls under the retention specifics that are not yet published.
- Will there be a record I can show an auditor?
- The design says what was held is accounted for at release. The intent is a record of which compute was held for which job and for how long, something an auditor could read. Its exact contents are not published.
KEEP READING