How long can a job run?
No maximum is published. A job's request declares how long it needs, or that it runs until done, and when it must finish. Long jobs are expected; scheduling them fairly beside short ones is still open.
· 3 min read
IN SHORT
It depends on what the job declares, and no maximum is published. The request part of a job states how much compute it needs, for how long or until done, and when it must be finished. The design intent is for a job to run unattended for that span with the meter running. Scheduling long jobs fairly beside short ones is an open problem the program has named.
It depends on the job, and there is no published ceiling. When a job is submitted, its request declares three things about time: how much compute it needs, for how long or until done, and when it must be finished. The design intent is that the job then runs unattended for that span, with the meter running while compute is held and stopping at release. A training run that takes most of a night and a processing pass that takes a few minutes are both jobs; one simply holds compute for longer.
What the design has not settled is how those two kinds of job share a pool fairly. That is on the published list of things the program has not solved, and it is the real answer underneath this question.
What the request declares
Of the four parts of a job, the request is the one about time. "For how long" can be a span: "I need eight GPUs for four hours to process this dataset" is a request with a duration built in. It can also be "until done", which suits a training run whose length is known only roughly. And it carries a deadline, "when it must be finished", which is different from duration: a job might need a few hours of compute but not be needed until next week.
That last distinction matters for the queue. A job with a distant deadline can wait longer for compute to be free, and the meter does not run while it waits. A job declared "until done" with a near deadline is harder to place. The how it works page walks through the request alongside the other three parts.
Long jobs and short jobs in one pool
The scheduling problem is easy to state. Imagine a job that needs eight GPUs for four hours arriving beside a job that needs a few minutes. If the long job goes first, the short one waits far longer than its own length. If short jobs always go first, a steady stream of them starves the long one. Any policy that favors one kind penalizes the other, and the program has not published which trade it would make.
Exactly what a job must declare to be scheduled well is itself on the unsolved list. The request as described above is a starting point, and it may not be enough. A job that could say "I can be split" or "I can start late but must finish by then" would be easier to place than one that only says "four hours".
What this means in practice
For now, assume that a job can be long, that the meter would run for the whole of it, and that nothing is owned when it ends. Do not assume a maximum, because none is published. And do not assume the scheduling has been worked out, because the program says it has not.
If you have a job whose length is hard to pin down, that is useful information. The design partners page describes the kind of team the program would want to work through such cases with, and notes that intake is not yet open. Whether a job that cannot wait at all belongs here is a different question, answered at what if my job cannot wait.
Questions this raises
- What happens if my job needs more time than it requested?
- The design has not published how overruns would be handled, and it is an honest gap. A job declared as running until done avoids the question; a job declared for a fixed number of hours raises it. Expect the answer to arrive with the rest of the scheduling design.
- Can a long job be paused and resumed?
- Nothing like that is published. The life of a job as designed is submit, rent, run, output, release, with the run step unattended and continuous. Checkpointing inside the job is the submitter's business, as it is on any compute.
- Does a longer job cost more per hour?
- No rates are published at all, so no shape of rate is published either. The only mechanism described is by the hour for the hours compute is held, with the meter stopping at release.
KEEP READING