Processing a backlog of footage

Cameras produce footage faster than anyone reviews it. A warehouse, a loading dock, a production line: each one records continuously, and the recordings pile up on storage until someone decides what to do with them. The illustrative request the program uses for this is "I have forty hours of warehouse camera footage that needs processing." The footage already exists. It is not going anywhere. It needs a model run over it once.

That is the shape E31 Network is being designed for. The footage is the data, the model pass is the work, the request is enough compute to get through the backlog in a reasonable window, and the output is whatever the pass extracts: detections, events, counts, clips, a report.

Why footage is a batch problem

A recorded video file has no urgency of its own. Nothing is waiting on frame-by-frame results in the moment, because the moment has passed. The question the footage answers is retrospective: what happened, how often, where, and when. A pass that runs tonight and reports in the morning answers it as well as a pass that runs now.

That makes footage one of the clearest cases of a workload that can wait for a queue. The job can be submitted when the backlog is ready, sit in the queue with the meter stopped, run when compute is free, and return its results. Compute is held only for the pass itself. The team does not need to keep GPUs around for the next backlog, because the next backlog can be its own job.

Describing the job

The work is the model or pipeline that runs over the footage: a detection model, a tracking step, a classifier for events of interest, whatever the team has built or chosen. The data is the footage itself, staged for the job. The request is how much compute, and whether the results are needed by a particular time or simply when done. The output is the structured result of the pass, and possibly a set of extracted clips.

Footage jobs split naturally. Each file, or each segment of a file, can be processed independently of the others, which means the pass can run across many GPUs at once if the team wants it fast, or across fewer for longer if it does not. The design intent is that the job declares which, and that the network schedules accordingly. How much the job must declare to be scheduled well is one of the open problems the program names.

Getting the footage to the compute

Footage is large. It is the workload where the program's first unsolved problem, getting large datasets to where the compute is, is most visible. A backlog measured in hours of video is a great deal of data to stage, and the pass cannot begin until it is staged.

The preview does not pretend this is finished. The design intent is that staging is a declared part of the job, that the team knows what it is moving and how, and that the time it takes is accounted for honestly. A team with a footage backlog that wants to be a design partner would be contributing to exactly this question. See /design-partners for what the program is looking for; intake is not yet open.

What the output looks like

The output of a footage job is almost never more footage. It is a smaller, structured artefact: a timeline of detected events, a table of counts per camera per interval, a set of short clips flagged for human review, a report. Returned to the submitter, it is what the team actually wanted from the recordings.

After the output is returned the compute is released and the hours it was held are accounted for. What happens to the staged footage after the job is a retention question whose specifics are not yet published. A team processing footage of people or of a sensitive site should treat that as an open item to raise, not as something already settled.

What stays on site

None of this applies to live camera feeds. A model watching a camera to stop a machine, to alert a supervisor in the moment, or to guide a robot through an aisle must answer now, and anything that must answer now stays on local infrastructure, at the edge, or on the customer's own systems. The network is batch AI infrastructure for the recordings. It is not a serving system for the floor.

The line is easy to draw with footage. If the frames are being produced right now and something acts on each one, it is local. If the frames were recorded earlier and are waiting on storage, they can be a job. See /examples#footage for the illustrative request, and Image processing at volume for the still-image version of the same work.