Uploading a whole photo library, even after the app is closed
A new family points us at a camera roll with a thousand photos in it, and the and we need to make sure the camera roll is uploaded even after the user backgrounds the app, locks the phone, or swipes it away. And iOS treats each of those as a cue to start taking resources back: time first, then memory, then the whole process — until all that’s left is a tiny, memory-starved extension to finish overnight. This page is about how we ensure the full upload will finish in a reasonable amount of time:do the expensive work while you still can, so the bottom layer has nothing hard left to do.
Part 1
Why it's hard
Two problems at once: the phone keeps clawing resources back — less room at every layer — and the most expensive work is the one most likely to get a layer killed.
Three layers, split by job
Every photo needs two kinds of work: an on-device Vision pass (face detection + a baby classifier — GPU/ANE work) and an upload that lands the bytes on the server. Because the two kinds of work have very different risk profiles, the pipeline hands each to the layer best suited to it.
While the app is open, the foreground drain ships already-analyzed photos as fast as it can. The moment the user leaves, a dedicated background task takes over the one job that can’t survive being interrupted — the Vision analysis — and races to bank a verdict on every photo. And once the app is fully closed, a tiny extension uploads whatever’s left. Different jobs, three altitudes — and the lower you go, the less room you get. Pick a layer below and watch the same backlog move through each one — a torrent up top, a trickle once the app is gone.
Budget 3, upload-first. Three streams move at once and the box is at its widest, so the camera roll ships fast. By now the background task has usually banked the verdicts, so the foreground mostly just uploads. This is the most bandwidth the pipeline ever gets — and it only lasts while the app is open.
Each layer does a different job — the background task analyzes, the other two upload — but the squeeze is the same: less room each step down. Here’s the full picture, and every value is the real one from the code:
| Foreground | Background | App closed | |
|---|---|---|---|
| Who runs it | The photo ingestion drain, in-app | A Vision-only analyzer, kept alive by a BGContinuedProcessingTask |
A separate iOS extension |
| Bandwidth | 3 concurrent slots | 2 concurrent analyses | ~1 at a time · 40-photo pages |
| Network | Any — cellular allowed | Any — Vision isn't Wi-Fi-gated | iOS decides per job |
| What it does | Direct API request to the server | Analyzes only — banks a verdict on each row, never uploads | Enqueues a PhotoKit job; iOS transfers out-of-process |
| Time limit | None — runs while the app is open | Seconds — ~30s no-progress expiry | ~100 ms to launch; re-invoked as needed |
| What can kill it | Nothing; navigating away hands off to the next layer | OS expiry, a Stop tap, or resource pressure — but only Vision in flight is ever at risk | Jetsam — a tight memory cap |
Two of the three never run Vision: the foreground ships photos the background task already analyzed, and the closed-app extension does the same — except it doesn’t even POST bytes itself. It hands iOS a PhotoKit upload job and lets the OS perform the transfer out-of-process, on its own schedule — the only way to keep uploading after the app is gone. The background task is the inverse: it only analyzes, and never uploads at all.
Two kinds of work, very different costs
Every photo travels the same two stages. They look similar on a flowchart and behave nothing alike when the OS pulls the plug halfway through:
Analyze (Vision)
Face detection + the baby classifier on the CPU/ANE. Expensive and memory-hungry. This is the work that is most likely to get a layer killed due to its resource consumption.
Upload (network)
POST the bytes (or hand off a PhotoKit job). Cheap on CPU/memory, just slow and I/O-bound. This is the work that depends most on network connectivity, and as such, is the most likely to fail and need to be retried.
Part 2
How we solved it
The foreground runs a small upload-first loop — the drain. The backgrounded task runs a separate, even smaller loop that only analyzes. The closed-app extension is a stripped-down third that mostly just enqueues uploads. All three advance one shared UploadedAsset record the server dedupes, so they never trip over each other.
How the foreground drain splits its budget
Only the foreground runs a drain — a loop that, on every freed slot, calls a tiny pure function, planSlots, to decide how many of its concurrent slots go to analysis vs. upload. It flushes upload-first: by the time the app is in the foreground, the background Vision task has usually already banked the verdicts, so there’s a queue of analyzed photos ready to ship. Uploads take every slot until there’s none left; Vision runs only as the cleanup fallback for whatever wasn’t analyzed yet. The background Vision task, by contrast, runs no planSlots at all — just two analysis slots, flat out, banking verdicts. Toggle between them below:
planSlots(budget, online, hasVisionWork, hasUploadWork, bias) from the foreground drain, ported verbatim — running inside the loop that calls it, upload-first at budget 3. The right tab is the background BGContinuedProcessingTask: no planner, no budget, just 2 analysis slots that bank verdicts and never upload. Step one iteration at a time, or hit Play. Drop in photos mid-flush and watch each loop react.- 1Evaluate state
- 2
planSlots() - 3Fill slots
- 4Work completes
While any analyzed photo is waiting, every freed slot re-asks planSlots and gets the same answer — upload.
In the foreground, watch planSlots flush upload-first: banked photos ship until the upload queue is empty, and Vision only kicks in to clean up stragglers that weren’t analyzed yet. In the BGCPT there’s no allocator to watch — both slots just analyze, every photo ends banked, and nothing ever uploads. Nobody flips a mode; each loop simply does its one job.
The goal: arrive at the bottom layer with nothing hard left
This is the real reason Vision gets its own dedicated background task. The moment the app is backgrounded, that task does nothing but analyze — two slots, flat out — so it races through the analysis backlog and banks verdicts far faster than uploads could ever drain. So the bet is this: if the user swipes the app away after even a minute or two, we’ve probably already banked every verdict — and the overnight extension can finish the actual uploading on its own, cheaply, well inside the 24-hour window. A dedicated Vision task is what turns “they closed the app” from a failure into a non-event.
When the app closes and only the tiny extension is left, most photos already have a banked verdict. For those, the extension does the cheap thing: just create an upload job. It never has to run Vision.
How far Vision got before the app closed decides the extension’s fate. The extension can still analyze whatever’s left — it has a fallback path — but doing GPU/ANE work under its tight memory cap is exactly what puts it at risk of a jetsam kill. So the more the background task banked, the less the extension has to risk: a fully-analyzed backlog means it never has to run Vision at all.
This is why the bottom layer is mostly just a job-enqueuing mechanism. In the steady state the background Vision task has already paid the Vision bill, so the extension’s hot path is the cheap one: read the banked verdict, create a PhotoKit job, move on. The fallback that does analyze is deliberately throttled — one page at a time, bail early on low memory — so even the bad case degrades instead of crashing.
How the pipeline survives being killed
Every layer lives in a hostile environment: networks flake, iCloud returns the occasional 5xx, and — most of all — the OS can tear a background task down mid-flight, with almost no warning. Resilience here means treating every one of those as routine rather than as failure. The clearest case: the background Vision task will be killed. It does nothing but analyze — it never uploads — so a kill can only ever cost the Vision that was in flight; every verdict already banked persists on its UploadedAsset row, and the next wake resumes from exactly where it left off.
No uploads happen here at all; this task's only output is a banked verdict on every UploadedAsset row.
That same resilience shows up as a handful of guardrails that keep the three layers honest:
Rotate, then give up
A transiently-failing photo (slow iCloud, a 5xx blip) rotates to the back of the queue by its lastAttemptAt so it can’t stall the page. Only after a few legitimate failures does it get marked failed — generous enough that only a genuinely broken asset reaches it.
Policy deferrals are free
A send the user blocked by disabling cellular for the app, or one cut off by the task being torn down, returns deferred-uncounted: the row stays pending and no attempt is charged. A time-sliced or interrupted session can’t slowly fail out photos that just needed more time.
Complete is one-shot
The background task can finish on the normal path or via the OS expiration handler. Both funnel through a thread-safe latch so the task is completed exactly once — whoever fires first wins, the other is a no-op. No double-completion, no stranded UI.
Memory is the extension's clock
The extension paginates its workload, runs at most 1 analysis page per wake, and bails the instant headroom drops — then returns “more work remains” so iOS re-invokes it with a fresh footprint.
From firehose to fully synced
The whole strategy comes down to a handful of moves. Every photo needs two kinds of work — expensive on-device Vision and a cheap-but-network-bound upload — and the pipeline never treats them as interchangeable. Instead of running both everywhere, it splits them by layer: a dedicated background task does nothing but the expensive Vision, banking a verdict on every photo the moment the app is backgrounded, while the foreground and the always-on extension handle the cheap, retryable uploads. So by the time the app closes, the only thing left for the memory-capped extension is cheap enqueuing it can’t get killed for — which is exactly what keeps the up-to-24-hours promise. Four timestamps on a single UploadedAsset row are the only coordination needed, one rule keeps the layers from re-doing each other’s work, and every exit path is built to degrade rather than crash. Once the bytes land, the server takes over: it breaks each photo into pieces the system can understand →, then forks that legible photo into matching faces to the right child → and building meaningful moments →, in parallel.