Building meaningful moments out of a family's photos
This picks up where ingestion leaves off. Every photo here has already had its rich data extracted — normalized, scored, placed, with its faces found. If you haven’t seen how a raw upload gets there, start with breaking each photo into pieces the system can understand →
Every moment is one small pipeline
It’s tempting to think a “trip” moment and a “first smile” moment are built by completely different machinery. They’re not. Each is the same three beats, with different things in each slot:
Trigger
What wakes the system up, and when. A nightly cron, a season closing, a backfill finishing, a photo finishing analysis.
Detect
Scan the photos for a candidate — a span of days, a recurring face, a milestone — that clears some thresholds. Persist it as a source row.
Curate
Keep the best photos, dedupe near-identical shots, then hand them — with all their context — to an LLM and let it arrange the titled, captioned story a parent sees.
Curation almost always ends the same way. A shareability filter keeps only the photos actually worth showing — sharp, well-composed, centered on the child, not the blurry or throwaway ones — dedupe caps near-identical shots so one burst can’t dominate, and an LLM arranges what’s left.
There are seven kinds of moments. What follows is one section per kind — its full pipeline, end to end — laid out in roughly the order a family meets them, from the day a child is born outward to the sweeping seasonal recaps.
Firsts & milestones
The first moment most families ever see. A first is a single milestone — Leo’s first smile, first steps, first solid food — pinned to one day and matched against a template.
Burst
Phones shoot in flurries. Ten near-identical frames of Leo laughing, all within a minute. A burst moment turns one of those rapid-fire sessions into a tiny story — and the only thing that defines it is time.
30-minute window into one burst.30-minute window they’re one burst; spread them past it and the burst splits. Each piece still needs ≥ 3 photos to become a moment.PhotoBurst.assign! drops every photo whose timestamp lands inside an existing burst's half-hour window into that same burst. Tight and rapid-fire is exactly what it's for.
Daily recap
The catch-all. At the end of a day with enough going on, a daily recap gathers everything the family shot — every child, the whole household — into one story of that single calendar day.
9–11 pm — so the recap lands in their evening, not at some server-clock 3 am.Daily recaps are the one family-wide kind: any child, anyone in the household.
Weekly recap
A gentler cadence, tied to the baby’s own clock. Every seven days since Leo was born, a weekly recap tells the story of his week.
9 pm local time for the family. It’s keyed to the birthday, not the calendar week: it fires when the child’s age in days is a clean multiple of seven.Because the window is anchored to the birthday, every family's weeks line up with their own child rather than the calendar.
Monthly recap
The same idea, one zoom level out. On each month-birthday — Leo turning one month, two months, three — a monthly recap gathers the whole age-month into one story.
9 pm local time for the family, on each age-month milestone.Weekly and monthly recaps are the steady heartbeat of the app for the first year.
Contiguous trip
Now the richest pipeline. A contiguous moment is a trip: a run of consecutive days the family spent away from home — the beach week, the visit to the grandparents. It’s the one kind that leans on where the photos were taken, and the one whose curation works day by day.
Detect: the trip detector
Detection classifies each day in a 21-day lookback window: as either away, home, silent if there are photos but no GPS data, or empty if there are none. A run of 3+ days becomes a candidate; 3 empty days in a row closes it.
A run of 3+ days away from home (50km+) is a trip. Silent days — photos with no GPS, like an airport — keep a run alive, but three empty days in a row close it. Try breaking the beach trip with a 'home' day in the middle.
Curate: how a trip is laid out
A trip is the one kind the LLM doesn’t arrange. After the shareable-keeper filter and the dedupe, the curator builds the structure itself, deterministically: it orders the surviving photos chronologically, drops in a text-only “Day N” slide at each day boundary, and interleaves that day’s photos beneath it. The LLM is called for just one thing — writing captions.
The dividers and ordering are the curator's, not the model's — the only thing the LLM contributes here is the captions, and only on the photos that need one.
Captions are reserved for what a photo can’t show on its own — never narrating a smile you can already see. And there’s a sibling detector for this same shape, a contiguous visit (one person, many consecutive days), that’s built but currently switched off.
Sparse visit
The subtlest kind. Some stories aren’t a block of days — they’re a face that keeps coming back. Nana doesn’t move in; she visits across the whole winter. A sparse moment catches that scattered pattern and tells it as one season-long story.
Detect: the visit detector
The detector works per season (or per year). For each visitor, it counts distinct days and total photos. Clear both thresholds and they’re a candidate.
Both gates green. A non-household face that recurs across enough distinct days, with enough photos, is a season-long story — even though the days are scattered, not consecutive.
What the detector produces is the candidate, not the moment — the curator is what turns it into the single season-long story a parent sees. And the visit detector is only the live half of the sparse family: its sibling, a sparse trip (a location that recurs across a season), is built but currently switched off, because its stories overlapped too much with the contiguous trip above.
The guardrails
Seven pipelines, all running over the same library, again and again. None of these guardrails are glamorous, but each one prevents a specific, real failure we’d otherwise ship:
The “≥3 photos or nothing” floor
Every recap-style moment needs at least three shareable photos to exist. A thin candidate is dropped, not padded with weak photos — the bar is “worth a parent’s attention,” not “technically possible.” (Firsts are the deliberate exception: one clear shot is the whole story.)
Idempotence by fingerprint
Source rows carry unique keys and curated moments are fingerprinted by their photo set. Re-running detection over the same library — which the nightly and post-backfill triggers absolutely will — produces no duplicates and no churn.
Per-family, per-timezone fairness
Detection is dispatched per family and per local timezone, on isolated jobs. One family’s huge backfill runs on its own queue and can’t delay another family’s nightly recap.
Seven kinds, one shape
Seven kinds of moment, and underneath they’re all the same shape: a trigger wakes a detector, the detector gathers the candidate photos, and a curator decides whether there’s enough there to be worth showing. Only what fills those slots changes from kind to kind. A first smile and a burst are decided together, per burst, the moment its photos finish analysis — one coordinator picking at most one of them. The daily, weekly and monthly recaps run on the clock, anchored to the child’s age. A trip is several days spent 50km+ from home. A recurring visitor surfaces from a non-household face that keeps showing up across a whole season. Same trigger → detect → curate skeleton every time — only the definition of “a moment worth keeping” differs.