Segun Olumbe

mobile · birds

Snap A Bird

Point the phone, get a species. An image model that knows 964 species runs on the phone, so it works on walks with no signal. Free, because I built it for my son Nathan. Public sightings go into an open Creative Commons dataset researchers can use.

  • Expo / React Native
  • TensorFlow Lite
  • VisionCamera
  • SQLite
  • Express
  • Prisma
  • PostgreSQL
  • Cloudflare R2
  • Claude API
snapabird.com ↗about ↗

Problem

Snap A Bird began with my son Nathan, who knows his birds. Most people see a small brown bird and think sparrow, when it could be a dunnock or a wren. I wanted an answer from one photo, on the walk, with no signal.

Constraints

  • Identification has to work with no connection, so the model ships inside the app.
  • A photo leaves the phone only when the user asks for a second opinion.
  • A guess under the confidence threshold is not shown.
  • Research data carries no names, emails, or photos, and only sightings the user made public.

Architecture

Offline first. Everything needed to name a bird and save the sighting is on the phone. The network is for a second opinion and for sharing.

  1. On the phone

    • CameraFrames from the camera, throttled and resized to the model's input.
    • ClassifierMobileNet V3 in TensorFlow Lite, quantised to about 12 MB, covering 964 species. No network call at inference.
    • Location filterIn the UK and Ireland, weighs each prediction by how likely that species is there and flags a rare sighting.
    • Local storeSightings saved to SQLite with species, confidence, place, and time.

  2. Second opinion

    • Cloud rescanFor a low-confidence result, or on request, the photo and the phone's top guess go to Claude Sonnet through the backend.
    • FallbackIf that call fails, the on-device answer stands.
    • Audit logEvery rescan is recorded with its source, status, and both confidence scores.

  3. Sync

    • APIExpress and Prisma on PostgreSQL. Sync is optional and started by the user.
    • Photo storageA saved sighting keeps its photo in Cloudflare R2.

  4. Open data

    • Research APIAnonymised public sightings as JSON or CSV, under CC BY 4.0.
    • Rare alertsAn uncommon or rare species triggers the alert pipeline.

Governance, written down

  • Model cardWhat each model is, what it is for, and what it cannot do.
  • DPIAThe data protection assessment for the AI processing.
  • AI Act assessmentHow the system is classified under the EU AI Act, and what follows from that.

Key decisions

  1. A model on the phone

    Instead of sending every photo to a server

    A bird does not wait for a signal. The classifier is bundled with the app and makes no network call, so identification works offline and the photo stays on the phone.

  2. A cloud model only as a second opinion

    Instead of identifying in the cloud first

    Image bytes leave the device only when a rescan is invoked. When that call fails the app falls back to the on-device answer, so the cloud can improve a result but cannot block one.

  3. Not showing a weak guess

    Instead of always naming a species

    Predictions under the confidence threshold are not surfaced. The model card is plain that the threshold is one global value, not calibrated per species.

  4. Stripping identity at the API

    Instead of publishing raw sightings

    Only sightings marked public are included. User ids, emails, display names, and photos are removed before anything reaches the research API.

What broke, and the fix

  1. The multi-model rescan was too slow for production.

    Cause
    The second opinion was planned as an ensemble of models with a fusion score. In production the latency was not acceptable.
    Fix
    Reverted to a single model on 17 April 2026. The fusion path stays disabled, and the model card says so.

Outcome

  • Shipping on iOS, at version 2.0.2.
  • The classifier is about 12 MB and covers 964 species, with a UK list of 170 used to weigh results.
  • Public sightings are available to researchers under CC BY 4.0.
  • Per-species accuracy has not been measured yet. The model card lists it as an open item.