Tutorial: Self-Tagging Image Gallery
A gallery that tags itself. You upload a photo, and by the time you refresh, it has a caption, five tags, and a place in vector space, so the gallery answers searches like “misty mountain ruins” even though nobody ever typed those words. The whole loop is platform primitives: file storage in S3 mode, a collection with a trigger, a plugin that runs a local vision model, and vector embeddings.
This page explains the mechanics. The complete, runnable project (docker compose stack with MinIO, IaC, the plugin, seed and search scripts, and a small live gallery UI) lives at vski.sh/x/example_gallery_tagger; its README has the full walkthrough.
flowchart TD U["upload<br/>(multipart POST)"] --> P["photos collection"] P -->|"object stored"| B[("S3 bucket<br/>photos/{recordId}/{filename}")] P -->|"trigger on update"| T["tagger plugin<br/>plugin://tagger/on_photo"] T -->|"1. download the image via /files"| B T -->|"2. caption and tags, 3. embed the caption"| O["Ollama<br/>vision + embed models"] T -->|"4. write caption, tags, state<br/>and upsert the caption vector"| P SR["semantic search"] --> PFiles live in the bucket
Section titled “Files live in the bucket”With S3_BUCKET set, the server stores every uploaded file as an object and never touches local disk:
S3_ENDPOINT=http://minio:9000 # host[:port]; empty means AWS S3S3_BUCKET=galleryS3_ACCESS_KEY=... S3_SECRET_KEY=...S3_FORCE_PATH_STYLE=true # for MinIO and most self-hosted S3Objects are keyed {collection}/{recordId}/{filename}, the same layout local storage uses, so the two modes read the same way. The download route GET /files/{collection}/{recordId}/{filename} streams objects straight from the bucket, and deleting a record deletes its objects. Replicas with the same S3 settings share the bucket, which is why the local file-sync journal turns itself off in S3 mode.
The upload, and why the trigger listens on update
Section titled “The upload, and why the trigger listens on update”An upload is one multipart POST to the normal record endpoint. Form values become record fields; each file part is stored under its field name, and the stored filename lands in that field:
record = await client.collection("photos").create( {"title": "Harbor morning", "state": "queued"}, files={"image": ("harbor.jpg", bytes, "image/jpeg")},)# record["image"] == "harbor.jpg"A file field can be required: the filenames from the file parts are merged into the record before validation, not after.
The multipart save publishes two record events. The create event fires before the bytes are stored; the update event, which the handler publishes after saving the files and writing the filenames into the record, arrives with everything in place. So the trigger listens on update:
photos: type: base fields: - {name: title, type: text, required: true} - {name: image, type: file, required: true} - {name: caption, type: text} - {name: tags, type: json} - {name: state, type: text} # queued -> tagged | failed - {name: taggedFile, type: text} # loop guard, see below - {name: error, type: text} rules: list: "'viewer' in @request.auth.roles" # the gallery UI reads view: "'viewer' in @request.auth.roles" update: "'plugin' in @request.auth.roles" # only the plugin writes back options: triggers: - action: update url: "plugin://tagger/on_photo" method: POSTEvery save of every photo calls the plugin. The interesting engineering question is how to do that without a loop.
The plugin: a trigger loop that cannot loop
Section titled “The plugin: a trigger loop that cannot loop”The plugin’s write-back (caption, tags, state) is itself an update, which fires the same trigger again. The guard is one field: taggedFile records which file the current tags describe.
@app.function("on_photo")async def on_photo(ctx, params): record = params or {} # the trigger payload is the record image = record.get("image") if not image: return {"skipped": "no file"} if record.get("taggedFile") == image: # our own write-back, or a title edit return {"skipped": "already tagged"}
asyncio.create_task(process(ctx, record)) # slow work, off the trigger's clock return {"queued": True}The guard has exactly the right failure semantics. A title edit skips. The plugin’s own write-back skips. A failed attempt records state: failed and still sets taggedFile, so the failure does not loop either. Replacing the image changes image, the guard no longer matches, and the new file is processed. Re-running on old records is one script away (make retag in the repo).
The background task is not decoration: trigger invocations are capped (30 seconds by default), and a local vision model needs most of a minute per photo. So the function acknowledges immediately and works behind a semaphore (one photo at a time, to keep the machine running Ollama responsive). The task downloads the image through the public /files route, asks the vision model for JSON, and writes everything back in one update.
From captions to semantic search
Section titled “From captions to semantic search”The last step of the task embeds the caption and upserts it against the record:
vector = await embed_text(ollama_url, "nomic-embed-text", f"{title}. {caption}")await ctx.client.embeddings.upsert("photos", record_id, vector, model="nomic-embed-text")The first tagged write pins the collection’s embedding model and dimensions, so searches must embed with the same model. The repo’s make search q="boat in a harbor" does exactly that: embed the query, vector-search the collection, print ranked results with file URLs.
Run it
Section titled “Run it”cp .env.example .envmake up # MinIO + vski (S3 mode) via docker composemake setup # bucket, first admin, admin + viewer API keysmake iac-apply # gallery db, photos collection, tagger instancemake seed # 12 photos; captions arrive about one per minutemake ui # live gallery at http://127.0.0.1:8090Browse the bucket at http://127.0.0.1:9001 while the queue drains to watch objects appear the moment photos upload.
Where to go next
Section titled “Where to go next”- Triggers for delivery semantics, retries, and the webhook log
- Plugins for the manifest, config, and the warm process model
- Vector Embeddings for model pinning and similarity search
- File Sync for how local storage and replication differ from S3 mode