Every frame becomes a paragraph. Every paragraph becomes searchable.
A vision-language model writes one dense description per frame or segment: every vehicle with type, exact colour and plate, every person, actions, crash signs. That paragraph is the searchable document. Everything downstream (embeddings, rewriting, reranking) works on text, so the GPU-heavy step is paid once per frame, not once per query.
Writes one paragraph per frame or segment: cars, people, actions, incidents. This is the indexed document.
Dense vectors for documents and queries. The instruct prefix is applied to queries only, never to stored text.
Expands the operator's query with visual synonyms so "truck" also finds "lorry" and "HGV" descriptions.
Scores candidates against the original query and cuts at a threshold. This is what produces "nothing found" instead of the nearest-looking wrong answer.
20,000 recordings, 10 million indexed segments
| Operator query | Result | Score |
|---|---|---|
| "truck at a junction" | Correct clip | 0.997 |
| "road accident" | Correct clip | 0.991 |
| "crowd gathering" | Correct clip | 0.739 |
Operators query in their own language. Queries shown translated to English; the rewriter keeps the query language unchanged.
Kafka → S3 → VLM → Qdrant
Face, plate, situational-analytics, sabotage and abandoned-object events arrive on Kafka. The matching frames are pulled from S3, described and written to the events collection.
ffmpeg → segments → parallel describe / embed
Uploads are sampled at 1 fps, split into segments and processed in parallel.
Rewrite → embed → Qdrant → rerank → NMS
Every query pays for a rewrite, an embedding and a rerank: small text calls on a text model. The vision model is never in the request path.