Photos in, catalogue out.
Each product is a folder with a few photos. The catalogue builder turns them into a search index, so adding a product never means collecting training data or retraining a model.
← All work
On-Prem AI-Enabled Smart Kiosk
Developed for a retail client: a vision system that learns a product from a few photos and logs every pick-up and put-back on the device.
Client: Retail technology company
Challenge
Unattended retail, from smart fridges and cabinets to open shelves, has to know what each shopper took. When recognition depends on a model trained for every product, each new item becomes a data-collection and retraining project, and the catalogue falls behind what is actually on the shelf. Operators also need shopper video to stay in the store.
Our approach
For this client we built a vision engine that recognises products by matching what the camera sees against a catalogue made from a few photos of each product. Nothing is trained: adding a product means dropping its photos in a folder and rebuilding the catalogue, which takes seconds to minutes. On the device, the engine detects the item, matches it with vector search, and tracks its direction across the shelf boundary to decide whether it was taken or put back. Guardrails stop a hesitating hand from creating duplicate events, and only product, action and session events are sent to the backend.
How it works
Video never leaves.
Detection, matching against the catalogue and direction tracking all run on the unit. Only three facts are sent on: which product, which action, which session.
It posts where you already work.
The cloud connector is a single function that builds and posts a payload. Swap it for your own API, message queue or point-of-sale system without touching the vision engine.
See it work in ten minutes.
This is the demo we run live, on your products if you send a few photos ahead. The drawing plays the part that happens in front of the camera.
3:00Live: the engine boxes and labels each product it knows.
Events sent to the backend
- Waiting…
- 0:00The catalogue
A folder with a few photos per product. That is all the data there is.
- 1:00Build
The catalogue builder runs and finishes in seconds to minutes, depending on catalogue size.
- 2:00Start a session
The session API starts and a demo session is opened.
- 3:00Go live
The vision engine starts, with direction arrows switched on.
- 4:00Pick up a product
You see the box, the label, the outgoing arrow and a remove event in the log.
- 5:00Put it back
The arrow reverses and an add event is logged.
- 6:00Hesitate
Wave the item at the boundary. Guardrails prevent duplicate events.
- 7:00Add a new product, live
Photograph a new product, drop the photos in a folder, rebuild and restart. It is recognised right away.
- 9:00The backend view
The session updates in the backend or the basket screen as events arrive.