> ## Documentation Index
> Fetch the complete documentation index at: https://docs.moorcheh.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Visual Memory

> How to remember people and objects on Ventuno Q with dual Moorcheh Edge stores, live gate UI, and spoken shelf guidance.

**Visual Memory** gives edge devices real-time recognition for both people and objects. Enroll a photo of a face or an item, and the live camera stream matches it against local vector embeddings with **Moorcheh Edge** - triggering actions like unlocking a gate or announcing where a product belongs on a shelf. Everything runs on a single board with no cloud dependency, so biometric and visual data stay private.

**Demo video:** [Visual Agentic Memory on Edge AI Devices](https://youtu.be/HGj5Ky9hMFs)

Source repo: [github.com/moorcheh-ai/ShelfIn](https://github.com/moorcheh-ai/ShelfIn)

> [!NOTE]
>
> Install Docker on the board first. For general Edge install on Arduino-class hardware, see [Arduino UNO Q](https://docs.moorcheh.ai/guides/arduino-uno-q). Visual Memory runs **two** Edge containers (faces + objects), not the default single `:8080` store.

## Overview

**Visual Memory** runs on **Ventuno Q** with **Moorcheh Edge** as the on-device match layer.

A USB webcam watches the scene. The app does two jobs in parallel:

| Path | What it does |
| - | - |
| **Faces** | OpenCV **YuNet** finds faces, **SFace** builds 128-d embeddings, **Moorcheh Edge faces** (`:8083`) searches the enrolled gallery → **ACCESS GRANTED** or **ACCESS DENIED** |
| **Objects** | **YOLO** proposes boxes, **ResNet** builds 256-d embeddings, **Moorcheh Edge objects** (`:8081`) searches enrolled items. Only a **known** match gets a name, shelf, and description. Unknown boxes stay visual-only |
| **UI** | Browser dashboard on **:8501** (live stream + enroll) |

When an item is known, the app builds a short spoken line:

* **Template mode** (default) - fixed sentence from name + shelf
* **LLM mode** (optional) - an **external** chat LLM (Ollama or OpenAI-compatible) rewrites the same facts already in memory into one natural sentence

Speech uses **standalone Piper** when installed under `~/piper/`, with **espeak** as fallback. The browser never plays audio; the board speaker does.

Everything that matters for matching stays on the board: models, vectors, and decisions. Spoken guidance does **not** use Moorcheh `/answer` or a third Edge RAG store. The optional LLM receives **text facts only** (already matched name / shelf / description) - never the camera image, and never an unknown detection.

Powered by **Moorcheh** - PyPI `moorcheh-edge`, Docker `moorcheh/moorcheh-edge`.

## Why Moorcheh Edge

Moorcheh Edge is built for **on-device vector search**. Visual Memory keeps two Edge stores on the same board:

| Store | Port | Container | Vectors |
| - | - | - | - |
| Faces | **8083** | `moorcheh-edge-faces` | SFace 128-d |
| Objects | **8081** | `moorcheh-edge-objects` | ResNet→256-d |

On edge hardware this matters:

* Fast similarity search as galleries grow
* Stable scores you can threshold for GRANT / DENY and known / unknown items
* No cloud round-trip for identity or product match
* Same Edge runtime pattern used across Moorcheh on-device projects

YuNet, SFace, YOLO, and ResNet stay in the Python app. Edge is the match layer. Spoken lines use a template or an external LLM; Piper / espeak is the voice layer.

## Why edge access + shelf guidance?

* **Privacy** - embeddings and decisions stay on the device
* **Reliability** - matching does not depend on internet uptime
* **Latency** - detect, embed, and search run on the board
* **Control** - you decide thresholds and who / what is enrolled
* **Retail clarity** - known items speak a shelf line; unknowns are never invented by the LLM
* **Simple ops** - USB-C power, USB camera, browser UI after one-time setup

## Architecture

End-to-end path from camera to UI and speaker:

![Visual Memory system architecture](https://mintcdn.com/acme-991ed283/x7S3iqq12BcG9h6U/images/visual-memory/architecture-hub.png?fit=max\&auto=format\&n=x7S3iqq12BcG9h6U\&q=85\&s=14f3e76faf7c20c8975df94d14dabf93)

Visual Memory on Ventuno Q - faces and objects through dual Moorcheh Edge stores, then optional guidance and voice.

Diagrams (Figma): [VENTUNO Face Access - System Architecture](https://www.figma.com/design/4ZReJYcUDZaEkj22gzJerR/VENTUNO-Face-Access-%E2%80%94-System-Architecture)

| Port | Service |
| - | - |
| **8083** | Moorcheh Edge faces (SFace 128-d) |
| **8081** | Moorcheh Edge objects (ResNet 256-d) |
| **8501** | Visual Memory web UI |
| **11434** | Optional Ollama (only if `GUIDANCE_MODE=llm`) |

```text theme={null}
USB camera
    │
    ├─ YuNet → SFace → Edge :8083 → GRANT / DENY
    │
    └─ YOLO → ResNet → Edge :8081 → known item?
              │
              └─ known → template / external LLM → Piper / espeak
```

Unknown objects get a red **Unknown** box only. They are **not** sent to the LLM.

### Plain-language flow

1. The USB camera sends frames into Visual Memory.
2. The **face path** asks: “Do we know this person?” → grant or deny on the screen.
3. The **object path** asks: “Is this an enrolled item?” → if yes, use its name, shelf, and description.
4. If the item is known, build one short sentence (template or external LLM).
5. Piper / espeak speaks that sentence on the board speaker.
6. The browser shows the live stream, boxes, and status (Wi-Fi or HDMI).

### Technical flow (faces)

1. Frame (cadence every **3** frames, scale **0.35**)
2. YuNet detect → SFace embed (**128-d**)
3. Search Moorcheh Edge faces
4. Score ≥ `FACE_MATCH_THRESHOLD` (default **0.28**) → **ACCESS GRANTED** + name

### Technical flow (objects)

1. Same camera frame pushed to a side thread (every **3** frames)
2. YOLO proposes boxes (not names)
3. ResNet embed (**256-d**) on each crop
4. Search Moorcheh Edge objects (top **3**, margin **0.03**)
5. Score ≥ `OBJECT_MATCH_THRESHOLD` (default **0.38**) → known name + shelf + description
6. Else → **Unknown** (draw only)

### Technical flow (guidance + voice)

1. Only **known** object facts (and optional person name) enter guidance
2. `GUIDANCE_MODE=template` → fixed sentence from those facts
3. `GUIDANCE_MODE=llm` → external chat LLM with the facts in the prompt; fallback to template if the LLM is down
4. Standalone Piper (or espeak) speaks the line on the board speaker

## What runs on Ventuno Q

| Piece | Role |
| - | - |
| **Moorcheh Edge faces** (Docker) | Face vector store + search |
| **Moorcheh Edge objects** (Docker) | Object vector store + search |
| **Visual Memory app** (FastAPI) | Live gate UI, enroll APIs, MJPEG stream |
| **YuNet / SFace** | Face detect + 128-d embed |
| **YOLO / ResNet** | Object box propose (LiteRT `.tflite` preferred, Torch fallback) + 256-d embed |
| **External LLM** (optional) | Spoken-line rewrite from in-memory facts |
| **Piper / espeak** | On-device TTS to the speaker |
| **USB webcam** | Live frames (`CAMERA_BACKEND=opencv`, index `0`) |
| **Browser / HDMI** | GRANT / DENY, item boxes, onboarding |

Day-to-day: start faces Edge, start objects Edge, start the UI. Camera, thresholds, audio runtime, and Piper paths are set by `start.sh` defaults.

## Features

**Entrance / visitor**

* Live camera gate with ACCESS GRANTED / DENIED
* Identity label and similarity score
* Known item boxes + shelf line
* Unknown object boxes without inventing names
* Browser UI over Wi-Fi or HDMI

**Staff**

* Multi-photo person onboarding
* Item upload with name, shelf, description
* Bulk CLI enroll for larger sets

**Developers**

* Dual Moorcheh Edge stores
* Optional external LLM guidance + Piper / espeak TTS
* Scripts: `start_faces_edge.sh`, `start_objects_edge.sh`, `start.sh`, `scripts/enroll_faces.py`, `scripts/enroll_objects.py`

## Hardware requirements

| Need | Why |
| - | - |
| **Ventuno Q** | Ubuntu / ARM Linux, Docker capable |
| **USB webcam** | Live gate (`CAMERA_BACKEND=opencv`, index `0` by default) |
| **USB sound + speaker** | Piper / espeak |
| **USB-C power** | Board power |
| **Network** | First-time clone / models; UI at `http://<board-ip>:8501/` |
| Optional HDMI / SSH | Local development; USB-C to PC for first SSH (Wi-Fi SSH works too) |

## Software stack

| Layer | Choice |
| - | - |
| Face detect | OpenCV YuNet (ONNX) |
| Face embed | OpenCV SFace, 128-d (ONNX) |
| Object propose | Ultralytics YOLOv8n |
| Object embed | ResNet18 → 256-d |
| Vector DB + search | Moorcheh Edge (Docker) × 2 |
| Spoken rewrite (optional) | External chat LLM (Ollama / OpenAI-compatible) |
| TTS | Standalone Piper or espeak |
| App / UI | Python FastAPI + static web UI |
| Camera | OpenCV VideoCapture (V4L2) |

## Repo layout

```text theme={null}
ShelfIn/                 # GitHub repo folder name
  access_app/            # FastAPI dashboard + enroll APIs
  core/                  # camera, face/object embed, TTS helpers
  scripts/               # bulk enroll, model download, LiteRT export
  known_faces/           # local photos (gitignored)
  known_objects/         # local photos (gitignored)
  models/                # YuNet / SFace ONNX
  start.sh
  start_faces_edge.sh
  start_objects_edge.sh
```

## How to: one-time setup

1. Clone [github.com/moorcheh-ai/ShelfIn](https://github.com/moorcheh-ai/ShelfIn)
2. Install Docker; add your user to the `docker` group
3. Create a venv and install deps
4. Download face models; place `yolov8n.pt` in the project root
5. Optional: install Piper under `~/piper/piper/` (otherwise use espeak)
6. Follow the repo **README.md** for any board-specific notes

```bash theme={null}
cd ~
git clone https://github.com/moorcheh-ai/ShelfIn.git
cd ShelfIn

sudo apt update && sudo apt install -y docker.io
sudo usermod -aG docker "$USER"   # log out/in
sudo systemctl enable --now docker

python3 -m venv faceenv
source faceenv/bin/activate
pip install -U pip
pip install -r requirements-retail.txt

bash ./scripts/download_models.sh
# place yolov8n.pt in the project root
```

Optional Piper paths (auto-detected by `start.sh`):

```text theme={null}
~/piper/piper/piper
~/piper/piper/en_US-lessac-medium.onnx
```

Otherwise install `espeak-ng` for spoken lines.

## How to: every boot

`start.sh` already sets camera (`opencv` / index `0`), object thresholds (`0.38` / `0.55` / `0.03`), `XDG_RUNTIME_DIR`, and Piper when installed. No exports required for the default demo.

```bash theme={null}
cd ~/ShelfIn
source faceenv/bin/activate
bash ./start_faces_edge.sh
bash ./start_objects_edge.sh
bash ./start.sh
```

Open:

* On the board: **[http://127.0.0.1:8501/](http://127.0.0.1:8501/)**
* From another PC: **http\://\<board-ip>:8501/**

Enroll at least one person and one item before relying on live recognition.

## How to: enroll faces and items

Staff enroll people and shelf items once. After that, live recognition can grant access and speak shelf lines.

![Enroll faces and objects](https://mintcdn.com/acme-991ed283/x7S3iqq12BcG9h6U/images/visual-memory/enroll-faces.png?fit=max\&auto=format\&n=x7S3iqq12BcG9h6U\&q=85\&s=7abd574e62d5a3f881b5e4bd3de933ec)

Enroll faces and objects into the two Moorcheh Edge stores.

### Enroll a person (UI)

1. Open the Visual Memory UI
2. Under **Onboard a person**, enter a display name
3. Drop or choose several photos (JPG / PNG)
4. Click **Insert & authorize**
5. Vectors upload to **Moorcheh Edge faces** (`:8083`)

### Enroll an item (UI)

1. Under **Add an item**, enter name, shelf phrase, and optional description
2. Drop or choose item photos
3. Click **Add item**
4. Vectors upload to **Moorcheh Edge objects** (`:8081`)

### Bulk enroll (backend scripts)

For larger galleries, skip the UI and run scripts with Edge up:

```bash theme={null}
# Photos under known_faces/<Name>/
python scripts/enroll_faces.py

# Photos + shelf.txt + description.txt under known_objects/<Item>/
python scripts/enroll_objects.py
```

Face and object images stay on the device (`known_faces/` and `known_objects/` are gitignored).

### Enroll checklist

1. Faces Edge up (`start_faces_edge.sh`)
2. Objects Edge up (`start_objects_edge.sh`)
3. Visual Memory UI running (`start.sh`) - or run the CLI enroll scripts with Edge up
4. At least one person and one item enrolled
5. Confirm both lists before relying on live recognition

Clear a store if dimensions conflict:

```bash theme={null}
moorcheh-edge clear-store -y --base-url http://localhost:8083
moorcheh-edge clear-store -y --base-url http://localhost:8081
```

## How to: live recognition

![Live recognition](https://mintcdn.com/acme-991ed283/x7S3iqq12BcG9h6U/images/visual-memory/live-recognition.png?fit=max\&auto=format\&n=x7S3iqq12BcG9h6U\&q=85\&s=174dc5e5d989850ea184b03b8f992976)

Live path: known face grants access; known item names its shelf and can be spoken.

1. USB camera frame arrives
2. **Face path** - YuNet → SFace → Edge faces → **ACCESS GRANTED** or **ACCESS DENIED**
3. **Object path** - YOLO → ResNet → Edge objects → known item or **Unknown**
4. If known → guidance sentence → Piper / espeak on the speaker
5. UI shows boxes, scores, and gate status

Face grant / deny never depends on the LLM. Unknown objects never go to the LLM.

### Defaults (no exports required)

| Knob | Default |
| - | - |
| Camera backend | `opencv` |
| Camera index | `0` |
| Face match | `0.28` |
| Object match | `0.38` |
| Object fallback | `0.55` |
| Object margin | `0.03` |
| Guidance | `template` |
| Piper | auto if `~/piper/piper/...` exists |

### Optional overrides

Only if you need them:

```bash theme={null}
# export CAMERA_INDEX=2
# export GUIDANCE_MODE=llm
# export PIPER_BIN=~/piper/piper/piper
# export PIPER_MODEL=~/piper/piper/en_US-lessac-medium.onnx
```

**External LLM guidance** (default is template - no LLM):

```bash theme={null}
export GUIDANCE_MODE=llm
export LLM_BASE_URL=http://127.0.0.1:11434
export LLM_MODEL=qwen2.5:0.5b-instruct
bash ./start.sh
```

**LiteRT YOLO** (export on Mac/x86 only, not aarch64):

```bash theme={null}
python scripts/export_yolo_litert.py
# copy yolov8n.tflite to the board project root
```

## How matching works

**Faces:** frame → YuNet → SFace **128-d** → Edge `:8083` → GRANT / DENY

**Objects:** frame → YOLO box → ResNet **256-d** → Edge `:8081` → known name or Unknown

**Guidance:** known facts only → template or external LLM → Piper / espeak

## Use cases

* Door / lobby face access on Ventuno Q
* Retail or warehouse shelf guidance for enrolled products
* Privacy-first demos without cloud face APIs
* Edge AI proof-of-concept before a fixed install

## Links

* Visual Memory source - [https://github.com/moorcheh-ai/ShelfIn](https://github.com/moorcheh-ai/ShelfIn)
* Demo video - [https://youtu.be/HGj5Ky9hMFs](https://youtu.be/HGj5Ky9hMFs)
* Moorcheh Edge - [https://github.com/moorcheh-ai/moorcheh-edge](https://github.com/moorcheh-ai/moorcheh-edge)
* Documentation - [https://docs.moorcheh.ai/on-edge/](https://docs.moorcheh.ai/on-edge/)
* PyPI - [https://pypi.org/project/moorcheh-edge/](https://pypi.org/project/moorcheh-edge/)
* Docker Hub - [https://hub.docker.com/r/moorcheh/moorcheh-edge](https://hub.docker.com/r/moorcheh/moorcheh-edge)
* Architecture diagrams (Figma) - [https://www.figma.com/design/4ZReJYcUDZaEkj22gzJerR/VENTUNO-Face-Access-%E2%80%94-System-Architecture](https://www.figma.com/design/4ZReJYcUDZaEkj22gzJerR/VENTUNO-Face-Access-%E2%80%94-System-Architecture)
* Moorcheh - [https://moorcheh.ai](https://moorcheh.ai)

## Next steps

> [card] [Arduino UNO Q](https://docs.moorcheh.ai/guides/arduino-uno-q)
>
> Board Docker + moorcheh-edge basics

> [card] [Upload vectors](https://docs.moorcheh.ai/cli/upload-vectors)
>
> Precomputed vector upload (same mode Visual Memory uses)

> [card] [Retail Kiosk](https://docs.moorcheh.ai/examples/retail-kiosk)
>
> Text + voice RAG kiosk on UNO Q

> [card] [Source on GitHub](https://github.com/moorcheh-ai/ShelfIn)
>
> Source repo and README


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.