Eyas: AI Security Camera Agent

GitHub HuggingFace Space Demo Video Field Notes Social Media

Competition

Hugging Face Build Small Hackathon — Backyard AI track. The challenge: build a practical tool for a real person you know using models with no more than 32 billion parameters, hosted as a Gradio app on a Hugging Face Space. Judging focuses on a specific real problem, evidence of actual use, honest small-model fit, and app polish.

Team: Javier Huang, Hanhee Lee, Joe Lee

What it does

Eyas is an on-device security camera agent built for our teammate’s family’s convenience store. It runs person tracking, event detection, and LLM reasoning over CCTV footage to surface theft, loitering, and suspicious activity as a structured, searchable log.

• Visual pipeline — person tracking → VLM observation → event structuring → LLM reasoning
• Event Timeline — scatter chart + table; click any event to seek the annotated video
• Summary & Alerts — risk gauge, flag breakdown, and per-camera narrative
• Ask Footage — natural-language Q&A over the event log via the on-device LLM
• Audio Report — spoken security brief via VoxCPM2 TTS
• Multi-camera — queue multiple clips, get a unified cross-camera session summary
• Korean — full UI and pipeline output translation, hot-swap without restart

Real-world context

Small retail owners have no affordable tool to automatically review CCTV footage for theft, loitering, and unusual activity. Manual review of 8-hour overnight recordings is impractical. Eyas was built for a teammate’s family, who runs a small Korean-owned convenience store — the tool runs on their existing laptop with no subscription, no cloud account, and no API keys.

We demoed on real four-camera aisle footage filmed at Joy Convenience Store (same layout and CCTV profile as the target store). The full pipeline — tracking, event log, bilingual summary, spoken audio brief — ran on that footage and produced reports the store operators could read immediately.

Architecture

The core pipeline chains five models end-to-end with no cloud APIs:

1. YOLO11n + BotSORT — detects and tracks people across frames, maintaining consistent track IDs through shelf occlusions
2. MiniCPM-V 4.6 — watches each tracked person and outputs structured JSON: activity, held objects, and whether a pickup occurred
3. Heuristic event structurer — merges per-track observations, applies keyword-based overrides, assigns zone labels
4. Nemotron 3 Nano 4B — reads the event log and produces risk-level summary, flags, suspicious clip list, and Q&A answers
5. TinyAya Global — translates all freeform VLM and LLM text to Korean on demand
6. VoxCPM2 — synthesizes a spoken audio security brief from the LLM summary

The UI is a custom React + Vite + MUI SPA. Gradio runs as a pure API backend with all native components hidden.

Models

All models download automatically on first run. No API keys required. Total loaded weight ~8.7B params / ~6 GB — runs fully on a laptop CPU; GPU optional.

Hackathon checklist

Live space: build-small-hackathon/eyas