Teaching Security Cameras to Describe What They See

Tue, 28 Jul 2026 00:00:00 +0000

A notification that says “person detected on doorbell” is useful. A notification that says “a delivery driver in a brown UPS uniform placed a medium-sized box on the porch and is walking back to the truck” is significantly more useful — especially when you’re in a meeting and trying to decide whether to get up.

I wanted that second kind of notification. The pieces were already in my homelab: Frigate NVR for camera management, a Coral USB TPU for real-time object detection, and a Mac Studio running llama-swap for LLM inference. The challenge was wiring them together so that every time Frigate detects a person or package, a vision model looks at the image and writes a one-sentence description.

Home-Automation on Rusty Bower

Teaching Security Cameras to Describe What They See