Teaching Security Cameras to Describe What They See
A notification that says “person detected on doorbell” is useful. A notification that says “a delivery driver in a brown UPS uniform placed a medium-sized box on the porch and is walking back to the truck” is significantly more useful — especially when you’re in a meeting and trying to decide whether to get up. I wanted that second kind of notification. The pieces were already in my homelab: Frigate NVR for camera management, a Coral USB TPU for real-time object detection, and a Mac Studio running llama-swap for LLM inference. The challenge was wiring them together so that every time Frigate detects a person or package, a vision model looks at the image and writes a one-sentence description. ...