Add Apple Silicon ZMQ detector for Frigate #206

Merged
eblume merged 7 commits from feature/frigate-zmq-detector into main 2026-02-17 19:03:29 -08:00
8 changed files with 106 additions and 10 deletions
Showing only changes of commit dc4d35024f - Show all commits

Add Apple Silicon ZMQ detector for Frigate

Moves object detection from ONNX CPU (~117ms/frame) to the
apple-silicon-detector running natively on indri via CoreML/Neural Engine
(~15ms), communicating with Frigate over ZMQ (tcp://host.minikube.internal:5555).

- New ansible role `frigate_detector` with LaunchAgent
- Switch Frigate configmap from ONNX to ZMQ detector
- Remove detect FPS cap (no longer needed with fast inference)
- Update docs and add changelog fragment

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Erich Blume 2026-02-17 17:11:35 -08:00

View file

@ -175,3 +175,5 @@
tags: jellyfin_metrics
- role: caddy
tags: caddy
- role: frigate_detector
tags: frigate_detector

View file

@ -0,0 +1,6 @@
---
frigate_detector_repo: https://github.com/frigate-nvr/apple-silicon-detector.git
frigate_detector_dir: "{{ ansible_env.HOME }}/code/3rd/apple-silicon-detector"
frigate_detector_endpoint: "tcp://*:5555"
frigate_detector_model: AUTO
frigate_detector_log_dir: "{{ ansible_env.HOME }}/Library/Logs"

View file

@ -0,0 +1,6 @@
---
- name: Restart frigate-detector
ansible.builtin.shell: |
launchctl unload ~/Library/LaunchAgents/mcquack.eblume.frigate-detector.plist 2>/dev/null || true
launchctl load ~/Library/LaunchAgents/mcquack.eblume.frigate-detector.plist
changed_when: true

View file

@ -0,0 +1,48 @@
---
# Apple Silicon ZMQ detector for Frigate
# Runs natively on macOS, using CoreML / Neural Engine for ~15ms inference.
# Communicates with Frigate via ZMQ over TCP.
#
# ONE-TIME SETUP (before running ansible):
#
# 1. Clone the repo:
# ssh indri 'git clone https://github.com/frigate-nvr/apple-silicon-detector.git ~/code/3rd/apple-silicon-detector'
#
# 2. Install dependencies:
# ssh indri 'cd ~/code/3rd/apple-silicon-detector && make install'
#
# 3. Run ansible to deploy LaunchAgent:
# mise run provision-indri -- --tags frigate_detector
- name: Verify apple-silicon-detector repo exists
ansible.builtin.stat:
path: "{{ frigate_detector_dir }}/Makefile"
register: frigate_detector_stat
- name: Fail if apple-silicon-detector not found
ansible.builtin.fail:
msg: |
apple-silicon-detector not found at {{ frigate_detector_dir }}.
Please clone and install first:
ssh indri 'git clone {{ frigate_detector_repo }} {{ frigate_detector_dir }}'
ssh indri 'cd {{ frigate_detector_dir }} && make install'
when: not frigate_detector_stat.stat.exists
- name: Deploy frigate-detector LaunchAgent plist
ansible.builtin.template:
src: mcquack.eblume.frigate-detector.plist.j2
dest: ~/Library/LaunchAgents/mcquack.eblume.frigate-detector.plist
mode: '0644'
notify: Restart frigate-detector
- name: Check if frigate-detector LaunchAgent is loaded
ansible.builtin.command: launchctl list mcquack.eblume.frigate-detector
register: frigate_detector_launchctl_check
changed_when: false
failed_when: false
- name: Load frigate-detector LaunchAgent if not loaded
ansible.builtin.command: launchctl load ~/Library/LaunchAgents/mcquack.eblume.frigate-detector.plist
when: frigate_detector_launchctl_check.rc != 0
changed_when: true
failed_when: false

View file

@ -0,0 +1,31 @@
<?xml version="1.0" encoding="UTF-8"?>
<!-- {{ ansible_managed }} -->
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>mcquack.eblume.frigate-detector</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/make</string>
<string>run</string>
</array>
<key>WorkingDirectory</key>
<string>{{ frigate_detector_dir }}</string>
<key>EnvironmentVariables</key>
<dict>
<key>ENDPOINT</key>
<string>{{ frigate_detector_endpoint }}</string>
<key>MODEL</key>
<string>{{ frigate_detector_model }}</string>
</dict>
<key>RunAtLoad</key>
<true/>
<key>KeepAlive</key>
<true/>
<key>StandardOutPath</key>
<string>{{ frigate_detector_log_dir }}/mcquack.frigate-detector.out.log</string>
<key>StandardErrorPath</key>
<string>{{ frigate_detector_log_dir }}/mcquack.frigate-detector.err.log</string>
</dict>
</plist>

View file

@ -30,7 +30,6 @@ data:
roles: [detect]
detect:
enabled: true
fps: 2
stationary:
max_frames:
default: 1500
@ -51,8 +50,9 @@ data:
track: [person, car, dog, cat, bird]
detectors:
onnx:
type: onnx
apple_silicon:
type: zmq
endpoint: tcp://host.minikube.internal:5555
model:
model_type: yolonas

View file

@ -0,0 +1 @@
Add Apple Silicon ZMQ detector for Frigate — inference moves from ONNX CPU (~117ms) to CoreML/Neural Engine (~15ms)

View file

@ -27,12 +27,14 @@ Open-source network video recorder (NVR) with object detection. Runs cloud-free
ReoLink Camera (GableCam)
│ RTSP
Frigate pod
├── go2rtc — RTSP restream proxy
├── FFmpeg — stream decoding
├── ONNX detector — object detection (YOLO-NAS-s, CPU)
├── /media/frigate — NFS recordings (sifaka)
└── /db — SQLite (local PVC)
Frigate pod (minikube)
├── go2rtc — RTSP restream proxy
├── FFmpeg — stream decoding
├── ZMQ detector ──tcp://host.minikube.internal:5555──→ apple-silicon-detector
│ ├── CoreML / Neural Engine
│ └── LaunchAgent (mcquack.eblume.frigate-detector)
├── /media/frigate — NFS recordings (sifaka)
└── /db — SQLite (local PVC)
└──→ MQTT (Mosquitto) → frigate-notify → ntfy → mobile
```
@ -47,7 +49,7 @@ Camera credentials are stored in 1Password and synced via [[external-secrets]] t
## Detection
Object detection uses ONNX with a YOLO-NAS-s model running on CPU (ARM64). The model file lives on the NFS recordings volume at `/media/frigate/models/yolo_nas_s.onnx`.
Object detection uses the [apple-silicon-detector](https://github.com/frigate-nvr/apple-silicon-detector), which runs natively on [[indri]] as a LaunchAgent (`mcquack.eblume.frigate-detector`). It communicates with Frigate via ZMQ over TCP (`tcp://host.minikube.internal:5555`), leveraging CoreML and the M1 Neural Engine for ~15ms inference (down from ~117ms with ONNX CPU).
A `driveway_entrance` zone is configured for alert filtering — only detections in this zone trigger review alerts.