Raspberry Pi 5 AI Projects with the DEEPX NPU

Sixfab's Raspberry Pi AI projects run on a Raspberry Pi 5 with the Sixfab AI HAT+ for Raspberry Pi 5 (DEEPX DX-M1M), whose NPU runs the detection and pose models. Five use Frigate NVR and two are Python apps. The Frigate projects live in the sixfab-dx-frigate repository on GitHub, and the Python apps live in sixfab-dx-examples. Frigate projects need Frigate with the DEEPX NPU and camera feeds and MQTT, set up once before you pick a project.

Frigate projects tested on: Raspberry Pi 5 (8 GB), Raspberry Pi OS Trixie 64-bit, Sixfab AI HAT+ (DEEPX DX-M1M), RT driver v2.6.0, PCIe driver v2.5.0, DX-RT v3.4.2 and v3.4.0, firmware v2.7.4, Frigate image ghcr.io/blakeblackshear/frigate:af0ba19-standard-arm64. Last tested 2026-10-05.

How do the NPU and the Pi 5 CPU share the work?

In every project, the NPU on the Sixfab AI HAT+ runs the detection or pose model, so the Raspberry Pi 5 CPU stays free for video decoding, tracking and your own logic. Each project watches a camera, finds something in the picture and acts on it. The Frigate projects work in four steps:

1. A camera streams video over RTSP.
2. Frigate NVR sends frames to the NPU, which returns detections.
3. Frigate publishes each tracked object on the frigate/events MQTT topic.
4. In some projects, a small Python listener applies a rule, such as "log every person" or "alert if a package vanishes".

Frigate's Debug view draws a box, a label and a confidence score around each car and person it detects. The side panel names the DEEPX detector doing the work.

Which AI projects can you build on a Raspberry Pi 5?

You can build seven AI projects on a Raspberry Pi 5 with the Sixfab AI HAT+: five Frigate NVR projects (person detection, package detection, face recognition, bird species ID and squirrel detection) and two Python pipelines (a people counter, and pose estimation). Fall detection is part of the pose estimation page.

Every project was tested on a Raspberry Pi 5 with the Sixfab AI HAT+ (DX-M1M). No project page publishes an accuracy figure.

Frigate or a Python pipeline: which should you use?

Use Frigate NVR for camera events and MQTT, and a Python pipeline for work Frigate cannot do, such as pose estimation.

What hardware do you need?

Every project was tested on a Raspberry Pi 5 with the Sixfab AI HAT+ (DX-M1M). The Frigate projects add a camera that streams RTSP and an MQTT broker.

In what order should you set up a Raspberry Pi 5 AI project?

Set up the board once, then Frigate, then a camera and a broker, then a project.

1. Fit and set up the board with the Sixfab AI HAT+ Quickstart.
2. Set up Frigate with the DEEPX detector: Frigate setup on Raspberry Pi 5.
3. Add an RTSP camera and an MQTT broker.
4. Open a project from the project table and follow its Install section.

The people counter and the pose apps do not use Frigate, so they skip steps 2 and 3; they need the board and a desktop session.

Home security

The three home security projects act on what a porch or door camera sees. Boss Detector, a Raspberry Pi person detection trigger, finds people with a ready-compiled YOLOv8n model, and a Python listener publishes an MQTT alert when it sees a person. Porch Pirate, named after US slang for someone who steals delivered packages, uses a custom package model and a porch zone for Frigate package detection. It publishes an alert when a package disappears after someone new entered the zone, and it is not a security product. The Frigate face recognition camera adds enrolled names to person events.

Garden and wildlife

COCO-80 has a bird class and no squirrel class, so these two projects differ. The DIY Raspberry Pi bird camera uses one NABirds model with 555 classes, so the detector itself names the species at a feeder camera. Frigate squirrel detection uses a squirrel model and publishes an MQTT trigger for a sprinkler; no valve is wired.

Retail analytics

The DIY people counter and heatmap is a standalone Python app, not a Frigate project. The people counter runs YOLOX-S on the NPU, counts entries and exits across a line with a tracker on the CPU, and draws live and total foot-traffic heatmaps.

Health and fitness

Pose estimation on the DEEPX NPU runs one YOLOv8n-pose engine behind three Python apps: a Pilates rep counter, a fall detection demo and a Fruit Ninja-style pose game. The fall detector is a demo, not a safety device or a medical or emergency alert device.

What are the known limitations?

These limits apply across the section; each project page lists its own under Limitations.

• Frigate projects need a Frigate build with the DEEPX detector, and no stable release includes it yet. Frigate itself publishes one on GitHub Container Registry: ghcr.io/blakeblackshear/frigate:af0ba19-standard-arm64, built from commit af0ba19 of the dev branch of the official Frigate repository.
• Two actions are placeholders. Boss Detector and squirrel detection publish an MQTT message and write a log line; no real action or valve is connected.
• Porch Pirate decides "someone new" by Frigate track ID, so it has no identity check.
• Face recognition runs on the Pi 5 CPU, not the NPU. Frigate's DEEPX detector runs object detection only. Upstream Frigate lists AVX support as a requirement for face recognition, but it ran effectively on the Raspberry Pi 5.
• Every Frigate project was tested with a live IP camera over RTSP. The people counter and pose pages use video files.
• Pose estimation and fall detection ran on a Pi 5 with video files only, and the fall thresholds were set on one sample clip.
• The Frigate projects ran with Frigate in Docker. Home Assistant OS and the Frigate add-on were not tested.
• If the DX-RT daemon (the DEEPX runtime) restarts, restart the Frigate container too.
• Face recognition, the people counter and the fall demo handle images of people; see each project page's Privacy section.
• Frigate chooses the DX-RT version its DEEPX detector uses. The tested build pins DX-RT v3.4.0, and this page cannot change it.
• The Mosquitto broker in the tested setup has no password and no TLS, so anyone on the network can publish to it.

Glossary

Project pages link these terms here.

NPU

An NPU (neural processing unit) is a chip built to run neural-network models, leaving the Pi 5 CPU free for video and app logic.

TOPS

TOPS (trillions of operations per second) is an NPU's peak INT8 math rate: a spec-sheet ceiling, not a measured speed.

Frigate

Frigate NVR is an open-source network video recorder that runs in Docker, detects objects in camera streams and publishes events over MQTT. See frigate.video.

RTSP

RTSP (Real Time Streaming Protocol) is how most IP cameras stream live video, at an address that starts with rtsp://. The camera feeds page shows how to connect an RTSP camera to Frigate.

MQTT

MQTT is a publish-and-subscribe messaging protocol: Frigate posts events to a broker such as Mosquitto, and your scripts subscribe to them. See Frigate MQTT topics and a Mosquitto broker.

.dxnn

A .dxnn file is a model compiled for DEEPX NPUs by the DX-COM compiler; the DX-M1 runs only this format. See compile your own model to .dxnn.

COCO-80

COCO-80 is the 80-class label set many pretrained detectors use, with person as class 0.

PPU

The Post-Processing Unit (PPU) is the part of the DEEPX NPU that decodes the detection boxes and applies non-maximum suppression (NMS) on the chip, so the CPU has less to do. The -ppu- model files in these projects use it.

FAQ

What AI can you run on a Raspberry Pi 5 with a DEEPX NPU?

Sixfab's projects run object detection and pose estimation on the DEEPX NPU. They use YOLOv8n, YOLOX-S, a custom package model, a squirrel model, a bird species model and YOLOv8n-pose, all compiled to .dxnn. The Pi 5 CPU handles tracking and rules.

Which DEEPX NPU project should I build first on a Raspberry Pi 5?

Start with Boss Detector, the Frigate person detection project. Boss Detector uses a ready-compiled YOLOv8n model, so you write only the action.

Do these projects need a cloud account or a Frigate+ subscription?

No. Detection runs locally on the Raspberry Pi 5 and its NPU, and events go to your own MQTT broker. Porch Pirate uses a custom model instead of Frigate+. The installers download a project's files from GitHub, and Frigate downloads the DX-RT Python bindings the first time a DEEPX detector is configured.

Does Frigate run face recognition or pose estimation on the DEEPX NPU?

No. Frigate's DEEPX detector runs object detection only. Face recognition runs on the Raspberry Pi 5 CPU, and Frigate has no pose estimation support, so the pose apps are Python scripts that talk to the NPU through dx_engine. The face recognition page has the details.

Can several projects share one Raspberry Pi 5 and one NPU?

Yes, for the Frigate projects: one Frigate instance on one NPU served several projects in testing. After adding a project, check inference times after adding a project.

Can I use these Raspberry Pi AI projects in a commercial product?

It depends on each model's license and on the code license (MIT). Several models are built on Ultralytics YOLOv8 and are licensed under AGPL-3.0, which sets conditions for commercial use; the bird model's NABirds data is limited to non-commercial use. The release notes of each project, on the releases pages of https://github.com/sixfab/sixfab-dx-frigate and https://github.com/sixfab/sixfab-dx-examples, list every model's license and details, so read them before any commercial use.

Related docs

These Sixfab docs pages cover the shared DEEPX software stack.

• Pre-compiled .dxnn models in the Sixfab Model Zoo
• Models supported on DEEPX NPUs
• Check NPU load with dxtop
• Compile a model with DX-COM
• Sixfab AI HAT+ Troubleshooting