AI Development on the ALPON

End-to-end developer reference for running Edge AI workloads on the ALPON, an industrial edge AI computer built on Raspberry Pi CM5. This page covers the DEEPX DX-M1 AI accelerator, the DEEPX toolchain, the pre-compiled model zoo, the path from an ONNX file to a running container, and the throughput you can expect on the ai gateway in production deployments.

Overview

The ALPON is an edge ai computer designed for image-based ai on edge inference. Hardware acceleration is provided by the DEEPX DX-M1, a 25 TOPS NPU with 4 GB of dedicated on-chip memory that is independent of the Raspberry Pi CM5 system RAM. All of the developer tooling described on this page targets that accelerator.

Two-stage pipeline

Host vs. container architecture

ALPON OS ships with the DEEPX kernel driver preinstalled. Your inference code runs inside Docker containers. This decouples host maintenance from application deployment: OS and driver updates arrive via OTA, while inference logic updates independently through container images.

What lives where

The toolchain is split cleanly between the host PC (your development laptop) and the target device (the ALPON). Keep this split in mind when wiring up build systems and CI pipelines.

Languages and APIs

The DEEPX Runtime ships first-class bindings for Python (dx_engine) and C++ (dxrt_api.h). Use Python for rapid prototyping, video analytics with OpenCV or GStreamer, and glue code. Use C++ when you need deterministic latency, embedded integration, or direct runtime control.

AI Accelerator (DEEPX DX-M1)

Supported model formats

Models must be exported to ONNX and compiled with the DEEPX compiler (dx-com) before they execute on the NPU. Source frameworks for ONNX export include PyTorch, TensorFlow, Keras, XGBoost, and MXNet. Any framework that emits a valid ONNX graph of supported operators works.

Supported workloads

The DX-M1 is optimized for image-based AI workloads. The runtime executes convolutional architectures reliably. Transformer-based models are not supported on the current runtime.

• Object detection: YOLO family (v5, v7, v8, YOLOX), SSD variants
• Classification: ResNet, MobileNet, EfficientNet
• Segmentation: U-Net and semantic segmentation backbones
• OCR: PaddleOCR-based detection, classification, and recognition heads
• Face detection and recognition: CNN-based pipelines
• Pose estimation: keypoint-regression CNNs

Known limitations

Plan model architecture and memory footprint around the following constraints before deployment.

DEEPX Runtime

Install or reinstall the runtime

The runtime ships with every ALPON out of the box, so first-time users can skip ahead to the Deploy Your First Model section. Use the command below only when you need to reinstall, recover from a broken state, or update after a kernel upgrade.

sudo apt update && sudo apt install sixfab-dx

Verify the runtime

Confirm the NPU is enumerated and the driver is healthy with dxrt-cli -s. This is the first command to run when diagnosing any issue.

dxrt-cli -s

DXRT v3.2.0
 * Device 0: M1, Accelerator type
 * RT Driver version  : v2.1.0
 * FW version         : v2.5.0
 * Memory : LPDDR5x 6000 Mbps, 3.92 GiB
 * PCIe   : Gen3 X1 [01:00:00]
NPU 0: voltage 750 mV, clock 1000 MHz, temperature 46°C
NPU 1: voltage 750 mV, clock 1000 MHz, temperature 46°C
NPU 2: voltage 750 mV, clock 1000 MHz, temperature 46°C

Installed CLI tools

The sixfab-dx package installs a small set of command-line tools for diagnostics, monitoring, and headless inference.

Using the Python API

The dx_engine Python library ships inside the runtime virtual environment. Activate it before importing.

source /usr/lib/libdxrt/dxrt-venv/bin/activate
python -c "import dx_engine; print(dx_engine.__version__)"

Concurrent models

The DEEPX Runtime schedules NPU time across multiple concurrently loaded models automatically. A typical ALPON deployment pairs a detector (for example YOLO) with a classifier or OCR head on the same device. You do not need to implement your own scheduler.

Model Zoo

Categories and representative models

Quantization modes: Q-Lite vs Q-Pro

Models in the zoo ship in two quantization flavors. Pick one based on your accuracy headroom and deployment timeline.

How do I download a model from the Model Zoo?

Models are distributed as .dxnn files from the DEEPX developer portal. On the ALPON, fetch the file directly with wget or curl and mount it into your container.

# Example layout: keep models under /opt/models on the host
sudo mkdir -p /opt/models
cd /opt/models
 
# Download a Q-Lite YOLOv5 nano model (example path)
sudo curl -L -O   https://developer.deepx.ai/modelzoo/download/yolov5n_qlite.dxnn

Review the per-model license on the DEEPX developer portal before you redistribute or ship a commercial product derived from a zoo model.

Deploy Your First Model

Prerequisites

• An ALPON powered on and reachable over SSH, with ALPON OS up to date.
• DEEPX Runtime healthy on the device. Verify with dxrt-cli -s; the runtime ships preinstalled.
• Docker available on the device (included with ALPON OS).
• A .dxnn model, either downloaded from the Model Zoo or compiled with dx-com on your host PC.

Step-by-step walkthrough

from ultralytics import YOLO
 
model = YOLO("yolov8n.pt")
model.export(format="onnx", opset=13, simplify=True)
# produces yolov8n.onnx

dx-com compile   --model yolov8n.onnx   --config yolov8n.cfg   --output yolov8n.dxnn   --ppu

scp yolov8n.dxnn alpon@<device-ip>:/opt/models/

import cv2
import numpy as np
from dx_engine import InferenceEngine
 
# 1) Load the compiled model onto the DX-M1 NPU
engine = InferenceEngine("/models/yolov8n.dxnn")
 
# 2) Prepare an input frame (BGR, 640x640 for YOLOv8n)
frame = cv2.imread("/models/sample.jpg")
input_tensor = cv2.resize(frame, (640, 640))
 
# 3) Run inference on the NPU
outputs = engine.run([input_tensor])
 
# 4) outputs is a list of numpy arrays; shape depends on the model
print("Output tensors:", [o.shape for o in outputs])

FROM debian:trixie-slim
 
RUN apt-get update && apt-get install -y       sixfab-dx python3-opencv
 
COPY infer.py /app/infer.py
WORKDIR /app
 
# Use the bundled DEEPX venv so dx_engine is importable
CMD ["/usr/lib/libdxrt/dxrt-venv/bin/python", "/app/infer.py"]

docker build -t alpon-infer:latest .
 
docker run --rm --privileged   --device /dev/dxrt0   -v /opt/models:/models   alpon-infer:latest

dxtop
# watch NPU utilization, memory, and temperature in real time.

Common pitfalls

Docker Access

Production ALPON deployments run inference code inside Docker containers. The kernel driver lives on the host, so the container only needs access to the device node and the runtime libraries.

# Minimal run command: privileged mode + NPU device node
docker run --privileged   --device /dev/dxrt0   -v $(pwd)/models:/models   -it your-image:tag

The equivalent in docker-compose.yml:

services:
  inference:
    image: your-image:tag
    privileged: true
    devices:
      - "/dev/dxrt0:/dev/dxrt0"
    volumes:
      - ./models:/models
    restart: unless-stopped

Performance & Benchmarks

YOLO throughput reference

How to benchmark your own model

Use a simple loop around engine.run() and measure wall-clock time over a warm window. Skip the first ~10 iterations to avoid warm-up noise.

import time, numpy as np
from dx_engine import InferenceEngine
 
engine = InferenceEngine("/models/yolov8n.dxnn")
dummy  = np.random.randint(0, 255, (640, 640, 3), dtype=np.uint8)
 
# Warm-up
for _ in range(10):
    engine.run([dummy])
 
# Timed window
N = 200
t0 = time.perf_counter()
for _ in range(N):
    engine.run([dummy])
dt = time.perf_counter() - t0
print(f"FPS: {N/dt:.1f} (n={N})")

Variables that move the numbers

• Model variant. Nano tier is the fastest; small, medium, and large variants drop FPS roughly proportional to compute.
• Input resolution. Doubling resolution roughly quarters FPS on detection models.
• PPU compilation. On or off is usually the largest single factor for YOLO.
• Quantization mode. Q-Lite runs slightly faster than Q-Pro; choose Q-Pro only if you measure an accuracy regression.
• Concurrent models. Running a detector and a classifier simultaneously shares NPU time; each will run slower than in isolation.
• Pre- and post-processing path. OpenCV decode plus color conversion on the CPU can bottleneck high-resolution pipelines. Consider GStreamer with hardware decode for sustained Full HD.

Optional: enable PCIe Gen3

The ALPON is tuned to run the DX-M1 on a Gen3 PCIe link. If you have modified the device tree or carrier configuration, verify the link speed reports Gen3 X1 after reboot.

dxrt-cli -s   # look for: PCIe : Gen3 X1

Hard limits

Monitoring & Power

Live monitor: dxtop

# On the host or inside a privileged container with /dev/dxrt0 mounted
dxtop

For headless monitoring from a remote system, run dxtop over the ALPON Cloud remote terminal or an SSH session. Pipe the output into your logging stack if you want long-running metrics rather than an interactive view.

Quick reference commands

# Full hardware status
dxrt-cli -s
 
# Real-time NPU utilisation (q to quit)
dxtop
 
# Test a compiled model headlessly
run_model --model_path yolov8n.dxnn
 
# Reinstall or update the runtime
sudo apt update && sudo apt install sixfab-dx

Can I control DEEPX NPU power modes from software?

No. The DX-M1 on the ALPON runs at a fixed performance profile. There is currently no user-facing API to switch between "maximum performance" and "low power" modes. In practice the NPU draws 2 W idle to 5 W under full load on supported AI workloads, which is a narrow enough envelope that dynamic throttling has little operational value at the edge.

If your deployment is power-constrained, control the total inference budget at the application layer: reduce the input frame rate, skip alternate frames, or stop inference between triggers rather than throttling the silicon.