Troubleshooting

Troubleshooting

Diagnose and resolve issues with the Sixfab Edge AI Expansion Board for Raspberry Pi 5: NPU detection, dxrt-runtime behavior, the 40 mm PCIe FFC and pogo-pin contact, the USB bridge that carries NVMe and cellular, host power, and the thermal envelope. Each issue lists the symptom, the most likely cause, and the verified fix.

DEEPX DX-M1M / DX-M1ML Raspberry Pi 5 PCIe Gen 2 / Gen 3 x1 Triple M.2 USB-C PD
Edge AI Expansion Board · Reference · Troubleshooting · Updated 2026-05-14
How do I troubleshoot the Sixfab Edge AI Expansion Board?

Start with the two quick diagnostic commands below: dxrt-cli -s and dxtop, then widen to lspci, lsusb, lsmod, and dmesg | grep -i dx. Their output narrows almost every issue to one of nine categories: NPU detection, runtime / DKMS, the 40 mm PCIe FFC cable, the USB bridge carrying NVMe and cellular, host power and pogo-pin contact, thermal throttling, slow inference, or a Raspberry Pi OS kernel update that left the DEEPX driver out of sync. Match your symptom to one of the issue cards below and follow the fix steps.

Quick diagnostics: run these first

These six commands cover the majority of issues across all three subsystems on the Edge AI Expansion Board. Run them in order and check the output before drilling into a specific issue card.

bash · run in order 6 checks
# 1. Full NPU status: runtime/firmware versions, PCIe link speed, per-core voltages and temperatures
dxrt-cli -s

# 2. Confirm the DX-M1 is visible on the PCIe bus
lspci

# 3. Confirm the USB bridge, NVMe, and cellular modem are enumerated on USB
lsusb

# 4. Confirm the DEEPX kernel module (dx_dma) is loaded
lsmod | grep -i dx

# 5. Check the kernel ring buffer for driver / PCIe errors
dmesg | grep -i dx

# 6. Live NPU telemetry: per-core utilisation, temperature, throttle observation
dxtop

Common issues

Each row below is one symptom. Click a row to open it; the body covers what you see, why it happens, and the verified fix. Rows are colour-coded by category. The dot, the category pill, and the left rail all match, so you can scan the list by category at a glance.

NPU

NPU not detected: lspci shows no DEEPX device

Symptom

lspci output contains no DEEPX entry. dxrt-cli -s reports no device found. The runtime is installed but the NPU is invisible to the host.

Cause

In the vast majority of reported cases the 40 mm PCIe FFC cable between the Edge AI Expansion Board and the Raspberry Pi 5 PCIe FPC connector is reversed, partially seated, or the pogo pins are not making contact because the M2.5 spacers are loose. The Pi 5 sees no PCIe link, so the DX-M1 never enumerates.

Fix

  1. 1
    Power off the Pi 5 and verify PCIe FFC orientation.

    The Edge AI Expansion Board does not support hot-plug. Disconnect the USB-C power cable before touching anything. The 40 mm Sixfab FFC has two labelled ends: RPI5 goes to the Raspberry Pi 5, EDGE_AI goes to the Expansion Board. The labels must face out (visible to you), and the metal contacts must face in toward each PCB. A reversed cable produces no link.

  2. 2
    Reseat the cable at both FFC connectors.

    Open both FFC latches, remove and reinsert the cable firmly at both ends, close the latches. Each latch must sit flush with the connector body. A latch that feels closed but is not flush is a common trap and will not make electrical contact with the PCIe lane.

  3. 3
    Tighten the M2.5 spacers to seat the pogo pins.

    The Edge AI Expansion Board back-powers the Pi 5 through four pogo pins (2× 5V + 2× GND) on the underside of the Pi 5 GPIO header. Loose spacers leave the pogo pins under-tensioned; the Pi 5 may either fail to power on or boot but show no PCIe link. Tighten all four M2.5 spacers finger-tight, not hard, but firmly seated.

  4. 4
    Inspect dmesg for PCIe enumeration errors.
    dmesg | grep -i dx

    On a healthy stack you should see dx_dma_pcie probe messages and [dx_dma_pcie_probe] Probe Done!!. If you instead see PCIe link failed, link training timeout, or simply silence after the kernel boot lines, the host saw the bus event but could not establish a link. This is almost always traceable back to the FFC cable or pogo pins above.

  5. 5
    Confirm the PCIe lane is enabled in config.txt.

    The Edge AI Expansion Board requires the Pi 5 PCIe lane enabled. Open /boot/firmware/config.txt and confirm both lines are present and not commented out:

    dtparam=pciex1 dtparam=pciex1_gen=3

    The first enables the PCIe lane; the second enables Gen 3 for full DX-M1 bandwidth. Without these lines, no PCIe link comes up regardless of how well the cable is seated. Reboot if you change anything.

  6. 6
    Verify enumeration after reboot.
    lspci

    You should see the expected DEEPX line on the bus:

    Expected output PCIe link up
    00:00.0 PCI bridge: Broadcom Inc. BCM2712 x1 PCI Express Bridge (rev 21)
    01:00.0 Processing accelerators: DEEPX Co., Ltd. DX-M1 AI Accelerator (rev 01)
Runtime

Driver installed but dxrt-cli -s reports "no device found"

Symptom

The sixfab-dx package is installed and dxrt-cli is on $PATH, but dxrt-cli -s prints no device found. lspci may or may not list the DEEPX entry.

Cause

The DEEPX kernel module (dx_dma) is not loaded into the running kernel. The Sixfab APT package builds the module via DKMS, so normally a reboot or kernel update triggers an automatic rebuild, but the load can fail if DKMS could not compile against the current kernel or if the system was not rebooted after install.

Fix

  1. 1
    Confirm whether the kernel module is loaded.
    lsmod | grep -i dx

    You should see a dx_dma entry. Empty output means the module is not loaded. Proceed to step 2.

  2. 2
    Reboot the Raspberry Pi 5.

    The DEEPX kernel module may not have been loaded into the current session, especially right after a fresh install. Reboot first, then re-check with lsmod | grep -i dx and dxrt-cli -s.

    sudo reboot
  3. 3
    Reinstall sixfab-dx to rebuild the kernel module.

    Reinstalling forces APT to recompile the DEEPX kernel module via DKMS against the currently running kernel.

    sudo apt update sudo apt install --reinstall sixfab-dx
  4. 4
    Re-check kernel messages for module load errors.
    dmesg | grep -i dx

    If lsmod still shows no DEEPX module after the reinstall, the kernel ring buffer is where the failure is logged. Look for DKMS build errors or missing kernel headers.

If the symptom started after an OS update

Skip to Driver stopped working after a Raspberry Pi OS update. The recovery sequence is different.

PCIe / FFC

PCIe link errors in dmesg or link does not train at Gen 3

Symptom

dmesg | grep -i dx shows messages such as link failed, link training timeout, or PCIe enumeration messages followed by silence. dxrt-cli -s may intermittently detect the device and report Gen2 X1 instead of the expected Gen3 X1, or errors during inference.

Cause

The PCIe link between the Pi 5 PCIe FPC connector and the Edge AI Expansion Board is unstable. Most often: the 40 mm PCIe FFC cable is partially seated, the cable is reversed (labels facing the wrong way), the dtparam=pciex1_gen=3 line is missing from config.txt, or a non-Sixfab cable was substituted for the supplied one.

Fix

  1. 1
    Power off the Pi 5 and reseat the 40 mm PCIe FFC cable.

    Open both FFC latches at the Pi 5 PCIe port and the Expansion Board PCIe port. Remove and reinsert the cable firmly, then close both latches. Verify each latch sits flush with the connector body.

  2. 2
    Re-verify cable orientation: RPI5 to Pi 5, EDGE_AI to Expansion Board.

    The two labelled ends are not interchangeable. The RPI5 end goes to the Raspberry Pi 5; the EDGE_AI end goes to the Expansion Board. Both labels face out (visible to you), and the metal contacts face in toward the respective circuit boards. A reversed cable produces no link.

  3. 3
    Use only the supplied 40 mm Sixfab FFC cable.

    The included cable is a specialized 40 mm Sixfab-manufactured part. Substituting a different cable or a longer one is not recommended. Pin assignment and signal integrity are tuned for this exact length and connector pair.

  4. 4
    Verify the PCIe lane is enabled in config.txt.

    The default Pi 5 PCIe lane runs at Gen 2 unless explicitly enabled at Gen 3. Open /boot/firmware/config.txt and confirm both lines are present and not commented out:

    dtparam=pciex1 dtparam=pciex1_gen=3

    Reboot if you change anything.

  5. 5
    Re-check the link after reboot.
    dxrt-cli -s

    The output should report PCIe : Gen3 X1. Measured bandwidth is approximately 400–450 MB/s at Gen 2 and 800–900 MB/s at Gen 3, so a stuck-at-Gen-2 link is a measurable performance regression. Persistent errors after these steps indicate a damaged cable. Contact Sixfab support for a replacement.

USB Bridge

NVMe SSD installed but does not appear in lsblk or lsusb

Symptom

An NVMe SSD is installed in the M.2-SSD slot, but lsblk does not show a /dev/sda device, or lsusb does not list the Realtek USB-to-NVMe bridge.

Cause

The NVMe slot is USB-attached, not native PCIe.

The NVMe slot is routed through the on-board USB 3.2 Gen 1 hub and a Realtek RTL9210B-CG USB-to-NVMe bridge, then up to the Pi 5 via the USB-C bridge between the boards. Two physical links carry NVMe traffic, the M.2 module itself and the USB-C bridge to the Pi 5, and either being loose breaks the path. The OS sees the SSD as a USB Mass Storage device, so lsblk (not nvme list) is the right tool to check it with.

Fix

  1. 1
    Power off the Pi 5 and reseat the M.2 module in the SSD slot.

    Loosen the M2 6 mm flat-head screw, remove the SSD, then reinsert it at a 30° angle until the gold contacts are fully seated, lower it flat, and tighten the screw finger-tight. The module should sit parallel to the Expansion Board.

  2. 2
    Confirm the USB-C bridge between the Expansion Board and the Pi 5 is firmly seated.

    This bridge carries both NVMe and cellular data. A loose USB-C bridge will silently drop both subsystems even though the M.2 modules are physically installed correctly. Disconnect and firmly reseat the USB 3.0 connection at both ends.

  3. 3
    Re-check enumeration on USB and as a block device.
    lsusb lsblk

    You should see a Realtek RTL9210 NVMe Bridge entry in lsusb, and /dev/sda with the expected SSD capacity in lsblk. If the SSD appears in lsusb but lsblk still does not show /dev/sda, the module is communicating but unpartitioned. Follow the partition / format procedure on the NVMe Storage page.

  4. 4
    Check power.

    An under-powered system can cause USB devices to disconnect intermittently. If the SSD appears and disappears, head to Power / under-voltage. The official Raspberry Pi 27 W USB-C PD supply is the documented minimum; 45 W is recommended for a full-stack configuration.

USB Bridge

LTE/5G modem installed but does not appear in lsusb

Symptom

A cellular modem (e.g. Quectel RM520N-GL) is installed in the CELLULAR LTE/5G M.2 Key-B slot, but lsusb does not list it and the modem's network interface does not appear in ifconfig.

Cause

Like NVMe, the cellular modem on this board is USB-attached. It routes through the on-board USB 3.0 hub via the USB Bridge PCBA. If the USB-C bridge between the Expansion Board and the Pi 5 is loose, or the modem is not fully seated in the M.2 Key-B slot, the modem never enumerates.

Fix

  1. 1
    Power off the Pi 5 and reseat the modem in the M.2 Key-B slot.

    Loosen the M2 6 mm flat-head screw, remove the modem, then reinsert it at a 30° angle until the gold contacts are fully seated, lower it flat, and tighten the screw finger-tight. The module should sit parallel to the Expansion Board.

  2. 2
    Confirm the USB-C bridge to the Pi 5 is firmly seated.

    The USB-C bridge between the Expansion Board and the Pi 5 carries both NVMe and cellular. A loose USB-C bridge will silently drop both subsystems even though the M.2 modems are physically installed correctly.

  3. 3
    Verify enumeration with lsusb.
    lsusb

    You should see the modem's VID:PID. For the Quectel RM520N-GL the expected line looks like ID 2c7c:0125 Quectel Wireless Solutions Co., Ltd. RM520N-GL. [NEED FROM SIXFAB: confirmed VID:PID list per tested modem model, QP49d.]

  4. 4
    Discover the modem's network interface name.

    The cellular network interface name is device-dependent. usb0 is the most common, but it is not guaranteed. Find the actual name with ifconfig first, then use it for any ping tests.

    ifconfig ping -I usb0 1.1.1.1
  5. 5
    If the modem enumerates but does not register on the network, continue on the LTE/5G Cellular page.

    Network registration, APN, SIM activation, and signal-quality monitoring are covered on the Cellular Connectivity page. [NEED FROM SIXFAB: registration-denied causes and AT-command flow, QP48k cluster.]

Power

System reboots, shows an under-voltage warning, or pauses at boot

Symptom

One of three patterns: (a) the Pi 5 pauses at boot and prompts you to press the on-board button to acknowledge USB current; (b) the Raspberry Pi OS taskbar shows a yellow lightning bolt (under-voltage); (c) the system reboots without warning under inference load, or the SSD intermittently disconnects.

Cause

Almost always a power supply or pogo-pin contact problem.

The Edge AI Expansion Board powers the entire stack, including back-powering the Pi 5 via four pogo pins, from one USB-C PD input. A supply below 27 W will trigger system crashes, SSD disconnects, and under-voltage warnings. The boot-pause symptom is specifically caused by the Pi 5's default USB current cap, which is lifted with the usb_max_current_enable=1 line in config.txt. And a loose M2.5 spacer leaves the pogo pins under-tensioned, producing intermittent power that looks like an under-voltage event.

Fix

  1. 1
    Use the official Raspberry Pi USB-C PD supply: 27 W minimum, 45 W recommended.

    The Edge AI Expansion Board exclusively negotiates the following USB PD profiles: 5V/5A, 9V/3A, or 12V/2.25A for the 27 W mode; 5V/5A, 9V/5A, or 12V/3.75A for the 45 W mode. For a full-stack configuration (DX-M1 + NVMe + LTE/5G all populated), the official Raspberry Pi 45 W USB-C PD supply is strongly recommended.

  2. 2
    Add usb_max_current_enable=1 to config.txt if you see the boot-pause prompt.

    Without this line, the Pi 5 pauses at boot and prompts you to press its on-board button to acknowledge that more than 1.6 A of USB current is being requested. The Expansion Board's full-stack draw exceeds that cap by design, so the line is required for unattended boot.

    usb_max_current_enable=1

    Add to the end of /boot/firmware/config.txt, then reboot.

  3. 3
    Tighten the M2.5 spacers to seat the pogo pins firmly.

    The four pogo pins (2× 5V + 2× GND on the Pi 5 GPIO header underside) carry power from the Expansion Board to the Pi 5. Loose spacers leave the pogo pins under-tensioned; the symptom is intermittent power that mimics an under-voltage event. Tighten all four spacers finger-tight.

  4. 4
    Do not power the Pi 5 from its own USB-C port at the same time.

    Power must come from the Expansion Board's USB-C input. If a power adapter is plugged into the Pi 5's own USB-C port while the Expansion Board is also powered, the system behaviour is undefined. Only the Pi 5 itself will operate, and the M.2 stack will be silent. Use exactly one USB-C input: the one on the Expansion Board.

  5. 5
    Confirm the warning has cleared.
    vcgencmd get_throttled

    Expected output: throttled=0x0. Any non-zero value indicates an under-voltage event has been recorded since boot.

If the USB-C cable is unplugged during operation

No physical damage occurs, but unsaved data is lost and filesystem errors may occur on the NVMe SSD. Always shut down the Pi 5 cleanly with sudo shutdown -h now before removing the power cable.

Thermal

Frame rate drops over time during sustained inference

Symptom

Inference starts at the expected FPS but throughput degrades after several minutes of continuous load. The system does not crash; FPS stabilises at a lower value and recovers when load is paused.

Cause

NPU temperature is rising into the throttling band. The DEEPX DX-M1 begins thermally throttling at approximately 90 °C. Passive cooling is sufficient for typical bursty workloads, but sustained 100 % NPU utilisation in a closed enclosure or warm ambient can push the per-core temperature above that threshold, at which point the runtime backs off clocks to stay within the safe envelope.

Fix

  1. 1
    Watch NPU temperature live with dxtop.
    dxtop

    The display reports per-core NPU voltage, clock, temperature, and utilisation in real time. Run it in a second terminal during sustained inference and watch for a steady climb toward 90 °C. dxrt-cli -s shows a snapshot of the same per-core values.

  2. 2
    Add active cooling for sustained 100 % NPU workloads.

    Passive cooling covers typical bursty inference, but benchmarks, multi-camera continuous inference, and deployment in a sealed enclosure benefit from active cooling. The Edge AI Expansion Board does not include a dedicated fan connector. The Pi 5 active cooler runs from the Pi 5's own JST-SH fan header, which the Expansion Board does not consume. Use the official Raspberry Pi 5 active cooler.

  3. 3
    Check ambient temperature and enclosure ventilation.

    If ambient temperature is near the upper end of the operating range, thermal headroom shrinks. Move the system to a cooler location or improve enclosure airflow before assuming a hardware issue. Sixfab does not prescribe a single enclosure spec. Ventilation requirements depend on ambient, workload duty cycle, and whether a fan is fitted.

  4. 4
    For deeper telemetry, hook dxtop into the monitoring page.

    Continuous logging of NPU temperature alongside other system metrics is covered on the System Monitoring page.

Runtime

Inference runs but frame rate is much lower than expected

Symptom

Inference completes without errors, but throughput is well below the published figure for the model on this configuration.

Cause

Most often one of four things: the PCIe link trained at Gen 2 instead of Gen 3, the model is not actually running on the NPU (a raw .onnx falls back to CPU), a YOLO model was compiled without PPU so non-maximum suppression serialises on the CPU, or CPU-side capture and preprocessing are starving the NPU.

Fix

  1. 1
    Confirm the PCIe link is at Gen 3.
    dxrt-cli -s

    The output should report PCIe : Gen3 X1. If it reports Gen2 X1, the host is not feeding the DX-M1 at full bandwidth: measured ceiling drops from 800–900 MB/s to 400–450 MB/s. Confirm dtparam=pciex1_gen=3 is present in /boot/firmware/config.txt, then reboot.

  2. 2
    Confirm the model is running on the NPU.

    If you passed a raw .onnx file to the runtime instead of a compiled .dxnn, the runtime falls back to CPU inference. That fallback can be one to two orders of magnitude slower. Always compile to DXNN with DX-COM first. See Custom Models (DXNN SDK).

  3. 3
    For YOLO models, recompile with PPU support.

    A YOLO model compiled without PPU (Post-Processing Unit) performs non-maximum suppression on the CPU after every NPU inference, which becomes the bottleneck at high frame rates. Recompile with the PPU flag enabled in DX-COM and the bounding-box output is returned directly from the NPU.

  4. 4
    Watch NPU utilisation during inference.

    Run dxtop in a second terminal. Near 100 % NPU utilisation with low FPS means CPU-side preprocessing is the bottleneck (capture, color conversion, resize). Low NPU utilisation means the NPU is starved and is waiting on the CPU. Fix capture and preprocessing first. Both are the CPU's responsibility on this board, not the NPU's.

  5. 5
    Use the async API.

    The dxrt API supports asynchronous inference. Submit the next frame to the NPU while it processes the current one. This hides preprocessing latency and significantly improves throughput.

NVMe and cellular load do not measurably degrade NPU throughput

The DX-M1 has a dedicated PCIe FFC link directly to the Pi 5; NVMe and cellular share a separate USB 3.0 path through the on-board hub. Simultaneous NVMe writes plus LTE/5G traffic plus AI inference shows no measurable AI-throughput degradation. If slow inference correlates with disk or network activity, the bottleneck is the CPU, not the NPU.

Runtime

Driver stopped working after a Raspberry Pi OS update

Symptom

The Edge AI Expansion Board was working before a routine sudo apt upgrade or kernel update. After reboot, dxrt-cli -s reports no device found, lsmod | grep -i dx returns empty, and lspci may or may not still list the DEEPX entry.

Cause

Why this happens

The Sixfab APT package installs the DEEPX kernel module via DKMS, which is supposed to rebuild it automatically against any new kernel installed by the system. In most cases DKMS handles this transparently and no manual recovery is needed. When it does not, usually because DKMS could not find matching kernel headers, or the build failed silently, the dx_dma module ends up missing from the new kernel and the NPU disappears. Reinstalling the package forces a fresh DKMS build against the now-running kernel.

Fix

Run the following sequence in order. The Q&A guidance is that a clean reinstall of sixfab-dx is sufficient to recover; the steps below add the verification checks needed to confirm the rebuild succeeded.

  1. 1
    Confirm the kernel version actually changed.

    This rules out unrelated regressions. If the kernel version is unchanged, the OS update did not invalidate the module. Re-run the Quick diagnostics at the top of this page first.

    uname -r
  2. 2
    Confirm the DEEPX kernel module is no longer loaded.
    lsmod | grep -i dx

    Empty output is consistent with the OS-update cause. DKMS did not produce a dx_dma module for the new kernel. Continue to the reinstall.

  3. 3
    Refresh the package index, then reinstall sixfab-dx.

    The --reinstall flag forces APT (and therefore DKMS) to recompile the kernel module against the current running kernel even if the package version on disk has not changed.

    bash
    # 1. Refresh package index
    sudo apt update
    
    # 2. Reinstall sixfab-dx (recompiles the dx_dma kernel module via DKMS)
    sudo apt install --reinstall sixfab-dx

    Compilation against the new kernel is the longest part of the reinstall. Lots of compiler output is normal.

  4. 4
    Reboot.

    A reboot ensures the freshly compiled module is loaded cleanly into the running kernel, rather than relying on the install-time hooks.

    sudo reboot
  5. 5
    Verify the module loaded and the NPU is reachable.
    bash
    # Module loaded?
    lsmod | grep -i dx
    
    # Runtime + NPU healthy?
    dxrt-cli -s
    Expected output Ready
    DXRT v3.3.0
    =======================================================
     * Device 0: M1, Accelerator type
    ---------------------   Version   ---------------------
     * RT Driver version   : v2.4.0
     * PCIe Driver version : v2.2.0
    -------------------------------------------------------
     * FW version          : v2.5.6
    --------------------- Device Info ---------------------
     * Memory : LPDDR5 5600 Mbps, 3.92GiB
     * Board  : M.2, Rev 1.0
     * Chip Offset : 0
     * PCIe   : Gen3 X1 [01:00:00]
    NPU 0: voltage 750 mV, clock 1000 MHz, temperature 45'C
    NPU 1: voltage 750 mV, clock 1000 MHz, temperature 45'C
    NPU 2: voltage 750 mV, clock 1000 MHz, temperature 45'C
    =======================================================
    If the reinstall fails to compile

    If apt install --reinstall sixfab-dx reports DKMS build errors against the new kernel, the most common root cause is missing kernel headers. The Sixfab APT package normally pulls raspberrypi-kernel-headers as a dependency, but on a manually patched system it may be absent. If that fails, the safest path is a fresh setup against the current kernel. Follow the Quickstart install steps from the start.

Still need help?

If your symptom is not covered above, contact Sixfab support. Including the output of the six Quick diagnostic commands in your report turns a multi-day debugging cycle into a single round-trip.

Sixfab support

Couldn't fix it with the cards above?

Attach the output of dxrt-cli -s, lspci, lsusb, lsmod | grep -i dx, dmesg | grep -i dx, and dxtop (a screenshot is fine). Those six outputs are usually enough to pinpoint the issue.