Verdict

On a Raspberry Pi 5 CPU, ncnn FP32 runs a median 2.04x faster than ONNX Runtime FP32 across 33 models, ranging from 1.25x to 2.91x. No model ran slower. The cost sits in one family: 5 YOLOX models lose half a point of mAP or more. Every other model stayed within half a point, including all the Ultralytics and YOLOv9 models. mAP is shown in percent form.

The Raspberry Pi 5 runs these models on its CPU, so both runtimes are CPU inference. ONNX Runtime is the common baseline. ncnn is the CPU-tuned alternative. This compares both on the same 33 models, same COCO protocol, same board. The set now includes the Ultralytics YOLOv5u, YOLOv8, YOLO11 and YOLO26 lines next to YOLOX, YOLOv9 and PicoDet.

ModelONNX Runtime FP32 FPSncnn FP32 FPSSpeedupmAP delta (pts)
YOLOv5xu0.31.02.91x+0.0
YOLOv8-X0.31.02.91x+0.0
YOLOX-X0.30.92.87x-0.9
YOLOX-L0.51.52.85x-1.4
YOLOv8-L0.51.52.81x+0.0
YOLOX-M1.12.92.72x-1.1
YOLOv8-M1.02.82.67x+0.0
YOLOv5lu0.61.62.65x+0.0
YOLOv5mu1.23.22.63x+0.0
YOLOv9-C0.71.92.61x+0.2
YOLOv9-M1.02.32.44x-0.0
YOLOv8-S2.45.72.37x+0.0
YOLOX-S2.55.72.26x-1.3
YOLOv5su2.86.22.23x+0.0
YOLO11-S2.65.72.17x+0.0
Per-model FPS under ONNX Runtime FP32 and ncnn FP32 on Raspberry Pi 5, with speedup and mAP delta. Top 15 by speedup.

Speedup varies by family

YOLOX gains a family median of 2.72x, YOLOv8 2.67x, YOLOv5u 2.63x and YOLOv9 2.44x. YOLO11 and YOLO26 gain less, 1.95x and 1.93x. PicoDet gains 1.33x. The top single gains are YOLOv5xu and YOLOv8-X at 2.91x, both from 0.3 to 1.0 FPS. The smallest gain is PicoDet-S at 1.25x.

The YOLOX accuracy cost

5 of the 33 models lose half a point of mAP or more, and all of them are YOLOX. YOLOX-L drops 1.4 points, YOLOX-S 1.2, YOLOX-M 1.1, YOLOX-X 1.0 and YOLOX-Nano 0.9. The other families stay within half a point through the same conversion. If you deploy a YOLOX model with ncnn, re-check mAP on your own data.

No regressions

No model ran slower under ncnn than under ONNX Runtime. On this board ncnn is the faster CPU runtime for every model we measured. The only trade to weigh is the YOLOX accuracy drop.

Every number on this page comes from the verified dataset: same 500-image COCO val2017 slice, conf 0.001, IoU 0.6, max 300 detections, pycocotools mAP, identical protocol across all hardware and runtimes. The full protocol is on the methodology page. To rerun this comparison with your own filters, open compare. Accuracy is measured on LibreYOLO retrained checkpoints; other weight sources can yield different values.