Verdict

Across 68 models on a Raspberry Pi 5, ONNX Runtime FP32 gives a median 1.39x over PyTorch FP32, ranging from 0.85x to 2.49x. The gain depends on the family: DEIMv2 and RT-DETRv2 gain 1.87x, while the Ultralytics families gain little or lose speed. 6 models run slower on ONNX, and 9 lose half a point of mAP or more.

PyTorch FP32 is the reference runtime. ONNX Runtime FP32 is the common first step off PyTorch on a CPU board. This compares both on the same 68 models, same COCO protocol, same board: a Raspberry Pi 5 running on its CPU.

ModelPyTorch FP32 FPSONNX Runtime FP32 FPSSpeedupmAP delta (pts)
YOLOX-Nano8.621.32.49x-1.1
DEIMv2-Atto12.730.32.39x+0.0
DEIMv2-Femto7.416.72.26x+0.0
DEIMv2-Pico2.96.22.16x+0.0
RT-DETR-R50m0.40.82.08x+0.0
RT-DETRv2-R50m0.40.82.08x+0.0
YOLOv9-T2.95.92.03x-0.7
DEIM-N2.34.31.88x+0.0
DEIMv2-N2.44.41.87x+0.0
D-FINE-N2.34.31.87x+0.0
RT-DETR-R500.30.61.87x+0.0
RT-DETRv2-R500.30.61.87x+0.0
YOLOX-Tiny5.59.41.72x-0.8
YOLOv9-S1.42.41.68x-0.9
D-FINE-L0.40.71.65x+0.0
Per-model FPS under PyTorch FP32 and ONNX Runtime FP32 on Raspberry Pi 5, with speedup and mAP delta. Top 15 by speedup.

Speedup varies by family

The detection transformers and the YOLOX and YOLOv9 families gain a family median of 1.57x to 1.87x. The biggest single gain is YOLOX-Nano at 2.49x, from 8.6 to 21.3 FPS. The Ultralytics families sit at the bottom: YOLO26 gains 1.13x, YOLO11 1.11x, YOLOv5u 1.01x, and YOLOv8 0.94x. PicoDet gains 1.26x and ECDet 1.07x.

Accuracy

mAP is shown in percent form. 9 of the 68 models lose half a point or more after conversion. YOLOv9-M drops the most, 1.7 points, from 55.3 to 53.6. The losses land on the YOLOX, YOLOv9 and PicoDet families. Every other family held its accuracy, including all the Ultralytics models.

Six regressions

6 models ran slower on ONNX than on PyTorch. Five are large Ultralytics models: YOLOv8-X at 0.85x, YOLOv5xu at 0.87x, YOLOv8-L at 0.91x, YOLOv8-M at 0.94x and YOLOv5lu at 0.98x. The sixth is PicoDet-L at 0.92x, from 1.9 to 1.7 FPS, which also lost 1.5 mAP, from 44.2 to 42.7. It is the only conversion here that costs both speed and accuracy. We did not determine the cause of any regression.

Every number on this page comes from the verified dataset: same 500-image COCO val2017 slice, conf 0.001, IoU 0.6, max 300 detections, pycocotools mAP, identical protocol across all hardware and runtimes. The full protocol is on the methodology page. To rerun this comparison with your own filters, open compare. Accuracy is measured on LibreYOLO retrained checkpoints; other weight sources can yield different values.