RK3588 NPU Research — Orange Pi 5 Plus

Open-source reverse engineering of the Rockchip RK3588 Neural Processing Unit. This project includes a standalone NPU driver (librocketnpu), Mesa Gallium optimization patches, a QEMU emulator for the NPU hardware, and research into the proprietary RKNN register programming model.

Project Components

librocketnpu — Standalone Open-Source NPU Driver

A zero-dependency C library that drives the RK3588 NPU directly via DRM IOCTLs, without Mesa or the proprietary RKNN stack.

librocketnpu/
  src/
    rnpu_tflite.c    # TFLite FlatBuffer parser (zero deps)
    rnpu_onnx.c      # ONNX protobuf parser (protobuf-c)
    rnpu_model.c     # Graph analysis, per-channel grouping, scheduling
    rnpu_task.c      # CBUF bank allocation, spatial tiling
    rnpu_coefs.c     # Weight/bias quantization formatting
    rnpu_regcmd.c    # Hardware register command generation
    rnpu_drm.c       # DRM IOCTL submission (CREATE_BO, SUBMIT)
    rnpu_rknn.c      # RKNN binary parser (BRDMA extraction)
    rnpu_sw_ops.c    # CPU fallback: concat, maxpool, pad, resize, sigmoid, softmax
    rnpu_convert.c   # NPU tensor format conversion (NHWC <-> NCHW interleaved)
  include/
    rocketnpu.h      # Public C API
  tests/
    test_sw_ops.c    # 27 unit tests (CPU-only, no hardware)
    test_rknpu_abi.c # 29 ABI regression tests (CPU-only)
    test_onnx_parse.c # ONNX parser validation
  docs/
    compiler_architecture.md  # Research: open-source NPU compiler design

Features:

Loads TFLite INT8 models directly — no conversion step
ONNX frontend (protobuf-c) for RKNN-toolkit graph consumption
Dual-driver: supports both Rocket (mainline) and RKNPU (vendor) kernel drivers
Per-channel quantization via scale-sorted grouping + BRDMA DMA
Multi-task batching for cross-operation chaining
56 unit tests, CI on GitHub Actions

Build:

# On board (aarch64, Armbian)
apt install libdrm-dev
cd librocketnpu && make

# With ONNX support
apt install libprotobuf-c-dev
make test_onnx_parse

# Run tests (no hardware needed)
make test

Mesa Optimization Patches

Performance patches for the upstream Rocket Gallium driver:

Patch	Description	Impact
`0003`	BO pool, cache sync reduction, NEON I/O conversion, cached submit	12% avg latency reduction
`0004`	SW ops: concat, maxpool, pad, resize, logistic	YOLO mixed HW/SW execution
`0005`	Fix INT8 regression: batch tasks per operation	Correctness fix for upstream

QEMU NPU Emulator

Full-system emulation of the RK3588 NPU for development without hardware:

Boots unmodified Armbian disk images
Emulates CRU (Clock Reset Unit) for kernel driver probe
NPU MMIO register model (PC, CNA, Core units)
IOMMU stub for DMA address translation

Benchmark Results

MobileNetV1 224x224 INT8 (single core)

Stack	Latency	Status
RKNN proprietary (vendor kernel)	2.6ms	Bit-exact
librocketnpu (vendor kernel, BRDMA)	10.2ms	Bit-exact (max_diff=0)
Mesa Rocket + patches (mainline kernel)	10.2ms	Bit-exact
Mesa Rocket stock (mainline kernel)	11.6ms	Bit-exact

YOLOv5s-relu 640x640 INT8

Stack	Latency	Accuracy vs RKNN golden
RKNN proprietary (3 cores)	9.5ms	Reference
RKNN simulator (x86, ONNX Runtime)	N/A	~0.2 mean diff
librocketnpu (vendor, per-channel groups)	292ms	~18-25 mean diff

The accuracy gap is due to per-channel quantization hardware limitations — the NVDLA-derived CNA applies one requantization scale per operation, while YOLO's per-axis weights need per-channel scaling. librocketnpu approximates this with scale-sorted channel grouping and BRDMA MUL correction.

NPU Hardware Architecture

The RK3588 NPU has 3 independent cores (6 TOPS total), each with:

Offset	Unit	Function
+0x0000	PC (Frontend)	DMA engine: reads register command buffers, writes to CNA
+0x1000	CNA	Convolution Neural Accelerator (INT8 MAC array)
+0x2000	DPU + RDMA	Data Processing: bias, batch norm, element-wise, output quantization
+0x3000	Core	Power, clock, interrupt control

The NPU is register-programmed — no instruction set. Each "instruction" is a (register_address, value) pair packed into 64-bit entries, DMA'd from memory by the PC unit. A typical convolution requires ~130 register writes across CNA, DPU, and RDMA units.

Research Documents

Document	Description
`librocketnpu/docs/compiler_architecture.md`	Open-source NPU compiler design (ONNC, TVM, MLIR comparison)
`optimization_report.md`	Mesa Rocket driver optimizations (12% latency reduction)
`rocket_ioctl_analysis.md`	Decoded IOCTL protocols for Rocket and RKNPU drivers
`npu_research_report.md`	Full research report: architecture, ftrace, driver comparison
`per_axis_quantization_research.md`	Per-channel quantization hardware investigation

Board Setup


Board	Orange Pi 5 Plus (RK3588, 16GB LPDDR4X, 233GB eMMC)
OS	Armbian 25.11.1 Noble (Ubuntu 24.04)
Kernels	6.18.10-current-rockchip64 (mainline) / 6.1.115-vendor-rk35xx (vendor)

License

librocketnpu: MIT. Research documents and patches: as noted per file.

widgetii/orangepi5plus-npu