Efficient Video Processing on Embedded GPU

Learn how to develop and test a 4K video processing and streaming application on the NVIDIA Jetson TX1/TX2 embedded platform with GStreamer. To achieve real-time video processing, the diverse processing resources of this high-performance embedded architecture need to be employed optimally.

The heterogeneous system architecture allows capturing, processing, and streaming of video with a single chip. The main challenges lie in the optimal utilization of the different hardware resources of the Jetson TX1 (CPU, GPU, dedicated hardware blocks) and in the extensive software stack (from Linux drivers to GStreamer application).

We'll discuss variants, identify bottlenecks, and show the interaction between hardware and software. Simple capturing and displaying video from the built-in camera can be achieved using out-of-the-box methods. However, for capturing 4K from HDMI we had to dig into the documentation, rewrite the drivers for the Video capture system, write a driver for an external HDMI to CSI converter and figure out efficient zero-copy methods. GPU-based enhancements were developed and integrated for real-time video processing tasks (scaling and video mixing).

Blog post about the drivers: https://blog.zhaw.ch/high-performance/2016/06/01/open-source-driver-for-hdmi2csi-module-released/​

Tobias Kammacher has worked for the last three years in the High Performance Multimedia and Data Acquisition Research Group in the Institute of Embedded Systems at Zurich University of Applied Sciences. He and his colleagues carry out research projects with industry partners, focused on implementing signal and video processing applications on SoC, FPGA, and mixed architectures. Tobias received his B.S. in electrical engineering and M.S. in engineering.