Embien is a recognized leader in working with cutting edge technologies, especially related to multimedia and communication. We have pushed the performance of the embedded devices to the most possible limits reliably and repeatedly. In this case study, we will explore how we helped a Europe-based defense product manufacturer create a multi-channel video mixer platform to stream multiple channels of video over a Tegra K1 based design. The customer wanted to realize a highly configurable PCIe Multi-channel video mixer system approached us based on our prior multi-channel video capture card design expertise.
In the system, multiple streams of independent video data were to be mixed and displayed over a single output. Further, many more proprietary processing algorithms were to be added along with other highly specific needs as per their standards. To be deployed in unmanned aerial vehicles, both performance and band width are of primary concerns in these systems. Embien took up the challenge of designing the entire data pipeline and provided with them an integrated solution.
Further to our requirement understanding, based on our prior GStreamer and Nvidia expertise, Embien proposed a system with two processing units - an ARM-based unit for video data generation and an FPGA based unit for video mixing and other custom logic. It was decided to go for NVIDIA Tegra K1 as the ARM processing unit and Intel/Altera Arria V as the FPGA platform. To meet the high data transfer requirement, PCIe bus was used for connectivity. The block diagram of the PCIe multi-channel video mixer is captured below.
The FPGA team developed the complete logic necessary for the implementation of the FPGA Video processing sub-system. Intel FPGA's IP were used whenever possible and new ones created when needed. Most of the blocks were connected using the standard Avalon ST interface. To reduce the load, DMA engines were created in the FPGA that transfers the data between endpoints. Various configuration options were made available to select the way the video channels were mixed and displayed. Many design considerations were provided for scalability such as coordinating display of the channel data, configuring the dimensions of the data channels, enabling/disabling the data path etc. Alpha blending was also supported so that the videos can be blended seamlessly. Since the video data could be in any color space such a YUV12, Y12, Y14, RGB888, BAYER etc. suitable conversion mechanisms were developed and incorporated. To achieve very high bandwidth in the PCIe multi-channel video mixer, multiple lanes were supported in the PCIe.
The Tegra K1 unit was developed based on a COTS module to reduce the development time and due to low volume requirements. Based on NVIDIA's Linux4Tegra kernel base, Embien developed the complete software subsystem. Underlying PCIe device driver was created to enable applications to manage the FPGA video logic. Upon detecting the FPGA device, on the PCIe bus the driver presents itself to the user space. For further ease of use, GStreamer plug-ins were developed on top of this interface, for developing multimedia applications quickly and also to leverage the power of the GStreamer media framework.
The video channel could be created either from decoding a video file like MPEG, WMV, etc. or from camera input. Or it could be complete graphics developed with OpenGL APIs. Either way, up to 1080p30 streams could be generated from the Nvidia Tegra K1. Further processing was done using the CUDA framework on the Nvidia Kepler device. The 192 GPU cores ran in parallel for faster image processing. The output was available from one of the following sources- OpenGL, CUDA programming logic, GStreamer, or user rendered buffers. The driver configured the DMA engines to transfer these data from DDR in the Nvidia Tegra K1 platform to the FPGA Video processing platform frame by frame. Once the frame was transferred, the driver flipped the frame buffer at the right moment to avoid any tearing effects. User-friendly APIs were provided for easy configuration of the multi-channel video mixer system.
Embien Technologies also developed a full-fledged demo to prove the performance of the overall PCIe FPGA card and Tegra K1 video system, especially its throughput and latency. It involved rendering complex designs using OpenGL developed with Eclipse Nsight, post-processing them with the CUDA programming framework, and blending them finally with other video channels. The whole implementation of PCIe Multi-channel video mixer was closely monitored, profiled with advanced benchmarking techniques and refined till phenomenal performance is obtained.
All the requirements of the customer were met in time. Some of the advantages the customer had by working with our team are
The customer was very satisfied with our delivery of the PCIe multi-channel video mixer and has been working with us continuously since. Feel free to get in touch with us to leverage our best quality and result oriented development processes in your FPGA video processing or Nvidia based development needs.