CASE STUDY SNAPSHOT
Customer : A leading technology research and development organisation in Brazil developing medical imaging solutions for the healthcare sectorSize : 200–1000
Project vertical : Medical Devices, Embedded Multimedia, Surgical Imaging
Challenge : Develop production multi-camera driver firmware on NVIDIA Jetson TX2 for a laparoscopic surgical recording system — integrating an IR camera, a laparoscopic camera, and a multichannel input camera simultaneously, with combined 2D surgical video recording and real-time display
Solution : Production multi-camera driver stack on NVIDIA Jetson TX2 for the LifeBox surgical recording system — integrating VM1120-IR-G4 infrared camera, laparoscopic surgical camera, and multichannel input camera through custom V4L2 drivers, GStreamer-based video pipeline, and combined 2D multi-camera recording
Services & Products Availed : Embedded Software Development, Device Driver Development, Embedded Multimedia, OS Porting and BSP Development
Tools and Technologies:
- Platform: NVIDIA Jetson TX2
- OS: Linux4Tegra (L4T)
- Cameras: VM1120-IR-G4 (IR), Laparoscopic camera, Multichannel input camera
- Driver Framework: V4L2 (Video for Linux 2)
- Multimedia: GStreamer
- GPU: CUDA
- Languages: C, C++
- Tools : JetPack SDK
Introduction
Laparoscopic surgery — minimally invasive surgical procedures performed through small incisions using a camera-guided surgical instrument — has become the standard approach for a wide range of abdominal and pelvic procedures. Recording and reviewing laparoscopic surgical video is essential for surgical training, quality assurance, patient documentation, and medico-legal purposes. As surgical imaging systems become more sophisticated, the ability to capture and combine video from multiple camera sources simultaneously — providing the surgical team with complementary views of the operative field — is increasingly valuable.
A leading technology research and development organisation in Brazil, developing medical imaging solutions for the healthcare sector, approached Embien to develop the multi-camera driver firmware for their LifeBox surgical recording system — a production medical device designed to capture, combine, and record video from multiple camera sources during laparoscopic surgical procedures on the NVIDIA Jetson TX2 platform.
The LifeBox had to integrate three distinct camera types — an infrared camera for thermal visualisation, a laparoscopic surgical camera for the primary operative field view, and a multichannel input camera for additional source integration — and combine their outputs into a synchronised 2D recording suitable for surgical documentation and review.
Challenge
Integrating three camera types with fundamentally different interfaces, pixel formats, and operating characteristics into a single coherent video pipeline on the NVIDIA Jetson TX2 required developing bespoke V4L2 driver support for each camera within the Linux4Tegra kernel environment. Each camera presented its own integration challenge — the VM1120-IR-G4 infrared camera used a proprietary interface and pixel encoding that required custom format handling, the laparoscopic camera had specific timing and synchronisation requirements appropriate for its surgical context, and the multichannel input camera required driver support for simultaneously managing multiple input channels through a single device interface.
Achieving synchronised multi-camera capture — ensuring that frames from all three cameras were temporally aligned for correct 2D composition — required careful management of the capture pipeline timing across independent camera interfaces. Frame timestamp management and inter-camera synchronisation had to be implemented at the driver level to ensure that combined recordings were temporally coherent — a requirement with direct implications for the clinical usefulness of the recorded material.
The video pipeline also had to operate reliably within the demanding constraints of a surgical environment — where the system must continue recording without interruption throughout a procedure that may last several hours, handling any camera connectivity events or signal interruptions gracefully without losing recorded data or requiring operator intervention.
The production medical device context imposed quality and reliability requirements beyond what a research prototype would demand — driver stability, graceful error handling, and validated performance under sustained operation were essential for a system intended for use in an active surgical setting.
Solution
NVIDIA Jetson TX2 Platform and L4T BSP
The LifeBox surgical recording system is built on the NVIDIA Jetson TX2 — a production-grade embedded AI computing module combining a dual-core Denver 2 and quad-core ARM Cortex-A57 CPU cluster with a 256-core Pascal GPU, 8GB LPDDR4 memory, and the NVIDIA Video Image Compositor (VIC) and video encode/decode engines required for efficient multi-camera video processing. Embien worked within the Linux4Tegra BSP framework — NVIDIA's production Linux distribution for Jetson platforms — developing all camera driver work within the L4T kernel to ensure compatibility with the Jetson's hardware video processing pipeline and JetPack SDK.
VM1120-IR-G4 Infrared Camera Driver
The VM1120-IR-G4 infrared camera provides thermal visualisation capability — enabling the surgical team to observe tissue perfusion, vascular structures, and thermal gradients in the operative field that are not visible in standard visible-light video. Embien developed a custom V4L2 source driver for the VM1120-IR-G4 within the L4T kernel, implementing the camera's proprietary interface protocol, pixel format decoding from the camera's native thermal encoding to a standard V4L2 pixel format compatible with the GStreamer pipeline, and gain and integration time control through the driver's V4L2 control interface. The driver presents the infrared camera to the GStreamer pipeline as a standard /dev/video node, enabling the video pipeline to consume its output without format-specific handling at the application level.
Laparoscopic Camera Driver
The laparoscopic surgical camera provides the primary operative field view — the camera inserted through the laparoscopic port that gives the surgical team their main visual access to the surgical field. Embien developed V4L2 driver support for the laparoscopic camera within L4T, handling the camera's specific interface timing requirements and signal characteristics. Particular attention was paid to the driver's handling of the camera's connection and disconnection events — in a surgical context, the laparoscopic camera may be removed and reinserted during the procedure, and the driver had to handle these events gracefully without disrupting the recording pipeline or requiring operator intervention to restore normal operation.
Multichannel Input Camera Driver
The multichannel input camera provides additional source integration capability — enabling the LifeBox to simultaneously capture video from auxiliary sources such as an overhead surgical room camera, an anaesthesia monitoring display, or a secondary endoscopic camera alongside the primary laparoscopic view. Embien developed driver support for the multichannel camera interface, managing the simultaneous acquisition of multiple input channels through the device and presenting each channel as an independent V4L2 video node to the GStreamer pipeline. Channel synchronisation at the driver level ensures that frames from different channels of the multichannel input are temporally aligned for correct composition in the video pipeline.
GStreamer Multi-Camera Video Pipeline
The three camera drivers — infrared, laparoscopic, and multichannel — feed into a GStreamer-based video pipeline that manages simultaneous capture from all sources, frame synchronisation, composition, encoding, and recording. The pipeline was architected to leverage the Jetson TX2's hardware video encode engine for efficient H.264 encoding of the combined video output — minimising CPU load and ensuring sustained encoding performance across multi-hour surgical procedures without thermal throttling.
Frame synchronisation across the three camera sources was implemented through a custom GStreamer element that buffers frames from each source and releases synchronised frame sets based on timestamp alignment — ensuring that the combined 2D recording reflects the simultaneous state of all camera views at each recorded moment. The synchronisation element handles frame rate differences between camera sources and manages buffer depth to absorb timing jitter in camera frame delivery without introducing visible artefacts in the combined output.
The combined video composition — assembling the infrared, laparoscopic, and multichannel views into the 2D layout required for the LifeBox recording format — was implemented using the Jetson TX2's Video Image Compositor hardware engine, offloading the composition operation from the CPU and GPU to dedicated hardware, further reducing the system's processing load during sustained recording.
Sustained Recording Reliability
For a surgical recording system, recording reliability over a multi-hour procedure is a non-negotiable requirement. The video pipeline was validated for sustained operation — running continuous recording sessions representative of the longest expected surgical procedures — confirming stable memory usage, consistent frame delivery, and correct handling of any transient camera events throughout. Storage management logic ensures that recording continues correctly as the output file grows, with appropriate handling of filesystem limits and storage capacity monitoring to alert the operator before available storage is exhausted.
Benefits
- Three-camera simultaneous integration Production V4L2 drivers for VM1120-IR-G4 infrared, laparoscopic surgical, and multichannel input cameras enabling simultaneous capture from all three sources within the Linux4Tegra kernel framework
- Hardware-accelerated video pipeline GStreamer pipeline leveraging Jetson TX2's VIC composition engine and hardware H.264 encoder delivers efficient multi-camera recording without CPU bottleneck — sustaining performance across multi-hour surgical procedures
- Temporal frame synchronisation Custom GStreamer synchronisation element aligns frames from all three camera sources by timestamp — ensuring the combined 2D recording is temporally coherent for clinical review and documentation purposes
- Surgical-context driver robustness Camera connection and disconnection events handled gracefully at the driver level without pipeline disruption — enabling laparoscopic camera removal and reinsertion during a procedure without operator intervention
- Production medical device quality Driver stability, sustained recording reliability, and validated performance under multi-hour continuous operation meeting the quality requirements of a production surgical recording system deployed in an active clinical environment
Conclusion
The LifeBox surgical recording system demonstrates Embien's capability to develop production-grade embedded multimedia solutions for medical imaging applications on the NVIDIA Jetson TX2 platform. By developing bespoke V4L2 drivers for three distinct camera types — infrared, laparoscopic, and multichannel — and integrating them into a hardware-accelerated, temporally synchronised GStreamer recording pipeline, Embien provided its medical device customer with a robust multi-camera surgical recording foundation suitable for production deployment in active surgical environments. This project reflects Embien's depth in NVIDIA Jetson platform development, Linux camera driver engineering, and embedded multimedia pipeline design — a combination increasingly relevant as medical imaging systems grow in camera complexity and real-time processing demands.