CASE STUDY SNAPSHOT
Customer : Electronics Manufacturing Services ProviderSize : 200-1000
Project vertical : Manufacturing
Challenge : To design and develop a secure face recognition-based access control system for protected area
Solution : Develop a secure offline NVIDIA Jetson Nano powered Access control system using Retinaface with Tensorflow and Arcface on Jetson Nano
Services Availed : Embedded ML development, Edge Video Analytics, Web Application Development, Electronics product development services
Tools and Technologies :
- MPU: NVIDIA Jetson Nano
- Framework: NCNN, ArcFace, RetinaFace
- Programming Language: C++
- Compiler: GCC
INTRODUCTION
Embien works with some of the leading EMS (Electronics Manufacturing Services) Providers to support its customers in prototyping and manufacturing electronics products. One of our EMS vendors needed a solution to monitor and restrict access to a part of their plant as it was dedicated for some specific customers and provided access only to a certain group of individuals. Knowing very well about Embien's prowess, they approached us to develop such a system.
CHALLENGE
The entire face recognition-based access control system had to be secure, and all data had to be stored locally for their security audit purposes. It had to be put up in a retrofit manner without many changes to the existing infrastructure. Further it had to be quite a low-cost system that could be deployed later in many of their plants.
SOLUTION
System Design
Our electronics product development team and our edge AI/ML development team brainstormed together about various approaches. Considering many factors like ease of availability, lower cost, feature set available along with a rough calculation of CPU/GPU processing power needed by the system, it was decided to go for NVIDIA Jetson Nano platform. At the time of development, it was the most affordable yet powerful module that had necessary features. A high-resolution USB camera was identified and connected to the Jetson Nano. The Ethernet interface was provided for configuration and some of the GPIO’s from the kit was used to drive the access control system after due level translation.
Web based user interface
Our Web Application Development team provided a sleek web-based user interface for the system. Users can log in over the web server running on the secure port over the Ethernet port at a predefined local domain name. After entering the password, the system can be configured. Separate pages were provided for performing training, viewing status, reviewing past detection and export reports, modified settings etc. Firewalls were added to prevent unauthorized access to the internet by our embedded Linux team. Secure boot was enabled to prevent un-trusted applications from running on the system and the file system was encrypted too.
Embedded ML Algorithm Development
Embien’s Embedded ML development engineers worked on the algorithms to achieve the necessary face recognition-based access control functionality. It had to be a two-step process post deployment – face detection and face recognition. Many face detection algorithms like OpenCV Haar cascade, OpenCV DNN (Deep Neural Network), Dlib, MTCNN etc. were considered and finally RetinaFace with Tensorflow was chosen. Similarly, many face recognition algorithms like FaceNet, DeepFace etc were deliberated up on before finalizing the Arcface model.
It is known that using RetinaFace for face detection and Arcface for face recognition will take a long time to calculate features, cosine distance and processing through CNN layers when running in CPU. As the frame rate is not very high, our team used Tencent’s NCNN framework. NCNN is a high-performance neural network inference framework that support common CNN networks including classical CNN (VGG AlexNet, GoogleNet Inception), Face detection (MTCNN, RetinaFace) and segmentation (FCN, PSPnet, UNet, YOLACt). Designed for mobile and embedded devices, NCNN can use Vulkan routines to accelerate the convolutions of a deep learning model.
Since the Jetson Nano has Vulkan support which can be leveraged by NCNN, it easily met the expected performance metrics. The pretrained mnet.25-opt.bin and mnet.25-opt.param files were used to detect face and localize the facial points. And the pretrained mobilefacenet.bin and mobilefacenet.param files were used to extract features from the detected faces.
Training Process
With all the necessary applications and frameworks in place, we provided a web browser-based mechanism where the members to be allowed to access can be trained with multiple photos from different angles. The photos taken from the camera were fed to a training process as follows:
Face Recognition System - Training Process
The results of the training were updated in the database along with the underlying data. Provision was given to disable access to a pre-trained person.
Recognition Process
The below picture depicts the process of recognition in the face recognition-based access control leveraging the Retinaface with TensorFlow and Arcface on Jetson Nano.
Face Recognition Process on NVIDIA Jetson Nano
NVIDIA Jetson Nano always kept the door in closed state and
- Continuously monitor the frames from camera and detect the faces from frame using RetinaFace.
- Extract the features of detected faces from frame using ArcFace.
- Calculate the cosine distance between the pretrained feature vector and extracted feature vector from frame.
- Based on the similarity score, it identifies the person by showing the names saved in labels file.
- Save the entries in text file along with the time.
- Based on the permission, activated the GPIO to open the door.
Finally, the system was packaged in a rugged enclosure and connected to the door access control system.
BENEFITS
Embien’s edge analytics capabilities helped yet another customer in developing their access control system with the following benefits.
- The entire system was secure protected by secure boot and encryption technologies.
- State of the art AI inference Jetson Nano module was used without compromising cost.
- Easy to use user interface provided over web interface that can be accessed over with any PC/mobile phone.
- The retrofit mechanism helped installing it without any changes in the existing infrastructure.
- Added anti-spoof feature to prevent erroneous detection with static photos.
CONCLUSION
Once the system was validated in one of their plants, the customer installed 3 more such systems in their other plants. The project helped manage access to their protected zone easily. Later we helped them with another system to detect if the technicians were entering the work area with proper personal protective equipment. But that is for another article!