Implementation of a Neural Network over a microcontroller of the ESP32 family using recently introduced TinyML library tools. The project describes prototype development of a smart door peephole capable of person identification.
Currently, there are billions of microcontrollers operating around the world, these devices have the advantage of being cheap, storge efficient, having low power consumption and maintaining privacy to name a few.
This project introduces a prototype of a smart door peephole that benefits from all the above advantages. This prototype holds a convolutional neural network (CNN) that was trained to distinguish between different classes of people’s faces.
During the offline phase, a dataset of faces was created using a front facing laptop camera and preprocessed to fit the CNN’s input requirements. To train the model, transfer learning method was used. In this efficient method new layers are trained upon a pretrained light weight version of MobileNet V1 model. For the final stage of this phase, the model was converted into a TFlite model. During the conversion, full integer quantization was used which resulted in an 8-bit representation of the model’s weights, input, output, and activation functions.
After the offline phase, the converted model was deployed over an ESP32 microcontroller based on an Arduino project. The model receives its input from an integrated camera module when a PIR sensor is triggered. After the model outputs its classification score vector, a classification decision is conducted according to a predefined threshold and the microcontroller sends an E-mail alert via the home WiFi network.
Project Video
