A Face Recognition system to be used for marking attendance in an organisation for a streamlined and centralized record of Employees or Members.
Phase includes the following stages:
- A C++ program to detect and store faces. (Detection)
- A Python Script to maintain and link available faces. (Linking)
- A C++ program to fetch faces from a camera and compare them with available database. (Recognition)
- A Python Script to update the record on Google Spreadsheets over a secure wireless connection. (Uploading)
Phase is and has been my most ambitious project because of the way it works. And also because it is composed of three of my most relished domains: Embedded Linux, Machine Learning and Internet of Things.
Each Functional component took considerable time worth mentioning in this catalog however I also do acknowledge that a lot of hows and whys will still be skipped because they are really large in number.
Last but most important, Kudos to Stack Overflow and every developer that asked relevant questions for this project of mine to be completed even after so many dead ends I have encountered.
Project Phase:
My initial setup was Ubuntu 15.10 on an Intel core i5 laptop. Linux was a choice because I already planned to deploy the final project on a Raspberry Pi. This maintained familiarity with the development platform.
The entire project is too long to discuss every working bit in a single blog post so an extended summary is what this post is.
What Phase Does:
It Sees You, Remembers You, Recognizes You and Keeps a note of it on Google Drive !
Pretty Cool when you think about it.
Each task stated above uses a separate program linked together to work seamlessly.
It Sees You:
The Video is captured via the integrated webcam (when developing on ubuntu) and via a USB webcam (when run on Raspbian OS [Raspberry Pi]).This is made easy by always fetching the video from the default connected device.As Pi doesn't come with an inbuilt camera,default device is the USB webcam.Voila!
A C++ program linked with the OpenCV (build from source:make,make install) running a cascade Haar's Frontal Face classifier detects the faces in an image.The task of detection is the following two things:
A Video Documenting the database creation is as shown:
It Remembers You:
The database of a face is created only when only a single face is detected by the classifier.This ensures that the database of a single individual contains images only of that individual.Before saving the faces are gray scaled and resized to 300 x 300 pixels.
A lot of guidance was received from OpenCV documentation.This includes the above folder structures to store faces.
The entire project is too long to discuss every working bit in a single blog post so an extended summary is what this post is.
What Phase Does:
It Sees You, Remembers You, Recognizes You and Keeps a note of it on Google Drive !
Pretty Cool when you think about it.
Each task stated above uses a separate program linked together to work seamlessly.
It Sees You:
The Video is captured via the integrated webcam (when developing on ubuntu) and via a USB webcam (when run on Raspbian OS [Raspberry Pi]).This is made easy by always fetching the video from the default connected device.As Pi doesn't come with an inbuilt camera,default device is the USB webcam.Voila!
A C++ program linked with the OpenCV (build from source:make,make install) running a cascade Haar's Frontal Face classifier detects the faces in an image.The task of detection is the following two things:
- Number of faces in the image
- Segmenting ,Cropping and Resizing Faces
 |
| Detecting One Face and Saving to Database |
 |
| Detecting One Face and Saving to Database |
A Video Documenting the database creation is as shown:
It Remembers You:
The database of a face is created only when only a single face is detected by the classifier.This ensures that the database of a single individual contains images only of that individual.Before saving the faces are gray scaled and resized to 300 x 300 pixels.
The structure of the folder is as :
A lot of guidance was received from OpenCV documentation.This includes the above folder structures to store faces.
 |
| Root Folder |
 |
| Database of an Individual |
Although the database of images is created successfully,For a program to actually "See" them,It is crucial that every image is properly documented along with the ID of the person they represent.This path creation and Labeling is done by a Python Script.
The Python script creates a record in .csv format which contains :
- Full Path of the image
- Label Corresponding to the Image
The .csv file created looks like:
The .CSV file created is used by the next segment to fetch,load and train the Face Recognizer algorithm.
It Recognizes You
The Task of Face Recognition is done by C++ Program written using OpenCV library.
The Face Recognition module is not native to the official source yet so the additional libraries are built using a new method I came up with as documented here.This method is more reliable than the conventional route.
The program fetches live feed from the default imaging device and processes it frame by frame.
The first task that the program performs is to train its Two classifiers on the training database and labels of images.The Two algorithms used are:
- Eigenface Algorithm ( Principal Component Analysis)
- Fisher Face Algorithm ( Linear Discriminant Analysis )
Eigenface is single class specific i.e. It finds the similarities between multiple images of same individual whereas FisherFace finds the differences between different individuals.The Collective and commonly agreed result of both these algorithms trained on the same set of images is used as a confirmation of a prediction.
The Haar's cascade is run to segment the faces which are the evaluated by the two algorithms and predictions are returned by both.The value of prediction is accurate 90% of the trials however it depends on the quality of images in the database.
Video Documenting Face Recognition:
Keeping a note on Google Drive:
The task of connecting securely to google cloud is done by a python script. It uses the following package to do the task of accessing and updating attendance on google spreadsheet.
- Oauth2client (Google Cloud Authentication Client)
- Gspread (Google Spreadsheet API client)
- PyOpenSSL (Python Open SSL package)
The Result of prediction (Roll No. or Unique ID) is given to the Cloud Connect Script as a command line argument. The script fetches the date of current day from the system.These two data elements are enough to mark a student as present.
The Logic here is always a tautology,
i.e. if a student 'A' arrives before the system ,he is marked as present for the current day.
if a student 'A' is absent, he never arrives for attendance before the system, hence he is not marked for that day thus stating him absent.
The Python Script connects to a google spreadsheet via valid security credentials and update the attendance onto it.The programming is done in such a way that it handles all the possible scenarios that can arise on the spreadsheet section. Few of the problem -> solution are:
- Date Row not found -> Create row for Current Date. (When taking attendance on a new day)
- Roll No not found -> Create column for Roll No. (When database is updated)
- Date Row found, Roll No column not found -> Add Roll No column and write "Present" in current date row (Database updated during current day)
- Date Row not found, Roll No. Column Found -> Add Date Row and write "Present" in current Roll No. column (Database Intact, Day changed )
The Data for the recognized individual is successfully updated in 3-4 seconds. This is slow compared to execution time of our Recognizer program however keeping in mind all the authorizations and Credential check every time, it for sure is a lead over other unsecured connections.
The Google spreadsheet is edited to give write access to our API token so that there is no conflict of permissions during write task.
Here is the video of Phase updating the attendance of a detected individual in real time:
The Pi Setup:
The Setup is done with a Dell VGA monitor using an HDMI to VGA converter to connect to Raspberry Pi. Additionally USB Webcam,Keyboard & Mouse are connected via USB port.The webcam lights are kept off because of high current surge of 6 LEDs. They barely make any improvements in lighting conditions anyway.
- An 8GB Sandisk MicroSD card is loaded with NOOBS and Raspbian OS is installed.
- OpenCV is built from source using my method for extra modules building as stated here.
- CodeBlocks is installed from apt-get and code is copied to from the ubuntu system to Pi using a thumb drive.
- Static path for database storage, database linking,fetching and cloud uploading are set to get around using command line arguments every time.The Detection stage still employs CL arguments to denote the person being databased.
The entire system is enclosed in a box as follows :
The LCD and the glowing Leds are part of a temperature monitoring system. It measures the temperature of the box internals to warn or ward off any heat damage. And that is an entirely different story for a later time.
-----------------------------------------------------------------------------------------------------------------
[ Update 17 November,2018 ] :
The code for a dlib variant of the face detection and recognition project is available for access on my github here : https://github.com/sanjeev309/face-recognition-dlib-tensorflow-knn
You will need to modify the core code to suit your requirement for an attendance system.
Pull requests are welcome.
-----------------------------------------------------------------------------------------------------------------
Success is not final, failure is not fatal: it is the courage to continue that counts
Used :
Code::Blocks IDE
PyCharm IDE
Raspbian OS
Atmel Studio 7.0
-----------------------------------------------------------------------------------------------------------------
[ Update 17 November,2018 ] :
The code for a dlib variant of the face detection and recognition project is available for access on my github here : https://github.com/sanjeev309/face-recognition-dlib-tensorflow-knn
You will need to modify the core code to suit your requirement for an attendance system.
Pull requests are welcome.
-----------------------------------------------------------------------------------------------------------------
Success is not final, failure is not fatal: it is the courage to continue that counts
Used :
Code::Blocks IDE
PyCharm IDE
Raspbian OS
Atmel Studio 7.0
nice project can i get the code ?? hahaha
ReplyDeleteBro can u tell me about the eigenfaces ??
Hello Lagan, Thank You for stopping by.
DeleteFor a little depth in EigenFaces I would recommend the following page: https://jeremykun.com/2011/07/27/eigenfaces/
For OpenCV specific information, nothing beats the official documentation :)
http://docs.opencv.org/2.4/modules/contrib/doc/facerec/facerec_tutorial.html
I work on my graduation project has the same idea but i face a lot of issuses , can you help me , give me your email to contact with you .
ReplyDeleteHi Alaa, Apologies for the late reply. What can I help you with?
Deleteya already did this project but my face recolonization is not properly working it can not recognize multiple face
ReplyDeleteHi Robin! Thank you for stopping by. This is an old project of mine therefore it uses [now] old methods of face recognition. Today we have way better approaches to face recognition which are also significantly faster. I would refer you to the following link Dlib Face Recognition using Deep Metric. And Yes it works on multiple faces! Let me know if that helps. Kudos!
DeleteBro does this code work in normal windows computer?
ReplyDeleteHi, The code will work on windows however it is cumbersome to configure OpenCV and an IDE on windows. I would happily recommend Linux for development task if you are just starting up.
DeleteCan i get the code please?
ReplyDeleteHi Jatin, I don't have the code uploaded anywhere anymore. Primarily because the methods used in my project are now obsolete and there are far better implementations available now. I would recommend you look into Dlib's Deep Face Recognition which claims to have an accuracy of 99.38%. Here's the link in case you consider it: http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html
Deletehello can i get that code plzz
ReplyDeletehello can i get the code plzz
ReplyDeleteHi Kavya,this project was done a long time ago with the techniques now being obsolete. I recommend you go through the following link for a better approach in C++ : http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html
DeleteIn Python: https://github.com/davisking/dlib/blob/master/python_examples/face_recognition.py .
Hello bro can you help me how to create attendance sheet In rasspery pi
ReplyDeleteI will be very thankfull
My mail is Hammadyounas2008@gmail.com
Kindly if you send me attendance report code or all code i will be very thankfull to you
ReplyDeleteHi Hammad, This project was done a long time ago with the techniques now being obsolete. I recommend you go through the following link for a better approach in C++ : http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html
DeleteIn Python: https://github.com/davisking/dlib/blob/master/python_examples/face_recognition.py It will work on Pi as well.
sir i recognized the face on pi but i dont know how to gernate the report kindly help me in that
DeleteHi Hammad, I don't think I understand your query properly as to what 'report' you are referring to. Kindly contact me via contact form, it will redirect to my mail. I will be glad to help if I can in any way.
Deletei want to know that for each student you would be uploading 30 to 40 images and at a school we have 1000 students on an average. So, how will it be managed? Such a huge database and what if the student gets some problem with there face?
ReplyDeleteHi Aman, Good Question! With the setup I had for the above project, I had an average size of 1 Mb collectively for an individual. At ~1000 individuals that would just scale up to be ~1 GB which in fact is way less than any standard of a huge database. Besides, the cropped faces were only used to train the two models which were then saved in yaml format to be reused again, following which the database of images can actually be deleted unless retraining required.The trained model file had a comparatively smaller disk footprint. As for changing facial features,this is something that no system can practically eliminate with 100% accuracy, deep learning systems however can give way better results in such scenarios.
DeleteHey, im new to all this but iireally want to make it so i tried going to contact form but it's not loading. I sent you request on hangout as well as LinkedIn please accept it and give me your email atleast. Thank you..
ReplyDeleteHi Vivek, I have accepted your connection requested on LinkedIn the same day. Let's discuss the queries you have on LinkedIn itself.
Deleteeverything is working just want to know how to save the attendance in spreadsheet or excel.Would be happy if you could help me
ReplyDeleteHi Anurag, The following article will help you automate saving data into an excel sheet: https://automatetheboringstuff.com/chapter12/ . Feel free to reach out if you get stuck somewhere.
Deletecan you tell me the functional and non functionla reuirements
ReplyDeletehey, can you tell me how to update the data to libreoffice? thank you before
ReplyDelete