As described in our previous posts, we created an ARKit-App with Face-Recognition.
I will explain how we created our Face-Recognition model.
Where to start?
Apple’s machine learning framework CoreML supports Keras and Caffe for neural network machine learning.
Never heard of these before and done anything with machine learning, I started with a Keras tutorial:
A simple dogs vs. cats example.
The results and the performance were not that good. Especially with a Macbook and ATI-Graphics.
Nvidia DIGITS and AWS
There are a couple of tools, that are beginner-friendly.
One of is Nvidia DIGITS, which offers
- Integrated frameworks Caffe, Torch and TensorFlow
- Pre-trained models such as AlexNet and GoogLeNet
- Easy to use UI
Rather than installing it on my local machine with no GPU support, I went for a AWS instance.
The newest image I found in the AWS was DIGITS 4. Even if DIGITS 6 is available.
1. Create an instance
Create a new instance from the AWS Marketplace
Now we need data
For our training data, we did a little photo-session and shot a couple of hundred photos for each person:
To get our face-detection running, we need to extract the same face/head proportion for our training photos as we do later in the app.
We created a litte script to extract the faces, downscale and save the new files:
Aside from our specific persons, we need an unknown category. So that other faces are not falsely identified as one of our classifications.
For this we searched the web, and downloaded lots of different faces.
Start the training
1. Upload our images
2. Create Dataset
- Set the image size to 227 x 227
- Caffe’s default cropped size is 227 x 227. Otherwise greater images will be cropped.
- Resize Transformation to Crop
- In order to keep the aspect ratio, we crop the images.
- Training images to /home/ubuntu/data
- Image Encoding to JPEG(lossy)
- To save space.
3. Train our model
4. Use a pre-trained model
The standard AlexNet model is not optimised for face-recognition.
There a many pre-trained model available, some of the are listed here:
In the end we chose to use FaceDetection-CNN model.
- Upload the pretrained model to /home/ubuntu/models/
- Go back to the DIGITS UI and click on the Clone Job of our previous run.
Integrate the model in iOS
1. Download model
2. Install CoreMLTools
To use our Caffe ML-model in our iOS-App we have convert it to a CoreML compatible format.
Apple provides a tool for this: coremltools
You can install it with: pip install -U coremltools
3. Create convert script
We need to write a little python script to convert our model. It’s based on examples and documentation available here: https://apple.github.io/coremltools/
# Convert a caffe model to a classifier in Core ML
coreml_model = coremltools.converters.caffe.convert(('snapshot_iter_360.caffemodel',
image_input_names = 'data',
class_labels = 'labels.txt',
# Now save the model
You may need to adjust the file-name of the caffemodel
Save this to the same folder as the model and run it with: python convert.py
4. Integration to project
We only need to copy faces_model.mlmodel to our Showcase-Project
All compile errors in Xcode should disappear.
Now we have a working face-recognition app 🙂