Fast face detection by Apple in iOS

by Krzystof Kunowski on 12th October 2011

Face detection with FaceDecorator and iOS5 Apple together with the Core Image Framework for iOS5 and Lion 10.7 introduces a really cool and easy to use face detection API.  The face detection technology was bought by Apple for over $22 million from Polar Rose, a Swedish company known from its face-tagging pictures service and facial recognition software. The API detection with the Core Image Framework in iOS5 looks pretty straightforward.

Let’s take a look at how it works

Setup detector

The most important option for the CIDetector is the accuracy. We need to find a compromise between accuracy of detection and performance depending on requirements. There are two possible values for CIDetectorAccuracy:
  • CIDetectorAccuracyLow
  • CIDetectionAccuracyHigh

NSDictionary *options = [NSDictionary dictionaryWithObject: CIDetectorAccuracyLow forKey: CIDetectorAccuracy];

In the applications, where the speed is crucial, for example when face detection is being processed on a live video stream, it is recommended to use low accuracy, especially when we are using high quality video stream. It is not recommended to use a resolution higher than 640×480 in AVCaptureSession together with face detection. With the still photo we could use high accuracy, as far as the device GPU is able to handle it in a desirable time. For instance the above photo was taken with low accuracy and detected a face on the hand drawing as well :) Finally, we can create a detector:

CIDetector  *detector = [CIDetector detectorOfType: CIDetectorTypeFace context: nil options: options];

It would be great if in the future apple extended the scope of possible detections with, for example a hand etc. but today we have only one option to choose from.

Detect features

Now, we can use lately created detector instance to detect features in the photo. In Core Image framework we, of course, use CIImage as a base image class that we can work on.  Let’s prepare CIImage in imagePickerController call back method:

UIImage *pickerImage = [info objectForKey: UIImagePickerControllerOriginalImage];

CIImage *ciImage = [CIImage imageWithCGImage: [pickerImage CGImage]]];

Next, we need to define the options for features in the photo detection. The most important option is the orientation. We need to make sure that our detector has the same orientation as the picture we use. The orientation has the same values as kCGImagePropertyOrientation for CGImageRef  but not the same as UIImage orientation. We can calculate an orientation in an easy way:

NSNumber *orientation = [NSNumber numberWithInt:[pickerImage imageOrientation]+1];

NSDictionary *fOptions = [NSDictionary dictionaryWithObject: forKey:orientation CIDetectorImageOrientation];

Finally, we can start features detection.  Detector will give us an array of CIIFeatures as a result:

NSArray *features = [detector featuresInImage:ciImage options:fOptions];

Let’s check what we can get in result:

for (CIFaceFeature *f in features) {

NSLog(@"left eye found: %@", (f. hasLeftEyePosition ? @"YES" : @"NO"));

NSLog(@"right eye found: %@", (f. hasRightEyePosition ? @"YES" : @"NO"));

NSLog(@"mouth found: %@", (f. hasMouthPosition ? @"YES" : @"NO"));

 if(f.hasLeftEyePosition)

NSLog(@”left eye position x = %d , y = %d”, f.leftEyePosition.x, f.leftEyePosition.y);

 if(f.hasRightEyePosition)

NSLog(@”right eye position x = %d , y = %d”, f.rightEyePosition.x, f.rightEyePosition.y);

if(f.hasMouthPosition)

NSLog(@”mouth position x = %d , y = %d”, f.mouthPosition.x, f.mouthPosition.y);

 }

Just give it a try

If you are curious what it looks like in action – check out our just released iPhone/iPad App FaceDecorator application [iTunes link]. We are curious about your opinion, feel free to share below!

  • http://www.facebook.com/profile.php?id=100002815945372 JustLook Biometrics

    Face detection is an emerging
    technology in India. It has been easy to spot a system using this technology in
    the day to day life in India. The use of this technology in social networking
    sites will be more than welcome in India.

  • http://www.goyello.com Krzystof Kunowski

    Indeed it has a potencial not only in India ;) both in security system or social networking to tag a friends etc.

    Unfortunately Apple does not provides a real recognition yet that is based on individual’s biometrics but only face detection.

    I am really looking forward to have it on iOS toys in upcoming time … then the real fun starts !

  • http://twitter.com/rgarge ram

    NSDictionary *fOptions = [NSDictionary dictionaryWithObject: forKey:orientation CIDetectorImageOrientation]; is giving error can you please verify. The error is use of undeclared identifier ‘forKey’.

  • http://twitter.com/kkunowski Krzysztof Kunowski

    Yep, there is a typo – it should be:NSDictionary *fOptions = [NSDictionary dictionaryWithObject:orientation forKey: CIDetectorImageOrientation];Thanx :)

  • Ian Keen

    no luck getting this working :(
    have tried both front and back cameras, both orientations.. won’t detect a face

  • Pingback: Easy face detection using iOS 5, face.com and Facebook APIs – part 1 | Blog Jerrys Apps

  • Pingback: Easy face detection using iOS 5, face.com and Facebook APIs – part 1 | Jerry's Mind Blog