Windows Phone Face Recognition

Face Recognition using 2D Fast Fourier Transform

Fast Fourier Transform

The Fourier Transform is an important image processing tool which is used to decompose an image into its sine and cosine components. The output of the transformation represents the image in the Fourier or frequency domain, while the input image is the spatial domain equivalent. In the Fourier domain image, each point represents a particular frequency contained in the spatial domain image.

Working with 2D FFT in Windows Phone 8

A new great Windows Phone’s feature is Camera Lenses that enables you to build apps into the camera app. Users can launch “Lenses” apps directly from the camera app making the facial recognition process a perfect solution for Lenses application.

I provided some tips I learned and found useful developing a lens application to save time during development.

  • Download Dsp.zip and extract DSP.cs from the zip. Then add it into your project. DSP.cs provides a namespace called DSP and a class FourierTransform containing a set of functions to compute the FFT.
  • face recog

Don’t forget to include the namespace DSP

using DSP;

Inside the WMAppManifest.xml file add the following capabilities:

<Capabilities>
          <Capability Name="ID_CAP_ISV_CAMERA" />
          <Capability Name="ID_CAP_MEDIALIB_PHOTO" />
</Capabilities>

<Extensions>
<Extension ExtensionName="Camera_Capture_App" ConsumerID="{5B04B775-356B-4AA0-AAF8-6491FFEA5631}" TaskID="_default" />
</Extensions>
  1. Each Extention element describes an App Connect extension and the Extention tag must be allocated after Tokens tag.
  2. ExtensionName ss the identifier for the type of extension support. The value is Camera_Capture_App
  3. ConsumerID restricts access to the extension to the consumer with the specified ProductID. All search extensions require the same value, 5B04B775-356B-4AA0-AAF8-6491FFEA5661.

Now your application is registered as Lenses app and can be found and called from the main Camera app.

To display the Camera flow into your application let’s add the following code:

<Button Content="Snap Picture" Click="SaveImage" />
<Grid x:Name="ContentPanel" Grid.Row="1" >
         <Grid.Background>
                  <VideoBrush x:Name="viewfinderBrush" />
         </Grid.Background>
</Grid>

Now let’s begin to build the app.

using Microsoft.Devices; // Needed for PhotoCamera
using Microsoft.Xna.Framework.Media;
using System.Windows.Media.Imaging;
using Microsoft.Phone; // Needed for PictureDecoder

namespace Face_Recognition
{
    public partial class MainPage : PhoneApplicationPage
    {
        PhotoCamera cam;
        MediaLibrary library = new MediaLibrary();

        public static WriteableBitmap CapturedImage;
        private static int W = 256;
        private static int H = 256;
        private static int matchSamples = 25;

        private double[] compareSignal = new Double[matchSamples];

        private Double[] pRealIn = new Double[W * H];
        private Double[] pImagIn = new Double[W * H];
        private Double[] pRealOut = new Double[W * H];
        private Double[] pImagOut = new Double[W * H];
        private Double[] pRealOut2 = new Double[W * H];
        private Double[] pImagOut2 = new Double[W * H];

        public MainPage()
        {
                InitializeComponent();

                this.Loaded += Lense_Loaded;
        }

        void Lense_Loaded(object sender, RoutedEventArgs e)
        {
              if (PhotoCamera.IsCameraTypeSupported(CameraType.FrontFacing))
              {
                cam = new Microsoft.Devices.PhotoCamera(CameraType.FrontFacing);
                cam.CaptureImageAvailable += cam_CaptureImageAvailable;
                viewfinderBrush.SetSource(cam);
              } else 
              if (PhotoCamera.IsCameraTypeSupported(CameraType.Primary))
              {
                cam = new Microsoft.Devices.PhotoCamera(CameraType.Primary);

                cam.CaptureImageAvailable += cam_CaptureImageAvailable;
                viewfinderBrush.SetSource(cam);
              }
        }

}

Converting a pixel to Grayscale

Here a useful function to convert a colored pixel into gray-scale. That operation allow us to save more computation,

        internal int ColorToGray(int color)
        {
            int gray = 0;

            int a = color >> 24;
            int r = (color & 0x00ff0000) >> 16;
            int g = (color & 0x0000ff00) >> 8;
            int b = (color & 0x000000ff);

            if ((r == g) && (g == b))
            {
                gray = color;
            }
            else
            {
                // Calculate for the illumination.
                // I =(int)(0.109375*R + 0.59375*G + 0.296875*B + 0.5)
                int i = (7 * r + 38 * g + 19 * b + 32) >> 6;

                gray = ((0x1) << 24) | ((i & 0xFF) << 16) | ((i & 0xFF) << 8) | (i & 0xFF);
            }
            return gray;
        }

FFT 2D

Tip: Don’t create a BitmapImage or WriteableBitmap on the UI thread because they are DependencyObjects. If you do you will get the ObjectDisposedException error because the stream for the image is closed already when the dispatcher handles your request. Move into your Dispatcher invocation.
void cam_CaptureImageAvailable(object sender, ContentReadyEventArgs e)
{         
            Deployment.Current.Dispatcher.BeginInvoke(delegate()
            {
                //Take JPEG stream and decode into a WriteableBitmap object
                MainPage.CapturedImage = PictureDecoder.DecodeJpeg(e.ImageStream,W,H);

                //Collapse visibility on the progress bar once writeable bitmap is visible.
                progressBar.Visibility = Visibility.Collapsed;

                int[] pixel = MainPage.CapturedImage.Pixels;            

                int color = 0;
                for (int y = 0; y < MainPage.CapturedImage.PixelHeight; y++)
                {
                                for (int x = 0; x < MainPage.CapturedImage.PixelWidth; x++)
                                {
                                                color = MainPage.CapturedImage.Pixels[x + (y * MainPage.CapturedImage.PixelWidth)];     
                                                pRealIn[x + (y * MainPage.CapturedImage.PixelWidth)] = DSP.Utilities.ColorToGray(color) & 0xFF;                        
                                }
                }

                Double[] match;
                double mse = 0;

                System.Diagnostics.Debug.WriteLine("Using Fourier");
                DSP.FourierTransform.Compute2D((uint)W, (uint)H, ref pRealIn, null, ref pRealOut, ref pImagOut, false);
                match = DSP.Utilities.triangularExtraction(ref pRealOut, (uint)W, (uint)H, (uint)matchSamples, 0);
                mse = DSP.Utilities.MSE(ref compareSignal, ref match, (int)matchSamples);

                // normalize
                mse /= 1000000000;

                mseResult.Text = "" + Math.Round(mse);
                System.Diagnostics.Debug.WriteLine("MSE:" + mseResult.Text);

                // Just as Demo we save the current image into a buffer called match in order to compare it with the next image. In real life situations the sample to compare is aved into a file or database.
                Array.Copy(match, 0, compareSignal, 0, matchSamples);

                // Sample code to reconstruct the image from FFT spectrum
                color = 0;
                for (int y = 0; y < MainPage.CapturedImage.PixelHeight; y++)
                {
                                for (int x = 0; x < MainPage.CapturedImage.PixelWidth; x++)
                                {                        
                                                 color = (int)Math.Floor(pRealIn[x + (y * MainPage.CapturedImage.PixelWidth)]);                   
                                                color = (color > 0) ? ((color > 255) ? 255 : color) : 0;
                                                MainPage.CapturedImage.Pixels[x + (y * MainPage.CapturedImage.PixelWidth)] = (1 << 24) | (color << 16) | (color << 8) | color;                       
                                }
                }
            });
 }
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s