OpenCV Eye Tracking with C#

A while back, I wrote a few articles about using OpenCV with C#. One of those articles was titled “OpenCV Eye Tracking with C#”. However, I had titled that article wrong. The article was actually about “head” tracking with OpenCV and C#. I just now caught that mistake and have corrected it. But, to make up for the mistake, I will now show you how to do what that title originally said. I will show you how to do “eye” tracking with C# and OpenCV. If you read the article about “OpenCV Head Tracking with C#“, this article will be a breeze for you as it only requires a couple of minor changes to the head tracking application from the last article.

Tracking eyes with OpenCV and C# is identical to head tracking. So, go ahead and read OpenCV Head Tracking with C# if you haven’t done so already. At the beginning of the code, you will notice variables for Scale, ScaleFactor, and MinNeighbors. As I mentioned in the head tracking article, you can alter these variables to get different results. Since we’re going to be tracking 2 eyes (right & left) and since they are smaller than a head, we need to modify the Scale and MinNeighbors variables. Start with the Scale variable. Drop it down to the neighborhood of 1.25. This will shrink the size of the circles that will be drawn on the screen and will also tell OpenCV to be expecting the objects being tracked to be a little smaller than what it expected for head tracking. Next, since we’re looking for 2 objects instead of 1, go ahead and change the MinNeighbors variable to 2.

The next part is just as easy as the last. In the head tracking article, you will have noticed that we used a cascade file that was provided to us in the OpenCV installation. Since we were tracking the head in that article, we went with the haarcascade_frontalface_alt2.xml cascade. For tracking eyes, you simply need to change this path to point the the haarcascade_eye.xml cascade instead. While you’re at it, you’ll probably want to go ahead and change all “face” references to say “eyes” instead, but that step isn’t completely necessary if all you’re doing is playing anyways.

That’s it! With those few changes, you can now track eyes using OpenCV and C#. If everything went correctly (and you already had the head tracking app working), you should see something like this:

In case you don’t want to go back and look at my head tracking article for the code, here it is for eye tracking. Simply copy and paste this code as EyeDetect.cs. Then, in your Program.cs, construct a new instance of EyeDetect by using “EyeDetect ed = new EyeDetect();“.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using OpenCvSharp;

namespace EdgeDetect
{
    class EyeDetect
    {
        public EyeDetect()
        {
            CvColor[] colors = new CvColor[]{
                new CvColor(0,0,255),
                new CvColor(0,128,255),
                new CvColor(0,255,255),
                new CvColor(0,255,0),
                new CvColor(255,128,0),
                new CvColor(255,255,0),
                new CvColor(255,0,0),
                new CvColor(255,0,255),
            };

            const double Scale = 1.25;
            const double ScaleFactor = 2.5;
            const int MinNeighbors = 2;

            using (CvCapture cap = CvCapture.FromCamera(2))
            using (CvWindow w = new CvWindow("Eye Tracker"))
            {
                while (CvWindow.WaitKey(10) < 0)
                {
                    using (IplImage img = cap.QueryFrame())
                    using (IplImage smallImg = new IplImage(new CvSize(Cv.Round(img.Width / Scale), Cv.Round(img.Height / Scale)), BitDepth.U8, 1))
                    {
                        
                        using (IplImage gray = new IplImage(img.Size, BitDepth.U8, 1))
                        {
                            Cv.CvtColor(img, gray, ColorConversion.BgrToGray);
                            Cv.Resize(gray, smallImg, Interpolation.Linear);
                            Cv.EqualizeHist(smallImg, smallImg);
                        }
                        
                        using (CvHaarClassifierCascade cascade = CvHaarClassifierCascade.FromFile(@"C:\Program Files\OpenCV\data\haarcascades\haarcascade_eye.xml"))
                        using (CvMemStorage storage = new CvMemStorage())
                        {
                            storage.Clear();

                            Stopwatch watch = Stopwatch.StartNew();
                            CvSeq<CvAvgComp> eyes = Cv.HaarDetectObjects(smallImg, cascade, storage, ScaleFactor, MinNeighbors, 0, new CvSize(30, 30));
                            watch.Stop();
                            Console.WriteLine("detection time = {0}msn", watch.ElapsedMilliseconds);

                            for (int i = 0; i < eyes.Total; i++)
                            {
                                CvRect r = eyes[i].Value.Rect;
                                CvPoint center = new CvPoint
                                {
                                    X = Cv.Round((r.X + r.Width * 0.5) * Scale),
                                    Y = Cv.Round((r.Y + r.Height * 0.5) * Scale)
                                };
                                int radius = Cv.Round((r.Width + r.Height) * 0.25 * Scale);
                                img.Circle(center, radius, colors[i % 8], 3, LineType.AntiAlias, 0);
                            }
                        }
                        
                        w.Image = img;
                    }
                }
            }
        }
    }
}

Originally posted at http://www.prodigyproductionsllc.com/articles/programming/opencv-eye-tracking-with-c/