Lane Detection with OpenCV and C#

It’s easy to recognize when a semester is drawing near. Every computer science college student on the planet begins scouring the internet, looking for projects they can call their own and submit as their senior projects. Personally, I wish everyone would develop their own / new projects as that’s how we get many of the amazing products we all come to love and rely on. But, I also know that many computer scientists need a platform to build on top of. Besides, as a great man once said, “we’re all standing on the shoulders of giants” – Isaac Newton. Since my website’s purpose is to educate and to give others the building blocks for developing their own products, you can imagine how hammered my web servers get when semester-ends get closer and closer.

Over the last few weeks, I have received hundreds of emails asking for source code to many of my computer vision projects. The most commonly requested project this semester is my lane detection application. A while back I had a harddrive crash and unfortunately did not have a backup, causing me to lose the source code for my original lane detection application. I also haven’t had time (or a reason) to rewrite the application. However, with the boom of excitement about products such as Google Glasses and the Vuzix Smart Glasses, I have decided to rewrite my lane detection app which I would like to use with the Google Glasses, Vuzix Smart Glasses, or modified Vuzix Wrap 1200 video glasses that I have mounted a camera onto. The code is by far no where close to being complete. But, I do think it is in a good place that I can share it. Plus, as already mentioned, the state of the code at this point is only the stepping stones for others to build on top of. I might decide to release the final source code once I have it completed, but I haven’t really thought that far ahead yet. Until then, here is the code as it is today.

You can find the entire Microsoft Visual Studio C# project download at the end of this article. The download also includes a video you can use for testing. There is also a YouTube video of the application in action at the bottom of this article.

Above is a screenshot of the lane detection application in action. As you can see, the application can detect the lane you are driving in as well as nearby lanes which are all indicated by the red lines. I have also included code to detect cars which are indicated by blue circles. Lanes are marked with red lines and only extend to the horizon since we are not interested in anything beyond that. I have included a video that I found on the web somewhere that you can use to test your code with. However, the code can easily be modified to work with live camera feeds instead. For example, I have tested my app using my Vuzix Wrap 1200 video glasses mentioned earlier while driving down the road. Even though the app still needs a lot of work, it still did a good job of marking the lanes and circling nearby cars. I even included an alarm that would sound in the Vuzix’s earbuds when a car entered my lane and was inside a predefined distance from myself. I didn’t include that code for this article as I expect others to do their own implementations.

If I decide to release another version of this application, I will include the alerts as well as lane markers that will contain arrows indicating which way the road is turning. I also plan on posting a video of me driving down the road wearing my Vuzix glasses while using this app so that you can see the app in action just like I do for my tests. At some point, I even plan on adding things like road sign recognition and pedestrian detection. With a little bit of enginuity, you could use this application as the building blocks for a self-navigating car. You could watch road signs to detect the speed limit and adjust your speed according to the speed limit and the presence of other cars in your lane. You could also have your car automatically stop when a pedestrian enters your proximity. The possibilities are endless.

using System;
using OpenCvSharp;

namespace LaneDetection
{
    class Program
    {
        [STAThread]
        static void Main()
        {
            CvCapture cap = CvCapture.FromFile("road.avi");
            CvWindow w = new CvWindow("Lane Detection");
            IplImage src, gray, dstCanny, halfFrame, smallImg;
            CvMemStorage storage = new CvMemStorage();
            CvSeq lines;
            CvHaarClassifierCascade cascade = CvHaarClassifierCascade.FromFile("haarcascade_cars3.xml");

            const double Scale = 2.0;
            const double ScaleFactor = 1.05;
            const int MinNeighbors = 2;

            CvSeq<CvAvgComp> cars;

            while (CvWindow.WaitKey(10) < 0)
            {
                src = cap.QueryFrame();
                halfFrame = new IplImage(new CvSize(src.Size.Width / 2, src.Size.Height / 2), BitDepth.U8, 3);
                Cv.PyrDown(src, halfFrame, CvFilter.Gaussian5x5);

                gray = new IplImage(src.Size, BitDepth.U8, 1);
                dstCanny = new IplImage(src.Size, BitDepth.U8, 1);


                smallImg = new IplImage(new CvSize(Cv.Round(src.Width / Scale), Cv.Round(src.Height / Scale)), BitDepth.U8, 1);
                using (IplImage grey = new IplImage(src.Size, BitDepth.U8, 1))
                {
                    Cv.CvtColor(src, grey, ColorConversion.BgrToGray);
                    Cv.Resize(grey, smallImg, Interpolation.Linear);
                    Cv.EqualizeHist(smallImg, smallImg);
                }

                storage.Clear();
                cars = Cv.HaarDetectObjects(smallImg, cascade, storage, ScaleFactor, MinNeighbors, HaarDetectionType.DoCannyPruning, new CvSize(30, 30));

                for (int i = 0; i < cars.Total; i++)
                {
                    CvRect r = cars[i].Value.Rect;
                    CvPoint center = new CvPoint
                    {
                        X = Cv.Round((r.X + r.Width * 0.5) * Scale),
                        Y = Cv.Round((r.Y + r.Height * 0.5) * Scale)
                    };
                    int radius = Cv.Round((r.Width + r.Height) * 0.25 * Scale);
                    src.Circle(center, radius, CvColor.Blue, 2, LineType.AntiAlias, 0);
                }

                // Crop off top half of image since we're only interested in the lower portion of the video
                int halfWidth = src.Width / 2;
                int halfHeight = src.Height / 2;
                int startX = halfWidth - (halfWidth / 2);
                src.SetROI(new CvRect(0, halfHeight - 0, src.Width - 1, src.Height - 1));

                gray.SetROI(src.GetROI());
                dstCanny.SetROI(src.GetROI());

                src.CvtColor(gray, ColorConversion.BgrToGray);
                Cv.Smooth(gray, gray, SmoothType.Gaussian, 5, 5);
                Cv.Canny(gray, dstCanny, 50, 200, ApertureSize.Size3);

                storage.Clear();
                lines = dstCanny.HoughLines2(storage, HoughLinesMethod.Probabilistic, 1, Math.PI / 180, 50, 50, 100);

                for (int i = 0; i < lines.Total; i++)
                {
                    CvLineSegmentPoint elem = lines.GetSeqElem<CvLineSegmentPoint>(i).Value;

                    int dx = elem.P2.X - elem.P1.X;
                    int dy = elem.P2.Y - elem.P1.Y;
                    double angle = Math.Atan2(dy, dx) * 180 / Math.PI;

                    if (Math.Abs(angle) <= 10)
                        continue;

                    if (elem.P1.Y > elem.P2.Y + 50 || elem.P1.Y < elem.P2.Y - 50)
                    {
                        src.Line(elem.P1, elem.P2, CvColor.Red, 2, LineType.AntiAlias, 0);
                    }
                }

                src.ResetROI();

                storage.Clear();
                w.Image = src;
            }
        }
    }
}

Originally posted at http://www.prodigyproductionsllc.com/articles/programming/lane-detection-with-opencv-and-c/