Face Detection & Tracking using OpenCV

Download source code and executable: FaceDetect

This is implemented using OpenCV v1.
It's actually a simple combination of two of the samples that come with the library:
 -Face Detect
 -Feature Tracking.


Since Face Detection takes much more time & processing than Feature Tracking, it tracks the face on each frame but it only tries to detect a face every 100 milliseconds and then only in the currently tracked area. Red frame indicates an actual face detected, blue frame indicates just feature tracking so rapid flickering between red/blue is a good sign, solid blue not so much.

When it cannot detect a face it keeps tracking the dots until they degrade too much or too much time (1.5 seconds) elapses since last face was detected in which case it tries to find a new face in the whole picture.

Little kids love to play with this program as it is. They compete trying to get their face captured by the program and trying to take the capture from whomever has it.

It can be used as input to play games when coupled with GlovePie. It sends detection results to the IP indicated as the first parameter into port 54934 (hardcoded).

I use this as an improvement on my virtual gaming experience (see old video below) I replaced one wiimote & the IR cap with this program as the virtual game controller. It's a poor man's Kinect.

This is the old video where I was using two wiimotes to play N.O.L.F.

Helicopter Auto Pilot - Helicopter Control

Back to Helicopter Auto Pilot - Introduction

Download source code & runtimes: Heli Control

Helicopter Control (d-L-i)

For controlling the helicopter I developed a program in C# called IR_HeliAutoPilot.exe you can find it in folder HeliComm.

(WORK IN PROGRESS more to come later)

Back to Helicopter Auto Pilot - Introduction

Helicopter Auto Pilot - Infrared Emitter

Back to Helicopter Auto Pilot - Introduction
Download source code & runtimes: Heli Control

Infrared Emitter (L-i-h)

In the original video I posted I was using the USB IR Toy together with a simple circuit that amplifies the IR signal with some extra IR Leds. In version 1 of the USB IR Toy couldn't do this so I had to reprogram the IR Toy to send the signal through the TX serial pin which had a header on the board where you could connect the amplifier circuit. This was a rather complicated development that required using the MPLAB IDE and C compiler to reprogram the PIC18F2550 at the heart of the IR Toy. Unfortunately for me during testing that part of the circuit was fried.

Based on a suggestion posted on the original video I decided to use instead the laptop's headphone jack to send the signal to a homemade IR emitter.

I must warn you, my knowledge of electronics is pretty limited so I'm sure this circuit can be improved upon and I'm open to suggestions. This circuit was designed with the assistance of the nl5 circuit simulator by Alexei Smirnov: http://nl5.sidelinesoft.com/.

The circuit is simple enough:

• T3 & T4 form a classical BJT astable multivibrator (oscillator) at ~ 32KHz. 
• T1 & T2 perform an AND operation with the incoming signal (Base).
• D1 & D2 serve to create a constant current driver for the IR Leds (D3-5).

To connect this circuit to the audio jack connect the Left wire to Base, Right to ground and ignore the ground wire.

PS. With a small modification this circuit also becomes compatible with the  i-Helicopter app for Androids and iPhone. The app is free and it uses the same protocol as my FalconX helicopter .

Back to Helicopter Auto Pilot - Introduction

Helicopter Auto Pilot - Decoding Helicopter Protocols

Back to Helicopter Auto Pilot - Introduction
Download source code & runtimes: Heli Control

Decoding Helicopter Protocols

For deciphering and decoding the specific protocols for each helicopter I used a tool called the USB IR Toy from Dangerous Prototypes: http://dangerousprototypes.com/docs/USB_Infrared_Toy version 1. It allows you to receive and send infrared signals. Although the emitter in version 1 was not strong enough to cover a big area.

I also used my program IR_HeliAutoPilot.exe that communicates with the IR Toy and helps decode the signals. This program was written in C# using Visual Studio 2005 and it's in the folder HeliComm. The subfolder hcomm has some of the definitions (.hcomm files) and trials I've made for different models.

This video explains the process of discovering the protocol, if you have questions you can post them in the comments section here:

Scripting the protocol
Once you know the details of the protocol the helicopter uses it can be scripted into the program and saved as an .hcomm file. That entails entering the different pulse lengths in microseconds in the IR Protocol tab and scripting the variables in the Heli Protocol and Script tabs.

(WORK IN PROGRESS, will expand later)

Back to Helicopter Auto Pilot - Introduction

Helicopter Auto Pilot - Camera Tracking (h-c-d-L)

Back to Helicopter Auto Pilot - Introduction
Download source code & runtimes: Heli Control

Camera Tracking (h-c-d-L)

For tracking the helicopter I'm using a Logitech HD 1080 webcam capable of 30 fps. After processing the images it slows down to approximately 27 fps.

For processing the image I adapted, in a very rudimentary fashion, a C++ program I found somewhere in the internet that can read the frames from the camera and processes them trying to find the brightest spot in the picture. For that reason the best situation to use this is indoors and at night when no sunlight can penetrate the room. You don't need to have the lights off but you must avoid having any strong light source like lamps, leds, screens or some reflective surfaces.

The program is in the folder called HeliFind. It was compiled using C++ Express 2010.
The program requires OpenCV 2.2.

Same as the wiimote script this program sends the tracking information to a specific ip and port.

The current version has the IP and port hardcoded in Video.CPP:

RecvAddr.sin_port = htons(53525);
RecvAddr.sin_addr.s_addr = inet_addr("192.168.5.13");

If someone is interested I can change this so it's a parameter when you run the program. Otherwise you can change it yourself.

After you run the program an empty window opens, click on Help/About and it starts to track the brightest point in the image. For better performance turn off all automatic processing from the camera like True Light, True Sound, Auto White Balance, Auto Focus, and reduce exposure and brightness to a minimum. Your image should be almost black except for the LED in the helicopter.

Back to Helicopter Auto Pilot - Introduction

Helicopter Auto Pilot - Wiimote Communication (w-d)

Back to Helicopter Auto Pilot - Introduction
Download source code & runtimes: Heli Control

Wiimote communication (w-d)

To connect the wiimote with the desktop (w-d) I use a Bluetooth connection and the program GlovePie from Carl Kenner: https://sites.google.com/site/carlkenner/glovepie

Glove Pie is an easy to use program (I love that it doesn't require installation) that can read the wiimote and send information to other programs. It comes with it's own special scripting language.

Download my script called HeliWii.PIE.

In particular I'm using it to send commands from the wiimote to the laptop program. The first two lines of the script indicate the IP and port to send the information to:

OSC.port = 53525

OSC.ip = "192.168.5.13"

Change them to suit your environment.

Wiimote  controls:

• Home: Toggle IR transmission on/off
• B: Go from No Power mode to Manual Power mode. (led 1 on)
• Nunchuk Pitch: Controls power in Manual Power mode.
• Nunchuk Joystick: Turn Left/Right, go Forward/Backward.
• C & Z together (nunchuk) : 
• First time: Auto Power! Go from Manual Power to Auto Power mode (4 leds on)
• Second time: Emergency stop! go from Auto Power to No Power mode (leds 1 & 4 on).
• minus (-)/plus (+): Adjust drift left/right.
• (1)/(2): Adjust elevation set point up/down when in Auto Power mode.

Back to Helicopter Auto Pilot - Introduction

Helicopter Auto Pilot - Introduction

Download source code & runtimes: Heli Control

You may have seen my video on how to control a Helicopter using the computer:

Since many people have shown interest in this little project of mine I wanted to share more in depth information in this blog for those who want to replicate and/or expand on this.

Let's start with a schematic of the solution:

Notice I have two control loops, a manual loop h-w-d-L-i-h:

• (h)elicopter
• Human with (w)iimote
• (d)esktop computer
• (L)aptop computer
• (i)nfrared emitter
• (h)elicopter

 and an automated loop h-c-d-L-i-h:

• (h)elicopter
• Web (c)am
• (d)esktop computer
• (L)aptop computer
• (i)nfrared emitter
• (h)elicopter

Notice I use two computers because my laptop is not terribly powerful and also it's easier to place the web cam and infrared emitter in different places. You can just as well use one computer.

I'll be adding new sections and expanding existing ones according to your questions and comments.

• Wiimote Communication (w-d)
• Camera Tracking (h-c-d-L)
• Decoding Helicopter Protocols
• Helicopter Control Program (d-L-i)
• Infrared Emitter (L-i-h)