Simple motion tracking

Background
The Pixy camera offers motion tracking thru color recognition. This works well with strong colors like glow-in-the-dark paint and such, but was not ideal for our project. We knew we had to modify the Pixy to get a more general tracking of motion.

To make it easier to work with, and to be able to visualize the output of the algorithms we decided to start working with the code base for Pixymon. Pixymon is the PC host for Pixy, that lets you change settings and view real-time motion tracking.

The idea is that once we get the code working nicely on our computer with Pixymon it will be possible to then port that code over to run on the Pixy.

The code
In the Pixymon codebase we added a method inside the renderer class to run our own algorithm on the video stream before it was shown on screen. This proved a good place to start as the video stream at this point in the code is broken down to individual pixels with corresponding x and y coordinates in each frame.

To avoid problems with color setting the RGB picture was first converted to greyscale by mixing all three channels like this:

gray = (R +G + B)/3

 -First, simple motion detection
We then created a simple algorithm that checks each pixel against the corresponding pixel in the last frame. Thresholding is used to remove small differences like uneven lighting.

If a pixel has changed since last the pixel is painted white. Else it is painted black.

This gives a fast and easy motion detection. The result can be shown in the picture below, and is a screenshot made when moving a hand in front of the camera:

Now this is a very simple process, and we will have to expand it with filtering and maths. But for now it's a good starting point for further work. 

The images are only 318x198 piksler due to a limitation on the transfer rate from the Pixy over USB. On the Pixy internally the resolution will be alot better.