The Brain

The Brain of the Robot was developed by Rafael Mizrahi, a GarageGeeks made-man and research manager at Feng-GUI the artificial vision lab.

GuitarHeroNoid source code home page at Google Code service.

Each song is presented on a set of five columns, resembling a real guitar fret
board, that scroll constantly towards the player. The five columns correspond to
the five fret buttons and appropriately colored notes appear in these columns.

Notes detection area

Detecting the notes could be accomplished by using several approaches to detect
moving objects; most of them might not be fast enough to implement and use. So
we came up with this idea: Looking at the game you can quickly realize that the
important information, such as the plates is brighter than the rest of the
image.

HSV color space

Representing the color of the pixels in each trapezoid from RGB as HSV (Hue,
Saturation, Value), also known as HSB (Hue, Saturation, Brightness), and
defining a threshold (Brightness Threshold at the properties dialog),
somewhere in the middle of the Brightness value, gives you a binary
representation of the pixels inside the trapezoid. Now, when the trapezoid is
filled with enough white pixels, there is probably a plate over there.

Binary black and white image

We connecting the game video output using a capture device into a computer. A
live video streaming filter captures the video frames as images and sends each
image into the image processing part of the program that detects the notes.

one Robot Server and one Image Processing

Delays, Delays, , , Delays
To play a note, the player must hold the correct fret button and press the strum bar. After playing and watching the game, you find out that the PlayStation add another delay to the equation. Pressing the frets buttons is recognized by the PlayStation game within some 100 milliseconds or so. Having this delay along with the delay it takes for the strum solenoids to go up and down, I realized that sending fret notes together with the strum action is not possible. I divided the protocol into two main actions:
The first, at the area of detection, pressing one or more of the fret notes and lift up the strum. The fret notes information is added into a FIFO queue. The second, after about 250 milliseconds (Strum Delay at the properties dialog), pop the fret notes from the queue and send them to the guitar with a strum down.

A nice TODO is to detect the BPM of the song at the first seconds of the song, and adjust all those delays according to that.

Sending notes to Robot Server

Multiplayer is split-screen. In a "dueling guitars" fashion,
two players tackle segments of the selected song. Unlike other modes, it is not
possible to fail a song in multiplayer, but scoring dictates that one player
will generally win.
Just by moving the area of detection to the side of the screen, using smaller trapezoids, all parameters in a different configurations file (guitarhero.ini), and there you go, you can play with or against the GuitarHeroNoid.

Multi Player mode