Automated Piano Tiles player

At the Game Jam on the 5th-6th of February at University of Applied Sciences Salzburg, we developed an automated Piano Tiles player, based on image recognition.

With the software you can setup ROIs which trigger a serial send to the Arduino, if the brightness is changing within the regions of interest. The Arduino triggers a relay which switches Ground on to a wire. The end of this wire is placed on a capacitive touchscreen. When Ground is applied a fake touch input is generated.

 

Arduino scheme if you want to try it yourself: