AnyJAMS

This is a Flask + MATLAB app. The game will give the player a song to play, and the player must play in real time alongside the song that is being played by the game. We use the pyaudio library to retrieve the audio frames from the user's microphone, and use a TCP socket to send the frames over to MATLAB. In MATLAB, we process the user's audio frames using an advanced mathematical algorithm. At the same time, we also process the mp3 file that is played to the user. We then run the xcorr algorithm to compare the two waves together. We get the two resultant vectors of amplitudes and run another advanced algorithm to determine how close they are to each other. This is constantly running throughout the song and the player is given feedback in real time throughout the song. The frontend was created using Flask and HTML/CSS/JS.