This is my passion project that will be in continuous development. So far I have created an algorithm that takes in microphone data, creates a wave file, creates an audio data profile consisting of the frequency values of a Fourier transform, and the MEL spectrogram of the audio. It then compares this profile with a MIDI database that matches the frequency values with their corresponding musical note values and returns the notes value.
Source wave is generated by microphone data.
Fourier Transform breaks down the source wave into its component sine waves.
In music terms, if you put in a G major chord, the Fourier transform returns the constituent notes of that chord (G,A,B,C,D,E,F#)
Even if you play the same note, no two notes are ever the same. Characteristics such as timbre, pitch, and intensity are easier to categorize in the format of a Mel spectrogram
A side note, this process of converting analog audio inputs into unique interpretations that can be used for data analysis, like Mel Spectrograms, is the same process used on submarines to detect and classify anomalies picked up by sonar.
This video is by one of my favorite YouTube channels "Smarter Every Day". The host, Destin, gives an amazingly intuitive example of how audio inputs are received, translated, and classified by using Mel spectrograms in the time stamp 4:20-5:58. I would recommend checking out the entire video though.
My goal with this is to create a tool that helps ease the transition from practical guitar inputs to music theory concepts. It is a well-known meme within the guitar community that we are not the most attune to music theory concepts. It is much easier to naturally play guitar than it is to understand the underlying theoretical foundations that help describe the music you're playing. My guitar teacher studied music theory at the University of Southern California and has always implied it is extremely useful to understand what you're playing and why it sounds the way it does. However, he tends to teach guitar from a practical standpoint rather than a heavily theoretical one. I took a music theory class in college to satisfy my art requirement and have been trying to deepen my understanding for years now.
Even the small progress I have made with this program has greatly contributed to my understanding of these fundamental music concepts.
I am currently working on a way to graphically display the notes and their relationships to each other using descriptions like intervals and scale degrees.
I believe that machine learning (ML) would be a great tool for this. I have been researching how to implement TensorFlow into an application like this. I am working on creating a database of .wav files labeled by their note. I believe that training a network to identify the MEL spectrogram of this audio data is a perfect application of ML.