𝄞watch, a chord detector that runs on Chrome

18 Jul 2024

As a musician, finding the chords for a song is something I do extremely often. Whether I’m arranging for acappella or simply replicating the backbone of a pop song on the piano, knowing the chords of a song are crucial for capturing its essential colour - and provides a contextual springboard from which more musical fun can happen: reharmonizations, new melody lines, mashups, and more!

Most times, you’ll be able to find the chords for the song you want online on a website like Ultimate Guitar. But these chords tend to come with a set of issues - maybe they are simplified, or they weren’t transcribed accurately. Sometimes, there’s a song that doesn’t have any transcription at all because it’s simply not well-known enough!

In those moments, I wished there was a convenient tool to help me detect chords instantaneously. That is what spurred me to build 𝄞watch, a Chrome extension that detects chords. It follows in the great tradition of lightweight Chrome extensions like Colorzilla and Transpose, both effective semi-automated tech solutions which automate away the non-essential, thereby allowing the user to exercise their expert judgement. I wanted my extension to be able to instantaneously capture the key of a song, just as Colorzilla could capture the hex code of a colour. I also wanted my extension to perform digital signal processing natively within Chrome, much like Transpose could perform key changes on music on the fly.

Introducing… 𝄞watch!

At Last, originally by Etta James, performed by Cynthia Erivo

Play music from any audio source on a browser tab, and 𝄞watch will near-instantaneously detect the musical chord for you, whilst also displaying the relevant notes that belong to that chord, so you can easily replicate it! For example, if C Major is detected, then C, E, and G will appear on the on-screen digital piano. Finally, you can also play and pause from within the Chrome extension, avoiding the need to exit the extension view every time you need to pause/play.

Architecture

This app was served to the user as a Chrome extension, which by construction uses HTML and CSS for the front-end design, while using Javascript for back-end scripting. Because one of my goals was to optimize the audio digital signal processing in a lower level language (in summary, I perform Fast Fourier Transforms, bin frequencies into chroma, before minimizing similarity to established chord-chroma profiles) - I coded all the backend processing in C++. However, because C++ would traditionally be run in an executable file, I use WebAssembly to transpile C++ into browser-readable assembly code. The Javascript backend calls the .wasm code for compute-intensive tasks, while maintaining the broader averaging functionalities needed to assert more stability in chord detection.

The happy result of choosing this architecture is: firstly, there is minimal installation required. Once the extension has been downloaded and installed into one’s browser from the Chrome Web Store (and it is a tiny program), there is no further configuration required. Secondly, the extension functions standalone, that is to say it does not need to ping a server to get audio processing results. This is great because we don’t need to spend resources on maintaining a dedicated server, and we are also saving on latency costs, assuming that every user’s computer and Chrome browser is competent enough to execute the .wasm code produced. Thirdly, chord detection happens live and near-instantaneously. Because the underlying C++ is so speedy, there is no need to do any patch-up work on the backend with respect to time alignments, and musicians can get what they need immediately, which is crucial for their workflows. No one wants to be bogged down at the boring, grindy work - we want to give them as much time as possible for the real creative work.

Conclusion

There’s still a lot of work to be done on the DSP front when it comes to more precisely extracting chords. Clear-cut chords are easily identified correctly, but real songs with complicated chords are a much harder task that my extension does not yet excel at. Version 0.1 will soon be available on the Chrome Web Store for free.