homeprojectsblogbookscontactmima

The AI Band

Cover art of White Knuckles

White Knuckles

The AI Band

The AI Band is an ML model that generates entire songs from scratch based solely on the music genre and lyrics the user provides.

I used it to create two songs that I sold as NFT because money is always good.

Although now (in 2024) a model like this is trivial to create and common to encounter (Suno, Udio, etc.) — back in September 2021 it wasn't so straightforward.

skills

AI
Machine Learning
Transformer
Finetunning
Crypto
Python
React
Next.js

ML model story


OpenAI's Jukebox was released on April 2020, and I immediatly started playing with it and was struck by how awesome it was.


However, for some of my favorite music genres, especially hip hop sub-genres like industrial and abstract hip hop, its perfomance was somewaht lacking. So, I decided to finetune it in order to try to improve it. Since this was my first time working with transformers and finetuning a model, it was hard & confusing.


Eventually, after months of trial and error, I had a finetuned Jukebox that, to my personal taste, sounded better than OpenAI's original. I showed it to a couple of friends, and that was that.



NFT story


In September 2021, during the peak of NFTs, @suhail launched Tunes: a NFT collection of 8k GPT-3 generated song titles.


I'm a big fan of Suhail's background as a founder and immediatly saw the opportunity to use the ML model I had created earlier in the year to make a quick buck. Here's how it went down:


    —
  • I minted two Tunes NFTs — each one is just an image of a AI-generated song title: White Knuckles and Rough Runts.
    • —
  • Created the lyrics for each NFT song using GPT-3;
    • —
  • Used my ML model to generate the songs;
    • —
  • Sold both for a total of ~1.6 ETH.

I felt I could have made more songs, but 1.6 ETH at the time was around ~40 minimum wage salaries in Brazil, which was good enough for me.


In my mind, this was a very successful side project: not only did I learn so much during the process, but people also liked what my model produced so much that they spent on it, either because they loved the songs or (much more likely) because they thought they could sell for a higher value in the future — either case sounded awesome to me.