The Illiac Suite - Music and AI

This is an automated AI transcript. Please forgive the mistakes!

Hello, humans. Do you recognize that noise?
If so, you are either living in Austin, Texas, or you just went to the South by
Southwest. The birds are making that noise every evening in the streets in Austin
before the humans take over. I tried really hard to find out what they are called,
but even my AI app that usually recognizes every bird pretty good was not able to
tell me the name. If you know the name of these birds, feel free to send me an
email. I would be more than happy to know what they are called. Let's get this
started with the South by Southwest edition of this lovely podcast. This is the
Iliac Suite, a podcast on AI -driven music. Join me as we dive into the ever
-evolving world of AI -generated music, where algorithms become the composers and
machines become the virtuosos. Yes, this music and text was written by a computer,
and I am not real, but… I am and my name is Dennis Kastrup.
So it's over. This year's edition of the South by a Southwest and what stays
probably this moment?
The video emerged on social media of the audience, loudly booing a conference sizzle
reel that featured several industry leaders speaking positively about AI.
I was not there. It made the big news. First time I saw it, I thought,
well, you are right. I don't agree with all of the positivity around AI.
Not at all. We are probably heading towards a society where it will be possible to
control the people with artificial intelligence because we will not be able to decide
what is real and what is fake anymore. I seriously think it is the biggest threat
for humanity right now. Don't get me wrong, not the AI is the problem,
the humans are the problem. They will use it. We are the problem.
If humans don't change, the AI will not change us. And when Peter Deng of OpenAI
says in the video, AI will make us more human, I cannot more agree.
But we should stop thinking being human is a good thing. Look at us. What have we
done throughout history so far? Humans can be gentle, Yes, and take care of each
other. That is what in our definition human means, but humanity starts wars,
destroys the planet and lets people starve while others have so much food.
That is human, and yes, if an AI makes us more human, then it will unfortunately
be cruel. So let's start with us as humans to change before we blame AI.
So how do I find a bridge now to music and AI? No,
I just make a deep cut. There's no bridge. I believe that with the help of AI in
music though, we will create beautiful art. And I also believe that there is enough
money out there that all the musicians can make living from that. But as we are
human and therefore also cruel and unjust. It is right now not well distributed.
It is the moment right now to change that. The money goes to the AI companies that
steal the creativity of artists. If there is one major problem to solve these days,
it is exactly that, pay the musicians and let them also play with AI. The solution
is very complicated, very complex, but I'm Sure, one day we will find a good
solution. And always remember, human art will never die. This is for sure.
Humanity will make always beautiful art. AI will just be a part of it. That's my
little speech for today. Coming back to the South by Southwest, I was there for one
week and talked to some AI music players. I went to see some panels and experienced
some innovation, hopefully. I collected the interviews and made a little recap out of
that.
Generative AI is everywhere and I have also talked about it here in the podcast
many times at the South by Southwest There were also some companies in that field
that were trying to get the attention of the visitors with their generative AI I
did not feel they were presenting anything new. I have to say I thought they were
Trying to enter the space where they see a lot of money to make for example in
the expo I found POSA labs. My name is Hitek Lim and I work for POSLF Music AI
Generating Startup in Korea. We generate music and we're a company that focuses on
creating instrumental tracks for content creators and people who are in need of
copyright free music. We have four AI vocalists. That technology is not actually
created by POSLF. It's the technology by a partner company that we have back in
Korea as well. So They focus on AI vocal synthesis and like TTS,
SVS and what not. So mostly on vocal and we do the instrumental part.
So there are two methods of creating music. One is if you have a song that you
would like to make similarly you can upload that music file. If you have obtained
it legally and the AI will not be using that to train this model but it will only
extract certain metadata to send it to the model and the model will use that to
create all the tracks from scratch and the other is you could just pick on a
template we call it templates basically you just pick the genre and the mood and
then from there the model will create the the song for you. Currently where live is
in the open beta phase and we will be closing that open beta phase and officially
launching mid -april from then we um users will be charged will be using tokens to
create um the songs and the tokens aren't free you have to pay to use the tokens
and there will be a certain license purchasing fee per song, if you like that song.
My question these days, when it comes to generative AI, is always what datasets did
they use? Did they treat the musicians fairly? Did they steer creativity? Chaitay
Klim says they are working on that in -house. Our datasets come from internally,
we own all we have a proprietary in -house developed data set so we have a team of
around 30 producers and artists who create the data sets and help the engineers
actually design the model itself and so that's where the data set comes from.
Our model uses, they create music samples for each individual track and then each
individual song form and then we put it all together in the very end so the
musicians and the artists In house art musicians, they are creating music samples
that would help the model be able to create those samples Of course, we have to
believe them because I have not seen the data set and this is what happens a lot
of times with data sets We don't know what's in there But if you want to be sure
and generate ethically fair -trained music with an AI you can use sound full I
talked about them the last time here in the podcast the music in this episode is
by the way way generated by Soundfull, they are part of a fairly trained platform
that gives away licenses to AI models that only use datasets in which the music is
in there with the consent of the artist. I support that.
One of the oldest use of AI in music is as a tool of production indoors digital
audio workstations. I remember Lander as being the first one to use artificial
intelligence to get a final output of a song that sounds great, so mastering as
time passes by and computers get faster other companies joined in that field at the
South by Southwest. I was sitting on a terrace enjoying my drink in the sun when I
met Christian Schultz, born and raised in Norway. I'm the CEO and co -founder of
Master Channel, one of the leading AI mastering and audio enhancement companies in
the music industry. We use a concept called reinforcement learning, where we have
trained the system with the knowledge from different engineers, from different
producers, and we encode their expertise into mathematical equations. So when you
upload a song, we create a couple of thousand different versions, and then the
system gives it a reward or a penalty based on the decision it makes. So it's
like, it's similar to how you train a child to walk, that it has to try to walk
and then fall down and try to walk and fall down and eventually it managed to
walk. That is the same process when you run through a song. It's not generative AI
or it's more about preserving the originality in the song but making sure that it
sounds as good as possible regardless of where the song is being played, and also
what streaming platform is being played on. So it has like the right quality
standards needed. So the quality sound can be that it doesn't peak when you play it
in a club, that the bass doesn't sound muffling if you play it in your car,
and all of those things. So it doesn't really like B,
It's not A and B, but it's more about looking at the whole song individually and
then working on it. It's similar to how a mastering engineer would do it, but just
fully auto -automatic. So Master Channel does the mastering for you. And one
interesting feature is you can use the presets of an engineer called Wes Clark. He
has done pop, dance, urban songs and has mixed records for Beyoncé, Sam Smith, Rudy
Mantle, Ed Sheeran and Jean Paul, just to name a few. What we also have is because
we cloned a Grammy Award -winning engineer, so we have his take on what is a good
sound, similar to how we have our own air -mastering. We have also cloned his sounds
whenever a song is being uploaded, his kind of style or mojo is being applied to
that song. Interesting thought that you can pay for a clone of an engineer to do
the work in their style I think. So check out Master Channel if you're into music
production. Let's jump to another continent and have a look at K -pop bands. In the
expo of the South by Southwest I found an interesting booth of the Hype Corporation,
the South Korean company behind all that K -pop music, well not all but a lot,
which has taken the music world in the last years. For example, BTS, the K -pop
sensation. The problem for them, their stars have so many fans from all over the
world who do not understand them. Of course, they would love to talk to them in
their language. In order to speak to their fans and connect better, Hype has started
an AI that translates the voice of the singers in different languages. I'm Sebum Tak
from Hype company. So I'm here here for say our AI top contents in South Spice.
Actually South Spice is the first time for our company to exhibit some kind of IT
technology.
And this content is concert film. So it's originally English version of this content
concert because tomorrow by together performed in LA last year so we filmed it and
we made that their no more voice no more meant make AI version in Spanish so the
the the technology team is We extract artists' original voice from original video and
then we make that into voice gene and then we dub with special voice actor in
Spanish and then both of them mix together and integrate and then we made Spanish
version AI .content. So they took the English words and a Spanish speaking person
says the same thing in Spanish Which is then changed to the original voice of the
member of the group But why did they choose these two languages and not the
original Korean? the mouse Working when they speak the mouse is similar English and
Spanish, but Korean is very very different when they speak.
The mouse working is not the same, so we prefer English
into Spanish. We do that, this AI job,
it's for fans. Fans can enjoy our artist's contents all,
are more lively and yeah that's why we do so when they sing we don't use this
technology because fans like listen to original voice singing actually I find that
information interesting we have the technology now to overdub voices in different
languages also singing It's working, yes. I mean, we can see it in voice cloning.
It's going good, I think, but Hype decided not to do that because the original is
still more loved, original, and better, which is the proof that whatever happens with
AI and music in the future, you cannot beat the original, the real thing, I guess.
And this is good news for humans.
One subject that was present in a lot of panels during the South by Southwest was
also the question about copyrights in artificial intelligence. I have talked about
this many times before here in the podcast and I was very happy to see the panel
AI Copyright, 471 billion melodies thwarting human creativity with Damien Reel,
who is lawyer, coder and technologist. He is the person behind all the music,
a fascinating project. I talked to him. I was hired by Facebook to investigate
Cambridge Analytica, so I spent a 14 -hour day at Facebook and during the evening
when I went back to the Hotel Lounge, I said to a buddy, who is also one of the
best programmers I've ever met. His name is Noah. I said, Noah, you know how we
can brute force a password by going A, A, A, A, A, A, B, A, A, C? He said,
yeah. I said, what if we could do that with music? What if we could go da -da -da
-do, da -da -da -ray, da -da -da -da -me, da -da -da -fa, until we mathematically exhaust
every melody that's ever been and every melody that ever can be? He said, oh yeah,
let's do that. So that night we had finished a proof of concept where we created
about 3 ,000 melodies. And at this point, now sitting here in 2024, we have now
generated about 471 billion melodies with a B. And that is arguably,
we've mathematically exhausted every melody that's ever been and every melody that
ever can be. What fascinates me about this project, I have thought about this many
times that one day all the melodies of the world must have been written somewhere
on this planet. And Daniel just told an AI to do it. Fascinating. With this
knowledge we started talking about copyrights in an era of AI of course and he as
a lawyer is a specialist in this. In that context check also out his TED Talk.
I highly recommend it. We as musicians should fight for what is ours. That is the
most important aspect of it. But really the question that we should ask ourselves is
what is truly ours that is no one else's. And then, that's thing number one. And
thing number two is what is the communal collective that we're all drawing from?
We're all drawing from the same wellspring of potential melodies. We're all drawing
from the wellspring of potential chords. So we, for decades, have never been able to
copyright a chord structure. We just can't because there are only so many chords.
You also can't copyright a drum beat. We just can't because there are only so many
drum beats. So I wonder if we also would say okay maybe melodies are amongst that.
Maybe we cannot copyright a melody because there are only so many melodies in the
world. So if we now set that aside and we say okay now if AI is training on my
music, what is original to me and also that is a monopoly to me,
that cannot be a monopoly to anyone else. And maybe a melody is not original to
me. The way that large language models work is they extract the ideas of what is
happening. So they extract, you think about two dimensions, the x -axis and the y
-axis, it's plotted on two dimensions. And then you add a third dimension, the y
-axis, the z -axis, and then now you have 3D. And now think about 4D, and you
can't, because it's impossible for our brains to think about 4D, all right? Now
think about 12 ,000 D and that's the number of vector spaces that the training is
happening on. So within those 12 ,000 dimensions is Bob Dylan -ness and Ernest
Hemingway -ness and Pablo Picasso -ness. So the ideas of those songwriters and those
artists and those writers, they're all somewhere in 12 ,000 dimensional vector space.
So when then I say write a song in the style of Bob Dylan. It's going to that.
And think about if Bob Dylan could then sue you over someone writing a song, a
human writing a song in Bob Dylan. And he couldn't, obviously, because if he could
sue someone for singing in the style of Bob Dylan, every singer -songwriter since the
1970s could be sued. So really, the question is then if he can't sue a human over
that, can he sue then a machine for doing the same thing, taking the idea of Bob
Dylan -ness and and then putting in the output. And this question remains until now
unsolved, because... It's too complicated and also very expensive to be able to hire
lawyers, very expensive. So if I'm a label that's thinking about sewing, do I pay a
million dollars to be able to get back $20 ,000 or $50 ,000? And similarly, if I'm
a musician, I'm more likely to get sued by a label than I am to sue someone else
over my song. So really, if I'm an artist, copyright is doing me more harm than it
is good because I have a target on my back for every new song that I have because
if I happen to make a big a label then will say now this is making enough money
that it's worthwhile to sue over so so really is copyright protecting artists or
harming artists and I would argue maybe it's it's not protecting us as much as we
think this interview had an interesting twist in the end is copyright protecting
artists or harming artists. What would you say? You can write me to mail
@deniscastrup .com. I am curious to know what you think about that. My opinion is it
is definitely harming art in these days of artificial intelligence because it is too
complicated. I think that we need a new copyright law that regulates things
differently. The old one just does not fit in our modern times. And if I could
make a bet in 20 years from now, just to be safe, I say 20 years, the law will
be different.
So, what do we do with all of this right now? AI is here,
and it will stay, or as one panel at the South by best put it, the new normal AI
in music creation. So it's normal to use it. Here is an excerpt of that discussion
with Danny DiCiaccio from Splice, Dia Elal from Soundful,
who was a guest on my podcast the last time, and even Bogart from Seeker Music
exactly in that order they will speak. They all respond to the question of Ethan
Milman from The Rolling Stone. - What excites the three of you about, I mean, beyond
like, as far as what's coming up soon, like something that you find very promising
or exciting about, you know, your tech and what artists are gonna be able to do?
- What comes to mind for me are two things. One is mobile. I think mobile is going
to be a revolution. And I think music tech has not done a very good job at
committing to mobile. In the last number of years, there've been like some apps that
got close. The GarageBand had a cool iteration, Machine had a cool iteration,
but no one really did it really, really well. And what mobile is going to give us
are different points of view across the world. There's like, I I think I saw stat
recently that there's like more smart phones than people, like they are ubiquitous at
this point. So having it in a mobile, mobile capacity is going to unlock different
points of view, more diverse opinions, more like cognitive profiles in music.
That's really exciting to me. The other thing I think about is and that I'm excited
about, but that also heartens me with this conversation is that I think people are
getting more and more excited about things like the provenance of certain sounds, the
paratext, the things that are going on outside of the sound,
the person that made it, how they made it, when they made it, what else have they
done? What's their lived experience that brought that to the table? I think,
surprisingly, I see people asking those questions and wanting to know about that.
Yeah, I think what excites me the most is really figuring out a way in an ethical
way to work with the artists and producers, not just to bring him closer to the
fans, but also to give them a utility to use in their camps, to give them a
utility to use in other avenues like gaming, Web 3,
etc. that they haven't been, you know, do you just don't have the bandwidth to
actually go in and do, but now we just gave them that opportunity. The second part
is really how it is exciting and it is terrifying about how we can,
every creator will become a super creator, meaning every creator will become a music
producer, a songwriter, a video creator, a graphic designer,
that's very exciting. But the part that scares me is how to, while doing that,
just making sure that we're making the right moves from paying the rights holders,
etc. But I will say to that is like, if everybody's on the same level,
you still have the cream that rises at the top, you still have to be great at
creating in order to get the the best out of it, right? Yes, anybody right now,
forget AI, anybody right now can create anything and put something out that sounds
like shit, right? And so I think even if you have these tools, you could still
create something that's like low bar, right? And in order to be great,
I think if it becomes noise, you still need the stuff that rises through the noise.
And so I think that people who are already inherently talented who embrace it will
become the people that are the leaders in it. And I also think that the other
thing that excites me is because I love what this means for the creator community
and for the productivity and for the inspiration and innovation that's coming with
it. I think there are also really smart people who are dealing with issues such as
name image likeness. I think that talking to the copyright office,
trying to educate them on what policies should be created around this and hopefully
educating the middle class of music creators on ways that they could get involved
with AI and embrace it and not be as fearful about their careers.
I mean, unfortunately, there is certain parts of music creation that can be easily
replicated by somebody pressing a button. But, um, and I would just,
you know, for me, I just try to encourage, like, how can you embrace that? Like,
can you make production music or production house? Like, how can you make AI part
of what you do instead of letting AI replace what you do? The most used phrase
when it comes to AI and music and also in this podcast, make AI part of the music
and don't let it replace you. I would like to finish this podcast with this quote,
because it is a perfect ending for this South by Southwest episode, but one thing
has to be said also, unfortunately. Yes, I did feel uncomfortable with the US Army
as the biggest sponsor of the festival. It just feels weird when you go through the
expo and look at great innovation with colorful and funny bands and funny music and
funny images. And then the booth of the US Army shows you how to destroy an enemy
with the newest tech inventions. It is weird. Choose your friends wisely.
That was the South by Southwest Iliac Suite Edition. Follow me, share me,
share the podcast. Until the next time, humans, take care and behave.