The Illiac Suite - Music and AI

This is an automated AI transcript. Please forgive the mistakes!

Welcome, humans. Welcome to a world which has decided on November 5th to live in a
reality which is not real anymore. A reality of lies and fake statistics and
statements. Welcome to a world that will have to face difficult times now.
And artificial intelligence is part of that as it shapes our lives and societies.
It gives us beautiful opportunities to create new music. It offers us ways to heal
sicknesses. It supports us in daily tasks. It is supposed to help humanity.
But it also destroys our nature with its immense use of energy.
It creates fake pictures, films and voices. It is used in the military.
It can support populism and it takes away the money from musicians who could live
by earnings quite okay in the last decades I always believe that no matter how big
the challenges are There are always humans who want to face them and make the world
a better place And I try to give them a voice here in the Iliac Suite
This is The Iliac Suite, a podcast on AI -driven music. Join me as we dive into
the ever -evolving world of AI -generated music, where algorithms become the composers
and machines become the virtuosos. Yes, this music and text was written by a
computer, and I am not real, but... I am and my name is Dennis Kastrup.
Episode 17 of this podcast. And this time I have for the first time two guests in
my little suite. There is on the one hand Syed Ilfan Ali -Milsa. He is a doctoral
student in computer engineering at the University of Tennessee. Together with
colleagues from Lee University he presented a few weeks ago Harmony Cloak, kind of
protective cloak that is Browning the music, with this technology, artists should be
able to manipulate their own songs before uploading them to the internet, so that
they are later unusable for artificial intelligence. And later on, you will hear
Virginie Berger, she is responsible for copyright issues at MatchTube, the French
software company based in Paris and Los Angeles, develops digital tools for video and
audio productions, However, it also offers a service to detect copyright infringements
of songs on streaming portals. For example, when a song uses a sample from another
artist without asking. Recently, Matchtune has been advertising that it can detect AI
-generated music. We will dig into that later. But let's start with Ziad and how the
that came up for Harmony Cloak. - My supervisor, Dr. Gianlu happens to be a musician
as well. So he plays a bass guitar and I play piano. So we both are kind of into
music. And then when we are doing like, they release couple of music generative AIs
like Music LM and there is Music Gen. Also there are a couple of like proprietary
generative tools that are generating music and they are trained on like normal
musicians music. So we are concerned that why like our music is not protected and
those companies are taking those music and scrapping those music from online without
any concern and we are like insecure that our music is maybe maybe fall into those
generative AI tools and now even if you think like I can train an AI model just
downloading music from YouTube. Downloading music from YouTube is like really easy
thing right and those musicians will never know that I used their music. So this
actually motivated us like, yeah, we should protect the musicians. - We all agree on
the fact that musicians should be paid if they are in a data set, right? No,
you disagree? Your argument is, hey, wait, but what about all these other musicians
who were inspired by someone before and they didn't pay them like, let's say Oasis
who listened to a lot of Beetles, and they didn't pay the beetles. It's one of the
big arguments of the tech companies. Yes, I get it, but back then when Oasis did
this, the money was always coming back to the musicians. It was a working chain.
You listen to something, you create something and you get paid for that. While
someone else listens to you and makes music and gets paid for this, right? This was
a closed chain. And this is now broken by the big tech companies who just let a
machine suck out the money. So I do understand why you might want to protect your
music. How does Harmony Cloak try to do that? We want our music to be available
online. But whenever you want to train any AI model on those music, AI model will
not learn anything. Because AI model needs AI model learns like us,
right? There should be some kind of a learning gap What do you know versus what is
new over there? There should be a gap and model will try to minimize that gap
That's how the model learns, but if there is no gap in between them the model will
think oh, I already know this music I already have learned that music. So what we
are trying to do is we are making music so that this gap is really low,
but without changing any music, but adding some kind of a noise onto the music. And
those noises, we cannot hear, but the AI model can perceive those noises. - Okay,
we have to get more detailed what exactly this is about. How does it sound? - We
are not adding any kind of like a specific sound, okay, because if you add anything
specific, you will be able to hear. So we are adding some kind of Gaussian noise,
kind of like a white noise. But what we are trying to do is like our air is like
if you take the psychoacoustic model, how our air perceive the noise, any kind of
sound. So it happens like in certain frequency, you need to have a certain amount
sound pressure for a year to perceive that sound. But if any sound exists below
that sound pressure, we will not be able to hear that. That's why we cannot hear
the dog whistle, because it's below our sound pressure level,
sensitivity. So we added those noises like that so that it's below those thresholds.
And also, Even if there exists a sound,
like suppose a certain frequency, there is a sound available, a loud sound, those to
be able to hear other sound, you need to have the other sound higher pressure than
that existing sound or at least equal to that existing sound. That's why sometimes
like if you play a really loud guitar, you'll not be able hear piano because it's
his pressure is really low and guitar is like high pressure equipment so instrument
so what we did is like we find out the most what I can say most loudest part of
the music where the most instrument instruments are colliding like the chord changes
or chord progressions where most instruments are colliding which are each other. So
there is the loudest sound available. We injected noise over there with a lower
sound pressure at those frequencies. So what happens? You cannot hear that, but AI
model will see those noises.
To see if Harmony Cloak works, the team around Sid had of course to test with
music. But where did the music come from? Did they reach out to labels maybe to
test with the music from the labels, the answer is more simple. - When we actually
developed this method, we had a like idea to collaborate with some musicians. But
during the development, we couldn't. We didn't get the chance to collaborate with
musicians. But now we are like trying to reach out musicians. Like our university
has a music school. We are trying to reach them out, hey recommend us some teachers
or some musicians or also friends and family and other things. So we are trying to
like make a community so that we can ask them and get their feedbacks from and
definitely we want musicians to be in that community like professional musicians so
that they can give us more better feedback than any other but while we were
developing this method we had couple of like you can say uh part -time musicians
like hobbyist they were participating they had we had a survey and there were like
we call them music lovers because they are not professional musicians but they are
hobbyist and they were like they gave us feedbacks We improved the method,
but now we need to collaborate with the professional musicians. I love an innovative
idea like this that could maybe save millions of musicians from being trained by an
AI, well, not the musicians, but their music, their songs. So far,
Harmony Cloak was trained on hobby musicians playing the best they can, so they
tested it with them. It is a good start, but of course, there's still a lot to do
for a perfect product. One thing is also Harmony Cloak doesn't know so far all the
existing genres. We tested our methadone like eight styles like rock,
metal, classical, jazz, blues and country music.
I forgot others. We tested on 10 genres and those genres were perfectly fine.
And we are trying to incorporate more genres and testing on more genres. The problem
with like testing new genre is like there are no like a lot of data available on
other genres that we can use to test our methods. So that's why we need the
musicians so that they can give us their music from their genre. Now we can test
because we like I come from a country called Bangladesh so we have some folk music
over Bangladesh and I contacted some of the folk artists that we can use your music
to test this model and they are perfectly okay and I tested with that it worked it
works but I'm not I'm I cannot guarantee you you it will work on the all the
genres because I haven't tested it yet So,
there are some insecurities still existing. They are early in the process. But if
you are interested in using Harmony Cloak to protect your music from artificial
intelligence, where can you get access to the software? - We are planning to like
release it as like a free to use software where you can use it,
make your music unlearnable, publish it. But we have like a,
we need to go through a lot of process before that. So right now our plan is to
release the source code so that you can make that model by yourself.
And by the end of next year, probably we will release the beta version of the
software. So if you're into coding, programming, feel free to use the open source.
If not, unfortunately, you have to wait until next year. The flip side of that
also, one year in the development of artificial intelligence is a long time,
to be honest. I give you a personal example. So many things happened since I
started this podcast one year ago when the Iliac Suite began. When I think about
what I said last summer in the first episodes, I feel at some point not really
ashamed but somehow uncomfortable because I would not say it that way today.
My opinion changed, the way we use AI changed and the discussions about AI changed
in 12 months. I have the feeling it was a totally different life back then.
So how does harmony cloak protect us from all the changes that might come and also
from all the air eyes that are just waiting to break the system? The thing is
that's the question about the robustness of our method and we cannot guarantee that
yet because One thing is good right now. Maybe someone will come up with a
brilliant idea or solution to break our method. So what we are planning to do is
we are planning to have continuous updates.
If there is some work that can break our method, we'll figure out how we can
prevent that as well. So if something comes out breaks our method we will figure it
out and release it because there is no like a one stop solution there's no way you
can just use one idea and it will work every time. Let's cross fingers that they
somehow keep up the pace and adapt quickly to the changes because if they do so
the software could really be interesting for musicians and some companies. We were
thinking about from like it would be really great that some like music platforms
like SoundCloud or other platform who actually shares people's music, right? It will
be great if they reach us out so that they said, hey, we want to incorporate this
in our website whenever someone uploads their music. we will add like protect that
for them. That will be great actually and we hope someday they like reach us.
- Thanks Ciot for that insight view into Harmony Cloak. By the way, there's already
another software existing that does kind of the same thing for images. So if you
want to protect your image from an AI that scrapes it from the internet and uses
it for that training, you can check out the software Glace. It kind of works the
same way, but it's a little bit different of course because it's not for music but
for images. So for me that idea from Harmony Clock sounds great and I'm wishing all
of them good luck and maybe it can protect us a little bit.
Let's come to the second guest of the show. A couple of weeks ago I read the news
that Matchtune said it had cracked the generative AI model Soonu.
Meaning it can detect if a song was generated by Soonu or not. There were some
other companies before that who said they can do that, but Matchtune seems to be
very successful with it, so I reached out to them and talked to Virgeny Bergé about
that. So actually, I'm the Chief Business Development and Rights Officer at Matchtune,
so I'm in charge of, and especially I'm working on Covernet, so Covernet is our
copyright infringement detection tool. It's larger than AI, and I'm in charge of
working with all rights holders. So it can be a media, it can be publisher, music
composers, you know, record labels and to make sure that we can help them with the
identification of copyright infringement on DSPs. What we call DSPs is the platforms
like YouTube 35. So copyright infringement, it can be like, you know,
the modification of the audio. So, you know, it can be pitch, modification of the
pitch, tempo. It's many things that we are seeing on TikTok. Now, slow down, and so
it's really something we are working on. On the covers side, it's mostly for
publishing music. And on AI, so AI, we have two sides.
We have the deep fake sides. So it's voice cloning, but it's also when you are
using the music of an artist and you add your voice so it can be the opposite.
And we also have gene AI, generative AI, so it's music created from scratch using
you know the new platforms like Suno, your music NLM, Bumi and everything. So I'm
in charge of this product actually. How do they do that? How do they recognize AI
generated music? It's not, I won't say it's a secret, but we can't say, we can't
tell, it's like virus and anti -virus. So it's like, you know, keeping updating all
the time. So what I can say is that, so the music, really we can identify the
music extremely quickly, it's really a question of, you know, seconds for us.
And so the technology can detect AI composition with 95 % of accuracy actually,
so we have 0 .01 false -positive and for Suno, because it would be different,
but for Suno we have 4 .99 false -negative,
what we call false -negative, is that when we are not able to detect if a song is
there generated or not, which means that we'll say It's not generated by AI, but at
the end, it will be generated by AI. So we have 4 .99 % for false,
it's false negative. But we think that it's much, much, much better to have almost
zero false positive, which means that we really never say that a real song,
it's an AI song, but from time to time, 4 percent, we can say that an AI song,
it's a real song, it's because now with the technology that they have, so they can
mix, you know, real music or real voice with AI. So sometimes,
you know, we can be a bit like, it can be a bit tricky, but we have an accuracy
of 99%, so we think it's not too bad. And actually what we are doing is that we
are analyzing patterns, trace, in the file,
in the AI music -generated file, which means that each company, like Suno Audio, when
they create a file, AI file, they leave like a trace, a pattern,
that we can recognize. And it's something that they can't change it, they can't
suppress it, they can't dilate it. It's how we identify the files, it's really
because they leave a trace, and for us it's to track the trace, you know,
it's like bounty hunter actually, it's really like, you know, being able to find it.
And for Uju actually, we are almost at 0 % false positive and 0 % false negative.
So Uju is a bit different from Suno.
Pretty good numbers. I'm impressed, but I have a question. How do we do this in
the future? I mean, the better the systems get, the harder it will be to detect
music that was fully generated by an AI. And even if they detect it, and always
will be, the possibility that this song was not created by an AI even if it is
detected like that, right? What I want to say, it is a cat and a mouse game.
Yes, it would be lovely to know what is AI generated, but on the other side, a
company can claim to have cracked the code and we must believe what they say. It
is a trust thing. We're getting into an age of trusting more because the more and
more music is out there the more and more it will be totally confusing to know
what is right what is wrong what is real what is not real but still to have a
tool like this is these days very interesting for different kind of fields we are
working from music libraries so I can I can name you know for instance so we are
working with APM music so it's the biggest music library in the world and so it's
on the production music side. When I say production music, it's synchronization music.
You know, it's the music behind commercials, for instance, video. So we are working
with them on, well, detecting copyright infringement and now GNI. But we are also
working with an I can't name because it's a race between, you know,
GNI company and companies, you know, NDSP is a low -wing copyright infringement,
and those are our clients. So that's why it's like, you know,
a race, it's like virus and anti -virus actually, it's exactly the same. So we are
working with big majors, very big majors actually. We are working with publishers,
music publishers, and we are are working what we call CMOs, Collective Management
Organization. It's the equivalent of GEMA in Germany or you know so can in Quebec.
So it's mostly our clients. So we're also you know approached by managers to work
with them but we are always very very careful with you know who owns the rights.
So we are mostly working with rights holder, which means that we won't track for
someone who doesn't own the rights.
So yeah. But you know, we have requests from people, you know, asking us to track
for these artists or these artists, because we are also able to detect where the
music is not monetized. So that's why we are very, very careful, but we're mostly
working from production music to
us two record labels to write an organization. So what is the deal in the end?
We know the music is made by an eye. What's the problem? And then? Or before? Or
in the future? We use music without consent, without licensing and without
monetization. So now we have to find a way to understand, I mean,
to monetize this music. So Maybe like with licensing on the input or maybe to find
a way to license the output. So, you know, the creation created from,
you know, Suno and Udyo. But no, no, it's the thing is that tracking first because
there is a lawsuit, but it won't be, you know, it won't start, I think, between 25
something like that. So I mean, in the meantime, we can have music absolutely
everywhere. Maybe also a way to track the music. So maybe you have like a pixel,
you know, something within the catalogs, it's, it's, there are plenty, plenty of
questions and people are working on it now to detect the music and to detect the
output. I know that some companies are working on detecting Exactly from the output,
you know, what type of music they used, which is, you know, another work. It's not
what we are doing because it's not exactly the same thing. It's very hard,
but it would be extremely important for us to know, okay, where come from the
output and how can we divide the licensing, but it's plenty of questions. But from
What's your worldwide point of view? I mean, from a right point of view, no, we
know. No. And I say yes to that, because I agree. We are right now in the process
of hopefully finding a solution for the problem. That already starts with the
question, is it a problem? A lot of AI companies in the States say no.
They say music for training is fair use. And the labels that the musicians say Yes,
it is a problem, it is not fair use. And I am on the side of the musicians here.
We have to find solutions to pay them. People are having ideas, they are making
suggestions. Like for example, the GEMA, the German equivalent of the RIAA in the
States. They take care of the musical and performing and reproduction rights in
Germany. They suggested recently a model in which 30 % of the profit of a generative
AI company goes into a pool, which then will be distributed to the artists.
Stays the question, who will get how much? Is it the amount of music in there or
the precious amount of music, like songs from stars and who defines this anyhow?
Like do we look at this to pay the money to those artists or does it in the and
count how much music was generated in the style of someone. So many open questions,
but maybe we do it step by step. What we are doing and how we are working, it's
also to help the industry to define what they want to do, to fight against, I
mean, to fight, to fight or to license or to find a solution or, you know,
to discuss with Janiyai companies, but I agree with you, it's now that we have to
define what we want to do with it and how we can monetize and how we can license
properly the consent is also very important, opting out if an artist wants his music
to be trained or not, so it's really the basis and it's something that we can do
now, so no, I agree with you. For me monetization, it's not fighting,
suing, yes, but it's just a way at the end to find a solution about how we will
license music, how we will pay for music and most important consent.
100 % agreed. I will keep you updated here in the Iliac Suite about What will
happen in the future? I can promise you it will not be boring. So stay tuned.
Thanks Virgini Berger from Matchtune for your insights and also Siert Ilfan Alimisa
for talking about Harmony Clock, both of you. I would like to say we need people
like you, don't give up. Thanks for listening humans, this was episode 17 of the
Iliac Suite. If you want to write me, give me a feedback, write to mail
@theeliaxsweet .com, mail @theeliaxsweet .com.
After 17 episodes, I finally made it to install an email address for me,
mail @theeliaxsweet .com. You will find all the information about this episode also in
the liner notes. That's it. Hope to see you. No, I'm not gonna see you hope to
hear you. No, I'm not gonna hear you. I hope that you will hear me That's how it
is next time take care and behave