I’ll be honest, I’ve been waiting for this moment since I was a kid.
I always dreamed of having intelligent machines that would push me to develop other ways of thinking and give me other means of creation. I’m a Star Wars fan. My life changed when I went to see it in late October 1977. It was obvious my future would be technological. Right after spending my teenage years discovering computing with the first home machines (the Commodore PET, the Atari 400, the Apple IIe), Philip K. Dick blew my mind in my twenties. Non-organic life… Artificial intelligence endowed with feelings. That was my thing. So yes, it took years before AI became accessible to us and the approach turned "friendly."
In 2016, I was playing around at a distance with GANs, style transfers, but I didn’t have my hands on the machine. I didn’t know how to use a PC because I was on Mac, and even less how to enter lines of code to run these tools. I’d regularly pick the brain of my buddy Eric (Eric Debeir, opinion piece in this issue — Ed.) so he’d do it for me… It was on September 5, 2022, very, very early in the morning that everything tipped over. Scrolling like a sheep on my Instagram, I stumbled on a crazy image with the hashtag #discodiffusion… I searched, I dug through that wretched network to understand what we were talking about, and I landed on #StableDiffusion. But what exactly are we talking about??
When I opened Dream Studio (which had just been put on the market the week before by Stability AI), I immediately knew I was going to switch over to this technology. So we’re talking about producing images by writing the idea… Brilliant! So I started producing thousands of images, to understand how Text2Img works, prompting, the key to the future. Writing words to produce images and sound, and soon for every task that passes through a computer interface.
AI suddenly brings two very important things to light: a new form of limitless creativity and an ancestral communication through words.
Then I got a better grasp of how this interface worked and focused on what I liked most, animation. My first tests consisted of taking a few seconds of a video I’d just shot, turning it into a sequence of images, running them through Dream Studio one by one, then bringing them back into my editing software to make a video entirely reinterpreted by Stable Diffusion… a super long process but satisfying, even though consistency wasn’t there. The images were all a little different, creating a permanent vibration bordering on unbearable. But a group of little geniuses had just developed Deforum, an animation software using Stable Diffusion that ran on a Jupyter Notebook in a Google Colab… so many new things to discover, but above all so much to learn. Before running Deforum, I had to learn this geek universe, these pages of scripts, this coded language that’s totally wild for a novice but so exciting. Token, Seed, Transformer, Tensors — these words didn’t ring any bell in my mind. After a few weeks of pain and disillusion, I finally put out my first video, and I’m absolutely stunned by the result.
Deforum works in Text2Img with sequences of prompts attached to keyframes. Which means the video evolves and can follow the ideas you place at precise moments… generative cinema makes its appearance. A few terabytes of video were generated, hundreds of hours of Google credits spent, before putting it all to use. At that point, I was handling the music for Fred Hazan’s series "Besoin d’amour" (OCS). He was looking for someone to do the opening credits. I took two days to think it over, and then I proposed doing it with my new tools. A music-and-animation package via Deforum. Everyone loved the result and I was getting a foot into directing. I then moved straight on to a video for my brother BoomBass and a video for "La Plage/Captain."
After that I went deeper into animation techniques, and the improvements came faster and faster. The arrival of ControlNet changed the game. ControlNet lets you grab a reference from the last generated image and apply it to the next, ensuring better consistency… but that was only the beginning of a nonstop, exhausting evolution. ControlNet, SDXL, Animatediff, IPAdapter… and more, would be the game changers in AI animation.
For almost a year now, I’ve been working with an open source software called ComfyUI, installed on my machine in Linux — that too was another challenge: learning to communicate in Python through a terminal, creating venvs, diving into the GitHub and Hugging Face universe and discovering this incredible open source community. ComfyUI is a node-based software that lets you do almost anything possible with Stable Diffusion, and even other truly crazy things like integrating an LLM (a GPT-type language model) at the prompting stage, or to analyze an image you feed in and pull a hyper-detailed description from it. It’s with ComfyUI that I made the videos for the album "So Sorry" using Animatediff, another process that’s trained on moving video.
But before I could make those videos, I had to learn how to train a model with my mug… kind of a gas factory back when I got into it. It consists of "fine-tuning" a base model, SDXL in my case, by creating a Lora. The Lora is a lightweight version of the model that gets added to the model like a plugin during the process. You feed the model with a dataset, in this case a collection of images of me and a detailed description of what you see in each photo. Then it’s a long experimentation before you get a satisfying model, and above all a fairly painful computing time when you only have a consumer-grade video card.
For "So Sorry," the idea was to see myself younger and grayer in the middle of a decomposing world, and to have this dual male-female identity — something I cared about because with all these debates on sexual identity and gender, I wanted to say that, for my part, I was a clever mix of everything. It’s pretty wild what this workflow allowed me to do, and I’m rather proud of it because it’s sometimes quite hard to get what you want. Because even if there’s an enormous share of the work done by the software, the research part is fairly long and sometimes disorienting. You have to choose and know where you’re going, otherwise you spend entire days testing renders that are all just as different and often all just as hallucinatory — an abundant source of possibilities that makes you lose your head. For "Irrational Interlude," I wanted a slightly acid trip vibe. The multiplication of the Sinclair self takes up all the space and the other selves are strange… a non-narcissistic ego trip.
I’ve been making mini films, music videos and visual objects for a long time, and I set up a mini video studio to experiment with all this. It’s my childhood fantasy, having a studio like George Lucas’s ILM… well, we’re not there yet, but the path is being walked.
While searching out everything being done in the AI field, I obviously played with the first Text2music tools that produced truly crazy results. MusicGen and Audiocraft are, to me, very experimental but fascinating. Everything’s in the prompt, but not only, because there’s this insane music2music function where you just feed in a chord sequence, a melody or a sample and transform it through the prompt… same thing, limitless, so pretty wild to handle.
Thanks to this process, I did the music for a 15-minute film for Robin Lopvet’s exhibition at the art and photography center in Lectoure. The film is about the history of New York photography revisited through AI. Robin works essentially on his photos in Midjourney, having them reinterpreted. We then upscaled the renders in Stable Diffusion and animated part of it in Gen2, the rest being an animation technique that’s Robin’s own. I told myself the music had to be partly AI-generated like the images. So I used parts of tracks I had in stock and others I’d built, and modified them with MusicGen, a fairly long but really trippy process. Sometimes the same original piece yields 3 absolutely distinct versions. We made a 4-minute version of this film and it will be screened at the Rencontres de la Photo in Arles on July 6.
What I’m passionate about is searching. I’ve always searched and pushed my projects to the frontiers of my skills. That’s where I have fun, that edge of the stage with one foot over the void. I spend an hour every morning reading forums, Discord servers or Reddit communities to keep up with the latest advances in AI in all its forms. I’m watching for the moment when we’ll be able to use OpenAI’s "Sora," this model that has ad agencies and Hollywood trembling, but which in my opinion needs five buildings full of Nvidia H200 cards to run it… I’m following Udio very closely, which to me is the musical revolution in motion. This tool is going to transform commercial music, distribution and copyright… and put a whole lot of singer-songwriters in a bind. I also use LLMstudio, to try quantized language models, because that’s kind of the starting point — language in AI. I think we’re at a moment identical to when the internet arrived in our homes.
We’re in the middle of a technological revolution, with effects less promising for the population than those of the internet were. When the sampler arrived, I dove into the machine and managed to produce my first tracks, take part in one of the first French rap albums and release my first album thanks to that new technology.
Artificial intelligence is my sampler of 2024.
I integrate it into all my processes, and I’m not even talking about ChatGPT, which has become essential to optimizing my work in general. To finish, I believe the future of creators is to fine-tune their own models, create unique versions with personal datasets, copyright them and put them into circulation. The economic model will fall into place, and we’ll completely forget today’s doubts, just as we forgot Napster… My future projects are mostly in producing tracks for the artists I work with, directing the videos that go with them, and moving forward on a short film project.
I’m about to celebrate the first anniversary of my label "Rockette Records," which we set up in Arles last summer. We’re going to close out the Escales du Cargo festival at the ancient theater on July 21. A big fiesta with the whole label crew! And video projections… in AI!
