With respect to real time, I'm actively working on generating StyleGAN visuals in real time in Python, but admit that the frame rate is a little low at 1024. Are there particles that generate on the beats? A clip you're playing? Etc. I'd perhaps start with defining what you'd like to see - find reference that you like and go back from there. There are a lot of VJ tools out there that can sync visuals to a beat and take other inputs (like the clothing sensors, MIDI controllers, etc). That sounds like a really interesting idea! I guess the first question might be if you really need a GAN or not to create the visuals.
#Video music visualization update
Also, of course, /u/gwern who posted an incredible guide to using StyleGAN.Įdit - Update - Second video has been released by Anjunadeep! Youtube link.Īwesome. Indeed, I also wanted to shout out /u/ C0D32 who shared an art-centered StyleGAN model that was really influential to me! Thanks for that. As a visual effects artist, I'm over the moon with the sorts of things that are possible.
#Video music visualization how to
It's been fascinating to learn how to use StyleGAN like this. There are some further StyleGAN experiments there too if you're interested. I've also posted some more images I created with Qrion's custom models here on my site. a keyframeable overall playback speed fader system a system I built that incorporates live input (you can tap the keyboard to add a jump to the playback in After Effects) The tempo of the GAN evolution is controlled by a few different things: Qrion picked images that matched the mood of each song (things like clouds, lava hitting the ocean, forest interiors, and snowy mountains) and I generated interpolation videos for each track. This was created using StyleGAN and doing a transfer learning with a custom dataset of images curated by the artist. Here's the first generated video - two more coming out soon.
I'm excited to share this generative video project I worked on with Japanese electronic music artist Qrion for the release of her Sine Wave Party EP. Metacademy is a great resource which compiles lesson plans on popular machine learning topics.įor Beginner questions please try /r/LearnMachineLearning, /r/MLQuestions or įor career related questions, visit /r/cscareerquestions/ Please have a look at our FAQ and Link-Collection Rules For Posts + Research + Discussion + Project + News on Twitter Chat with us on Slack Beginners:
0 kommentar(er)
