Data Machina #211
AI and The Future of TV & Movies. AI Video Generation. Llama 2 Finetuning & Tricks. FreeWilly2. Flash Attention 2.0. Quantised Whisper. ReLoRa. kNN-LM. Meta Transformer. RetNet. TableGPT.
AI and the Future TV & Movies. I guess it’s not a surprise why the battle over AI is being fought in Hollywood. I’d bet that in less than 2 years, AI filmmakers and AI studios will be able to release fully AI-generated feature films in less than a month, with a small team of people. AI filmmaking is a huge booming business, VCs throwing millions at AI studios startups. Still early days, but watch what’s happening today, imagine in 2 years!
[click Open in Browser (top right) if your email client doesn’t render all the vids in this post]
Creating AI Actors with NeRFs. A startup called Synthesia is using Neural Radiance Fields to create AI actors. Watch the video below. Also see: Neural Radiance Field (NeRF): A Gentle Introduction.
Google Dreamix, diffusion models and video editing. Back in February, Google introduced the first diffusion-based method that is able to perform text-based motion and appearance editing of general videos. Checkout the explainer video:
Adobe INVE: Interactive Neural Video Editing. A few days ago, Adobe published a real-time video editing solution, powered by hash-grids encoding and NNs. INVE can assist the video editing process by consistently propagating sparse frame edits to the entire video clip. Checkout the demo, quite impressive.
Automation of video creation with AI. Also a few days ago, Ray Ventura published a new powerful framework for automating video content creation. ShortGPT simplifies video creation, footage sourcing, voiceover synthesis, and editing tasks.
Southpark TV Show, Diffusion Models & Multi-Agents. Fable Studio just published an approach to generating HQ TV episodes using LLMs, custom SoTA diffusion models and multi-agent simulation. Paper: Generating HQ South Park Episodes w/ Diffusion Models & Multi-Agents. Watch the video below, pretty funny.
Barbie, Oppenheiemer… and generative AI. This week Curious Refuge released an awesome short trailer called Barbenheimer: a mash up of Barbie and Oppenheimer. Brilliant! Curious Refuge also announced: The World’s First Bootcamp on AI Filmmaking which looks like very interesting.
Generative AI western. Using Midjourney, Manuel explores an array of aspect ratios, camera lenses, and focal settings mastered by legendary film directors such as John Ford, Sergio Leone, and Akira Kurosawa, to produce an amazing Generative AI Western Film Character.
A short AI-generated film. The Carnival of the Ages: a Cinematic Odyssey Through Time and Space! This is a fully AI-generated short with DALLE, Midjourney, RunawaML, and GPT4. It’s a bit uncannily disturbing and ghostly, but worth watching.
Sci-fi and text+speech to video. Wayne Hills, a South Korean startup, just released a sci-fi short using text & speech to video. AI New Cinema Movie, follows Aiden, a hyperreal human-like character, on his own spaceship, narrating his own adventure. Watch the trailer:
Video Text-to-3D, Apple Vision Pro & VR. This is quite crazy and impressive. The winners of the Scale AI Generative Hackathon 2023, demoing a lucid text-to-3D tool built with the Apple VisionOS SDK, Scale AI Spellbook, OpenAI's GPT-4 and several other tools. This may be the future of VR/ video entertainment.
The Great Catspy. A funny, amazing text-to-video trailer of a fictional movie generated with RunawayML GenAI-2 videos, Kaliber gen AI stories, ElevenLabs gen AI voices, and other AI tools like Musicfy gen AI music. All 500 shots in this movie trailer were created with text prompts. Amazing.
A great tutorial on AI video gen and RunawayML Gen-2. If you are interested in exploring and learning how to get started in AI-filming and AI-film generation, Checkout this tutorial.
Have a nice week.
10 Link-o-Troned
Update from the Llama 2 stables…
the ML Pythonista
Deep & Other Learning Bits
AI/ DL ResearchDocs
data v-i-s-i-o-n-s
MLOps Untangled
AI startups -> radar
ML Datasets & Stuff
Pick-a-Pic - 0.5 Million Human Preferences over AI Gen images
Google Android in the Wild - Large-scale dataset of Human-Mobile Device Interactions
Postscript, etc
Tips? Suggestions? Feedback? email Carlos
Curated by @ds_ldn in the middle of the night.
They are already so far behind on VFX work that the programmers are overwhelmed. They don't pay them well enough. The minute they do this is the minute people start picking up books again.
Hell, video games have better cinematic pieces than the cinema,
My children are in their 50s.
When they were young and watched some Saturday morning cartoons I mostly saw their formulaic nature and disliked it.
When my oldest was 12 I was scanning the channels and saw a PSA with a putatively 5 or 6 year old boy approaching a girl on the lone swing in the park. He looked aggressive. I paused to watch what kind of villainy was programmed that morning. The girl showed apprehension.
The boy slowed his approach and asked the girl if she would like a push. She said yes. The boy pushed carefully.
I broke down in tears. I had watched some amount of children's programming for ten years and this was the first time I had seen humanity.
Hollywood will not evolve humanity with or without AI although they will inject lots of brain fog along their way.
We need to grasp the tools that let us communicate person to person and not let go.
Generative AI be damned .