https://www.technologyreview.com/2023/06/01/1073858/surreal-ai-generative-video-changing-film
The Frost nails its uncanny, disconcerting vibe in its first few shots. Vast icy mountains, a makeshift camp of military-style tents, a group of people huddled around a fire, barking dogs. It’s familiar stuff, yet weird enough to plant a growing seed of dread. There’s something wrong here.
“Pass me the tail,” someone says. Cut to a close-up of a man by the fire gnawing on a pink piece of jerky. It’s grotesque. The way his lips are moving isn’t quite right. For a beat it looks as if he’s chewing on his own frozen tongue.
Welcome to the unsettling world of AI moviemaking. “We kind of hit a point where we just stopped fighting the desire for photographic accuracy and started leaning into the weirdness that is DALL-E,” says Stephen Parker at Waymark, the Detroit-based video creation company behind The Frost.
The Frost is a 12-minute movie in which every shot is generated by an image-making AI. It’s one of the most impressive—and bizarre—examples yet of this strange new genre. You can watch the film below in an exclusive reveal from MIT Technology Review.
To make The Frost, Waymark took a script written by Josh Rubin, an executive producer at the company who directed the film, and fed it to OpenAI’s image-making model DALL-E 2. After some trial and error to get the model to produce images in a style they were happy with, the filmmakers used DALL-E 2 to generate every single shot. Then they used D-ID, an AI tool that can add movement to still images, to animate these shots, making eyes blink and lips move.
“We built a world out of what DALL-E was giving back to us,” says Rubin. “It’s a strange aesthetic, but we welcomed it with open arms. It became the look of the film.”
“This is certainly the first generative AI film I’ve seen where the style feels consistent,” says Souki Mehdaoui, an independent filmmaker and cofounder of Bell & Whistle, a consultancy specializing in creative technologies. “Generating still images and puppeteering them gives it a fun collaged vibe.”
The Frost joins a string of short films made using various generative AI tools that have been released in the last few months. The best generative video models can still produce only a few seconds of video. So the current crop of films exhibit a wide range of styles and techniques, ranging from storyboard-like sequences of still images, as in The Frost, to mash-ups of many different seconds-long video clips.
