Google has developed a new video generation AI model called Lumiere, which uses a diffusion model called Space-Time-U-Net (STUNet). This model allows Lumiere to determine the location of objects in a video and how they move and change over time. Unlike other methods that stitch together still frames, Lumiere creates videos in one process. It starts by creating a base frame and then uses STUNet to approximate the movement of objects, creating seamless motion. Lumiere generates 80 frames compared to the 25 frames of its competitors. Google’s Lumiere is a significant advancement in AI video generation, moving closer to realistic videos. It competes with platforms like Runway and Stable Video Diffusion. Lumiere also offers features like image-to-video generation, stylized generation, cinemagraphs, and inpainting. However, there is a concern about the potential misuse of this technology, and Google acknowledges the need for tools to detect biases and malicious use cases.
