Adobe teases new generative AI tool with text-to-video and image-to-video features

The advanced AI video model will be the latest addition to Adobe’s Firefly suite of AI offerings.

First look at the Adobe Firefly Video Model. (Screenshot: Adobe)

Adobe has unveiled a new generative AI tool that can be used to create videos based on text prompts and images. The AI tool called Adobe Firefly Video Model is the latest addition to the designing software giant’s suite of generative AI services known as Adobe Firefly.

However, the new Firefly-powered AI model with text-to-video and image-to-video capabilities will only be available in beta later this year. Adobe also said that it is upgrading its video-editing software Premiere Pro with Generative Extend, which extends the canvas of an image or video frame with newly-generated backgrounds.

“The Firefly Video Model is designed to be commercially safe and is only trained on content we have permission to use – never on Adobe customer content,” the company said.

Story continues below this ad

In June this year, users raised concerns that the company could potentially use their data for AI training after Adobe updated its terms of service to reflect that it may access, view, or listen to user content “through both automated and manual methods,” in addition to holding licences “ to use, reproduce, publicly display, distribute, modify, create derivative works based on, publicly perform, and translate the Content.”

How to use Adobe’s Firefly video model

Adobe’s latest generative AI tool is capable of creating videos from text prompts. It can also be used to finetune videos by adjusting angle, motion and zoom. Video editors can use this tool to generate B-Roll based on certain reference images.

It also comes with image-to-video capabilities that can be used to convert still images or illustrations into live action videos.

Also Read | Adobe brings AI Assistant to Reader and Acrobat: Here’s how it works

“Building upon our foundational Firefly models for imaging, design and vector creation, our Firefly foundation video model is designed to help the professional video community unlock new possibilities, streamline workflows and support their creative ideation,” Ashley Still, senior vice president, Creative Product Group at Adobe, said in a statement.

Story continues below this ad

Some other use cases of the AI video model are creating animations with control over camera angle and motion, generating an insert into a key video shot with camera motion, pairing AI-generated video with footage captured in real-life, and more.

Since its launch in March 2023, Adobe Firefly has been gradually expanded to include a generative AI image model, vector model, design model, and soon, a video model. Generative Fill in Photoshop, Generative Remove in Lightroom, Generative Shape Fill in Illustrator, and Text-to-Template in Express are a few other popular AI features offered by Adobe.