Vidu AI Video Generator is Incredibly Fast!

Vidu, an emerging text-to-video AI platform, has received some significant upgrades since its initial announcement in April 2024. One of the most impressive updates is the reference-to-video feature.

Back in April, I published an article about Vidu, sharing my initial thoughts on its remarkable speed and quality. Months later, its creators have proven they’re working tirelessly to push the technology to its limits, going head-to-head with major competitors like Runway and Kling AI.

What is Vidu?

Vidu is an AI-powered tool that can do the following video generation capabilities:

• Text-to-video
• Image-to-video
• Reference-to-video

The AI model is built on a proprietary visual transformation model architecture called the Universal Vision Transformer (U-ViT). This integrates two text-to-video AI models: the Diffusion and the Transformer.

This architecture enables the creation of high-quality videos with dynamic camera movements, intricate facial expressions, and authentic lighting and shadow effects.

Vidu is the first to introduce this world’s first technical framework in 2022.

Now, let’s explore the video generator dashboard. On the left side, you can choose to generate video either from an existing image, an existing subject, or from text.

Several settings can be adjusted, including the video style (general or animated), duration (4 or 8- seconds), or mode (switch the priority between speed and quality).

Example Videos

Let’s take an example to illustrate this.

Prompt: A man sitting at a table, eating noodles with chopsticks

Once the processing is done, you’ll get the result on the right side. This time, you can either edit the video by modifying the prompt or upscaling it for an additional 4 credit points. The upscaled video resolution is 2K (1934 × 1080).

In just 30 seconds, Vidu generated a 4-second, 688 × 384 video file that beautifully captures the scene. The impressive part here is not just the quality of the generated video but the speed with which it was created.

For context, other AI video generators I’ve tried either take longer or deliver subpar results at similar speeds. Vidu stands out for this reason.

“We’re proud to bring this feature to market and believe it will significantly enhance how our users interact with and utilize AI in their creative processes.” — Jiayu Tang, Cofounder and CEO of Shengshu Technology.

Here are more examples:

Prompt: A medieval sailboat sailing on the sea, foggy nights, bright moonlight, eerie atmosphere

Vidu generated a video that perfectly captures the eeriness described in the text prompt. All the elements from the prompt are present in the final video. The foggy night, the bright moonlight casting an eerie glow, and the medieval sailboat sailing silently across the sea—all come together to create a hauntingly beautiful scene.

Not all AI video generators can handle such details, especially with moving subjects and dynamic environments. It’s a really great tool, and this example showcases just how advanced Vidu’s technology is.

How to Access Vidu

Accessing Vidu is easy. Simply head over to vidu.studio, and the first thing you’ll notice is a modal window showcasing details on the platform’s latest updates.

1. Newly upgraded reference-generated video capability
2. New generation mode configuration

We’ll try these features later on. You can close the window and start by creating an account.

We’ll get into these features in more detail later, but right away, the user interface is clean, modern, and intuitive — something I personally appreciate in creative tools. It makes the process of generating videos less overwhelming, even for first-time users.

New Features of Vidu

The reference-to-video feature ensures that the core subject or scene of the video remains consistent throughout, which might sound basic but is actually crucial in maintaining the viewer’s attention and the video’s integrity.

For instance, if you’re generating a video about a character walking through different environments, Vidu can maintain the identity and appearance of the character consistently throughout the scenes.

Many competing tools struggle with this, often delivering results where the character’s appearance changes subtly from frame to frame — something that can be quite distracting.

Here’s an example AI-generated with Midjourney:

Prompt: Ice driving fast in Norways with a porsche

Upload the image into Vidu’s image-to-video tool and describe it in the prompt field.

Here’s the final video output:

Another notable feature is the reference-generated video capability, which allows users to input a reference image or video that helps guide the style or atmosphere of the generated video.

Let’s try an example:

Prompt: A business woman sitting relaxed on a white fluffy cloud, she has short brown curly hair, she is wearing round glasses, coworking on the laptop, flying over a traditional european bank, downtown, wearing blue and white business clothes, brown pants and white sneakers, high angle view, heaven, clouds.

Upload the image to Vidu and select a single subject in the image for the best results. Also, don’t forget to describe the video in the prompt field.

Vidu Prompt Guide

To get the most out of Vidu, the platform offers a prompt guide that helps users create the best possible video prompts. The documentation is comprehensive, offering example structures, keywords, and techniques for creating more effective and visually appealing videos.

You can explore various prompt keywords related to film styles, artistic styles, shooting settings, and text effects.

How does prompting affect the output?

The quality of the text prompt you provide hugely affects the final result of the video. When prompts follow the basic structure of subject, scene, environment, and style, they can enhance the effectiveness of video generation to a certain extent.

Take a look at the example below:

Prompt: A corgi is swimming

As expected, the output is a video of a corgi swimming, but it’s pretty straightforward — nothing too flashy. Now, let’s improve the prompt.

Prompt: Capture a serene moment featuring a baby corgi swimming gracefully in a large, sunlit pool. The underwater perspective showcases the puppy, its gentle smile illuminated by soft, golden hour lighting that filters through the water, creating a dance of light and shadow on the pool’s bottom. The scene is set in soft pastel colors, enhancing the dreamlike, ethereal quality of the atmosphere. The high-resolution photography captures every delicate detail of the water’s texture and the Corgi’s joyful expression, creating a simple yet cinematic portrait tranquility and innocence. This minimalist yet emotive setup conveys a sense of calm and happiness, ideal for a serene and visually captivating film sequence.

As you can see, the result is vastly improved, with better lighting, more dynamic camera angles, and an overall cinematic quality. Vidu excels at capturing these details when the prompt is structured thoughtfully.

To strengthen the consistency of the effect and atmosphere, it is necessary to constantly emphasize and refine the overall ambiance.

How Much Does It Cost?

Vidu offers free credits to try out the tool and also offers paid subscription plans:

• Free: 80 credits monthly, generate 4-second video, upscale resolution, no commercial use, 1 task at a time.
• Standard: $9.99 per month (50% off, usually $19.99), 320 credits monthly, generate 4-second and 8-second video, upscale resolution, commercial use, remove watermark after upscaling, 2 tasks at a time.
• Advanced: $29.99 per month (50% off, usually $59.99), 880 credits monthly, generate 4-second and 8-second video, upscale resolution, commercial use, remove watermark after upscaling, 3 tasks at a time, priority for new features.
• Premium: $99.99 per month (50% off, usually $199.99), 2960 credits monthly, generate 4-second and 8-second video, upscale resolution, commercial use, remove watermark after upscaling, 4 tasks at a time, priority for new features.

The free plan offers a decent number of credits to test the waters, but you’ll quickly want to move up to the Standard or Advanced plan if you’re serious about generating high-quality content consistently.

Users can also opt for annual subscriptions and get a 20% discount.

API Access for Developers

The API is not publicly available yet, but you can sign up to get early access. Complete this form to apply for API access.

We are excited to offer our API to support the community in developing various applications based on Vidu, bringing the power of multimodal large models to everyone. We are looking to select some beta users to test the stability of our API services so we can open it up to everyone as soon as possible.

I haven’t seen any documentation yet about the API usage and cost. I will update this article once I get that information.

Final Thoughts

I am still in awe with all the latest updates and improvements of AI video generators. In the recent months we’ve seen upgrades from the likes of Runway Gen-3 and Kling AI. Today, Vidu joins the list of top-tier AI video generators. Never mind Sora, because OpenAI doesn’t seem to have plans to publicly release it.

According to Vidu’s CEO, the company is actively exploring the commercial potential of generative AI in areas such as art design, game development, film post-production, and content socialization. Their ultimate vision is to use this multimodal model to enhance human creativity and productivity through AI.

Generative AI is already in use today in various fields such as gaming, social media, and art. It won’t be long before we see AI-generated media such as full-blown movies and TV-shows, games generated on-demand with AI, and even interact with AI companions that are indistinguishable from a real human being.

So, what’s your take on Vidu? Do you like this new AI video tool? How do you think it stacks up against Kling and Runway? I’d love to know your thoughts.

‍