Google Launches Genie 4, the Most Realistic Interactive Virtual World Model with Synchronized Audio

icon币界网
Share
AI summary iconSummary
Google DeepMind has launched Genie 4, a new world model that delivers the most realistic and interactive virtual environments with synchronized audio, including music and dialogue. The model supports real-time scene editing and doubles simulation duration to two minutes. Integrated with Gemini, it enables applications in gaming and robotics. This update brings fresh news on real-world assets (RWA) and is among the latest crypto developments in AI-driven innovation. Genie 4 is now available to trusted testers, with broader access coming soon.
CoinDesk reports:

IT Home, October 28: Google DeepMind launched its latest world model, Genie 4, on Tuesday, currently in research preview. This newest model can generate the most realistic and interactive virtual worlds to date, allowing users to modify scenes by entering prompts during simulation. The model can also generate real-time audio synchronized with the virtual world, including music, ambient sounds, and interactive dialogue.

IT Home noted that just months before the release of Genie 4, AI video startup Runway launched its second world model, GWM Worlds 2—a dynamically interactive model capable of responding to user inputs and supporting audio generation. Meanwhile, consolidation is emerging in the world model space: World Labs, founded by Fei-Fei Li, has been acquired by AMD; Amazon is in talks to acquire Decart; and Rivian has recently acquired Third Dimension to build internal simulation capabilities for autonomous driving testing.

Launch of Genie 4 is Google's latest move to maintain its leadership in a highly competitive field.

Over a year ago, Google released the previous model, Genie 3, in a research preview. In addition to significantly improved visual realism and newly added audio capabilities, Genie 4 supports free-form editing operations, such as real-time adjustments to weather, camera angles, or the addition of characters and objects. It can maintain a coherent scene for up to two minutes—twice as long as the previous model—with enhanced physics and more lifelike virtual characters.

The integration of Google’s large models, Gemini and Genie 4, has also been further enhanced: Gemini acts as the director, responsible for planning steps and validating video frames, while Genie 4 generates visual content. In a research experiment related to the Gemini robotics project, the Gemini 4 Argon model broke down the task into multiple steps, which were then simulated in real time by Genie 4. This experiment demonstrated a viable approach: verifying scenarios in a simulated environment before conducting robotic tests in the real world.

Tejas Kulkarni, Director of Research at Google, said at a briefing: “In the future, we will see more and more of these hybrid world models, where Gemini generates code and text, and Genie 4 reads this content in real time to simulate reality at different levels of abstraction.”

Like other world models, its primary commercial applications include gaming, interactive learning, and robotics. One envisioned use case for interactive learning is users entering a painting depicting a historical scene and engaging in conversations with the figures within to learn about the events recorded in the artwork.

However, "trusted testers"—the first users granted access to Google's cutting-edge products—discovered a potential new use: film prototyping.

Google DeepMind engineer Diego Rivas said at a media briefing: "The value of real-time capabilities is especially evident in scenarios requiring sustained creative flow, such as the ideation phase and storyboarding—all stages where rapid idea generation is essential."

Google also launched Veo, an AI video generation model that creates high-definition short videos from text, image, or video prompts, complete with synchronized audio—a tool whose popularity continues to rise among film creators.

Rivas added that users may one day use Gemini Omni, a multimodal large model, to transform prototype materials into finalized creative assets, as Gemini Omni can generate videos based on images, audio, and text. However, he noted that these capabilities are still some distance from practical implementation.

Throughout the briefing, Rivas repeatedly emphasized that Genie 4 is still in its early development stages, and the final user experience of the full product has not yet been determined. The team also acknowledged that Genie 4 still has limitations: the longer the simulation runs, the more likely details and objects are to drift; the team has not yet found a reliable way to ensure accurate audio while keeping text in the virtual world clearly legible.

Genie 4 is currently available in a research preview to trusted testers. Over the coming weeks, access will be extended to Google AI Ultra subscribers, the Genie project—which is Google’s AI tool for generating interactive virtual worlds from text or image prompts—and early collaborators using the Gemini API and Google AI Studio.

Disclaimer: The information on this page may have been obtained from third parties and does not necessarily reflect the views or opinions of KuCoin. This content is provided for general informational purposes only, without any representation or warranty of any kind, nor shall it be construed as financial or investment advice. KuCoin shall not be liable for any errors or omissions, or for any outcomes resulting from the use of this information. Investments in digital assets can be risky. Please carefully evaluate the risks of a product and your risk tolerance based on your own financial circumstances. For more information, please refer to our Terms of Use and Risk Disclosure.