How the Metaverse and AI Collaborate

In early March, a Tribune headline said: “Zuckerberg quietly buries Metaverse” and the subhead read, “After a failed idea and its expected revolutionary impact, Metaverse is quietly taken down to focus on AI.”

The End of the Metaverse?

Has Meta (META) CEO Mark Zuckerberg truly buried the metaverse? Or is this simply the Tribune’s own interpretation? The headline was based on a Zuckerberg post on Facebook and Instagram about Meta’s technology plans. It was not the actual quote from Zuckerberg’s post, but rather the Tribune’s inference of its meaning.

Here’s what Zuckerberg actually wrote: “We’re creating a new top-level product group at Meta focused on generative AI to turbocharge our work in this area.”

Considering the latest hype around ChatGPT, Meta’s shift to focus on Artificial Intelligence (AI) and Generative AI is not surprising. Meta is simply joining the strategic AI positioning of other big tech companies such as Google, Apple and Microsoft, which has a stake in OpenAI, the creator of ChatGPT, with a multi-year, multi-billion investment plan.

Tim Cook, Apple’s CEO, recently said: “It [AI] is a major focus of ours. It’s incredible in terms of how it can enrich customers’ lives…Apple sees an enormous potential in this space to affect virtually everything we do,” adding that “it will affect every product in every service that we have.”

Cook’s comment that AI “will affect every product in every service that we have” is key, and it is most likely how other tech giants are envisioning and positioning the impact of AI, and why they are motivated to heavily invest in this technology.

After all, “every product or service” includes the metaverse.

Zuckerberg said that Meta would pull together teams working on generative AI across the company into one group to build “delightful experiences around this technology into all of our different products.” This statement aligns with Cook’s view on the impact of AI. Zuckerberg also mentioned that “in the short term, we’ll focus on building creative and expressive tools. Over the longer term, we’ll focus on developing AI personas that can help people in a variety of ways.”

Meta’s long-term view is quite telling, and a possible indication that Meta is not “burying” the metaverse, but rather positioning to invest in means that further enhance the metaverse experience and its utility, which could conceivably have a greater impact on the way the metaverse will function and operate.

Why is AI important to the existence and scalability of the metaverse? And how would AI turbocharge the metaverse to create a seamless virtual reality beyond our imagination?

To answer these questions, let’s first consider how virtual spaces, such as the metaverse, have been depicted in science fiction literature and movies. It’s worth paying attention to this particular element because pop culture has a way of influencing real-life innovation.

OASIS from Ready Player One

“These days, the reality is a bummer. Everyone is looking for a way to escape,” said Wade Watts, the protagonist of the Ernest Cline’s science fiction novel Ready Player One. In the novel and movie, which takes place in 2045, the world is on the verge of collapse, but a virtual reality universe, OASIS, has given people something to find hope in.

While Ready Player One is a science fiction story about a contest taking place in a virtual community, it draws many parallels to the rise of the metaverse. In OASIS, hosted avatars could be customized according to individual preferences, people could purchase items and outfits to use in the game and those purchases had real value. OASIS could be read as a description of the metaverse. Science fiction is becoming less of a fiction and more of an attainable reality due to the exponential advances in technology, and in particular in AI and Generative AI.

How will AI impact the Metaverse?

While virtual spaces can technically exist without artificial intelligence, the combination of AI with Virtual Reality (VR) can unlock a whole new degree of verisimilitude, including but not limited to:

Accurate Avatar Creation

Users are at the center of the metaverse, and user avatar’s accuracy will determine the quality of experience for the user and other participants. An AI engine can analyze 2D user images or 3D scans to come up with a highly realistic simulated rendition. It can then plot a variety of facial expressions, emotions, hairstyles, features brought on by aging, etc. to make the avatar more dynamic. Companies like Ready Player Me are already using AI to help build avatars for the metaverse, and Meta is working on its own version of the technology.

Powering 3D chatbots – Digital Humans

Digital Humans shouldn’t be confused with avatars. While avatars are a controlled representation of the user and are replicas of another person, digital humans are more like AI-enabled non-playing characters (NPCs) in a video game that can react and respond to users’ actions in a VR world. Digital humans are built entirely using AI technology and are essential to the landscape of the metaverse. From NPCs in gameplay to automated assistants in VR workplaces, there are myriad applications, and companies like Unreal Engine and Soul Machines have already invested in this direction. 

Multilingual Accessibility

In the Metaverse, users will be able to communicate smoothly, even if they don’t speak the same language, all due to AI. AI can recognize the user’s natural language and converts it into machine language, from where it is transformed back into its natural form and delivered to another user. The whole process takes a fraction of a second – like a real conversation. The best part is that the results could be converted into any language, depending on the AI’s training, so that users from around the world can access the metaverse. 

Enabling data learning – VR world expansion at scale

Artificial intelligence finds patterns in large amounts of data and predicts possible outcomes due to machine learning. Over time the predictions improve and become more accurate and superior. AI data learning is valuable because it is expected to replace the human factor in data-collecting jobs in the future. Eventually, the AI will be able to perform the task and provide output almost as well as human beings. Companies like NVIDIA are training AI to create entire virtual worlds. This breakthrough will be instrumental in driving scalability for the metaverse, as new virtual worlds can be added without the intervention of humans.

Intuitive Interfacing

AI can also assist in human-computer interactions (HCI). When you put on a sophisticated AI-enabled VR headset, its sensors will be able to read and predict your electrical and muscular patterns to know exactly how you’d want to move inside the metaverse. AI can help recreate an authentic sense of touch in VR and aid in voice-enabled navigation, so you can interact with virtual objects without having to use hand controllers.

Bear in mind that Meta will continue to invest in and produce VR headsets. The headsets are the hardware that provides access to the metaverse. Without the hardware, we cannot experience the virtual space. The headset also incorporates software, which allows for data collection and monitoring, and with the integration of AI, will also allow all the features and interactions mentioned above.

If Meta collects data via its headsets, this data could be used for not only enhancing their existing AI algorithms but also for the creation of new and more sophisticated ones based on the information collected via the headsets on how people interact in the metaverse, and their physical as well as mental behavior. And if brainwaves are collected through Brian-Computer Interface (BCI) then a direct link to someone’s brain opens up new types of data to be collected and analyzed. 

The Importance of AI for the Metaverse

AI in the Metaverse enables user interaction, data processing and protection, the construction of avatars and chatbots, and a wide range of additional features that produce a genuine experience.

AI can analyze a user’s image, creating an avatar that mimics their gestures and facial expressions and even simulates their speech. Millions of users from all over the world may understand each other because AI processes natural language. AI helps to broaden inclusiveness and lessen discrimination in this way.

Users and businesses, both of which provide unique entertainment, learning, or employment opportunities, can benefit from AI in the Metaverse. AI utilization will allow them to analyze user experiences to enhance their services and product line, and eventually increase income.

Challenges of AI integration in the Metaverse

The metaverse is a new area of research and operation, and AI implementation could run into issues. The following are a few challenges to consider:

• Ownership for AI-created content: Who holds the copyright to and can profit from the content and VR worlds created by AI? 
• Deepfakes and user transparency: How do you ensure that users know they are interacting with AI or a human? How do you prevent deepfakes and fraud?
• Fair use of AI: Can users legally apply AI technologies to metaverse interactions? For instance, can they use AI code to win games?
• Right to use data for AI model training: How can we ethically train AI for the metaverse? What are the consent mechanisms involved?
• Accountability for AI bias: If a digital human or similar AI algorithm displays bias, what is the possible recourse a person can take?

AI is a pivotal technology; without AI, it will be difficult to create an engaging, authentic, and scalable metaverse experience. Understanding the technology’s great potential as well as its risks, companies like Meta are working closely with think tanks and ethics groups to stem the risks of AI without curbing the technology’s potential.

Meta’s Approach on AI in the Metaverse

In March 2022 Meta conducted an event titled “Inside the lab: Building for the metaverse with AI,” a step in Meta’s quest to unlock the metaverse with AI.

“We work on a lot of different technologies here at Meta – everything from virtual reality to designing our own data centers. And we’re particularly focused on foundational technologies that can make entirely new things possible. Today, we’re going to focus on perhaps the most important foundational technology of our time: artificial intelligence,” said Zuckerberg.

Jérôme Pesenti, leader of Meta AI and co-managing director at Meta AI Research, noted that AI is one of the keys to the metaverse. He said the mission of Meta AI is to bring the world closer together by advancing AI through AI research breakthroughs and improving Meta’s products through them.

Clearly, Meta has not buried the Metaverse. Just the opposite.

It seems that Meta has always invested heavily in AI to enhance its products and services, not just its metaverse exploration. The event in March underscores Meta’s understanding of the important role and impact of AI on the metaverse and the reason they focus on AI. The recent hype over ChatGPT simply gave Zuckerberg the opportunity to reinforce his thoughts, vision, and investment decisions in AI. Zuckerberg certainly understands that commanding AI may lead to commanding the metaverse.

The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.