In a recent report from the New York Times, Meta revealed new AI features for its smart glasses product – that Meta developed using Ray Ban frames – following the firm’s promise of broader virtual assistant tools for the product range at last year’s Connect event. It appears that the new prototype features follow Meta’s moves to establish a partnership with LG to develop AI-powered HMDs, which is a very timely series of events.
The Ray Ban Stories product already had voice integration to access features like video/photo capture and chat services. However, the most recent experimental features – that Meta is only opening as early access for developers – include fresh new leaps towards integrating consumer AI use cases into the emerging product. The AI features are available to select developers, with a broader rollout planned for next month.
According to reports, Meta’s smartglasses will gain AI integration to understand a user’s surroundings and identify information, such as brands, pets, and language. The glasses will translate a user’s surrounding spatial data as helpful augmented information; the New York Times featured how the device could tell the nutritional content of a food item; alternatively, the glasses also help a user with instructional cooking guidelines.
Other features highlighted include information on art, landmarks, and buildings, with users able to reply to the AI assistance and gain further details. Notably, the Ray-Ban Stories product follows the longstanding trend of smart glasses translating written languages.
The Ray-Ban Meta smart glasses are affordable for those interested in augmented reality. Starting at $299, they come with stylish eyewear, an integrated camera, and open-ear headphones. Although they don’t offer comprehensive AR or MR features, they allow you to capture video and communicate while moving.
The basic specifications of the glasses include a 12MP ultra-wide portrait camera, 1080px video resolution, live streaming mode for up to 30 minutes, 32GB of storage for videos and photos, 133-gram weight, custom-built speakers, five mics with immersive audio recording, Bluetooth 5.2 and Wi-Fi 6 connectivity, IPX4 water resistance, and 36 hours of battery life (with the case).
Meta to Drive AI-Powered XR Devices
The upcoming AI features come as Meta seemingly sets its last hurrah for the previous Oculus branding, with all future sails featuring Meta. This month, Meta will remove all Oculus accounts and related data by the end of the month.
Meta has announced changes to its Oculus account system, which means users will no longer be able to log into their XR devices with an Oculus account, and all data linked to that account will be deleted by the end of March. Meta will enforce these changes today, on March 29th.
Once an Oculus account is deleted, a user can no longer use a Meta Quest with an Oculus account without reactivating methods to retrieve information from the account. However, Oculus account users can still migrate their data to Meta’s website.
The move is part of Meta’s expansion of its enterprise XR services. Meta Quest for Business service allows companies to use a framework of XR hardware, software, and management tools to help enterprise end-users get started with the firm’s immersive portfolio. The premium additions to Meta’s enterprise service allow end-users to bypass sign-in requirements, simplifying onboarding processes.
Mark Zuckerberg, the CEO of Meta, met with LG’s CEO, William Cho and Park Hyoung-sei, the President of the Home Entertainment Company, to discuss potential collaborations on XR devices, beginning a strategic partnership between LG and Meta’s technology portfolios.
The partnership aims to drive innovation in XR devices and customer experiences by creating a technology ecosystem that combines LG’s TV division services with Meta’s XR portfolio. LG plans to leverage the benefits of its smart device OS alongside Meta’s AI systems, which will lead to the developing of a competitive XR device. Currently, LG’s smart OS is worth approximately $750.5 million from app revenue, which could become a valuable source of income and services for Meta’s XR hardware ecosystem.
Meta and AI: The Long Game
Meta’s recent AI smartglasses integration should surprise those watching the firm closely. Through efforts such as its established service ecosystem, Meta is working to become a leader in AI solutions for various use cases.
Regarding XR, Meta also kept no secrets regarding its interest in a marriage between the two technologies. Late last year, Meta’s Head of Hardware Partnerships at Meta, Jenniffer Hom, said:
Connect was a huge success for us [Meta]. We announced the Meta Quest 3, the Ray-Ban/Meta smart glasses, and gave the world a glimpse into the future of the metaverse. The most important takeaway for our developers and partners is that AI has the power to transform XR experiences. For creators, it will open up so many new opportunities to create amazing experiences and give you and me – the consumer – superhuman capabilities. Very soon, you will be able to ask your Meta AI a question, and the AI will answer you with the context of seeing and hearing through your smart glasses.
Moreover, during the event, Qualcomm’s VP and GM of XR, Hugo Swart, said that they are busy working on ways to address the “full spectrum of realities” that make emerging spatial computing from “immersive VR in 100 per cent virtual worlds, to mixed reality where you experience immersive content while being able to see and merge your physical space using video see-through, to augmented reality where virtual objects overlay on your world through thinner, lighter glasses.”
Major XR Partnerships to Deepen AI Integration
Qualcomm released the XR2+ Gen 2 chipset earlier this year, promoting the technology to power AI-optimised immersive hardware. Qualcomm is working in a partnership with Samsung and Google to develop an XR headset.
In a recent interview, Jay Latta, Speaker and Founder of The Fusionists, noted that XR developments are getting expensive. As new technologies emerge, they bring new costs and time investments for vendors and end-users. The XR industry is no exception – it comes with its risks and investments. In recent developments, LG has partnered with Meta, while Qualcomm is collaborating with Samsung and Google, as many firms are forming groups to manage the challenges of investing in XR technology.
Jay explained:
All these developments are becoming so expensive, and no one knows what direction you should go. Setting up partnerships, you see the Qualcomm and Samsung partnership. So, of course, LG needs someone as well, and Meta needs someone as well. – Currently, in these uncertainties, they partner because they are simply sharing the risk.
Many recent technology giants entering the XR device marketplace share a background in transitioning from smartphones to XR devices. However, the XR marketplace isn’t just headsets; end devices that leverage spatial data include smartphones, computer monitors, automated cars, and robotics—any form of hardware translating spatial XR data into an end-user experience or solution.
Could AI Improve XR Adoption?
A key element behind various enterprise-facing XR AI solutions in multiple forms and the influence of such large technology firms entering the XR marketplace with AI-ready devices could be vast. With improved AI tools emerging alongside XR solutions, the accessibility improvements may lead to broader adoption.
At GTC, NVIDIA highlighted how AI will boost its Omiverse and Maxine immersive collaboration solutions. Greg Jones, NVIDIA’s Director of XR Global Business Development and Product Manager for the Maxine Developer Platform, noted that when large language models came out, “the pipelines were simplified to a huge degree – This is a technology that’s here now, and everybody can use it.”
Jones pointed out that AI-powered Omniverse services can enhance accessibility to tools like Vred by incorporating large language models and voice commands. This means that a user who may not have a deep understanding of the sophisticated VRed UI and operations can modify the AR rendering of a product, such as the BAC car, without requiring extensive software knowledge. In other words, the user can use voice commands instead of interacting with the interface through touch or mouse.
Jones’ example spans outwards, away from Omniverse-style collaboration tools. With AI-powered hand and eye tracking mixed voice-powered assistance tools, as seen in the Meta/Ray-Ban product, accessibility and understanding of XR device usage would increase with a lesser requirement for controllers or complex input methods and UI.
Outside headsets, AI creates new XR devices, such as 3D monitors, which use eye tracking to move an RT3D image in correlation with a user’s real-life position.
AI is here to stay, having an extensive mass market appeal. XR is on a stricter journey with more hurdles restricting the adoption of the technologies. However, it appears that XR will follow off the coattails of AI’s success – and industry leaders know this.
