In September 2020, Meta Platforms, formerly Facebook, announced an augmented reality (AR) glasses initiative to help users interact with their surroundings with digital content overlays to assist their day-to-day activities.

Dubbed Project Aria, Meta launched the initiative purely for research purposes, using the company’s already deep knowledge on immersive technology.

VR has dominated conversations in immersive tech for a while now, forcing AR to take a back seat, but Facebook remains determined to reverse the tide. Project Aria is the company’s newest strategy to leverage the full potential of AR and hopefully, power better products in the future.

What Is Project Aria?

Project Aria refers to a family of AR smart glasses that superimpose contextual information and messages around the wearer’s field of view (FoV).

For example, users visiting a new city would automatically see directions and shopping info whilst looking down a street with Meta’s AR glasses. However, the project is still under development with two outstanding requirements remaining:

• Technological breakthroughs – Meta notes the devices require sophisticated audio and visual input systems, contextual AI, and innovative lightweight frames for Project Aria, with the product estimated to launch in five to ten years, according to The Information.
• High volume, localised data – Testers will also need to collect information for the AR glasses to display. Project Aria will invite participants wearing these AR glasses to collect data in their surroundings whilst wearing the head-mounted displays (HMDs).

Project Aria builds Meta’s efforts in a bid to to create a body of knowledge to serve as the foundational data repository for contextual AR experiences through smart glasses.

Sources indicate Meta will not name the final product Project Aria as the current devices are meant solely for research. The firm has not yet envisioned a consumer-facing product and importantly, research participants cannot view or listen to any of the data they capture.

Key Updates: What’s New?

Meta unveiled Project Aria at its Connect 2020 event, where the company spoke about its progress, revealing the following updates:

Predictive Capabilities

Meta has determined that, in the future, AR glasses could potentially anticipate what the wearer intends to do next. For example, the glasses could predict a user is searching for their TV remote and highlight where it’s located.

At Connect event last year, Meta announced it had made progress in achieving this by using a combination of AR glasses and handheld devices.

Recording with consent

The company outlined its Responsible Innovation Principles to guide data collection and technology development.

Meta announced some of its contractors and employees in the UK, EU, Switzerland, Singapore, Canada, and Israel captured data in their households, with consent from family members, with additional testers mining data in public spaces at select locations in the US and Singapore.

Scale and reach

This key update saw Meta deploy 800 research devices worldwide and plans to ramp up numbers up to roughly 3,000 devices over the next few quarters.

Diversity, Equity & Inclusion

Meta’s initial announcements did not include much on diversity, equity and inclusion (DE&I), but efforts have ramped up this year. Meta realised the risks of limiting research projects to developed or privileged regions, leading to skewed datasets against minority groups.

From November last year, it opened up participation to a much wider group of employees and contractors, as well as paid research participants, factoring in diverse genders, heights, body types, ethnicities, and capabilities.

Contributions to Academia

Meta also announced Ego4D last year, a consolidated knowledge base of egocentric (first-person) data that could soon power contextual AI. It is currently funded via academic contributions to institutions.

While it is unclear whether Meta will share all data collected through Ego4D, it is definitely a move in the right direction to unite 13 universities and labs across nine countries.

New Partnership with BMW

Meta also launched its partnership with Carnegie Mellon University and later teamed up with German automaker BMW – its first industry partner.

The two firms explored how companies can integrate AR in next-gen vehicles and how data from vehicles can help AR-assisted drivers. Meta also partnered with the National University of Singapore for further research.

User Manual Released

While the Menlo Park-based firm does not intend to release Aria as a consumer-facing product, Meta has uploaded a user manual document to the Federal Communications Commission (FCC).

The document is exhaustive and reveals a lot of new information about Meta’s Project Aria glasses. For instance, Meta does not have a visual AR component (data collection only) and the smart glasses, named ‘Gemini,’ are available with prescription lenses, work alongside a mobile app called Ariane, use a 4-camera setup, and rely on a Qualcomm chipset.

Future Directions

Meta aims to sign on more research participants, industry partners, and academic institutions for Project Aria to date.

Individuals collecting data with its smart glasses will wear special clothing and identification, allowing test subjects to delete their data at any time.

Currently, the company launched its real-world application of Project Aria: a 3D map called LiveMaps that users can access with future contextualised AI-powered AR glasses. Its newest Ray-Ban Stories smart glasses also hit the shelves this year, which allows users to capture photos and videos, listen to music, or take calls without AR overlays.