Behind Mark Zuckerberg big plans for AR glasses

Meta is racing to release its first AR glasses in 2024, but sources believe it’ll take a lot longer for them to become mainstream

Mark Zuckerberg has a grandiose vision for the metaverse, and he hopes that you’ll one day see the same thing, too — quite literally, through a pair of augmented reality glasses.

Zuckerberg calls AR goggles a “holy grail” device that will “redefine our relationship with technology,” akin to the introduction of smartphones. During the special effect-laden video announcing Facebook’s corporate rebrand to Meta last October, they acted as the connective tissue for his metaverse pitch, letting people play games and work with virtual humans Star Trek-style. At one point, Zuckerberg wore them while fencing with a hologram. “Don’t be scared to stab,” his virtual sparring partner quipped.

Zuckerberg may have big hopes for smart glasses, but the near-term reality of the technology is far less lofty. The demonstrations during Zuckerberg’s Meta presentation, such as playing virtual chess on a real table with someone’s avatar, weren’t based on any functioning hardware or software. And Meta doesn’t yet have a working, wearable prototype of its planned AR glasses but rather a stationary demonstration that sits on a table.

Still, Zuckerberg has ambitious goals for when his high-tech glasses will be a reality. Employees are racing to deliver the first generation by 2024 and are already working on a lighter, more advanced design for 2026, followed by a third version in 2028.​​ The details, which together give the first comprehensive look at Meta’s AR hardware ambitions, were shared with The Verge by people familiar with the roadmap who weren’t authorized to speak publicly. A spokesperson for Meta declined to comment for this story.

Mark Zuckerberg fences with a hologram during his metaversion vision presentation last October.Image: Meta

Animating the push for AR glasses and Facebook’s rebrand to Meta is a desire by Zuckerberg to cast the company he founded as innovative once again, people familiar with his thinking say. The social network’s reputation has been stained by a series of privacy and content moderation scandals, hurting employee morale and faith in leadership. Regulators are trying to break the company up and curb its business of personalized advertising. And among its Silicon Valley peers, it has become known as a ruthless copycat.

If the AR glasses and the other futuristic hardware Meta is building eventually catch on, they could cast the company, and by extension Zuckerberg, in a new light. “Zuck’s ego is intertwined with [the glasses],” a former employee who worked on the project tells me. “He wants it to be an iPhone moment.”

Meta’s CEO also sees the AR glasses, dubbed Project Nazare, as a way to get out from under the thumb of Apple and Google, which together dictate the terms that apps like Facebook have to abide by on mobile phones. The first version of Nazare is designed to work independently from a mobile phone with the assistance of a wireless, phone-shaped device that offloads parts of the computing required for the glasses to operate. A marquee feature will be the ability to communicate and interact with holograms of other people through the glasses, which Zuckerberg believes will, over time, provide people with a more immersive, compelling experience than the video calling that exists today.

Despite already spending billions on developing its AR glasses, Meta internally has tepid sales expectations in the low tens of thousands for the first version, which will be aimed at early adopters and developers. A price point hasn’t been decided, but the device will certainly be pricier than the company’s $299 Quest VR headset, given that the AR glasses bill of materials is multiple thousands of dollars. The cost will test Zuckerberg’s willingness to subsidize the price of the hardware to encourage adoption — a competitive strategy intended in part to undercut the margins enjoyed by other players like Apple.

In addition to Nazare, a separate, previously unreported pair of cheaper smart glasses codenamed Hypernova are also planned for 2024. Nazare is designed to operate independently of a smartphone, but Hypernova will pair with a nearby phone to show incoming messages and other notifications through a smaller, heads-up display, similar to the North smart glasses Google acquired two years ago.

Together with Nazare, Hypernova, and future versions of the Ray-Ban camera-equipped glasses it recently introduced, Meta hopes to be selling tens of millions of smart glasses towards the end of this decade, its VP of AR, Alex Himel, has told employees.

A demo illustrates how Meta imagines AR glasses could work to let someone play chess with a hologram.Image: Meta

It’s unclear if people will find AR glasses useful in the next few years. Similar products from Microsoft, Snap and others are far from mainstream. And the stakes couldn’t be higher for Meta. Its division making metaverse hardware and related software has swelled to roughly 18,000 people, costing the company $10 billion last year alone. To build the glasses and future VR hardware, Meta has aggressively poached from Microsoft, Apple, Google, and others, driving up the price of talent across the industry.

Zuckerberg has said he plans to increase spending on building AR and VR hardware in the years to come, an immense bet happening while his business is under pressure from all sides. Meta’s stock has been hammered due to its slowing social media business and younger users flocking to competitors like TikTok. Antitrust scrutiny has essentially ruled out big acquisitions that kickstarted growth in the past, like the purchase of Instagram and WhatsApp. And after kneecapping Meta’s core ads business with recent tracking changes in iOS, Apple is readying an assault on Zuckerberg’s hardware strategy, too, starting with a high-end, mixed reality headset as soon as this year and eventually its own AR glasses.

Since Nazare was brought out of research in 2018 with the internal codename Orion, Zuckerberg has shown a special interest in the project. “It’s like the eye of Sauron,” one former team member says, referring to the all-seeing eye in Lord of the Rings. (Zuckerberg himself recently admitted in a podcast that employees sometimes use the phrase to explain his intense involvement in a project.)

Zuckerberg has insisted that the first version of Nazare offer a full AR experience with 3D graphics, a large field of view, and a socially acceptable design. The team originally hoped for it to boast a 70-degree field of view — far wider than what’s currently on the market — but that goal likely won’t be met. The current design of the glasses slightly resembles Superman’s black frames when he’s disguised as Clark Kent. They weigh 100 grams, which is about four times more than a typical pair of normal glasses.

While Meta is racing towards a 2024 ship goal, there’s no guarantee that Nazare will meet that target. Its ship year has already slipped multiple times. Work is still very much underway on the product experience, especially on the software side. An effort to build a custom microkernel operating system for the device off a version of Google’s open-source Fuchsia OS was scrapped at the end of last year, in part because it wasn’t going to be ready in time for 2024. (The Information earlier reported the decision to scrap the microkernel OS.) Now Meta is pursuing a version built on top of Android for the first version of the AR glasses — a similar approach to what powers the company’s current Quest VR headset.

Nazare won’t be a mainstream device, at least not at first. Its current battery life is only four hours, and the glasses are intended to be used mostly indoors. Even though it will take a while before the glasses sell in high volume, Zuckerberg has spared no expense. The displays are powered by costly custom waveguides and microLED projectors. The first version will have eye tracking and a front-facing camera, along with stereo audio in the frame. Employees are working with semiconductor fabs in Asia to build custom chips for the planned roadmap through the latter half of this decade.

Last October, Meta presented a conceptual demo of how its EMG wristband could work to let people type without a physical keyboard.Image: Meta

Perhaps the most futuristic aspect of the first versions of both Nazare and Hypernova is a wrist device Meta plans to bundle with the glasses for controlling them, hypothetically, with the wearer’s mind — something that will likely be the company’s next big privacy hurdle. The wristband uses differential electromyography, or EMG, to measure electrical pulses in the arm’s neurons, essentially creating the effect of a phantom limb the wearer can use to interact with the glasses. The result is that someone can essentially think to type or control a virtual interface, which Meta believes will aid in interacting with smart glasses that don’t have a touchscreen, mouse, or keyboard. The tech is based on the company’s roughly $1 billion acquisition of a startup called CTRL-Labs in 2019.

Everyone I’ve talked to who has tried a prototype of the band Meta is working on says it’s one of the most impressive tech demos they’ve ever experienced. If it works at scale, the company thinks it could have the next mouse and keyboard. A focus has been getting the EMG to work through the display and other tech built onto the wristband. “If CTLR-Labs works, none of this other stuff needs to matter,” according to one former senior Meta employee.

In the nearer term, Meta is planning to debut its first-ever smartwatch as soon as this year. While the first and second versions won’t have CTRL-Labs tech built in, the plan is for the third generation to include it and tie up with the debut of Nazare and Hypernova in 2024. A second version of the company’s smart glasses with Ray-Ban is also in the works, as The Information recently reported. Meta sold about 120,000 pairs of the glasses with Ray-Ban during the period when it went on sale last September through December, missing its initial goal of 300,000. On the VR side, a higher-end headset codenamed Cambria with pass-through video capable of blending the real and virtual worlds is being readied for later this year, ahead of a similar device Apple is planning to release.

As Zuckerberg’s involvement with Nazare has deepened and the project has become more of a priority, the team has seen a fair amount of turnover, with the recent heads of product, design, and software departing. He has installed longtime company leaders in key positions, reporting up to CTO Andrew Bosworth. The VP of AR who oversees all glasses products under Bosworth, Alex Himel, has been at the company for 13 years. And the direct leader for Nazare under Himel is Sue Young, who has worked at Facebook for a decade. Two other senior leaders on the team are hardware engineering head Caitlin Kalinowski, who recently moved over from running hardware for Oculus, and former Microsoft executive Don Box, who now runs software engineering for the glasses.

Even though Zuckerberg has already staked his claim on the metaverse, it’s going to be a long time for AR glasses to possibly garner mainstream appeal. “You have to really be a missionary to see this through,” someone who has been involved with Nazare says. “It will take decades.”