Mark Zuckerbergs augmented reality

Mark Zuckerberg has a magnificent vision for the meta-verse, and he hopes that one day you will also see the same thing – literally through a pair of augmented reality glasses.

Zuckerberg calls AR glasses a “holy grail” device that will “redefine our relationship to technology,” similar to the introduction of smartphones. During the special effects video announcing Facebook’s corporate branding for Meta last October, they acted as the connective tissue of his metaverse pitch, letting people play games and work with virtual people. Star Trek-style. At one point, Zuckerberg wore them while fencing with a hologram. “Do not be afraid to poke,” said his virtual sparring partner.

Zuckerberg may have high hopes for smart glasses, but the reality of the technology in the short term is far less high. The demonstrations during Zuckerberg’s Meta presentation, such as playing virtual chess on a real table with someone’s avatar, were not based on any working hardware or software. And Meta does not yet have a working, portable prototype of its planned AR glasses, but rather a stationary demonstration sitting on a table.

Still, Zuckerberg has ambitious goals for when his high-tech glasses become a reality. Employees race to deliver the first generation by 2024 and are already working on a lighter, more advanced design for 2026, followed by a third version in 2028. The details, which together provide the first comprehensive look at Meta’s AR hardware ambitions, were shared with The edge of people who are familiar with the roadmap and who were not authorized to speak in public. A spokesman for Meta declined to comment for comment.

Animating the push for AR glasses and Facebook’s rebrand to Meta is a wish on Zuckerberg’s part to once again make the company he founded as innovative, say people familiar with his mindset. The social network’s reputation has been tarnished by a series of scandals over privacy and content moderation, damaging employees’ morale and belief in management. Regulators are trying to break the company up and limit its business with personal advertising. And among its Silicon Valley peers, it has become known as a ruthless copycat.

If the AR glasses and the other futuristic hardware that Meta builds eventually catch on, they can put the company, and by extension Zuckerberg, in a new light. “Zuck’s ego is intertwined with [the glasses], ”Says a former employee who has worked on the project. “He wants it to be an iPhone moment.”

Meta’s CEO also sees the AR glasses, called Project Nazare, as a way to get out from under the thumbs of Apple and Google, which together dictate the terms that apps like Facebook must adhere to on mobile phones. The first version of Nazare is designed to work independently of a mobile phone using a cordless, phone-shaped device that relieves parts of the computer required for the glasses to work. A highlight feature will be the ability to communicate and interact with holograms of other people through the glasses, which Zuckerberg believes will eventually give people a more immersive, compelling experience than the video calls available today.

Despite already spending billions on developing its AR glasses, Meta has internally weak sales expectations in the low tens of thousands for the first version, which will be aimed at early adopters and developers. A price point has not been decided, but the device will definitely be more expensive than the company’s 299 $ Quest VR headset, as the AR glasses’ bill of materials is several thousand dollars. The costs will test Zuckerberg’s willingness to subsidize the price of hardware to encourage adoption – a competitive strategy that is partly aimed at undercutting the margins of other players such as Apple.

In addition to Nazare, a separate, previously unreported pair of cheaper smart glasses with the code name Hypernova is also planned for 2024. Nazare is designed to work independently of a smartphone, but Hypernova will pair with a nearby phone to show incoming messages and other messages via a smaller heads-up display similar to the northern smart glasses that Google bought two years ago.

Along with Nazare, Hypernova and future versions of the Ray-Ban camera-equipped glasses that it recently introduced, Meta hopes to sell tens of thousands of smart glasses by the end of this decade, its VP of AR, Alex Himel, has told employees.

It is unclear whether people will find AR glasses useful in the next few years. Similar products from Microsoft, Snap and others are far from mainstream. And the stakes could not be higher for Meta. Its department, which manufactures metaverse hardware and related software, has grown to about 18,000 people, costing the company $ 10 billion last year alone. To build the glasses and the VR hardware of the future, Meta has aggressively poached from Microsoft, Apple, Google and others, which has driven up the price of talent in the industry.

Zuckerberg has said he plans to increase the cost of building AR and VR hardware in the coming years, a huge effort that is happening while his business is under pressure from all sides. Meta’s stock has been hammered due to its declining social media business and younger users flocking to competitors like TikTok. Antitrust control has essentially ruled out major acquisitions that previously kick-started growth, such as the acquisition of Instagram and WhatsApp. And after beating Meta’s core ad business with the latest tracking changes in iOS, Apple is also preparing an attack on Zuckerberg’s hardware strategy, starting with an advanced mixed reality headset already this year and finally its own AR glasses.

Since Nazare was taken out of the research in 2018 with the internal codename Orion, Zuckerberg has shown a particular interest in the project. “It’s like Sauron’s eye,” says a former team member, referring to the all-seeing eye in Lord of the Rings. (Zuckerberg himself recently admitted in a podcast that employees sometimes use the phrase to explain his intense involvement in a project.)

Zuckerberg has insisted that the first version of Nazare offers a full AR experience with 3D graphics, a large field of view and a socially acceptable design. The team initially hoped it could boast a 70-degree field of view – far wider than what is currently on the market – but that goal is unlikely to be met. The current design of the glasses is a bit reminiscent of Superman’s black frame when disguised as Clark Kent. They weigh 100 grams, which is about four times more than a typical pair of normal glasses.

While Meta is racing towards a ship goal in 2024, there is no guarantee that Nazare will reach that goal. Its shipwreck has already slipped several times. There is still a lot of work on the product experience, especially on the software side. An attempt to build a custom micro-core operating system for the device based on a version of Google’s open source Fuchsia OS was scrapped at the end of last year, in part because it would not be ready in time by 2024. (The information previously reported the decision to scrap the micro-core OS.) Now Meta is pursuing a version built on top of Android for the first version of the AR glasses – a similar approach to what drives the company’s current Quest VR headset.

Nazare will not be a mainstream entity, at least not in the beginning. Its current battery life is only four hours and the glasses are intended to be used mostly indoors. Although it will take a while before the glasses sell in high volume, Zuckerberg has not spared the cost. The screens are powered by expensive custom waveguides and microLED projectors. The first version will have eye tracking and a front-facing camera along with stereo sound in the frame. Employees are working with semiconductor factories in Asia to build custom chips for the planned roadmap through the latter half of this decade.

Perhaps the most futuristic aspect of the first versions of both Nazare and Hypernova is a wrist unit that Meta plans to bundle together with the glasses to control them, hypothetically, with the wearer’s mind – something that is likely to be the company’s next big privacy hurdle. The bracelet uses differential electromyography or EMG to measure electrical impulses in the neurons of the arm, which essentially creates the effect of a phantom limb that the wearer can use to interact with the glasses. The result is that anyone can basically think of typing or controlling a virtual interface that Meta believes will help interact with smart glasses that do not have a touch screen, mouse, or keyboard. The technique is based on the company’s acquisition of about $ 1 billion from a startup called CTRL-Labs in 2019.

Everyone I’ve talked to and who’s tried a prototype of the band Meta is working on says it’s one of the most impressive tech demos they’ve ever experienced. If it works on a large scale, the company thinks it could have the next mouse and keyboard. One focus has been to get EMG to work through the screen and other technology built on the bracelet. “If CTRL-Labs works, none of these other things need to matter,” according to a former senior Meta employee.

In the short term, Meta plans to debut its first smartwatch ever as soon as this year. While the first and second versions will not have CTRL-Labs technology built in, the third generation is scheduled to include it and end with the debut of Nazare and Hypernova in 2024. Another version of the company’s smart glasses with Ray – Ban is also on his way, as The information recently reported. Meta sold about 120,000 pairs of the glasses with Ray-Ban during the period when they went on sale from September to December last year, and lacked its original target of 300,000. On the VR side, an advanced headset codenamed Cambria with pass-through video capable of blending the real and virtual worlds is ready for later this year, ahead of a similar device that Apple plans to release.

As Zuckerberg’s commitment to Nazare has deepened and the project has become more of a priority, the team has experienced a fair amount of revenue, with the recent heads of product, design and software going. He has installed longtime business leaders in key positions and reports up to CTO Andrew Bosworth. The Deputy Director of AR, who oversees all spectacle products under Bosworth, Alex Himel, has been with the company for 13 years. And the direct leader of Nazare under Heaven is Sue Young, who has been working on Facebook for a decade. Two other senior executives on the team are hardware engineer Caitlin Kalinowski, who recently switched from running hardware for Oculus, and former Microsoft director Don Box, who now runs software development for the glasses.

Although Zuckerberg has already put his claim to the meta-verse, it is going to take a long time for AR glasses to possibly get mainstream appeal. “You really have to be a missionary to see this through,” says one who has been involved with Nazare. “It will take decades.”