Meta late-night bombshell: The first sky-high AR glasses debut! Hands-on experience, Jensen Huang personally takes the stage

Meta's first AR glasses Orion finally unveiled!

The annual event Meta Connect 2024 was held in Menlo Park, California. As usual, Mark Zuckerberg carefully took out the long-awaited trump card from the safe with handcuffs worn by Meta employees - the first AR glasses Orion. According to Meta, the production cost of each pair of glasses is about $10,000 (approximately 70,000 RMB), making it possibly the most expensive AR glasses in history.

Orion features a split design, uses waveguide design, and has a heavy black frame, weighing 98g. These AR glasses can be paired with Meta's self-developed EMG wristband, offering various interactive features such as hand tracking, eye tracking, and head tracking.

From the demonstration video, users wearing Orion can browse the web, watch short videos, and even play some simple 3D games. Of course, you can also listen to music and make video calls on Orion.

However, the current Orion glasses are only used for demonstration and internal development kits, with only about 1,000 pairs produced, and not officially available for sale.

This time, Meta officials hardly provided any parameter information about Orion, such as chips, optics, and battery life. Mark only invited a group of friends to experience these AR glasses and endorse them. NVIDIA CEO Jensen Huang commented when experiencing these AR glasses, "The display is great, tracking is great, and the colors are also very good."

But don't be too surprised yet, as many tech bloggers in the industry have already received Orion for in-depth testing.

We have further summarized detailed information about Orion:

1. Split design for AR glasses: Orion glasses themselves (98g), independent computing unit, EMG wristband ;

2. Basic hardware configuration: 7 cameras, custom chips, silicon carbide waveguide lenses, Micro LED micro displays ;

3. Cost: The cost of each pair of Orion glasses is as high as $10,000, with only 1,000 pairs produced and not for sale ;

4. Orion 2nd generation: Meta's first consumer-grade AR glasses, thinner lenses, non-silicon carbide waveguide lenses, higher resolution ;

In addition, Meta also launched the affordable version of Quest 3 called Quest 3S, with the main chip using the Qualcomm Snapdragon XR2 Gen 2 chip. The basic configuration is similar to Quest 2, with the 128GB version priced at $299.99, available for sale on September 25th, and shelves on October 15th Meta also demonstrated how generative AI can be integrated with its own social ecosystem, launching Meta AI advanced voice functions, AI videos, and AI translation features.

In terms of large models, Meta introduced the first open-source model Llama 3.2 that can simultaneously process images and text. This model can create AI applications such as AR applications that provide real-time video understanding, content-based image search engines, or long-text analysis.

After the conference, Meta's stock price continued to rise, reaching a peak of $575.77 per share at the time of writing.

01 Silicon Carbide + Micro LED Waveguide Solution, with an ultra-wide 70-degree field of view

One of the most impressive displays of Meta's strength is Orion's optical design, showcasing Meta's generous use of materials.

Orion adopts a diffraction optical waveguide design, with waveguide lenses made of silicon carbide, which are lighter, more durable, and have a high refractive index. Meta's Chief Technology Officer Andrew Bosworth commented on these lenses, saying, "You can't imagine how low the yield is."

Furthermore, Meta has chosen Micro LED microdisplay, and from the images displayed at the conference, we can infer that a three-in-one light engine solution is most likely used.

In terms of video display effects, Orion glasses feature a significantly large rhomboid quadrilateral waveguide lens in the center, which can provide a wider display area. Evaluations show that Orion has a 70-degree ultra-wide FOV, far exceeding the industry standard of 25 degrees FOV for conventional waveguide AR glasses.

02 Separate design, using customized chips

Orion adopts a separate design, which means that the AR glasses also have a separate computing unit that handles the main computing power. Meta refers to it as "a hockey puck that can be ignored."

Specifically, Orion's position tracking, hand tracking, eye tracking, and some graphic algorithms will run on the glasses. Some rendering and regular computing processes run on the controller and wirelessly transmit the data results to the glasses.

Meta mentioned that Orion's glasses end uses a custom chip. With Jensen Huang's rare appearance, many netizens speculate that this custom chip may be created by NVIDIA. At the same time, the lightweight magnesium frame on the glasses end can dissipate heat quickly.

03 Compatible with electromyography (EMG) wristband, supporting four major interaction methods

In terms of interaction, Orion may be the AR glasses that support the most interaction forms in the industry, supporting eye movement, head movement, voice, and gesture as the four major interaction methods.

Meta has specifically developed an EMG wristband for Orion. The wristband mainly uses electromyography (EMG) technology, adopts high-performance textile materials, and can track subtle finger movements by sensing muscle activity on the wrist.

Currently, the wristband can recognize several basic operations: pinching the index finger with the thumb can select content; pinching the middle finger and thumb can call up or hide the application launcher; making a coin flipping gesture with the thumb on a closed palm can scroll up or down.

Combined with eye movement interaction, the EMG wristband of Orion mainly uses the eyes as a pointer and pinching fingers as clicks. This is also similar to Apple's interaction logic.

Meta stated that the EMG wristband is not limited by the glasses' camera's field of view, allowing users to swipe, click, scroll, and perform other operations from any position. At the Connect conference two years ago, Meta demonstrated a series of cool operations with EMG wristbands, among which the coolest was the AR glasses projecting a keyboard, and users input text using the EMG wristband.

04 Equipped with 7 cameras, supporting SLAM spatial positioning

According to The Verge's reporter, the frame of the glasses is embedded with seven cameras, which can fix virtual objects in real space, assist in eye and hand tracking, and let Meta's AI assistant understand what you are looking at. The design of these cameras around the frame of the glasses is similar to Apple's Vision Pro design.

Users can keep virtual windows open, turn their heads and walk away, as long as the glasses are still on, the window will still be there when the user comes back. This means that Orion has SLAM spatial positioning function, which can be considered as true AR glasses.

05 Games with almost no delay, playing table tennis with Mark Zuckerberg

In terms of content, Orion's main function currently is to combine with Meta AI for some basic interactions, play some simple AR games, achieve hands-free video calls, and share multiple virtual screens simultaneously.

For example, The Verge's reporter had a 3D virtual table tennis match with Mark Zuckerberg face to face. The two Orion users need to scan a QR code to pair the glasses, and then use gesture tracking to control the paddles. "The game has almost no delay," said The Verge's reporter.

The reporter also used Meta AI in Orion to identify ingredients placed on the table and make smoothies. A few seconds later, it correctly labeled various ingredients and generated a smoothie recipe guide in a window above the ingredients.

06 Orion 2nd generation will be launched in the coming years, possibly the first consumer-grade AR glasses

Meta revealed that the second generation of Orion will be launched in the coming years, which may be Meta's first consumer-grade AR glasses, with a price comparable to today's smartphones and laptops.

It is understood that the lenses of the first consumer-grade AR glasses will not be made of silicon carbide, the field of view will not be as large, but the resolution will be higher. If everything goes according to plan, the thickness of the frame will be only about half of the current Orion glasses Orion product manager Rahul Prasad revealed: "Having a broader vision is a scientific issue. The next stage is an engineering problem, that is, to achieve higher resolution, higher brightness, and lower cost.

Zuckerberg mentioned that AR glasses must be good for users to want to wear them on their faces. AR glasses need to provide some basic services even when users are not using AR. He also mentioned that the development of AR glasses is not a one-time thing, different smart glasses will serve different functions, such as Ray-Ban smart glasses suitable for some light application scenarios.

07 AI Video, AI Translation, AI Advanced Voice, Meta AI Builds Content Creation Toolbox

Meta and OpenAI successively launched AI Advanced Voice function, which can help AI models match the voices of many well-known actors. Users can choose the voices of Dame Judi Dench, John Cena, Keegan Michael Key, and Kristen Bell to interact with.

At the conference, Zuckerberg had a conversation with AI version Stevenson. Stevenson's AI assistant answered questions about his creative process. At the same time, Meta AI can directly convert Spanish videos into English videos. "Compared to text, interacting with AI through voice is a better way." Zuckerberg said.

At the same time, Meta AI also helps in the function upgrade of Ray-Ban smart glasses. Now, after combining Ray-Ban smart glasses with AI, users can directly use smart glasses to record their cooking process or one-click record their parking location.

If you come across a poster on the street, you can also directly use smart glasses to scan related QR codes or make a phone call with one click. Nowadays, Ray-Ban smart glasses also support AI translation function. Users can directly translate their speech with the help of AI. It can be said that this function directly creates a simultaneous translation assistant for users.

However, during the live demonstration, Meta AI's translation seemed to fail and could not translate the long conversation between Zuckerberg and the Spanish-speaking presenter.

Zuckerberg said that the conversation function of Meta Ray-Ban smart glasses is getting stronger, they will be able to better remember what you see. They will also be equipped with multimodal artificial intelligence to provide real-time assistance to users

08 AR Glasses Boosted, Meta Regains Throne with Generative AI

In 2024, Meta successfully reversed the dilemma of its metaverse development with breakthroughs in generative AI, causing its stock price to soar and reclaim a spot in the top ten market value of US stocks. As a leader in the AVR field, Meta's products and technologies showcased at the Connect conference often become the focus of the industry.

In the past few years, the headsets represented by the Quest series have always been the main theme of Meta Connect conferences. We have also seen Meta's exploration in different areas with the high-end Meta Quest Pro, consumer-level Quest 3, and Quest 3. However, this year, Meta finally made a long-awaited move in the AR direction - the first AR glasses, Orion.

Compared to common AR glasses in the industry, Orion is quite generous in materials. The most stunning feature is the ultra-wide field of view, which showcases Meta's ceiling in AR glasses. However, from the existing videos, it seems that there are still issues like dropped frames and unclear display.

Furthermore, Meta possesses powerful AI underlying capabilities that other AR manufacturers do not have. How will the combination of Ray-Ban smart glasses and AI gameplay be received by users? This may affect the changes in the market for smart glasses without display functions.

"I once thought that the metaverse could be achieved before AI appeared." Zuckerberg said, "The twist of fate is interesting, as AI is erupting in the metaverse before it can truly become popular."

Silicon Rabbit, original title: "Meta's Late-Night Bombshell: First High-priced AR Glasses Released! Hands-on Experience, Jensen Huang Personally Presents"