Meta's new AI device is in full swing: exploring the development of AI earphones with cameras that can translate

As multiple tech companies are seeking to develop AI wearable devices, Meta has also begun the research and development of new AI devices. It is understood that the Meta platform is exploring the development of AI earbuds with cameras, hoping that these devices can be used to identify objects and translate foreign languages.

Product design undecided, technical and privacy issues unresolved

Insiders revealed that Meta's CEO Mark Zuckerberg has reviewed several possible device designs but is not satisfied with any of them. The internal project is named Camerabuds, and it is currently unclear whether the final design of the product will be in-ear earbuds or over-ear headphones.

Furthermore, the project's timeline is also unclear. An insider stated that the company's leadership originally expected to approve a design in the first quarter. However, employees discovered several potential issues with the project, including long hair possibly obstructing the camera on the earbuds.

At the same time, placing cameras and batteries in a small device may make the earbuds bulky and the potential heat generated uncomfortable for users. Moreover, installing hidden cameras on wearable devices may also raise privacy concerns, as seen with Google Glass. Meta declined to comment on this.

Tech companies making a big push for AI earbuds

Currently, several other tech companies are also developing AI wearable devices, exploring new uses for AI, hoping to be as revolutionary as smartphones. OpenAI's CEO Sam Altman recently told a partner that he and former Apple Chief Designer Jony Ive's design company LoveFrom have been exploring the development of camera-equipped earbud devices.

There are also reports that Apple is exploring adding cameras and AI to its headphones. In addition, several startups, including Humane's AI Pin and Rabbit's R1 pocket device, have recently launched small AI devices, although these products have received criticism. Altman and a spokesperson for LoveFrom did not respond to requests for comments.

Insiders at Meta said that Meta's leadership, including Zuckerberg and Chief Technology Officer Andrew Bosworth, want to develop AI earbuds because they believe Meta's competitors will develop similar devices.

Media reports state that Bear Clark of electronic product company Ear Micro mentioned that tech companies' interest in developing "hearable devices" has surged in the past eight to ten months. Clark said that two large tech companies recently contacted him to develop smart earbuds, although he refused to disclose the names of these companies. When asked about collaborating with Meta, he declined to comment, citing a confidentiality agreement. However, he mentioned that like many other tech companies, Meta is "very interested in this field."

Clark also mentioned that to manufacture earbuds with cameras, the cameras can be placed on the circular exterior of the earbuds to capture an ultra-wide angle. He said that the images from both earbuds "can then be stitched together" to create a "complete 360-degree image." Analysis believes that for companies like Meta, adding a camera to headphones will allow users to draw their surroundings. Using AI, the device can tell users what they are looking at and how far they are from the office.

However, just like the previous hype surrounding augmented reality and virtual reality devices, it is unclear whether consumer demand for AI wearable devices will reach the level hyped by the industry.

Crowded Track Meta has learned from past experiences

It is understood that Meta previously attempted to develop headphones capable of recognizing simple commands in 2019, such as asking for the time, whether any recent Facebook posts had received comments, and requesting reminders, but eventually abandoned the project because the leadership believed that such a product would have a hard time standing out in the already mature headphone market.

By 2022, Meta applied for a patent for in-ear earbuds that can transmit sound directly to the eardrum bypassing the ear canal; the patent was approved in October 2023. A prototype of this device was internally demonstrated in 2021, designed to make audio sound as if the people in the same space as the wearer were speaking.

A former Meta employee revealed that after the popularity of OpenAI's ChatGPT, Meta's leadership began asking employees what AI-led devices they could develop. Since then, Meta has accelerated the work of introducing generative AI into more products and has invested billions of dollars, although the return on investment path is not yet clear.

In addition, Meta has heavily invested in audio and acoustic technology, including acquiring Audio Analytic, a manufacturer of sound recognition software, in 2022, and acquiring Whisper, a developer of AI-enabled hearing aid systems, in 2023.

However, Meta has faced difficulties in developing successful hardware products in the past. The company previously abandoned plans to manufacture smartwatches, with the first watch including two cameras and allowing users to detach the dial to take photos. Meta also scaled back plans to develop AR glasses and canceled its Portal smart home device series.

Analysis suggests that Portal has faced challenges to some extent, partly because the product was launched several years after Amazon and Google began selling smart home devices and gained consumer popularity. Meta may face a similar situation in the headphone market, as competitors like Apple and Google are already manufacturing headphones. Additionally, Meta does not produce natural products like phones that are used in conjunction with glasses and headphones, putting it at a further disadvantage in the hardware field.

Meta's stock price fell 2% in mid-Monday trading to $466.67. The company's stock price has risen more than 34% year-to-date.