Jensen Huang talks to Mark Zuckerberg: AI open source, robots, smart glasses

Editor | Li Xinma

Caption | Bilibili @NVIDIA

The emergence of generative AI has completely overturned modern science. I don't remember any other technology that can impact consumers, businesses, industries, and science at such a rapid pace, crossing different fields from climate technology to biotechnology to physical science, and to every field. Generative AI is in the midst of this fundamental transformation...

On July 30th Beijing time, Monday evening local time at SIGGRAPH 2024 computer graphics conference held in Denver, NVIDIA CEO Jensen Huang made the above remarks.

This year marks the 50th anniversary of the SIGGRAPH conference, a special moment. Old Huang invited a special guest - Mark Zuckerberg (Meta founder, hereinafter referred to as "Zuckerberg"), and the two had a conversation about AI for about an hour, mainly covering two aspects:

How basic research drives breakthroughs in the field of AI

The empowering role of generative AI and open-source technology for developers and creators

In the era of artificial intelligence, the collision of ideas between the leaders of the two tech giants has attracted widespread attention in the industry. DoNews has summarized the views of the two individuals as follows:

01.

"Open source has great industry value"

On the day of the conference, Meta released a new tool called AI Studio, based on the LlaMa 3.1 model, which allows users to create, share, and design personalized AI chatbots. NVIDIA also deployed LlaMa 3.1 through the new NVIDIA AI Foundry service and NVIDIA NIM inference microservices.

Zuckerberg also showcased the first unified open-source AI model Meta Segment Anything Model (SAM2) that can segment objects in real-time in images and videos. SAM2 can segment any object in any image without any annotations and can track all shots in a video in real-time.

According to reports, Meta will release SAM2 under the Apache2.0 license, allowing anyone to use it.

Since developing large models on its own, Meta has been an important customer of NVIDIA. Huang revealed during the conversation, "Meta may have up to 600,000 NVIDIA GPUs." During the conversation with Jensen Huang, Mark Zuckerberg elaborated on his "open source philosophy."

He mentioned that Meta started building things like distributed computing infrastructure and data centers after some other tech companies, which from a technical perspective can be said to lack a competitive advantage. "Well, let's make it open, and then we can benefit from the ecosystem around it."

Meta has always been a leader in the "open" AI field. Recently, Meta launched free models like LlaMa and Segment Anything. "We do this not because we are altruistic - even though I think it helps the ecosystem. We do this because we believe it will make what we are building the best," Zuckerberg said.

In his view, there will always be a closed and an open, and there is a reason for both - it benefits both sides. But for the computing platform that the entire industry is building, "open" has great value. "This really shapes my philosophy in this regard, and also shapes my work with LlaMa and our work in AI virtual reality."

Jensen Huang also agreed to adhere to the open source route for large models, using a metaphor: "Like the leather jacket I'm wearing, I don't want to make it myself, but I hope someone will make it for me, so these leather materials need to be open source for everyone to use."

02.

"The next wave of AI is robots"

In the dialogue, Jensen Huang mentioned the concept of "world models," stating that the next wave of AI will be robots.

Currently, humanoid robots are becoming a field where various tech giants are competing. In the article "Humanoid Robots 'Eighteen Arhats' Who Will Stay Until the End? | Artificial Intelligence +," DoNews has compiled the performance of mainstream manufacturers in this field. The humanoid robot market is exploding with huge potential and energy, attracting capital injection. According to incomplete statistics, in the first half of 2024, there were over 22 financing events in the global humanoid robot field, with financing amounts exceeding 7 billion yuan.

NVIDIA also has many actions in the robot field. At this week's SiGGRAPH conference, NVIDIA announced that it will provide a full set of services, models, and software platforms for leading robot manufacturers, AI model developers, and software manufacturers globally to develop, train, and build the next generation of humanoid robots. The suite of products includes NVIDIA NIM microservices and frameworks for robot simulation and learning, as well as NVIDIA OSMO orchestration services for running multi-stage robot workloads.

Jensen Huang said, "We are advancing the development of the entire NVIDIA robot stack, opening access to global robot developers and companies, allowing them to use the platform, accelerated libraries, and AI models that best suit their needs."

However, the commercialization of humanoid robots is still at a primitive level, and due to the imperfect supply chain, many manufacturers are still some time away from achieving mass production and delivery 03.

"All glasses wearers will upgrade to smart glasses."

In the second half of the conversation, Jensen Huang asked Mark Zuckerberg about his views on the future of smart glasses. Zuckerberg stated that smart glasses will be an important form of future computer development. He believes that smart glasses will become the mobile version of the next generation of computing platforms, "almost everyone who wears glasses today will eventually upgrade to smart glasses."

"On the one hand, we have been building the technology needed for the ideal holographic AR glasses, and in the next few years, I think we will be close to it, it will still be quite expensive, but I think it will start to become a product. Another aspect is collaborating with the world's best glasses manufacturers," Zuckerberg mentioned.

When researching, DoNews found that Francesco Miller, CEO of Luxottica, the parent company of Ray-Ban, recently stated that Meta may invest in the company. Meta and Luxottica have established a strong and long-term partnership in the field of smart glasses, according to Miller.

Zuckerberg also mentioned previously that Meta is expected to release a brand new AR smart glasses by the end of this year, with performance far exceeding current Ray-Ban products, potentially completely disrupting traditional glasses.

"In the end, it will be like a beautiful pair of glasses." Zuckerberg expressed confidence in the conversation, "Currently, we already have camera sensors, you can take photos and videos, can be shared on Instagram, can make video calls on WhatsApp and livestream what you see to the other person... this sensor is exactly what you need to interact with AI."

Zuckerberg admitted that he had always thought that holographic AR glasses would appear before the AI era, but they have not been mass-produced until now. However, they are "luckily in a favorable position," with very high-quality AI, and will produce a range of smart glasses products at different price points and technological levels in the future, with the $300 smart glasses expected to be the most popular style, potentially attracting hundreds of millions of consumers.

As the conversation neared its end, Zuckerberg took out a box from behind him, inside of which was a new black jacket he bought for Jensen Huang. Huang also gave Zuckerberg his iconic leather jacket prepared for SIGGRAPH.

"Your jacket is obviously more valuable because it has been worn by (you)." Zuckerberg joked.

(Images in the text are from Bilibili @NVIDIA video screenshots, delete if infringing)