No code, purely trained by video feeding! Tesla FSD 12 debut, "ChatGPT Moment" is here?

Elon Musk's highly anticipated Full Self-Driving (FSD) system may have reached its "ChatGPT moment."

Recently, Musk showcased the latest progress of Tesla's FSD Beta V12 in a 45-minute livestream. This is the first-ever end-to-end artificial intelligence (AI) autonomous driving system, which is considered Tesla's most significant upgrade.

During the livestream, it was evident that the Model S equipped with FSD Beta V12 performed smoothly, successfully navigating obstacles, yielding to pedestrians, making turns at intersections, and slowing down when necessary.

It is worth noting that Musk only intervened once during the livestream when the Model S, which was supposed to go straight, started following the green light for left turns while waiting at a red light. Fortunately, Musk and the engineer beside him intervened in time.

In the eyes of analysts, this update of FSD may herald a new era of semi-autonomous driving under human supervision. What makes it so special?

It should be noted that all driving decisions made by FSD Beta V12 are not based on familiar code, programming, or human settings. Its ability to complete driving tasks relies solely on a large amount of "fed" video data and the resulting training.

According to further explanations by NewSmart:

FSD AI now only inputs a large amount of video data to identify what needs to be done in different situations, without individually encoding each road element or scenario.

This allows Tesla to eliminate hundreds of thousands of lines of code in FSD V12, making it lighter and more flexible, while still being able to operate without a data connection in unfamiliar terrain.

The key point is that while FSD 11 still had over 300,000 lines of C++ code in its exclusive control stack, this is no longer the case with FSD 12.

Musk previously stated that the reason for choosing an "end-to-end" autonomous driving solution is because it is similar to how humans drive: "Input photons, output actions (control). Cameras and neural network AI are the correct general solution."

However, Musk also emphasized the importance of high-quality driving videos for training, as using data from skilled drivers is crucial for the success of FSD.

Furthermore, Musk mentioned that despite already spending $2 billion on training this year, more efforts are still needed. Tesla is about to launch a GPU cluster consisting of 10,000 H100 chips to train the new version of FSD. However, the current training still relies almost entirely on NVIDIA's GPUs, and Tesla's own Dojo supercomputer is used relatively sparingly, for the time being serving as a supplementary tool.

Technically, the most challenging aspect may still be the need for high-speed InfiniBand network to provide more computing power, but currently "InfiniBand is scarcer than GPUs".