What does GPT-4o mean? OpenAI joining forces with Apple, the unstoppable trend of AI smartphones!

Author: Li Xiaoyin

Source: Hard AI

ChatGPT Boosting Siri?

On the morning of May 13th local time, OpenAI unveiled the iterative version of GPT-4, named GPT-4o, at its spring conference. It is reported that GPT-4o is twice as fast as its predecessor and more user-friendly: voice activation, real-time conversation, no registration required, free to use.

OpenAI announced that GPT-4o, along with its text and image capabilities, will be rolled out to APIs and users starting today, with voice and video capabilities coming soon.

From a positioning perspective, the brand new flagship version GPT-4o seems to be challenging Siri.

However, recent reports revealed that Apple has reached an agreement with OpenAI to introduce ChatGPT technology in the new operating system iOS 18 to enhance Siri's conversational experience.

This raises the question of how GPT-4o and Siri will integrate and how ChatGPT and Apple will define the new generation of AI smartphones.

What's new in GPT-4o?

1) Multimedia Capabilities: GPT-4o has the ability to process text, images, videos, and audio, accepting inputs in various forms and generating responses in the same media format.

2) Faster Speed: GPT-4o is 5 times faster than the previous generation, with significantly improved voice latency. It can respond to audio inputs within 232 milliseconds, averaging at 320 milliseconds, approaching human conversation reaction times. This means users can have real-time conversations with GPT-4o, even directly engaging in video calls for instant answers.

3) Free and Open: Despite the ongoing "price war" in the AI industry, OpenAI is taking a different approach. Starting from the conference, GPT-4o will be available to all paying and free users of ChatGPT, removing all other restrictions, and reducing API prices by 50%.

As mentioned at the conference, the "o" in GPT-4o stands for "omni," meaning all-encompassing. With the current feature updates, the fully optimized GPT-4o has truly become an AI real-time voice assistant, surpassing Siri in performance.

During the demonstration, GPT-4o showcased additional features beyond its promotional points, such as real-time translation, emotion recognition, and the ability to analyze charts and graphs by recognizing scenes through the camera

How will "Apple + OpenAI" define AI smartphones?

With the next generation of the iPhone operating system planning to release new features based on LLM (Large Language Models), Apple is seeking third-party partners for this, including Google and OpenAI.

Currently, it seems that Apple and OpenAI are more compatible.

Analysis suggests that the collaboration between Apple and OpenAI can address each other's pain points in developing cutting-edge AI, truly achieving a win-win situation—

What does OpenAI need the most? Edge-side application permissions, system-level permissions, which only Apple can provide.

What does Apple need the most? The best AI technology, the most compatible large language model, and GPT-4o is undoubtedly the best choice.

Moreover, Apple also has unique advantages in self-developed chips and closed ecosystem. As Jim Fan, a senior research scientist at NVIDIA, commented on the X platform: Whoever wins Apple first, wins the victory.

I believe there will be 3 levels of integration with iOS:

1. Abandon Siri. OpenAI refines a smaller, pure device-based GPT-4o for iOS, with the option to pay for upgrades to use the cloud.

2. Transmitting the camera or screen to the model's native functions. Chip-level support for neural audio/video codecs.

3. Integrating iOS system operation APIs and smart home APIs. It's time for Siri shortcuts to make a comeback.

This could become an AI product with 1 billion users from the start, with OpenAI to Apple similar to "FSD for smartphones."

Looking ahead, what new growth stories can ChatGPT create by being introduced to the iPhone?

Wedbush analyst Dan Ives stated in a report on Monday:

"Embedding the OpenAI chat box in iPhone 16 will also open up new growth paths, allowing key developers and the Microsoft developer ecosystem to enter the Apple ecosystem together."

"Essentially, establishing a close partnership with OpenAI will change the game, and for Microsoft/OpenAI, bundling with the world's largest consumer electronics brand would be a wise choice."

Ives predicts that Apple will announce its partnership with OpenAI at the WWDC conference on June 10 and launch an AI chatbot based on Apple's LLM