Meta Platforms is planning to release Llama 3 in July, with capabilities close to GPT-4, with up to 1.4 trillion parameters.
Meta Platforms hopes that Llama 3 can rival GPT-4, but has not yet decided whether to develop it into a multimodal model. With a maximum of 140 billion parameters, it is less than one-tenth of GPT-4.
On Wednesday local time, tech media The Information cited sources reporting that Meta Platforms is planning to release the Llama 3 models in July this year.
Llama 3 carries a significant mission.
According to the report, Meta Platforms hopes that Llama 3 can rival OpenAI's GPT-4, which has become a powerful multimodal model capable of handling longer texts and supporting image inputs.
However, a Meta Platforms employee revealed that since researchers have not yet started fine-tuning the model, the company has not decided whether Llama 3 will be multimodal. Fine-tuning is the process where developers provide additional data to existing models for them to learn new information or perform tasks.
The Meta Platforms employee also mentioned that Llama 3 could potentially have over 140 billion parameters, compared to Llama 2 released in July last year with a maximum of 70 billion parameters.
According to previous reports, the parameter scale of the GPT-4 model is around 1.8 trillion, which is less than a tenth of Llama 3.
Furthermore, before launching Llama 3, Meta Platforms is working to overcome an issue found in Llama 2 - the inability to handle any controversial questions.
Due to the safety guardrails added by developers in Llama 2, it would refuse to answer a series of questions considered controversial.
According to Meta Platforms employees, these guardrails made Llama 2 appear "too safe" in the eyes of senior leadership and model researchers. Researchers plan to relax the restrictions in this aspect for Llama 3, allowing it to interact more with users, provide background information, rather than just refusing to answer.
Expectations for Llama 3 are rising, but Meta Platforms still faces enduring talent competition.
Two sources mentioned that Louis Martin, the researcher responsible for the security of Llama 2 and Llama 3, left the company this month. One of the sources also stated that Kevin Stone, the head of reinforcement learning, also resigned this month.