Physical Address
304 North Cardinal St.
Dorchester Center, MA 02124
Executives and researchers leading the Mater AI effort were busy beating OpenAI’s GPT-4 model while developing Llama 3. The internal messages were released by a court on Tuesday In one of the company’s ongoing AI copyright cases, Kadrey v. Meta.
In an October 2023 message to Meta researcher Hugo Tuvron, Meta VP of Generative AI Ahmad Al-Dahle said, “Really… we need to aim for GPT-4. “We have 64k GPUs coming! We have to learn how to build boundaries and win this race.”
Although Meta publishes open AI models, the company’s AI leaders have been more focused on beating competitors who don’t typically publish their model weights, such as Anthropic and OpenAI, and instead gate them behind an API. Meta’s executives and researchers hold Anthropic’s Claude and OpenAI’s GPT-4 as the gold standard.
French AI startup Mistral, one of Meta’s biggest open source competitors, was mentioned several times in internal messages, but the tone was dismissive.
“Mistral is peanuts for us,” Al-Dahleh said in a message. “We should be able to do better,” he said later.
Tech companies are racing to one-up each other with cutting-edge AI models these days, but these court filings reveal just how competitive Meta’s AI leaders really were — and seemingly. still have. At various points in the message exchange, Mater talked about how the AI was “very aggressive” in getting the right data to train the lead llama; At one point, one executive even said that “Llama 3 is literally meant to take care of me,” in a message to colleagues.
Prosecutors in the case allege that Meta’s executives sometimes cut corners in their mad dash to train AI models by shipping, copyrighted books in the process.
Touvron noted in a message that the mix of datasets used for Llama 2 was “bad” and talked about how Meta could use a better mix of data sources to improve Llama 3. Touvron and Al-Dahle then talked about clearing the way Use the LibGen dataset, which contains copyrighted works From Cengage Learning, Macmillan Learning, McGraw Hill, and Pearson Education.
“Do we have the right dataset out there[?]said Al-Dahle. “Is there something you wanted to use but couldn’t for some stupid reason?”
Meta CEO Mark Zuckerberg has previously said he’s trying to close the performance gap between Lamar’s AI model and closed models from OpenAI, Google and others. Internal messages reveal intense pressure within the company to do so.
“This year, Lama 3 is competitive with the most advanced models and in some cases leading,” said Zuckerberg the letter From July 2024. “Starting next year, we expect future llama models to be the most advanced in the industry.”
Finally when meta Lama 3 was released in April 2024The Open AI model was competitive with the leading closed models of Google, OpenAI, and Anthropic, and outperformed Mistral’s open option. However, the data Meta used to train its models — data Zuckerberg gave the green light to use despite its copyright status — is facing scrutiny in several ongoing lawsuits.
