EleutherAI releases massive AI training dataset of licensed and open domain text

AI research agency Eliotherai has published licensed for AI models and one of the largest collections in the open-domain text claims.

The datasate, known as Common Pile V 0.1, took about two years to cooperate with AI Startups Pullide, Embrace face and several academic organizations. In size 8 terabytes, ordinary pile V 0.1 Eliotherai, comma V 0.1-1 T and comma V.1-2 were used to train two new AI models from 0.1-2, which aliuothirai claimed that he did equal performance with developed models using licensed, copyrighted data.

AI agencies including Openai are involved Case Their AI training practices depend on web scraping with copyrighted materials such as books and research journals to create model training datasets. Some AI agencies have licenses with specific content suppliers, but most are maintained that the legal doctrine of fair use in the United States protects them from their responsibility where they were trained without permission.

Eliotherai argued that these cases were “severely reduced” from AI companies, which the organization said that the models damaged the area of ​​AI research field to understand how models work and what their defects might be.

“[Copyright] Data Sourcing has not changed in practice with the case litigation [model] Training, however Blog post Time to hug early on Friday. “Researchers have also quoted the litigation, especially the researchers we have talked to because they cannot publish the research in the high information -centric region.”

Common Pile V 0.1, which can be downloaded from the AI ​​Dev platform of the hug face and Github, was created with law experts, and it draws sources, including 300,000 public domain books digitized by the Congress library and Internet archive. Eliotherai also used the Open Source Speech-to-Text model whipper in the Open Source to replicate audio content.

Eliotherai claims that there are evidence that comma V 0.1-1 T and comma V 0.1-2 evidence that in common pile V.1, there was considerable correction with caution to enable developers to create competitive models owned with alternatives. According to Alytherai, both models, both in size and coding for coding, image understanding and math for math, were trained in a fraction of only one Common Pile V 0.1 in the first Lama AI model like Meta AI model.

Parameters, sometimes referred to as weight, are an internal component of an AI model that guides its behavior and answers.

“In general, we think that the general idea of ​​running a licensed text drive performance is unreasonable.” “As accessible is publicly licensed and public domain data increases we can expect to improve the quality of trained models in publicly licensed content.”

The general pile V0.1 seems to be present as an attempt to entrust the right to the Historical Tihasic wrongdoing of Eliotherai. A few years ago, the company published The Pile, an open collection of training texts that contains copyrighted material. AI companies are for the use of pile for training – and are under legal pressure.

Elethirai is promising to publish open datasets to move forward more frequently with the help of its research and infrastructure partners.