Apple has unveiled its own language model that is capable of running locally on its devices. OpenELM is a publicly available model that empowers developers to create generative AI. This compact model is essential for local operations.
OpenELM, also known as Open-source Efficient Language Models, is a small language model primarily designed for text generation by artificial intelligence. While Apple has not specified the exact purpose of this tool, it is likely intended for developing AI applications that can operate on iPhones or Macbooks locally. Previously, Apple hinted at entering the generative AI landscape, albeit with vague details.
The OpenELM model comprises four variants, each varying in size, with parameters ranging from 270 million to 3 billion. In comparison, Meta's recent large language model for WhatsApp contains between 8 billion and 400 billion parameters, possible due to cloud-based operations. However, local processing demands smaller models, and an increasing number of companies are shifting focus towards compact models, as demonstrated by Microsoft's recent endeavors.
Apple has published a paper detailing the functioning of the model, indicating that it will be available as open-source software. OpenELM underwent training on various datasets, including the RefinedWeb dataset and a portion of RedPajama, a compilation of predominantly English-language content from platforms like GitHub, Wikipedia, and StackExchange. Furthermore, the model was trained using data from Reddit and Project Gutenberg.
English (US)