Apple has introduced the Open-source Efficient Language Models family, specifically designed to provide precise results on devices like laptops while using fewer training tokens compared to other AI models such as OLMo. Known as OpenELM, this family emphasizes a layered scaling strategy to efficiently assign parameters within each layer of the model, enabling users to achieve more accurate results for specific tasks.
The OpenELM family consists of four large language models (LLMs) available in various sizes: 270 million parameters, 450 million, 1.1 billion, and 3 billion parameters. Each model has two versions: pre-trained and optimized. The pre-trained variant is a generic model trained with data sets from the CoreNet library on GitHub, while the optimized version is fine-tuned for specific purposes, as detailed in a research document published on Arxiv.org.
Apple researchers conducted tests using OpenELM on a MacBook Pro with an M2 Max SoC and 64 GB of RAM running macOS 14.4.1, as well as on a computer with an Intel i9-13900KF CPU, DDR5-4000 DRAM, and an NVIDIA RTX 4090 GPU with 24 GB of VRAM. The results showed that OpenELM performs more efficiently than similar LLMs like Elm, boasting a 2.36 percent improvement in accuracy while requiring half the pre-training tokens.
Apple has emphasized that OpenELM models have been trained using publicly available data sets and do not come with any security guarantee, warning of the possibility of inaccurate results, harm, or manipulation. The OpenELM repository is available on Hugging Face, offering users access to these powerful language models for a range of tasks.