Phi-3 Mini: Microsoft’s Small Language Model for Smartphone Use

Microsoft has introduced a new small language model called Phi-3 mini, which is designed to run on modern smartphones and offers performance similar to OpenAI’s GPT-3.5. This new iteration of Microsoft’s lighter language model has been trained with 3.3 billion tokens from “larger and more advanced” data sets compared to its predecessor model, Phi-2, which was trained with 1.4 billion tokens. Phi-3 mini consists of 3.8 billion parameters, making it suitable for use in modern smartphones as it only occupies around 1.8GB of memory and can be quantified to 4 bits, according to a text published on Arxiv.org.

Researchers tested Phi-3 mini on an iPhone 14 with an A16 Bionic chip and found that it runs natively and completely offline, achieving more than 12 tokens per second. The overall performance of this model “rivals” that of larger models like Mixtral 8x7B and GPT-3.5. The model utilizes a transformer decoder architecture that supports a 4K text length and is based on a block structure similar to Meta’s Llama 2, benefiting the open-source community and supporting all packages developed for Llama 2. Phi-3 mini supports a conversational chat format and aligns with Microsoft’s robustness and security values.

In addition to Phi-3 mini, Microsoft has also trained two other models from the same family: Phi-3 medium with 14 billion parameters and Phi-3 small with 7 billion parameters, both trained with 4.8 billion tokens. The technology company’s emphasis on innovation and performance in the field of language models is evident in their latest offerings.