Microsoft has announced the release of its third lightweight AI language model, Phi-3 mini, which it claims offers similar performance to competitors while being optimized for mobile use. With 3.8 billion parameters trained on 3.3 trillion tokens, it builds on the previous Phi-2 model launched in late 2023 with 2.7 billion parameters.
The company conducted assessments on Phi-3 mini against academic benchmarks and internal testing, indicating its ability to compete with models like “Mixtral 8x7B and GPT-3.5”. It achieved a rating of 69% in massive multitask language understanding (MMLU) and 8.38 in multi-turn (MT) benchmarks.
Microsoft revealed that Phi-3 mini was developed using a scaled-up dataset training approach compared to Phi-2, incorporating heavily filtered web and synthetic data. Despite its competitive performance, the company acknowledged limitations due to the model’s size in storing factual knowledge, proposing augmentation with a search engine to address this issue.
The development of Phi-3 mini was guided by Microsoft’s responsible AI principles, with input from internal teams to curate tailored datasets aimed at reducing harmful response rates. The model is now available through Microsoft’s Azure cloud platform, as reported by Reuters.