Fantastic in depth guide about Microsoft MAI by @eliebakouch
tl;dr about the model: Respect where respect is due.
-zero synthetic data or distillation from previous models. -1T model with 35B active, trained on 33.5T tokens
microsoft MAI tech report is a gold mine, one of the most transparent for a model at this scale. this model uses zero synthetic data or distillation from previo...