I've been trying to grapple with what the key inputs are to the open-closed performance gap， and how they're changing.

Until the training paradigm changes， open weight models will pretty clearly be able to fast-follow closed labs. There are sources of uncertainty， but that fact of keeping up seems hard to shake. I spent a long time looking for evidence of or arguments supporting open models falling behind， but it's not there at all today.

Things I consider include： - How benchmarks evolve over time， becoming more or less correlated with how people actually use models， - How different models' real-world performance relates to their benchmark rankings， and - How training regimes evolve over time to move said benchmarks.

I roughly conclude that this equilibrium will last until economic forces initiate a change in strategy， or training needs shift.