With release of Deepseek V4 I wanted see how the model sizes are trending over time. Open source models are constantly getting smaller and better. The trend is that by this time next year, we probably will have Opus 4.5 level models on consumer grade laptops (sounds unlikely?!).
I was surprised to see that Deepseek V4 Flash is extremely smart and small enough to fit in setup that can be built with < $50,000. Expensive, but not a datacenter. So I wanted to see the trend over time of model sizes and their scores and created above plots using Opus/Sonnet 5.
I am not an expert in LLMs and I am sure there are physical limitations to small models. But I also don't know how far we are from reaching the limits - maybe the small models have a long way to go before being saturated. I am hoping the trend continues and, if it does, then we should get Opus 4.5 level models on Macbook Air/pro (AA score 30-40) by next year!
[I don't trust trend of scores above 40 that as there are very few data points]