most benchmarks test if a model can solve a problem knowing everything up front, but what makes SlopCodeBench super interesting is that it discloses parts of the problem incrementally, forcing the LLM to redesign the codebase on the fly, lest it suffer the growing slop mountain
if you're thinking about building software factories or dark factories - this ep with @vaibcode is worth a watch
https://x.com/dexhorthy/status/2085764187507265980
🦄New Ep alert - If you've been following along, I'm obsessed with a new benchmark out of UW called SlopCodeBench - the best models in the world (Fable, Sol, Ki...