A benchmark for long-horizon software building: seventeen full-stack apps, 241 feature requests, and 544 test plans, showing that even top models only ship fully working products 49–65% of the time.
Why letting frontier models choose their own effort level, handoffs, and subagents beats prescriptive routing — the bitter lesson of harness design, applied to Replit Agent.