The same AI model can cost up to five times more to run depending on the coding agent harness it’s paired with, according to a new benchmark that tested 22 configurations across five open-weight models. Runta and Baseten ran those combinations 27,280 times for FrontierHarness Eval v1.1 , consuming roughly 83 billion tokens in the process. Nearly 65% of estimated inference spending went to runs that failed verification or timed out. The model itself accounted for 92.8% of the cost differences, but the harness still made a considerable difference. One configuration could be the cheapest option...
Source: The New Stack