If they fixed it, then it wouldn't be SWE-Bench Pro anymore, right? It'd be "SWE-Bench-Pro-Fixed-OpenAI." I think it's better optics for the independence of the benchmark if the OpenAI team lets some third party do the fixing and release the improved benchmark.
...Although OpenAI did exactly that when they released SWE-Bench Verified, so maybe I'm talking out of my butt here.
...Although OpenAI did exactly that when they released SWE-Bench Verified, so maybe I'm talking out of my butt here.