OpenAI stated that it has enhanced GPT-6's capabilities in legacy programming languages such as COBOL, Fortran, and Visual Basic, achieving progress in two evaluations. The disclosed results include: GPT-6 Astra achieving an average pass rate of 87.4% across 20 programming languages, and GPT-6.1 Sol reaching 97.9% accuracy on the COBOL migration benchmark.
There is less training code available for these languages compared to languages like Python, but OpenAI states that they underpin critical global infrastructure. IBM estimates that COBOL supports over 40% of online banking systems and processes 95% of ATM transactions.
OpenAI developed two benchmarks to measure model progress: one tests the ability to build databases across programming languages, and the other evaluates the capability to translate COBOL into other languages.
How OpenAI tests GPT-6's legacy programming capabilities
MultiLangBench requires the model to create a SQLite-compatible database engine from scratch. OpenAI reports that, at maximum inference intensity, GPT-6 Astra achieved an average pass rate of 87.4% across 20 languages.
In comparison, GPT-5.6 Terra scored 37.9%. GPT-6 Luna achieved 40.3% in the same evaluation, slightly higher than Terra. OpenAI’s comparative results show that the improvements across different models vary.
Another metric evaluating GPT-6’s legacy programming performance focuses on code migration. The COBOL benchmark measures the translation of code into C++, Java, Kotlin, Python, or Rust, rather than building a database engine.
In this evaluation, GPT-6.1 Sol achieved an accuracy of 86.6% under low reasoning intensity, while GPT-5.6 Sol reached only 5.4%. After increasing the reasoning intensity, GPT-6.1 Sol's accuracy rose to 97.9%.
This article was generated with the assistance of artificial intelligence and reviewed by the editorial team.
