OpenAI has made its Decisions API model widely available to users, entering the fast-growing, low-cost intelligent automation market—a market that recently propelled TypeSafe, a startup founded by a former Frontier Labs researcher, to a $7.6 billion valuation.
TypeSafe's Jev model pioneered a class of deep learning models that function like "super classifiers." Given a set of options, these models can rapidly make selections and provide a confidence score for their decisions. Software developers quickly realized that these decision models, or System One models, can replace large language models (LLMs) in various workflows, operating faster and at significantly lower cost.
Nikunj Handa, Head of API Products at OpenAI, told TechCrunch that after his colleagues saw Jev, they said, “Wow, we have about a million use cases.” Handa’s team then accelerated the launch of their own version: instead of training a dedicated, calibrated decision model, they reconfigured ChatGPT Luna to improve inference speed and reduce costs.
Handa said: "The biggest breakthrough of the Decisions API is speed—reducing processing time from 1 to 3 seconds down to 100, 120, or 180 milliseconds. In the future, we may conduct specialized training to make the model even better in these areas. But right now, it’s this ordinary Luna that’s getting the job done."
OpenAI unveiled this product at its Developer Day on September 29 and made it available to all customers on Thursday. The company also added new voice and visual modalities to the public version and has worked to reduce latency worldwide by deploying inference resources geographically.
In initial tests with customers such as Box, Datadog, Ramp, and Speak, engineers used this API to replace OpenAI’s LLM with higher throughput. Andrew Hsu, Chief Technology Officer at Speak, which develops an AI language tutor product, said the Decisions API reduced the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used this product to improve the response speed of its own Dots voice assistant.
However, what constitutes a good decision model is not yet fully understood, beyond “it works for you.” Optimization around intelligence, cost, calibration, and speed appears to vary by workload. Diogo Almeida, CEO of TypeSafe, is skeptical of benchmarks and encourages people to test these models in their own workflows—yet TypeSafe also released its own ranking last week alongside the launch of its latest Jev model.
Public benchmarks such as the AIM System One Index and the Artificial Analysis Decision Index tend to favor Liquid’s newly released D1 Ultra decision model and Base Lab’s Elephant-1. The latest Jev model remains competitive on both indices, ranking seventh and third respectively, while OpenAI’s Decision Model ranks eighth and fourteenth. However, at present, TypeSafe’s products are the most frequently queried on OpenRouter.
However, perhaps the most interesting Jev benchmark is JevMan, in which the decision-making model plays Pac-Man. The latest Jev models rank first in this category. Felix Wunderlich, Chief Growth Officer at Opper AI, which built this benchmark, says that this classic game “is an interesting test because it requires the model to respond quickly amid a continuous stream of high-stakes decisions. At every intersection, the model must decide which way to go within a few hundred milliseconds.”
OpenAI may have an advantage in multimodal capabilities. Handa told TechCrunch that “Luna is very strong in image classification and performs better in calibration,” and added that OpenAI clients also want the Decisions API to be more flexible when handling images as outputs.
Handa also believes that further increases in speed will enable new use cases: "Can we get it down to about 60 milliseconds? Then you could imagine calling an API for every frame in a video."
As these products become faster and cheaper, they may validate the economist Jevons named: that a cheap good becomes even more ubiquitous. As Handa said, decision models have revealed the vast array of use cases that were obvious but previously overlooked.