A receptionist at a dental clinic makes small judgments all day. A patient writes in, and someone has to work out what they want and whether the request can go straight to the calendar. None of this is hard, but it has to be right, and on a phone line or a chat it has to be quick.
Our products include front-desk agents, so we test the models behind them on the work itself. On 27 September we ran an internal production benchmark: one controlled dental booking scenario, the same for both models, one call each. The models were Jev 1.13 and GPT-6 Luna.
The two models
Jev comes from TypeSafe AI, which announced it on 15 September. TypeSafe calls it a System One model. It doesn't write text. You give it the current state (here, the patient's message) and a set of typed questions, each with a fixed set of allowed answers, and it returns an answer to each with a probability attached. TypeSafe prices it at $0.042 per million input tokens, with output free, and quotes response times of 70 to 500 milliseconds. TypeSafe is open about the limits too: Jev gives up text generation, and guides to it note that it can't do arithmetic or compare dates.
GPT-6 Luna is the efficient model in OpenAI's GPT-6 series, meant for high-volume work such as classification and chat. It lists at $0.10 per million input tokens and $0.50 per million output tokens. It has several reasoning effort settings, and we ran it with reasoning off.
How we set it up
We split the job in two. The model reads the patient's words and makes five judgments about them. Everything after that, such as checking open slots and making the booking, is handled by our own scheduling code, and both models fed the same code. We did this on purpose. Calendars need exact answers, and we would rather have plain code check a slot than trust any language model with date arithmetic. Jev isn't meant for that job anyway.
The split also keeps the comparison fair. The only thing that changed between the two runs was the model reading the message.
What we saw
Both models got all five judgments right. On accuracy, this run doesn't separate them.
Speed and cost did. Jev 1.13 answered in 806 milliseconds. GPT-6 Luna, with reasoning off, took 2,208 milliseconds. Our estimated cost was $0.0341 per 1,000 calls for Jev and $0.0841 per 1,000 calls for Luna. Jev took 63.5% less time and cost 59.4% less, in this run.
Our benchmark · Dental booking · One call per model
Both got 5 of 5 right.
The models read the patient's message. Our shared code did the scheduling.
Response time
Estimated cost per 1,000 calls
Both cost figures are tiny. Neither model would be a real line in a clinic's budget, so for us the time matters more than the money. A second and a half is a long pause on a phone call.
Our 806 milliseconds is slower than the 70 to 500 milliseconds TypeSafe quotes. We timed the call from our side, so the figure includes the network and our own code around the call. We haven't broken it down yet.
What this does and doesn't show
This was one call per model on one controlled scenario. It shows that both models handled this message correctly, and that Jev was faster and cheaper on that single call. It doesn't show which model is better in general. It also doesn't give us a response time we would plan around, because one timing says nothing about how much the next call might vary. A messier message could change the picture, and so could switching Luna's reasoning on.
We are also comparing two different kinds of model. Luna can write a reply to the patient and Jev can't. In our setup that doesn't matter, because the reply and the booking come from elsewhere. In a setup where one model does everything, it would.
What we'll test next
The next round runs each model many times on the same scenario, so we can report a typical response time and how slow the slowest calls get. Then we add harder messages: vague requests, a patient who changes their mind, someone who mentions pain, and messages in Arabic. We want to see where each model gets a judgment wrong, and whether Jev's probabilities drop on exactly those cases. If they do, that gives us a clean rule for when the agent should hand a conversation to a person.
Until then, the result is what it is. Same five judgments, less time and cost in this run.
Sources
- ADROLAB internal production benchmark, dental booking scenario, 27 September 2026 (our own run)
- TypeSafe AI, Introducing System One Models and Jev
- DataCamp, Jev: TypeSafe's System One model explained
- Flavio Copes, A deep dive into Jev (version and limits)
- OpenAI API docs: GPT-6 Luna