Same request · two output paths
Generate the answer or score the options
MESSAGE
“I was charged twice for the same order.”
SYSTEM PROMPT
Classify this ticket. Choose billing for payment issues, technical for product failures, or review when unclear.
Text generation
Build the answer in sequence
Read message + system prompt
Predict next token→Append to output
Repeat with the growing answer
Generated JSON
Generating answer…Answer ready~1–3 s · ~$0.000015/request
Decision inference
Evaluate three options
Read message + system prompt
Score the allowed answers
Option probabilities
Billing92%
Technical5%
Review3%
Generating decision…Decision ready~100 ms · ~$0.000010/request
From judgment to customer response
Decision model selects a queue
Clear resultCode routes the ticket
A person or LLM handles the issue and writes a reply.
Uncertain resultA person reviews it
The reviewer clarifies the issue and chooses the right team.
Customer receives a response