One GPU with at least 90 GB (NVIDIA H100 94 GB or RTX PRO 6000 96 GB)
Thinking
Default: on, with effort low. Our gateway sets chat_template_kwargs: {"reasoning_effort": "low"};
the model's own default would be xhigh. A reasoning_effort in your request overrides it.
reasoning_effort: low, medium, xhigh. minimal, high and max are rejected with error 400:
the model's own chat template only knows these three levels.
The levels add an instruction to the prompt (low: keep thinking brief, xhigh: think carefully).
In our tests the amount of thinking hardly changed between the levels.
Numeric effort: no.
Turn off: reasoning_effort: "none" or chat_template_kwargs: {"enable_thinking": false}.