Routing Behavior
An explanation of how PaletteAI Inference Launchpad decides which model answers each request: the Tier map, the semantic router, where routing rules live, and how the semantic router shares the GPUs.
An explanation of how PaletteAI Inference Launchpad decides which model answers each request: the Tier map, the semantic router, where routing rules live, and how the semantic router shares the GPUs.
Step-by-step guidance for platform administrators on how to set the Thinking directive for a tier on the Tier Map of a PaletteAI Inference Launchpad appliance.
How the Thinking directive on the Tier Map controls reasoning depth by effort level, why the off option is disabled for some providers, how each engine interprets the level, and the shared output allowance that reasoning and answer tokens draw from.
Reference of the Thinking directive modes on the Tier map, the effort levels, per-engine interpretation, the console states that override the setting, and the Thinking values of semantic routing rules.