Set Up Mid-Task Model Switching
This guide explains how a platform administrator sets up Mid-Task Model Switching on a PaletteAI Inference Launchpad appliance. You turn the feature on in Settings. Then you choose the two models for a client's alias on the client's Routing tab. To learn how a task moves between the two models, refer to Mid-Task Model Switching.
Prerequisites
-
A running PaletteAI Inference Launchpad appliance, with the console reachable.
-
(Settings steps only) Console access as a person who holds the Platform administrator role.
-
(Client steps only) Console access with permission to update routing rules.
-
(Client steps only) An existing client. To create one, refer to Create a Client.
-
(Client steps only) An efficient model for most turns and a stronger capable model for the hard turns. To deploy a model, refer to Deploy a Model.
-
(External capable model only) External inferencing enabled for the client. To enable it, refer to Manage a Client's Model Access.
-
(Judge model only) The semantic router running on the appliance. To turn it on, refer to Configure Semantic Routing.
Turn Mid-Task Model Switching On or Off
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Mid-Task Model Switching.
-
Turn the Use Mid-Task Model Switching on Inference Launchpad toggle on or off. While the toggle is on, the Switching settings appear.
-
Select Apply.
Change the Switching Settings
Use these steps while Mid-Task Model Switching is on. For what each setting does and its default, refer to The Switching Settings.
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Mid-Task Model Switching.
-
Under Switching settings, change any of these values.
-
In Confidence threshold (%), enter a percentage.
-
In Turns held on the capable model, enter a whole number.
-
Turn the Judge model toggle on or off.
To return all three settings to their defaults, select Reset to defaults instead.
-
-
Select Apply.
-
In the Restart Mid-Task Model Switching? dialog, select Apply and restart.
An
Applied the Mid-Task Model Switching settings and restarted it.message appears. The Switching settings show the saved values.
Choose the Models for a Client's Alias
Use these steps while Mid-Task Model Switching is on. To turn it on, refer to Turn Mid-Task Model Switching On or Off.
-
In the workspace drop-down menu in the console header, select the client's workspace.
-
From the left main menu, select Access Management, and then select the Clients tab.
-
Select the client to open its detail panel, and then select the Routing tab.
-
In the Tier map, find the alias that the client's coding assistant sends. Set its Model to Use Mid-Task Model Switching for routing.
For the aliases that each coding assistant sends, refer to its configuration reference, such as Model Aliases for Claude Code.
The Mid-Task Model Switching card appears.
-
From the Efficient model drop-down menu on the card, select the model for most turns.
-
From the Capable model drop-down menu, select the model for the hard turns.
-
Map every other alias in the Tier map to a model. Save stays unavailable until every alias is mapped.
-
On the Tier map, select Save, and then select Save in the Save alias routing dialog.
Validate
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Mid-Task Model Switching.
-
Confirm that the Use Mid-Task Model Switching on Inference Launchpad toggle shows the state that you applied.
-
If the toggle is on, confirm that the Switching settings show the values that you applied.
-
In the workspace drop-down menu in the console header, select the client's workspace.
-
From the left main menu, select Access Management, and then select the Clients tab.
-
Select the client to open its detail panel, and then select the Routing tab.
-
In the Tier map, confirm that the alias reads Use Mid-Task Model Switching for routing.
-
On the Mid-Task Model Switching card, confirm that Efficient model and Capable model show the models that you chose.
Next Steps
-
Use Claude Code to connect a coding assistant to the appliance.
-
Use Cursor, Use OpenAI Codex, or Use OpenCode for the other supported tools.