Skip to main content

Set Up Mid-Task Model Switching

This guide explains how a platform administrator sets up Mid-Task Model Switching on a PaletteAI Inference Launchpad appliance. You turn the feature on in Settings. Then you choose the two models for a client's alias on the client's Routing tab. To learn how a task moves between the two models, refer to Mid-Task Model Switching.

Prerequisites​

  • A running PaletteAI Inference Launchpad appliance, with the console reachable.

  • (Settings steps only) Console access as a person who holds the Platform administrator role.

  • (Client steps only) Console access with permission to update routing rules.

  • (Client steps only) An existing client. To create one, refer to Create a Client.

  • (Client steps only) An efficient model for most turns and a stronger capable model for the hard turns. To deploy a model, refer to Deploy a Model.

  • (External capable model only) External inferencing enabled for the client. To enable it, refer to Manage a Client's Model Access.

  • (Judge model only) The semantic router running on the appliance. To turn it on, refer to Configure Semantic Routing.

Turn Mid-Task Model Switching On or Off​

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Mid-Task Model Switching.

  3. Turn the Use Mid-Task Model Switching on Inference Launchpad toggle on or off. While the toggle is on, the Switching settings appear.

  4. Select Apply.

Change the Switching Settings​

Use these steps while Mid-Task Model Switching is on. For what each setting does and its default, refer to The Switching Settings.

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Mid-Task Model Switching.

  3. Under Switching settings, change any of these values.

    • In Confidence threshold (%), enter a percentage.

    • In Turns held on the capable model, enter a whole number.

    • Turn the Judge model toggle on or off.

    To return all three settings to their defaults, select Reset to defaults instead.

  4. Select Apply.

  5. In the Restart Mid-Task Model Switching? dialog, select Apply and restart.

    An Applied the Mid-Task Model Switching settings and restarted it. message appears. The Switching settings show the saved values.

Choose the Models for a Client's Alias​

Use these steps while Mid-Task Model Switching is on. To turn it on, refer to Turn Mid-Task Model Switching On or Off.

  1. In the workspace drop-down menu in the console header, select the client's workspace.

  2. From the left main menu, select Access Management, and then select the Clients tab.

  3. Select the client to open its detail panel, and then select the Routing tab.

  4. In the Tier map, find the alias that the client's coding assistant sends. Set its Model to Use Mid-Task Model Switching for routing.

    For the aliases that each coding assistant sends, refer to its configuration reference, such as Model Aliases for Claude Code.

    The Mid-Task Model Switching card appears.

  5. From the Efficient model drop-down menu on the card, select the model for most turns.

  6. From the Capable model drop-down menu, select the model for the hard turns.

  7. Map every other alias in the Tier map to a model. Save stays unavailable until every alias is mapped.

  8. On the Tier map, select Save, and then select Save in the Save alias routing dialog.

Validate​

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Mid-Task Model Switching.

  3. Confirm that the Use Mid-Task Model Switching on Inference Launchpad toggle shows the state that you applied.

  4. If the toggle is on, confirm that the Switching settings show the values that you applied.

  5. In the workspace drop-down menu in the console header, select the client's workspace.

  6. From the left main menu, select Access Management, and then select the Clients tab.

  7. Select the client to open its detail panel, and then select the Routing tab.

  8. In the Tier map, confirm that the alias reads Use Mid-Task Model Switching for routing.

  9. On the Mid-Task Model Switching card, confirm that Efficient model and Capable model show the models that you chose.

Next Steps​