Skip to main content

Set a System Prompt

This guide explains how to set a system prompt on a PaletteAI Inference Launchpad appliance. In the console, a system prompt that you set is an instruction. You set one default instruction for every model, and you can give a model its own instruction that replaces the default. You can also choose whether a request that carries its own system prompt receives the instruction. To learn which requests receive the instruction, refer to System Prompts.

Prerequisites​

  • A running PaletteAI Inference Launchpad appliance, with the console reachable.

  • Console access as a person who holds the Platform administrator role.

  • (Per-model instructions only) A model in the appliance catalog. To place a model in the catalog, refer to Upload a Model.

  • (Validation only) A model that is deployed and serving. To deploy a model, refer to Deploy a Model.

Set the Default Instruction​

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Prompt Instructions.

  3. In the Default instruction field, enter the instruction for every model that has no instruction of its own. For example, Answer every question in Spanish.

  4. Select Apply.

    A Prompt settings saved message appears.

Set a Per-Model Instruction​

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Prompt Instructions.

  3. Under Per model instructions, select Add per model instruction.

  4. From the Model drop-down menu, select the model.

  5. In the Instruction field, enter the instruction for that model.

  6. (Optional) To give another model its own instruction, repeat steps 3 - 5.

  7. Select Apply.

    A Prompt settings saved message appears.

Turn Apply to Every Turn On or Off​

Apply to every turn is off by default. To decide whether to turn it on, refer to System Prompts.

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Prompt Instructions.

  3. Under the Default instruction field, turn the Apply to every turn toggle on or off.

    • To also add the instruction after the system prompt that a client sends, turn it on.

    • To add the instruction only to a request without a system prompt, turn it off.

  4. Select Apply.

    A Prompt settings saved message appears.

Remove an Instruction​

  1. From the left main menu, select Settings.

  2. On the Settings menu, under Request handling, select Prompt Instructions.

  3. Remove the instruction that you no longer need.

    • To remove the default instruction, clear the Default instruction field.

    • To remove the instruction for one model, select the trash icon on that model's row under Per model instructions.

  4. Select Apply.

    A Prompt settings saved message appears.

Validate​

Run this check from the console, not from a coding assistant. For the requests that do not receive the instruction, refer to System Prompts.

  1. From the left main menu, select Diagnose.

  2. Select the Chat tab.

  3. From the Model drop-down menu, select a model that the instruction applies to. For a per-model instruction, select that model. For the default instruction, select a model that has no instruction of its own.

  4. Send a message that the instruction affects. For example, with the instruction Answer every question in Spanish., send What is Kubernetes?.

  5. Confirm that the reply follows the instruction. With the example instruction, the reply is in Spanish.

Next Steps​