Set a System Prompt
This guide explains how to set a system prompt on a PaletteAI Inference Launchpad appliance. In the console, a system prompt that you set is an instruction. You set one default instruction for every model, and you can give a model its own instruction that replaces the default. You can also choose whether a request that carries its own system prompt receives the instruction. To learn which requests receive the instruction, refer to System Prompts.
Prerequisites
-
A running PaletteAI Inference Launchpad appliance, with the console reachable.
-
Console access as a person who holds the Platform administrator role.
-
(Per-model instructions only) A model in the appliance catalog. To place a model in the catalog, refer to Upload a Model.
-
(Validation only) A model that is deployed and serving. To deploy a model, refer to Deploy a Model.
Set the Default Instruction
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Prompt Instructions.
-
In the Default instruction field, enter the instruction for every model that has no instruction of its own. For example,
Answer every question in Spanish. -
Select Apply.
A
Prompt settings savedmessage appears.
Set a Per-Model Instruction
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Prompt Instructions.
-
Under Per model instructions, select Add per model instruction.
-
From the Model drop-down menu, select the model.
-
In the Instruction field, enter the instruction for that model.
-
(Optional) To give another model its own instruction, repeat steps 3 - 5.
-
Select Apply.
A
Prompt settings savedmessage appears.
Turn Apply to Every Turn On or Off
Apply to every turn is off by default. To decide whether to turn it on, refer to System Prompts.
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Prompt Instructions.
-
Under the Default instruction field, turn the Apply to every turn toggle on or off.
-
To also add the instruction after the system prompt that a client sends, turn it on.
-
To add the instruction only to a request without a system prompt, turn it off.
-
-
Select Apply.
A
Prompt settings savedmessage appears.
Remove an Instruction
-
From the left main menu, select Settings.
-
On the Settings menu, under Request handling, select Prompt Instructions.
-
Remove the instruction that you no longer need.
-
To remove the default instruction, clear the Default instruction field.
-
To remove the instruction for one model, select the trash icon on that model's row under Per model instructions.
-
-
Select Apply.
A
Prompt settings savedmessage appears.
Validate
Run this check from the console, not from a coding assistant. For the requests that do not receive the instruction, refer to System Prompts.
-
From the left main menu, select Diagnose.
-
Select the Chat tab.
-
From the Model drop-down menu, select a model that the instruction applies to. For a per-model instruction, select that model. For the default instruction, select a model that has no instruction of its own.
-
Send a message that the instruction affects. For example, with the instruction
Answer every question in Spanish., sendWhat is Kubernetes?. -
Confirm that the reply follows the instruction. With the example instruction, the reply is in Spanish.