Skip to main content

2 docs tagged with "policy"

View all tags

Prompt Cache Sharing

How the policy a workspace follows decides which clients share the prompt cache, why workspaces never share, how sharing works on two or more nodes, and the timing side channel that sharing opens.

Set Prompt Cache Sharing for a Policy

On a PaletteAI Inference Launchpad appliance, choose whether clients share the prompt cache or each keep their own. Check why a repeated prompt shows no cache reads.