Prompt Cache Sharing
How the policy a workspace follows decides which clients share the prompt cache, why workspaces never share, how sharing works on two or more nodes, and the timing side channel that sharing opens.
How the policy a workspace follows decides which clients share the prompt cache, why workspaces never share, how sharing works on two or more nodes, and the timing side channel that sharing opens.
On a PaletteAI Inference Launchpad appliance, choose whether clients share the prompt cache or each keep their own. Check why a repeated prompt shows no cache reads.