A system prompt is an instruction inserted at the beginning of the context window in a dedicated role, framing the model’s behaviour for the whole conversation. Technically it is not a separate channel – after tokenisation it consists of the same tokens as user input – and its special authority is established only by training: during instruction tuning and RLHF, models are taught a hierarchy of roles in which the system message takes precedence over the user’s. It typically defines persona and tone, the language of replies, output format, available tools, the range of permitted topics and safety boundaries. Stable context is also placed in the system prompt, which allows it to be precomputed into the KV cache for cheaper operation. It is essential to be aware of its limits: because it is part of the same stream of text, it is not a security boundary and can be circumvented by prompt injection. Secrets such as keys or passwords therefore never belong in it, nor may it be the sole mechanism for controlling permissions – those must be enforced at the level of the application and the tools the model calls.
Think of the staff rules you hand a new employee on day one: “You are an employee of our bank, you always speak to customers yourself, you never promise a return, and you do not comment on competitors.” After that he simply takes client queries and holds to that frame. But the difference from a real employee is substantial: he has the rules taped to his monitor, not in a signed contract. A sufficiently inventive client can talk him into forgetting them. Which is why you do not lock the vault with staff rules but with a key – that is, with permissions on the system side.