We propose, analyze, and validate a method for guiding LLM behavior at inference time by applying steering vectors to query and value representations.