We propose a novel inference-time personalized alignment method that elicits the user's preferences with a few preference queries.