Transformers can learn self-verifying reflection without language, and reinforcement learning enhances performance through shallow statistical patterns.