We propose a policy gradient algorithm for fine-tuning discrete diffusion models over non-differentiable rewards.