We propose a efficient robust policy evaluation method for non-rectangular robust MDPs with uncertainty sets bounded by $L_p$ norms.