Learning Parametric Closed-Loop Policies for Markov Potential Games