In the last decades, important progress in nonlinear elasticity derived from the quantitative rigidity estimate obtained by Friesecke, James and Müller, that states that if the gradient of a Sobolev map $u:\Omega\subseteq\mathbb{R}^n\to\mathbb{R}^n$ is pointwise $L^p$-close to a rotation then it is globally $L^p$-close to a rotation. We extend this result to maps from a compact, connected, oriented Riemannian manifold into itself. Key ingredients in the proof are the weak Riemannian Piola identity derived by Kupferman, Maor and Shachar in 2019, a uniform $C^{1,\alpha}$ approximation through the harmonic map heat flow, and a linearization argument which reduces the estimate to the Riemannian version of Korn's inequality. This is joint work with Georg Dolzmann (Regensburg) and Stefan Müller (Bonn).