dg.differential geometry - Must a surface obtained by exponentiating a plane in a tangent space of a Riemannian manifold be geodesically convex?

I don't think that's true. If the dimension of $M$ is bigger than that of $P$, then a necessary condition is that every tangent hyperplane $Psubset T_pM$ develops locally into a totally geodesic submanifold of $M$. This is not true for arbitrary manifolds. Some examples of manifolds in which this is true include those of constant sectional curvature. Symmetric spaces being a special subclass which verifies this conditions. Someone more awake can probably come up with a sufficient criterion.

I can't think of a very good example right now, but I am pretty sure that if take the 3-dimensional Riemannian Schwarzschild solution, start from a $r$-orthogonal plane outside of the apparent horizon, you'd get a counterexample.

If the dimension of $P$ is the same as dimension of $M$, however, then you should be okay as long as you make $U$ small enough.

Edit: Ah, for $P$ 2 dimensional and $M$ 3, by applying the Codazzi equations one sees that a necessary condition for locally developing the hyperplanes to totally geodesic submanifolds is that $Ric(X,Y) = 0$ whenever $g(X,Y) = 0$. This is obviously a very strong condition that is not satisfied by most manifolds.