ct.category theory - Standard name of "atomic morphisms"?

As mentioned in the comments, I would probably call such a morphism "irreducible" or "prime." A "less evil," and perhaps more useful, version would be to ask that if $f = g circ h$, then either $g$ or $h$ is an isomorphism. In this form, if you regard the multiplicative monoid of a ring as a category with one object, the (noninvertible) irreducible morphisms are precisely the irreducible elements of the ring.

I agree that in "concrete categories" such morphisms are unlikely to be very common or useful, but one other situation in which they arise is free categories on directed graphs. In such a category, the nonidentity irreducible morphisms are precisely the generators (the images of the edges of the directed graph you started from).