gr.group theory - Is there any criteria for whether the automorphism group of G is homomorphic to G itself?

This answer is essentially a series of remarks, but ones which I hope will be helpful to you.

(1) There are two ways to interpret the condition that $G$ be isomorphic to its automorphism group: canonically and non-canonically.

a) Say that $G$ is complete if every automorphism of $G$ is inner (i.e., conjugation by some element of $G$) and $G$ has trivial center. In this case, there is a canonical isomorphism $G stackrel{sim}{rightarrow} operatorname{Aut}(G)$.

The linked wikipedia article gives some interesting information about complete groups. As above, by definition having trivial center is a necessary condition; all nonabelian simple groups satisfy this. On the other hand, an interesting sufficient condition is that for any nonabelian simple group $G$, its automorphism group $operatorname{Aut}(G)$ is complete, i.e., we have canonically $operatorname{Aut}(G) = operatorname{Aut}(operatorname{Aut}(G))$.

b) It is possible for a group to have nontrivial center and outer automorphisms and for these two defects to "cancel each other out" and make $G$ noncanonically isomorphic to $operatorname{Aut}(G)$. This happens for instance with the dihedral group of order $8$.

2) It seems extremely unlikely to me that there is a reasonable necessary and sufficient condition for a general finite group to be isomorphic to its automorphism group in either of the two sense above.

But a lot of specific examples are certainly known: see for instance

http://en.wikipedia.org/wiki/List_of_finite_simple_groups

in which the order of the outer automorphism group of each of the finite simple groups is given. So, for instance, exactly $14$ of the $26$ sporadic simple groups have trivial outer automorphism group, hence satisfy $G cong operatorname{Aut}(G)$.

I wouldn't be surprised if the outer automorphism groups of all finite groups of Lie type were known (they are not all known to me, but I'm no expert in these matters).