Why does Banner finally allow the Hulk to come out?

Banner's basic problem isn't anger. It's trust.

He not only learns how to trust himself as the Hulk, but he also learns to trust the Avengers - that they will protect the people from the Hulk if necessary, They will be his shield when needed, his direction when needed, and, in the latest movie, his path back to being Banner.

He's still very conflicted, but he was able to trust the people around him and himself more due to recent experiences, and thus felt comfortable in his role as the Hulk under extenuating circumstances.

In The Age of Ultron he finds, though, that he can still be controlled or misdirected, by the scarlet witch in this case, and hurt both the people he trusts, and those around him. As much as he trusts the Avengers, he loses trust in himself.

There's also a hint of mistrust in the Avengers as well. It's easy for him to guess whether the feelings he has for Black Widow are real, but are the actions and feelings she's expressing real, or simply a method to keep him under control, to direct him as a tool for others to use?