Friday, 4 May 2012

pr.probability - Why is Beta the maximum entropy distribution over Bernoulli's parameter?

Why is Beta(1,1) the maximum entropy distribution over the bias of a coin expressed as a probability given that:



  • If we express the bias as odds (which is over the support $[0, infty)$), then Beta-prime(1,1) is the corresponding distribution to Beta(1,1). Isn't the maximum entropy distribution over the positive reals the exponential distribution (which is not Beta-prime(1,1))?


  • If we express the bias in log odds (which is over the support of the reals), then the logistic distribution (with mean 0 and scale 1) is the corresponding distribution to Beta(1,1).


Beta(1,1) makes sense as maximum entropy because it's flat over its support. The other distributions are not flat. If we had chosen a different parametrization, we should clearly arrive at the corresponding distribution (not something else). How are the other two distributions the maximum entropy distributions over their support? There must be some other requirement that I'm missing. What is it?

No comments:

Post a Comment