Sunday, 27 April 2008

st.statistics - Distribution of the sum of the $m$ smallest values in a sample of size $n$

This is following on from Douglas Zare's answer. While not an answer in its own right,it was getting too long for a comment. Briefly, we can hit the question with brute force and look for a generating function for the desired expected values.



So, put
$$f_j(y) = sum_{k=0}^{j-1} {n choose k} y^k (1-y)^{n-k}$$
so that using the notation of Douglas' answer, $g_j(x)=f_j(F(x))$, and the expectation of the $j$th order statistic is
$$ E_j := int_0^infty f_j(F(x)) dx $$



Put $G(y,z) = sum_{j=1}^n f_j(y) z^j$ where $z$ is a formal variable. By linearity we have
$$ sum_{j=1}^n E_j z^j = int_0^infty G(F(x),z) dx $$



We can try to write $G$ as a rational function in $y$ and $z$.
Expanding out and interchanging the order of summation gives
$$ eqalign{
G(y,z) & = sum_{j=1}^nsum_{k=0}^{j-1} {nchoose k} y^k (1-y)^{n-k} z^j \\
& = sum_{k=0}^{n-1} sum_{j=k+1}^n z^j {nchoose k} y^k (1-y)^{n-k} \\
& = sum_{k=0}^{n-1} left( sum_{j=1}^{n-k} z^j right) cdot z^k{nchoose k} y^k (1-y)^{n-k} \\
}
$$



Now
$$ eqalign{
sum_{j=1}^{M-1}jz^j
= z frac{d}{dz}sum_{j=0}^{M-1} z^j
& = z frac{d}{dz}left[frac{1-z^M}{1-z}right] \\
& = frac{z-z^{M+1}}{(1-z)^2} - frac{Mz^M}{1-z}
& = frac{z-z^M}{(1-z)^2} - frac{(M-1)z^M}{1-z}
} $$
so substituting this back in we get
$$ eqalign{
G(y,z)
& = sum_{k=0}^{n-1} left[ frac{z-z^{n-k+1}}{(1-z)^2} - frac{(n-k)z^{n-k+1}}{1-z} right] cdot z^k{nchoose k} y^k (1-y)^{n-k} \\
& = sum_{k=0}^{n-1} frac{z}{(1-z)^2}{nchoose k} (yz)^k (1-y)^{n-k} \\
& - sum_{k=0}^{n-1} frac{z^{n+1}}{(1-z)^2} {nchoose k} y^k (1-y)^{n-k} \\
& - sum_{k=0}^{n-1} frac{nz^{n+1}}{1-z} { {n-1} choose k } y^k(1-y)^{n-k} \\
& = frac{z}{(1-z)^2} left[ (1-y+yz)^n - (yz)^n right]
- frac{z^{n+1}}{(1-z)^2} (1-y^n)
- frac{nz^{n+1}}{1-z}
} $$



I guess that in theory one could plug this back in to obtain a "formula" for the generating function $sum_{j=1}^n E_j z^j$, but I can't see how that formula might then simplify to something calculable, unless $F$ has a rather special form.

No comments:

Post a Comment