45-733 PROBABILITY AND STATISTICS I Notes #7B
February 2000
_{Ù} If E(q) = q, then q is an unbiased estimator.
If the variance of one estimator is smaller than a second estimator, then it is relatively more efficient. Example: Note that E(X_{1} + X_{2})/2 = m. However, if n > 2, the variance of this estimator is larger _ than the variance of X_{n} so it is less efficient.
_{Ù} _{ Ù} If E(q) ® q as n ® +¥, then q is a consistent estimator of q.
_{Ù} In particular, E(s^{2}) = [(n-1)/n]s^{2} To see this, recall that: VAR(X) = E(X^{2}) - [E(X)]^{2} so that E(X^{2}) = s^{2} + m^{2}; and _ _ _ _ VAR(X_{n}) = E[(X_{n})^{2}] - [E(X_{n})]^{2} so that E[(X_{n})^{2}] = s^{2}/n + m^{2} Therefore: _{Ù} _ E(s^{2}) = E[å_{i=1,n} (x_{i} - X_{n})^{2}]/n = _ _ (1/n)E{[å_{i=1,n} (x_{i})^{2}] - 2(X_{n})[å_{i=1,n} (x_{i})] + n(X_{n})^{2}} = _ (1/n)E[å_{i=1,n} (x_{i})^{2}] - E[(X_{n})^{2}] = (1/n)[n(s^{2} + m^{2})] - (s^{2}/n + m^{2}) = [(n-1)/n]s^{2}
_{Ù} Although s^{2} is the maximum likelihood estimator for s^{2} it is biased. _ This bias stems from the fact that X_{n} is used in the formula. _ Replacing X_{n} with m removes the bias. This is due to the fact that _ X_{n} in a finite random sample is very unlikely to be exactly on m. Hence, the sum of the squares of the sample will be slightly underestimated. To obtain an unbiased estimator, denoted as s^{2}, the sum of the squares of the sample is divided by n-1 and this serves to slighly inflate the value and produces an unbiased estimator; namely: _ s^{2} = [å_{i=1,n} (x_{i} - X_{n})^{2}]/(n-1) and it is easy to show that: E(s^{2}) = s^{2}