Introductory treatments of inferential statistics focus on normal populations. In that case, the two parameters characterizing the distribution, μ and σ2, coincide with expected value, the first-order moment, and variance, the second-order central moment. Hence, students might believe that parameter estimation is just about calculating sample means and variances. It is easy to see that this is not the case.
Example 9.34 Consider a uniform random variable X ∼ u[a, b]. We know from section 7.6.1 that
Clearly, sample mean 
and sample variance S2 do not provide us with direct estimates of parameters a and b. However, we might consider the following way of transforming the statistics to estimates of parameters. If we substitute μ and σ2 with their estimates, we get
Note that in taking the square root of variance, we should only consider the positive root to get a positive standard deviation. Solving this system yields
This example suggests a general strategy to estimate parameters:
- Estimate moments by a random sample.
- Set up a system of equations relating parameters and moments, and solve it.
Indeed, this is the starting point of a general parameter estimation approach called method of moments. However, a more careful look at the example should raise an issue. Consider the order statistics of a random sample of n observations of a uniform distribution:
If we take the above approach, all of these observations play a role in estimating a and b. But this is a bit counterintuitive; to characterize a uniform distribution we need a lower and an upper bound on its realizations. So, it seems that only U(1) and U(n), i.e., the smallest and the largest observations should play a role. We might suspect that there are alternative strategies for finding point estimators. In this section we outline two approaches: the method of moments and the method of maximum likelihood. Since there are alternative ways to build estimators, it is just natural to wonder how we can compare them. Therefore, we should first list the desirable properties that make a good estimator.
Leave a Reply