Fundamentals of Probability and Statistics for Engineers

Theorem 9. 2: the Crame ́r– R a o ineq ua lit y.LetX 1 ,X 2 ,...,Xn denote a sample

of size n from a population X with pdf f(x; ), where is the unknown param-

eter, and let h(X 1 ,X 2 ,...,Xn) be an unbiased estimator for. Then, the

variance of satisfies the inequality

if the indicated expectation and differentiation exist. An analogous result with

p(X; ) rep lacing f(X; ) is obtained when X is discrete.

ProofofTheorem9.2:the joint probability density function (j pdf) of X 1 ,X 2 ,...,

and Xn is, because of their mutual independence, The

mean of statistic is

and, since is unbiased, it gives

Another relation we need is the identity:

Upon differentiating both sides of each of Equations (9.27) and (9.28) with

respect to , we have

Parameter Estimation 267

 

^ˆ 

^

varf^g nE

qlnf…X;†

q

) 2  1

; … 9 : 26 †

 

fx 1 ;)fx 2 ;)fxn;).

^,^ hX 1 ,X 2 ,...,Xn),

Ef^gˆEfh…X 1 ;X 2 ;...;Xn†g;

^

ˆ

Z 1

1



Z 1

1

h…x 1 ;...;xn†f…x 1 ;†f…xn;†dx 1 dxn: … 9 : 27 †

1 ˆ

Z 1

1

f…xi;†dxi; iˆ 1 ; 2 ;...;n: … 9 : 28 †



1 ˆ

Z 1

1



Z 1

1

h…x 1 ;...;xn†

Xn

jˆ 1

1

f…xj;†

qf…xj;†

q

"#

f…x 1 ;†f…xn;†dx 1 dxn

ˆ

Z 1

1



Z 1

1

h…x 1 ;...;xn†

Xn

jˆ 1

qlnf…xj;†

q

"#

f…x 1 ;†f…xn;†dx 1 dxn;

… 9 : 30 †

0 ˆ

Z 1

1

qf…xi;†

q

dxi

ˆ

Z 1

1

qlnf…xi;†

q

f…xi;†dxi; iˆ 1 ; 2 ;...;n:

… 9 : 30 †

)

ˆ