74 5 Positive Definiteness
Remark 5.1.6 Let f : M™ M- R andx G Rn be the local minimum, Vf(x) =
0 and V^2 f(x) is positive definite. We are able to explore the neighborhood
of x by means of x* + Ax, where \Ax\ is sufficiently small (such that the
second order Taylor's approximation is pretty good) and positive. Then,
f(x* + Ax) S f(x*) + AxTVf(x*) + ^AxTV^2 f(x*)Ax.The second term is zero since x is a critical point and the third term is
positive since the Hessian evaluated at x is positive definite. Thus, the left
hand side is always strictly greater than the right hand side, indicating the
local minimality of x*.
5.2 Detecting Positive-Definiteness
Theorem 5.2.1 A real symmetric matrix A is positive definite if and only if
one of the following holds:
i. xTAx > 0, Vz ^ 6;
ii. All the eigen values of A satisfy A; > 0;
Hi. All the submatrices Ak have positive determinants;
iv. All the pivots (without row exchanges) satisfy di > 0;
v. 3 a nonsingular matrix W B A = WTW (called Cholesky Decomposition);
Proof. A is positive definite.
- (i) <& (ii)
(i) => (ii): Let x, be the unit eigen vector corresponding to eigen value
Xi.
J\Xi == AJXJ \^ XA J\-X{ = Xi AjXj — Aj.
Then, Ai > 0 since A is positive definite.
(i) <= (ii): Since symmetric matrices have a full set of orthonormal eigen
vectors
(Exercise!).x — 2_,aixi ^* Ax = 2~\onAxi =>• xTAx = (2~]aixf)(2~]cti\iXi).Because of orthonormality xTAx = ^ ctf Aj > 0.- (i) <£• (Hi) <=> (iv) <& (v)
(i) => (Hi): det A = \\ • X 2 • • • A„, since (i) o (ii).
Claim: If A is positive definite, so is every Ak-
Xk
Proof: If x
0
, then