25. Orthogonal bases

We start with a few concepts from geometry, which are often important in mathematics, and limit for the time being consideration to real numbers. Two real vectors

are said to be orthogonal to each other when their inner product vanishes:

a'b = b'a = a₁b₁ + ··· a_nb_n = 0. *

The Norm N(a) of the real vector A is the inner product of a with itself:

N(a) = a'a = a²₁ + ··· a²_n.

Always, N(a) 0; it only vanishes when a = 0, since the sum of the squares of real numbers can only vanish when every term in the sum vanishes. The absolute value |a| of a is defined as the positive square root of N(a), that is,

Obviously, |a| 0 and only |a| = 0 when a = 0.

* The term orthogonal comes from Analytic Geometry, where the equation a'b = 0 is the condition for vectors a and b to be orthogonal to each other, assuming that it is based on a orthogonal coordinate system.

** In Analytic Geometry, in an orthogonal coordinate system, the absolute value |a| is the expression for the length of the vector.

There may arise at times the need to extend these concepts to vectors of complex numbers. If it were required to define in the complex regime a norm as the sum of the squares of the elements of a vector, one would realize soon enough that this definition does not serve its purpose. In fact, it is obvious that the sum of the squares of complex numbers can vanish without all numbers equalling zero and, moreover , it can be complex. It could also happen that the absolute value |a| = (N(a))^1/2 of a complex vector a vanishes without a = 0, and when N(a) is negative or complex, one would encounter more problems with the absolute value |a| . Hence it turns out that one can in another way reach a useful generalization of the above definition for complex vectors.

We muct now discuss calculations with complex matrices. If A is an arbitrary matrix, you denote by the conjugate complex matrix, that is, the matrix the elements of which are conjugate to the corresponding elements of A. A real matrix is characterized by the equation = A; the conjugate complex matrix of A is again A, that is, = A. For the sum and product you have the rules:

They follow directly from the corresponding rules for complex numbers.

The norm N(a) of an arbitrary vector

is defined by

By this definition, always N(a) 0, and N(a) = 0 only when a = 0, because the absolute values |a_i| on the right hand side are 0 and |a_i| = 0 only for all a_i=0. In the case that a is real, that is, = a, you obtain again the former expression for the norm of a real vector.

The absolute value |a| of a is defined, as above, by the positive square root of the norm:

Obviously, |a | is always a non-negative, real number, and it follows from |a|=0 that a = 0. A normalized vector is a vector with the absolute value 1. Every vector a 0 can be normalized by multiplication by |a|^-1, because

Finally, you say that two arbitrary vectors a and b are mutually orthogonal when

This statement is justified, because orthogonality is a symmetric relationship. In fact, from = 0 follows by transition to the conjugate complex numbers

Moreover, since as a (1,1)-matrix equals its transposed matrix, one finds that which establishes the symmetry.

A system of r vectors c₁, ··· c_r of an n-dimensional vector space V is a normalized orthogonal system, if c₁, ··· c_r are normalized and orthogonal in pairs. This condition is expressed by

We will now prove: The vectors of every orthogonal system are linearly independent.

In fact, the relation

l₁c₁ + ··· + l_ic_i + ··· + l_rc_r = 0 (25.3)

yields, on multiplication by ,

Then, by (25.2),

l_i ·1 = 0,

that is, l_i = 0. Since this holds for i = 1, ···, r, all coefficients l_i· in (25.3) vanish, whence the c₁, ··· c_r are linearly independent.

There cannot exist in the n-dimensional vector space V more than n linearly independent vectors, that is, a normalized orthogonal system comprises at the most n vectors. Such a system with exactly n vectors is called a normalized orthogonal system.

The rows of the n-row unit matrix form for every natural number n an example of a completely normalized orthogonal system. By selection of any r columns of the unit matrix, you obtain for every natural number r n an example of a normalized orthogonal system of r vectors.

A generalization of the evident fact that there exist in plane systems and in three-dimensional space two and three pairwise perpendicular, unit length vectors, respectively, is given by

Theorem 13

Every vector space, in which a norm is defined, has bases which form a normalized orthogonal system.

Proof: Let T be a vector space in V of dimension r. Then T has a base of r linearly independent vectors a₁, ··· a_r. We will now show that one can find in T a system of r vectors c₁, ···, c_r which form a normalized, orthogonal system. As a result of the linear independence of the c₁, ··· c_r, discussed above, they form a base for T.

The method which we will employ for the construction of the c₁, ··· c_r is due to E. Schmidt. *

* E. Schmidt's original method involved vectors with countable, infinitely many elements. It has a fundamental role in the Theory of Integral Equations.

Due to their linear independence, none of the vectors a₁, ··· a_r is the null-vector, in particular, a₁ 0. Hence we can set

c₁ = a₁/|a₁|,

so that c₁ is normalized.

We let the vector c₂ have the form

c₂ = a₁c₁ + a₂a₂

and seek a₁ and a₂ so that c₂ is normalized and orthogonal to c₁. The condition for c₁ to be orthogonal to c₂ is

Hence a₁ = - a₂a₂ and

c₂ = - a₂[a₂ - (a₂)c₁].

The vector in the square brackets is not a null-vector, because a₂ does not depend linearly on a₁ and therefore also not on c₁. Hence the number a₂ can be determined and c₂ normalized. In fact, with a₂ = |a₂ - (a₂)c₁|^-1, you find

c₂ = (a₂ - (a₂)c₁/|a₂ - (a₂)c₁|),

a normalized vector which is orthogonal to c₁.

For c₂, you start with

c₃ = b₁c₁ + b₂c₂ + b₃a₃ =

and seek b₁, b₂, b₃, so that c₃ becomes normalized and is orthogonal with respect to c₁ and c₂. Since c₁ and c₂ are already orthogonal, you find

c₃ = b₁ + b₃a₃ = 0,
c₃ = b₂ + b₃a₃ = 0,

whence, to start with,

c₃ = b₃[a₃ - (a₃)c₁ - (a₃)c₃ ].

The vector in the square brackets is again not a null-vector, because a₃ does not depend linearly on a₁, a₂ - and hence also not on c₁ ,c₂. You can now find b₃ so that c₃ is normalized:

,

You will now readily see how to continue this process. If for i r the vectors c₃ , ··· , c_i-1 have already been determined in the above manner - normalized and orthogonalized - you write

c_i = r₁c₁+ c_i r_i-1c_i-1+ r_ic_i

and find r₁, ··· , r_i normalizing c_i and making it orthogonal to c₁ , ··· , c_i-1. The orthogonal conditions yield

= r_k + r_i a_i = 0 (k = 1, ··· , i - 1),

whence

The expression in square brackets is not a null-vector, because is does not depend linearly on a₁ , ··· , a_i-1 and hence also not on c₁ , ··· , c_i-1, which, as we know, are linear sums of a₁ , ··· , a_i-1. Consequently, you can normalize c_i by a suitable choice of r_i:

After r steps, you arrive at a normalized, orthogonal system of vectors c₁,·· ,c_r, whence Theorem 13 has been proved. Note yet that in the case of real vectors you simply omit the bars on top of the symbols, which indicate transition to conjugate complex vectors. Given a system of real vectors a₁ , ··· , a_r, you obtain thus a real, normalized orthogonal system c₁ , ··· , c_r.

It is readily seen that the constructed vectors c₁ , ··· , c_r are by no means the only normalized orthogonal base of T. The vectors of a complete, normalized. orthogonal system c₁ , ··· , c_n should be viewed as the columns of an (n,n)-matrix C. Equations (25.2) for i, k = 1, ··· , n can then be combined in the matrix equation

C = E. (25.4)

Such a matrix C is said to be unitary. For a unitary matrix C, you have also
= C^-1. Since every matrix can interchanged with its inverse, you have in addition to (25.40) the equation

C = E. (25.5)

This equation says that also the rows of C form a complete, normalized, orthogonal system.

For a real matrix C, you can write simply instead of (25.5) CC' = E and obtain (18.6), whence a real unitary matrix is orthogonal.

It is readily seen that the product of two unitary matrices is again unitary. If both A and B are unitary, that is,

then you have for the product C = AB

Hence it is readily confirmed that the set of all unitary (n,n)-matrices forms a group.

Exercises

18. The adjoint minor determinants of of the (n,n)-determinant A are denoted by A_ik. Form the (n,n)-matrix B = (A_ik) and prove that |B| = |A^n-1|. (Beware of |A| = 0!).

19. Find a normalized orthogonal base for the vector space determined by the vectors

.

20. Let there be given in a vector space V with base e₁, ··· e_n the r < n linearly independent vectors a₁, ··· a_n. Show that from e₁, ··· e_n you can select n - r suitable vectors so that they form together with a₁, ··· a_n a base for V.

21. The determinant of every unitary matrix has the absolute value 1.

Answers

last next