Preliminaries

1.1 INTRODUCTION

In this chapter we introduce some notation and discuss some results which will be found very useful for the development of the theory of both Kronecker products and matrix differentiation. Our aim will be to make the notation as simple as possible although inevitably it will be complicated. Some simplification may be obtained at the expense of generality. For example, we may show that a result holds for a square matrix of order n X n and state that it holds in the more general case when A is of order m X n. We will leave it to the interested reader to modify the proof for the more general case.

Further, we will often write

[MATHEMATICAL EXPRESSION OMITTED]

when the summation limits are obvious from the context.

Many other simplifications will be used as the opportunities arise. Unless of particular importance, we shall not state the order of the matrices considered. It will be assumed that, for example, when taking the product AB or ABC the matrices are conformable.

1.2 UNIT VECTORS AND ELEMENTARY MATRICES

[MATHEMATICAL EXPRESSION OMITTED] (1.1)

The unit vectors of order n are defined as

The one vector of order n is defined as

[MATHEMATICAL EXPRESSION OMITTED] (1.2)

From (1.1) and (1.2), obtain the relation

e = [summation]Zei (1.3)

The elementary matrixEij is defined as the matrix (of order m X n) which has a unity in the (i,j)th position and all other elements are zero.

For example,

[MATHEMATICAL EXPRESSION OMITTED] (1.4)

The relation between ei, ej and Eij is as follows

Eij = eie'j (1.5)

where e'j denotes the transposed vector (that is, the row vector) of ej.

Example 1.1

Using the unit vectors of order 3

(i) form E11, E21, and E23

(ii) write the unit matrix of order 3 X 3 as a sum of the elementary matrices.

Solution

(i) [MATHEMATICAL EXPRESSION OMITTED]

(ii) [MATHEMATICAL EXPRESSION OMITTED]

The Kronecker delta dij is defined as

[MATHEMATICAL EXPRESSION OMITTED]

it can be expressed as

dij = e'iej = e'jei (1.6)

We can now determine some relations between unit vectors and elementary matrices.

[MATHEMATICAL EXPRESSION OMITTED] (1.7)

and

[MATHEMATICAL EXPRESSION OMITTED] (1.8)

Also

[MATHEMATICAL EXPRESSION OMITTED] (1.9)

In particular if r = j, we have

EijEjs = djjEis = Eis

and more generally

EijEjsEsm = EisEsm = Eim. (1.10)

Notice from (1.9) that

EijErs = 0 if j ? r.

1.3 DECOMPOSITIONS OF A MATRIX

We consider a matrix A of order m X n having the following form

[MATHEMATICAL EXPRESSION OMITTED] (1.11)

We denote the n columns of A by A.1, A.2, ... A.n. So that

[MATHEMATICAL EXPRESSION OMITTED] (1.12)

and the m rows of A by A1., A.2, ... Am. so that

[MATHEMATICAL EXPRESSION OMITTED] (1.13)

Both the A.j and the Ai. are column vectors. In this notation we can write A as the (partitioned) matrix

A = [A.1 A.2 ... A.n] (1.14)

or as

A = [A.1 A.2 ... Am.]' (1.15)

(where the prime means 'the transpose of').

For example, let

[MATHEMATICAL EXPRESSION OMITTED]

so that

[MATHEMATICAL EXPRESSION OMITTED]

then

[MATHEMATICAL EXPRESSION OMITTED]

The elements, the columns and the rows of A can be expressed in terms of the unit vectors as follows:

The jth column A.j = Aej (1.16)

The ith row Ai,' = e'iA. (1.17)

So that

Ai. = (e'iA)' = A'ei. (1.18)

The (i,j)th element of A can now be written as

aij = e'iAej = e'jA'ei. (1.19)

We can express A as the sum

A = [summation][summation]aijEij (1.20)

(where the Eij are of course of the same order as A) so that

[MATHEMATICAL EXPRESSION OMITTED] (1.21)

From (1.16) and (1.21)

[MATHEMATICAL EXPRESSION OMITTED] (1.22)

Similarly

Ai. = [summation over j]aijej (1.23)

so that 7

A'i. = [summation over j]aije'j (1.23)

It follows from (1.21), (1.22), and (1.24) that

A = [summation]A.je'j (1.25)

and

A = [summation]ejAi.'. (1.26)

Example 1.2

Write the matrix

[MATHEMATICAL EXPRESSION OMITTED]

as a sum of: (i) column vectors of A; (ii) row vectors of A.

Solutions

(i) Using (1.25)

[MATHEMATICAL EXPRESSION OMITTED]

Using (1.26)

[MATHEMATICAL EXPRESSION OMITTED]

There exist interesting relations involving the elementary matrices operating on the matrix A.

For example

[MATHEMATICAL EXPRESSION OMITTED] (1.27)

Similarly

AEij = Aeie'j = A.ie'j (by 1.16) (1.28)

so that

AEjj A.je'j (1.29)

AEijB = Aeie'jB = A.iBj'. (by 1.28 and 1.27) (1.30)

[MATHEMATICAL EXPRESSION OMITTED] (1.31)

In particular

EjjAErr = ajrEjr (1.32)

Example 1.3

Use elementary matrices and/or unit vectors to find an expression for

(i) The product AB of the matrices A = [aij] and B = [bij].

(ii) The kth column of the product AB

(iii) The kth column of the product XYZ of the matrices X= [xij], Y = [yij] and Z = [zij]

Solutions

(i) By (1.25) and (1.29)

A = [summation]A.je'j = [summation]AEjj,

hence

[MATHEMATICAL EXPRESSION OMITTED]

(ii) (a)

(AB).k = (AB)ek = A(Bek) = AB.k by (1.16)

(b) From (i) above we can write

[MATHEMATICAL EXPRESSION OMITTED]

(iii) [MATHEMATICAL EXPRESSION OMITTED]

1.4 THE TRACE FUNCTION

The trace (or the spur) of a square matrix A of order (n X n) is the sum of the diagonal terms

[MATHEMATICAL EXPRESSION OMITTED]

We write

tr A = [summation]aii. (1.33)

From (1.19) we have

aii = e'iAei,

so that

tr A = [summation]e'iAei. (1.34)

From (1.16) and (1.34) we find

tr A = [summation]e'iA.i (1.35)

and from (1.17) and (1.34)

tr A = [summation]Ai'.e'i. (1.36)

We can obtain similar expression for the trace of a product AB of matrices.

For example

[MATHEMATICAL EXPRESSION OMITTED] (1.37)

[MATHEMATICAL EXPRESSION OMITTED] (1.38)

Similarly

[MATHEMATICAL EXPRESSION OMITTED] (1.39)

From (1.38) and (1.39) we find...