Lecture 13 Math221 

Column Space (Range Space) 

It is very important that you read the complete lecture before working the exercises.  The easy way to work the problem will discussed after some modivation.   

We would like to now  study the spaces related to the solutions of systems Ax = b.  If you have two different b's  where there are solutions i.e.   Ax1 = b1 and Ax2 = b2.  Note here that the A is the same while the other variables are different.    Then Typesetting:-mrow(Typesetting:-mi(Ax1 = Typesetting:-mrow(Typesetting:-mi(b1 or ATypesetting:-mrow(Typesetting:-mi(x1=Typesetting:-mrow(Typesetting:-mi(b1 and 

Ax1+Ax2 = b1 + b2 or A(x1 +x2)=b1 +b2 .   We see that the b's form a subspace of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( where A is an mxn matrix.  We will look at how this space is related to the columns of the matrix.  

 

 

> with(linalg):
 

> with(LinearAlgebra):
 

> A:=<<2,2,-2>|<2,-2,2>|<3,6,9>>;
 

Matrix(%id = 2841144)
 

> X:=<x,y,a>;
 

Vector[column](%id = 2674588)
 

> bb:=<a,b,c>;
 

Vector[column](%id = 3110508)
 

> A.X=bb;
 

Vector[column](%id = 3219128) = Vector[column](%id = 3110508)
 

Recall in the second section of chapter 1 we could have rewritten the system in the following manner.  First let us look at a command to take off the columns of A one at a time.  

 

 

> c1A:=Column(A,1); c2A:=Column(A,2); c3A:=Column(A,3);
 

 

 

Vector[column](%id = 1068300)
Vector[column](%id = 3265348)
Vector[column](%id = 3274788) (1.1)
 

> x*c1A+y*c2A+z*c3A=bb;
 

Vector[column](%id = 3411812) = Vector[column](%id = 3110508)
 

This tells us that the vectors Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi( must be in the space span by the columns of the matrix A. 

We now want to know something about the columns of the matrix A.   First are the columns of A linearly independent.   To do this we will look at the homogenous system of equation and see if we get a solution.  If we get a solution other than zero then they are linearly depandent on the other hand if we get only the zero solution then the columns are linearly independent.   

 

 

> zero:=<0,0,0>;
 

Vector[column](%id = 3094688)
 

> BackwardSubstitute(GaussianElimination(<A|zero>));
 

Vector[column](%id = 1430012)
 

This tell us that the columns of A are linearly independent so form a basis for all of Typesetting:-mrow(Typesetting:-mi( so any vector b will have a solution.    So far we have found that the range of A or the column space of A is all of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( if A is an nxn matrix with the columns linearly independent.  What else do we know about the matrix A?  Is it non-singular?   

 

Consider now a second example.  You must read further than the following to learn how the problems should be worked.  Please read the following modivation and do not just try to go to where you can work the problems.  The following will help you better understand why we want to look at the short cuts for finding different basis elements for what we are going to call the row and column spaces of a matirx.   

 

 

> A1:=<<2,2,1>|<2,-2,-1>|<4,6,3>>;
 

Matrix(%id = 3583332)
 

> A1.X=bb;
 

Vector[column](%id = 3599220) = Vector[column](%id = 3110508)
 

> GaussianElimination(<A1|bb>);
 

Matrix(%id = 3656504)
 

>
 

We see that c=b/2 in order to have a solution   and  we have the columns of the coefficient matrix are not linearly independent.    The solutions wee then be of the form   Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(  or Typesetting:-mrow(Typesetting:-mi( which tell us that the solutions are in the following: span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi( , Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(}  This tells us that the columns of the origional matrix are not linearly independent. 

Why?  Think about dimension.   We see that none of the columns of the origional matrix are in this set of spanning vectors.  But a close look at the first two columns tells us that  

2 times the first column  plus 2 times the second column of the basis will give the first column of the matrix.  Also, 2 times the first column plus -2 times the second column of the basis will give the second column.    We can see that different basis for the column or range space is given by 

span{ Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(,Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} = span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(,Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} .  A question that we might ask is can be get the second set of spanning vectors from the origional matrix without having to augment the matrix with the arbitrary vector and do Gauss elimination.  Gauss elimination works on the rows so it is doing linear combination of the rows.  And what we would like to do is to do linear combinations on the columns 

.  How can we come up with more than one set of basis vectors for the column space?  We will not go through the process of augmenting the matrix with an arbitrary vector and then selecting constants.  The following is how to find two different basis element and even a third. The way to work the homework follows.  The above is a modivation of what makes this work.  Gauss elimination does row operations to a matrix. In order to see wich  columns are linearly independent we will take the transpose of the matrix and then do GaussianElimination and Gauss Jordan in the with(LinearAlgebra) package ReducedRowEchelonForm to the transposed matrix.  If gausselim does not interchange any row then we can obtain a third set of basis vectors.  This proceedure is to help you better understant that basis vectors for a subspace are not unique.  For the matrix A1 we will first take the transpose and then do GaussianElimination and ReducedRowEchelonForm to the matrix.  We want the row of the GaussianElimination  and ReducedRowEchelonForm of the transpose as basis vectors for the column space.  Do not list them as a matrix.  We are looking for vectors.    

 

   

> A1T:=A1^%T;
 

Matrix(%id = 2215580)
 

> AA1T:=GaussianElimination(A1T);
 

Matrix(%id = 2086188)
 

> AA2T:=ReducedRowEchelonForm(A1T);
 

Matrix(%id = 3785096)
 

> C1A1:=Row(AA1T,1)^%T;
 

Vector[column](%id = 3763232) (1.2)
 

> C1A2:=Row(AA1T,2)^%T;
 

Vector[column](%id = 1099212) (1.3)
 

The above commands will give you the tow vectors and you could then state that a basis is span{C1A1,C1A2}  You can used the same command on AA2T   

 

This gives us some insight into how to write three differnt represetation of a basis for the column space 

 

span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(,Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} = span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} = span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(}  

The basis elements are the first two rows of the transpose matrix , the two non-zero elements of the matrix obtained by gausselimination of the transpose and finially the two non-zero elements of the matrix by going through the steps for Gauss Jordan for the transpose matrix.   We have three different sets of spanning vectors.  Some of the vectors in each set of the same but we have three distinct set of basis vectors for the column or range space.   Here you can see that the second row is not a multiple of the first row of the transpose so they will be linearly independent and a basis for the space.  If you are not for sue for larger matrices then you may be able only to see two different sets of basis vectors by using gausselim and gaussjord.  Maple has a colspace command that will give one of the above sets.  You should always check to see which one Maple is giving you.  We will use the command that you might want to use if the matrices are larger and you are not for sure whether there was a row switch.  We will use it here but for this problem it is not necessry.  

. 

 

 

> v1:=Column(A1,1);
 

Vector[column](%id = 3797372) (1.1.1)
 

> v2:=Column(A1,2);
 

Vector[column](%id = 807836) (1.1.2)
 

> v3:=Column(A1,3);
 

Vector[column](%id = 903876) (1.1.3)
 

> Basis({v1,v2,v3});
 

 

This tells you that there were no row switches and that v1 and v2 are linearly independent.  You could use the command and see that any two of the three columns are linearly independent.  The above discussion also holds for the row space.  So look at the results for the row space in light of the discussion just given for the column space.   

{Vector[column](%id = 903876), Vector[column](%id = 807836)} (1.1.4)
 

Null Space 

 

Another subspace that is related to the solution of equations of the form Ax=b 

 

If we look at the homogeous equation Ax = 0.  Let us look the at two different solutions 

Ax = 0 and Ay = 0 then A(x + y) = 0 .  We can see that a scalar multiple we also hold.   

Aax = aO.  Then Aax = 0.  So the solution to the homogeous equation form a subspace called the nullspace of the matrix A.    If A is nonsingular then the only solution to the homogeious equation is the zero vector.  Why?   Recall when A is nonsingular we have a unique solution to the equation.  You should know that every homogeous equation has one solution whether the matrix is non-singular or not.  What solution is this?  

 

We will look at the last example above to try to find the null space of the matrix. 

A1x = 0.   

 

> BackwardSubstitute(GaussianElimination(<A1|zero>));
 

Vector[column](%id = 3400452)
 

The space is span by the single vector Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(.  So the null space of this matrix has dimension 1. 

 

The dimension of the column space is 2 and the dimension of the null space is one  

which is the number  of entries in each column.   What happens if the matrix is non singular like the matrix A in the first example?  Notice that the vectors in the column space  3 and columns are  linearly independent.  The dimension of the null space for A is zero since the zero vector is the only vector in the null space and it does not have dimension.    Returning to the matrix A1 we see every basis set for column space has two linearly independent vectors and the vector in the null space is linearly independent to the vectors in the column space.  Since the three vectors are linearly independent they will form a baisis for Typesetting:-mrow(Typesetting:-mi(.                    

Row Space or Range of the transpose of A. 

In order to look at the linearly independent rows in the same way that we looked at the linearly independent columns of a matrix we will need to look at a system of equations of the form 

Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( u = d .  Here d is a linear combination of the columns of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( which are the row of the orgional matrix.  Another way to look at this is to say that d is in the span of the columns of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi(.   Let us look at the second example since the first example is of little interest.  Why would that be?  Remember that if A is non-singular then so is Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi(.  This means that the columns of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( will be a basis for all of Typesetting:-mrow(Typesetting:-mi( and that we do not have an interesting subspace.    Let us now turn our attention to the second example 

    

 

> AT:=A1^%T;
 

Matrix(%id = 1412012)
 

> uu:=<u,v,w>;
 

Vector[column](%id = 1067148)
 

> dd:=<d,e,f>;
 

Vector[column](%id = 1065780)
 

> GaussianElimination(<AT|dd>);
 

Matrix(%id = 812708)
 

To have a solution then  f -5/2d + 1/2e =0 or f = 5/2d -1/2e.  We can write the vector of the form Typesetting:-mrow(Typesetting:-mi( = dTypesetting:-mrow(Typesetting:-mi( +  eTypesetting:-mrow(Typesetting:-mi( .   This tell us that a basis for the space span by the row of A or the columns of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( is given by 

 

span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(}  which again is not any of the columns of Typesetting:-mrow(Typesetting:-msup(Typesetting:-mi( or the row of A.   Since the transpose of the transpose is back to A then we can do Gauss elimination and Gauss Jordan to 

the origional matrix to see if we can come up with the above spanning set.   

 

 

> GaussianElimination(A1);
 

Matrix(%id = 1857312)
 

> ReducedRowEchelonForm(A1);
 

Matrix(%id = 3511388)
 

We see that the last computation gives the two vectors we found by the previous method.  Useing GaussinElimination and ReducedRowEchelonForm commands are the correct way to find the Row and Column space and not the longer way of setting the last row equal to zero,  There is also a short-cut to finding a set of Row and Column  space vectors.  Are not to use this command for this home work but I will inform you when you can use it.   I will need to put the command at the end of this discussion.    

>
 

We can now put together three different spans.  The first two rows of the origional matrix are linearly independent so we can now write the different basis.  I will divide by 2 and by -4 to simplify matters.   span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} = span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} = span{Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(, Typesetting:-mrow(Typesetting:-mfenced(Typesetting:-mrow(Typesetting:-mi(} .  The study of row and column spaces have shown us that there are three different basis for each space that are very easy to find.  One last observation that is very important.  The vectors in the row space  

are orthogonal to the vectors in the null space.  Why is this true?  The inner product will be zero so perpendicular which is by definition.   

 

You should also be aware from you text that the dimension of the row space plus null space should be equal to the number of columns  in the matrix.  Also, the dimension of the column space is always equal to the dimension of the row space.   

> RowSpace(A1);
 

[Vector[row](%id = 1786300), Vector[row](%id = 1857052)] (1.3.1)
 

> ColumnSpace(A1);
 

[Vector[column](%id = 2820652), Vector[column](%id = 1881488)] (1.3.2)
 

>
 

>
 

>
 

Notice that this the same set of vector you found by using the Reduced RowEchelonForm command.  The ReducedRowEchelonForm is nothing more than Gauss Jordan.     

Exercises  2.3 

Exercise 2.3.1 

Exercise 2.3.3  Find the nullspace and two different sets of basis vectors for the column and row space for (a) and (b).  Do not express the solutionn in the form Typesetting:-mrow(Typesetting:-msub(Typesetting:-mi(+ Typesetting:-mrow(Typesetting:-msub(Typesetting:-mi(this time around we will address this in another lecture.   

Exercise 2.3.8  Find the nullspace and two different sets of basis vectors for the column and row  space for (b), (e) .   

You can use GaussianElimination  , ReducedRowEchelonForm to find the different sets of basis vectors.  Due October 17, 2008