Scalar, Vector, and Matrix Mathematics

Preface to the Revised and Expanded Edition xviiPreface to the Second Edition xixPreface to the First Edition xxiSpecial Symbols xxvConventions, Notation, and Terminology xxxvii1. Sets, Logic, Numbers, Relations, Orderings, Graphs, and Functions 11.1 Sets 11.2 Logic 21.3 Relations and Orderings 51.4 Directed and Symmetric Graphs 91.5 Numbers 121.6 Functions and Their Inverses 161.7 Facts on Logic 211.8 Facts on Sets 221.9 Facts on Graphs 251.10 Facts on Functions 261.11 Facts on Integers 281.12 Facts on Finite Sums 361.13 Facts on Factorials 491.14 Facts on Finite Products 521.15 Facts on Numbers 521.16 Facts on Binomial Coefficients 541.17 Facts on Fibonacci, Lucas, and Pell Numbers 951.18 Facts on Arrangement, Derangement, and Catalan Numbers 1031.19 Facts on Cycle, Subset, Eulerian, Bell, and Ordered Bell Numbers 1051.20 Facts on Partition Numbers, the Totient Function, and Divisor Sums 1131.21 Facts on Convex Functions 1161.22 Notes 1182. Equalities and Inequalities 1192.1 Facts on Equalities and Inequalities in One Variable 1192.2 Facts on Equalities and Inequalities in Two Variables 1292.3 Facts on Equalities and Inequalities in Three Variables 1462.4 Facts on Equalities and Inequalities in Four Variables 1772.5 Facts on Equalities and Inequalities in Five Variables 1832.6 Facts on Equalities and Inequalities in Six Variables 1842.7 Facts on Equalities and Inequalities in Seven Variables 1862.8 Facts on Equalities and Inequalities in Eight Variables 1872.9 Facts on Equalities and Inequalities in Nine Variables 1872.10 Facts on Equalities and Inequalities in Sixteen Variables 1872.11 Facts on Equalities and Inequalities in n Variables 1882.12 Facts on Equalities and Inequalities in 2n Variables 2152.13 Facts on Equalities and Inequalities in 3n Variables 2262.14 Facts on Equalities and Inequalities in 4n Variables 2262.15 Facts on Equalities and Inequalities for the Logarithm Function 2262.16 Facts on Equalities for Trigonometric Functions 2312.17 Facts on Inequalities for Trigonometric Functions 2462.18 Facts on Equalities and Inequalities for Inverse Trigonometric Functions 2542.19 Facts on Equalities and Inequalities for Hyperbolic Functions 2612.20 Facts on Equalities and Inequalities for Inverse Hyperbolic Functions 2642.21 Facts on Equalities and Inequalities in Complex Variables 2662.22 Notes 2763. Basic Matrix Properties 2773.1 Vectors 2773.2 Matrices 2803.3 Transpose and Inner Product 2853.4 Geometrically Defined Sets 2903.5 Range and Null Space 2903.6 Rank and Defect 2923.7 Invertibility 2943.8 The Determinant 2993.9 Partitioned Matrices 3023.10 Majorization 3053.11 Facts on One Set 3063.12 Facts on Two or More Sets 3103.13 Facts on Range, Null Space, Rank, and Defect 3153.14 Facts on the Range, Rank, Null Space, and Defect of Partitioned Matrices 3203.15 Facts on the Inner Product, Outer Product, Trace, and Matrix Powers 3263.16 Facts on the Determinant 3293.17 Facts on the Determinant of Partitioned Matrices 3343.18 Facts on Left and Right Inverses 3423.19 Facts on the Adjugate 3453.20 Facts on the Inverse 3483.21 Facts on Bordered Matrices 3513.22 Facts on the Inverse of Partitioned Matrices 3523.23 Facts on Commutators 3543.24 Facts on Complex Matrices 3563.25 Facts on Majorization 3593.26 Notes 3624. Matrix Classes and Transformations 3634.1 Types of Matrices 3634.2 Matrices Related to Graphs 3674.3 Lie Algebras 3684.4 Abstract Groups 3694.5 Addition Groups 3714.6 Multiplication Groups 3714.7 Matrix Transformations 3734.8 Projectors, Idempotent Matrices, and Subspaces 3744.9 Facts on Elementary, Group-Invertible, Range-Hermitian, Range-Disjoint, and Range-Spanning Matrices 3764.10 Facts on Normal, Hermitian, and Skew-Hermitian Matrices 3774.11 Facts on Linear Interpolation 3834.12 Facts on the Cross Product 3844.13 Facts on Inner, Unitary, and Shifted-Unitary Matrices 3874.14 Facts on Rotation Matrices 3914.15 Facts on One Idempotent Matrix 3964.16 Facts on Two or More Idempotent Matrices 3984.17 Facts on One Projector 4074.18 Facts on Two or More Projectors 4094.19 Facts on Reflectors 4164.20 Facts on Involutory Matrices 4174.21 Facts on Tripotent Matrices 4174.22 Facts on Nilpotent Matrices 4184.23 Facts on Hankel and Toeplitz Matrices 4204.24 Facts on Tridiagonal Matrices 4224.25 Facts on Triangular, Hessenberg, and Irreducible Matrices 4244.26 Facts on Matrices Related to Graphs 4264.27 Facts on Dissipative, Contractive, Cauchy, and Centrosymmetric Matrices 4274.28 Facts on Hamiltonian and Symplectic Matrices 4274.29 Facts on Commutators 4284.30 Facts on Partial Orderings 4304.31 Facts on Groups 4324.32 Facts on Quaternions 4374.33 Notes 4405. Geometry 4415.1 Facts on Angles, Lines, and Planes 4415.2 Facts on Triangles 4435.3 Facts on Polygons and Polyhedra 4895.4 Facts on Polytopes 4935.5 Facts on Circles, Ellipses, Spheres, and Ellipsoids 4956. Polynomial Matrices and Rational Transfer Functions 4996.1 Polynomials 4996.2 Polynomial Matrices 5016.3 The Smith Form and Similarity Invariants 5036.4 Eigenvalues 5066.5 Eigenvectors 5116.6 The Minimal Polynomial 5126.7 Rational Transfer Functions and the Smith-McMillan Form 5136.8 Facts on Polynomials and Rational Functions 5176.9 Facts on the Characteristic and Minimal Polynomials 5246.10 Facts on the Spectrum 5306.11 Facts on Graphs and Nonnegative Matrices 5376.12 Notes 5447. Matrix Decompositions 5457.1 Smith Decomposition 5457.2 Reduced Row Echelon Decomposition 5457.3 Multicompanion and Elementary Multicompanion Decompositions 5467.4 Jordan Decomposition 5497.5 Schur Decomposition 5537.6 Singular Value Decomposition, Polar Decomposition, and Full-Rank Factorization 5557.7 Eigenstructure Properties 5587.8 Pencils and the Kronecker Canonical Form 5637.9 Facts on the Inertia 5657.10 Facts on Matrix Transformations for One Matrix 5697.11 Facts on Matrix Transformations for Two or More Matrices 5757.12 Facts on Eigenvalues and Singular Values for One Matrix 5797.13 Facts on Eigenvalues and Singular Values for Two or More Matrices 5897.14 Facts on Matrix Pencils 5977.15 Facts on Eigenstructure for One Matrix 5977.16 Facts on Eigenstructure for Two or More Matrices 6037.17 Facts on Matrix Factorizations 6057.18 Facts on Companion, Vandermonde, Circulant, Permutation, and Hadamard Matrices 6107.19 Facts on Simultaneous Transformations 6177.20 Facts on Additive Decompositions 6187.21 Notes 6198. Generalized Inverses 6218.1 Moore-Penrose Generalized Inverse 6218.2 Drazin Generalized Inverse 6258.3 Facts on the Moore-Penrose Generalized Inverse for One Matrix 6288.4 Facts on the Moore-Penrose Generalized Inverse for Two or More Matrices 6328.5 Facts on the Moore-Penrose Generalized Inverse for Range-Hermitian, Range-Disjoint, and Range-Spanning Matrices 6418.6 Facts on the Moore-Penrose Generalized Inverse for Normal Matrices, Hermitian Matrices, and Partial Isometries 6498.7 Facts on the Moore-Penrose Generalized Inverse for Idempotent Matrices 6508.8 Facts on the Moore-Penrose Generalized Inverse for Projectors 6528.9 Facts on the Moore-Penrose Generalized Inverse for Partitioned Matrices 6598.10 Facts on the Drazin and Group Generalized Inverses for One Matrix 6698.11 Facts on the Drazin and Group Generalized Inverses for Two or More Matrices 6748.12 Facts on the Drazin and Group Generalized Inverses for Partitioned Matrices 6788.13 Notes 6799. Kronecker and Schur Algebra 6819.1 Kronecker Product 6819.2 Kronecker Sum and Linear Matrix Equations 6839.3 Schur Product 6859.4 Facts on the Kronecker Product 6859.5 Facts on the Kronecker Sum 6919.6 Facts on the Schur Product 6979.7 Notes 70110.Positive-Semidefinite Matrices 70310.1 Positive-Semidefinite and Positive-Definite Orderings 70310.2 Submatrices and Schur Complements 70410.3 Simultaneous Diagonalization 70710.4 Eigenvalue Inequalities 70910.5 Exponential, Square Root, and Logarithm of Hermitian Matrices 71310.6 Matrix Inequalities 71410.7 Facts on Range and Rank 72210.8 Facts on Unitary Matrices and the Polar Decomposition 72310.9 Facts on Structured Positive-Semidefinite Matrices 72410.10 Facts on Equalities and Inequalities for One Matrix 73010.11 Facts on Equalities and Inequalities for Two or More Matrices 73510.12 Facts on Equalities and Inequalities for Partitioned Matrices 74910.13 Facts on the Trace for One Matrix 76110.14 Facts on the Trace for Two or More Matrices 76310.15 Facts on the Determinant for One Matrix 77410.16 Facts on the Determinant for Two or More Matrices 77610.17 Facts on Convex Sets and Convex Functions 78510.18 Facts on Quadratic Forms for One Matrix 79210.19 Facts on Quadratic Forms for Two or More Matrices 79510.20 Facts on Simultaneous Diagonalization 79910.21 Facts on Eigenvalues and Singular Values for One Matrix 80010.22 Facts on Eigenvalues and Singular Values for Two or More Matrices 80410.23 Facts on Alternative Partial Orderings 81310.24 Facts on Generalized Inverses 81510.25 Facts on the Kronecker and Schur Products 82010.26 Notes 83111.Norms 83311.1 Vector Norms 83311.2 Matrix Norms 83511.3 Compatible Norms 83811.4 Induced Norms 84111.5 Induced Lower Bound 84511.6 Singular Value Inequalities 84711.7 Facts on Vector Norms 84911.8 Facts on Vector p-Norms 85311.9 Facts on Matrix Norms for One Matrix 86011.10 Facts on Matrix Norms for Two or More Matrices 86811.11 Facts on Matrix Norms for Commutators 88411.12 Facts on Matrix Norms for Partitioned Matrices 88511.13 Facts on Matrix Norms and Eigenvalues for One Matrix 89011.14 Facts on Matrix Norms and Eigenvalues for Two or More Matrices 89211.15 Facts on Matrix Norms and Singular Values for One Matrix 89511.16 Facts on Matrix Norms and Singular Values for Two or More Matrices 89911.17 Facts on Linear Equations and Least Squares 90911.18 Notes 91212.Functions, Limits, Sequences, Series, Infinite Products, and Derivatives 91312.1 Open Sets and Closed Sets 91312.2 Limits of Sequences 91512.3 Series, Power Series, and Bi-power Series 91912.4 Continuity 92112.5 Derivatives 92412.6 Complex-Valued Functions 92612.7 Infinite Products 92912.8 Functions of a Matrix 93012.9 Matrix Square Root and Matrix Sign Functions 93212.10 Vector and Matrix Derivatives 93212.11 Facts on One Set 93412.12 Facts on Two or More Sets 93712.13 Facts on Functions 94112.14 Facts on Functions of a Complex Variable 94512.15 Facts on Functions of a Matrix 94812.16 Facts on Derivatives 94912.17 Facts on Limits of Functions 95412.18 Facts on Limits of Sequences and Series 95712.19 Notes 97413.Infinite Series, Infinite Products, and Special Functions 97513.1 Facts on Series for Subset, Eulerian, Partition, Bell, Ordered Bell, Bernoulli, Euler, and Up/Down Numbers 97513.2 Facts on Bernoulli, Euler, Chebyshev, Legendre, Laguerre, Hermite, Bell, Ordered Bell, Harmonic, Fibonacci, and Lucas Polynomials 98113.3 Facts on the Zeta, Gamma, Digamma, Generalized Harmonic, Dilogarithm, and Dirichlet L Functions 99413.4 Facts on Power Series, Laurent Series, and Partial Fraction Expansions 100413.5 Facts on Series of Rational Functions 102113.6 Facts on Series of Trigonometric and Hyperbolic Functions 105713.7 Facts on Series of Binomial Coefficients 106313.8 Facts on Double-Summation Series 107113.9 Facts on Miscellaneous Series 107413.10 Facts on Infinite Products 108013.11 Notes 109214.Integrals 109314.1 Facts on Indefinite Integrals 109314.2 Facts on Definite Integrals of Rational Functions 109614.3 Facts on Definite Integrals of Radicals 111114.4 Facts on Definite Integrals of Trigonometric Functions 111414.5 Facts on Definite Integrals of Inverse Trigonometric Functions 113014.6 Facts on Definite Integrals of Logarithmic Functions 113214.7 Facts on Definite Integrals of Logarithmic, Trigonometric, and Hyperbolic Functions 115014.8 Facts on Definite Integrals of Exponential Functions 115714.9 Facts on Integral Representations of G and y 116914.10 Facts on Definite Integrals of the Gamma Function 117114.11 Facts on Integral Inequalities 117114.12 Facts on the Gaussian Density 117214.13 Facts on Multiple Integrals 117314.14 Notes 117815.The Matrix Exponential and Stability Theory 117915.1 Definition of the Matrix Exponential 117915.2 Structure of the Matrix Exponential 118115.3 Explicit Expressions 118515.4 Matrix Logarithms 118715.5 Principal Logarithm 119015.6 Lie Groups 119115.7 Linear Time-Varying Differential Equations 119315.8 Lyapunov Stability Theory 119515.9 Linear Stability Theory 119815.10 The Lyapunov Equation 120115.11 Discrete-Time Stability Theory 120315.12 Facts on Matrix Exponential Formulas 120415.13 Facts on the Matrix Sine and Cosine 120915.14 Facts on the Matrix Exponential for One Matrix 120915.15 Facts on the Matrix Exponential for Two or More Matrices 121115.16 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for One Matrix 121715.17 Facts on the Matrix Exponential and Eigenvalues, Singular Values, and Norms for Two or More Matrices 122015.18 Facts on Stable Polynomials 122315.19 Facts on Stable Matrices 122615.20 Facts on Almost Nonnegative Matrices 123215.21 Facts on Discrete-Time-Stable Polynomials 123415.22 Facts on Discrete-Time-Stable Matrices 123915.23 Facts on Lie Groups 124315.24 Facts on Subspace Decomposition 124315.25 Notes 124716.Linear Systems and Control Theory 124916.1 State Space Models 124916.2 Laplace Transform Analysis and Transfer Functions 125216.3 The Unobservable Subspace and Observability 125316.4 Observable Asymptotic Stability 125716.5 Detectability 125916.6 The Controllable Subspace and Controllability 125916.7 Controllable Asymptotic Stability 126616.8 Stabilizability 126816.9 Realization Theory 127016.10 Zeros 127816.11 H2 System Norm 128516.12 Harmonic Steady-State Response 128816.13 System Interconnections 128916.14 Standard Control Problem 129116.15 Linear-Quadratic Control 129316.16 Solutions of the Riccati Equation 129516.17 The Stabilizing Solution of the Riccati Equation 129816.18 The Maximal Solution of the Riccati Equation 130216.19 Positive-Semidefinite and Positive-Definite Solutions of the Riccati Equation 130416.20 Facts on Linear Differential Equations 130516.21 Facts on Stability, Observability, and Controllability 130716.22 Facts on the Lyapunov Equation and Inertia 130916.23 Facts on the Discrete-Time Lyapunov Equation 131316.24 Facts on Realizations and the H2 System Norm 131316.25 Facts on the Riccati Equation 131616.26 Notes 1319Bibliography 1321Author Index 1433Subject Index 1449