Numerical Methods for Simulation and Optimization of Piecewise Deterministic Markov Processes

Preface ix Introduction xi Part 1. Piecewise Deterministic Markov Processes and Quantization 1 Chapter 1. Piecewise Deterministic Markov Processes 3 1.1. Introduction 3 1.2. Notation 4 1.3. Definition of a PDMP 5 1.4. Regularityassumptions 8 1.4.1. Lipschitz continuity along the flow 8 1.4.2. Regularity assumptions on the local characteristics 9 1.5. Time-augmentedprocess 11 1.6. EmbeddedMarkovchain 15 1.7. Stopping times 16 1.8. ExamplesofPDMPs 20 1.8.1. Poisson processwith trend 20 1.8.2. TCP 21 1.8.3. Air conditioningunit 22 1.8.4. Crack propagationmodel 23 1.8.5. Repairworkshopmodel 24 Chapter 2. Examples in Reliability 27 2.1. Introduction 27 2.2. Structure subject to corrosion 28 2.2.1. PDMPmodel 29 2.2.2. Deterministic time to reach the boundary 32 2.3. The heatedhold-uptank 33 2.3.1. Tank dynamics 34 2.3.2. PDMPmodel 36 Chapter 3. Quantization Technique 39 3.1. Introduction 39 3.2. Optimal quantization 40 3.2.1. Optimal quantization of a random variable 40 3.2.2. Optimal quantization of a Markovchain 42 3.3. SimulationofPDMPs 44 3.3.1. Simulation of time-dependent intensity 45 3.3.2. Simulation of trajectories 45 3.4. QuantizationofPDMPs 47 3.4.1. Scale of coordinates of the state variable 48 3.4.2. Cardinality of the mode variable 50 Part 2. Simulation of Functionals 53 Chapter 4. Expectation of Functionals 55 4.1. Introduction 55 4.2. Recursive formulation 57 4.2.1. Lipschitz continuity 58 4.2.2. Iterated operator 60 4.2.3. Approximationscheme 61 4.3. Lipschitz regularity 62 4.4. Rate of convergence 69 4.5. Time-dependent functionals 71 4.6. Deterministic time horizon 74 4.6.1. Direct estimation of the running cost term 74 4.6.2. Bounds of the boundary jump cost term 77 4.6.3. Bounds in the general case 79 4.7. Example 81 4.8. Conclusion 84 Chapter 5. Exit Time 87 5.1. Introduction 87 5.2. Problem setting 88 5.2.1. Distribution 90 5.2.2. Moments 91 5.2.3. Computationhorizon 92 5.3. Approximationschemes 92 5.4. Convergence 95 5.4.1. Distribution 95 5.4.2. Moments 100 5.5. Example 101 5.6. Conclusion 108 Chapter 6. Example in Reliability: Service Time 109 6.1. Mean thickness loss 109 6.2. Service time 112 6.2.1. Mean service time 114 6.2.2. Distribution of the service time 118 6.3. Conclusion 121 Part 3. Optimization 123 Chapter 7. Optimal Stopping 125 7.1. Introduction 125 7.2. Dynamic programming equation 128 7.3. Approximation of the value function 130 7.4. Lipschitz continuity properties 132 7.4.1. Lipschitz properties of J and K 132 7.4.2. Lipschitz properties of the value functions 135 7.5. Error estimation for the value function 138 7.5.1. Second term 140 7.5.2. Third term 141 7.5.3. Fourth term 147 7.5.4. Proof of theorem 7.1 148 7.6. Numerical construction of an [1]-optimal stopping time 149 7.7. Example 161 Chapter 8. Partially Observed Optimal Stopping Problem 165 8.1. Introduction 165 8.2. Problem formulation and assumptions 167 8.3. Optimal filtering 170 8.4. Dynamicprogramming 175 8.4.1. Preliminaryresults 176 8.4.2. Optimal stopping problem under complete observation 180 8.4.3. Dynamic programming equation 181 8.5. Numerical approximation by quantization 188 8.5.1. Lipschitz properties 189 8.5.2. Discretization scheme 195 8.5.3. Numerical construction of an [1]-optimal stopping time 205 8.6. Numerical example 211 Chapter 9. Example in Reliability: Maintenance Optimization 215 9.1. Introduction 215 9.2. Corrosionprocess 216 9.3. Air conditioningunit 219 9.4. The heatedhold-uptank 221 9.4.1. Problem setting and simulation 222 9.4.2. Numerical results and validation 224 9.5. Conclusion 228 Chapter 10. Optimal Impulse Control 231 10.1. Introduction 231 10.2. Impulse controlproblem233 10.3. Lipschitz-continuity properties 236 10.3.1. Lipschitz properties of the operators 236 10.3.2. Lipschitz properties of the operator L 239 10.4. Approximation of the value function 242 10.4.1. Time discretization 245 10.4.2. Approximation of the value functions on the control grid U 246 10.4.3. Approximation of the value function 255 10.4.4. Step-by-step description of the algorithm 259 10.4.5. Practical implementation 259 10.5. Example 262 10.6. Conclusion 264 Bibliography 269 Index 277