Modeling and Analysis of Stochastic Systems

IntroductionWhat in the World is a Stochastic Process? How to Characterize a Stochastic Process What Do We Do with a Stochastic Process? Discrete-Time Markov Chains: Transient Behaviour Definition and Characterization Examples DTMCs in Other Fields Marginal Distributions Occupancy Times Computation of Matrix Powers Modeling Exercises Computational Exercises Conceptual Exercises Discrete-Time Markov Chains: First Passage Times Definitions Cumulative Distribution Function of T Absorption Probabilities Expectation of T Generating Function and Higher Moments of T Computational Exercises Conceptual Exercises Discrete-Time Markox Chains: Limiting Behaviour Exploring the Limiting Behaviour by Examples Classification of States Determining Recurrence and Transience: Finite DTMCs Determining Recurrence and Transience: Infinite DTMSc Limiting Behaviour of Irreducible DTMCs Examples: Limiting Behaviour of Infinite State-Space Irreducible DTMCs Limiting Behaviour of Reducible DTMCs DTMCs with Costs and Rewards Reversibility Computational Exercises Conceptual Exercises Poisson Processes Exponential Distributions Poisson Process: Definitions Event Times in a Poisson Process Superposition and Splitting of Poisson Processes Non-Homogeneous Poisson Process Compound Poisson Process Computational Exercises Conceptual Exercises Continuous-Time Markov Chains Definitions and Sample Path Properties Examples CTMCs in Other Fields Transient Behaviour: Marginal Distribution Transient Behaviour: Occupancy Times Computation of P(t): Finite State-Space Computation of P(t): Infinite State-Space First-Passage Times Exploring the Limiting Behaviour by Examples Classification of States Limiting Behaviour of Irreducible CTMCs Limiting Behaviour of Reducible CTMCs CTMCs with Costs and Rewards Phase Type Distributions Reversibility Modeling Exercises Computational Exercises Conceptual Exercises Queueing Models Introduction Properties of General Queueing Systems Birth and Death Queues Open Queueing Networks Closed Queueing Networks Single Server Queues Retrial Queue Infinite Server Queue Modeling Exercises Computational Exercises Renewal Processes Introduction Properties of N(t) The Renewal Function Renewal-Type Equation Key Renewal Theorem Recurrence Times Delayed Renewal Processes Semi-Markov Processes Renewal Processes with Costs/Rewards Regenerative Processes Computational Exercises Conceptual Exercises Markov Regenerative Processes Definitions and Examples Markov Renewal Process and Markov Renewal Function Key Renewal Theorem for MRPs Semi-Markov Processes: Further Results Markov Regenerative Processes Applications to Queues Modeling Exercises Computational Exercises Conceptual Exercises Diffusion Process Brownian Motion Sample Path Properties of BM Kolmogorov Equations for Standard Brownian Motion First Passage Times Reflected SBM Reflected BM and Limiting Distributions BM and Martingales Cost/Reward Models Stochastic Integration Stochastic Differential Equations and Ito's Formula Applications to Finance Computational Exercises Conceptual Exercises Epilogue Probability of Events AppendicesUnivariate Random Variables Multivariate Random Variables Generating Functions Laplace-Stieltjes Transforms Laplace Transforms Modes of Convergence Results from Analysis Difference and Differential Equations Answers to Selected Problems References Index