Cyber-Physical-Human Systems

A Note from the Series Editor xvii About the Editors xviii List of Contributors xix Introduction xxvii Part I Fundamental Concepts and Methods 1 1 Human-in-the-Loop Control and Cyber–Physical–Human Systems: Applications and Categorization 3 Tariq Samad 1.1 Introduction 3 1.2 Cyber + Physical + Human 4 1.2.1 Cyberphysical Systems 5 1.2.2 Physical–Human Systems 6 1.2.3 Cyber–Human Systems 6 1.3 Categorizing Human-in-the-Loop Control Systems 6 1.3.1 Human-in-the-Plant 8 1.3.2 Human-in-the-Controller 8 1.3.3 Human–Machine Control Symbiosis 10 1.3.4 Humans-in-Multiagent-Loops 11 1.4 A Roadmap for Human-in-the-Loop Control 13 1.4.1 Self- and Human-Driven Cars on Urban Roads 13 1.4.2 Climate Change Mitigation and Smart Grids 14 1.5 Discussion 15 1.5.1 Other Ways of Classifying Human-in-the-Loop Control 15 1.5.2 Modeling Human Understanding and Decision-Making 16 1.5.3 Ethics and CPHS 18 1.6 Conclusions 19 Acknowledgments 19 References 20 2 Human Behavioral Models Using Utility Theory and Prospect Theory 25 Anuradha M. Annaswamy and Vineet Jagadeesan Nair 2.1 Introduction 25 2.2 Utility Theory 26 2.2.1 An Example 27 2.3 Prospect Theory 27 2.3.1 An Example: CPT Modeling for SRS 30 2.3.1.1 Detection of CPT Effects via Lotteries 32 2.3.2 Theoretical Implications of CPT 33 2.3.2.1 Implication I: Fourfold Pattern of Risk Attitudes 34 2.3.2.2 Implication II: Strong Risk Aversion Over Mixed Prospects 36 2.3.2.3 Implication III: Effects of Self-Reference 37 2.4 Summary and Conclusions 38 Acknowledgments 39 References 39 3 Social Diffusion Dynamics in Cyber–Physical–Human Systems 43 Lorenzo Zino and Ming Cao 3.1 Introduction 43 3.2 General Formalism for Social Diffusion in CPHS 45 3.2.1 Complex and Multiplex Networks 45 3.2.2 General Framework for Social Diffusion 46 3.2.3 Main Theoretical Approaches 48 3.3 Modeling Decision-Making 49 3.3.1 Pairwise Interaction Models 49 3.3.2 Linear Threshold Models 52 3.3.3 Game-Theoretic Models 53 3.4 Dynamics in CPHS 55 3.4.1 Social Diffusion in Multiplex Networks 56 3.4.2 Co-Evolutionary Social Dynamics 58 3.5 Ongoing Efforts Toward Controlling Social Diffusion and Future Challenges 62 Acknowledgments 63 References 63 4 Opportunities and Threats of Interactions Between Humans and Cyber–Physical Systems – Integration and Inclusion Approaches for Cphs 71 Frédéric Vanderhaegen and Victor Díaz Benito Jiménez 4.1 CPHS and Shared Control 72 4.2 “Tailor-made” Principles for Human–CPS Integration 73 4.3 “All-in-one” based Principles for Human–CPS Inclusion 74 4.4 Dissonances, Opportunities, and Threats in a CPHS 76 4.5 Examples of Opportunities and Threats 79 4.6 Conclusions 85 References 86 5 Enabling Human-Aware Autonomy Through Cognitive Modeling and Feedback Control 91 Neera Jain, Tahira Reid, Kumar Akash, Madeleine Yuh, and Jacob Hunter 5.1 Introduction 91 5.1.1 Important Cognitive Factors in HAI 92 5.1.2 Challenges with Existing CPHS Methods 93 5.1.3 How to Read This Chapter 95 5.2 Cognitive Modeling 95 5.2.1 Modeling Considerations 95 5.2.2 Cognitive Architectures 97 5.2.3 Computational Cognitive Models 98 5.2.3.1 ARMAV and Deterministic Linear Models 99 5.2.3.2 Dynamic Bayesian Models 99 5.2.3.3 Decision Analytical Models 100 5.2.3.4 POMDP Models 102 5.3 Study Design and Data Collection 103 5.3.1 Frame Research Questions and Identify Variables 104 5.3.2 Formulate Hypotheses or Determine the Data Needed 105 5.3.2.1 Hypothesis Testing Approach 105 5.3.2.2 Model Training Approach 105 5.3.3 Design Experiment and/or Study Scenario 107 5.3.3.1 Hypothesis Testing Approach 107 5.3.3.2 Model Training Approach 107 5.3.4 Conduct Pilot Studies and Get Initial Feedback; Do Preliminary Analysis 108 5.3.5 A Note about Institutional Review Boards and Recruiting Participants 109 5.4 Cognitive Feedback Control 109 5.4.1 Considerations for Feedback Control 110 5.4.2 Approaches 111 5.4.2.1 Heuristics-Based Planning 111 5.4.2.2 Measurement-Based Feedback 112 5.4.2.3 Goal-Oriented Feedback 112 5.4.2.4 Case Study 112 5.4.3 Evaluation Methods 113 5.5 Summary and Opportunities for Further Investigation 113 5.5.1 Model Generalizability and Adaptability 114 5.5.2 Measurement of Cognitive States 114 5.5.3 Human Subject Study Design 114 References 115 6 Shared Control with Human Trust and Workload Models 125 Murat Cubuktepe, Nils Jansen, and Ufuk Topcu 6.1 Introduction 125 6.1.1 Review of Shared Control Methods 126 6.1.2 Contribution and Approach 127 6.1.3 Review of IRL Methods Under Partial Information 128 6.1.3.1 Organization 129 6.2 Preliminaries 129 6.2.1 Markov Decision Processes 129 6.2.2 Partially Observable Markov Decision Processes 130 6.2.3 Specifications 130 6.3 Conceptual Description of Shared Control 131 6.4 Synthesis of the Autonomy Protocol 132 6.4.1 Strategy Blending 132 6.4.2 Solution to the Shared Control Synthesis Problem 133 6.4.2.1 Nonlinear Programming Formulation for POMDPs 133 6.4.2.2 Strategy Repair Using Sequential Convex Programming 134 6.4.3 Sequential Convex Programming Formulation 135 6.4.4 Linearizing Nonconvex Problem 135 6.4.4.1 Linearizing Nonconvex Constraints and Adding Slack Variables 135 6.4.4.2 Trust Region Constraints 136 6.4.4.3 Complete Algorithm 136 6.4.4.4 Additional Specifications 136 6.4.4.5 Additional Measures 137 6.5 Numerical Examples 137 6.5.1 Modeling Robot Dynamics as POMDPs 138 6.5.2 Generating Human Demonstrations 138 6.5.3 Learning a Human Strategy 139 6.5.4 Task Specification 139 6.5.5 Results 140 6.6 Conclusion 140 Acknowledgments 140 References 140 7 Parallel Intelligence for CPHS: An ACP Approach 145 Xiao Wang, Jing Yang, Xiaoshuang Li, and Fei-Yue Wang 7.1 Background and Motivation 145 7.2 Early Development in China 147 7.3 Key Elements and Framework 149 7.4 Operation and Process 151 7.4.1 Construction of Artificial Systems 152 7.4.2 Computational Experiments in Parallel Intelligent Systems 152 7.4.3 Closed-Loop Optimization Based on Parallel Execution 153 7.5 Applications 153 7.5.1 Parallel Control and Intelligent Control 154 7.5.2 Parallel Robotics and Parallel Manufacturing 156 7.5.3 Parallel Management and Intelligent Organizations 157 7.5.4 Parallel Medicine and Smart Healthcare 158 7.5.5 Parallel Ecology and Parallel Societies 160 7.5.6 Parallel Economic Systems and Social Computing 161 7.5.7 Parallel Military Systems 163 7.5.8 Parallel Cognition and Parallel Philosophy 164 7.6 Conclusion and Prospect 165 References 165 Part II Transportation 171 8 Regularities of Human Operator Behavior and Its Modeling 173 Aleksandr V. Efremov 8.1 Introduction 173 8.2 The Key Variables in Man–Machine Systems 174 8.3 Human Responses 177 8.4 Regularities of Man–Machine System in Manual Control 180 8.4.1 Man–Machine System in Single-loop Compensatory System 180 8.4.2 Man–Machine System in Multiloop, Multichannel, and Multimodal Tasks 185 8.4.2.1 Man–Machine System in the Multiloop Tracking Task 185 8.4.2.2 Man–Machine System in the Multichannel Tracking Task 187 8.4.2.3 Man–Machine System in Multimodal Tracking Tasks 188 8.4.2.4 Human Operator Behavior in Pursuit and Preview Tracking Tasks 191 8.5 Mathematical Modeling of Human Operator Behavior in Manual Control Task 194 8.5.1 McRuer’s Model for the Pilot Describing Function 194 8.5.1.1 Single-Loop Compensatory Model 194 8.5.1.2 Multiloop and Multimodal Compensatory Model 197 8.5.2 Structural Human Operator Model 197 8.5.3 Pilot Optimal Control Model 199 8.5.4 Pilot Models in Preview and Pursuit Tracking Tasks 201 8.6 Applications of the Man–Machine System Approach 202 8.6.1 Development of Criteria for Flying Qualities and PIO Prediction 203 8.6.1.1 Criteria of FQ and PIO Prediction as a Requirement for the Parameters of the Pilot-Aircraft System 203 8.6.1.2 Calculated Piloting Rating of FQ as the Criteria 205 8.6.2 Interfaces Design 206 8.6.3 Optimization of Control System and Vehicle Dynamics Parameters 210 8.7 Future Research Challenges and Visions 213 8.8 Conclusion 214 References 215 9 Safe Shared Control Between Pilots and Autopilots in the Face of Anomalies 219 Emre Eraslan, Yildiray Yildiz, and Anuradha M. Annaswamy 9.1 Introduction 219 9.2 Shared Control Architectures: A Taxonomy 221 9.3 Recent Research Results 222 9.3.1 Autopilot 224 9.3.1.1 Dynamic Model of the Aircraft 224 9.3.1.2 Advanced Autopilot Based on Adaptive Control 225 9.3.1.3 Autopilot Based on Proportional Derivative Control 228 9.3.2 Human Pilot 228 9.3.2.1 Pilot Models in the Absence of Anomaly 228 9.3.2.2 Pilot Models in the Presence of Anomaly 229 9.3.3 Shared Control 230 9.3.3.1 SCA1: A Pilot with a CfM-Based Perception and a Fixed-Gain Autopilot 231 9.3.3.2 SCA2: A Pilot with a CfM-Based Decision-Making and an Advanced Adaptive Autopilot 232 9.3.4 Validation with Human-in-the-Loop Simulations 232 9.3.5 Validation of Shared Control Architecture 1 234 9.3.5.1 Experimental Setup 234 9.3.5.2 Anomaly 235 9.3.5.3 Experimental Procedure 235 9.3.5.4 Details of the Human Subjects 236 9.3.5.5 Pilot-Model Parameters 237 9.3.5.6 Results and Observations 237 9.3.6 Validation of Shared Control Architecture 2 240 9.3.6.1 Experimental Setup 241 9.3.6.2 Anomaly 241 9.3.6.3 Experimental Procedure 242 9.3.6.4 Details of the Human Subjects 243 9.3.6.5 Results and Observations 244 9.4 Summary and Future Work 246 References 247 10 Safe Teleoperation of Connected and Automated Vehicles 251 Frank J. Jiang, Jonas Mårtensson, and Karl H. Johansson 10.1 Introduction 251 10.2 Safe Teleoperation 254 10.2.1 The Advent of 5G 258 10.3 CPHS Design Challenges in Safe Teleoperation 259 10.4 Recent Research Advances 261 10.4.1 Enhancing Operator Perception 261 10.4.2 Safe Shared Autonomy 264 10.5 Future Research Challenges 267 10.5.1 Full Utilization of V2X Networks 267 10.5.2 Mixed Autonomy Traffic Modeling 268 10.5.3 5G Experimentation 268 10.6 Conclusions 269 References 270 11 Charging Behavior of Electric Vehicles 273 Qing-Shan Jia and Teng Long 11.1 History, Challenges, and Opportunities 274 11.1.1 The History and Status Quo of EVs 274 11.1.2 The Current Challenge 276 11.1.3 The Opportunities 277 11.2 Data Sets and Problem Modeling 278 11.2.1 Data Sets of EV Charging Behavior 278 11.2.1.1 Trend Data Sets 279 11.2.1.2 Driving Data Sets 279 11.2.1.3 Battery Data Sets 279 11.2.1.4 Charging Data Sets 279 11.2.2 Problem Modeling 281 11.3 Control and Optimization Methods 284 11.3.1 The Difficulty of the Control and Optimization 284 11.3.2 Charging Location Selection and Routing Optimization 285 11.3.3 Charging Process Control 286 11.3.4 Control and Optimization Framework 287 11.3.4.1 Centralized Optimization 287 11.3.4.2 Decentralized Optimization 288 11.3.4.3 Hierarchical Optimization 288 11.3.5 The Impact of Human Behaviors 289 11.4 Conclusion and Discussion 289 References 290 Part III Robotics 299 12 Trust-Triggered Robot–Human Handovers Using Kinematic Redundancy for Collaborative Assembly in Flexible Manufacturing 301 S. M. Mizanoor Rahman, Behzad Sadrfaridpour, Ian D. Walker, and Yue Wang 12.1 Introduction 301 12.2 The Task Context and the Handover 303 12.3 The Underlying Trust Model 304 12.4 Trust-Based Handover Motion Planning Algorithm 305 12.4.1 The Overall Motion Planning Strategy 305 12.4.2 Manipulator Kinematics and Kinetics Models 305 12.4.3 Dynamic Impact Ellipsoid 306 12.4.4 The Novel Motion Control Approach 307 12.4.5 Illustration of the Novel Algorithm 308 12.5 Development of the Experimental Settings 310 12.5.1 Experimental Setup 310 12.5.1.1 Type I: Center Console Assembly 310 12.5.1.2 Type II: Hose Assembly 311 12.5.2 Real-Time Measurement and Display of Trust 311 12.5.2.1 Type I: Center Console Assembly 311 12.5.2.2 Type II: Hose Assembly 313 12.5.2.3 Trust Computation 313 12.5.3 Plans to Execute the Trust-Triggered Handover Strategy 314 12.5.3.1 Type I Assembly 314 12.5.3.2 Type II Assembly 314 12.6 Evaluation of the Motion Planning Algorithm 315 12.6.1 Objective 315 12.6.2 Experiment Design 315 12.6.3 Evaluation Scheme 315 12.6.4 Subjects 316 12.6.5 Experimental Procedures 316 12.6.5.1 Type I Assembly 317 12.6.5.2 Type II Assembly 317 12.7 Results and Analyses, Type I Assembly 318 12.8 Results and Analyses, Type II Assembly 322 12.9 Conclusions and Future Work 323 Acknowledgment 324 References 324 13 Fusing Electrical Stimulation and Wearable Robots with Humans to Restore and Enhance Mobility 329 Thomas Schauer, Eduard Fosch-Villaronga, and Juan C. Moreno 13.1 Introduction 329 13.1.1 Functional Electrical Stimulation 330 13.1.2 Spinal Cord Stimulation 331 13.1.3 Wearable Robotics (WR) 332 13.1.4 Fusing FES/SCS and Wearable Robotics 334 13.2 Control Challenges 335 13.2.1 Feedback Approaches to Promote Volition 336 13.2.2 Principles of Assist-as-Needed 336 13.2.3 Tracking Control Problem Formulation 336 13.2.4 Co-operative Control Strategies 337 13.2.5 EMG- and MMG-Based Assessment of Muscle Activation 344 13.3 Examples 345 13.3.1 A Hybrid Robotic System for Arm Training of Stroke Survivors 345 13.3.2 First Certified Hybrid Robotic Exoskeleton for Gait Rehabilitation Settings 347 13.3.3 Body Weight-Supported Robotic Gait Training with tSCS 348 13.3.4 Modular FES and Wearable Robots to Customize Hybrid Solutions 348 13.4 Transfer into Daily Practice: Integrating Ethical, Legal, and Societal Aspects into the Design 350 13.5 Summary and Outlook 352 Acknowledgments 353 Acronyms 353 References 354 14 Contemporary Issues and Advances in Human–Robot Collaborations 365 Takeshi Hatanaka, Junya Yamauchi, Masayuki Fujita, and Hiroyuki Handa 14.1 Overview of Human–Robot Collaborations 365 14.1.1 Task Architecture 366 14.1.2 Human–Robot Team Formation 368 14.1.3 Human Modeling: Control and Decision 369 14.1.4 Human Modeling: Other Human Factors 371 14.1.5 Industrial Perspective 372 14.1.6 What Is in This Chapter 375 14.2 Passivity-Based Human-Enabled Multirobot Navigation 376 14.2.1 Architecture Design 377 14.2.2 Human Passivity Analysis 379 14.2.3 Human Workload Analysis 381 14.3 Operation Support with Variable Autonomy via Gaussian Process 383 14.3.1 Design of the Operation Support System with Variable Autonomy 385 14.3.2 User Study 388 14.3.2.1 Operational Verification 388 14.3.2.2 Usability Test 390 14.4 Summary 391 Acknowledgments 393 References 393 Part IV Healthcare 401 15 Overview and Perspectives on the Assessment and Mitigation of Cognitive Fatigue in Operational Settings 403 Mike Salomone, Michel Audiffren, and Bruno Berberian 15.1 Introduction 403 15.2 Cognitive Fatigue 404 15.2.1 Definition 404 15.2.2 Origin of Cognitive Fatigue 404 15.2.3 Effects on Adaptive Capacities 406 15.3 Cyber–Physical System and Cognitive Fatigue: More Automation Does Not Imply Less Cognitive Fatigue 406 15.4 Assessing Cognitive Fatigue 409 15.4.1 Subjective Measures 409 15.4.2 Behavioral Measures 410 15.4.3 Physiological Measurements 410 15.5 Limitations and Benefits of These Measures 412 15.6 Current and Future Solutions and Countermeasures 412 15.6.1 Physiological Computing: Toward Real-Time Detection and Adaptation 412 15.7 System Design and Explainability 414 15.8 Future Challenges 415 15.8.1 Generalizing the Results Observed in the Laboratory to Ecological Situations 415 15.8.2 Determining the Specificity of Cognitive Fatigue 415 15.8.3 Recovering from Cognitive Fatigue 417 15.9 Conclusion 418 References 419 16 Epidemics Spread Over Networks: Influence of Infrastructure and Opinions 429 Baike She, Sebin Gracy, Shreyas Sundaram, Henrik Sandberg, Karl H. Johansson, andPhilipE.Paré 16.1 Introduction 429 16.1.1 Infectious Diseases 429 16.1.2 Modeling Epidemic Spreading Processes 430 16.1.3 Susceptible–Infected–Susceptible (SIS) Compartmental Models 431 16.2 Epidemics on Networks 432 16.2.1 Motivation 432 16.2.2 Modeling Epidemics over Networks 433 16.2.3 Networked Susceptible–Infected–Susceptible Epidemic Models 434 16.3 Epidemics and Cyber–Physical–Human Systems 436 16.3.1 Epidemic and Opinion Spreading Processes 437 16.3.2 Epidemic and Infrastructure 438 16.4 Recent Research Advances 439 16.4.1 Notation 439 16.4.2 Epidemic and Opinion Spreading Processes 440 16.4.2.1 Opinions Over Networks with Both Cooperative and Antagonistic Interactions 440 16.4.2.2 Coupled Epidemic and Opinion Dynamics 441 16.4.2.3 Opinion-Dependent Reproduction Number 443 16.4.2.4 Simulations 444 16.4.3 Epidemic Spreading with Shared Resources 445 16.4.3.1 The Multi-Virus SIWS Model 445 16.4.3.2 Problem Statements 447 16.4.3.3 Analysis of the Eradicated State of a Virus 448 16.4.3.4 Persistence of a Virus 449 16.4.3.5 Simulations 449 16.5 Future Research Challenges and Visions 450 References 451 17 Digital Twins and Automation of Care in the Intensive Care Unit 457 J. Geoffrey Chase, Cong Zhou, Jennifer L. Knopp, Knut Moeller, Balázs Benyo, Thomas Desaive, Jennifer H. K. Wong, Sanna Malinen, Katharina Naswall, Geoffrey M. Shaw, Bernard Lambermont, and Yeong S. Chiew 17.1 Introduction 457 17.1.1 Economic Context 458 17.1.2 Healthcare Context 459 17.1.3 Technology Context 460 17.1.4 Overall Problem and Need 460 17.2 Digital Twins and CPHS 461 17.2.1 Digital Twin/Virtual Patient Definition 461 17.2.2 Requirements in an ICU Context 463 17.2.3 Digital Twin Models in Key Areas of ICU Care and Relative to Requirements 464 17.2.4 Review of Digital Twins in Automation of ICU Care 466 17.2.5 Summary 467 17.3 Role of Social-Behavioral Sciences 467 17.3.1 Introduction 467 17.3.2 Barriers to Innovation Adoption 467 17.3.3 Ergonomics and Codesign 468 17.3.4 Summary (Key Takeaways) 469 17.4 Future Research Challenges and Visions 470 17.4.1 Technology Vision of the Future of CPHS in ICU Care 470 17.4.2 Social-Behavioral Sciences Vision of the Future of CPHS in ICU Care 471 17.4.3 Joint Vision of the Future and Challenges to Overcome 473 17.5 Conclusions 473 References 474 Part V Sociotechnical Systems 491 18 Online Attention Dynamics in Social Media 493 Maria Castaldo, Paolo Frasca, and Tommaso Venturini 18.1 Introduction to Attention Economy and Attention Dynamics 493 18.2 Online Attention Dynamics 494 18.2.1 Collective Attention Is Limited 494 18.2.2 Skewed Attention Distribution 495 18.2.3 The Role of Novelty 496 18.2.4 The Role of Popularity 496 18.2.5 Individual Activity Is Bursty 499 18.2.6 Recommendation Systems Are the Main Gateways for Information 500 18.2.7 Change Is the Only Constant 500 18.3 The New Challenge: Understanding Recommendation Systems Effect in Attention Dynamics 501 18.3.1 Model Description 502 18.3.2 Results and Discussion 503 18.4 Conclusion 505 Acknowledgments 505 References 505 19 Cyber–Physical–Social Systems for Smart City 511 Gang Xiong, Noreen Anwar, Peijun Ye, Xiaoyu Chen, Hongxia Zhao, Yisheng Lv, Fenghua Zhu, Hongxin Zhang, Xu Zhou, and Ryan W. Liu 19.1 Introduction 511 19.2 Social Community and Smart Cities 513 19.2.1 Smart Infrastructure 513 19.2.2 Smart Energy 515 19.2.3 Smart Transportation 515 19.2.4 Smart Healthcare 517 19.3 CPSS Concepts, Tools, and Techniques 518 19.3.1 CPSS Concepts 518 19.3.2 CPSS Tools 519 19.3.3 CPSS Techniques 520 19.3.3.1 IoT in Smart Cities 520 19.3.3.2 Big Data in Smart Cities 525 19.4 Recent Research Advances 528 19.4.1 Recent Research Advances of CASIA 528 19.4.2 Recent Research in European Union 531 19.4.3 Future Research Challenges and Visions 533 19.5 Conclusions 537 Acknowledgments 538 References 538 Part VI Concluding Remarks 543 20 Conclusion and Perspectives 545 Anuradha M. Annaswamy, Pramod P. Khargonekar, Françoise Lamnabhi-Lagarrigue, and Sarah K. Spurgeon 20.1 Benefits to Humankind: Synthesis of the Chapters and their Open Directions 545 20.2 Selected Areas for Current and Future Development in CPHS 547 20.2.1 Driver Modeling for the Design of Advanced Driver Assistance Systems 547 20.2.2 Cognitive Cyber–Physical Systems and CPHS 547 20.2.3 Emotion–Cognition Interactions 548 20.3 Ethical and Social Concerns: Few Directions 549 20.3.1 Frameworks for Ethics 550 20.3.2 Technical Approaches 550 20.4 Afterword 551 References 551 Index 555