Trade-off Analytics

List of Contributors xix About the Authors xxi Foreword xxxi Preface xxxiii Acknowledgments xli About the Companion Website xlv 1 Introduction to Trade-off Analysis 1 Gregory S. Parnell, Matthew Cilli, Azad M. Madni and Garry Roedler 1.1 Introduction 2 1.2 Trade-off Analyses Throughout the Life Cycle 3 1.3 Trade-off Analysis to Identify System Value 3 1.4 Trade-off Analysis to Identify System Uncertainties and Risks 6 1.5 Trade-off Analyses can Integrate Value and Risk Analysis 6 1.6 Trade-off Analysis in the Systems Engineering Decision Management Process 8 1.7 Trade-off Analysis Mistakes of Omission and Commission 9 1.7.1 Mistakes of Omission 12 1.7.2 Mistakes of Commission 15 1.7.3 Impacts of the Trade-Off Analysis Mistakes 18 1.8 Overview of the Book 20 1.8.1 Illustrative Examples and Techniques Used in the Book 24 1.9 Key Terms 24 1.10 Exercises 25 References 26 2 A Conceptual Framework and Mathematical Foundation for Trade-Off Analysis 29 Gregory S. Parnell, Azad M. Madni and Robert F. Bordley 2.1 Introduction 29 2.2 Trade-Off Analysis Terms 30 2.3 Influence Diagram of the Tradespace 31 2.3.1 Stakeholder Needs System Functions and Requirements 33 2.3.2 Objectives 33 2.3.3 System Alternatives 34 2.3.4 Uncertainty 36 2.3.5 Preferences and Evaluation of Alternatives 37 2.3.6 Resource Analysis 44 2.3.7 An Integrated Trade-Off Analyses 44 2.4 Tradespace Exploration 46 2.5 Summary 46 2.6 Key Words 47 2.7 Exercises 48 References 48 3 Quantifying Uncertainty 51 Robert F. Bordley 3.1 Sources of Uncertainty in Systems Engineering 51 3.2 The Rules of Probability and Human Intuition 52 3.3 Probability Distributions 56 3.3.1 Calculating Probabilities from Experiments 56 3.3.2 Calculating Complex Probabilities from Simpler Probabilities 58 3.3.3 Calculating Probabilities Using Parametric Distributions 59 3.3.4 Applications of Parametric Probability Distributions 62 3.4 Estimating Probabilities 66 3.4.1 Using Historical Data 66 3.4.2 Using Human Judgment 68 3.4.3 Biases in Judgment 70 3.5 Modeling Using Probability 72 3.5.1 Bayes Nets 72 3.5.2 Monte Carlo Simulation 75 3.5.3 Monte Carlo Simulation with Dependent Uncertainties 76 3.5.4 Monte Carlo Simulation with Partial Information on Output Values 77 3.5.5 Variations on Monte Carlo Simulation 78 3.5.6 Sensitivity Analysis 78 3.6 Summary 81 3.7 Key Terms 81 3.8 Exercises 83 References 86 4 Analyzing Resources 91 Edward A. Pohl, Simon R. Goerger and Kirk Michealson 4.1 Introduction 91 4.2 Resources 92 4.2.1 People 92 4.2.2 Facilities 95 4.2.3 Costs 95 4.2.4 Resource Space 99 4.3 Cost Analysis 99 4.3.1 Cost Estimation 102 4.3.2 Cost Estimation Techniques 108 4.3.3 Learning Curves 120 4.3.4 Net Present Value 125 4.3.5 Monte Carlo Simulation 130 4.3.6 Sensitivity Analysis 134 4.4 Affordability Analysis 135 4.4.1 Background 136 4.4.2 The Basics of Affordability Analysis Are Not Difficult 137 4.4.3 DoD Comparison of Cost Analysis and Affordability Analysis 138 4.4.4 Affordability Analysis Definitions 139 4.4.5 “Big A” Affordability Analysis Process Guide 141 4.5 Key Terms 147 4.6 Excercises 149 References 152 5 Understanding Decision Management 155 Matthew Cilli and Gregory S. Parnell 5.1 Introduction 155 5.2 Decision Process Context 156 5.3 Decision Process Activities 157 5.3.1 Frame Decision 159 5.3.2 Develop Objectives and Measures 163 5.3.3 Generate Creative Alternatives 171 5.3.4 Assess Alternatives via Deterministic Analysis 180 5.3.5 Synthesize Results 183 5.3.6 Develop Multidimensional Value Model 187 5.3.7 Identify Uncertainty and Conduct Probabilistic Analysis 190 5.3.8 Assess Impact of Uncertainty 192 5.3.9 Improve Alternatives 196 5.3.10 Communicating Trade-Offs 197 5.3.11 Present Recommendation and Implementation Plan 197 5.4 Summary 199 5.5 Key Terms 199 5.6 Exercises 200 References 201 6 Identifying Opportunities 203 Donna H. Rhodes and Simon R. Goerger 6.1 Introduction 203 6.2 Knowledge 205 6.2.1 Domain Knowledge 205 6.2.2 Technical Knowledge 205 6.2.3 Business Knowledge 205 6.2.4 Expert Knowledge 206 6.2.5 Stakeholder Knowledge 206 6.3 Decision Traps 207 6.4 Techniques 210 6.4.1 Interviews 210 6.4.2 Focus Groups 213 6.4.3 Surveys 215 6.5 Tools 219 6.5.1 Concept Map 219 6.5.2 System Boundary 220 6.5.3 Decision Hierarchy 220 6.5.4 Issues List 221 6.5.5 Vision Statement 221 6.5.6 Influence Diagram 222 6.5.7 Selecting Appropriate Tools and Techniques 223 6.6 Illustrative Examples 223 6.6.1 Commercial 223 6.6.2 Defense 226 6.7 Key Terms 228 6.8 Exercises 230 References 230 7 Identifying Objectives and Value Measures 233 Gregory S. Parnell and William D. Miller 7.1 Introduction 233 7.2 Value-Focused Thinking 234 7.2.1 Four Major VFT Ideas 235 7.2.2 Benefits of VFT 235 7.3 Shareholder and Stakeholder Value 236 7.3.1 Private Company Example 237 7.3.2 Government Agency Example 237 7.4 Challenges in Identifying Objectives 238 7.5 Identifying the Decision Objectives 239 7.5.1 Questions to Help Identify Decision Objectives 239 7.5.2 How to Get Answers to the Questions 240 7.6 The Financial or Cost Objective 241 7.6.1 Financial Objectives for Private Companies 241 7.6.2 Cost Objective for Public Organizations 242 7.7 Developing Value Measures 243 7.8 Structuring Multiple Objectives 243 7.8.1 Value Hierarchies 244 7.8.2 Techniques for Developing Value Hierarchies 245 7.8.3 Value Hierarchy Best Practices 247 7.8.4 Cautions about Cost and Risk Objectives 248 7.9 Illustrative Examples 248 7.9.1 Military Illustrative Example 248 7.9.2 Homeland Security Illustrative Example 250 7.10 Summary 250 7.11 Key Terms 252 7.12 Exercises 253 References 255 8 Developing and Evaluating Alternatives 257 C. Robert Kenley, Clifford Whitcomb and Gregory S. Parnell 8.1 Introduction 257 8.2 Overview of Decision-making Creativity and Teams 258 8.2.1 Approaches to Decision-Making 258 8.2.2 Cognitive Methods for Creating Alternatives 260 8.2.3 Key Concepts for Building and Operating Teams 260 8.3 Alternative Development Techniques 263 8.3.1 Structured Creativity Methods 263 8.3.2 Morphological Box 266 8.3.3 Pugh Method for Alternative Generation 270 8.3.4 TRIZ for Alternative Development 271 8.4 Assessment of Alternative Development Techniques 275 8.5 Alternative Evaluation Techniques 276 8.5.1 Decision-Theory-Based Approaches 276 8.5.2 Pugh Method for Alternative Evaluation 276 8.5.3 Axiomatic Approach to Design (AAD) 277 8.5.4 TRIZ for Alternative Evaluation 280 8.5.5 Design of Experiments (DOE) 280 8.5.6 Taguchi Approach 282 8.5.7 Quality Function Deployment (QFD) 283 8.5.8 Analytic Hierarchy Process AHP 287 8.6 Assessment of Alternative Evaluation Techniques 290 8.7 Key Terms 290 8.8 Exercises 290 References 293 9 An Integrated Model for Trade-Off Analysis 297 Alexander D. MacCalman, Gregory S. Parnell and Sam Savage 9.1 Introduction 297 9.2 Conceptual Design Example 298 9.3 Integrated Approach Influence Diagram 300 9.3.1 Decision Nodes 300 9.3.2 Uncertainty Nodes 303 9.3.3 Constant Node 310 9.3.4 Value Nodes 314 9.4 Other Types of Trade-Off Analysis 322 9.5 Simulation Tools 322 9.5.1 Monte Carlo Simulation Proprietary Add-Ins 324 9.5.2 The Discipline of Probability Management 324 9.5.3 SIPmathTM Tool in Native Excel 324 9.5.4 Model Building Steps 325 9.6 Summary 329 9.7 Key Terms 330 9.8 Exercises 331 References 335 10 Exploring Concept Trade-Offs 337 Azad M. Madni and Adam M. Ross 10.1 Introduction 337 10.1.1 Key Concepts Concept Trade-Offs and Concept Exploration 341 10.2 Defining the Concept Space and System Concept of Operations 345 10.3 Exploring the Concept Space 346 10.3.1 Storytelling-Enabled Tradespace Exploration 346 10.3.2 Decisions and Outcomes 347 10.3.3 Contingent Decision-Making 347 10.4 Trade-off Analysis Frameworks 348 10.5 Tradespace and System Design Life Cycle 349 10.6 From Point Trade-offs to Tradespace Exploration 351 10.7 Value-based Multiattribute Tradespace Analysis 351 10.7.1 Tradespace Exploration and Sensitivity Analysis 353 10.7.2 Tradespace Exploration and Uncertainty 354 10.7.3 Tradespace Exploration with Spiral Development 356 10.7.4 Tradespace Exploration in Relation to Optimization and Decision Theory 356 10.8 Illustrative Example 359 10.8.1 Step 1: Determine Key Decision-Makers 359 10.8.2 Step 2: Scope and Bound the Mission 360 10.8.3 Step 3: Elicit Attributes and Utilities (Preference Capture) 360 10.8.4 Step 4: Define Design Vector Elements (Concept Generation) 362 10.8.5 Step 5: Develop Model(s) (Evaluation) 362 10.8.6 Step 6: Generate the Tradespace (Computation) 364 10.8.7 Step 7: Explore the Tradespace (Analysis and Synthesis) 365 10.9 Conclusions 369 10.10 Key Terms 371 10.11 Exercises 372 References 372 11 Architecture Evaluation Framework 377 James N. Martin 11.1 Introduction 377 11.1.1 Architecture in the Decision Space 378 11.1.2 Architecture Evaluation 379 11.1.3 Architecture Views and Viewpoints 380 11.1.4 Stakeholders 382 11.1.5 Stakeholder Concerns 382 11.1.6 Architecture versus Design 383 11.1.7 On the Uses of Architecture 384 11.1.8 Standardizing on an Architecture Evaluation Strategy 384 11.2 Key Considerations in Evaluating Architectures 385 11.2.1 Plan-Driven Evaluation Effort 386 11.2.2 Objectives-Driven Evaluation 387 11.2.3 Assessment versus Analysis 387 11.3 Architecture Evaluation Elements 389 11.3.1 Architecture Evaluation Approach 389 11.3.2 Architecture Evaluation Objectives 390 11.3.3 Evaluation Approach Examples 391 11.3.4 Value Assessment Methods 391 11.3.5 Value Assessment Criteria 393 11.3.6 Architecture Analysis Methods 394 11.4 Steps in an Architecture Evaluation Process 396 11.5 Example Evaluation Taxonomy 398 11.5.1 Business Impact Factors 398 11.5.2 Mission Impact Factors 398 11.5.3 Architecture Attributes 399 11.6 Summary 400 11.7 Key Terms 400 11.8 Exercises 402 References 402 12 Exploring the Design Space 405 Clifford Whitcomb and Paul Beery 12.1 Introduction 405 12.2 Example 1: Liftboat 406 12.2.1 Liftboat Fractional Factorial Design of Experiments 406 12.2.2 Liftboat Design Trade-Off Space 409 12.2.3 Liftboat Uncertainty Analysis 411 12.2.4 Liftboat Example Summary 411 12.3 Example 2: Cruise Ship Design 411 12.3.1 Cruise Ship Taguchi Design of Experiments 411 12.3.2 Cruise Ship Design Trade-Off Space 412 12.3.3 Cruise Ship Example Summary 416 12.4 Example 3: NATO Naval Surface Combatant Ship 417 12.4.1 NATO Surface Combatant Ship Stakeholder Need 418 12.4.2 NATO Surface Combatant Ship Box–Behnken Design of Experiments 420 12.4.3 NATO Surface Combatant Ship Cost-Effectiveness Trade-Off 421 12.4.4 NATO Surface Combatant Ship Design Tradespace 421 12.4.5 NATO Surface Combatant Ship Design Trade-Off 422 12.4.6 NATO Surface Combatant Ship Trade-Off Summary 430 12.5 Key Terms 431 12.6 Exercises 433 References 435 13 Sustainment Related Models and Trade Studies 437 John E. MacCarthy and Andres Vargas 13.1 Introduction 437 13.2 Availability Modeling and Trade Studies 439 13.2.1 FMDS Background 439 13.2.2 FMDS Availability Trade Studies 449 13.2.3 Section Synopsis 453 13.3 Sustainment Life Cycle Cost Modeling and Trade Studies14 454 13.3.1 The Total System Life Cycle Model 454 13.3.2 The O&S Cost Model 456 13.3.3 Life Cycle Cost Trade Study 459 13.4 Optimization in Availability Trade Studies 464 13.4.1 Setting Up the Optimization Problem 464 13.4.2 Instantiating the Optimization Model 465 13.4.3 Discussion of the Optimization Model Results 468 13.4.4 Deterministic Sensitivity Analysis 469 13.5 Monte Carlo Modeling 471 13.5.1 Input Probability Distributions for the Monte Carlo Model 471 13.5.2 Monte Carlo Simulation Results 472 13.5.3 Stochastic Sensitivity Analysis 473 13.6 Chapter Summary 475 13.7 Key Terms 476 13.8 Exercises 478 References 482 14 Performing Programmatic Trade-Off Analyses 483 Gina Guillaume-Joseph and John E. MacCarthy 14.1 Introduction 483 14.2 System Acceptance Decisions and Trade Studies 485 14.2.1 Acceptance Decision Framework 486 14.2.2 Calculating the Confidence That a System Is “Good” 491 14.2.3 Acceptance Test Design and Trade Studies 493 14.2.4 A “Delay Fix and Test” Cost Model 499 14.2.5 The Integrated Decision Model 504 14.2.6 Conclusions 511 14.3 Product Cancelation Decision Trade Study 512 14.3.1 Introduction 512 14.3.2 Significance 513 14.3.3 Defining Failure 514 14.3.4 Developing the Predictive Model 519 14.3.5 Research Results 522 14.3.6 Model Implementation In Industry 528 14.3.7 Predictive Model Deployment in Industry 530 14.3.8 When the Decision Has Been Made to Cancel the System 536 14.3.9 Conclusion 537 14.4 Product Retirement Decision Trade Study 538 14.4.1 Introduction 538 14.4.2 Legacy HR Systems 539 14.4.3 The US NAVY Retirement and Decommission Program for Nuclear-Powered Vessels 544 14.4.4 Decision Analysis for Decommissioning Offshore Oil and Gas Platforms in California 551 14.4.5 System Retirement and Decommissioning Strategy 559 14.4.6 Conclusion 561 14.5 Key Terms 562 14.6 Exercises 564 References 566 15 Summary and Future Trends 571 Gregory S. Parnell and Simon R. Goerger 15.1 Introduction 571 15.2 Major Trade-Off Analysis Themes 572 15.2.1 Use Standard Systems Engineering Terminology 572 15.2.2 Avoid the Mistakes of Omission and Commission 572 15.2.3 Use a Decision Management Framework 572 15.2.4 Use Decision Analysis as the Mathematical Foundation 573 15.2.5 Explicitly Define the Decision Opportunity 573 15.2.6 Identify and Structure Decision Objectives and Measures 574 15.2.7 Identify Creative Doable Alternatives 574 15.2.8 Use the Most Appropriate Modeling and Simulation Technique for the Life Cycle Stage 575 15.2.9 Include Resource Analysis in the Trade-Off Analysis 575 15.2.10 Explicitly Consider Uncertainty 575 15.2.11 Identify the Cost Value Schedule and Risk Drivers 575 15.2.12 Provide an Integrated Framework for Cost Value and Risk Analyses 576 15.3 Future of Trade-Off Analysis 576 15.3.1 Education and Training of Systems Engineers 577 15.3.2 Systems Engineering Methodologies and Tools 577 15.3.3 Emergent Tradespace Factors 580 15.4 Summary 581 References 581 Index 583