Software Quality Assurance

Preface xv Acknowledgments xxiii 1. Software Quality Fundamentals 1 1.1 Introduction 1 1.2 Defining Software Quality 2 1.3 Software Errors, Defects, and Failures 4 1.3.1 Problems with Defining Requirements 10 1.3.2 Maintaining Effective Communications Between Client and Developer 13 1.3.3 Deviations from Specifications 14 1.3.4 Architecture and Design Errors 15 1.3.5 Coding Errors 15 1.3.6 Non-Compliance with Current Processes/Procedures 16 1.3.7 Inadequate Reviews and Tests 17 1.3.8 Documentation Errors 17 1.4 Software Quality 19 1.5 Software Quality Assurance 20 1.6 Business Models and the Choice of Software Engineering Practices 22 1.6.1 Description of the Context 23 1.6.2 Anxiety and Fear 24 1.6.3 Choice of Software Practices 25 1.6.4 Business Model Descriptions 25 1.6.5 Description of Generic Situational Factors 26 1.6.6 Detailed Description of Each Business Model 27 1.7 Success Factors 32 1.8 Further Reading 33 1.9 Exercises 34 2. Quality Culture 35 2.1 Introduction 35 2.2 Cost of Quality 39 2.3 Quality Culture 49 2.4 The Five Dimensions of a Software Project 53 2.5 The Software Engineering Code of Ethics 56 2.5.1 Abridged Version: Preamble 58 2.5.2 The Example of the Code of Ethics of the Ordre des ing'enieurs du Qu'ebec 60 2.5.3 Whistle Blowers 61 2.6 Success Factors 62 2.7 Further Reading 63 2.8 Exercises 63 3. Software Quality Requirements 66 3.1 Introduction 66 3.2 Software Quality Models 69 3.2.1 Initial Model Proposed by McCall 71 3.2.2 The First Standardized Model: IEEE 1061 73 3.2.3 Current Standardized Model: ISO 25000 Set of Standards 77 3.3 Definition of Software Quality Requirements 86 3.3.1 Specifying Quality Requirements: The Process 91 3.4 Requirement Traceability During the Software Life Cycle 95 3.5 Software Quality Requirements and the Software Quality Plan 95 3.6 Success Factors 96 3.7 Further Reading 97 3.8 Exercises 97 4. Software Engineering Standards and Models 101 4.1 Introduction 101 4.2 Standards, Cost of Quality, and Business Models 108 4.3 Main Standards for Quality Management 109 4.3.1 ISO 9000 Family 109 4.3.2 ISO/IEC 90003 Standard 115 4.4 ISO/IEC/IEEE 12207 Standard 117 4.4.1 Limitations of the ISO 12207 Standard 121 4.5 ISO/IEC/IEEE 15289 Standard for the Description of Information Elements 121 4.6 IEEE 730 Standard for SQA Processes 123 4.6.1 Activities and Tasks of SQA 125 4.7 Other Quality Models, Standards, References, and Processes 129 4.7.1 Process Maturity Models of the SEI 130 4.7.2 Software Maintenance Maturity Model (S3m) 135 4.7.3 ITIL Framework and ISO/IEC 20000 138 4.7.4 CobiT Process 142 4.7.5 ISO/IEC 27000 Family of Standards for Information Security 143 4.7.6 ISO/IEC 29110 Standards and Guides for Very Small Entities 144 4.7.7 ISO/IEC 29110 Standards for VSEs Developing Systems 155 4.8 Specific Standards for an Application Domain 156 4.8.1 DO-178 and ED-12 Guidance for Airborne Systems 156 4.8.2 EN 50128 Standard for Railway Applications 159 4.8.3 ISO 13485 Standard for Medical Devices 161 4.9 Standards and the SQAP 163 4.10 Success Factors 165 4.11 Further Reading 165 4.12 Exercises 166 5. Reviews 167 5.1 Introduction 167 5.2 Personal Review and Desk-Check Review 172 5.2.1 Personal Review 172 5.2.2 Desk-Check Reviews 175 5.3 Standards and Models 179 5.3.1 ISO/IEC 20246 Software and Systems Engineering: Work Product Reviews 179 5.3.2 Capability Maturity Model Integration 180 5.3.3 The IEEE 1028 Standard 181 5.4 Walk-Through 184 5.4.1 Usefulness of a Walk-Through 184 5.4.2 Identification of Roles and Responsibilities 186 5.5 Inspection Review 187 5.6 Project Launch Reviews and Project Assessments 189 5.6.1 Project Launch Review 190 5.6.2 Project Retrospectives 192 5.7 Agile Meetings 197 5.8 Measures 199 5.9 Selecting the Type of Review 202 5.10 Reviews and Business Models 205 5.11 Software Quality Assurance Plan 205 5.12 Success Factors 206 5.13 Tools 208 5.14 Further Reading 208 5.15 Exercises 208 6. Software Audits 210 6.1 Introduction 210 6.2 Types of Audits 215 6.2.1 Internal Audit 215 6.2.2 Second-Party Audit 215 6.2.3 Third-Party Audit 217 6.3 Audit and Software Problem Resolution According to ISO/IEC/IEEE 12207 217 6.3.1 Project Assessment and Control Process 218 6.3.2 Decision Management Process 218 6.4 Audit According to the IEEE 1028 Standard 218 6.4.1 Roles and Responsibilities 220 6.4.2 IEEE 1028 Audit Clause 221 6.4.3 Audit Conducted According to IEEE 1028 222 6.5 Audit Process and the ISO 9001 Standard 225 6.5.1 Steps of a Software Audit 226 6.6 Audit According to the CMMI 230 6.6.1 SCAMPI Assessment Method 231 6.7 Corrective Actions 233 6.7.1 Corrective Actions Process 234 6.8 Audits for Very Small Entities 238 6.9 Audit and the SQA Plan 239 6.10 Presentation of an Audit Case Study 241 6.11 Success Factors 246 6.12 Further Reading 247 6.13 Exercises 247 7. Verification and Validation 249 7.1 Introduction 249 7.2 Benefits and Costs of V&V 255 7.2.1 V&V and the Business Models 257 7.3 V&V Standards and Process Models 257 7.3.1 IEEE 1012 V&V Standard 258 7.3.2 Integrity Levels 260 7.3.3 Recommended V&V Activities for Software Requirements 262 7.4 V&V According to ISO/IEC/IEEE 12207 263 7.4.1 Verification Process 265 7.4.2 Validation Process 265 7.5 V&V According to the CMMI Model 266 7.6 ISO/IEC 29110 and V&V 267 7.7 Independent V&V 268 7.7.1 IV&V Advantages with Regards to SQA 271 7.8 Traceability 271 7.8.1 Traceability Matrix 273 7.8.2 Implementing Traceability 276 7.9 Validation Phase of Software Development 277 7.9.1 Validation Plan 279 7.10 Tests 281 7.11 Checklists 282 7.11.1 How to Develop a Checklist 283 7.11.2 How to Use a Checklist 285 7.11.3 How to Improve and Manage a Checklist 286 7.12 V&V Techniques 287 7.12.1 Introduction to V&V Techniques 287 7.12.2 Some V&V Techniques 288 7.13 V&V Plan 289 7.14 Limitations of V&V 290 7.15 V&V in the SQA Plan 291 7.16 Success Factors 292 7.17 Further Reading 293 7.18 Exercises 293 8. Software Configuration Management 295 8.1 Introduction 295 8.2 Software Configuration Management 296 8.3 Benefits of Good Configuration Management 297 8.3.1 CM According to ISO 12207 298 8.3.2 CM According to IEEE 828 299 8.3.3 CM According to the CMMI 299 8.4 SCM Activities 301 8.4.1 Organizational Context of SCM 301 8.4.2 Developing a SCM Plan 302 8.4.3 Identification of CI to be Controlled 303 8.5 Baselines 309 8.6 Software Repository and Its Branches 311 8.6.1 A Simple Branching Strategy 315 8.6.2 A Typical Branching Strategy 316 8.7 Configuration Control 318 8.7.1 Requests, Evaluation, and Approval of Changes 319 8.7.2 Configuration Control Board 321 8.7.3 Request for Waivers 322 8.7.4 Change Management Policy 322 8.8 Configuration Status Accounting 323 8.8.1 Information Concerning the Status of CI 323 8.8.2 Configuration Item Status Reporting 325 8.9 Software Configuration Audit 325 8.9.1 Functional Configuration Audit 327 8.9.2 Physical Configuration Audit 327 8.9.3 Audits Performed During a Project 328 8.10 Implementing SCM in Very Small Entities with ISO/IEC 29110 329 8.11 SCM and the SQAP 330 8.12 Success Factors 331 8.13 Further Reading 333 8.14 Exercises 333 9. Policies, Processes, and Procedures 335 9.1 Introduction 335 9.1.1 Standards, theCost ofQuality, and Business Models 341 9.2 Policies 341 9.3 Processes 345 9.4 Procedures 351 9.5 Organizational Standards 352 9.6 Graphical Representation of Processes and Procedures 353 9.6.1 Some Pitfalls to Avoid 356 9.6.2 Process Mapping 357 9.6.3 ETVX Process Notation 357 9.6.4 IDEF Notation 366 9.6.5 BPMN Notation 370 9.7 Process Notation of ISO/IEC 29110 376 9.8 Case Study 383 9.9 Personal Improvement Process 388 9.10 Policies, Processes, and Procedures in the SQA Plan 393 9.11 Success Factors 394 9.12 Further Reading 395 9.13 Exercises 396 10. Measurement 397 10.1 Introduction-the Importance of Measurement 397 10.1.1 Standards, the Cost of Quality, and Software Business Models 401 10.2 Software Measurement According to ISO/IEC/IEEE 12207 402 10.3 Measurement According to ISO 9001 403 10.4 The Practical Software and Systems Measurement Method 404 10.5 ISO/IEC/IEEE 15939 Standard 411 10.5.1 Measurement Process According to ISO 15939 412 10.5.2 Activities and Tasks of the Measurement Process 412 10.5.3 An Information Measurement Model of ISO 15939 412 10.6 Measurement According to the CMMI Model 418 10.7 Measurement in Very Small Entities 421 10.8 The Survey as a Measurement Tool 421 10.9 Implementing a Measurement Program 425 10.9.1 Step 1: Management Commitment Build-Up 426 10.9.2 Step 2: Staff Commitment Build-Up 427 10.9.3 Step 3: Selection of Key Processes to be Improved 427 10.9.4 Step 4: Identification of the Goals and Objectives Related to the Key Process 427 10.9.5 Step 5: Design of the Measurement Program 427 10.9.6 Step 6: Description of the Information System to Support Measurement 428 10.9.7 Step 7: Deployment of the Measurement Program 428 10.10 Practical Considerations 430 10.10.1 Some Pitfalls with Regards to Measurement 432 10.11 The Human Side of Measurement 435 10.11.1 Cost of Measurement 438 10.12 Measurement and the IEEE 730 SQAP 439 10.12.1 Software Process Measurement 440 10.12.2 Software Product Measurement 441 10.13 Success Factors 443 10.14 Further Reading 443 10.15 Exercises 444 11. Risk Management 445 11.1 Introduction 445 11.1.1 Risk, the Cost of Quality and Business Models 451 11.1.2 Costs and Benefits of Risk Management 453 11.2 Risk Management According to Standards and Models 454 11.2.1 Risk Management According to ISO 9001 454 11.2.2 Risk Management According to ISO/IEC/IEEE 12207 455 11.2.3 Risk Management According to ISO/IEC/IEEE 16085 456 11.2.4 Risk Management According to the CMMI Model 459 11.2.5 Risk Management According to PMBOK (R) Guide 461 11.2.6 Risk Management According to ISO 29110 462 11.2.7 Risk Management and the SQA According to IEEE 730 465 11.3 Practical Considerations for Risk Management 466 11.3.1 Risk Evaluation Step 468 11.3.2 Risk Control Step 474 11.3.3 Lessons Learned Activity 477 11.4 Risk Management Roles 478 11.5 Measurement and Risk Management 479 11.6 Human Factors and Risk Management 483 11.7 Success Factors 485 11.8 Conclusion 486 11.9 Further Reading 487 11.10 Exercises 487 12. Supplier Management and Agreements 489 12.1 Introduction 489 12.2 Supplier Requirements of ISO 9001 490 12.3 Agreement Processes of ISO 12207 491 12.4 Supplier Agreement Management According to the CMMI 494 12.5 Managing Suppliers 496 12.6 Software Acquisition Life Cycle 497 12.7 Software Contract Types 499 12.7.1 Fixed Price Contract 501 12.7.2 Cost plus Percentage of Cost 502 12.7.3 Cost plus Fixed Fee 502 12.7.4 Risk Sharing 502 12.8 Software Contract Reviews 505 12.8.1 Two Reviews: Initial and Final 505 12.8.2 Initial Contract Review 506 12.8.3 Final Contract Review 509 12.9 Supplier and Acquirer Relationship and the SQAP 510 12.10 Success Factors 511 12.11 Further Reading 512 12.12 Exercises 512 13. Software Quality Assurance Plan 514 13.1 Introduction 514 13.2 SQA Planning 518 13.2.1 Purpose and Scope 518 13.2.2 Definitions and Acronyms 518 13.2.3 Reference Documents 519 13.2.4 SQAP Overview-Organization and Independence 520 13.2.5 SQAP Overview-Software Product Risk 524 13.2.6 SQAP Overview-Tools 525 13.2.7 SQAP Overview-Standards, Practices, and Conventions 525 13.2.8 SQAP Overview-Effort, Resources, and Schedule 526 13.2.9 Activities, Outcomes, and Tasks-Product Assurance 528 13.2.10 Activities, Outcomes, and Tasks-Process Assurance 529 13.2.11 Additional Considerations 531 13.2.12 SQA Records 536 13.3 Executing the SQAP 537 13.4 Conclusion 539 13.5 Further Reading 539 13.6 Exercises 540 Appendix 1. Software Engineering Code of Ethics and Professional Practice (Version 5.2) 541 Appendix 2. Incidents and Horror Stories Involving Software 549 Glossary - Abbreviations - Acronyms 555 References 576 Index 591