Machine Learning in Production

I SETTING THE STAGE 2 1 Introduction 3 1.1 Motivating Example: An Automated Transcription Startup 4 1.2 Data Scientists and Software Engineers 6 1.3 Machine-Learning Challenges in Software Projects 8 1.4 A Foundation for MLOps and Responsible Engineering 13 1.5 Summary 15 1.6 Further Readings 16 2 From Models to Systems 19 2.1 ML and Non-ML Components in a System 19 2.2 Beyond the Model 24 2.3 On Terminology 29 2.4 Summary 30 2.5 Further Readings 30 3 Machine Learning for Software Engineers, in a Nutshell 33 3.1 Basic Terms: Machine Learning, Models, Predictions 33 3.2 Technical Concepts: Model Parameters, Hyperparameters, Model Storage 34 3.3 Machine Learning Pipelines 35 3.4 Foundation Models and Prompting 36 3.5 On Terminology 37 3.6 Summary 38 3.7 Further Readings 38 II REQUIREMENTS ENGINEERING 39 4 When to use Machine Learning 41 4.1 Problems that Benefit from Machine Learning 41 4.2 Tolerating Mistakes and ML Risk 42 4.3 Continuous Learning 43 4.4 Costs and Benefits 43 4.5 The Business Case: Machine Learning as Predictions 44 4.6 Summary 45 4.7 Further Readings 45 5 Setting and Measuring Goals 47 5.1 Scenario: Self-help legal chatbot 47 5.2 Setting Goals 48 5.3 Measurement in a Nutshell 51 5.4 Summary 57 5.5 Further Readings 57 6 Gathering Requirements 59 6.1 Scenario: Fall Detection with a Smart Watch 60 6.2 Untangling Requirements 60 6.3 Eliciting Requirements 66 6.4 How Much Requirements Engineering and When? 71 6.5 Summary 72 6.6 Further Readings 73 7 Planning for Mistakes 75 7.1 Mistakes Will Happen 76 7.2 Designing for Failures 78 7.3 Hazard Analysis and Risk Analysis 84 7.4 Summary 89 7.5 Further Readings 90 III ARCHITECTURE AND DESIGN 92 8 Thinking like a Software Architect 93 8.1 Quality Requirements Drive Architecture Design 94 8.2 The Role of Abstraction 97 8.3 Common Architectural Design Challenges for ML-Enabled Systems 97 8.4 Codifying Design Knowledge 100 8.5 Summary 105 8.6 Further Readings 105 9 Quality Attributes of ML Components 109 9.1 Scenario: Detecting Credit Card Fraud 109 9.2 From System Quality to Model and Pipeline Quality 109 9.3 Common Quality Attributes 111 9.4 Constraints and Tradeoffs 115 9.5 Summary 117 9.6 Further Readings 118 10 Deploying a Model 119 10.1 Scenario: Augmented Reality Translation 119 10.2 Model Inference Function 120 10.3 Feature Encoding 120 10.4 Model Serving Infrastructure 123 10.5 Deployment Architecture Tradeoffs 126 10.6 Model Inference in a System 131 10.7 Documenting Model-Inference Interfaces 135 10.8 Summary 136 10.9 Further Readings 138 11 Automating the Pipeline 141 11.1 Scenario: Home Value Prediction 141 11.2 Supporting Evolution and Experimentation by Designing for Change 142 11.3 Pipeline Thinking 143 11.4 Stages of Machine-Learning Pipelines 144 11.5 Automation and Infrastructure Design 149 11.6 Summary 151 11.7 Further Readings 152 12 Scaling the System 155 12.1 Scenario: Google-Scale Photo Hosting and Search 155 12.2 Scaling by Distributing Work 156 12.3 Data Storage at Scale 157 12.4 Distributed Data Processing 166 12.5 Distributed Machine-Learning Algorithms 176 12.6 Performance Planning and Monitoring 178 12.7 Summary 178 12.8 Further Readings 179 13 Planning for Operations 181 13.1 Scenario: Blogging Platform with Spam Filter 182 13.2 Service Level Objectives 182 13.3 Observability 183 13.4 Automating Deployments 185 13.5 Infrastructure as Code and Virtualization 186 13.6 Orchestrating and Scaling Deployments 188 13.7 Elevating Data Engineering 189 13.8 Incident Response Planning 190 13.9 DevOps and MLOps Principles 191 13.10 DevOps and MLOps Tooling 192 13.11 Summary 195 13.12 Further Readings 195 IV QUALITY ASSURANCE 197 14 Quality Assurance Basics 199 14.1 Testing 200 14.2 Code Review 204 14.3 Static Analysis 205 14.4 Other Quality Assurance Approaches 206 14.5 Planning and Process Integration 207 14.6 Summary 209 14.7 Further Readings 209 15 Model Quality 211 15.1 Scenario: Cancer Prognosis 211 15.2 Defining Correctness and Fit 212 15.3 Measuring Prediction Accuracy 217 15.4 Model Evaluation Beyond Accuracy 231 15.5 Test Data Adequacy 244 15.6 Model Inspection 245 15.7 Summary 245 15.8 Further Readings 246 16 Data Quality 251 16.1 Scenario: Inventory Management 251 16.2 Data Quality Challenges 252 16.3 Data Quality Checks 255 16.4 Drift and Data Quality Monitoring 260 16.5 Data Quality is a System-Wide Concern 264 16.6 Summary 268 16.7 Further Readings 269 17 Pipeline Quality 273 17.1 Silent Mistakes in ML Pipelines 273 17.2 Code Review for ML Pipelines 274 17.3 Testing Pipeline Components 275 17.4 Static Analysis of ML Pipelines 284 17.5 Process Integration and Test Maturity 284 17.6 Summary 285 17.7 Further Readings 286 18 System Quality 287 18.1 Limits of Modular Reasoning 287 18.2 System Testing 289 18.3 Testing Component Interactions and Safeguards 291 18.4 Testing Operations (Deployment, Monitoring) 293 18.5 Summary 293 18.6 Further Readings 294 19 Testing and Experimenting in Production 295 19.1 A Brief History of Testing in Production 295 19.2 Scenario: Meeting Minutes for Video Calls 297 19.3 Measuring System Success in Production 297 19.4 Measuring Model Quality in Production 298 19.5 Designing and Implementing Quality Measures with Telemetry 302 19.6 Experimenting in Production 306 19.7 Summary 311 19.8 Further Readings 312 V PROCESS AND TEAMS 314 20 Data Science and Software Engineering Process Models 315 20.1 Data-Science Process 315 20.2 Software-Engineering Process 318 20.3 Tensions between Data Science and Software Engineering Processes 321 20.4 Integrated Processes for AI-Enabled Systems 323 20.5 Summary 327 20.6 Further Readings 327 21 Interdisciplinary Teams 329 21.1 Scenario: Fighting Depression on Social Media 329 21.2 Unicorns are not Enough 330 21.3 Conflicts Within and Between Teams are Common 331 21.4 Coordination Costs 332 21.5 Conflicting Goals and T-Shaped People 337 21.6 Groupthink 339 21.7 Team Structure and Allocating Experts 340 21.8 Learning from DevOps and MLOps Culture 342 21.9 Summary 345 21.10 Further Readings 346 22 Technical Debt 349 22.1 Scenario: Automated Delivery Robots 349 22.2 Deliberate and Prudent Technical Debt 349 22.3 Technical Debt in Machine Learning Projects 351 22.4 Managing Technical Debt 353 22.5 Summary 354 22.6 Further Readings 355 VI RESPONSIBLE ML ENGINEERING 356 23 Responsible Engineering 357 23.1 Legal and Ethical Responsibilities 357 23.2 Why Responsible Engineering Matters for ML-Enabled Systems 359 23.3 Facets of Responsible ML Engineering 362 23.4 Regulation is Coming 363 23.5 Summary 366 23.6 Further Readings 366 24 Versioning, Provenance, and Reproducibility 369 24.1 Scenario: Debugging a Loan Decision 370 24.2 Versioning 370 24.3 Data Provenance and Lineage 375 24.4 Reproducibility 378 24.5 Putting the Pieces Together 380 24.6 Summary 381 24.7 Further Readings 382 25 Explainability 385 25.1 Scenario: Proprietary Opaque Models for Recidivism Risk Assessment 385 25.2 Defining Explainability 386 25.3 Explaining a Model 389 25.4 Explaining a Prediction 392 25.5 Explaining Data and Training 397 25.6 The Dark Side of Explanations 397 25.7 Summary 398 25.8 Further Readings 398 26 Fairness 401 26.1 Scenario: Mortgage Applications 402 26.2 Fairness Concepts 403 26.3 Measuring and Improving Fairness at the Model Level 410 26.4 Fairness is a System-Wide Concern 416 26.5 Summary 428 26.6 Further Readings 429 27 Safety 433 27.1 Safety and Reliability 433 27.2 Improving Model Reliability 434 27.3 Building Safer Systems 438 27.4 The AI Alignment Problem 442 27.5 Summary 444 27.6 Further Readings 444 28 Security and Privacy 447 28.1 Scenario: Content Moderation 447 28.2 Security Requirements 448 28.3 Attacks and Defenses 449 28.4 ML-Specific Attacks 450 28.5 Threat Modeling 459 28.6 Designing for Security 462 28.7 Data Privacy 466 28.8 Summary 470 28.9 Further Readings 470 29 Transparency and Accountability 473 29.1 Transparency of the Model’s Existence 473 29.2 Transparency of How the Model Works 474 29.3 Human Oversight and Appeals 477 29.4 Accountability and Culpability 478 29.5 Summary 479 29.6 Further Readings 479