Designing Machine Learning Systems: An Iterative Process for Production-Ready Applications

1. Machine Learning Systems in Production....5 When to Use Machine Learning....9 Machine Learning Use Cases....15 Understanding Machine Learning Systems....19 Mind vs. Data....19 Machine learning in research vs. in production....25 Machine learning systems vs. traditional software....39 Designing ML Systems in Production....42 Requirements for ML Systems....43 Iterative Process....46 Summary....50 2. Data Engineering Fundamentals....54 Data Sources....55 Data Formats....57 JSON....59 Row-major vs. Column-major Format....60 Text vs. Binary Format....64 Data Models....66 Relational Model....66 NoSQL....75 Structured vs. Unstructured Data....80 Data Storage Engines and Processing....83 Transactional and Analytical Processing....84 ETL: Extract, Transform, and Load....87 Modes of Dataflow....88 Data Passing Through Databases....88 Data Passing Through Services....89 Data Passing Through Real-time Transport....90 Batch Processing vs. Stream Processing....94 Summary....96 3. Training Data....99 Sampling....99 Non-Probability Sampling....100 Simple Random Sampling....101 Stratified Sampling....101 Weighted Sampling....101 Importance Sampling....102 Reservoir Sampling....102 Labeling....104 Hand Labels....105 Handling the Lack of Hand Labels....108 Class Imbalance....118 Challenges of Class Imbalance....119 Handling Class Imbalance....121 Data Augmentation....134 Simple Label-Preserving Transformations....135 Perturbation....138 Data Synthesis....139 Summary....141 4. Feature Engineering....143 Learned Features vs. Engineered Features....143 Common Feature Engineering Operations....147 Handling Missing Values....148 Scaling....152 Discretization....154 Encoding Categorical Features....155 Feature Crossing....157 Discrete and Continuous Positional Embeddings....159 Data Leakage....162 Common Causes for Data Leakage....163 Detecting Data Leakage....165 Engineering Good Features....166 Feature Importance....166 Feature Generalization....169 Summary....170 5. Model Development....172 Framing ML Problems....173 Types of ML Tasks....173 Objective Functions....181 Model Development and Training....184 Evaluating ML Models....184 Ensembles....193 Experiment Tracking and Versioning....205 Distributed Training....213 AutoML....219 Model Offline Evaluation....229 Baselines....230 Evaluation Methods....237 Summary....258 6. Model Deployment....261 Machine Learning Deployment Myths....265 Batch Prediction vs. Online Prediction....271 From Batch Prediction To Online Prediction....279 Unifying Batch Pipeline And Streaming Pipeline....282 Model Compression....285 Low-rank Factorization....286 Knowledge Distillation....287 Pruning....287 Quantization....288 ML on the Cloud and on the Edge....293 Compiling and Optimizing Models for Edge Devices....299 ML in Browsers....312 Summary....313 7. Why Machine Learning Systems Fail in Production....317 Natural Labels and Feedback Loop....317 Causes of ML System Failures....319 Production Data Differing From Training Data....320 Edge Cases....321 Degenerate Feedback Loop....323 Data Distribution Shifts....328 Types of Data Distribution Shifts....328 General Data Distribution Shifts....330 Handling Data Distribution Shifts....331 Summary....336 About the Author....339