Machine Learning Model Serving Patterns and Best Practices: A definitive guide to deploying, monitoring, and providing accessibility to ML models in production

Cover Page....2 Table of Contents....3 Preface....5 Part 1:Introduction to Model Serving....13 Chapter 1: Introducing Model Serving....14 Technical requirements....14 What is serving?....15 What are models?....17 What is model serving?....19 Understanding the importance of model serving....24 Using existing tools to serve models....26 Summary....27 Chapter 2: Introducing Model Serving Patterns....29 Design patterns in software engineering....29 Understanding the value of model serving patterns....33 ML serving patterns....37 Summary....51 Further reading....52 Part 2:Patterns and Best Practices of Model Serving....53 Chapter 3: Stateless Model Serving....54 Technical requirements....54 Understanding stateful and stateless functions....55 States in machine learning models....62 Summary....97 Chapter 4: Continuous Model Evaluation....99 Technical requirements....99 Introducing continuous model evaluation....100 The necessity of continuous model evaluation....105 Continuous model evaluation use cases....124 Evaluating a model continuously....130 Monitoring model performance when predicting rare classes....137 Summary....139 Further reading....140 Chapter 5: Keyed Prediction....141 Technical requirements....141 Introducing keyed prediction....142 Exploring keyed prediction use cases....145 Exploring techniques for keyed prediction....162 Summary....174 Further reading....174 Chapter 6: Batch Model Serving....175 Technical requirements....175 Introducing batch model serving....176 Different types of batch model serving....179 Example scenarios of batch model serving....191 Techniques in batch model serving....193 Limitations of batch serving....199 Summary....201 Further reading....201 Chapter 7: Online Learning Model Serving....202 Technical requirements....202 Introducing online model serving....202 Use cases for online model serving....215 Challenges in online model serving....219 Implementing online model serving....225 Summary....229 Further reading....229 Chapter 8: Two-Phase Model Serving....231 Technical requirements....231 Introducing two-phase model serving....232 Exploring two-phase model serving techniques....234 Use cases of two-phase model serving....249 Summary....254 Further reading....254 Chapter 9: Pipeline Pattern Model Serving....256 Technical requirements....256 Introducing the pipeline pattern....257 Introducing Apache Airflow....260 Demonstrating a machine learning pipeline using Airflow....271 Advantages and disadvantages of the pipeline pattern....276 Summary....277 Further reading....278 Chapter 10: Ensemble Model Serving Pattern....279 Technical requirements....279 Introducing the ensemble pattern....280 Using ensemble pattern techniques....282 End-to-end dummy example of serving the model....291 Summary....293 Chapter 11: Business Logic Pattern....295 Technical requirements....295 Introducing the business logic pattern....295 Technical approaches to business logic in model serving....299 Summary....304 Part 3:Introduction to Tools for Model Serving....306 Chapter 12: Exploring TensorFlow Serving....307 Technical requirements....307 Introducing TensorFlow Serving....308 Using TensorFlow Serving to serve models....312 Summary....323 Further reading....324 Chapter 13: Using Ray Serve....325 Technical requirements....325 Introducing Ray Serve....325 Using Ray Serve to serve a model....335 Summary....345 Further reading....346 Chapter 14: Using BentoML....347 Technical requirements....347 Introducing BentoML....347 Using BentoML to serve a model....362 Summary....366 Further reading....366 Part 4:Exploring Cloud Solutions....367 Chapter 15: Serving ML Models using a Fully Managed AWS Sagemaker Cloud Solution....368 Technical requirements....368 Introducing Amazon SageMaker....368 Using Amazon SageMaker to serve a model....373 Summary....386 Index....388 Why subscribe?....410 Other Books You May Enjoy....412 Packt is searching for authors like you....416 Share Your Thoughts....416 Download a free PDF copy of this book....417