Speed Up Your Python with Rust: Optimize Python performance by creating Python pip modules in Rust with Py03

Speed Up Your Python with Rust....2 Contributors....5 About the author....5 About the reviewers....6 Preface....22 Who this book is for....23 What this book covers....24 To get the most out of this book....25 Download the example code files....25 Download the color images....25 Conventions used....25 Get in touch....26 Share Your Thoughts....27 Section 1: Getting to Understand Rust....28 Chapter 1: An Introduction to Rust from a Python Perspective....29 Technical requirements....29 Understanding the differences between Python and Rust....30 Why fuse Python with Rust?....30 Passing strings in Rust....32 Sizing up floats and integers in Rust....35 Managing data in Rust's vectors and arrays....37 Replacing dictionaries with hashmaps....38 Error handling in Rust....42 Understanding variable ownership....45 Copy....46 Move....47 Immutable borrow....48 Mutable borrow....50 Keeping track of scopes and lifetimes....51 Building structs instead of objects....55 Metaprogramming with macros instead of decorators....58 Summary....61 Questions....62 Answers....62 Further reading....62 Chapter 2: Structuring Code in Rust....63 Technical requirements....63 Managing our code with crates and Cargo instead of pip....63 Structuring code over multiple files and modules....70 Building module interfaces....75 Benefits of documentation when coding....83 Interacting with the environment....84 Summary....87 Questions....87 Answers....88 Further reading....88 Chapter 3: Understanding Concurrency....89 Technical requirements....89 Introducing concurrency....89 Threads....89 Processes....91 Basic asynchronous programming with threads....92 Running multiple processes....99 Customizing threads and processes safely....112 Amdahl's law....113 Deadlocks....114 Race conditions....117 Summary....117 Questions....118 Answers....118 Further reading....119 Section 2: Fusing Rust with Python....120 Chapter 4: Building pip Modules in Python....121 Technical requirements....121 Configuring setup tools for a Python pip module....121 Creating a GitHub repository....122 Defining the basic parameters....125 Defining a README file....126 Defining a basic module....127 Packaging Python code in a pip module....128 Building our Fibonacci calculation code....129 Creating a command-line interface....131 Building unit tests....133 Configuring continuous integration....138 Manually deploying onto PyPI....139 Managing dependencies....141 Setting up type checking for Python....142 Setting up and running tests and type-checking with GitHub Actions....143 Create automatic versioning for our pip package....147 Deploying onto PyPI using GitHub Actions....150 Summary....153 Questions....153 Answers....154 Further reading....154 Chapter 5: Creating a Rust Interface for Our pip Module....155 Technical requirements....155 Packaging Rust with pip....155 Define gitignore and Cargo for our package....156 Configuring the Python setup process for our package....157 Installing our Rust library for our package....160 Building a Rust interface with the pyO3 crate....161 Building our Fibonacci Rust code....162 Creating command-line tools for our package....165 Creating adapters for our package....167 Building tests for our Rust package....175 Comparing speed with Python, Rust, and Numba....178 Summary....180 Questions....181 Answers....181 Further reading....181 Chapter 6: Working with Python Objects in Rust....182 Technical requirements....182 Passing complex Python objects into Rust....182 Updating our setup.py file to support .yml loading....183 Defining our .yml loading command....184 Processing data from our Python dictionary....185 Extracting data from our config file....188 Returning our Rust dictionary to our Python system....189 Inspecting and working with custom Python objects....192 Creating an object for our Rust interface....192 Acquiring the Python GIL in Rust....193 Adding data to our newly created PyDict struct....194 Setting the attributes of our custom object....197 Constructing our own custom Python objects in Rust....198 Defining a Python class with the required attributes....199 Defining class static methods to process input numbers....199 Defining a class constructor....200 Wrapping up and testing our module....201 Summary....204 Questions....205 Answers....205 Further reading....205 Chapter 7: Using Python Modules with Rust....206 Technical requirements....206 Exploring NumPy....206 Adding vectors in NumPy....207 Adding vectors in pure Python....208 Adding vectors using NumPy in Rust....210 Building a model in NumPy....213 Defining our model....213 Building a Python object that executes our model....216 Using NumPy and other Python modules in Rust....219 Recreating our NumPy model in Rust....222 Building get_weight_matrix and inverse_weight_matrix functions....223 Building get_parameters, get_times, and get_input_vector functions....224 Building calculate_parameters and calculate_times functions....225 Adding calculate functions to the Python bindings and adding a NumPy dependency to our setup.py file....226 Building our Python interface....227 Summary....228 Questions....229 Answers....229 Further reading....230 Chapter 8: Structuring an End-to-End Python Package in Rust....231 Technical requirements....231 Breaking down a catastrophe modeling problem for our package....231 Building an end-to-end solution as a package....237 Building a footprint merging process....238 Building the vulnerability merge process....241 Building a Python interface in Rust....244 Building our interface in Python....246 Building package installation instructions....247 Utilizing and testing our package....249 Building a Python construct model using pandas....250 Building a random event ID generator function....251 Timing our Python and Rust implementations with a series of different data sizes....252 Summary....254 Further reading....254 Section 3: Infusing Rust into a Web Application....256 Chapter 9: Structuring a Python Flask App for Rust....257 Technical requirements....257 Building a basic Flask application....257 Building an entry point for our application....258 Building our Fibonacci number calculator module....259 Building a Docker image for our application....261 Building our NGINX service....263 Connecting and running our Nginx service....264 Defining our data access layer....267 Defining a PostgreSQL database in docker-compose....268 Building a config loading system....269 Building our data access layer....271 Setting up the application database migration system....273 Building database models....275 Applying the database access layer to the fib calculation view....277 Building a message bus....279 Building a Celery broker for Flask....280 Building a Fibonacci calculation task for Celery....282 Updating our calculation view....283 Defining our Celery service in Docker....283 Summary....287 Questions....287 Answers....287 Further reading....288 Chapter 10: Injecting Rust into a Python Flask App....289 Technical requirements....289 Fusing Rust into Flask and Celery....289 Defining our dependency on the Rust Fibonacci number calculation package....290 Building our calculation model with Rust....290 Creating a calculation view using Rust....293 Inserting Rust into our Celery task....294 Deploying Flask and Celery with Rust....295 Deploying with a private GitHub repository....297 Building a Bash script that orchestrates the whole process....299 Reconfiguring the Rust Fib package installment in our Dockerfile....299 Fusing Rust with data access....300 Setting up our database cloning package....301 Setting up the diesel environment....303 Autogenerating and configuring our database models and schema....304 Defining our database connection in Rust....306 Creating a Rust function that gets all the Fibonacci records and returns them....307 Deploying Rust nightly in Flask....309 Summary....310 Questions....311 Answers....311 Further reading....311 Chapter 11: Best Practices for Integrating Rust....312 Technical requirements....312 Keeping our Rust implementation simple by piping data to and from Rust....312 Building a Python script that formulates the numbers for calculation....313 Building a Rust file that accepts the numbers, calculates the Fibonacci numbers, and returns the calculated numbers....314 Building another Python script that accepts the calculated numbers and prints them out....315 Giving the interface a native feel with objects....317 Defining traits....323 Defining struct behavior with traits....325 Passing traits through functions....327 Storing structs with common traits....329 Running our traits in the main file....329 Keeping data-parallelism simple with Rayon....332 Further reading....334 Why subscribe?....335 Other Books You May Enjoy....335 Packt is searching for authors like you....338 Share Your Thoughts....338