Python for Mathematical Thinking

Preface....7 Contents....10 About the Authors....17 List of Common Abbreviations....19 1 Introduction....21 1.1 Why Python for Mathematics?....22 1.1.1 Advantages of Python in Mathematical Computation....22 Overview....22 Worked Examples....22 Common Pitfalls....23 1.1.2 Comparison with Other Mathematical Tools....24 Python vs. MATLAB....24 Python vs. R....24 Python vs. Mathematica....25 1.2 Setting Up the Python Environment....25 1.2.1 Installing Python and Essential Libraries....25 CPython and Anaconda Distributions....25 Managing Virtual Environments....26 1.2.2 Python Syntax and Semantics....26 Variables, Expressions, and Statements....26 Indentation, Blocks, and Style....27 1.2.3 Python as an Interpreted Language....27 Bytecode and the CPython VM....27 Interactive vs. Script Mode....28 1.3 Data Types and Variables....28 1.3.1 Primitive Data Types....28 Integers, Floats, and Complex Numbers....28 Booleans and NoneType....29 1.3.2 Mathematical Operations on Data Types....29 Arithmetic Operators....29 Bitwise and Logical Operators....30 1.3.3 Working with Variables....30 Naming Conventions....30 Dynamic Typing and Type Hints....31 1.3.4 Containers and Collections....31 Lists and Tuples....31 Sets and Frozen Sets....31 Dictionaries....32 1.4 Control Flow in Python....32 1.4.1 Conditional Statements....33 if-elif-else Blocks....33 Ternary Expressions....33 1.4.2 Loops in Python....34 for Loops....34 while Loops....34 1.4.3 Comprehensions in Python....35 List Comprehensions....35 Set and Dictionary Comprehensions....35 1.4.4 Error Handling and Exceptions....36 try-except-finally....36 Custom Exception Classes....37 1.5 Functions and Modular Programming....37 1.5.1 Defining Functions....38 Positional and Keyword Arguments....38 Default and Variable-Length Parameters....38 1.5.2 Advanced Function Concepts....39 First-Class Functions and Closures....39 Decorators and Higher-Order Functions....39 1.5.3 Modular Programming with Python....40 Packages and __init__.py....40 Import Mechanics and Namespaces....40 1.5.4 Recursive Functions....40 Mathematical Recurrence Relations....40 Tail Recursion and Optimisation....41 1.6 Input and Output in Python....41 1.6.1 Handling User Input....42 input() and Command-Line Arguments....42 1.6.2 File Handling in Python....42 Text vs. Binary Modes....42 Context Managers....43 1.6.3 Formatted Output....43 f'-strings'....43 str.format()....43 1.6.4 Best Practices in Mathematical Coding....44 Code Styling and Linting....44 Unit Testing with pytest....44 1.7 Object-Oriented Programming (OOP) in Python....44 1.7.1 Basic Concepts of OOP....45 Classes and Objects....45 Encapsulation and Abstraction....46 1.7.2 Advanced OOP Concepts....46 Inheritance and Polymorphism....46 Mixins and Multiple Inheritance....46 1.7.3 Special Methods and Operator Overloading....47 __str__, __repr__, etc.....47 Arithmetic and Comparison Overloads....48 1.7.4 Working with Mathematical Structures in OOP....48 Vectors and Matrices as Classes....48 Symbolic Algebra Objects....49 1.8 Iterators and Generators....50 1.8.1 Understanding Iterators in Python....50 Iterator Protocol....50 Custom Iterators....50 1.8.2 Generators and Lazy Evaluation....50 Generator Functions....50 Generator Expressions....51 1.8.3 Combinatorics with Iterators and Generators....51 Cartesian Products and Permutations....51 Infinite Sequences....51 1.9 Advanced Topics in Python....51 1.9.1 Regular Expressions and String Matching....52 Basic Pattern Syntax....52 Advanced Regex Features....52 1.9.2 Working with Dates and Times....53 datetime Module....53 Time-Zone Handling....53 1.10 Exercises....54 2 Mathematical Foundations in Python....57 2.1 Basic Arithmetic and Algebra....57 2.1.1 Arithmetic Operations....57 Integer Arithmetic....57 Floating-Point Precision....58 2.1.2 Algebraic Expressions....59 Symbolic Manipulation with sympy....59 2.1.3 Polynomial Algebra....60 Roots and Factorisation....60 2.1.4 Inequalities and Absolute Values....61 Chain and Triangle Inequalities....61 2.2 Functions and Graphs....62 2.2.1 Defining Functions in Python....62 Lambda Functions....62 2.2.2 Graphing Functions....63 Plotting with matplotlib....63 2.2.3 Analysing Function Properties....64 Limits and Continuity....64 2.3 Matrices and Linear Algebra....65 2.3.1 Matrix Operations....65 Addition and Multiplication....65 Kronecker Products....66 2.3.2 Determinants and EigenvaluesEigenvectors....67 Characteristic Polynomials....67 2.3.3 Solving Systems of Linear Equations....67 Gaussian Elimination....67 2.4 Exercises....68 3 Calculus with Python....72 3.1 Differentiation....72 3.1.1 Numerical Differentiation Techniques....72 Finite Difference Approaches....72 Automatic Differentiation with autograd....73 Error and Stability Analysis....74 3.1.2 Symbolic Differentiation Using SymPy....75 Differentiation Rules and Operators....75 Higher-Order Derivatives....76 Simplification and Optimisation....77 3.1.3 Applications of Differentiation....78 Optimisation Problems....78 Curve Sketching and Analysis....79 Physical System Modelling....80 3.2 Integration....81 3.2.1 Numerical Integration Methods....82 Riemann and Trapezoidal Rules....82 Simpson's Rule and Gaussian Quadrature....83 Adaptive Quadrature Algorithms....84 3.2.2 Symbolic Integration with SymPy....84 Indefinite Integrals and Antiderivatives....85 Definite Integrals and Limits....86 Special Functions and Integral Tables....87 3.2.3 Applications of Integration....88 Area and Volume Computations....88 Probability and Expectation....89 Solving Differential Equations....90 3.3 Multivariable Calculus....91 3.3.1 Partial Derivatives and Gradients....91 Directional Derivatives....92 Gradient Fields and Level Sets....93 Multivariate Optimisation....94 3.3.2 Multiple Integrals....95 Double Integrals....95 Triple Integrals....96 Change of Variables and Jacobians....97 3.3.3 Line Integrals and Surface Integrals....98 Scalar Line Integrals....98 Vector Line Integrals....99 Flux Through Surfaces....100 3.4 Advanced Topics in Calculus....101 3.4.1 Differential Equations....102 First-Order ODEs....102 Higher-Order ODEs and Systems....103 Numerical Solvers in SciPy....104 3.4.2 Calculus of Variations....106 Euler–Lagrange Formalism....107 Constraints and Lagrange Multipliers....107 Applications in Physics....109 3.4.3 Tensor Calculus....110 Index Notation and Einstein Summation....110 Covariant and Contravariant Tensors....111 General Relativity Applications....112 3.4.4 Fractional Calculus....113 Fractional Derivatives....113 Fractional Integrals....114 Modelling with Fractional Dynamics....115 3.5 Exercises....117 4 Data Structures and Algorithms with Python....120 4.1 Introduction to Data Structures....120 4.1.1 Overview of Data Structures....120 Role of Data Structures in Computational Efficiency....121 Types of Data Structures and Their Use Cases....121 4.1.2 Basic Data Structures....123 Stacks....123 Queues....124 Linked Lists....125 Trees....127 Hash Tables and Dictionary Internals....129 4.2 Search Algorithms....131 4.2.1 Linear Search....131 Algorithm and Complexity Analysis....131 Python Implementation....132 4.2.2 Binary Search....133 Algorithm and Complexity Analysis....133 Python Implementation....134 4.3 Sorting Algorithms....136 4.3.1 Basic Sorting Algorithms....136 Selection Sort....136 Bubble Sort....137 Insertion Sort....138 4.3.2 Divide and Conquer Techniques....139 Merge Sort....139 Quick Sort....141 4.4 Graph Theory and Algorithms....143 4.4.1 Introduction to Graph Theory....143 Graphs as Mathematical Structures....143 Graph Representation in Python....144 4.4.2 Basic Graph Algorithms....146 Breadth-First Search (BFS)....146 Depth-First Search (DFS)....147 Shortest Path Algorithms: Dijkstra's Algorithm....148 4.5 Exercises....150 5 Probability and Statistics....153 5.1 Probability Theory....153 5.1.1 Basic Probability Concepts....153 Probability Axioms and Theorems....153 Combinatorial Probability: Permutations and Combinations....155 Conditional Probability and Bayes' Theorem....157 Independence and Dependence of Events....159 5.1.2 Random Variables and Distributions....160 Discrete and Continuous Distributions....160 Probability Density Functions and Cumulative Distribution Functions....161 Simulation of Random Variables in Python....162 5.1.3 Expectation, Variance, and Moments....163 Mathematical Expectation and Properties....163 Variance, Covariance, and Standard Deviation....164 Moment Generating Functions....165 5.1.4 Common Probability Distributions....167 Binomial, Poisson, and Geometric Distributions....167 Normal, Exponential, and Gamma Distributions....168 Central Limit Theorem and Law of Large Numbers....168 5.2 Descriptive Statistics....170 5.2.1 Measures of Central Tendency....170 Mean, Median, Mode in Python....170 Weighted Mean and Percentiles....171 Geometric Mean and Harmonic Mean....171 5.2.2 Measures of Dispersion....172 Variance, Standard Deviation, and Range....173 Interquartile Range and Outlier Detection....173 Skewness and Kurtosis....174 5.2.3 Data Visualisation and Analysis....175 Histograms, Box Plots, and Scatter Plots....175 Correlation and Covariance....178 Heatmaps and Pair Plots for Multivariate Data....178 Data Cleaning and Preprocessing Techniques....179 5.3 Inferential Statistics....180 5.3.1 Hypothesis Testing....181 Null and Alternative Hypotheses....181 p-values, t-tests, and Chi-square Tests....181 Type I and Type II Errors....182 Power of a Test and Sample Size Determination....183 5.3.2 Confidence Intervals....184 Constructing Confidence Intervals in Python....184 Applications in Scientific Research....185 Bootstrap Methods for Confidence Intervals....186 5.3.3 Regression Analysis....187 Simple Linear Regression....187 Multiple Regression and Model Selection....188 Curve Fitting and Polynomial Regression....188 Logistic Regression for Classification....189 5.3.4 Advanced Topics in Regression....189 Ridge and Lasso Regression....190 Time Series Analysis and ARIMA Models....191 Principal Component Analysis (PCA) for Dimensionality Reduction....192 5.4 Stochastic Processes and Applications....194 5.4.1 Markov Chains....194 Transition Matrices and Long-Term Behaviour....195 Applications of Markov Chains in Python....196 5.4.2 Poisson Processes....197 Modelling Events in Time and Space....197 Applications in Queuing Theory and Reliability....198 5.4.3 Brownian Motion and Applications....200 Simulating Brownian Motion in Python....200 Applications in Finance: Stock Price Modelling....201 5.4.4 Bayesian Statistics....203 Bayesian Inference and Prior Distributions....204 Markov Chain Monte Carlo (MCMC) Methods....205 Applications in Machine Learning and Decision-Making....206 5.5 Exercises....207 6 Differential Equations....210 6.1 Ordinary Differential Equations (ODEs)....210 6.1.1 First-Order ODEs....210 Separable and Linear ODEs....211 Exact ODEs and Integrating Factors....213 Autonomous Equations and Stability Analysis....214 Bifurcation Theory and Phase Plane Analysis....216 6.1.2 Higher-Order ODEs....218 Reduction of Order and Method of Undetermined Coefficients....218 Variation of Parameters and Cauchy–Euler Equations....219 Green's Functions and Boundary Value Problems....220 Series Solutions of ODEs: Frobenius Method....220 6.1.3 Systems of ODEs....221 Linear Systems and Matrix Methods....221 Nonlinear Systems and Stability....222 Lyapunov's Direct Method....223 6.1.4 Numerical Solutions of ODEs....224 Euler's Method and Runge–Kutta Methods....224 Stiff Equations and Implicit Methods....225 Applications in Population Dynamics and Mechanics....226 Shooting Method and Boundary Value Problems....227 6.2 Partial Differential Equations (PDEs)....228 6.2.1 Numerical Methods for Solving PDEs....231 Finite Difference Method and Finite Element Method....231 Spectral Methods for PDEs....233 Applications in Engineering and Physics....234 Stability and Convergence of Numerical Methods....234 6.2.2 Advanced Topics in PDEs....234 Fourier Series Solutions....234 Transform Methods for PDEs....236 Nonlinear PDEs and Soliton Solutions....236 PDEs in Higher Dimensions....237 6.3 Special Functions and Transform Techniques....237 6.3.1 Laplace Transforms and Applications....237 Inverse Laplace Transform....238 Laplace Transform in Solving ODEs and PDEs....239 6.3.2 Fourier Transforms and Applications....240 Discrete Fourier Transform and Fast Fourier Transform....240 Fourier Transform in Signal Processing and PDEs....242 6.3.3 Special Functions in ODEs and PDEs....243 Bessel Functions and Their Applications....243 Legendre Polynomials and Spherical Harmonics....245 6.3.4 Green's Functions and Integral Equations....246 Green's Function for ODEs....246 Applications to PDEs and Boundary Value Problems....247 Integral Equations and Fredholm Theory....248 6.4 Stochastic Differential Equations (SDEs)....249 6.4.1 Introduction to SDEs....249 Brownian Motion and Stochastic Processes....249 It's Lemma and Stochastic Calculus....250 6.4.2 Numerical Solutions of SDEs....252 Euler–Maruyama Method....252 Milstein Method and Higher-Order Approximations....253 6.4.3 Applications of SDEs....253 Financial Modelling: Black–Scholes Equation....253 Stochastic Population Models....254 Physics Applications: Langevin Equation....255 6.5 Exercises....255 7 Discrete Mathematics and Combinatorics....259 7.1 Number Theory....259 7.1.1 Divisibility and Modular Arithmetic....260 Prime Numbers and the Euclidean Algorithm....260 Modular Exponentiation and Applications in Cryptography....260 Greatest Common Divisor (GCD) and Least Common Multiple (LCM)....261 Fermat's Little Theorem and Wilson's Theorem....261 7.1.2 Congruences and Number Theoretic Functions....262 Chinese Remainder Theorem and Euler's Totient Function....262 Applications in Coding Theory and Cryptanalysis....262 Quadratic Residues and Legendre Symbols....263 Elliptic Curves and Cryptography....263 7.1.3 Advanced Topics in Number Theory....264 Diophantine Equations and Applications....264 Modular Forms and Their Applications....265 Analytic Number Theory and the Riemann Hypothesis....266 7.2 Combinatorics....267 7.2.1 Basic Counting Principles....267 Permutations and Combinations....267 Pigeonhole Principle and Inclusion–Exclusion Principle....268 Derangements and Catalan Numbers....268 Stirling Numbers and Bell Numbers....269 7.2.2 Generating Functions....270 Ordinary and Exponential Generating Functions....270 Applications in Recurrence Relations and Partition Theory....271 Combinatorial Identities and the Binomial Theorem....271 Multivariate Generating Functions and Applications....272 7.2.3 Advanced Topics in Combinatorics....272 Plya's Enumeration Theorem and Group Actions....273 Ramsey Theory and Extremal Combinatorics....273 Probabilistic Method in Combinatorics....274 7.3 Graph Theory....275 7.3.1 Basic Concepts in Graph Theory....275 Graph Representation in Python....276 Graph Isomorphism and Subgraph Isomorphism....276 Planar Graphs and Euler's Formula....277 7.3.2 Advanced Topics in Graph Theory....277 Graph Colouring and Chromatic Polynomials....277 Spectral Graph Theory and Applications....278 Random Graphs and Erds–Rnyi Model....279 7.3.3 Applications of Graph Theory....281 Applications in Network Analysis and Optimisation....281 Social Network Analysis and Community Detection....281 Graph-Based Machine Learning Algorithms....282 7.4 Boolean Algebra and Logic....283 7.4.1 Propositional Logic and Proof Techniques....283 Logical Connectives, Truth Tables, and Tautologies....283 Proof Techniques: Direct, Indirect, and Contradiction....284 Applications in Automated Theorem Proving....285 7.4.2 Boolean Algebra and Circuit Design....285 Boolean Functions and Simplification Techniques....286 Karnaugh Maps and Quine–McCluskey Method....286 Applications in Digital Circuit Design....287 7.4.3 Advanced Topics in Logic....288 Predicate Logic and Quantifiers....288 Gdel's Incompleteness Theorems....289 Non-classical Logics and Their Applications....289 7.5 Discrete Structures and Applications....290 7.5.1 Sets, Relations, and Functions....291 Set Theory Basics and Venn Diagrams....291 Relations: Reflexivity, Symmetry, and Transitivity....291 Functions: Injective, Surjective, and Bijective Mappings....292 7.5.2 Algebraic Structures....293 Groups, Rings, and Fields....293 Lattices and Boolean Algebras....294 Applications in Cryptography and Coding Theory....294 7.5.3 Matroids and Their Applications....295 Introduction to Matroids and Examples....295 Applications in Optimisation and Graph Theory....296 Greedy Algorithms and Matroid Theory....297 7.6 Exercises....298 8 Numerical Methods ....301 8.1 Root-Finding Algorithms....302 8.1.1 Bisection Method....302 Convergence and Implementation in Python....302 Applications in Engineering and Finance....303 Error Analysis and Stopping Criteria....303 8.1.2 Newton–Raphson Method....304 8.1.3 Other Root-Finding Methods....306 Secant Method and False Position Method....307 Brent's Method and Hybrid Techniques....307 Root-Finding in Complex Domains....308 8.2 Optimisation Techniques....310 8.2.1 Gradient Descent....310 Basic Gradient Descent and Variants....310 Applications in Machine Learning and Data Fitting....311 Stochastic Gradient Descent and Mini-Batch Methods....312 8.2.2 Simplex Method....313 Linear Programming Problems....313 Implementation in Python....314 Dual Simplex Method and Sensitivity Analysis....314 8.2.3 Nonlinear Optimisation....315 Trust-Region Methods and Quasi-Newton Methods....315 Applications in Economics and Engineering....316 Global Optimisation Techniques: Genetic Algorithms and Simulated Annealing....317 8.2.4 Direct Methods for Linear Systems....318 LU Decomposition and Cholesky Decomposition....318 Applications in Scientific Computing....319 Pivoting Strategies and Numerical Stability....320 8.2.5 Iterative Methods for Linear Systems....321 Jacobi Method, Gauss–Seidel Method....321 Convergence Criteria and Applications....322 Krylov Subspace Methods: GMRES and Conjugate Gradient....322 8.2.6 Solving Nonlinear Systems....324 Fixed-Point Iteration and Newton's Method....324 Applications in Fluid Dynamics and Chemical Engineering....325 Continuation Methods and Bifurcation Analysis....326 8.3 Numerical Integration and Differentiation....327 8.3.1 Numerical Differentiation Techniques....327 Finite Difference Approximations....327 Error Analysis in Numerical Differentiation....329 8.3.2 Numerical Integration Methods....331 Trapezoidal Rule, Simpson's Rule....331 Gaussian Quadrature and Adaptive Methods....332 Monte Carlo Integration and Applications....333 8.3.3 Applications of Numerical Integration....334 Computing Areas and Volumes....334 Applications in Physics and Engineering....335 Numerical Solutions to Differential Equations....336 8.4 Eigenvalue Problems and Matrix Decompositions....337 8.4.1 Power Method and Inverse Iteration....337 Applications in Structural Engineering and Vibrations....338 Convergence and Numerical Stability....339 8.4.2 QR Algorithm and Schur Decomposition....340 Eigenvalue Computation in Python....340 Applications in Quantum Mechanics and Control Theory....341 8.4.3 Singular Value Decomposition (SVD)....342 Applications in Data Compression and Principal Component Analysis (PCA)....342 Low-Rank Approximations and Image Processing....344 8.5 Advanced Topics in Numerical Methods....344 8.5.1 Numerical Solutions to Differential Equations....344 Finite Difference Methods for ODEs and PDEs....345 Method of Lines and Applications....346 Stability and Convergence Analysis....346 8.5.2 Spectral Methods....347 Chebyshev Polynomials and Fourier Spectral Methods....347 Applications in Fluid Dynamics and Weather Modelling....348 8.5.3 Parallel Computing in Numerical Methods....349 Applications in High-Performance Computing and Big Data Analytics....350 8.5.4 Error Analysis and Stability in Numerical Methods....350 Round-Off Errors and Machine Precision....351 Stability in Numerical Algorithms....351 8.6 Exercises....352 9 Chaos Theory and Dynamical Systems ....355 9.1 Introduction to Dynamical Systems....356 9.1.1 Fixed Points and Stability....356 Linear Stability Analysis....356 Bifurcation Theory and Chaos....357 Bifurcation Diagrams and Continuation Methods....358 Stability of Nonlinear Systems....359 9.1.2 Discrete Dynamical Systems....360 Logistic Map and Cobweb Diagrams....360 Lyapunov Exponents and Their Calculation....362 Period-Doubling Route to Chaos....363 Symbolic Dynamics and Subshifts of Finite Type....363 State-Space Reconstruction (Takens Embedding)....364 9.1.3 Continuous Dynamical Systems....365 Phase Portraits and Limit Cycles....365 Strange Attractors and Chaos in Continuous Systems....366 Poincar Maps and Recurrence Plots....367 Hamiltonian Systems and Chaos....368 9.2 Chaos and Fractals....370 9.2.1 The Mandelbrot Set....370 Fractal Geometry and Self-Similarity....370 Rendering the Mandelbrot Set in Python....371 Complex Dynamics and the Mandelbrot Set....372 9.2.2 Julia Sets....373 Relationship with the Mandelbrot Set....373 Visualising Julia Sets in Python....374 Exploring Parameter Space in Julia Sets....375 9.2.3 Fractals in Nature and Art....376 Self-Similarity in Natural Systems....376 Applications of Fractals in Art and Design....378 Multifractals and Their Applications....378 9.2.4 Applications of Chaos Theory....379 Predicting Weather Patterns and Stock-Market Dynamics....380 Chaos in Biological Systems....381 Applications in Secure Communications....381 Chaos in Mechanical and Electrical Systems....382 9.3 Lyapunov Exponents....383 9.3.1 Definition and Interpretation....383 Calculation Methods....383 Implications for System Behaviour....384 Local and Global Lyapunov Exponents....384 9.4 Advanced Topics in Chaos Theory....385 9.4.1 Entropy and Chaos....385 Kolmogorov–Sinai Entropy....385 Relationship Between Entropy and Lyapunov Exponents....386 9.4.2 Chaos Control and Synchronisation....387 OGY and Time-Delayed Feedback Control (Ott1990)....387 Coupled Map Lattices and Chaos Synchronisation....388 Applications in Secure Communications....388 9.4.3 Complex Networks and Chaos....389 Chaotic Behaviour in Complex Networks....389 Applications in Social and Biological Networks....390 9.4.4 Quantum Chaos....390 Quantum Signatures of Chaos....391 Applications in Quantum Computing....391 9.5 Data-Driven Dynamics and Modern Tools....392 9.5.1 Koopman Operator Theory....392 Dynamic Mode Decomposition (DMD)....393 Koopman Spectral Analysis in Python....393 9.5.2 Reservoir Computing for Chaotic Time Series....393 Echo-State Networks and Forecasting....394 9.5.3 State-Space Reconstruction in Practice....395 False-Nearest-Neighbour Criterion....395 Predictability Horizons and Forecast Skill....397 9.6 Complex Systems and Nonlinear Dynamics....398 9.6.1 Lyapunov Exponents and Strange Attractors....398 9.6.2 Applications in Weather Forecasting and Financial Markets....399 9.6.3 Applications in Biological Systems and Population Dynamics....399 9.6.4 Numerical Methods for Nonlinear Dynamics....400 9.6.5 Complex Networks and Emergent Behaviour....401 Network Theory and Small-World Phenomena....401 Applications in Epidemics and Social Dynamics....402 Modelling Emergent Behaviour in Complex Systems....403 9.7 Exercises....403 10 Data Science and Machine Learning ....406 10.1 Introduction to Data Science....406 10.1.1 Data Manipulation and Cleaning....406 Using Pandas for Data Cleaning....407 Handling Missing Data and Outliers....407 Data Transformation and Normalisation Techniques....408 Feature Encoding: One-Hot Encoding and Label Encoding....408 10.1.2 Exploratory Data Analysis (EDA)....409 Descriptive Statistics and Visualisation Techniques....409 Correlation Analysis and Feature Engineering....409 Dimensionality Reduction for EDA....410 Advanced Visualisation: Heatmaps, Pair Plots, and 3D Plots....410 10.1.3 Data Wrangling and Integration....411 Merging, Joining, and Concatenating Dataframes....411 Working with Time Series Data....412 Handling Large Datasets with Dask and PySpark....413 10.2 Machine Learning Algorithms....414 10.2.1 Supervised Learning....414 Linear Regression, Logistic Regression....414 Decision Trees and Random Forests....415 Support Vector Machines (SVM) and Kernel Methods....415 Ensemble Methods: Bagging, Boosting, and Stacking....416 10.2.2 Unsupervised Learning....417 Clustering: k-Means and Hierarchical Clustering....417 Dimensionality Reduction: PCA, t-SNE....418 Anomaly Detection Techniques....418 Association Rule Mining: Apriori and FP-Growth....419 10.2.3 Reinforcement Learning....420 Markov Decision Processes (MDPs)....420 Q-Learning and Deep Q-Networks (DQN)....421 Policy Gradient Methods and Actor–Critic Algorithms....421 Applications of Reinforcement Learning in Game AI and Robotics....422 10.3 Deep Learning....423 10.3.1 Mathematical Foundations of Neural Networks....423 Perceptrons, Activation Functions, and Backpropagation....423 Regularisation Techniques and Optimisation Algorithms....424 Loss Functions: Cross-Entropy, MSE, and Custom Losses....425 10.3.2 Implementing Neural Networks from Scratch....425 Forward and Backward Propagation in Python....426 Training and Evaluating Neural Networks....427 Hyperparameter Tuning and Model Selection....428 10.3.3 Convolutional and Recurrent Neural Networks....429 CNNs for Image Recognition....429 RNNs for Sequential Data Analysis....430 Advanced Architectures: LSTM, GRU, and Transformers....430 Transfer Learning and Fine-Tuning Pretrained Models....431 10.4 Advanced Topics in Machine Learning....432 10.4.1 Model Interpretability and Explainability....432 SHAP Values, LIME, and Feature Importance....432 Interpreting Black-Box Models in Python....434 10.4.2 Model Evaluation and Validation Techniques....434 Cross-Validation and Resampling Methods....435 Handling Imbalanced Datasets: SMOTE and ADASYN....436 Model Selection with AIC, BIC, and Information Criteria....436 10.4.3 Generative Models....437 Autoencoders and Variational Autoencoders (VAEs)....437 Generative Adversarial Networks (GANs) and Applications....438 Bayesian Networks and Probabilistic Graphical Models....439 10.4.4 Big Data and Scalable Machine Learning....440 Distributed Machine Learning with Apache Spark....440 Scaling ML Models with Kubernetes and Docker....441 Real-Time Machine Learning with Streaming Data....442 10.5 Ethics and Fairness in Machine Learning....443 10.5.1 Bias and Fairness in AI....443 Identifying and Mitigating Bias in Machine Learning Models....443 Fairness Metrics and Tools in Python....444 10.5.2 Privacy–Preserving Machine Learning....445 Federated Learning and Differential Privacy....446 Secure Multi-party Computation....447 10.5.3 Responsible AI and Ethical Considerations....448 Ethical Frameworks and Guidelines for AI Development....448 Impact of AI on Society and Future Implications....448 10.6 Exercises....449 11 Advanced Topics ....452 11.1 Symbolic Computation....453 11.1.1 Advanced Symbolic Mathematics with SymPy....453 Solving Complex Equations and Systems Symbolically....454 Applications in Algebra and Calculus....454 Symbolic Linear Algebra and Matrix Computations....455 Differential Equations and Special Functions....456 11.1.2 Automated Theorem Proving....457 Introduction to Theorem Proving....457 Using SymPy for Automated Proofs....457 First-Order Logic and Quantifiers in Theorem Proving....458 Applications in Formal Verification and Cryptography....459 11.1.3 Fourier Transforms....460 Discrete and Continuous Fourier Transforms....460 Fast Fourier Transform (FFT) and Computational Efficiency....460 Applications in Signal Processing and Image Compression....461 Fourier Analysis in Heat and Wave Equations....461 11.1.4 Wavelet Transforms....462 Introduction to Wavelets....462 Wavelet Transform vs. Fourier Transform....462 Applications in Time-Frequency Analysis....463 Multiresolution Analysis and Denoising Techniques....463 11.2 Time-Series and Signal Processing....464 11.2.1 Statistical Time-Series Models....464 Autoregressive (AR), Moving-Average (MA), and ARIMA Models....464 State-Space Models and the Kalman Filter....465 ARCHGARCH and Volatility Modelling....466 Trend-Seasonality Decomposition and Stationarity Tests....466 11.2.2 Digital Filtering in Python....467 FIR vs. IIR Filters: Design and Stability....467 Window Functions and Smoothing Techniques....468 Butterworth, Chebyshev, and Elliptic Filters....468 Real-Time Filtering and Stream Processing....469 11.2.3 Spectral Analysis....469 Periodogram and Power Spectral Density Estimation....470 Welch's Method and Multi-taper Spectral Estimates....470 Short-Time Fourier Transform (STFT) and Spectrograms....471 Cross-Spectral Density and Coherence Analysis....471 11.2.4 Time-Frequency Representations....472 Continuous Wavelet Transform (CWT) Revisited....472 Empirical Mode Decomposition and Hilbert–Huang Transform....473 Synchrosqueezed and High-Resolution Methods....473 11.2.5 Applications....474 Financial Market Forecasting....474 Biomedical Signals: ECG and EEG....475 Environmental and Climate Data Analytics....476 Audio, Speech, and Music Processing....476 11.3 Topological Data Analysis....477 11.3.1 Introduction to TDA....477 Applications in Data Analysis and Machine Learning....477 Topological Features in High-Dimensional Data....478 11.3.2 Computational Topology....479 Simplicial Complexes and Homology Groups....480 Applications in Biology and Sensor Networks....481 Mapper Algorithm for Data Visualisation....481 Advanced Topics in Computational Topology....482 11.4 Quantum Computing....483 11.4.1 Mathematical Foundations of Quantum Mechanics....483 Linear Algebra in Quantum Computing....483 Quantum Gates and Circuits....483 Quantum Entanglement and Bell's Theorem....484 Applications in Cryptography and Secure Communication....484 11.4.2 Quantum Algorithms....485 Shor's Algorithm and Grover's Algorithm....485 Simulating Quantum Circuits in Python....486 Quantum Error Correction and Fault Tolerance....487 Quantum Machine Learning and Optimisation....487 11.4.3 Spectral Methods and Applications....488 Chebyshev Polynomials and Fourier Spectral Methods....489 Applications in Fluid Dynamics and Weather Modelling....490 Stability and Convergence in Spectral Methods....490 11.4.4 Parallel and Distributed Computing for Numerical Methods....491 Parallel Algorithms for Large-Scale Systems....491 Applications in High-Performance Computing and Big Data Analytics....492 GPU Computing and TensorFlow for Numerical Simulations....492 11.5 Exercises....493 Appendix....497 Numerical Methods in SciPy....499 Symbolic Computation in SymPy....500 Online Resources....501 Python Libraries and Documentation....501 References....502