A Mathematical Introduction to Data Science with Python

Preface....5 Objective and Approach....6 Outline of This Book....6 Software Requirements....8 Hardware Used for Timings....8 How to Contact Us....9 Contents....11 1 Introduction....16 1.1 Getting Started with Two Essential Built-in Functions....16 1.2 Data Types in Python....19 1.2.1 Strings....19 1.2.2 Sequence Types....20 1.2.3 Dictionaries....23 1.2.4 Booleans....25 1.3 Python Library Structure Guide....26 1.4 Introduction to NumPy....30 1.5 Introduction to SymPy....36 1.6 Data Manipulation with Pandas....39 1.7 Machine Learning with scikit-learn....42 1.8 Visualisation with Matplotlib....44 1.9 Control Flow in Python....47 1.9.1 If Statement....48 1.9.2 Inline If....49 1.9.3 For Loop....49 1.9.4 While Loop....50 1.10 Define Your Own Functions....51 1.10.1 How to Define A Function?....51 1.10.2 Define Your Own Functions Using Lambda....53 2 Sets and Functions....55 2.1 Sets....55 2.1.1 Set Construction....55 2.1.2 Subsets and Power Sets....59 2.1.3 Set Operations....61 2.1.4 Intervals....64 2.1.5 Sets Written in Comprehension....65 2.1.6 Cartesian Product Sets....67 2.2 Mathematical Functions....68 2.2.1 More About NumPy Functions....70 2.3 Composition of Two Functions....75 3 Linear Algebra....77 3.1 Vectors....77 3.1.1 Creating Vectors....77 3.1.2 Basic Vector Operations....79 3.1.3 Norm and Unit Vector....80 3.1.4 Distance and Angle Between Two Vectors....81 3.2 Linear Combinations and Span....82 3.3 Matrices....85 3.3.1 Build Up Matrices and Access Elements of a Matrix....85 3.3.2 Diagonal and Trace....90 3.4 More Using Matrices in Python....92 3.4.1 Operations with Scalars....92 3.4.2 Operations with Matrices....93 3.5 Matrices as Linear Transformations....100 4 Matrix Decomposition....104 4.1 Eigendecomposition....104 4.2 Understand PCA Through a Simple Example....108 4.3 Principal Component Analysis Using scikit-learn....113 4.4 Understanding SVD Through a Simple Example....114 4.4.1 The Relationship Between PCA and SVD....122 4.5 Compressing an Image Using Singular Vector Decomposition....123 5 Calculus....129 5.1 Finding Limits of Functions....129 5.2 Finding Derivatives of Functions....133 5.2.1 Finding Derivatives of Functions Using SymPy....133 5.2.2 The Chain Rule for Differentiation....138 5.3 Finding Critical Points of A Function with One Variable Using Derivatives....141 5.4 Integrals....145 5.4.1 Definite Integrals and Indefinite Integrals with SymPy....145 5.4.2 An Illustration of the Definite Integral....146 5.4.3 Integration by Substitution....149 5.4.4 Integration by Parts....150 6 Advanced Calculus....153 6.1 Partial Derivatives....153 6.1.1 Jacobian Matrix....154 6.1.2 Hessian Matrix....157 6.2 Find Maxima and Minima for Functions with Two Variables....158 6.3 Method of Lagrange Multipliers for Maxima and Minima....162 6.3.1 An Illustration of the Use of np.meshgrid()....167 6.4 Gradient Descent Algorithm....170 6.5 Double Integrals....174 7 Algorithms 1Principal Component Analysis....179 7.1 Relevant Mathematical Knowledge....179 7.1.1 The Variance of Projections....179 7.1.2 Two Derivatives with Respect to A Vector....182 7.2 The Mathematical Solution Behind the PCA....185 7.3 Data Normalisation....188 8 Algorithms 2Linear Regression....195 8.1 Simple Linear Regression with One Variable....195 8.2 Linear Regression with Multiple Variables....199 8.3 Linear Regression with scikit-learn–LinearRegression....202 8.4 Linear Regression with scikit-learn–SGDRegressor....205 8.4.1 Case Study 1An Overview of the Data and the Key Results....206 8.4.2 The Demonstration of the Use of SGDRegressor....206 9 Algorithms 3Neural Networks....215 9.1 Simple One-Layer Neural Networks....215 9.1.1 Linear Activation Function....216 9.1.2 Logistic Sigmoid Activation Function....221 9.2 An Example of a Simple Two-Layer Neural Network....226 9.3 Neural Network Models with scikit-learn....230 9.3.1 Regression....230 9.3.2 Classification....234 10 Probability....239 10.1 Combinatorial Analysis....239 10.2 Probability....243 10.2.1 Simple Probability....243 10.2.2 Discrete Random Variables....245 10.2.3 Continuous Random Variables....247 10.3 Mean and Variance....250 10.3.1 Mean....250 10.3.2 Variance....254 10.4 Special Univariate Distributions....257 10.4.1 Discrete Uniform Distribution....257 10.4.2 Bernoulli Distribution....260 10.4.3 Binomial Distribution....263 10.4.4 Poisson Distribution....266 10.4.5 Continuous Uniform Distribution....269 10.4.6 Gaussian or Normal Distribution....271 11 Further Probability....278 11.1 The Law of Large Numbers and the Central Limit Theorem....278 11.1.1 The Law of Large Numbers (LLN)....278 11.1.2 Central Limit Theorem....280 11.2 Multiple Random Variables....282 11.2.1 Joint Discrete Random Variables....282 11.2.2 Joint Continuous Random Variables....284 11.2.3 Multinomial Distribution....287 11.2.4 Multivariate Normal Distribution....289 11.3 Conditional Probability and Corresponding Rules....294 11.3.1 Conditional Probability for Two Discrete Random Variables....294 11.3.2 Conditional Probability for Two Continuous Random Variables....299 11.3.3 Conditional Mean for Two Continuous Random Variables....301 11.3.4 The Law of Total Probability....302 11.4 Bayes' Theorem....304 12 Elements of Statistics....307 12.1 Descriptive Statistics....307 12.1.1 Measures of Centre....307 12.1.2 Measures of Variation....313 12.1.3 The Boxplot....315 12.2 Elementary Sampling Theory....318 12.2.1 Sampling Distribution of Means....319 12.2.2 Sampling Distribution of Proportions....320 12.2.3 Student's t-Distribution....322 12.2.4 The Chi-Square Distribution....324 12.3 Interval Estimation....326 12.3.1 Confidence Interval for Means....326 12.3.2 Confidence Interval for Proportions....328 12.3.3 Confidence Interval for chi-squared....330 12.4 Testing Hypothesis....332 12.4.1 The t-test for mean....332 12.4.2 The t-test on Difference between Means of Two Samples....334 12.4.3 The Chi-Square Test for One-way Classification Tables....336 12.4.4 The Chi-Square Test for Two-Way Classification Tables....339 13 Algorithms 4Maximum Likelihood Estimation and Its Application to Regression....342 13.1 Maximum Likelihood Estimation....342 13.2 Revisiting Linear Regression....348 13.2.1 Linear Regression with Maximum Likelihood Estimation....349 13.2.2 Confidence Interval Estimation for MLEs in Linear Regression....351 13.3 The Logistic Regression Algorithm....356 14 Data Modelling in Practice....366 14.1 Data Description and Preprocessing....366 14.1.1 Loading the Dataset....366 14.1.2 Data Preprocessing....368 14.1.3 Dataset Splitting....373 14.2 Feature Selection and Dimensionality Reduction....375 14.2.1 A Simple Feature Selection Method....375 14.2.2 Feature Reduction via Principal Component Analysis....379 14.3 Model Selection....381 14.3.1 Overview of the Model Selection Procedure....381 14.3.2 Model Evaluation via Validation Set....382 14.3.3 Ridge Regression....383 14.3.4 Understanding the Trade-Off Between Bias and Variance....384 14.3.5 Final Model Evaluation....386 14.4 More About Model Evaluation for Classification....389 14.5 Early Stopping....394 Appendix A Setting up Python and Jupyter Notebook....397 A.1 Install Jupyter Notebook Locally....397 A.2 Verifying the Setup....397 Appendix References....399 ....399 Index....400