Introduction to Data Science: Data Analysis and Prediction Algorithms with R

 Preface....10 Acknowledgements....11 Introduction....12 Case studies....12 Who will find this book useful?....12 What does this book cover?....13 What is not covered by this book?....13 I R....14 Installing R and RStudio....15 Installing R....15 Installing RStudio....22 Getting Started with R and RStudio....28 Why R?....28 The R console....28 Scripts....29 RStudio....30 Installing R packages....39 R Basics....42 Case study: US Gun Murders....42 The very basics....44 Exercises....48 Data types....49 Data frames....49 Exercises....54 Vectors....55 Coercion....57 Exercises....58 Sorting....58 Exercise....61 Vector arithmetics....61 Exercises....63 Indexing....63 Exercises....65 Basic plots....66 Exercises....68 Programming basics....69 Conditional expressions....69 Defining functions....71 Namespaces....71 For-loops....72 Vectorization and functionals....74 Exercises....75 The tidyverse....76 Tidy data....76 Exercises....77 Manipulating data frames....78 Exercises....79 The pipe: %>%....80 Exercises....81 Summarizing data....82 Sorting data frames....85 Exercises....86 Tibbles....87 The dot operator....90 do....91 The purrr package....92 Tidyverse conditionals....93 Exercises....94 Importing data....95 Paths and the working directory....96 The readr and readxl packages....98 Exercises....100 Downloading files....100 R-base importing functions....101 Text versus binary files....101 Unicode versus ASCII....102 Organizing Data with Spreadsheets....102 Exercises....103 II Data Visualization....104 Introduction to data visualization....105 ggplot2....109 The components of a graph....109 ggplot objects....111 Geometries....111 Aesthetic mappings....113 Layers....114 Global versus local aesthetic mappings....115 Scales....117 Labels and titles....118 Categories as colors....118 Annotation, shapes, and adjustments....119 Add-on packages....120 Putting it all together....121 Quick plots with qplot....122 Grids of plots....123 Exercises....124 Visualizing data distributions....127 Variable types....127 Case study: describing student heights....128 Distribution function....128 Cumulative distribution functions....129 Histograms....130 Smoothed density....131 Exercises....136 The normal distribution....139 Standard units....141 Quantile-quantile plots....142 Percentiles....143 Boxplots....144 Stratification....145 Case study: describing student heights (continued)....145 Exercises....147 ggplot2 geometries....148 Exercises....157 Data visualization in practice....159 Case study: new insights on poverty....159 Scatterplots....161 Faceting....162 Time series plots....165 Data transformations....168 Visualizing multimodal distributions....172 Comparing multiple distributions with boxplots and ridge plots....172 The ecological fallacy and importance of showing the data....185 Data visualization principles....188 Encoding data using visual cues....188 Know when to include 0....190 Do not distort quantities....194 Order categories by a meaningful value....195 Show the data....197 Ease comparisons....200 Think of the color blind....205 Plots for two variables....205 Encoding a third variable....208 Avoid pseudo-three-dimensional plots....210 Avoid too many significant digits....211 Know your audience....212 Exercises....212 Case study: impact of vaccines on battling infectious diseases....216 Exercises....219 Robust summaries....220 Outliers....220 Median....221 The inter quartile range (IQR)....222 Tukey's definition of an outlier....222 Median absolute deviation....223 Exercises....223 Case study: self-reported student heights....224 III Statistics with R....227 Introduction to Statistics with R....228 Probability....229 Discrete probability....229 Monte Carlo simulations for categorical data....230 Independence....232 Conditional probabilities....233 Addition and multiplication rules....233 Combinations and permutations....234 Examples....238 Infinity in practice....241 Exercises....242 Continuous probability....243 Theoretical continuous distributions....244 Monte Carlo simulations for continuous variables....247 Continuous distributions....248 Exercises....249 Random variables....251 Random variables....251 Sampling models....252 The probability distribution of a random variable....252 Distributions versus probability distributions....254 Notation for random variables....255 The expected value and standard error....255 Central Limit Theorem....258 Statistical properties of averages....259 Law of large numbers....260 Exercises....260 Case study: The Big Short....262 Exercises....267 Statistical Inference....268 Polls....268 Populations, samples, parameters and estimates....271 Exercises....273 Central Limit Theorem in practice....274 Exercises....278 Confidence intervals....279 Exercises....283 Power....284 p-values....284 Association Tests....285 Exercises....290 Statistical models....292 Poll aggregators....292 Data driven models....297 Exercises....299 Bayesian statistics....301 Bayes Theorem simulation....302 Hierarchical models....304 Exercises....306 Case study: Election forecasting....307 Exercise....319 The t-distribution....320 Regression....323 Case study: is height hereditary?....323 The correlation coefficient....324 Conditional expectations....328 The regression line....331 Exercises....337 Linear Models....338 Case Study: Moneyball....338 Confounding....346 Least Squared Estimates....350 Exercises....355 Linear regression in the tidyverse....356 Exercises....361 Case study: Moneyball (continued)....361 The regression fallacy....368 Measurement error models....371 Exercises....373 Association is not causation....375 Spurious correlation....375 Outliers....377 Reversing cause and effect....379 Confounders....380 Simpson's paradox....384 Exercises....385 IV Data Wrangling....387 Introduction to Data Wrangling....388 Reshaping data....389 gather....389 spread....390 separate....391 unite....393 Exercises....394 Joining tables....396 Joins....397 Binding....400 Set operators....401 Exercises....403 Web Scraping....404 HTML....405 The rvest package....406 CSS selectors....407 JSON....408 Exercises....409 String Processing....411 The stringr package....411 Case study 1: US murders data....413 Case study 2: self reported heights....414 How to escape when defining strings....416 Regular expressions....417 Search and replace with regex....423 Testing and improving....425 Trimming....427 Changing lettercase....428 Case study 2: self reported heights (continued)....428 String splitting....431 Case study 3: extracting tables from a PDF....433 Recoding....435 Exercises....437 Parsing Dates and Times....440 The date data type....440 The lubridate package....441 Exercises....444 Text mining....446 Case study: Trump tweets....446 Text as data....448 Sentiment analysis....452 Exercises....456 V Machine Learning....458 Introduction to Machine Learning....459 Notation....459 An example....460 Exercises....462 Evaluation Metrics....462 Exercises....475 Conditional probabilities and expectations....476 Exercises....478 Case study: is it a 2 or a 7?....478 Smoothing....483 Bin smoothing....485 Kernels....487 Local weighted regression (loess)....489 Connecting smoothing to machine learning....495 Exercises....496 Cross validation....498 Motivation with k-nearest neighbors....498 Mathematical description of cross validation....503 K-fold cross validation....504 Exercises....509 Bootstrap....510 Exercises....512 The caret package....514 The caret train functon....514 Cross validation....515 Example: fitting with loess....517 Examples of algorithms....520 Linear regression....520 Exercises....522 Logistic regression....523 Exercises....528 k-nearest neighbors....529 Exercises....530 Generative models....530 Exercises....541 Classification and Regression Trees (CART)....542 Random Forests....555 Exercises....558 Machine learning in practice....561 Preprocessing....562 k-Nearest Neighbor and Random Forest....563 Variable importance....566 Visual assessments....566 Ensembles....567 Exercises....568 Large datasets....570 Matrix algebra....570 Exercises....579 Distance....579 Exercises....584 Dimension reduction....584 Exercises....599 Recommendation systems....600 Exercises....607 Regularization....608 Exercises....614 Matrix factorization....615 Exercises....625 Clustering....630 Hierarchical clustering....631 k-means....634 Heatmaps....636 Filtering features....637 Exercises....638 VI Productivity tools....639 Introduction to productivity tools....640 Accessing the terminal and installing Git....641 Accessing the terminal on a Mac....641 Installing Git on the Mac....642 Installing Git and Git Bash on Windows....645 Accessing the terminal on Windows....648 Organizing with Unix....651 Naming convention....651 The terminal....652 The filesystem....652 Unix commands....656 Some examples....659 More Unix commands....661 Preparing for a data science project....663 Advanced Unix....663 Git and GitHub....668 Why use Git and GitHub?....668 GitHub accounts....668 GitHub repositories....673 Overview of Git....676 Initializing a Git directory....681 Using Git and GitHub in RStudio....683 Reproducible projects with RStudio and R markdown....691 RStudio projects....691 R markdown....695 Organizing a data science project....701