Ultimate Neural Network Programming with Python: Create Powerful Modern AI Systems by Harnessing Neural Networks with Python, Keras, and TensorFlow

Cover Page....2 Title Page....3 Copyright Page....4 Dedication Page....5 About the Author....6 About the Technical Reviewers....7 Welcome note....9 Acknowledgements....13 Preface....14 Errata....17 Table of Contents....20 1. Understanding AI History....28 Structure....28 Evolution of AI....28 The early history of AI....28 The most crucial development in the History of AI....31 AI started evolving into new fields....32 AI starts taking its modern form....32 Understanding Intelligent Behavior....33 AI beats humans at chess....33 AI learning reasoning and language....34 AI starts playing poker....35 Conquering GO and Dota 2....36 An experience with ChatGPT....37 Difference between Artificial Intelligence, Machine Learning, and Deep Learning....39 Formally defining AI terms....40 Learning representations from data....42 Sub-Fields of AI....44 Artificial Intelligence (AI)....44 Machine Learning (ML)....45 Deep Learning (DL)....45 Early Models of Neuron-Inspired Networks....46 Understanding biological neurons....46 McCulloch-Pitts model of a neuron....47 Multilayer Perceptron (MLP)....48 Conclusion....52 2. Setting up Python Workflow for AI Development....54 Structure....54 Setting up Python Environment....56 Installing Python....56 Getting Anaconda for Data Science Environment Setup....57 Setting up a Virtual Environment....57 Installing packages....58 Setting up VS Code....58 Installing Git....60 Setting up GitHub with VS Code....61 Concepts of OOPS....65 Encapsulation....67 Accessing Variables....69 Inheritance....71 Conclusion....76 3. Python Libraries for Data Scientists....77 Structure....77 Web Scraping....77 Regex....83 Multi-Threading and Multi-Processing....91 Multi-Threading....91 Multi-Processing....93 Pandas Basics....95 Conclusion....109 4. Foundational Concepts for Effective Neural Network Training....111 Structure....111 Activation Functions....111 RBF, Universal Approximators, and Curse of Dimensionality....115 Radial Bias Function....115 Neural Networks are universal approximators....117 The curse of dimensionality....118 Overfitting, Bias-Variance, and Generalization....120 Overfitting problem....120 Regularization and effective parameters....122 Dropout....124 Early stopping and validation set....124 Bias-Variance trade-off....125 Generalization....126 Conclusion....127 5. Dimensionality Reduction, Unsupervised Learning and Optimizations....129 Structure....129 Dimensionality reduction....129 Principal component analysis (PCA)....129 T-SNE....132 Non-linear PCA....137 Unsupervised learning....137 Clustering....138 Semi-supervised learning....144 Generalizing active learning to multi-class....147 Self-supervised learning....150 Version space....153 Understanding optimization through SVM....158 Conclusion....165 6. Building Deep Neural Networks from Scratch....167 Structure....167 Coding neurons....167 A single neuron....167 Layer of neurons....168 Understanding lists, arrays, tensors, and their operations....171 Dot product and vector addition....172 Cross-product, transpose, and order....173 Understanding neural networks through NumPy....174 Neural networks using NumPy....174 Processing batch of data....175 Creating a multi-layer network....176 Dense layers....177 Activation functions....181 Calculating loss through categorical cross-entropy loss....187 Calculating accuracy s....194 Conclusion....197 7. Derivatives, Backpropagation, and Optimizers....199 Structure....199 Weights Optimization....199 Derivatives....204 Partial Derivatives....208 Backpropagation....209 Optimizers: SGD, Adam, and so on....220 Gradient-based optimization....221 Momentum-based optimization....222 RMSProp....222 Adam....223 Conclusion....242 8. Understanding Convolution and CNN Architectures....243 Structure....243 Intricacies of CNN....243 Local Patterns and Global Patterns....244 Spatial Hierarchies and Abstraction....244 Convolution Operation and Feature Maps....246 Pooling....248 Padding....248 Stride....249 Introduction to CNN-based Networks....253 Understanding the Complete Flow of CNN-based Network....254 VGG16....257 Inception Module: Naïve and Improved Version....258 ResNet....261 Other Variants of ResNet....263 FractalNet and DenseNet....264 Scaling Conv Networks: Efficient Net Architecture....266 Different Types of Convolutions....268 Depth-Separable Convolution....268 Conclusion....270 9. Understanding Basics of TensorFlow and Keras....272 Structure....272 A Brief Look at Keras....272 Understanding TensorFlow Internals....278 Tensors....278 Computational Graphs....281 Operations (Ops)....282 Automatic Differentiation....282 Sessions....283 Variables....283 Eager Execution....284 Layers and Models (Keras)....285 TensorFlow vs. PyTorch vs. Theano....285 TensorFlow vs. PyTorch....285 TensorFlow vs. Theano....286 TensorFlow: Layers, Activations, and More....287 Types of Layers....287 Dense Layer (Fully Connected Layer)....287 Convolution Layer....289 Max Pooling Layer....289 Dropout Layer....289 Recurrent Layer (LSTM)....290 Embedding Layer....290 Flatten Layer....290 Batch Normalization Layer....290 Global Average Pooling Layer....291 Upsampling/Transposed Convolution Layer....291 Activation Functions....291 Optimizers....294 Weight Initialization....294 Loss Functions....296 Multi-Input Single-Output Network with Custom Callbacks....297 Conclusion....302 10. Building End-to-end Image Segmentation Pipeline....303 Structure....303 Fine-tuning and Interpretability....303 Power of Fine-Tuning in Deep Learning....303 SHAP - An Intuitive Way to Interpret Machine Learning Models....304 Structuring Deep Learning Code....307 Project Structure....307 Python modules and packages....308 Documentation....309 Unit testing....310 Debugging....311 Logging....313 Building End-to-end Segmentation Pipeline....315 UNet and Attention Gates....316 Config....319 Dataloader....319 Model building....323 Understanding Attention block....324 Executor....332 Utils....335 Evaluation....338 main....341 Conclusion....345 11. Latest Advancements in AI....346 Structure....346 Transformers: Improving NLP Using Attention....346 Recurrent Neural Network (RNN)....346 Long-Short Term Memory (LSTM)....347 Self-Attention....348 Example to understand the concept:....348 Understanding Key, Query, and Value....350 Example to understand the concept:....350 Transformer Architecture....354 ChatGPT/GPT Overview....356 Object Detection: Understanding YOLO....357 Object Detector Architecture Breakdown....360 Backbone, Neck, and Head....361 Bag of Freebies (BoF)....363 CmBN: Cross-mini-Batch Normalization....365 Bag of Specials (BoS)....366 Cross-Stage Partial (CSP) Connection....367 YOLO A rchitecture S election....368 Spatial Pyramid Pooling (SPP)....369 PAN Path — Aggregation Block....370 Spatial Attention Module (SAM)....371 Image Generation: GAN’s and Diffusion models....372 Generative Adversarial Networks....372 Generative Discriminative models....373 Variational Autoencoders....375 GANs....376 Diffusion Models....380 DALL-E 2 Architecture....381 The Encoder: Prior Diffusion Model....383 The Decoder: GLIDE....385 Conclusion....386 Index....387