Introduction to Generative AI

inside front cover....2 Introduction to Generative AI....3 Copyright....4 dedication....6 contents....7 front matter....14 foreword....14 preface....16 acknowledgments....18 about this book....19 Who should read this book....20 How this book is organized: A road map....21 liveBook discussion forums....23 Other online resources....24 about the author....24 about the cover illustration....27 1 Large language models: The power of AI....28 Evolution of natural language processing....31 The birth of LLMs: Attention is all you need....36 Explosion of LLMs....41 What are LLMs used for?....42 Language modeling....42 Question answering....44 Coding....46 Content generation....48 Logical reasoning....50 Other natural language tasks....52 Where do LLMs fall short?....53 Training data and bias....54 Limitations in controlling machine outputs....58 Sustainability of LLMs....60 Revolutionizing dialogue: Conversational LLMs....62 OpenAI’s ChatGPT....62 Google’s Bard/LaMDA....64 Microsoft’s Bing AI....66 Meta’s LLaMa/Stanford’s Alpaca....69 Summary....71 2 Training large language models....73 How are LLMs trained?....74 Exploring open web data collection....75 Demystifying autoregression and bidirectional token prediction....78 Fine-tuning LLMs....79 The unexpected: Emergent properties of LLMs....80 Quick study: Learning with few examples....81 Is emergence an illusion?....86 What’s in the training data?....87 Encoding bias....87 Sensitive information....93 Summary....96 3 Data privacy and safety with LLMs....98 Safety-focused improvements for LLM generations....99 Post-processing detection algorithms....101 Content filtering or conditional pre-training....103 Reinforcement learning from human feedback....104 Reinforcement learning from AI feedback....108 Navigating user privacy and commercial risks....111 Inadvertent data leakage....112 Best practices when interacting with chatbots....114 Understanding the rules of the road: Data policies and regulations....116 International standards and data protection laws....116 Are chatbots compliant with GDPR?....121 Privacy regulations in academia....123 Corporate policies....124 Summary....126 4 The evolution of created content....127 The rise of synthetic media....128 Popular techniques for creating synthetic media....130 The good and the bad of synthetic media....133 AI or genuine: Detecting synthetic media....135 Generative AI: Transforming creative workflows....139 Marketing applications....139 Artwork creation....143 Intellectual property in the LLM era....148 Copyright law and fair use....148 Open source and licenses....159 Summary....164 5 Misuse and adversarial attacks....166 Cybersecurity and social engineering....167 Information disorder: Adversarial narratives....185 Political bias and electioneering....197 Why do LLMs hallucinate?....202 Misuse of LLMs in the professional world....212 Summary....220 6 Accelerating productivity: Machine-augmented work....222 Using LLMs in the professional space....223 LLMs assisting doctors with administrative tasks....224 LLMs for legal research, discovery, and documentation....228 LLMs augmenting financial investing and bank customer service....231 LLMs as collaborators in creativity....232 LLMs as a programming sidekick....236 LLMs in daily life....241 Generative AI’s footprint on education....251 Detecting AI-generated text....256 How LLMs affect jobs and the economy....263 Summary....266 7 Making social connections with chatbots....268 Chatbots for social interaction....269 Why humans are turning to chatbots for relationship....278 The loneliness epidemic....279 Emotional attachment theory and chatbots....282 The good and bad of human-chatbot relationships....286 Charting a path for beneficial chatbot interaction....296 Summary....306 8 What’s next for AI and LLMs....307 Where are LLM developments headed?....308 Language: The universal interface....309 LLM agents unlock new possibilities....311 The personalization wave....313 Social and technical risks of LLMs....315 Data inputs and outputs....315 Data privacy....318 Adversarial attacks....319 Misuse....323 How society is affected....324 Using LLMs responsibly: Best practices....326 Curating datasets and standardizing documentation....327 Protecting data privacy....330 Explainability, transparency, and bias....332 Model training strategies for safety....339 Enhanced detection....342 Boundaries for user engagement and metrics....345 Humans in the loop....348 AI regulations: An ethics perspective....350 North America overview....351 EU overview....357 China overview....363 Corporate self-governance....366 Toward an AI governance framework....370 Summary....374 9 Broadening the horizon: Exploratory topics in AI....378 The quest for artificial general intelligence....379 AI sentience and consciousness?....390 How LLMs affect the environment....398 The game changer: Open source community....404 Summary....412 references....415 Chapter 1....419 Chapter 2....428 Chapter 3....433 Chapter 4....441 Chapter 5....451 Chapter 6....462 Chapter 7....471 Chapter 8....479 Chapter 9....488 index....492 inside back cover....505