Prompt Engineering for Generative AI: Future-Proof Inputs for Reliable AI Outputs

PrefaceSoftware Requirements for This BookConventions Used in This BookUsing Code ExamplesO’Reilly Online LearningHow to Contact UsAcknowledgments1. The Five Principles of PromptingOverview of the Five Principles of Prompting1. Give Direction2. Specify Format3. Provide Examples4. Evaluate Quality5. Divide LaborSummary2. Introduction to Large Language Models for Text GenerationWhat Are Text Generation Models?Vector Representations: The Numerical Essence of LanguageTransformer Architecture: Orchestrating Contextual RelationshipsProbabilistic Text Generation: The Decision MechanismHistorical Underpinnings: The Rise of Transformer ArchitecturesOpenAI’s Generative Pretrained TransformersGPT-3.5-turbo and ChatGPTGPT-4Google’s GeminiMeta’s Llama and Open SourceLeveraging Quantization and LoRAMistralAnthropic: ClaudeGPT-4V(ision)Model ComparisonSummary3. Standard Practices for Text Generation with ChatGPTGenerating ListsHierarchical List GenerationWhen to Avoid Using Regular ExpressionsGenerating JSONYAMLFiltering YAML PayloadsHandling Invalid Payloads in YAMLDiverse Format Generation with ChatGPTMock CSV DataExplain It like I’m FiveUniversal Translation Through LLMsAsk for ContextText Style UnbundlingIdentifying the Desired Textual FeaturesGenerating New Content with the Extracted FeaturesExtracting Specific Textual Features with LLMsSummarizationSummarizing Given Context Window LimitationsChunking TextBenefits of Chunking TextScenarios for Chunking TextPoor Chunking ExampleChunking StrategiesSentence Detection Using SpaCyBuilding a Simple Chunking Algorithm in PythonSliding Window ChunkingText Chunking PackagesText Chunking with TiktokenEncodingsUnderstanding the Tokenization of StringsEstimating Token Usage for Chat API CallsSentiment AnalysisTechniques for Improving Sentiment AnalysisLimitations and Challenges in Sentiment AnalysisLeast to MostPlanning the ArchitectureCoding Individual FunctionsAdding TestsBenefits of the Least to Most TechniqueChallenges with the Least to Most TechniqueRole PromptingBenefits of Role PromptingChallenges of Role PromptingWhen to Use Role PromptingGPT Prompting TacticsAvoiding Hallucinations with ReferenceGive GPTs “Thinking Time”The Inner Monologue TacticSelf-Eval LLM ResponsesClassification with LLMsBuilding a Classification ModelMajority Vote for ClassificationCriteria EvaluationMeta PromptingSummary4. Advanced Techniques for Text Generation with LangChainIntroduction to LangChainEnvironment SetupChat ModelsStreaming Chat ModelsCreating Multiple LLM GenerationsLangChain Prompt TemplatesLangChain Expression Language (LCEL)Using PromptTemplate with Chat ModelsOutput ParsersLangChain EvalsOpenAI Function CallingParallel Function CallingFunction Calling in LangChainExtracting Data with LangChainQuery PlanningCreating Few-Shot Prompt TemplatesFixed-Length Few-Shot ExamplesFormatting the ExamplesSelecting Few-Shot Examples by LengthLimitations with Few-Shot ExamplesSaving and Loading LLM PromptsData ConnectionDocument LoadersText SplittersText Splitting by Length and Token SizeText Splitting with Recursive Character SplittingTask DecompositionPrompt ChainingSequential Chainitemgetter and Dictionary Key ExtractionStructuring LCEL ChainsDocument ChainsStuffRefineMap ReduceMap Re-rankSummary5. Vector Databases with FAISS and PineconeRetrieval Augmented Generation (RAG)Introducing EmbeddingsDocument LoadingMemory Retrieval with FAISSRAG with LangChainHosted Vector Databases with PineconeSelf-QueryingAlternative Retrieval MechanismsSummary6. Autonomous Agents with Memory and ToolsChain-of-ThoughtAgentsReason and Act (ReAct)Reason and Act ImplementationUsing ToolsUsing LLMs as an API (OpenAI Functions)Comparing OpenAI Functions and ReActUse Cases for OpenAI FunctionsReActUse Cases for ReActAgent ToolkitsCustomizing Standard AgentsCustom Agents in LCELUnderstanding and Using MemoryLong-Term MemoryShort-Term MemoryShort-Term Memory in QA Conversation AgentsMemory in LangChainPreserving the StateQuerying the StateConversationBufferMemoryOther Popular Memory Types in LangChainConversationBufferWindowMemoryConversationSummaryMemoryConversationSummaryBufferMemoryConversationTokenBufferMemoryOpenAI Functions Agent with MemoryAdvanced Agent FrameworksPlan-and-Execute AgentsTree of ThoughtsCallbacksGlobal (Constructor) CallbacksRequest-Specific CallbacksThe Verbose ArgumentWhen to Use Which?Token Counting with LangChainSummary7. Introduction to Diffusion Models for Image GenerationOpenAI DALL-EMidjourneyStable DiffusionGoogle GeminiText to VideoModel ComparisonSummary8. Standard Practices for Image Generation with MidjourneyFormat ModifiersArt Style ModifiersReverse Engineering PromptsQuality BoostersNegative PromptsWeighted TermsPrompting with an ImageInpaintingOutpaintingConsistent CharactersPrompt RewritingMeme UnbundlingMeme MappingPrompt AnalysisSummary9. Advanced Techniques for Image Generation with Stable DiffusionRunning Stable DiffusionAUTOMATIC1111 Web User InterfaceImg2ImgUpscaling ImagesInterrogate CLIPSD Inpainting and OutpaintingControlNetSegment Anything Model (SAM)DreamBooth Fine-TuningStable Diffusion XL RefinerSummary10. Building AI-Powered ApplicationsAI Blog WritingTopic ResearchExpert InterviewGenerate OutlineText GenerationWriting StyleTitle OptimizationAI Blog ImagesUser InterfaceSummaryIndexAbout the Authors