MongoDB in Action: Building on the Atlas Data Platform. 3 Ed

MongoDB 8.0 in Action, Third Edition....1 brief contents....8 contents....10 preface....20 acknowledgments....22 about this book....24 Who should read this book....24 How this book is organized: A road map....24 About the code....26 liveBook discussion forum....27 about the author....28 about the cover illustration....29 Part 1 A database for modern ....31 1 Understanding the world of MongoDB....33 1.1	Examining the document-oriented data model....34 1.2	Scaling data horizontally....36 1.3	Exploring the MongoDB ecosystem....38 1.3.1	Learning the core MongoDB server features....39 1.3.2	Learning MongoDB Atlas concepts....39 1.4	Enhancing the TCMalloc version....41 1.5	Discovering MongoDB Query API....41 2 Getting started with Atlas and MongoDB data....44 2.1	Setting up your first Atlas cluster using Atlas CLI....45 2.1.1	Installing the Atlas CLI....45 2.1.2	Creating an Atlas account....45 2.1.3	Creating an organization....46 2.1.4	Creating an Atlas project....47 2.1.5	Creating a MongoDB Atlas cluster....48 2.1.6	Navigating the Atlas user interface....49 2.2	Loading a sample data set....49 2.3	Adding an IP address to the project access list....50 2.4	Creating a user....51 2.5	Establishing a connection to MongoDB through MongoDB Shell....51 2.6	Managing data with databases, collections, and documents....53 2.6.1	Working with dynamic schema....53 2.6.2	Working with databases....55 2.6.3	Working with collections....56 2.6.4	Working with documents....61 3 Communicating with MongoDB....64 3.1	Interacting via MongoDB Wire Protocol....65 3.2	Discovering mongosh....65 3.2.1	Connecting to MongoDB Atlas....66 3.2.2	Connecting to self-hosted deployments....66 3.2.3	Performing operations....66 3.2.4	Viewing mongosh logs....68 3.2.5	Running scripts in mongosh....68 3.2.6	Configuring mongosh....70 3.2.7	Using .mongoshrc.js....72 3.3	Playing with MongoDB Compass....73 3.4	Connecting using MongoDB drivers....75 3.5	Using the Node.js driver....75 3.6	Employing Python drivers....79 3.6.1	PyMongo....79 3.6.2	Motor....81 3.6.3	PyMongo vs. Motor....83 3.7	Integrating Ruby drivers....83 3.8	Learning Mongoid....85 4 Executing CRUD operations....87 4.1	Connecting to mongosh for CRUD operations....88 4.2	Inserting documents....88 4.3	Updating documents....91 4.3.1	Using update operators....92 4.3.2	Updating many documents....93 4.4	Updating arrays....94 4.4.1	Adding elements to an array....94 4.4.2	Removing elements from an array....96 4.4.3	Updating array elements....97 4.4.4	Updating using array filters....98 4.5	Replacing documents....100 4.6	Reading documents....101 4.6.1	Using logical operators....102 4.6.2	Using comparison operators....104 4.6.3	Working with projections....105 4.6.4	Searching for null values and absent fields....106 4.7	Performing regular-expression searches....107 4.8	Querying arrays....108 4.9	Querying embedded/nested documents....111 4.9.1	Querying on a nested field with dot notation....112 4.9.2	Matching an embedded/nested document....113 4.9.3	Querying an array of embedded documents....113 4.10	Sorting, skipping, and limiting....114 4.10.1	The sort operation....114 4.10.2	The skip operation....114 4.10.3	The limit operation....115 4.11	Deleting documents....115 4.12	Using bulkWrite()....116 4.13	Understanding cursors....118 4.13.1	Using manual iteration....118 4.13.2	Returning an array of all documents....119 4.14	Employing MongoDB Stable API....119 5 Designing a MongoDB schema....122 5.1	Organizing the MongoDB data model....123 5.1.1	Determining the workload of the application....123 5.1.2	Mapping the schema relationship....125 5.1.3	Applying a design pattern....127 5.2	Embedding vs. referencing....128 5.3	Understanding schema design patterns....131 5.3.1	Approximation pattern....132 5.3.2	Archive pattern....132 5.3.3	Attribute pattern....133 5.3.4	Bucket pattern....134 5.3.5	Computed pattern....135 5.3.6	Document Versioning pattern....136 5.3.7	Extended Reference pattern....136 5.3.8	Outlier pattern....137 5.3.9	Polymorphic pattern....138 5.3.10	Preallocation pattern....139 5.3.11	Schema Versioning pattern....140 5.3.12	Subset pattern....141 5.3.13	Tree pattern....142 5.4	Schema validations....143 5.4.1	Specifying JSON schema validation....144 5.4.2	Testing a schema validation rule....145 5.4.3	Modifying schema validator behavior....147 5.4.4	Bypassing schema validation....148 5.5	MongoDB schema antipatterns....149 6 Building aggregation pipelines....151 6.1	Understanding the aggregation framework....152 6.1.1	Writing an aggregation pipeline....153 6.1.2	Viewing the aggregation pipeline stages....154 6.1.3	Using $set and $unset instead of $project....157 6.1.4	Scenarios for $set and $unset operators....158 6.1.5	Scenario for the $project operator....159 6.1.6	Saving the results of aggregation pipelines....159 6.2	Joining collections....161 6.2.1	Creating a MongoDB view using $lookup....162 6.2.2	Using $lookup with $mergeobjects....163 6.3	Deconstructing arrays with $unwind....164 6.4	Working with accumulators....166 6.5	Using the MongoDB Atlas aggregation pipeline builder....167 7 Indexing for query performance....170 7.1	MongoDB query planner....171 7.1.1	Viewing query plan cache information....171 7.1.2	MongoDB plan cache purges....175 7.2	Supported index types....175 7.2.1	Creating single field indexes....176 7.2.2	Understanding compound indexes....181 7.2.3	Using multikey indexes....186 7.2.4	Using text indexes....188 7.2.5	Creating wildcard index....190 7.2.6	Geospatial indexes....192 7.2.7	Hashed indexes....194 7.3	Dropping indexes....195 7.4	MongoDB index attributes....195 7.4.1	Partial indexes....195 7.4.2	Sparse indexes....196 7.4.3	Time-to-live indexes....197 7.4.4	Hidden indexes....199 7.5	Understanding index builds....200 7.5.1	Monitoring in-progress index builds....201 7.5.2	Terminating in-progress index builds....202 7.6	Managing indexes....202 7.6.1	Discovering the $indexStats aggregation pipeline stage....202 7.6.2	Modifying indexes....203 7.6.3	Controlling index use with hint ()....204 7.6.4	Using indexes with $OR queries....204 7.6.5	Using indexes with the $NE, $NIN, and $NOT operators....205 7.6.6	Ensuring that indexes fit in RAM....205 7.6.7	Sorting on multiple fields....206 7.6.8	Introducing covered queries....207 7.7	When to not use an index....207 8 Executing multidocument ACID transactions....210 8.1	WiredTiger storage engine....211 8.1.1	Snapshots and checkpoints....211 8.1.2	Journaling....212 8.1.3	Compression....212 8.1.4	Memory use....212 8.2	Single-document transaction....212 8.3	Defining ACID....213 8.4	Multidocument transactions....214 8.4.1	Differentiating the Core and Callback APIs....214 8.4.2	Using transactions with mongosh....215 8.4.3	Using transactions with the Callback API....217 8.5	MongoDB transaction considerations....224 9 Using replication and sharding....226 9.1	Ensuring data high availability with replication....227 9.1.1	Distinguishing replica set members....227 9.1.2	Electing primary replica-set member....230 9.1.3	Understanding the oplog collection....231 9.2	Understanding change streams....236 9.2.1	Connections for a change stream....237 9.2.2	Changing streams with Node.js....239 9.2.3	Modifying the output of a change stream....240 9.3	Scaling data horizontally through sharding....241 9.3.1	Viewing sharded cluster architecture....242 9.3.2	Creating sharded clusters via Atlas CLI....244 9.3.3	Working with a shard key....246 9.3.4	Choosing a shard key....246 9.3.5	Using a shard-key analyzer....247 9.3.6	Detecting shard-data imbalance or uneven data distribution....252 9.3.7	Resharding a collection....252 9.3.8	Understanding chunk balancing....254 9.3.9	Administrating chunks....255 9.3.10	Automerging chunks....258 9.4	MongoDB 8.0 sharded cluster features....259 9.4.1	Embedding config servers in sharded clusters....260 9.4.2	Moving unsharded collections seamlessly between shards....260 9.4.3	Fragmentation....261 9.4.4	Faster resharding....261 9.4.5	Unsharding collections....262 9.5	Managing data consistency and availability....263 9.5.1	Write Concern....263 9.5.2	Read Concern....265 9.5.3	Read Preference....267 Part 2 MongoDB Atlas data platform....271 10 Delving into Database as a Service....273 10.1	Shared M0 and Flex clusters....274 10.2	Dedicated clusters....277 10.2.1	Atlas clusters for low-traffic applications....278 10.2.2	Atlas clusters for high-traffic applications....279 10.2.3	Autoscaling clusters and storage....279 10.2.4	Customizing Atlas cluster storage....281 10.3	Atlas Global Clusters....283 10.4	Going multiregion with workload isolation....284 10.4.1	Adding electable nodes for high availability....286 10.4.2	Adding read-only nodes for local reads....286 10.4.3	Using analytics nodes for workload isolation....286 10.5	Using predefined replica set tags for querying....287 10.5.1	Routing queries to analytics nodes....288 10.5.2	Isolating normal application secondary reads from analytics nodes....288 10.5.3	Routing local reads for geographically distributed applications....288 10.6	Understanding the Atlas custom Write Concerns....289 11 Carrying out full-text search using Atlas Search....292 11.1	Implementing full-text search....293 11.2	Understanding Apache Lucene....295 11.3	Getting to know Atlas Search....297 11.3.1	Learning Atlas Search architecture....298 11.3.2	Using Atlas Search Nodes....299 11.3.3	Atlas Search indexes....300 11.4	Building an Atlas Search index....304 11.5	Running Atlas Search queries....308 11.5.1	Using the $search aggregation pipeline stage....309 11.5.2	Executing the $searchMeta aggregation pipeline stage....324 11.6	Learning Atlas Search commands....327 11.7	Using Atlas Search Playground....328 12 Learning semantic techniques and Atlas Vector Search....332 12.1	Starting with embeddings....333 12.1.1	Converting text to embeddings....335 12.1.2	Understanding vector databases....338 12.2	Using embeddings with Atlas Vector Search....339 12.2.1	Building an Atlas Vector Search index....340 12.2.2	Selecting a Vector Search source....340 12.2.3	Defining your Vector Search index....342 12.2.4	Creating an Atlas Vector Search index....344 12.3	Running Atlas Vector Search queries....345 12.3.1	Querying with embeddings....346 12.3.2	Using prefiltering with Atlas Vector Search....350 12.4	Executing vector search with programming languages....352 12.4.1	Using vector search with JavaScript....352 12.4.2	Using vector search and prefiltering with Python....353 12.4.3	Using vector search with prefilters in Ruby....355 12.5	Using Atlas Triggers for automated embeddings creation....356 12.6	Workload isolation with vector search dedicated nodes....361 12.7	Improving Atlas Vector Search performance....361 13 Developing AI applications locally with the Atlas CLI....365 13.1	Introducing local Atlas clusters....366 13.2	Creating an Atlas cluster locally with Atlas CLI....367 13.2.1	Configuring Docker....368 13.2.2	Building your first local Atlas cluster....370 13.3	Managing your local Atlas cluster....371 13.3.1	Stopping, starting, checking, and deleting your local cluster....371 13.3.2	Loading a sample data set....374 13.4	Diving into a local Atlas cluster....376 13.4.1	Displaying processes....379 13.4.2	Executing into the container....380 13.5	Creating search indexes....382 13.5.1	Executing full-text search locally....382 13.5.2	Executing vector search locally....385 14 Building retrieval-augmented generation AI chatbots....388 14.1	Gaining insight into retrieval-augmented generation....389 14.2	Embedding LangChain in the RAG ecosystem....390 14.3	Introducing the MongoDB Atlas Vector Search RAG template....392 14.4	Getting started with AI chatbots....392 14.4.1	Describing LangChain capabilities....393 14.4.2	Starting with the LangChain CLI....394 14.5	Creating an AI-powered MongoDB chatbot....394 14.5.1	Setting up a new application....395 14.5.2	Inserting embeddings into MongoDB Atlas....397 14.5.3	Creating an Atlas Vector Search index....404 14.5.4	Testing a chatbot with LangServe....406 14.5.5	Communicating programmatically with a chatbot....411 15 Building event-driven applications....414 15.1	Understanding event-driven technology....415 15.2	Examining the concepts of stream processing....417 15.2.1	Differentiating event time and processing time....417 15.2.2	Using time windows....418 15.3	Starting with Atlas Stream Processing....418 15.4	Exploring Atlas Stream Processing....420 15.4.1	Discovering Atlas Stream Processing components....420 15.4.2	Understanding Atlas Stream Processing capabilities....422 15.5	Structuring a stream processor aggregation pipeline....424 15.5.1	Taking a deep dive into the $source aggregation stage....425 15.5.2	Using the stream processor $validate aggregation stage....430 15.5.3	Viewing all supported aggregation pipeline stages....430 15.6	Mastering Atlas Stream Processing....432 15.6.1	Adopting new stream processor methods....432 15.6.2	Using the Atlas CLI with stream processing....432 15.6.3	Creating your first stream processor....434 15.6.4	Learning the anatomy of a stream processor....436 15.6.5	Setting up a streams Connection Registry....449 15.6.6	Ensuring persistence in stream processing....450 15.7	Controlling the stream processing flow....455 15.7.1	Capturing the state....455 15.7.2	Using a dead-letter queue....455 15.8	Securing Atlas Stream Processing....456 15.8.1	Discovering new roles....457 15.8.2	Learning new privilege actions....457 15.8.3	Protecting network access....457 15.8.4	Auditing events....457 16 Optimizing data processing with Atlas Data Federation....460 16.1	Querying Amazon S3 and Azure Blob Store data via the Query API....460 16.2	Learning Atlas Data Federation architecture....462 16.3	Deploying an Atlas Federated Database instance....463 16.4	Limitations of Atlas Data Federation....464 16.5	Charges for Atlas Data Federation....465 17 Archiving online with Atlas Online Archive....467 17.1	Archiving your data....468 17.1.1	Seeing how Atlas archives data....468 17.1.2	Deleting archived documents....470 17.2	Initializing Online Archive....470 17.3	Connecting and querying Online Archive....474 17.4	Restoring archived data....476 18 Querying Atlas using SQL....479 18.1	Introducing the Atlas SQL interface....480 18.2	Connecting to the Atlas SQL interface....481 18.2.1	Enabling the interface....481 18.2.2	Accessing the interface....482 18.3	Querying MongoDB using SQL....483 18.3.1	Aggregation pipeline Atlas SQL syntax....483 18.3.2	Short-form Atlas SQL syntax....484 18.3.3	UNWIND and FLATTEN with Atlas SQL....485 19 Creating charts, database triggers, and functions....489 19.1	Visualizing data with Atlas Charts....490 19.1.1	Using natural language to build visualizations....492 19.1.2	Using billing dashboards....495 19.2	Atlas Application Services....496 19.2.1	Triggering server-side logic with Atlas Database Triggers....497 19.2.2	Writing Atlas Functions....504 Part 3 MongoDB security and operations....509 20 Understanding Atlas and MongoDB security features....511 20.1	Understanding the shared responsibility model....512 20.2	Managing authentication....515 20.2.1	Choosing an Atlas database cluster authentication method....516 20.2.2	Integrating with HashiCorp Vault....517 20.2.3	Choosing the authentication method....518 20.3	Handling authorization....518 20.3.1	Understanding the principle of least privilege....519 20.3.2	Differentiating Atlas user roles....519 20.3.3	Using MongoDB RBAC....520 20.4	Auditing Atlas....522 20.5	Encrypting data in Atlas....525 20.5.1	Encrypting data in transit....525 20.5.2	Encrypting data at rest....526 20.5.3	Managing encryption keys yourself....526 20.5.4	Encrypting during processing....527 20.6	Securing the network....530 20.6.1	Using an IP access list....530 20.6.2	Peering networks....531 20.6.3	Using private endpoints....532 20.7	Implementing defense in depth....533 21 Operational excellence with Atlas....537 21.1	Crafting backup strategies and practices....538 21.1.1	Discovering Atlas backup methods....538 21.1.2	Restoring an Atlas cluster....542 21.2	Inspecting the performance of your Atlas cluster....546 21.2.1	Finding slow queries....546 21.2.2	Improving your schema....550 21.2.3	Using native MongoDB diagnostic commands....551 21.3	Alerting and logging....554 21.3.1	Setting alert conditions....554 21.3.2	Logging in Atlas....556 21.4	Upgrading your Atlas cluster....557 index....561