Node.js Design Patterns: Level up your Node.js skills and design production-grade applications using proven techniques. 4 Ed

Cover....1 Title Page....2 Copyright and Credits....3 Forewords....4 Contributors....8 Table of Contents....12 Preface....26 Chapter 1: The Node.js Platform....38 The Node.js philosophy....39 Small core....40 Small modules....40 Small surface area....42 Simplicity and pragmatism....42 How Node.js works....43 I/O is often the bottleneck....44 Blocking I/O....44 Non-blocking I/O....45 Event demultiplexing....46 The reactor pattern....48 libuv, the I/O engine of Node.js....50 The complete recipe for Node.js....51 JavaScript in Node.js....52 Run the latest JavaScript with confidence....52 The module system....53 Full access to operating system services....53 Running native code....54 Node.js and TypeScript....55 Using TypeScript with Node.js....55 The @types/node package....56 Summary....57 Chapter 2: The Module System....58 The need for modules....59 Module systems in JavaScript and Node.js....59 The revealing module pattern....60 ES modules....62 Using ES modules in Node.js....62 The ES module syntax....64 Named exports and imports....64 Default exports and imports....66 Mixed exports....67 Module identifiers....69 Static and dynamic imports....70 The module resolution algorithm....72 Module loading in depth....74 Loading phases....74 Read-only live bindings....76 Circular dependencies....77 Modules that modify other modules....83 How monkey patching affects type safety in TypeScript projects....89 CommonJS modules....90 ES modules and CommonJS—differences and interoperability....91 Strict mode....91 Top-level await....92 Behavior of `this`....92 Missing references in ES modules....93 Import interoperability....94 Import CommonJS modules from ES modules....94 Import ES modules from CommonJS....96 Importing JSON files....97 Using modules in TypeScript....99 The role of the TypeScript compiler....99 Configuring the module output format....101 Input module syntax and output emission....101 Module resolution....101 Summary....102 Chapter 3: Callbacks and Events....104 The Callback pattern....105 The continuation-passing style....105 Synchronous CPS....106 Asynchronous CPS....107 Non-CPS callbacks....110 Synchronous or asynchronous?....110 Writing an inconsistent function....110 Unleashing Zalgo....111 Using synchronous APIs....113 Guaranteeing asynchronicity with deferred execution....114 Node.js callback conventions....117 The callback is the last argument....117 Any error always comes first....117 Propagating errors....118 Avoiding uncaught exceptions....119 The Observer pattern....121 The EventEmitter....121 Creating and using the EventEmitter....122 Propagating errors....123 Making any object observable....124 The risk of memory leaks....125 Synchronous and asynchronous events....127 EventEmitter versus callbacks....128 Combining callbacks and events....129 Summary....133 Exercises....134 Chapter 4: Asynchronous Control Flow Patterns with Callbacks....136 The challenges of asynchronous programming....137 Creating a simple web spider....137 Callback hell....140 Callback best practices....142 Callback discipline....143 Applying the callback discipline....143 Control flow patterns....146 Sequential execution....146 Executing a known set of tasks in sequence....146 Sequential iteration....147 Concurrent execution....152 Web spider version 3....154 The pattern....155 Fixing race conditions with concurrent tasks....156 Limited concurrent execution....158 Limiting concurrency....159 Globally limiting concurrency....161 Summary....167 Exercises....167 Chapter 5: Asynchronous Control Flow Patterns with Promises and Async/Await....170 Promises....171 What is a promise?....171 Promises/A+ and thenables....174 The promise API....175 Creating a promise....177 Promisification....178 Sequential execution and iteration....180 Concurrent execution....184 Limited concurrent execution....185 Implementing the TaskQueue class with promises....185 Updating the web spider....187 Lazy promises....190 Async/await....194 Async functions and the await expression....194 Top-level await....195 Error handling with async/await....196 A unified try...catch experience....197 The “return” versus “return await” trap....198 Sequential execution and iteration....199 Antipattern – using async/await with Array.forEach for serial execution....201 Concurrent execution....201 Limited concurrent execution....203 The problem with infinite recursive promise resolution chains....204 Summary....207 Exercises....208 Chapter 6: Coding with Streams....210 Discovering the importance of streams....212 Buffering versus streaming....212 Spatial efficiency....213 Gzipping using a buffered API....214 Gzipping using streams....215 Time efficiency....215 Composability....220 Adding client-side encryption....221 Adding server-side decryption....222 Getting started with streams....223 Anatomy of streams....224 Readable streams....224 Reading from a stream....224 Implementing Readable streams....227 Writable streams....232 Writing to a stream....232 Backpressure....234 Implementing Writable streams....236 Duplex streams....238 Transform streams....239 Implementing Transform streams....240 Filtering and aggregating data with Transform streams....243 PassThrough streams....247 Observability....247 Late piping....248 Lazy streams....251 Connecting streams using pipes....252 Pipes and error handling....253 Better error handling with pipeline()....254 Asynchronous control flow patterns with streams....255 Sequential execution....256 Unordered concurrent execution....258 Implementing an unordered concurrent stream....258 Implementing a URL status monitoring application....260 Unordered limited concurrent execution....262 Ordered concurrent execution....264 Piping patterns....266 Combining streams....266 Implementing a combined stream....268 Forking streams....270 Implementing a multiple checksum generator....271 Merging streams....272 Merging text files....272 Multiplexing and demultiplexing....274 Building a remote logger....275 Multiplexing and demultiplexing object streams....279 Readable stream utilities....280 Mapping and transformation....280 Filtering and iteration....281 Searching and evaluation....281 Limiting and reducing....281 Web Streams....283 Converting Node.js streams to Web Streams....284 Converting Web Streams to Node.js streams....284 Stream consumer utilities....286 Summary....288 Exercises....288 Chapter 7: Creational Design Patterns....290 Factory....291 Decoupling object creation and implementation....292 A mechanism to enforce encapsulation....293 Building a simple code profiler....294 In the wild....297 Builder....297 Implementing a URL object builder....302 In the wild....305 Revealing Constructor....308 Building an immutable buffer....309 In the wild....311 Singleton....312 Wiring modules....315 Singleton dependencies....316 Dependency Injection....319 Summary....323 Exercises....324 Chapter 8: Structural Design Patterns....326 Proxy....326 Techniques for implementing proxies....327 Object composition....329 Object augmentation....332 The built-in Proxy object....333 A comparison of the different proxying techniques....336 Creating a logging Writable stream....337 Change Observer with Proxy....338 In the wild....341 Decorator....341 Techniques for implementing decorators....341 Composition....342 Object decoration....343 Decorating with the Proxy object....344 Decorating a Level database....346 Introducing Level and LevelDB....346 Implementing a Level plugin....346 In the wild....348 The decorator proposal for ECMAScript....349 The line between Proxy and Decorator....350 Adapter....351 Using Level through the filesystem API....351 In the wild....354 Summary....355 Exercises....355 Chapter 9: Behavioral Design Patterns....358 Strategy....359 Multi-format configuration objects....362 In the wild....367 State....368 Implementing a basic failsafe socket....369 In the wild....377 Template....378 A configuration manager template....379 In the wild....381 Iterator....381 The iterator protocol....382 The iterable protocol....385 Iterators and iterables as a native JavaScript interface....387 Implementing the iterable protocol on iterators....389 Iterator utilities....390 Generators....396 Generators in theory....396 A simple generator function....397 Controlling a generator iterator....398 How to use generators in place of iterators....399 Async iterators....401 Async generators....405 Async iterators and Node.js streams....406 Async iterator utilities....407 In the wild....409 Middleware....410 Middleware in Express....410 Middleware as a pattern....411 Creating a middleware framework for ZeroMQ....412 The middleware manager....412 Implementing the middleware to process messages....415 Using the ZeroMQ middleware framework....416 In the wild....418 Command....419 The Task pattern....420 A more complex command....420 In the wild....424 Summary....425 Exercises....426 Chapter 10: Testing: Patterns and Best Practices....428 An introduction to software testing....429 Definitions....430 System under test....430 Arrange, Act, Assert....430 Code coverage....431 Test doubles: Stubs, spies, and mocks....431 Test-driven development....432 Behavior-driven development....433 Continuous integration....434 Continuous delivery and continuous deployment....434 Types of tests....435 Unit tests....436 Integration tests....437 End-to-end tests....437 Other types of tests....439 The testing pyramid....440 Writing tests with Node.js....441 Our first unit test....441 The Node.js test runner....444 Our first test with the Node.js test runner....445 Organizing tests....446 Subtests....448 Subtest concurrency....449 Parametrized test cases....450 Test suites....452 Test runner tips and tricks....453 Watch mode....453 Targeted test execution with custom glob patterns....454 Test reporters....456 Collecting code coverage....457 Using the test runner with TypeScript....460 Writing unit tests....460 Testing asynchronous code....461 Mocking....467 Creating spies with mock.fn()....467 Mocking HTTP requests with the built-in test mock....469 Mocking HTTP requests with Undici....472 Mocking Node.js core modules....475 Mocking other dependencies....478 Problems with mocking imports....482 Mocking imports versus dependency injection....483 Writing integration tests....486 Testing with a local database....486 Testing a web application....494 Setting up the project....495 Writing E2E tests....504 The application structure....505 Home page....505 Sign-in and sign-up forms....506 Event Page....506 My reservations....507 The user flow....507 Browser automation....509 Writing an E2E test with Playwright....510 Setting up a new Playwright project....510 Understanding the Playwright API....512 Understanding timeouts....516 Testing our user flow....516 Summary....521 Exercises....521 Chapter 11: Advanced Recipes....524 Dealing with asynchronously initialized components....525 The issue with asynchronously initialized components....525 Local initialization check....527 Delayed startup....528 Pre-initialization queues....529 Using the State pattern....532 In the wild....535 Asynchronous request batching and caching....535 What’s asynchronous request batching?....535 Optimal asynchronous request caching....537 An API server without caching or batching....539 Batching and caching with promises....542 Batching requests....542 Batching and caching requests....544 Canceling asynchronous operations....546 A basic recipe for creating cancelable functions....546 Wrapping asynchronous invocations....548 Cancelable async functions with AbortController....549 Running CPU-bound tasks....553 Solving the subset sum problem....553 Interleaving with setImmediate....557 Interleaving the steps of the subset sum algorithm....557 Considerations on the interleaving approach....559 Using external processes....559 Delegating the subset sum task to an external process....560 Considerations for the multi-process approach....566 Using worker threads....567 Running the subset sum task in a worker thread....568 Running CPU-bound tasks in production....571 Summary....571 Exercises....571 Chapter 12: Scalability and Architectural Patterns....574 An introduction to application scaling....575 Scaling Node.js applications....575 The three dimensions of scalability....575 X-axis: cloning....576 Y-axis: decomposing by service or functionality....577 Z-axis: splitting by data partition....577 Combining strategies across the scale cube....577 Cloning and load balancing....578 The cluster module....579 Notes on the behavior of the cluster module....580 Building a simple HTTP server....581 Scaling with the cluster module....582 Resiliency and availability with the cluster module....584 Zero-downtime restart....586 Dealing with stateful communications....589 Sharing the state across multiple instances....590 Sticky load balancing....591 Scaling with a reverse proxy....592 Load balancing with Nginx....594 Dynamic horizontal scaling....598 Using a service registry....599 Implementing a dynamic load balancer....600 Peer-to-peer load balancing....607 Implementing an HTTP client that can balance requests across multiple servers....609 Scaling applications using containers....611 What is a container?....611 Creating and running a container with Docker....612 What is Kubernetes?....615 Deploying and scaling an application on Kubernetes....617 Decomposing complex applications....620 Monolithic architecture....620 The microservice architecture....622 An example of a microservice architecture....623 Microservices: advantages and disadvantages....624 Integration patterns in a microservice architecture....625 The API proxy....626 API orchestration....627 Integration with a message broker....630 Summary....632 Exercises....632 Chapter 13: Messaging and Integration Patterns....634 Fundamentals of a messaging system....635 One way versus request/reply patterns....635 Message types....636 Command messages....636 Event messages....637 Document messages....637 Push versus pull delivery semantics....638 Pull delivery (consumer-initiated)....638 Push delivery (producer-initiated)....638 Choosing between pull and push delivery....638 Asynchronous messaging, queues, and streams....639 Peer-to-peer or broker-based messaging....641 Publish/Subscribe pattern....642 Building a minimalist real-time chat application....643 Implementing the server side....643 Implementing the client side....644 Running and scaling the chat application....647 Using Redis as a simple message broker....648 Peer-to-peer Publish/Subscribe with ZeroMQ....652 Introducing ZeroMQ....652 Designing a peer-to-peer architecture for the chat server....652 Using the ZeroMQ PUB/SUB sockets....653 Reliable message delivery with queues....656 Introducing AMQP....658 Durable subscribers with AMQP and RabbitMQ....660 Reliable messaging with streams....666 Characteristics of a streaming platform....667 Streams versus message queues....668 Implementing the chat application using Redis streams....668 Task distribution patterns....672 The ZeroMQ fan-out/fan-in pattern....674 PUSH/PULL sockets....675 Building a distributed hashsum cracker with ZeroMQ....675 Pipelines and competing consumers in AMQP....683 Point-to-point communications and competing consumers....683 Implementing the hashsum cracker using AMQP....684 Distributing tasks with Redis streams....687 Redis consumer groups....687 Implementing the hashsum cracker using Redis streams....688 Request/Reply patterns....693 Correlation Identifier....693 Implementing a request/reply abstraction using correlation identifiers....694 Return address....699 Implementing the Return Address pattern in AMQP....699 Summary....705 Exercises....705 Other Books You May Enjoy....710 Index....714