Java Microservices and Containers in the Cloud: With Spring Boot, Kafka, PostgreSQL, Kubernetes, Helm, Terraform and AWS EKS

Table of Contents....5 About the Author....21 About the Technical Reviewer....22 Acknowledgments....23 Introduction....24 Chapter 1: Microservices for the Enterprise....26 Computing Architecture Paradigms....27 Centralized Computing....27 Distributed Computing....29 Decentralized Computing....32 Eventually Consistent Software Systems....34 Atomic Consistency....34 Eventual Consistency....36 Eventually Consistent, Decentralized Systems....38 Service Granularity....40 Monolith....40 Macroservice....41 Mini Service....41 Microservices....42 The Hexagonal Architecture Metaphor....43 Layered Architecture....44 The Ports and Adapter Architecture....44 Hexagonal Architecture....47 Microservice Metamodel....47 Your First Java Microservice....50 Spring Boot....51 Design Your First Hexagonal Microservice....51 Code Organization....53 Understanding the Code....55 Build and Run the Code....60 Test the Microservice Using UI....63 Test the Microservice Using cURL....68 Summary....68 Chapter 2: More Hands-on Microservices....69 Microservices Using MongoDB and RestTemplate....70 Design the Microservices....70 Code Organization....71 Understanding the Code....74 Build and Run the Microservice....81 Testing the Microservices....84 Microservices Using Spring Cloud....85 Design the Microservices....85 Understanding the Code....86 Build and Run the Microservice....89 Testing the Microservices....91 HATEOAS and HAL....91 HATEOAS Explained....92 HAL Explained....93 Microservices Using HATEAOS and HAL....94 Design the Microservices....94 Code Organization....95 Understanding the Code....96 Build and Run the Microservice....99 Testing the Microservice....101 Summary....102 Chapter 3: Onion and Hexagonal Architecture in Practice....104 Microservices Using PostgreSQL and RestTemplate....105 Design the Microservice....105 Understanding the Code....106 Build and Run the Microservice....115 Testing the Microservice....118 Microservices Using MongoDB and CrudRepository....119 Hexagonal Architecture Revisited....120 Understanding the Code....120 Build and Run the Microservice....123 Testing the Microservices....125 The Onion Architecture....127 Onion Architecture Design....127 Onion vs. Hexagonal Architecture....129 GraphQL....130 GraphQL Explained....130 Onion Architecture Microservice Example....131 Onion Design for the Microservice....132 Code Organization....133 Understanding the Code....134 Build and Run the Microservice....146 Testing the Microservice....148 Summary....155 Chapter 4: Message Oriented Microservices....156 Characteristics of Microservices....157 Microservices Are Autonomous....157 Microservices Are Evolutionary....158 The Honeycomb Analogy....160 Async, Still Request-Reply....160 Async Request....160 Async Request-Reply....161 Sync Over Async Microservices....163 Design Microservices Over Async Channel....163 Understanding the Code....164 Build and Run the Microservice....174 Testing the Microservice....176 SEDA and Microservices....177 SEDA Architecture....177 Microservices Architecture as a Network of Stages....179 Async HTTP on Sync over Async Microservices....180 Understanding the Code....180 Build and Run the Microservice....185 Testing the Microservice....188 Summary....191 Chapter 5: Microservices Integration in Practice....193 Microservices Scalability in the Cloud....194 Vertical Scalability....194 Horizontal Scalability....195 Microservices Scalability....195 Sync over Async Dynamics....197 Publisher Subscriber Combinations....198 Publisher Subscriber Instance Combinations....198 Microservices Sequential Processing....201 Design Sequential Processing Microservices....201 Understanding the Code....203 Build and Run the Microservice Sequential Processing....218 Testing Sequential Request Processing....224 Inspecting the Microservice Partition assignment....229 Microservices Parallel Processing....231 Design Parallel Processing Microservices....232 Parallelism and Thread Safety....233 Understanding the Code....234 Build and Run the Microservice Parallel Processing....236 Testing Parallel Request Processing....240 Summary....244 Chapter 6: Production Grade Message Oriented Microservices....245 Inter-Microservices Wire Level Options....246 XML-RPC and SOAP....246 REST and JSON-RPC....247 RPC vs. Messaging....248 RPC Style Messaging....248 HTTP Methods for CRUD on Resources....249 CRUD Microservices Over Kafka on MongoDB....251 Design CRUD Over Async Channel on MongoDB....252 Understanding the Source Code....253 Build and Run the Microservice....262 Testing the Microservices....268 CRUD Microservices Over Kafka on PostgreSQL....270 Design CRUD Over Async Channel on PostgreSQL....271 Understanding the Source Code....271 Build and Run the Microservice....272 Testing the Microservice....277 Aggregating Responses from Multiple Microservices....280 Aggregator Integration Pattern....280 Design the Microservice Aggregator....282 Understanding the Source Code....283 Build and Run the Microservice....289 Testing the Microservice....294 Summary....301 Chapter 7: Introducing Docker....302 Different Types of Virtualization....303 Traditional Deployment....303 Virtualization....304 Containers....306 Containers in Detail....306 History of Containers....306 Container Internals....307 Virtualization vs. Containerization....308 Docker Concepts....310 Layered Filesystems....310 Docker Images....310 Docker Images vs. Containers....314 Docker Registry....318 Hello World Tomcat in Docker....320 Run Tomcat Container....320 Build and Run Java App without Java....328 Using a Maven Container....329 Creating a Maven Archetype....329 Build and Package Using Maven Container....332 Deploy Webapp Using Maven Container....335 Build and Run Java App with Docker Compose....337 Using Docker Compose....338 Build the First Java Microservice with Docker....341 Dockerizing the Microservice Using Jib....341 Understanding the Source Code....342 Build and Push Images to Docker Hub....345 Docker Hub Registry and Repository....347 Pull Image and Run Container....349 Build a Microservice Using a Dockerfile....354 Dockerfiles....355 Understanding the Source Code....355 Build and Run the Microservice Using a Dockerfile....357 Summary....364 Chapter 8: Microservice Containers....365 Container Networking....366 Links....366 Networks....367 Container Logs....368 Console Output Logging....368 Microservice and MongoDB Containers Using Links....370 Designing the Container Topology....370 Understanding the Source Code....373 Run Containers Using Dockerfiles....373 Testing the Microservice Containers....378 Microservice and MongoDB Containers Using a Network....380 Designing the Container Topology....381 Understanding the Source Code....381 Run Containers Using Dockerfiles....381 Testing the Microservice Containers....387 Microservice Container with PostgreSQL in Host....389 Designing the Container Topology....389 Understanding the Source Code....391 Container to PostgreSQL in Host Configuration....391 Run Containers Using Dockerfiles....392 Testing the Microservice Containers....396 Microservice and PostgreSQL in Container....401 Designing the Container Topology....402 Understanding the Source Code....403 Run Containers Using Dockerfiles....403 Testing the Microservice Containers....408 Container Storage....413 Volumes....414 Microservice and MongoDB Containers Using File Mount....414 Designing the Container Topology....415 Understanding the Source Code....416 Run Containers Using Dockerfiles....417 Testing the Microservice Containers....421 Summary....424 Chapter 9: Composing Multi-Service Containers....425 Introducing Docker Compose....426 Docker Compose YAML....427 Docker Compose Use Cases....428 Composing Microservice with PostgreSQL Containers....429 Composing the Container Topology....430 Understanding the Source Code....431 Run Containers Using Docker Compose....436 Testing the Microservice Containers....441 Composing Microservice with MongoDB Containers....444 Composing the Container Topology....444 Understanding the Source Code....445 Run Containers Using Docker Compose....447 Testing the Microservice Containers....450 Composing Microservice with PostgreSQL and Kafka....452 Composing the Container Topology....452 Understanding the Source Code....453 Run Containers Using Docker Compose....459 Testing the Microservice Containers....464 Composing Microservices with MongoDB and Kafka....467 Composing the Container Topology....467 Understanding the Source Code....468 Run Containers Using Docker Compose....469 Testing the Microservice Containers....472 Summary....474 Chapter 10: Microservices with Kubernetes....475 The World of Kubernetes....476 Kubernetes Architecture....477 The Control Plane....478 The Node....479 Kubernetes Deployment....479 Containers....480 Pods....480 ReplicaSets....481 Deployment....481 Accessing Kubernetes Services....482 Accessing Kubernetes Deployments....482 Kubernetes Service....483 Traffic Routing to Kubernetes....486 Cluster IP....486 NodePorts....487 Load Balancer....489 Ingress....490 State Management....492 Scale Out vs. State Management....492 StatefulSets....493 Persistent Volume Claim....495 Persistent Volume....495 Kubernetes Clusters....496 Minikube....496 Microservices and MongoDB in Kubernetes....496 Design Microservices Deployment Topology....497 Understanding the Source Code....498 Run Microservices in Kubernetes....509 Testing the Microservice Pods....519 Accessing Kubernetes Deployments....520 Microservices and PostgreSQL in Kubernetes....527 Design Microservices Deployment Topology....527 Understanding the Source Code....528 Run Microservices in Kubernetes....534 Testing the Microservice Pods....536 Resiliency of Kubernetes Pods....539 State Retention....539 Summary....543 Chapter 11: Message Oriented Microservices in Kubernetes....544 Microservices Over Kafka with PostgreSQL in k8s....545 Design the Microservices Orchestration Topology....545 Understanding the Source Code....546 Run Microservices in Kubernetes....553 Testing the Microservice Pods....556 Load Testing the Microservice Pods....560 Microservices Over Kafka with MongoDB in k8s....563 Design the Microservices Orchestration Topology....563 Understanding the Source Code....564 Run Microservices in Kubernetes....566 Testing the Microservice Pods....568 Load Testing the Microservice Pods....568 Connect to the MongoDB Pod Using kubectl....569 Ingress Routing of Microservices....572 Design Ingress Routing Topology....573 Environment Configurations....573 Understanding the Source Code....575 Run Ingress and Microservices in Kubernetes....580 Describing Ingress....582 Testing the Microservice Pods....583 Summary....588 Chapter 12: Automating Kubernetes Deployment and Helm....589 Introducing a Simple Java Microservice....590 Designing Your Simple Microservices....590 Code Organization....591 Understanding the Source Code....592 Build and Run the Microservice....593 Testing the Microservice....595 Automating the Docker Build....595 Understanding the Source Code....595 Build and Run the Microservice....597 Testing the Microservice....603 Automating Docker Push....604 Understanding the Source Code....604 Build and Run the Microservice....606 Testing the Microservice....609 Automating Kubernetes Deployment....609 Code Organization....609 Build and Run the Microservice....611 Testing the Microservice....612 Helm....612 What Is Helm?....613 Helm Nomenclature....613 Client-Server Helm Architecture....614 How Helm Works....615 Helm Package Microservice....617 Code Organization....617 Creating Your First Helm Chart....618 Confirming Helm Chart Accuracy....621 Build and Run the Microservice....623 Testing the Microservice....626 Helm Release Upgrades....627 Helm Release Rollback....630 Helm-Packaging Multi-Microservices....632 Designing Helm-Based Deployment Topology....633 Code Organization....633 Understanding the Source Code....637 Helm Chart for PostgreSQL....637 Common Chart for Microservices and Adminer....642 Helm Chart for Ingress....643 Build and Run the Microservice....645 Testing the Microservice....647 Helmfile Packaging Multi-Microservices....649 Code Organization....650 Understanding the Source Code....652 Build and Run the Microservice....655 Testing the Microservice....658 Summary....659 Chapter 13: CI/CD for Microservice Containers....660 CI and CD....661 DevOps....661 Continuous Integration (CI)....662 Continuous Delivery (CD)....663 Continuous Deployment (CD)....663 Google Skaffold....663 Skaffold Workflow....664 CI and CD Example for Microservices....665 Code Organization....665 Understanding the Source Code....666 Build and Run the Microservice....670 Testing the Microservices....672 Summary....676 Chapter 14: Microservices in AWS Elastic Compute Cloud....678 Cloud Computing....679 Public Clouds....679 Cloud Deployment Models....681 Public Cloud....681 Private Cloud....682 Hybrid Cloud....682 Multi-Cloud....682 Distributed Computing....682 Amazon Web Services (AWS)....684 VPC....685 Subnet....685 Public Proxy Subnet....686 NAT Gateway....686 EC2....687 Route Tables....687 Internet Gateway....687 Infrastructure as Code (IaC)....688 Terraform....689 Setting Up AWS EC2 Using Terraform....690 Design a Mini Data Center in the Cloud....690 Code Organization....691 Understanding the Source Code....692 Build and Spin EC2 Server in the Cloud....700 Accessing Your EC2 in AWS Cloud Using SSH....703 Installing JRE in AWS EC2....704 Deploying Microservice in AWS EC2....705 Copy the Microservice Executable to the Cloud....706 Run the Microservice....706 Test the Microservice Using UI....708 Summary....709 Chapter 15: Microservices in AWS Elastic Kubernetes Service....710 The Amazon Elastic Kubernetes Service....711 EKS Control Plane....711 EKS Worker Nodes....711 Setting Up AWS EKS Using Terraform....712 Designing an EKS Topology in an AWS Cloud....712 Code Organization....714 Understanding the Source Code....714 Setups Required to Operate an EKS Cluster....725 Build and Spin Your EKS Cluster in AWS....725 Deploying Containers in AWS EKS....727 Testing the Microservice in EKS....730 Summary....732 Appendix A: Using cURL and Postman....734 cURL Operations for HTTP....735 Using HTTP GET to Retrieve an Entity....735 Using HTTP POST to Create an Entity....736 Using HTTP PUT to Replace an Entity....737 Using HTTP PATCH to Modify an Entity....738 Using HTTP DELETE to Delete an Entity....740 Postman for HTTP Operations....740 Using HTTP GET to Retrieve an Entity with Postman....740 Using HTTP POST to Create an Entity with Postman....741 Summary....743 Appendix B: Using MongoDB....744 Install MongoDB on the macOS....744 Downloading and Extracting the MongoDB Tarball....744 Installing the Tarball....746 Running the MongoDB Server....747 Opening the MongoDB Terminal....749 Install MongoDB on Windows....750 Downloading and Extracting the MongoDB Archive....750 Installing the ZIP Archive....751 Running the MongoDB Server....751 Opening the MongoDB Terminal....752 Run Commands Against the MongoDB Server....753 Creating a New Document....753 Reading a Document....754 Updating a Document....754 Deleting a Document....755 Dropping a Collection....755 Exiting the Mongo Terminal....756 Summary....756 Appendix C: Using PostgreSQL....757 Install PostgreSQL on macOS....757 Downloading and Extracting the Image....757 Installing the Image....759 Starting and Stopping PostgreSQL Server....771 Interacting Using psql Terminal....772 Opening the PostgreSQL Server for Remote Clients....772 Opening PostgreSQL on the Host Machine for Docker Clients....773 Install PostgreSQL in Windows OS....774 Downloading and Extracting the ZIP Archive....774 Installing the ZIP Archive....775 Starting and Stopping PostgreSQL Server....777 Interacting Using the psql Terminal....778 Create a Database Using pgAdmin....779 Logging In to PostgreSQL Server....779 Run Command Against PostgreSQL Using psql....784 Creating a New Table....786 Listing Existing Tables....786 Describing the Structure of an Existing Table....787 Inserting a Row into a Table....788 Reading Data from the Table....788 Updating a Table....789 Deleting Rows from a Table....789 Dropping a Table....790 Exiting the psql Terminal....790 Summary....791 Appendix D: Using Kafka....792 Installing Kafka....792 Downloading and Extracting the Image....792 Starting and Stopping Kafka Broker....794 Run Commands Against Kafka Broker....795 Creating a Topic....795 Viewing a Topic....796 Reading Events from a Topic....796 Publishing Events to a Topic....797 Deleting a Topic....798 Summary....798 Appendix E: Container Tools....799 Docker Env for Intel based Mac....799 Uninstalling the Existing Docker Desktop....800 Installing HyperKit....800 Installing Docker CLI....802 Container Orchestration....804 Installing kubectl....805 Installing Minikube....806 Configuring Minikube....808 Starting Minikube....809 Starting Docker env in the Terminal....811 Accessing Docker from the Terminal....811 Installing Docker Compose....812 Stopping Minikube....814 Deleting Minikube....814 Troubleshooting Minikube....815 Expose Minikube Services....817 Enabling the Ingress Addon....817 Enabling the Ingress DNS Addon....819 Manage Kubernetes Applications....819 Deploying Kubernetes Applications....819 Inspecting the Kubernetes Deployment....820 Accessing the Kubernetes Application....821 Docker Env for Apple Silicon-Based Mac....822 Installing Podman....822 Creating and Starting a VM....822 Starting Minikube....824 Installing Helm on the Mac....826 Summary....826 Appendix F: Cloud Tools....827 AWS CLI....827 Downloading AWS CLI....828 Installing AWS CLI....828 Configuring AWS CLI....829 AWS IAM....829 Downloading AWS IAM....829 Configuring AWS IAM....830 kubectl for EKS....832 Joining Worker Nodes to EKS....833 Terraform....834 Installing Terraform....835 Upgrading Terraform....835 Verifying Terraform....836 Summary....836 Index....837