Terraform Cookbook: Provision, run, and scale cloud architecture with real-world examples using Terraform. 2 Ed

Cover....1 Copyright....3 Contributors....7 Table of Contents....10 Preface....36 Chapter 1: Setting Up the Terraform Environment....44 Technical requirements....45 Overviewing Terraform best practices....45 Downloading and installing Terraform on Windows manually....47 Getting ready....47 How to do it…....47 How it works…....52 Installing Terraform using Chocolatey on Windows....54 Getting ready....54 How to do it…....55 How it works…....55 There’s more…....56 See also....56 Installing Terraform on Linux using the APT package manager....56 Getting ready....56 How to do it…....56 How it works…....57 See also....57 Installing Terraform using a script on Linux....57 Getting ready....57 How to do it…....58 How it works…....58 There’s more…....59 See also....60 Executing Terraform in a Docker container....60 Getting ready....60 How to do it…....61 How it works…....63 There’s more…....63 See also....63 Switching between multiple Terraform versions....64 Getting ready....65 How to do it…....65 How it works…....68 There’s more…....68 See also....70 Upgrading Terraform providers....70 Getting ready....70 How to do it…....71 How it works…....74 There’s more…....74 See also....75 Chapter 2: Writing Terraform Configurations....76 Technical requirements....77 Configuring Terraform and the provider version to use....77 Getting ready....78 How to do it…....80 How it works…....80 There’s more…....82 See also....83 Adding alias to a provider to use multiple instances of the same provider....83 Getting ready....83 How to do it…....84 How it works…....85 See also....85 Manipulating variables....86 Getting ready....86 How to do it…....86 How it works…....87 There’s more…....88 See also....89 Keeping sensitive variables safe....90 Getting ready....90 How to do it…....90 How it works…....91 There’s more…....91 See also....92 Using local variables for custom functions....92 Getting ready....92 How to do it…....93 How it works…....94 There’s more…....94 See also....94 Using outputs to expose Terraform provisioned data....95 Getting ready....95 How to do it…....96 How it works…....96 There’s more…....96 See also....97 Calling Terraform’s built-in functions....97 Getting ready....97 How to do it…....98 How it works…....98 See also....99 Using YAML files in Terraform configuration....100 Getting ready....100 How to do it…....100 How it works…....102 There’s more…....102 See also....102 Writing conditional expressions....102 Getting ready....103 How to do it…....103 How it works…....103 There’s more…....104 See also....104 Generating passwords with Terraform....104 Getting ready....105 How to do it…....105 How it works…....106 See also....106 Managing Terraform resource dependencies....107 Getting ready....107 How to do it…....108 How it works…....108 There’s more…....109 See also....109 Adding custom pre and postconditions....109 Getting ready....110 How to do it…....110 How it works…....111 See also....113 Using checks for infrastructure validation....113 Getting ready....113 How to do it…....114 How it works…....114 There’s more…....115 See also....115 Chapter 3: Scaling Your Infrastructure with Terraform....118 Technical requirements....119 Provisioning infrastructure in multiple environments....119 Getting ready....119 How to do it…....120 How it works…....121 See also....124 Provisioning multiple resources with the count meta-argument....124 Getting ready....124 How to do it…....125 How it works…....125 There’s more…....127 See also....128 Using maps....128 Getting ready....129 How to do it…....129 How it works…....130 There’s more…....131 See also....132 Looping over a map of objects....132 Getting ready....133 How to do it…....133 How it works…....134 There’s more…....135 See also....136 Generating multiple blocks with the dynamic block....136 Getting ready....137 How to do it…....137 How it works…....139 There’s more…....140 See also....140 Filtering maps....140 Getting ready....141 How to do it…....141 How it works…....142 There’s more…....143 See also....143 Chapter 4: Using Terraform with External Data....146 Technical requirements....147 Obtaining external data with data sources....147 Getting ready....148 How to do it…....148 How it works…....148 There’s more…....149 See also....150 Querying external data with Terraform....150 Getting ready....151 How to do it…....151 How it works…....152 There’s more…....154 See also....154 Manipulating local files with Terraform....154 Getting ready....155 How to do it…....155 How it works…....156 There’s more…....156 See also....157 Executing local programs with Terraform....157 Getting ready....157 How to do it…....157 How it works…....158 There’s more…....159 See also....160 Executing shell scripts using the Terraform Shell provider....160 Getting ready....160 How to do it…....160 How it works…....162 There’s more…....164 See also....164 Chapter 5: Managing Terraform State....166 Using the local Terraform state....167 Getting ready....167 How to do it…....167 How it works…....168 There’s more…....168 See also....170 Managing resources in Terraform state....170 Getting ready....170 How to do it…....171 Displaying the content of state....171 Listing Terraform resource names within state....173 Showing detailed resource properties in state....173 Deleting resources from state....174 There’s more…....175 See also....175 Synchronizing Terraform state....175 Getting ready....176 How to do it…....177 How it works…....178 There’s more…....180 See also....180 Importing existing resources....180 Getting ready....181 How to do it…....182 How it works…....183 There’s more…....183 See also....186 Using external resources from other Terraform state files....186 Getting ready....187 How to do it…....188 How it works…....188 There’s more…....189 See also....189 Refactoring resources in configuration....189 Getting ready....190 How to do it…....192 How it works…....193 There’s more…....196 See also....197 Chapter 6: Applying a Basic Terraform Workflow....200 Technical requirements....201 Keeping your Terraform configuration clean....201 Getting ready....202 How to do it…....202 How it works…....202 There’s more…....203 See also....205 Validating the code syntax....205 Getting ready....205 How to do it…....206 How it works…....207 There’s more…....207 See also....209 Destroying infrastructure resources....209 Getting ready....209 How to do it…....210 How it works…....210 There’s more…....210 See also....211 Displaying a list of providers used in a configuration....212 Getting ready....212 How to do it…....213 How it works…....213 There’s more…....213 See also....214 Generating one Terraform lock file with Windows and Linux compatibility....214 Getting ready....215 How to do it…....216 How it works…....216 See also....217 Copying a Terraform module configuration....218 Getting ready....218 How to do it…....218 How it works…....218 There’s more…....219 See also....219 Using workspaces to manage environments....220 Getting ready....220 How to do it…....220 How it works…....221 There’s more…....223 See also....224 Exporting the output in JSON....224 Getting ready....224 How to do it…....225 How it works…....225 There’s more…....227 See also....228 Tainting resources....228 Getting ready....228 How to do it…....228 How it works…....229 There’s more…....230 See also....231 Generating the dependency graph....231 Getting ready....232 How to do it…....232 How it works…....232 See also....233 Using different Terraform configuration directories....233 Getting ready....234 How to do it…....235 How it works…....235 There’s more…....235 See also....236 Testing and evaluating a Terraform expression....236 Getting ready....236 How to do it…....237 How it works…....237 There’s more…....238 See also....239 Debugging the Terraform execution....239 Getting ready....239 How to do it…....239 How it works…....240 There’s more…....241 See also....241 Chapter 7: Sharing Terraform Configuration with Modules....242 Technical requirements....243 Creating a Terraform module and using it locally....243 Getting ready....244 How to do it…....244 How it works…....247 There’s more…....247 See also....249 Provisioning multiple instances of a Terraform module....249 Getting ready....249 How to do it…....250 How it works…....251 There’s more…....251 See also....252 Using modules from the public registry....252 Getting ready....252 How to do it…....253 How it works…....254 There’s more…....255 See also....256 Sharing a Terraform module in the public registry using GitHub....256 Getting ready....256 How to do it…....256 How it works…....259 There’s more…....260 See also....261 Using another file inside a custom module....261 Getting ready....261 How to do it…....262 How it works…....262 There’s more…....263 See also....264 Using the Terraform module generator....264 Getting ready....264 How to do it…....265 How it works…....265 There’s more…....267 See also....267 Generating module documentation....268 Getting ready....268 How to do it…....269 How it works…....269 There’s more…....271 See also....272 Using a private Git repository for sharing a Terraform module....272 Getting ready....273 How to do it…....274 How it works…....276 There’s more…....277 See also....278 Applying a Terrafile pattern for using modules....278 Getting ready....279 How to do it…....279 How it works…....281 There’s more…....281 See also....282 Chapter 8: Provisioning Azure Infrastructure with Terraform....284 Technical requirements....285 Using Terraform in Azure Cloud Shell....285 Getting ready....286 How to do it…....286 How it works…....287 There’s more…....288 See also....289 Protecting the Azure credential provider....289 Getting ready....290 How to do it…....290 How it works…....292 There’s more.......293 See also....294 Protecting the state file in the Azure remote backend....294 Getting ready....295 How to do it…....295 How it works…....296 There’s more…....297 See also....299 Executing ARM templates in Terraform....299 Getting ready....300 How to do it…....300 How it works…....302 There’s more…....303 See also....303 Executing Azure CLI commands in Terraform....304 Getting ready....304 How to do it…....305 How it works…....306 There’s more…....306 See also....307 Using Azure Key Vault with Terraform to protect secrets....308 Getting ready....308 How to do it…....309 How it works…....311 There’s more…....312 See also....314 Provisioning and configuring an Azure VM with Terraform....314 Getting ready....314 How to do it…....315 How it works…....318 There’s more…....319 See also....320 Building Azure serverless infrastructure with Terraform....320 Getting ready....320 How to do it…....321 How it works…....322 There’s more…....322 See also....323 Generating a Terraform configuration for existing Azure infrastructure....323 Getting ready....324 How to do it…....326 How it works…....327 There’s more…....329 See also....330 Enabling optional Azure features....330 Getting ready....330 How to do it…....331 How it works…....331 There’s more…....331 Estimating Azure cost of infrastructure using Infracost....331 Getting ready....332 How to do it…....334 How it works…....336 There’s more…....336 See also....337 Using the AzApi Terraform provider....337 Getting ready....338 How to do it…....338 How it works…....340 There’s more…....341 See also....342 Chapter 9: Provisioning Simple AWS and GCP Infrastructure Using Terraform....344 Technical requirements....345 Getting started using Terraform for AWS....345 Getting ready....345 How to do it…....347 How it works…....348 There’s more…....349 See also....350 Using the S3 backend in AWS....350 Getting ready....350 How to do it…....353 How it works…....353 There’s more…....354 See also....354 Getting started using Terraform for GCP....355 Getting ready....355 How to do it…....359 How it works…....360 There’s more…....362 See also....362 Using a GCS backend in GCP....362 Getting ready....362 How to do it…....365 How it works…....365 See also....366 Executing Terraform in GCP Cloud Shell....366 Getting ready....367 How to do it…....367 How it works…....370 There’s more…....370 See also....371 Chapter 10: Using Terraform for Docker and Kubernetes Deployment....372 Technical requirements....373 Creating a Docker container using Terraform....374 Getting ready....374 How to do it…....374 How it works…....375 There’s more…....377 See also....377 Deploying Kubernetes resources using Terraform....377 Getting ready....378 How to do it…....378 How it works…....381 There’s more…....382 See also....383 Deploying a Helm chart in Kubernetes using Terraform....383 Getting ready....383 How to do it…....384 How it works…....385 There’s more…....386 See also....387 Using a Kubernetes controller as a Terraform reconciliation loop....387 Getting ready....388 How to do it…....390 How it works…....391 There’s more…....393 See also....394 Chapter 11: Running Test and Compliance Security on Terraform Configuration....396 Technical requirements....398 Using PowerShell’s Pester framework to perform Terraform testing....398 Getting ready....399 How to do it…....400 How it works…....403 There’s more…....404 See also....405 Testing the Terraform configuration using Python....405 Getting ready....405 How to do it…....406 How it works…....408 There’s more…....409 See also....409 Using OPA to check the Terraform configuration....409 Getting ready....410 How to do it…....412 How it works…....414 There’s more…....415 See also....416 Using tfsec to analyze the compliance of Terraform configuration....416 Getting ready....417 How to do it…....418 How it works…....419 There’s more…....419 See also....420 Applying Terraform compliance using terraform-compliance....420 Getting ready....420 How to do it…....421 How it works…....423 There’s more…....425 See also....425 Testing Terraform module code with Terratest....425 Getting ready....425 How to do it…....426 How it works…....429 There’s more…....430 See also....431 Testing the Terraform configuration using Kitchen-Terraform....431 Getting ready....431 How to do it…....432 How it works…....435 There’s more…....437 See also....437 Using the new integrated Terraform module integration test....438 Getting ready....438 How to do it…....439 How it works…....441 There’s more…....442 See also....443 Chapter 12: Deep-Diving into Terraform....444 Technical requirements....445 Preventing resources from being destroyed....445 Getting ready....445 How to do it…....446 How it works…....447 There’s more…....447 See also....449 Ignoring manual changes....449 Getting ready....450 How to do it…....451 How it works…....452 There’s more…....452 See also....453 Using Terraform’s templating feature....453 Getting ready....453 How to do it…....455 How it works…....456 There’s more…....457 See also....457 Zero-downtime deployment with Terraform....458 Getting ready....458 How to do it…....459 How it works…....460 There’s more…....460 See also....461 Managing Terraform configuration dependencies using Terragrunt....461 Getting ready....461 How to do it…....463 How it works…....463 There’s more…....464 See also....464 Using Terragrunt as a wrapper for Terraform....465 Getting ready....465 How to do it…....466 How it works…....467 See also....467 Generating a self-signed SSL certificate using Terraform....468 Getting ready....468 How to do it…....468 How it works…....469 There’s more…....469 See also....471 Checking the configuration before committing code using Git hooks....472 Getting ready....472 How to do it…....473 How it works…....474 There’s more…....475 See also....475 Visualizing Terraform resource dependencies with Rover....476 Getting ready....476 How to do it…....476 How it works…....477 See also....478 Using the Terraform CDK for developers....478 Getting ready....479 How to do it…....480 How it works…....484 There’s more…....486 See also....487 Chapter 13: Automating Terraform Execution in a CI/CD Pipeline....488 Running Terraform in automation mode....489 Getting ready....489 How to do it…....490 How it works…....491 There’s more…....491 See also....491 Displaying a summary of the execution of terraform plan....492 Getting ready....492 How to do it…....492 There’s more…....494 See also....494 Building CI/CD pipelines to apply Terraform configurations in Azure Pipelines....494 Getting ready....495 How to do it…....497 How it works…....501 There’s more…....502 See also....503 Automating Terraform execution in GitHub Actions....503 Getting ready....504 How to do it…....504 How it works…....506 There’s more…....506 See also....508 Working with workspaces in CI/CD....508 Getting ready....508 How to do it…....509 How it works…....511 There’s more…....512 See also....512 Building CI/CD for Terraform modules in Azure Pipelines....512 Getting ready....513 How to do it…....513 How it works…....518 There’s more…....518 See also....518 Building a workflow for publishing Terraform modules using GitHub Actions....519 Getting ready....519 How to do it…....519 How it works…....521 There’s more…....524 See also....524 Chapter 14: Using Terraform Cloud to Improve Team Collaboration....526 Technical requirements....528 Authenticating Terraform to Terraform Cloud....529 Getting ready....529 How to do it…....530 There’s more…....532 See also....533 Managing workspaces in Terraform Cloud....533 Getting ready....534 How to do it…....534 How it works…....541 There’s more…....542 See also....543 Using the remote backend in Terraform Cloud....543 Getting ready....544 How to do it…....545 How it works…....548 There’s more…....549 See also....549 Migrating Terraform State to Terraform Cloud....549 Getting ready....550 How to do it…....551 How it works…....553 See also....553 Using Terraform Cloud as a private module registry....553 Getting ready....554 How to do it…....556 How it works…....558 There’s more…....558 See also....558 Executing Terraform configuration remotely in Terraform Cloud....559 Getting ready....559 How to do it…....560 How it works…....564 There’s more…....564 See also....569 Checking the compliance of Terraform configurations using OPA in Terraform Cloud....569 Getting ready....569 How to do it…....571 There’s more…....578 See also....578 Using integrated cost estimation for cloud resources....578 Getting ready....579 How to do it…....579 How it works…....582 There’s more…....583 See also....583 Integrating the Infracost run task during the Terraform Cloud run....583 Getting ready....584 How to do it…....586 How it works…....590 There’s more…....590 See also....590 Configuring Terraform Cloud with the Terraform TFE provider....590 Getting ready....591 How to do it…....592 There’s more…....594 See also....595 Chapter 15: Troubleshooting Terraform Errors....596 Fixing interpolation errors....597 Getting ready....597 How to do it…....598 How it works…....599 See also....599 Fixing cycle errors....599 Getting ready....599 How to do it…....600 How it works…....601 There’s more…....601 Fixing for_each errors....602 Getting ready....602 How to do it…....604 There’s more…....605 Fixing output errors....605 Getting ready....605 How to do it…....606 How it works…....607 There’s more…....607 Appendix A: Terraform Cheat Sheet....608 Basic commands....608 Format Terraform configuration....609 Terraform providers management....609 Terraform dependency file....609 Basic workflow commands....610 Backend configuration....610 Validate configuration....611 Get outputs....611 Import resources....611 Terraform workspaces....611 Terraform debug....612 State management....612 Display Terraform graph dependencies....613 Taint/untaint resources....613 Terraform Cloud/Enterprise....613 Appendix B:Terraform Resources....614 Terraform official resources....614 Documentation....614 Registry....614 Providers development....614 Terraform community resources....615 Terraform news feed....615 Terraform certifications and certification preparation....615 Terraform certification program pages....615 Terraform certification preparation....615 PacktPage....616 Other Books You May Enjoy....618 Index....2