Software Testing Strategies: A testing guide for the 2020s

Software Testing Strategies....2 Contributors....7 About the authors....7 About the reviewers:....9 Preface....23 Our scope - beyond button-pushing....23 What is Testing?....25 Who this book is for....28 What this book covers....29 To get the most out of this book....30 Download the example code files....31 Download the color images....31 Conventions used....31 Get in touch....32 Share your thoughts....32 Download a free PDF copy of this book....32 Part 1:The Practice of Software Testing....34 Chapter 1: Testing and Designing Tests....35 Jumping into testing....35 The impossibility of complete testing....36 What is the EVP asking for?....37 Toward a theory of error....43 Testing software – an example....44 Testing software – an analysis....51 Quick attacks – the airdropped tester....52 Test design – input space coverage....54 Equivalence classes and boundaries....57 Decision tables....61 Decision trees....63 All-pairs and pairwise testing....67 High volume automated approaches....71 Other approaches....73 Data and predictability – the oracle problem....74 Summary....76 Further reading....76 Chapter 2: Fundamental Issues in Tooling and Automation....77 Technical requirements....77 No silver bullets – you can’t radically fix the test cycle....78 The minefield regression problem....79 Coverage model....80 The Battleships problem – testing versus checking....84 Comparing Battleships to bugs....86 Automation rarely beats human intuition....87 The maintenance problem....88 The dangers of comprehensive tests....92 The money problem....93 Lessons from the fundamental issues....96 Summary....97 Chapter 3: Programmer-Facing Testing....99 Technical requirements....99 The programmer’s view....100 Testing and reliability....101 The hexagonal architecture....101 Introducing FizzBuzz....104 Unit tests....106 TDD....110 Consequences of TDD....112 Unit test and unit code design....113 Using test doubles to create seams....115 Mutation testing....116 Web APIs from a test perspective....120 Web API testing strategy....121 Testing functional and legacy code....123 A Roman Numerals Kata....123 Summary....124 Chapter 4: Customer-Facing Tests....126 Technical requirements....126 A word of warning....126 Human Or Tooling—is it either/or?....128 GUI test automation patterns....129 Eliminating redundancy with domain-specific libraries....132 Eliminating redundancy through object locators....133 Do you need conditionals, looping structures, and variables?....134 The tradeoff between assertion and image recognition....137 Designing your own system....139 Toward specification by example....140 Specification by example....142 Low-code and no-code test automation....144 Batch- and model-driven test automation....145 This chapter is completely wrong....149 Summary....149 Chapter 5: Specialized Testing....151 Technical requirements....151 Understanding load and performance testing....151 Getting to know the basics of load testing....152 Setting up a load test....152 Exploring security testing....156 Security concepts....156 Checking out common security vulnerabilities....157 Learning about industry standards....157 Investigating some security testing tools....158 Delving into accessibility testing....158 What is accessibility?....158 Advocating for accessibility....159 Investigating the distinctions between accessibility and inclusive design....160 Learning about the WCAG standard....160 Investigating some accessibility testing tools....160 Internationalization and localization....161 Preparing for internationalization and localization....162 Investigating tools for internationalization and localization....162 CI....163 CI and the pipeline....163 Getting involved with build management as a tester....163 Investigating CI tools....163 Regulated testing....164 Navigating regulatory requirements....164 Summary....165 Chapter 6: Testing Related Skills....166 Technical requirements....166 Finding bugs....167 Oracles in software testing....168 Inattentional blindness and Oracles....170 About the word bug....172 Writing bug reports....172 Effective bug reports....174 Effective reproduction steps....176 Planning testing – cases and plans....179 Test cases in practice....187 Metrics and measurement....188 Metric dysfunction....189 Project projections....191 Influencing change....192 Summarizing information....192 Summary....194 Chapter 7: Test Data Management....196 Technical requirements....196 The test data problem....197 Test data and database apps....199 The standard data seed....201 Table-driven – when test data drives behavior....203 Scriptable users and structure....204 Exploring synthetic users....204 Leveraging production refreshes....207 Exploring development, test, and production environments....208 Understanding the regulatory issues in test data management....209 The user spy feature....210 Summary....210 Part 2:Testing and Software Delivery....212 Chapter 8: Delivery Models and Testing....213 Technical requirements....214 Waterfall....214 The advantages of and place for the waterfall....217 The V-Model....218 Iterative, incremental, and mini waterfalls....219 Extreme Programming (XP)....222 The context of XP....222 Scrum and SAFe....223 The context of Scrum....227 SAFe and its context....228 House-rules software development....232 The Agile Manifesto....242 Context-driven testing....244 Illustrations of the principles in action....244 Kanban as a house rule....245 Continuous delivery and deployment....246 DevOps, Platform Engineering, SRE....247 Summary....248 Further reading....249 Chapter 9: The Puzzle Pieces of Good Testing....250 Technical requirements....250 Recipes – how to do hard things....251 Defining recipes....251 Shouldn’t recipes just be automated?....254 Do recipes overlap with technical documentation?....255 Coverage – did we test the right things well enough?....255 Precise code coverage measures....259 Closing out coverage....260 Defects – what is the status of the software?....260 Schedule and risk – too many test ideas, not enough time....261 Iterative testing....262 Strategy – what are our risks and priorities?....263 Dashboard – how do we communicate what we know?....264 Summary....269 Chapter 10: Putting Your Test Strategy Together....271 What are we doing now?....271 Getting the form filled out....276 The elevator pitch....276 A census of risk....279 Defining a real risk census....281 Setting priorities, time management, and scope....283 Today’s strategy versus tomorrow’s goals....285 Summary....286 Chapter 11: Lean Software Testing....288 Lean software testing defined....288 From ideas in practice to the term “Lean”....289 The seven wastes....291 Waste #1 – transport....292 Waste #2 – inventory....293 Waste #3 – motion....294 Waste #4 – waiting....296 Waste #5 – overprocessing....297 Waste #6 – overproduction....298 Waste #7 – defects....298 (New) waste #8 – ability....300 Removing waste in testing....301 Flow....302 Visualizing flow – an example....303 Multitasking....305 Measurement – lead time versus cycle time....307 Efficiency and congestion....308 Metric – touch time....308 Batch size....309 Queues and efficiency....310 Arrival time pacing....311 Limiting work in progress to create a pull system....311 Release cadence....312 One-piece flow and CD....313 Summary....314 Part 3:Practicing Politics....315 Chapter 12: Case Studies and Experience Reports....316 RCRCRC at scale....317 The 1-day test plan....318 RCRCRC in the enterprise....318 A coverage dashboard....320 Test coverage blinders....322 Pair and tri-programming....323 Discovering the expertise paradox....324 Making expertise transparent....326 The evolution of the test strategy....326 The alternative to evolution – information hiding....328 Professional pushback – dealing with bullies....329 Power in the workplace....330 One way to say it....331 Boundaries in the enterprise....337 Narcissistic communication....339 Boundary enforcement – the power move....341 If you choose to stay....341 Summary....342 Chapter 13: Testing Activities or a Testing Role?....343 Technical requirements....343 The cultural conflict with a testing role....343 How we got here – the bad news....344 How we got here – developing at internet speed....346 Building a risk mitigation team....348 The purpose of the risk exercise....349 Faith-based versus empirical test automation....352 The math behind faith-based test automation....353 Possible outcomes....353 Shift left and shift right....354 (Actually) continuous testing....355 Summary....356 Chapter 14: Philosophy and Ethics in Software Testing....358 Philosophy and why it matters in testing....358 Sprint Length: it depends....359 Shu Ha Ri: learning by osmosis....360 A tough question: what do you want?....361 Ethics and ethical reasoning in testing....363 Ethical frameworks....364 Classic ethical challenges....366 Practical ethical issues in testing....366 Skipping test steps and the good news....367 Decisions are not made in meetings....368 Scientific thinking and logical fallacies....369 Logical fallacies....370 How we wind up in hell and how to escape....371 Put the responsibility in the right place....372 Summary....373 Additional readings for the serious philosopher....373 Chapter 15: Words and Language About Work....375 Context-driven testing and the other schools....375 The Agile school....376 The DevOps or CD school....376 The analytical school....376 The factory school....377 The quality school....377 The context-driven school....377 Precise language....379 Wordsmatter....380 The benefits of being imprecise....381 Process versus skill....384 Testing and checking....386 Yes, we can assure quality....386 Summary....389 Further reading....389 Chapter 16: Testing Strategy Applied....390 A mobile test strategy example....390 Our sample mobile app....391 Designing a test program....392 Mobile system coordination and deployment....394 The human element....395 AI in software testing....396 The state of AI tools....397 Possible ways to use AI for testing....398 Other forms of AI in testing....401 The bottom line for AI testing....402 A few thoughts to leave with....402 Thoughts on documentation....403 Summary....404 Index....405 Why subscribe?....427 Other Books You May Enjoy....427 Packt is searching for authors like you....431 Share your thoughts....431 Download a free PDF copy of this book....431