Game Testing: All in One. 4 Ed

Front Cover....1 Half-Title Page....2 LICENSE, DISCLAIMER OF LIABILITY, AND LIMITED WARRANTY....3 Title Page....4 Copyright Page....5 Contents....6 Foreword....16 Preface to the Fourth Edition....20 Acknowledgments....24 Chapter 1: Your Role on the Game Development Team....26 “You Get Paid to Play Games?”....26 Elements of a Game Development Team....27 Programmers....27 Artists....28 Animators....28 Level Designers....29 Sound Designers....29 Game Designers....30 Producers....30 Testers....31 Putting It All Together....32 The Game Production Cycle....33 Concept Development....33 Development....39 Alpha....40 Beta....40 Code Lock....42 Gold....42 Patches....42 Updates....42 Ready, Tester One?....43 Exercises....43 Reference....43 Chapter 2: The Basics of Game Testing....44 Not All Players Are Alike....45 Black Box Testing....46 White Box Testing....47 Your First Test Suite....49 Writing Test Cases and Test Suites....53 The Life Cycle of a Build....55 The Testing Feedback Loop....57 The Essence of a Bug Report....57 Testing “Around” a Bug....57 Not All Testers Are Alike....59 Exercises....61 References....61 Chapter 3: Bug Report Writing and Defect Tracking....62 How to Write a Bug Report....62 Just the Facts, Please....63 The Brief Description....64 The Full Description....65 Great Expectations....67 Habits to Avoid....69 Using the Bug Database....70 Category....71 Summary....71 (Full) Description....71 Severity....72 Priority....72 Attachments and Recordings....73 Other Database Fields....73 Exercises....76 References....76 Chapter 4: How Bugs Happen....78 Who Cares?....79 Defect Types....79 Functions....81 Assignments....83 Checking....85 Timing....86 Build/Package/Merge....87 Algorithms....91 Documentation....94 Interfaces....96 Testing Happens....98 Exercises....99 References....101 Chapter 5: The Phases of Game Quality Assurance....102 Pre-Production....103 Planning Tasks....103 Determine the Scope of Testing the Project Will Require....104 Assign a Lead Tester....106 Establish Milestone Acceptance Criteria....106 Participate in Game Design Reviews....107 Set Up the Defect Tracking Database....108 Draft Test Plans and Design Tests....110 Testing Before Testing Begins....111 Test Kickoffs....112 Configuration Preparation....119 Smoke Testing....121 Alpha Testing....121 Alpha Phase Entry Criteria....122 Regression Testing....123 Beta Testing....125 Beta Phase Entry Criteria....125 Design Lock....126 Living with Bugs (For Now)....127 “The Swing Set of Death”....128 Patches, Updates, and Hotfixes....129 Gold Testing....130 Last-Minute Defects....131 Release Certification....132 Post-Release Testing....132 Live Teams....133 Exercises....135 Reference....135 Chapter 6: Exploratory Testing and Gameplay Testing....136 Ad Hoc Testing....136 Free Testing Comes from the Right Side of Your Brain....137 “Fresh Eyes”....137 Directed Testing Makes Order Out of Chaos....139 If You Are Not Recording, You Are Not Testing....140 Avoid Groupthink....142 Testing as Detective Work....143 The Benefits of Reproduction....144 The Scientific Method....146 Gameplay Testing....147 A Balancing Act....147 “It’s Just a Suggestion”....150 Making a Game Easy is Hard Work....151 External Testing....152 Who Decides?....155 Exercises....155 References....156 Chapter 7: The Two Rules of Test Management....158 Rule One: Do Not Panic....158 Unfamiliar....159 Unprepared....159 Under Pressure....160 Unrested....161 “Nearsighted”....163 Rule Two: Trust No One....163 Balancing Act....164 Word Games....164 Last Chance....165 Trust Fund....166 Managing Crunch Time....166 Give and Take....168 The Rest of the Story....169 Exercises....170 Reference....170 Chapter 8: Combinatorial Testing....172 Parameters....172 Values....173 Defaults....173 Enumerations....174 Ranges....174 Boundaries....175 Constructing Tables....177 Combinatorial Templates....197 Combinatorial Test Generation....200 Combinatorial Economics....204 Exercises....205 Chapter 9: Test Flow Diagrams....206 TFD Elements....206 Events....207 Actions....207 States....208 Primitives....208 Terminators....208 TFD Design Activities....208 Preparation....209 Allocation....209 Construction....209 A TFD Example....211 Data Dictionary....217 Data Dictionary Application....217 Data Dictionary Reuse....217 Data Dictionary Example....218 TFD Paths....222 Minimum Path Generation....222 Baseline Path Method....223 Expert Constructed Paths....225 Combining Path Strategies....227 Producing Test Cases from Paths....228 TFD Templates....232 When Should You Create a TFD?....233 Exercises....234 Chapter 10: Cleanroom Testing....236 Usage Probabilities....237 Mode-Based Usage....237 Player-Type Usage....238 Real-Life Usage....240 Cleanroom Test Generation....240 Cleanroom Combinatorial Tables....241 Cleanroom Combinatorial Examples....243 TFD Cleanroom Paths....249 TFD Cleanroom Path Example....250 Flow Usage Maintenance....253 Flow Usage Profiles....254 Inverted Usage....256 Calculating Inverted Usage....257 Combinatorial Table Usage Inversion....258 TFD Flow Usage Inversion....259 Exercises....263 References....263 Chapter 11: Test Trees....264 Test Case Trees....264 Tree Feature Tests....267 Test Tree Designs....272 Exercises....280 Chapter 12: Defect Triggers....282 Operating Regions....282 Pre-Game Operating Region....282 Game Start Operating Region....283 In-Game Operating Region....283 Post-Game Operating Region....284 The Triggers....284 The Configuration Trigger....284 The Startup Trigger....286 The Exception Trigger....287 The Stress Trigger....288 The Normal Trigger....288 The Restart Trigger....289 Classifying Defects....289 Defect Triggers and Test Designs....292 Combinatorial Design Trigger Examples....293 TFD Trigger Examples....299 Exercises....306 Reference....307 Chapter 13: Regression Testing and Test Reuse....308 Regression Testing....308 The A-B-C Method....308 Defect Modeling....311 Time Keeps on Ticking....314 Expanding Possibilities....316 Test Reuse....317 TFD Design Patterns....318 Looking Back and Forth....322 Combinatorial Expansion....323 Exercises....331 Reference....331 Chapter 14: Testing by the Numbers....332 Testing Progress....332 Testing Effectiveness....336 Tester Performance....339 Exercises....342 Chapter 15: Software Quality....344 Game Quality Factors....345 Game Quality Appraisal....346 Walkthroughs....347 Reviews....349 Checklist-based Reviews....349 Inspections....350 Game Standards....352 User Interface Standards....353 Coding Standards....354 Game Quality Measurements....356 Six Sigma Software....356 Phase Containment....358 Quality Plans....362 QA Personnel....362 Standards....363 Reviews and Audits....363 Feedback and Reports....364 Problem Reporting and Corrective Action....365 Tools, Techniques, and Methods....365 Supplier Control....368 Training....369 Risk Management....369 Exercises....370 References....370 Appendix A: Odd-Numbered Answers to Exercises....372 Chapter 1 — Your Role on the Game Development Team....372 Chapter 2 — The Basics of Game Testing....372 Chapter 3 — Bug Report Writing and Defect Tracking....373 Chapter 4 — How Bugs Happen....373 Chapter 5 — The Phases of Game Quality Assurance....373 Chapter 6 — Exploratory Testing and Gameplay Testing....374 Chapter 7 — The Two Rules of Test Management....374 Chapter 8 — Combinatorial Testing....374 Chapter 9 — Test Flow Diagrams....377 Chapter 10 — Cleanroom Testing....378 Chapter 11 — Test Trees....379 Chapter 12 — Defect Triggers....380 Chapter 13 — Regression Testing and Test Reuse....381 Chapter 14 — Testing by the Numbers....383 Chapter 15 — Software Quality....384 Appendix B: Sample Test Suite: RTS Building Checklist....386 Appendix C: Basic Test Plan Template....396 Section I: QA Team (and areas of responsibility)....396 Section II: Testing Procedures....397 Section III: How Testing Requirements are Generated....401 Section IV: Bug Tracking Software....401 Section V: Bug Classifications....401 Section VI: Bug Tracking....401 Section VII: Scheduling and Loading....402 Section VIII: Equipment Budget and Costs....402 Appendix D: Combinatorial Test Templates....404 Tables of Parameters with Two Test Values....404 Tables of Parameters with Three Test Values....407 Appendix E: Test Flow Diagram (TFD) Templates....414 Power-Ups....414 Craft Item....415 Heal Character....416 Create/Save....416 Unlock and Buy Item....417 Update Song List....418 Complete a Mission or Quest....419 Get Weapon and Ammo....420 Glossary....422 Index....426