Clean Code Principles And Patterns: Python Edition

1: About the Author....12 2: Introduction....13 3: Architectural Principles and Patterns....19 3.1: Software Hierarchy....20 3.2: Single Responsibility Principle....23 3.3: Uniform Naming Principle....28 3.4: Encapsulation Principle....30 3.5: Service Aggregation Principle....31 3.6: High Cohesion, Low Coupling Principle....47 3.7: Library Composition Principle....48 3.8: Avoid Duplication Principle....49 3.9: Externalized Service Configuration Principle....50 3.9.1: Environment Variables....51 3.9.2: Kubernetes ConfigMaps....53 3.9.3: Kubernetes Secrets....55 3.10: Service Substitution Principle....56 3.11: Inter-Service Communication Methods....57 3.11.1: Synchronous Communication Method....57 3.11.2: Asynchronous Communication Method....58 3.11.3: Shared Data Communication Method....60 3.12: Strategic Domain-Driven Design Principle....61 3.12.1: Design Example 1: Mobile Telecom Network Analytics Software System....64 3.12.2: Design Example 2: Banking Software System....69 3.13: Autopilot Microservices Principle....71 3.13.1: Stateless Microservices Principle....72 3.13.2: Resilient Microservices Principle....72 3.13.3: Horizontally Autoscaling Microservices Principle....75 3.13.4: Highly-Available Microservices Principle....76 3.13.5: Observable Microservices Principle....78 3.14: Software Versioning Principles....78 3.14.1: Use Semantic Versioning Principle....79 3.14.2: Avoid Using 0.x Versions Principle....79 3.14.3: Don’t Increase Major Version Principle....79 3.14.4: Implement Security Patches and Bug Corrections to All Major Versions Principle....81 3.14.5: Avoid Using Non-LTS Versions in Production Principle....81 3.15: Git Version Control Principle....81 3.15.1: Feature Branch....82 3.15.2: Feature Toggle....83 3.16: Architectural Patterns....85 3.16.1: Event Sourcing Pattern....85 3.16.2: Command Query Responsibility Segregation (CQRS) Pattern....87 3.16.3: Distributed Transaction Patterns....88 3.16.3.1: Saga Orchestration Pattern....89 3.16.3.2: Saga Choreography Pattern....93 3.17: Preferred Technology Stacks Principle....95 4: Object-Oriented Design Principles....99 4.1: Object-Oriented Programming Concepts....100 4.1.1: Classes/Objects....101 4.1.2: Encapsulation....102 4.1.3: Abstraction....102 4.1.4: Inheritance....102 4.1.5: Interface....103 4.1.5.1: Interface evolution....105 4.1.6: Polymorphism....106 4.2: Programming Paradigms....107 4.2.1: Imperative Programming....107 4.2.2: Functional Programming....108 4.3: Multi-Paradigm Programming Principle....112 4.4: Why is Object-Oriented Programming Hard?....113 4.5: SOLID Principles....115 4.5.1: Single Responsibility Principle....116 4.5.2: Open-Closed Principle....122 4.5.3: Liskov’s Substitution Principle....127 4.5.4: Interface Segregation and Multiple Inheritance Principle....130 4.5.5: Program Against Interfaces Principle (Generalized Dependency Inversion Principle)....139 4.6: Clean Microservice Design Principle....148 4.6.1: Real-Life Example....152 4.7: Vertical Slice Design Principle....164 4.8: Class Organization Principle....166 4.9: Uniform Naming Principle....168 4.9.1: Naming Interfaces and Classes....168 4.9.2: Naming Functions....171 4.9.2.1: Preposition in Function Name....174 4.9.2.2: Naming Method Pairs....176 4.9.2.3: Naming Boolean Functions (Predicates)....178 4.9.2.4: Naming Builder Methods....181 4.9.2.5: Naming Methods with Implicit Verbs....182 4.9.2.6: Naming Lifecycle Methods....183 4.9.2.7: Naming Function Parameters....183 4.10: Encapsulation Principle....184 4.10.1: Immutable Objects....185 4.10.2: Don’t Leak Modifiable Internal State Outside an Object Principle....186 4.10.3: Don’t Assign From a Method Parameter to a Modifiable Attribute....188 4.11: Prefer Composition Over Inheritance Principle....188 4.12: Tactical Domain-Driven Design Principle....196 4.12.1: Tactical DDD Concepts....197 4.12.1.1: Entities....197 4.12.1.2: Value Objects....197 4.12.1.3: Aggregates....198 4.12.1.4: Aggregate Roots....199 4.12.2: Actors....201 4.12.2.1: Factories....201 4.12.2.2: Repositories....201 4.12.2.3: Services....201 4.12.2.4: Events....202 4.12.2.5: Event Storming....202 4.12.3: Domain-Driven Design Example: Data Exporter Microservice....202 4.12.4: Domain-Driven Design Example 2: Anomaly Detection Microservice....216 4.13: Design Patterns....219 4.13.1: Design Patterns for Creating Objects....220 4.13.1.1: Factory Pattern....220 4.13.1.2: Abstract Factory Pattern....222 4.13.1.3: Static Factory Method Pattern....225 4.13.1.4: Builder Pattern....229 4.13.1.5: Singleton Pattern....232 4.13.1.6: Prototype Pattern....235 4.13.1.7: Object Pool Pattern....237 4.13.2: Structural Design Patterns....239 4.13.2.1: Composite Pattern....239 4.13.2.2: Facade Pattern....242 4.13.2.3: Bridge Pattern....243 4.13.2.4: Strategy Pattern....248 4.13.2.5: Adapter Pattern....249 4.13.2.6: Proxy Pattern....254 4.13.2.7: Decorator Pattern....257 4.13.2.8: Flyweight Pattern....261 4.13.3: Behavioral Design Patterns....264 4.13.3.1: Chain of Responsibility Pattern....264 4.13.3.2: Observer Pattern....266 4.13.3.3: Command/Action Pattern....268 4.13.3.4: Iterator Pattern....271 4.13.3.5: State Pattern....272 4.13.3.6: Mediator Pattern....275 4.13.3.7: Template Method Pattern....295 4.13.3.8: Memento Pattern....297 4.13.3.9: Visitor Pattern....298 4.13.3.10: Null Object Pattern....303 4.14: Don’t Ask, Tell Principle....304 4.15: Law of Demeter....307 4.16: Avoid Primitive Obsession Principle....308 4.17: Dependency Injection (DI) Principle....317 4.18: Avoid Code Duplication Principle....324 4.19: Inheritance in Cascading Style Sheets (CSS)....327 5: Coding Principles....330 5.1: Uniform Variable Naming Principle....330 5.1.1: Naming Integer Variables....331 5.1.2: Naming Floating-Point Number Variables....332 5.1.3: Naming Boolean Variables....333 5.1.4: Naming String Variables....335 5.1.5: Naming Enum Variables....336 5.1.6: Naming Collection (List and Set) Variables....336 5.1.7: Naming Dictionary Variables....337 5.1.8: Naming Pair and Tuple Variables....338 5.1.9: Naming Object Variables....339 5.1.10: Naming Optional Variables....339 5.1.11: Naming Function Variables (Callbacks)....340 5.1.12: Naming Class Attributes....342 5.1.13: General Naming Rules....343 5.1.13.1: Use Short, Common Names....343 5.1.13.2: Pick One Term And Use It Consistently....343 5.1.13.3: Avoid Obscure Abbreviations....343 5.1.13.4: Avoid Too Short Or Meaningless Names....344 5.2: Uniform Source Code Repository Structure Principle....345 5.3: Domain-Based Source Code Structure Principle....346 5.4: Avoid Comments Principle....356 5.4.1: Name Things Properly....357 5.4.2: Single Return Of Named Value At The End Of Function....360 5.4.3: Return Type Aliasing....362 5.4.4: Extract Constant for Boolean Expression....363 5.4.5: Extract Named Constant or Enumerated Type....363 5.4.6: Extract Function....364 5.4.7: Avoid Comments for Regular Expression....365 5.4.8: Avoiding Comments in Bash Shell Scripts....366 5.5: Function Single Return Principle....368 5.6: Use Type Annotations for Production Code Principle....372 5.6.1: Function Arguments Might Be Given in Wrong Order....372 5.6.2: Function Argument Might Be Given with Wrong Type....373 5.6.3: Function Return Value Type Might Be Misunderstood....373 5.6.4: Refactoring Code Is More Difficult....373 5.6.5: Forced to Write Public API Comments....374 5.6.6: Type Errors Are Not Found in Testing....374 5.7: Refactoring Principle....374 5.7.1: Rename....376 5.7.2: Extract Method....377 5.7.3: Extract Class....377 5.7.4: Extract Constant....379 5.7.5: Replace Conditionals with Polymorphism....381 5.7.6: Introduce Parameter Object....383 5.7.7: Invert If-Statement....384 5.7.8: Creating Rich Object....385 5.8: Static Code Analysis Principle....389 5.8.1: Common Static Code Analysis Issues....390 5.9: Error/Exception Handling Principle....393 5.9.1: Returning Errors....405 5.9.1.1: Returning Failure Indicator....406 5.9.1.2: Returning an Optional Value....406 5.9.1.3: Returning an Error Object....407 5.9.1.4: Adapt to Wanted Error Handling Mechanism....411 5.9.1.5: Functional Exception Handling....412 5.10: Avoid Off-By-One Errors Principle....418 5.11: Be Critical When Googling or Using Generative AI Principle....419 5.12: Make One Change At A Time Principle....421 5.13: Use Appropriate Data Structure Principle....422 5.13.1: List....422 5.13.2: Dictionary....423 5.13.3: Tuple....423 5.13.4: Set....423 5.13.5: String....424 5.13.6: Bytearray....424 5.13.7: Counter....424 5.13.8: OrderedDict....424 5.13.9: Deque (Double Ended Queue)....425 5.13.10: Customized Built-In Collections....425 5.13.11: Stack (LIFO Queue)....425 5.13.12: Queue (FIFO Queue)....425 5.13.13: Priority Queue....426 5.13.14: Synchronized Queues (LIFO/FIFO)....426 5.14: Optimization Principle....426 5.14.1: Optimization Patterns....427 5.14.1.1: Optimize Busy Loops Only Pattern....427 5.14.1.2: Remove Unnecessary Functionality Pattern....428 5.14.1.3: Object Pool Pattern....429 5.14.1.4: Use Optimal Data Structures Pattern....429 5.14.1.5: Algorithm Complexity Reduction Pattern....429 5.14.1.6: Cache Function Results Pattern....430 5.14.1.7: Buffer File I/O Pattern....431 5.14.1.8: Share Identical Objects a.k.a Flyweight Pattern....431 6: Testing Principles....432 6.1: Functional Testing Principles....432 6.1.1: Unit Testing Principle....433 6.1.1.1: Test-Driven Development (TDD)....437 6.1.1.2: Unit Specification-Driven Development (USDD)....451 6.1.1.3: Naming Conventions....457 6.1.1.4: Mocking....457 6.1.1.5: Web UI Component Unit Testing....473 6.1.2: Software Component Integration Testing Principle....475 6.1.2.1: Web UI Integration Testing....484 6.1.2.2: Setting Up Integration Testing Environment....486 6.1.3: Complete Example with BDD, ATDD, DDD, OOD and TDD....489 6.1.4: End-to-End (E2E) Testing Principle....530 6.2: Non-Functional Testing Principle....534 6.2.1: Performance Testing....534 6.2.2: Data Volume Testing....536 6.2.3: Stability Testing....536 6.2.4: Reliability Testing....537 6.2.5: Stress and Scalability Testing....538 6.2.6: Security Testing....539 6.2.7: Other Non-Functional Testing....540 6.2.7.1: Visual Testing....540 7: Security Principles....541 7.1: Shift Security to Left Principle....541 7.2: Have a Product Security Lead Principle....541 7.3: Use Threat Modelling Process Principle....542 7.3.1: Decompose Application....542 7.3.2: Determine and Rank Threats....542 7.3.2.1: STRIDE Threat Examples....543 7.3.3: Determine Countermeasures and Mitigation....545 7.3.4: Threat Modeling Example using STRIDE....546 7.3.4.1: Decompose Application....546 7.3.4.2: Determine and Rank Threats....548 7.3.5: Determine Countermeasures and Mitigation....550 7.3.6: Threat Modeling Example Using ASF....553 7.4: Security Features....555 7.4.1: Authentication and Authorization....556 7.4.1.1: OpenID Connect Authentication and Authorization in Frontend....556 7.4.1.2: OAuth2 Authorization in Backend....570 7.4.2: Password Policy....579 7.4.3: Cryptography....580 7.4.3.1: Encryption Key Lifetime and Rotation....581 7.4.4: Denial-of-service (DoS) Prevention....582 7.4.5: Database Security....582 7.4.6: SQL Injection Prevention....583 7.4.7: OS Command Injection Prevention....583 7.4.8: Security Configuration....584 7.4.9: Automatic Vulnerability Scanning....585 7.4.10: Integrity....586 7.4.11: Error Handling....586 7.4.12: Logging....586 7.4.13: Audit Logging....587 7.4.14: Input Validation....587 7.4.14.1: Validating Numbers....589 7.4.14.2: Validating Strings....589 7.4.14.3: Validating Timestamps....589 7.4.14.4: Validating Arrays....589 7.4.14.5: Validating Objects....589 7.4.14.6: Validating Files Uploaded to Server....590 8: API Design Principles....591 8.1: Frontend Facing API Design Principles....591 8.1.1: JSON-based RPC API Design Principle....591 8.1.2: REST API Design Principle....596 8.1.2.1: Creating a Resource....596 8.1.2.2: Reading Resources....599 8.1.2.3: Updating Resources....603 8.1.2.4: Deleting Resources....606 8.1.2.5: Executing Non-CRUD Actions on Resources....606 8.1.2.6: Resource Composition....607 8.1.2.7: HTTP Status Codes....609 8.1.2.8: HATEOAS and HAL....611 8.1.2.9: API Versioning....612 8.1.2.10: Documentation....613 8.1.2.11: Implementation Example....614 8.1.3: GraphQL API Design....627 8.1.4: Subscription-Based API Design....644 8.1.4.1: Server-Sent Events (SSE)....644 8.1.4.2: GraphQL Subscriptions....646 8.1.5: WebSocket Example....648 8.2: Inter-Microservice API Design Principles....663 8.2.1: Synchronous API Design Principle....663 8.2.1.1: gRPC-Based API Design Example....664 8.2.2: Asynchronous API Design Principle....672 8.2.2.1: Request-Only Asynchronous API Design....672 8.2.2.2: Request-Response Asynchronous API Design....674 9: Databases And Database Principles....677 9.1: Relational Databases....677 9.1.1: Structure of Relational Database....677 9.1.2: Use Object Relational Mapper (ORM) Principle....678 9.1.3: Entity/Table Relationships....684 9.1.3.1: One-To-One/Many Relationships....684 9.1.3.2: Many-To-Many Relationships....685 9.1.3.3: Sales Item Repository Example....688 9.1.4: Use Parameterized SQL Statements Principle....690 9.1.5: Normalization Rules....699 9.1.5.1: First Normal Form (1NF)....700 9.1.5.2: Second Normal Form (2NF)....700 9.1.5.3: Third Normal Form (3NF)....700 9.2: Document Databases....701 9.3: Key-Value Database Principle....706 9.4: Wide-Column Databases....707 9.5: Search Engines....713 10: Concurrent Programming Principles....714 10.1: Threading Principle....714 10.1.1: Parallel Executors....716 10.2: Thread Safety Principle....716 10.2.1: Use Locking for Mutual Exclusion (Mutex)....717 10.2.2: Atomic Variables....717 10.2.3: Concurrent Collections....718 10.3: Publish/Subscribe Shared State Change Principle....719 11: Teamwork Principles....721 11.1: Use Agile Framework Principle....721 11.2: Define the Done Principle....723 11.3: You Write Code for Other People Principle....725 11.4: Avoid Technical Debt Principle....725 11.5: Software Component Documentation Principle....727 11.6: Code Review Principle....730 11.6.1: Focus on Object-Oriented Design....730 11.6.2: Focus on Function Specification by Unit Tests....731 11.6.3: Focus on Proper and Uniform Naming....731 11.6.4: Don’t Focus on Premature Optimization....731 11.6.5: Detect Possible Malicious Code....731 11.7: Uniform Code Formatting Principle....732 11.8: Highly Concurrent Development Principle....732 11.8.1: Dedicated Microservices and Microlibraries....732 11.8.2: Dedicated Domains....733 11.8.3: Follow Open-Closed Principle....733 11.9: Pair Programming Principle....734 11.10: Well-Defined Development Team Roles Principle....735 11.10.1: Product Owner....735 11.10.2: Scrum Master....736 11.10.3: Software Developer....736 11.10.4: Test Automation Developer....737 11.10.5: DevOps Engineer....737 11.10.6: UI Designer....738 11.11: Competence Transfer Principle....738 12: DevSecOps....740 12.1: SecOps Lifecycle....741 12.2: DevOps Lifecycle....741 12.2.1: Plan....742 12.2.2: Code....742 12.2.3: Build and Test....743 12.2.4: Release....744 12.2.4.1: Example Dockerfile....745 12.2.4.2: Example Kubernetes Deployment....745 12.2.4.3: Example CI/CD Pipeline....749 12.2.5: Deploy....755 12.2.6: Operate....756 12.2.7: Monitor....757 12.2.7.1: Logging to Standard Input....757 12.2.7.2: Distributed Tracing....758 12.2.7.3: Metrics Collection....758 12.2.7.4: Metrics Visualization....759 12.2.7.5: Alerting....759 12.2.8: Software System Alerts Dashboard Example....760 12.2.9: Microservice Grafana Dashboard Example....760 12.2.9.1: Logging....762 12.2.9.2: OpenTelemetry Log Data Model....763 12.2.9.3: PrometheusRule Example....764 13: Conclusion....765 14: Appendix A....769 15: Appendix B....773