C Programming for System, Network, and Cloud Engineers: Techniques for Processes, Memory, Network Applications, and Linux Namespaces

Table of Contents....5 About the Author....13 About the Technical Reviewer....14 Acknowledgments....15 Introduction....16 Chapter 1: Basic Computer System Architecture and Essential Operating System Concepts....18 Basic Computer System Architecture....19 CPU Architecture....20 Memory Hierarchy....22 CPU and IO Device Interactions....23 Inspect System Configurations on a Linux OS....24 Operating System Responsibilities....28 Users Program Access to OS Services....30 OS Process Management Activities and Services....33 OS Role in Processes Management....34 OS Offering Services for User Programs....37 OS Role in Memory Management....38 OS IO Devices, File Management, and Protection Services....39 Importance of Compiler, Linker, and Loader....40 Compiler....41 Compiler Options for Application Performance Tuning....42 Compiler Options to Protect Application Code....43 Compiler Options to Handle Runtime Errors....45 Linking and Loading Options....46 Developers Must Know Linking Options to Handle Runtime Errors....47 Runtime Environment of a Process....48 Process Memory Layout....48 Process Execution Environment....51 Process and Memory Access-Related Issues....52 Process and IO Devices Access-Related Issues....55 Process Reliability and Security Issues....57 Process Memory Layout-Related Access Checkpoints....58 Careful About Shared Libraries Access....60 Interprocess Communication-Related Access Checkpoints....60 C Programmer Powers and Responsibilities....61 C Programming Powers....62 C Programmer Responsibilities....64 Summary....67 Chapter 2: Quick Revision of Powerful C Constructs....68 Procedure-Oriented Programming....69 Understand C Functions Memory Layout and Execution Procedure....73 Recursive Functions....88 Nested Functions in GNU C....94 Arrays....98 Powerful Ways for Organizing Arrays Data Elements and Accessing....101 Arrays: Indexing Ways for Accessing Data Structures....103 Hands-On Activities for Organizing Arrays Data Elements and Accessing Them Efficiently....108 Strings and Buffers....117 Strings Processing Hands-On Activities....122 C Structures....129 Passing Structure Variables Between Functions....132 Structure and Its Data Members Memory Layout....134 Hands-On Activity-1....135 Power of C Structures for Modeling....140 Data Structures Modeling....140 Protocols and Protocol Stack Modeling....143 Hands-On Activity-2....145 C Unions....149 An Important Use Case of Unions....151 Hands-On Activity....152 Summary....157 Practice Tasks....158 Chapter 3: Practice C Programming with Important Tools....160 Compilation and Linking Techniques for Enhancing Programming Skills....161 gcc Compilation Options for Performance Tuning....162 gcc Compilation Options for Linking....168 gcc Compilation Options for Secured Code....178 Memory Access Errors Handling Techniques and Tools....189 Memory and Other Important Errors Handling gcc Options for Reliable Code....189 Memory Error Handling Using Valgrind....198 Code Coverage Analysis for Avoiding Performance Issues....200 Code Coverage Tools (gcc and gcov) Usage Steps....202 Hands-On Activity....205 GDB: The GNU Project Debugger....212 Go Through a Code Execution and Inspect the Code and Data....212 Inspect the Code and Data Using gdb Commands....219 Handle Application Runtime Errors....226 Summary....236 Practice Tasks....237 Chapter 4: Power of C: Pointers....239 Importance of Pointer Programming....240 Basic Concepts of C Pointers....241 Basic Pointers....241 Multilevel Pointers....242 Use Cases of C Pointers....244 Efficient Communication Approach for Functions....245 Flexible Data Accessing Approaches....247 Pointers to Data Structures....248 Pointers to Arrays....248 Pointers to Structures....249 Pointers Flexible Traversals....250 Void Pointers to Implement Generic Data Structures....251 Efficient and Reliable Ways to Access Arrays and Strings Using Pointers....252 Pointers to Array Subparts....253 Pointers to Multidimensional Arrays....254 Pointers to Strings....255 Pointers to Simple Strings....256 Pointers to Array of Strings....256 Pointers to Array of Array of Strings....257 Hands-On Activities for Organizing Arrays Data Elements and Accessing Them Efficiently....258 Efficient and Reliable Ways to Access Structures Using Pointers....262 Basic Pointers Accessing Ways for Structures....263 Efficient Ways for Structure Variables Passing to Functions....263 Advanced Pointers Accessing Ways for Structures....265 Hands-On Activity....266 Pointers and Functions for Powerful Programming Techniques....271 Pointers to a Variety of Functions....272 Basic Function Pointers....272 Advanced Function Pointers....273 Typedef....274 Declare Array of Function Pointers....274 Declare a Pointer to a Function Returning an Array of Function Pointers....275 C Ways for Polymorphism....277 A Quick Example....277 Use typedef to Simplify Interface Name....278 Hands-On Activity....278 Summary....283 Practice Tasks....283 Chapter 5: C Programming for Memory Management....285 Introduction to Dynamic Memory Management....286 Importance of Dynamic Memory Management....286 Important C System Calls for Memory Management....287 Important C System Calls for Memory Management....290 C Library Functions for Basic Dynamic Memory Management....290 C System Calls for Extending Memory Blocks Dynamically....293 C System Calls for Handling Multiple Blocks of Memory Allocation....295 Error Handling Programming Techniques for Memory Management....297 Best Dynamic Memory Management Practices....297 Memory Error Handling Techniques Using Valgrind....301 Hands-On Activity Using Valgrind Tools....304 Setting Up Memory Buffers and Accessing Them in a Reliable Manner....309 Hands-On Activity-1....310 Hands-On Activity-2....316 Hands-On Activity-3....322 Summary....328 Practice Tasks....328 Chapter 6: Process and Thread Management Tasks for C Programmers....330 Introduction to Processes and Threads....331 Process Management Basic Activities Using Fork....332 Thread Management Basic Activities....334 Explore the Power of Process Management Using C Programming....339 Developers Role and Activities in Process Management....339 Creating a Specific Number of Processes....342 Control the Multiple Processes Execution Order....346 Create N Processes and Load Them with Executable Images....350 Safe and Reliable Programming Ways for Process Management....355 Importance of Parent and Child Processes Exit Order....357 Orphan Process....357 Zombie Process....359 Handle Carefully Data Sharing Activities Between Parent and Child Processes....360 Parent and Child Processes Segments....361 Use Valgrind to Check Processes for Memory Leaks....364 Sharing of File Descriptors Between Parent and Child Processes....365 Use Valgrind to Check File Descriptor Leaks....367 Reliable Ways of Executing a New Process in a Child Process....369 Importance of Interrupt Handling in Process Management....372 Explore Power of Threads Management Using C Programming....378 Developers Role in Threads Management....378 Creating Threads and Assigning Tasks....378 Challenges in Assigning Specific Tasks and Data to Threads....382 Common Threads Issues and Respective Solutions....385 Critical Section and Synchronization Issues Handling Tasks....385 Pthreads Mutex Objects for Locking and Synchronization....386 Mutex and Conditional Variable for Synchronization....388 Common Data Sharing Issues and Threads Performance....391 False Sharing....391 Hands-On Activity-1....393 Hands-On Activity-2....398 Safe and Reliable Programming Ways for Threads Management....402 Improve the Performance of a Multithreaded Application....402 Multithreading Application Performance Tuning....403 First Approach....406 Second Approach....407 Third Approach....409 Quickly Identify and Handle Threads Synchronization Issues....411 Identify Data Race Conditions Quickly....411 Identify Locking Issues Quickly....413 Summary....416 Practice Tasks....417 Chapter 7: Handle Interprocess Activities Using C Programming....419 Essential Linux Interprocess Communication (IPC) Approaches....420 Linux Basic Services for File Handling....421 Importance of dup and dup2....422 PIPEs....423 OS Role in Pipes Management....423 Use Case of PIPEs....424 FIFO....424 OS Role in FIFO Management....425 Use Case of FIFOs....426 Message Queues....426 OS Role in Message Queues Management....427 Use Case of Message Queues....428 Shared Memory....428 OS Role in Shared Memory Management....429 Use Case of Shared Memory....429 Semaphores....430 OS Role in Semaphores Management....430 Use Case of Semaphores....431 Explore C Programming and Linux IPC Tools for IPC Management....432 Usage of PIPEs in C....432 Sample C Program Using a Pipe....433 Developers Role in Case of Pipe Usage....435 Usage of FIFO in C....436 Sample C Program Using a FIFO....437 Developers Role in Case of FIFO Usage....440 Usage of Message Queues in C....441 Sample C Program Using a Message Queue....443 Developers Role in Case of Message Queue Usage....447 Usage of Shared Memory in C....448 Sample C Program Using a Shared Memory....449 Developers Role in Case of Shared Memory Usage....453 Explore C Ways to Implement Service Chaining and Asynchronous Programming....454 Hands-On Activity-1....455 Hands-On Activity-2....459 Hands-On Activity-3....465 Essential IPC Synchronization Handling Tools....471 Semaphores....472 Usage of Semaphore in C....473 Developers Role in Case of Shared Memory Usage....474 Hands-On Activity-1....475 Hands-On Activity-2....480 Summary....485 Practice Tasks....486 Chapter 8: Essential Network Socket Programming Skills....487 Quick Introduction to TCPIP Stack and Network Applications....488 TCPIP Stack....489 TCPIP Layers....489 TCPIP Stack and Network Interactions....491 Network Devices and Their TCPIP Stack Layers....492 Basics of Protocol Address....492 IP Address....493 TCP Socket Programming....495 TCP Rules....496 TCP Connection Setup....496 TCP Connection Shutdown....497 TCP Socket Programming Constructs....500 General TCP Server Program Structure....510 Multiprocess TCP Server Program Structure....512 General TCP Client Program Structure....514 TCP Socket Programming Practice....515 TCP Server Program for Waiting for Messages....516 TCP Client Program for Messages Exchange....520 Multiprocess TCP Server Program....524 UDP Socket Programming....529 UDP Socket Programming in C....531 Importance of Connection-Oriented Socket for UDP Clients....532 UDP Traffic Exchange System Calls....533 UDP Socket Programming in C....537 General UDP Server Program Structure....537 General UDP Connection-Less Client Program Structure....539 General UDP Connection-Oriented Client Program Structure....541 Hands-On Activity-1....543 C Programming Ways for Monitoring and Inspecting TCPUDP Applications Traffic....553 Packet Monitoring and Filtering Activities....553 Basic Traffic Monitoring and Inspecting Program Structure....554 Hands-On Activity-1....556 Hands-On Activity-2....561 Summary....565 Practice Tasks....565 Chapter 9: Essential Advanced Socket Programming Ways Using C Sockets....567 Important Socket Programming Approaches....568 Multiprocessing and Multithreading....570 Must-Know Socket Options....573 Non-blocking Socket Programming....578 Non-blocking Ways of Socket Programming....579 Usage of fcntl and select....579 Hands-On Activity-1....582 Hands-On Activity-2....592 Handle TCP Client-Server Performance and Reliability Issues....598 The Reliable TCP Server and Client....601 Hands-On Activity-1....611 Hands-On Activity-2....613 Hands-On Activity-3....616 Handle UDP Client-Server Performance and Reliability Issues....618 Hands-On Activity-1....619 Hands-On Activity-2....626 Summary....635 Practice Tasks....636 Chapter 10: Learn C Programming Skills for Virtualization....637 Introduction to Linux Namespaces....638 Linux Namespaces....639 Linux cgroups....641 Learn How to Program Linux Namespaces and cgroups....644 Detach a Process from Inherited Namespaces....644 Unshare System Call Usage....645 Detach from the UTS Namespace Using unshare....646 Detach from the User Namespace Using unshare....648 Detach from the Network Namespace Using unshare....649 Attach a Process with Existing Namespaces....650 Set Up a New Namespace Using unshare and Attach It with setns....651 Attach with an Existing UTS Namespace Using setns....652 Attach with Existing User and Network Namespaces....654 Clone New Processes with Suitable Namespaces....657 Start Multiple Processes with Necessary New Namespaces....659 Start a Process with Multiple Namespaces....662 Start a Process with Limited Namespaces and Attach New Namespaces Later....663 Learn Setting Up and Using Control Groups....666 Setting Up a Suitable Resource Control Group....667 Configure Resource Control Group Limits....667 Assign Processes to a cgroup....668 Experiment with Process and Memory Namespaces....669 Hands-On Activity-1....670 Hands-On Activity-2....675 Hands-On Activity-3....680 Hands-On Activity-4....685 Hands-On Activity-5....688 Experiment with Networking Namespaces....693 Learn Setting Up Network Namespaces....693 Practice Using iproute2....694 Various Virtual Networks Setups....696 Basic Virtual Network Setup....696 Interconnect Network Namespace and Host....698 Interconnect Two Different Subnet Network Namespaces....699 Hands-On Activity-1....702 Hands-On Activity-2....705 Hands-On Activity-3....708 Summary....713