UNIX Network Programming, Volume 1 The Sockets Networking API, 3rd Edition (W. Richard Stevens, Bill Fenner etc.) (Z-Library)

UNIX Network Programming The Sockets Networking API Stevens_title.fm Page i Tuesday, October 21, 2003 11:29 AM

Addison-Wesley Professional Computing Series Brian W. Kernighan and Craig Partridge, Consulting Editors Matthew H. Austern, Generic Programming and the STL: Using and Extending the C++ Standard Template Library David R. Butenhof, Programming with POSIX® Threads Brent Callaghan, NFS Illustrated Tom Cargill, C++ Programming Style William R. Cheswick/Steven M. Bellovin/Aviel D. Rubin, Firewalls and Internet Security, Second Edition: Repelling the Wily Hacker David A. Curry, UNIX® System Security: A Guide for Users and System Administrators Stephen C. Dewhurst, C++ Gotchas: Avoiding Common Problems in Coding and Design Erich Gamma/Richard Helm/Ralph Johnson/John Vlissides, Design Patterns: Elements of Reusable Object- Oriented Software Erich Gamma/Richard Helm/Ralph Johnson/John Vlissides, Design Patterns CD: Elements of Reusable Object- Oriented Software Peter Haggar, Practical Java™ Programming Language Guide David R. Hanson, C Interfaces and Implementations: Techniques for Creating Reusable Software Mark Harrison/Michael McLennan, Effective Tcl/Tk Programming: Writing Better Programs with Tcl and Tk Michi Henning/Steve Vinoski, Advanced CORBA® Programming with C++ Brian W. Kernighan/Rob Pike, The Practice of Programming S. Keshav, An Engineering Approach to Computer Networking: ATM Networks, the Internet, and the Telephone Network John Lakos, Large-Scale C++ Software Design Scott Meyers, Effective C++ CD: 85 Specific Ways to Improve Your Programs and Designs Scott Meyers, Effective C++, Second Edition: 50 Specific Ways to Improve Your Programs and Designs Scott Meyers, More Effective C++: 35 New Ways to Improve Your Programs and Designs Scott Meyers, Effective STL: 50 Specific Ways to Improve Your Use of the Standard Template Library Robert B. Murray, C++ Strategies and Tactics David R. Musser/Gillmer J. Derge/Atul Saini, STL Tutorial and Reference Guide, Second Edition: C++ Programming with the Standard Template Library John K. Ousterhout, Tcl and the Tk Toolkit Craig Partridge, Gigabit Networking Radia Perlman, Interconnections, Second Edition: Bridges, Routers, Switches, and Internetworking Protocols Stephen A. Rago, UNIX® System V Network Programming Curt Schimmel, UNIX® Systems for Modern Architectures: Symmetric Multiprocessing and Caching for Kernel Programmers W. Richard Stevens/Bill Fenner/Andrew M. Rudoff, UNIX Network Programming Volume 1, Third Edition: The Sockets Networking API W. Richard Stevens, Advanced Programming in the UNIX® Environment W. Richard Stevens, TCP/IP Illustrated, Volume 1: The Protocols W. Richard Stevens, TCP/IP Illustrated, Volume 3: TCP for Transactions, HTTP, NNTP, and the UNIX® Domain Protocols W. Richard Stevens/Gary R. Wright, TCP/IP Illustrated Volumes 1-3 Boxed Set John Viega/Gary McGraw, Building Secure Software: How to Avoid Security Problems the Right Way Gary R. Wright/W. Richard Stevens, TCP/IP Illustrated, Volume 2: The Implementation Ruixi Yuan/ W. Timothy Strayer, Virtual Private Networks: Technologies and Solutions Visit www.awprofessional.com/series/professionalcomputing for more information about these titles. PCS_Network_SP_7x9.25 9/3/03 12:17 PM Page 1

UNIX Network Programming The Sockets Networking API Volume 1 • Third Edition W. Richard Stevens Bill Fenner Andrew M. Rudoff Boston • San Francisco • New York • Toronto • Montreal London • Munich • Paris • Madrid Capetown • Sydney • Tokyo • Singapore • Mexico City Stevens_title.fm Page iii Tuesday, October 21, 2003 11:29 AM

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in this book, and Addison- Wesley was aware of a trademark claim, the designations have been printed with initial capital letters or in all capitals. The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions. No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein. The publisher offers discounts on this book when ordered in quantity for bulk purchases and special sales. For more information, please contact: U.S. Corporate and Government Sales (800) 382-3419 corpsales@pearsontechgroup.com For sales outside of the U.S., please contact: International Sales (317) 581-3793 international@pearsontechgroup.com Visit Addison-Wesley on the Web: www.awprofessional.com Library of Congress Cataloging-in-Publication Data A CIP catalog record for this book can be obtained from the Library of Congress. Copyright © 2004 by Pearson Education, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form, or by any means, electronic, mechanical, photocopying, recording, or otherwise, without the prior consent of the publisher. Printed in the United States of America. Published simultaneously in Canada. For information on obtaining permission for use of material from this work, please submit a written request to: Pearson Education, Inc. Rights and Contracts Department 75 Arlington Street, Suite 300 Boston, MA 02116 Fax: (617) 848-7047 ISBN: 0-13-141155-1 Text printed on recycled paper First printing Stevens_title.fm Page iv Tuesday, October 21, 2003 11:29 AM

To Rich. Aloha nui loa.

Stevens_title.fm Page ii Tuesday, October 21, 2003 11:29 AM

Contents Foreword xvii Preface xix Part 1. Introduction and TCP/IP 1 Chapter 1. Introduction 3 1.1 Introduction 3 1.2 A Simple Daytime Client 6 1.3 Protocol Independence 10 1.4 Error Handling: Wrapper Functions 11 1.5 A Simple Daytime Server 13 1.6 Roadmap to Client/Server Examples in the Text 16 1.7 OSI Model 18 1.8 BSD Networ king Histor y 20 1.9 Test Networ ks and Hosts 22 1.10 Unix Standards 25 1.11 64-Bit Architectures 28 1.12 Summary 29 Chapter 2. The Transpor t Layer: TCP, UDP, and SCTP 31 2.1 Introduction 31 2.2 The Big Picture 32 2.3 User Datagram Protocol (UDP) 34 vii

viii UNIX Network Programming Contents 2.4 Transmission Control Protocol (TCP) 35 2.5 Stream Control Transmission Protocol (SCTP) 36 2.6 TCP Connection Establishment and Ter mination 37 2.7 TIME_WAIT State 43 2.8 SCTP Association Establishment and Ter mination 44 2.9 Por t Numbers 50 2.10 TCP Port Numbers and Concurrent Servers 52 2.11 Buffer Sizes and Limitations 55 2.12 Standard Inter net Ser vices 61 2.13 Protocol Usage by Common Internet Applications 62 2.14 Summary 63 Part 2. Elementar y Sockets 65 Chapter 3. Sockets Introduction 67 3.1 Introduction 67 3.2 Socket Address Structures 67 3.3 Value-Result Arguments 74 3.4 Byte Order ing Functions 77 3.5 Byte Manipulation Functions 80 3.6 inet_aton, inet_addr, and inet_ntoa Functions 82 3.7 inet_pton and inet_ntop Functions 83 3.8 sock_ntop and Related Functions 86 3.9 readn, writen, and readline Functions 88 3.10 Summary 92 Chapter 4. Elementar y TCP Sockets 95 4.1 Introduction 95 4.2 socket Function 95 4.3 connect Function 99 4.4 bind Function 101 4.5 listen Function 104 4.6 accept Function 109 4.7 fork and exec Functions 111 4.8 Concurrent Ser vers 114 4.9 close Function 117 4.10 getsockname and getpeername Functions 117 4.11 Summary 120 Chapter 5. TCP Client/Server Example 121 5.1 Introduction 121 5.2 TCP Echo Server: main Function 122 5.3 TCP Echo Server: str_echo Function 123 5.4 TCP Echo Client: main Function 124 5.5 TCP Echo Client: str_cli Function 125 5.6 Normal Startup 126 5.7 Normal Ter mination 128

UNIX Network Programming Contents ix 5.8 POSIX Signal Handling 129 5.9 Handling SIGCHLD Signals 132 5.10 wait and waitpid Functions 135 5.11 Connection Abor t before accept Retur ns 139 5.12 Ter mination of Server Process 141 5.13 SIGPIPE Signal 142 5.14 Crashing of Server Host 144 5.15 Crashing and Rebooting of Server Host 144 5.16 Shutdown of Server Host 145 5.17 Summary of TCP Example 146 5.18 Data Format 147 5.19 Summary 151 Chapter 6. I/O Multiplexing: The select and poll Functions 153 6.1 Introduction 153 6.2 I/O Models 154 6.3 select Function 160 6.4 str_cli Function (Revisited) 167 6.5 Batch Input and Buffer ing 169 6.6 shutdown Function 172 6.7 str_cli Function (Revisited Again) 173 6.8 TCP Echo Server (Revisited) 175 6.9 pselect Function 181 6.10 poll Function 182 6.11 TCP Echo Server (Revisited Again) 185 6.12 Summary 188 Chapter 7. Socket Options 191 7.1 Introduction 191 7.2 getsockopt and setsockopt Functions 192 7.3 Checking if an Option Is Supported and Obtaining the Default 194 7.4 Socket States 198 7.5 Generic Socket Options 198 7.6 IPv4 Socket Options 214 7.7 ICMPv6 Socket Option 216 7.8 IPv6 Socket Options 216 7.9 TCP Socket Options 219 7.10 SCTP Socket Options 222 7.11 fcntl Function 233 7.12 Summary 236 Chapter 8. Elementar y UDP Sockets 239 8.1 Introduction 239 8.2 recvfrom and sendto Functions 240 8.3 UDP Echo Server: main Function 241 8.4 UDP Echo Server: dg_echo Function 242 8.5 UDP Echo Client: main Function 244 8.6 UDP Echo Client: dg_cli Function 245

x UNIX Network Programming Contents 8.7 Lost Datagrams 245 8.8 Ver ifying Received Response 246 8.9 Server Not Running 248 8.10 Summary of UDP Example 250 8.11 connect Function with UDP 252 8.12 dg_cli Function (Revisited) 256 8.13 Lack of Flow Control with UDP 257 8.14 Determining Outgoing Interface with UDP 261 8.15 TCP and UDP Echo Server Using select 262 8.16 Summary 264 Chapter 9. Elementar y SCTP Sockets 267 9.1 Introduction 267 9.2 Interface Models 268 9.3 sctp_bindx Function 272 9.4 sctp_connectx Function 274 9.5 sctp_getpaddrs Function 275 9.6 sctp_freepaddrs Function 275 9.7 sctp_getladdrs Function 275 9.8 sctp_freeladdrs Function 276 9.9 sctp_sendmsg Function 276 9.10 sctp_recvmsg Function 277 9.11 sctp_opt_info Function 278 9.12 sctp_peeloff Function 278 9.13 shutdown Function 278 9.14 Notifications 280 9.15 Summary 286 Chapter 10. SCTP Client/Server Example 287 10.1 Introduction 287 10.2 SCTP One-to-Many-Style Streaming Echo Server: main Function 288 10.3 SCTP One-to-Many-Style Streaming Echo Client: main Function 290 10.4 SCTP Streaming Echo Client: str_cli Function 292 10.5 Exploring Head-of-Line Blocking 293 10.6 Controlling the Number of Streams 299 10.7 Controlling Termination 300 10.8 Summary 301 Chapter 11. Name and Address Conversions 303 11.1 Introduction 303 11.2 Domain Name System (DNS) 303 11.3 gethostbyname Function 307 11.4 gethostbyaddr Function 310 11.5 getservbyname and getservbyport Functions 311 11.6 getaddrinfo Function 315 11.7 gai_strerror Function 320 11.8 freeaddrinfo Function 321 11.9 getaddrinfo Function: IPv6 322

UNIX Network Programming Contents xi 11.10 getaddrinfo Function: Examples 324 11.11 host_serv Function 325 11.12 tcp_connect Function 326 11.13 tcp_listen Function 330 11.14 udp_client Function 334 11.15 udp_connect Function 337 11.16 udp_server Function 338 11.17 getnameinfo Function 340 11.18 Re-entrant Functions 341 11.19 gethostbyname_r and gethostbyaddr_r Functions 344 11.20 Obsolete IPv6 Address Lookup Functions 346 11.21 Other Networ king Infor mation 348 11.22 Summary 349 Part 3. Advanced Sockets 351 Chapter 12. IPv4 and IPv6 Interoperability 353 12.1 Introduction 353 12.2 IPv4 Client, IPv6 Server 354 12.3 IPv6 Client, IPv4 Server 357 12.4 IPv6 Address-Testing Macros 360 12.5 Source Code Por tability 361 12.6 Summary 362 Chapter 13. Daemon Processes and the inetd Superser ver 363 13.1 Introduction 363 13.2 syslogd Daemon 364 13.3 syslog Function 365 13.4 daemon_init Function 367 13.5 inetd Daemon 371 13.6 daemon_inetd Function 377 13.7 Summary 379 Chapter 14. Advanced I/O Functions 381 14.1 Introduction 381 14.2 Socket Timeouts 381 14.3 recv and send Functions 387 14.4 readv and writev Functions 389 14.5 recvmsg and sendmsg Functions 390 14.6 Ancillary Data 395 14.7 How Much Data Is Queued? 398 14.8 Sockets and Standard I/O 399 14.9 Advanced Polling 402 14.10 Summary 408

xii UNIX Network Programming Contents Chapter 15. Unix Domain Protocols 411 15.1 Introduction 411 15.2 Unix Domain Socket Address Structure 412 15.3 socketpair Function 414 15.4 Socket Functions 415 15.5 Unix Domain Stream Client/Server 416 15.6 Unix Domain Datagram Client/Ser ver 418 15.7 Passing Descriptors 420 15.8 Receiving Sender Credentials 429 15.9 Summary 432 Chapter 16. Nonblocking I/O 435 16.1 Introduction 435 16.2 Nonblocking Reads and Writes: str_cli Function (Revisited) 437 16.3 Nonblocking connect 448 16.4 Nonblocking connect: Daytime Client 449 16.5 Nonblocking connect: Web Client 452 16.6 Nonblocking accept 461 16.7 Summary 463 Chapter 17. ioctl Operations 465 17.1 Introduction 465 17.2 ioctl Function 466 17.3 Socket Operations 466 17.4 File Operations 468 17.5 Interface Configuration 468 17.6 get_ifi_info Function 469 17.7 Interface Operations 480 17.8 ARP Cache Operations 481 17.9 Routing Table Operations 483 17.10 Summary 484 Chapter 18. Routing Sockets 485 18.1 Introduction 485 18.2 Datalink Socket Address Structure 486 18.3 Reading and Writing 487 18.4 sysctl Operations 495 18.5 get_ifi_info Function (Revisited) 500 18.6 Interface Name and Index Functions 504 18.7 Summary 508 Chapter 19. Key Management Sockets 511 19.1 Introduction 511 19.2 Reading and Writing 512 19.3 Dumping the Security Association Database (SADB) 514 19.4 Creating a Static Security Association (SA) 517 19.5 Dynamically Maintaining SAs 524 19.6 Summary 528

UNIX Network Programming Contents xiii Chapter 20. Broadcasting 529 20.1 Introduction 529 20.2 Broadcast Addresses 531 20.3 Unicast versus Broadcast 532 20.4 dg_cli Function Using Broadcasting 535 20.5 Race Conditions 538 20.6 Summary 547 Chapter 21. Multicasting 549 21.1 Introduction 549 21.2 Multicast Addresses 549 21.3 Multicasting versus Broadcasting on a LAN 553 21.4 Multicasting on a WAN 556 21.5 Source-Specific Multicast 558 21.6 Multicast Socket Options 559 21.7 mcast_join and Related Functions 565 21.8 dg_cli Function Using Multicasting 570 21.9 Receiving IP Multicast Infrastr ucture Session Announcements 571 21.10 Sending and Receiving 575 21.11 Simple Networ k Time Protocol (SNTP) 579 21.12 Summary 584 Chapter 22. Advanced UDP Sockets 587 22.1 Introduction 587 22.2 Receiving Flags, Destination IP Address, and Interface Index 588 22.3 Datagram Truncation 594 22.4 When to Use UDP Instead of TCP 594 22.5 Adding Reliability to a UDP Application 597 22.6 Binding Interface Addresses 608 22.7 Concurrent UDP Servers 612 22.8 IPv6 Packet Infor mation 615 22.9 IPv6 Path MTU Control 618 22.10 Summary 620 Chapter 23. Advanced SCTP Sockets 621 23.1 Introduction 621 23.2 An Autoclosing One-to-Many-Style Server 621 23.3 Par tial Deliver y 622 23.4 Notifications 625 23.5 Unordered Data 629 23.6 Binding a Subset of Addresses 630 23.7 Determining Peer and Local Address Infor mation 631 23.8 Finding an Association ID Given an IP Address 635 23.9 Heartbeating and Address Failure 636 23.10 Peeling Off an Association 637 23.11 Controlling Timing 639 23.12 When to Use SCTP Instead of TCP 641 23.13 Summary 643

xiv UNIX Network Programming Contents Chapter 24. Out-of-Band Data 645 24.1 Introduction 645 24.2 TCP Out-of-Band Data 645 24.3 sockatmark Function 654 24.4 TCP Out-of-Band Data Recap 661 24.5 Summary 662 Chapter 25. Signal-Driven I/O 663 25.1 Introduction 663 25.2 Signal-Dr iven I/O for Sockets 664 25.3 UDP Echo Server Using SIGIO 666 25.4 Summary 672 Chapter 26. Threads 675 26.1 Introduction 675 26.2 Basic Thread Functions: Creation and Ter mination 676 26.3 str_cli Function Using Threads 679 26.4 TCP Echo Server Using Threads 681 26.5 Thread-Specific Data 686 26.6 Web Client and Simultaneous Connections (Continued) 694 26.7 Mutexes: Mutual Exclusion 697 26.8 Condition Variables 701 26.9 Web Client and Simultaneous Connections (Continued) 705 26.10 Summary 707 Chapter 27. IP Options 709 27.1 Introduction 709 27.2 IPv4 Options 709 27.3 IPv4 Source Route Options 711 27.4 IPv6 Extension Headers 719 27.5 IPv6 Hop-by-Hop Options and Destination Options 719 27.6 IPv6 Routing Header 725 27.7 IPv6 Sticky Options 731 27.8 Historical IPv6 Advanced API 732 27.9 Summary 733 Chapter 28. Raw Sockets 735 28.1 Introduction 735 28.2 Raw Socket Creation 736 28.3 Raw Socket Output 737 28.4 Raw Socket Input 739 28.5 ping Program 741 28.6 traceroute Program 755 28.7 An ICMP Message Daemon 769 28.8 Summary 786

UNIX Network Programming Contents xv Chapter 29. Datalink Access 787 29.1 Introduction 787 29.2 BSD Packet Filter (BPF) 788 29.3 Datalink Provider Interface (DLPI) 790 29.4 Linux: SOCK_PACKET and PF_PACKET 791 29.5 libpcap: Packet Capture Librar y 792 29.6 libnet: Packet Creation and Injection Librar y 793 29.7 Examining the UDP Checksum Field 793 29.8 Summary 815 Chapter 30. Client/Ser ver Design Alternatives 817 30.1 Introduction 817 30.2 TCP Client Alternatives 819 30.3 TCP Test Client 820 30.4 TCP Iterative Ser ver 821 30.5 TCP Concurrent Server, One Child per Client 822 30.6 TCP Prefor ked Server, No Locking Around accept 826 30.7 TCP Prefor ked Server, File Locking Around accept 832 30.8 TCP Prefor ked Server, Thread Locking Around accept 835 30.9 TCP Prefor ked Server, Descr iptor Passing 836 30.10 TCP Concurrent Server, One Thread per Client 842 30.11 TCP Prethreaded Server, per-Thread accept 844 30.12 TCP Prethreaded Server, Main Thread accept 846 30.13 Summary 849 Chapter 31. STREAMS 851 31.1 Introduction 851 31.2 Over view 851 31.3 getmsg and putmsg Functions 856 31.4 getpmsg and putpmsg Functions 857 31.5 ioctl Function 857 31.6 Transpor t Provider Interface (TPI) 858 31.7 Summary 868 Appendix A. IPv4, IPv6, ICMPv4, and ICMPv6 869 A.1 Introduction 869 A.2 IPv4 Header 869 A.3 IPv6 Header 871 A.4 IPv4 Addresses 874 A.5 IPv6 Addresses 877 A.6 Internet Control Message Protocols (ICMPv4 and ICMPv6) 882 Appendix B. Vir tual Networks 885 B.1 Introduction 885 B.2 The MBone 885 B.3 The 6bone 887 B.4 IPv6 Transition: 6to4 889

xvi UNIX Network Programming Contents Appendix C. Debugging Techniques 891 C.1 System Call Tracing 891 C.2 Standard Internet Services 893 C.3 sock Program 893 C.4 Small Test Programs 896 C.5 tcpdump Program 896 C.6 netstat Program 896 C.7 lsof Program 897 Appendix D. Miscellaneous Source Code 899 D.1 unp.h Header 899 D.2 config.h Header 904 D.3 Standard Error Functions 910 Appendix E. Solutions to Selected Exercises 913 Bibliography 947 Index 955

Foreword When the original text of this book arrived in 1990, it was quickly recognized as the definitive reference for programmers to learn network programming techniques. Since then, the art of computer networking has changed dramatically. All it takes is a look at the return address for comments from the original text (‘‘uunet!hsi!netbook’’) to make this clear. (How many readers will even recognize this as an address in the UUCP dialup network that was commonplace in the 1980s?) Today, UUCP networks are a rarity and new technologies such as wireless networks are becoming ubiquitous! With these changes, new network protocols and program- ming paradigms have been developed. But, programmers have lacked a good reference from which to learn the intricacies of these new techniques. This book fills that void. Readers who have a dog-eared copy of the original book will want a new copy for the updated programming techniques and the substantial new material describing next-generation protocols such as IPv6. Everyone will want this book because it provides a great mix of practical experience, historical perspective, and a depth of understanding that only comes from being intimately involved in the field. I’ve already enjoyed and learned from reading this book, and surely you will, too. Sam Leffler xvii

Stevens_title.fm Page ii Tuesday, October 21, 2003 11:29 AM

Preface Introduction This book is for people who want to write programs that communicate with each other using an application program interface (API) known as sockets. Some readers may be very familiar with sockets already, as that model has become synonymous with network programming. Others may need an introduction to sockets from the ground up. The goal of this book is to offer guidance on network programming for beginners as well as professionals, for those developing new network-aware applications as well as those maintaining existing code, and for people who simply want to understand how the net- working components of their system function. All the examples in this text are actual, runnable code tested on Unix systems. However, many non-Unix systems support the sockets API and the examples are largely operating system-independent, as are the general concepts we present. Virtually every operating system (OS) provides numerous network-aware applications such as Web browsers, email clients, and file-sharing servers. We discuss the usual partitioning of these applications into client and server and write our own small examples of these many times throughout the text. xix