Network Programming with Go Essential Skills for Using and Securing Networks (Jan Newmarch) (Z-Library)

Page 1

Network Programming with Go Essential Skills for Using and Securing Networks — Jan Newmarch

Page 2

Network Programming with Go Essential Skills for Using and Securing Networks Jan Newmarch

Page 3

Network Programming with Go: Essential Skills for Using and Securing Networks Jan Newmarch Oakleigh, Victoria Australia ISBN-13 (pbk): 978-1-4842-2691-9 ISBN-13 (electronic): 978-1-4842-2692-6 DOI 10.1007/978-1-4842-2692-6 Library of Congress Control Number: 2017941517 Copyright © 2017 by Jan Newmarch This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein. Managing Director: Welmoed Spahr Editorial Director: Todd Green Acquisitions Editor: Steve Anglin Development Editor: Matthew Moodie Technical Reviewer: Ronald Petty Coordinating Editor: Mark Powers Copy Editor: Kezia Endsley Compositor: SPi Global Indexer: SPi Global Artist: SPi Global Cover image designed by Freepik Distributed to the book trade worldwide by Springer Science+Business Media New York, 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer-sbm.com, or visit www.springeronline.com. Apress Media, LLC is a California LLC and the sole member (owner) is Springer Science + Business Media Finance Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation. For information on translations, please e-mail rights@apress.com, or visit http://www.apress.com/ rights-permissions. Apress titles may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Print and eBook Bulk Sales web page at http://www.apress.com/bulk-sales. Any source code or other supplementary material referenced by the author in this book is available to readers on GitHub via the book's product page, located at www.apress.com/9781484226919. For more detailed information, please visit http://www.apress.com/source-code. Printed on acid-free paper

Page 4

(This page has no text content)

Page 5

(This page has no text content)

Page 6

(This page has no text content)

Page 7

(This page has no text content)

Page 8

(This page has no text content)

Page 9

(This page has no text content)

Page 10

(This page has no text content)

Page 11

(This page has no text content)

Page 12

(This page has no text content)

Page 13

(This page has no text content)

Page 14

(This page has no text content)

Page 15

(This page has no text content)

Page 16

xvii About the Author Jan Newmarch is head of ICT (higher education) at Box Hill Institute, adjunct professor at Canberra University, and adjunct lecturer in the School of Information Technology, Computing and Mathematics at Charles Sturt University. He is interested in more aspects of computing than he has time to pursue, but the major thrust over the last few years has developed from user interfaces under UNIX into Java, the Web, and then into general distributed systems. Jan developed a number of publicly available software systems in these areas. For the last few years, he has been looking at sound for Linux systems and programming the Raspberry Pi’s GPU. He is now exploring aspects of the IoT. He lives in Melbourne, Australia and enjoys the food and culture there, but is not so impressed by the weather.

Page 17

xix About the Technical Reviewer Ronald Petty, M.B.A., M.S. is the founder of Minimum Distance LLC, a management consulting firm based in San Francisco. He spends his time helping technology-based startups do the right thing. He is also an instructor at UC Berkeley Extension.

Page 18

xxi Preface It’s always fun to learn a new programming language, especially when it turns out to be a major one. Prior to the release of Go in 2009, I was teaching a Master’s level subject in network programming at Monash University. It’s good to have a goal when learning a new language, but this time, instead of building yet another wine cellar program, I decided to orient my lecture notes around Go instead of my (then) standard delivery vehicle of Java. The experiment worked well: apart from the richness of the Java libraries that Go was yet to match, all the programming examples transferred remarkably well, and in many cases were more elegant than the original Java programs. This book is the result. I have updated it as Go has evolved and as new technologies such as HTTP/2 have arisen. But if it reads like a textbook, well, that is because it is one. There is a large body of theoretical and practical concepts involved in network programming and this book covers some of these as well as the practicalities of building systems in Go. In terms of language popularity, Go is clearly rising. It has climbed to 16th in the TIOBE index, is 18th in the PYPL (Popularity of Programming Language), and is 15th in the RedMonk Programming Language rankings. It is generally rated as one of the fastest growing languages. There is a growing community of developers both of the core language and libraries and of the independent projects. I have tried to limit the scope of this book to the standard libraries only and to the “sub-repositories” of the Go tree. While this eliminates many excellent projects that no doubt make many programming tasks easier, restricting the book to the official Go libraries provides a clear bound. This book assumes a basic knowledge of Go. The focus is on using Go to build network applications, not on the basics of the language. Network applications are different than command-line applications, are different than applications with a graphical user interface, and so on. So the first chapter discusses architectural aspects of network programs. The second chapter is an overview of the features of Go that we use in this book. The third chapter on sockets covers the Go version of the basics underlying all TCP/IP systems. Chapters 4, 5, and 6 are more unusual in network programming books. They cover the topics of what representations of data will be used, how a network interaction will proceed, and for text, which language formats are used. Then in Chapter 7, we look at the increasingly important topic of security. In Chapter 8, we look at one of the most common application layer protocols in use, HTTP. The next four chapters are about topics related to HTTP and common data formats carried above HTTP—HTML and XML. In Chapter 13, we look at an alternative approach to network programming, remote procedure calls. Chapters 14 and 15 consider further aspects of network programming using HTTP.

Page 19

1© Jan Newmarch 2017 J. Newmarch, Network Programming with Go, DOI 10.1007/978-1-4842-2692-6_1 CHAPTER 1 Architecture This chapter covers the major architectural features of distributed systems. You can’t build a system without some idea of what you want to build. And you can’t build it if you don't know the environment in which it will work. GUI programs are different than batch processing programs; games programs are different than business programs; and distributed programs are different than standalone programs. They each have their approaches, their common patterns, the problems that typically arise, and the solutions that are often used. This chapter covers the high-level architectural aspects of distributed systems. There are many ways of looking at such systems, and many of these are dealt with. Protocol Layers Distributed systems are hard. There are multiple computers involved, which have to be connected in some way. Programs have to be written to run on each computer in the system and they all have to cooperate to get a distributed task done. The common way to deal with complexity is to break it down into smaller and simpler parts. These parts have their own structure, but they also have defined means of communicating with other related parts. In distributed systems, the parts are called protocol layers and they have clearly defined functions. They form a stack, with each layer communicating with the layer above and the layer below. The communication between layers is defined by protocols. Network communications requires protocols to cover high-level application communication all the way down to wire communication and the complexity handled by encapsulation in protocol layers.

Page 20

Chapter 1 ■ arChiteCture 2 ISO OSI Protocol Although it was never properly implemented, the OSI (Open Systems Interconnect) protocol has been a major influence in ways of talking about and influencing distributed systems design. It is commonly given as shown in Figure 1-1. OSI Layers The function of each layer from bottom to top is as follows: • The Physical layer conveys the bit stream using electrical, optical, or radio technologies. • The Data link layer puts the information packets into network frames for transmission across the physical layer, and back into information packets. • The Network layer provides switching and routing technologies. • The Transport layer provides transparent transfer of data between end systems and is responsible for end-to-end error recovery and flow control. • The Session layer establishes, manages, and terminates connections between applications. • The Presentation layer provides independence from differences in data representation (e.g., encryption). • The Application layer supports application and end-user processes. Application Presentation Session Transport Network Data Link Physical Application Presentation Session Transport Network Data Link Physical Figure 1-1. The Open Systems Interconnect protocol