Statistics
46
Views
0
Downloads
0
Donations
Author：Josiah L. Carlson
Summary Redis in Action introduces Redis and walks you through examples that demonstrate how to use it effectively. You'll begin by getting Redis set up properly and then exploring the key-value model. Then, you'll dive into real use cases including simple caching, distributed ad targeting, and more. You'll learn how to scale Redis from small jobs to massive datasets. Experienced developers will appreciate chapters on clustering and internal scripting to make Redis easier to use. About the Technology When you need near-real-time access to a fast-moving data stream, key-value stores like Redis are the way to go. Redis expands on the key-value pattern by accepting a wide variety of data types, including hashes, strings, lists, and other structures. It provides lightning-fast operations on in-memory datasets, and also makes it easy to persist to disk on the fly. Plus, it's free and open source. About this book Redis in Action introduces Redis and the key-value model. You'll quickly dive into real use cases including simple caching, distributed ad targeting, and more. You'll learn how to scale Redis from small jobs to massive datasets and discover how to integrate with traditional RDBMS or other NoSQL stores. Experienced developers will appreciate the in-depth chapters on clustering and internal scripting. Written for developers familiar with database concepts. No prior exposure to NoSQL database concepts nor to Redis itself is required. Appropriate for systems administrators comfortable with programming. What's Inside Redis from the ground up Preprocessing real-time data Managing in-memory datasets Pub/sub and configuration Persisting to disk About the Author Dr. Josiah L. Carlson is a seasoned database professional and an active contributor to the Redis community. Purchase of the print book includes a free eBook in PDF, Kindle, and ePub formats from Manning Publications. Table of Contents PART 1 GETTING STARTED Getting to know Redis Anatomy of a Redis web
Tags
Support Statistics
¥.00 ·
0times
Text Preview (First 20 pages)
Registered users can read the full content for free
Register as a Gaohf Library member to read the complete e-book online for free and enjoy a better reading experience.
Page
1
M A N N I N G Josiah L. Carlson FOREWORD BY Salvatore Sanfilippo IN ACTION
Page
2
Redis in Action Download from Wow! eBook <www.wowebook.com>
Page
3
Download from Wow! eBook <www.wowebook.com>
Page
4
Redis in Action JOSIAH L. CARLSON M A N N I N G Shelter IslandDownload from Wow! eBook <www.wowebook.com>
Page
5
For online information and ordering of this and other Manning books, please visit www.manning.com. The publisher offers discounts on this book when ordered in quantity. For more information, please contact Special Sales Department Manning Publications Co. 20 Baldwin Road PO Box 261 Shelter Island, NY 11964 Email: orders@manning.com ©2013 by Manning Publications Co. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher. Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps. Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine. Manning Publications Co. Development editor: Elizabeth Lexleigh 20 Baldwin Road Technical proofreaders: James Philips, Kevin Chang, PO Box 261 and Nicholas Lindgren Shelter Island, NY 11964 Java translator: Eric Van Dewoestine Copyeditor: Benjamin Berg Proofreader: Katie Tennant Typesetter: Gordan Salinovic Cover designer: Marija Tudor ISBN 9781935182054 Printed in the United States of America 1 2 3 4 5 6 7 8 9 10 – MAL – 18 17 16 15 14 13Download from Wow! eBook <www.wowebook.com>
Page
6
To my dear wife, See Luan, and to our baby girl, MikelaDownload from Wow! eBook <www.wowebook.com>
Page
7
Download from Wow! eBook <www.wowebook.com>
Page
8
brief contents PART 1 GETTING STARTED . ..........................................................1 1 ■ Getting to know Redis 3 2 ■ Anatomy of a Redis web application 24 PART 2 CORE CONCEPTS.............................................................37 3 ■ Commands in Redis 39 4 ■ Keeping data safe and ensuring performance 63 5 ■ Using Redis for application support 90 6 ■ Application components in Redis 110 7 ■ Search-based applications 154 8 ■ Building a simple social network 185 PART 3 NEXT STEPS. ................................................................207 9 ■ Reducing memory use 209 10 ■ Scaling Redis 228 11 ■ Scripting Redis with Lua 249vii Download from Wow! eBook <www.wowebook.com>
Page
9
Download from Wow! eBook <www.wowebook.com>
Page
10
contents foreword xv preface xvii acknowledgments xix about this book xxi about the cover illustration xxv PART 1 GETTING STARTED ..................................................1 1 Getting to know Redis 3 1.1 What is Redis? 4 Redis compared to other databases and software 4 ■ Other features 6 ■ Why Redis? 6 1.2 What Redis data structures look like 7 Strings in Redis 9 ■ Lists in Redis 10 ■ Sets in Redis 11 Hashes in Redis 12 ■ Sorted sets in Redis 13 1.3 Hello Redis 15 Voting on articles 15 ■ Posting and fetching articles 19 Grouping articles 20 1.4 Getting help 22 1.5 Summary 22ix Download from Wow! eBook <www.wowebook.com>
Page
11
CONTENTSx2 Anatomy of a Redis web application 24 2.1 Login and cookie caching 25 2.2 Shopping carts in Redis 29 2.3 Web page caching 30 2.4 Database row caching 31 2.5 Web page analytics 34 2.6 Summary 36 PART 2 CORE CONCEPTS...................................................37 3 Commands in Redis 39 3.1 Strings 40 3.2 Lists 43 3.3 Sets 46 3.4 Hashes 48 3.5 Sorted sets 50 3.6 Publish/subscribe 54 3.7 Other commands 57 Sorting 57 ■ Basic Redis transactions 58 ■ Expiring keys 61 3.8 Summary 62 4 Keeping data safe and ensuring performance 63 4.1 Persistence options 64 Persisting to disk with snapshots 65 ■ Append-only file persistence 68 ■ Rewriting/compacting append-only files 70 4.2 Replication 70 Configuring Redis for replication 71 ■ Redis replication startup process 72 ■ Master/slave chains 73 ■ Verifying disk writes 74 4.3 Handling system failures 75 Verifying snapshots and append-only files 76 ■ Replacing a failed master 77 4.4 Redis transactions 78 Defining users and their inventory 79 ■ Listing items in the marketplace 80 ■ Purchasing items 82 4.5 Non-transactional pipelines 84Download from Wow! eBook <www.wowebook.com>
Page
12
CONTENTS xi4.6 Performance considerations 87 4.7 Summary 89 5 Using Redis for application support 90 5.1 Logging to Redis 91 Recent logs 91 ■ Common logs 92 5.2 Counters and statistics 93 Storing counters in Redis 94 ■ Storing statistics in Redis 98 Simplifying our statistics recording and discovery 100 5.3 IP-to-city and -country lookup 102 Loading the location tables 102 ■ Looking up cities 104 5.4 Service discovery and configuration 104 Using Redis to store configuration information 105 ■ One Redis server per application component 106 ■ Automatic Redis connection management 107 5.5 Summary 109 6 Application components in Redis 110 6.1 Autocomplete 111 Autocomplete for recent contacts 111 ■ Address book autocomplete 113 6.2 Distributed locking 116 Why locks are important 117 ■ Simple locks 119 ■ Building a lock in Redis 120 ■ Fine-grained locking 123 ■ Locks with timeouts 126 6.3 Counting semaphores 127 Building a basic counting semaphore 127 ■ Fair semaphores 129 Refreshing semaphores 132 ■ Preventing race conditions 132 6.4 Task queues 134 First-in, first-out queues 134 ■ Delayed tasks 137 6.5 Pull messaging 140 Single-recipient publish/subscribe replacement 140 ■ Multiple-recipient publish/subscribe replacement 141 6.6 Distributing files with Redis 146 Aggregating users by location 146 ■ Sending files 148 Receiving files 149 ■ Processing files 150 6.7 Summary 152Download from Wow! eBook <www.wowebook.com>
Page
13
CONTENTSxii7 Search-based applications 154 7.1 Searching in Redis 155 Basic search theory 155 ■ Sorting search results 161 7.2 Sorted indexes 163 Sorting search results with ZSETs 163 ■ Non-numeric sorting with ZSETs 165 7.3 Ad targeting 167 What’s an ad server? 168 ■ Indexing ads 168 ■ Targeting ads 171 Learning from user behavior 175 7.4 Job search 181 Approaching the problem one job at a time 181 ■ Approaching the problem like search 182 7.5 Summary 183 8 Building a simple social network 185 8.1 Users and statuses 186 User information 186 ■ Status messages 187 8.2 Home timeline 188 8.3 Followers/following lists 190 8.4 Posting or deleting a status update 192 8.5 Streaming API 196 Data to be streamed 196 ■ Serving the data 197 ■ Filtering streamed messages 200 8.6 Summary 206 PART 3 NEXT STEPS .......................................................207 9 Reducing memory use 209 9.1 Short structures 210 The ziplist representation 210 ■ The intset encoding for SETs 212 ■ Performance issues for long ziplists and intsets 213 9.2 Sharded structures 215 HASHes 216 ■ SETs 219 9.3 Packing bits and bytes 222 What location information should we store? 222 ■ Storing packed data 224 ■ Calculating aggregates over sharded STRINGs 225 9.4 Summary 227Download from Wow! eBook <www.wowebook.com>
Page
14
CONTENTS xiii10 Scaling Redis 228 10.1 Scaling reads 228 10.2 Scaling writes and memory capacity 232 Handling shard configuration 233 ■ Creating a server-sharded connection decorator 234 10.3 Scaling complex queries 236 Scaling search query volume 236 ■ Scaling search index size 236 Scaling a social network 241 10.4 Summary 248 11 Scripting Redis with Lua 249 11.1 Adding functionality without writing C 250 Loading Lua scripts into Redis 250 ■ Creating a new status message 252 11.2 Rewriting locks and semaphores with Lua 255 Why locks in Lua? 255 ■ Rewriting our lock 256 Counting semaphores in Lua 258 11.3 Doing away with WATCH/MULTI/EXEC 260 Revisiting group autocomplete 260 ■ Improving the marketplace, again 262 11.4 Sharding LISTs with Lua 265 Structuring a sharded LIST 265 ■ Pushing items onto the sharded LIST 266 ■ Popping items from the sharded LIST 268 Performing blocking pops from the sharded LIST 269 11.5 Summary 271 appendix A Quick and dirty setup 273 appendix B Other resources and references 281 index 284Download from Wow! eBook <www.wowebook.com>
Page
15
Download from Wow! eBook <www.wowebook.com>
Page
16
foreword Redis was created about three years ago for practical reasons: basically, I was trying to do the impossible with an on-disk SQL database. I was handling a large write-heavy load with the only hardware I was able to afford—a little virtualized instance. My problem was conceptually simple: my server was receiving a stream of page views from multiple websites using a small JavaScript tracker. I needed to store the lat- est n page views for every site and show them in real time to users connected to a web interface, while maintaining a small history. With a peak load of a few thousand page views per second, whatever my database schema was, and whatever trade-offs I was willing to pick, there was no way for my SQL store to handle the load with such poor hardware. My inability to upgrade the hard- ware for cost concerns coupled with the feeling that to handle a capped list of values shouldn’t have been so hard, after all, gave me the idea of creating a throw-away pro- totype of an in-memory data store that could handle lists as a native data type, with constant-time pop and push operations on both sides of the lists. To make a long story short, the concept worked, I rewrote the first prototype using the C language, added a fork-based persistence feature, and Redis was born. Fast-forward to the present. After three years, the project has evolved in significant ways. We have a more robust system now, and with Redis 2.6 just released and the major work in progress being cluster and HA features, Redis is entering its maturity period. One of the most remarkable advancements in the Redis ecosystem, in my opinion, is its community of users and contributors, from the redis.io website to the Redis Google Group. Stemming from the GitHub issues system, there are thousandsxv Download from Wow! eBook <www.wowebook.com>
Page
17
FOREWORDxviof people involved in the project, writing client libraries, contributing fixes, and help- ing other users. Redis is still a community project: it’s BSD licensed. There are no closed source add-ons or enhanced versions you need to pay for. The reference documentation is as accurate as possible, and it’s extremely easy to get help and get in touch with Redis developers or experts. Redis started in a pragmatic way, with a programmer who needed to get things done and couldn’t find the right tool for the tasks at hand. This is why I think a theo- retical book wouldn’t serve Redis well, and why I like Redis in Action: it’s a book for people that want to get things done. It doesn’t limit itself to a sterile description of the API; Redis features and data types are explored in depth using compelling examples. At the same time, Redis in Action comes from the Redis community, and more spe- cifically from someone who, before publishing this book, has already helped hun- dreds of Redis users in many different ways—from schema designs to hardware latency issues. The Redis Group is full of Josiah’s advice and contributions. The fact that system operation topics are also covered is a big plus. The reality is that most people need to both develop the application software and handle the deployment of the server. And I’d say that you need to understand system operations and the fundamental limits of the hardware and the system software you’re using in order to write an application that makes the best use of both. The result of these efforts is a book that will get you into Redis in a direct way, pointing your attention in the right directions to avoid common pitfalls. I think Redis in Action is a great addition to the Redis ecosystem and will be welcomed by the com- munity of Redis users. SALVATORE SANFILIPPO CREATOR OF REDISDownload from Wow! eBook <www.wowebook.com>
Page
18
preface In March of 2010 I began working in Beverly Hills with Chris Testa, a friend I’d met while at Google in Santa Monica. He had hired me to be the architect of a small startup that he was team lead/director for; I was to be the research branch. While talking one afternoon about how to solve an unrelated problem, Chris men- tioned Redis as a database that I might find interesting (given my education in theoret- ical computer science). Several weeks later, after using and patching Redis for our purposes, I started participating on the mailing list, offering advice and a patch or two. As time went on, I used Redis for a wider variety of projects at our startup: search- ing, an ad targeting engine, a Twitter analytics engine, and many pieces to connect the different parts of our infrastructure. Each project forced me to learn more about Redis. And as I saw others on the mailing list using Redis, asking questions, I couldn’t help but offer more and more advice (my all-time favorite was actually a job-search problem, which became section 7.4), becoming one of the most prolific posters on the Redis mailing list. In late September 2011, while on my honeymoon in Paris, I received a call from a Manning Publications acquisitions editor named Michael Stephens. I didn’t receive the call immediately, because my phone doesn’t work outside the United States. And due to bugs in my phone’s firmware, I didn’t even receive the message until the sec- ond week of October. When I finally got the message and spoke to Michael, I learned that someone at Manning had decided that it was about time to publish Redis in Action. After reading the relevant mailing lists and asking around for suggestions as to who should write thexvii Download from Wow! eBook <www.wowebook.com>
Page
19
PREFACExviiibook, my name came up. Luckily, Manning was still taking book proposals when I called. After a few weeks of discussions and a few book proposal revisions (primarily resulting from farming several dozen of my past Redis mailing list advice posts), Man- ning accepted my proposal, and I started writing. It’s now around 17 months since I first spoke with Michael, and Redis in Action is essentially complete, missing only a few details that I’m finishing up now. I’ve spent a full year of evenings and weekends pro- ducing a book to help others understand and utilize one of the most interesting tech- nologies I’ve come across—more interesting than almost anything I’ve run into since the day I sat down at my family’s first computer, 20 years ago this past Christmas. My only regret in all of this is not having had the foresight to invent Redis in the first place. But at least I had the opportunity to write the book on it!Download from Wow! eBook <www.wowebook.com>
Page
20
acknowledgments I thank my editor at Manning, Beth Lexleigh, for her help throughout the writing process: your patience and gentle pushes have kept me on track. I also thank my development editor Bert Bates: thank you for pointing out that my writing style needed to change for a book audience. Your influence on my writing early on continued during my work on the entire book, which I hope is reasonably accessible to readers with a wide range of expertise. Thank you, Salvatore Sanfilippo: without you, there would be no Redis, and no one would be reading this. And a huge thank you for writing the foreword on such short notice. Thank you, Pieter Noordhuis: in addition to working on Redis, you shared several drinks with me during RedisConf 2012 while listening to my ideas on Redis data struc- ture layouts. Even if none of them happen (which is likely), it was great getting a chance to meet and talk about Redis internals. A huge thank you goes to my technical proofreading team (in alphabetical order by first name): James Phillips, Kevin Chang, and Nicholas Lindgren. You helped when I needed help, and through your collective efforts, Redis in Action has been polished even further. Thank you, my friend and colleague Eric Van Dewoestine: you took the time to write Java versions of examples from the book, which are available at GitHub: https:// github.com/josiahcarlson/redis-in-action. Thank you, all of my reviewers, for the first, second, and third reviews of my manu- script during development, and to my final QA reviewer. I tried to take all of yourxix Download from Wow! eBook <www.wowebook.com>
Comments 0
Loading comments...
Reply to Comment
Edit Comment