Learn how to use, deploy, and maintain Apache Spark with this comprehensive guide, written by the creators of the open-source cluster-computing framework. With an emphasis on improvements and new features in Spark 2.0, authors Bill Chambers and Matei Zaharia break down Spark topics into distinct sections, each with unique goals.
You’ll explore the basic operations and common functions of Spark’s structured APIs, as well as Structured Streaming, a new high-level API for building end-to-end streaming applications. Developers and system administrators will learn the fundamentals of monitoring, tuning, and debugging Spark, and explore machine learning techniques and scenarios for employing MLlib, Spark’s scalable machine-learning library.
Get a gentle overview of big data and Spark
Learn about DataFrames, SQL, and Datasets—Spark’s core APIs—through worked examples
Dive into Spark’s low-level APIs, RDDs, and execution of SQL and DataFrames
Understand how Spark runs on a cluster
Debug, monitor, and tune Spark clusters and applications
Learn the power of Structured Streaming, Spark’s stream-processing engine
Learn how you can apply MLlib to a variety of problems, including classification or recommendation
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
Tip the Site
Support this siteYour recognition and a small knowledge-service contribution help keep this technical work open source.Scan the WeChat Pay or Alipay code below. Logged-in and guest visitors can both tip.
WeChat Pay
Alipay
Open WeChat or Alipay and scan. No login required.
Brief outline
【One-Line Pitch】
Learn how to use, deploy, and maintain Apache Spark with this comprehensive guide, written by the creators of the ope…
【Book Arc】
- **Opening (~0%–12%)**: This can be one of three core cluster managers: Spark’s standalone cluster manager, YARN, or Mesos.; d see drastically different runtimes.
- **Early (~12%–35%)**: single structured API query from user code to executed code.; "Croatia") .show(2) # in Python df.where(col("count") 2).where(col("ORIGIN_COUNTRY_NAME") !
- **Middle (~35%–65%)**: In a DataFrame, each column has a unique ID within Spark’s SQL Engine, Catalyst.; l work in a similar way using MySQL or PostgreSQL, as well.
- **Late (~65%–88%)**: This method is accessible within serialized functions without having to serialize the data.; is one big partition in our dataset, we will have one task.
- **Ending (~88%–100%)**: You can find up-to-date information about how to use this tool in the Spark documentation.; d into a UDF or function, or if you’re working with strange data types that cannot be serialized.
【Key Takeaways】
- **This can be one of thr…** (Opening): This can be one of three core cluster managers: Spark’s standalone cluster manager, YARN, or Mesos.
- **d see drastically diff…** (Opening): d see drastically different runtimes.
- **Spark’s structured API…** (Opening): Spark’s structured APIs and run them in a streaming fashion.
- **single structured API…** (Early): single structured API query from user code to executed code.
- **"Croatia") .show(2) #…** (Early): "Croatia") .show(2) # in Python df.where(col("count") 2).where(col("ORIGIN_COUNTRY_NAME") !
- **----------------------…** (Early): ---------------------------------- We can also do this quite easily in Python.
【Reading Tips】
- Use Passage locations below to jump into the text and set reading anchors
- If this is a brief outline, click Regenerate (top right) for a synthesized guide
【Coverage Limits】
Compressed outline without the model (~33 index chunks). Full structured guide needs AI available.
Excerpt 1
t. In this diagram, we removed the concept of cluster nodes. The user can specify how many executors should fall on each node through configurations. NOTE Sp...
k.sql.types.{StructField, StructType, StringType, LongType} val myManualSchema = new StructType(Array( new StructField("some", StringType, true), new StructF...
t way to explain this is to begin performing some groupings. The first will be a count, just as we did before. We will group by each unique invoice number an...
maximum number of concurrent numPartitions JDBC connections. If the number of partitions to write exceeds this limit, we decrease it to this limit by calling...
y a fixed-size random sample from your RDD. You can specify whether this should be done by using withReplacement, the number of values, as well as the random...
the intermediate results to memory or disk after each step. The same kind of pipelining happens for a DataFrame or SQL computation that does a select, filter...
This includes monitoring things like CPU, network, and I/O. These are often reported in cluster-level monitoring solutions; however, there are more specific...
such as the Vectorized UDF extension for Python that gives your code multiple records at once using a Pandas data frame. We discussed UDFs and their costs in...
Support this siteYour recognition and a small knowledge-service contribution help keep this technical work open source.
Scan the WeChat Pay or Alipay code below. Logged-in and guest visitors can both tip.
WeChat PayAlipay
Open WeChat or Alipay and scan. No login required.
Add Tag
Enter tag name (max 50 characters)
Share E-Book
Spark The Definitive Guide Big Data Processing Made Simple (Bill Chambers, Matei Zaharia) (Z-Library)
Scan QR code with your phone to access
Copy the link or scan the QR code to access this e-book on your phone
Share E-Book via Email
Please enter email address
Donation Statistics
¥.00
Total Donations
0
Donation Count
Spark The Definitive Guide Big Data Processing Made Simple (Bill Chambers, Matei Zaharia) (Z-Library)
Find Your Favorite Books
Only registered users can comment after logging in. Comments need to be reviewed by administrators before being displayed
Loading comments...
Reply to Comment
Edit Comment