本书作为数据挖掘入门读物,介绍了数据挖掘的基础知识,基本工具和实践方法,通过循序渐进地讲解算法,带读者轻松踏上数据挖掘之旅.
AI Reading Assistant
Whole-book reading guide from stratified index samples; jump to passages in the text
AI guide
【One-Line Pitch】
A hands-on tour of practical data mining with Python, where each chapter pairs a real dataset with a working algorithm so beginners can build intuition by doing. Best for readers who can already write basic Python and want to see how classification, recommendation, text mining, and neural networks actually get applied.
【Book Arc】
- **Opening (~0%–10%)**: Sets up the toolkit and mindset — installing Python 3.4, IPython Notebook, and scikit-learn, then framing data mining as "letting computers make decisions from data" through tiny affinity-analysis and OneR examples.
- **Early (~10%–30%)**: Builds the standard scikit-learn workflow — estimators, `fit`/`predict`, cross-validation, preprocessing and pipelines — then applies it to real problems like predicting NBA winners with decision trees and random forests.
- **Middle (~30%–55%)**: Moves into recommendation and feature work — Apriori-based movie recommendation on MovieLens, sparse matrices, and a deep chapter on feature extraction, selection, and building your own transformers.
- **Late (~55%–85%)**: Applies the toolkit to messier domains — Naive Bayes for social media text, graph mining for finding interesting people, neural networks for CAPTCHA cracking, and authorship attribution with SVMs.
- **Ending (~85%–100%)**: Closes with clustering news articles, deep learning for image classification (Theano/Lasagne/nolearn, GPU notes), and a MapReduce/Hadoop chapter for big-data processing, plus an appendix pointing to next steps.
【Key Takeaways】
- **Feature engineering usually matters more than algorithm choice** (Early): the NBA chapter shows accuracy climbing mainly through new features and grid search, not just swapping classifiers.
- **A standard scikit-learn workflow is the book's backbone** (Early): estimators, cross-validation, and pipelines recur in nearly every chapter, so mastering them early pays off.
- **Preprocessing fixes real failures** (Early): scaling broken features restores accuracy from 71.5% back to 82.3%, a concrete lesson in why normalization matters.
- **Ensembles reduce variance** (Early): random forests average many decorrelated trees, trading a bit of interpretability for more stable predictions.
- **Sparse data needs special handling** (Middle): MovieLens is mostly empty, so sparse matrices and Apriori's frequent-itemset pruning replace brute-force search.
- **Text becomes numbers through feature extraction** (Late): bag-of-words, n-grams, and function words turn tweets and books into classifier-ready inputs.
- **Different domains demand different algorithms** (Late): Naive Bayes, SVMs, neural networks, and clustering each appear where they fit, not as one universal tool.
- **Scale changes the method** (Ending): when data grows, MapReduce and Hadoop-style processing become necessary rather than optional.
【Reading Tips】
- Deep-read Chapters 1–3 for the core workflow; skim installation details if your environment is already set up.
- Treat each chapter as a standalone project — run the code, then try the author's suggested "improve this" exercise before moving on.
- Don't skip Chapter 5 on feature extraction; it underpins everything later and is easy to underestimate.
- Expect jumps between topics; keep notes on which algorithm maps to which problem type.
- For the deep learning and big-data chapters, focus on concepts and setup rather than memorizing framework-specific code.
【Coverage Limits】
This guide is based on stratified excerpts covering the table of contents, preface, and selected chapters; detailed content of later chapters (especially deep learning and MapReduce) is only partially represented.
Excerpt 1
9.5 使用安然公司数据集 9.5.1 获取安然数据集 9.5.2 创建数据集加载工具 9.5.3 组装起来 9.5.4 评估 9.6 小结 第 10 章 新闻语料分类 10.1 获取新闻文章 10.1.1 使用Web API获取数据 10.1.2 数据资源宝库reddit 10.1.3 获取数据 10.2 从任...
View in text
Excerpt 2
可能无法运行。 如何通过编译源文件进行安装,以及更多的安装指南,请见官方文档: http://scikit-learn.org/stable/install.html 。 未知 1.4 分类问题的简单示例 在上述亲和性分析例子中,我们寻找的是数据集中不同变量之间的相关性。而分类问题,只关注 类别 (也叫作目标)这...
View in text
Excerpt 3
差来表示这种不一致。 方差 是由训练集的变化引起的。决策树这类方差大的算法极易受到训练集变化的影响,从而产生过拟合问题。 对比来说, 偏误 (bias)是由算法中的假设引起的,而与数据集没有关系。比如,算法错误地假定所有特征呈正态分布,就会导致较高的误差。通过分析分类器的数据模型和实际数据集的匹配情况,就能降低...
View in text
Excerpt 4
着,用一个函数来实现步骤(2)和(3),它接收新发现的频繁项集,创建超集,检测频繁程度。下面为函数声明及字典初始化代码。 from collections import defaultdict def find_frequent_itemsets(favorable_reviews_by_users, k_1_i...
View in text
Excerpt 5
可以知道都使用了哪些特征。还是看下代码: print(transformer.scores_) 输出结果如下: [ 8.60061182e+03 2.40142178e+03 8.21924671e+07 1.37214589e+06 6.47640900e+03] 相关性最好的分别是第一、三、四列,分别对应着...
View in text
Excerpt 6
元语法 比起用单个词作特征,使用 N元语法 能更好地描述文档,具体优势稍后会讲。N元语法是指由几个连续的词组成的子序列。拿我们的数据集来讲,N元语法指的是每条消息里一组连续的词。 N元语法的计算方法跟计算单个词语方法相同,我们把构成N元语法的几个词看成是 词袋 中的一个 词 。数据集 5 中每一项就变成了N元语法...
View in text
Excerpt 7
能还有其他原因引发的 TypeError 异常。对于这种异常,需要单独处理,因此需要返回具体错误信息,方便查找错误起因。代码如下: except TypeError as e: if results is None: print("You probably reached your API limit, wait...
View in text
Excerpt 8
中用户相同的情况: for user1 in friends.keys(): for user2 in friends.keys(): if user1 == user2: continue 计算两个用户之间边的权重: weight = compute_similarity(friends[user1], fri...
View in text
Tags
AI categories
DataPythonArtificial Intelligence
Loading comments...
Reply to Comment
Edit Comment