本书详细地介绍了大数据、人工智能等项目中不可或缺的环节和内容：数据准备和特征工程。书中的每节首先以简明方式介绍了基本知识；然后通过实际案例演示了基本知识的实际应用，并提供了针对性练习项目，将“知识、案例、练习”融为一体；最后以“扩展探究”方式引导读者进入更深广的领域。本书既适合作为大学相关专业的教材，也适合作为大数据、人工智能等领域的开发人员的参考读物。

前言 在计算机科学中，有一句名言：“Garbage in, garbage out”(GIGO)。这句话用到数据科学上也同样成立。另外，数据科学业界中还流传着另一句话：“数据和特征决定了机器学习的上限，而模型和算法只是逼近这个上限而已。” 除了“名言”，很多数据科学实践者的项目经验也一再证明高质量的数据永远是排在第一位的。 然而，现实世界的数据存在不完整、噪声、不一致、错误值、离群值、重复等问题。不仅如此，数据集的特征也是形形色色的，有的特征与项目无关，有的特征彼此强相关，还有的数据集因为特征太多而导致耗费极大的计算资源。诸如此类现象，可以概括为一句话：“理想很丰满，现实很骨感。” 因此，数据准备和特征工程的工作就成为数据科学项目中不可或缺的环节，每个从业者必须熟练掌握相关操作技能，并能耐心地从事这项工作。实践经验表明，数据准备和特征工程会占用项目开发的绝大部分时间。 本书相对于已有的类似书籍而言，在以下方面更具有特色。 ? 强调工程实践，这也是本书作者所有书籍的共同特点。书中通过大量案例，向读者演示了各种方法的具体实现方式。 ? 基础与前沿结合。虽然本书在“基础知识”中介绍了相关的基本实现方法，但因为现实项目的复杂性，在具体项目中还会用到各种工具及最新的研发成果，为此专设了“扩展探究”供读者了解更精彩的内容。 ? 以案例为载体，传授思想方法。数据科学项目需要严谨、科学的思想方法，这些方法并非通过简单说教就能让读者掌握，本书以“项目案例”为载体，不仅讲述操作技法，而且还让读者体验其中的思想方法，并且在“动手练习”中提供了练习项目，供读者检验和巩固所学内容。 为了给读者使用本书提供更多的资源支持，在此推荐本书作者的微信公众号：老齐教室。通过此微信公众号，可以得到如下资源： ? 使用本书配套的在线实验平台。在实验平台中，读者可以运行本书的所有源码，应用书中所要求的数据集。 ? 观看本书配套的视频课程。 ? 及时获得本书的勘误内容。 ? 阅读与本书相关的其他技术资料。 ? 与本书的作者及其他读者进行专业交流。 非常感谢为本书的出版而辛苦工作的各位编辑。 书中内容难免错误，恳请读者不吝赐教。 齐 伟

目录 第1 章 感知数据 ·································.001 1.0 了解数据科学项目 ································001 1.1 文件中的数据 ··································003 1.1.1 CSV文件 ····································003 1.1.2 Excel文件 ···································009 1.1.3 图像文件 ···································015 1.2 数据库中的数据 ·································019 1.3 网页上的数据 ··································029 1.4 来自API 的数据 ·································039 第2 章 数据清理 ··································044 2.0 基本概念 ····································045 2.1 转化数据类型 ··································046 2.2 处理重复数据 ··································054 2.3 处理缺失数据 ··································057 2.3.1 检查缺失数据 ·································058 2.3.2 用指定值填补 ·································063 2.3.3 根据规律填补 ·································069 2.4 处理离群数据 ··································076 第3 章 特征变换 ···································083 3.0 特征的类型 ···································084 3.1 特征数值化 ···································085 3.2 特征二值化 ···································088 3.3 OneHot编码 ···································093 3.4 数据变换 ····································098 3.5 特征离散化 ···································104 3.5.1 无监督离散化 ·································104 3.5.2 有监督离散化 ·································110 3.6 数据规范化 ···································113 第4 章 特征选择 ···································124 4.0 特征选择简述 ··································124 4.1 封装器法 ····································127 4.1.1 循序特征选择 ·································127 4.1.2 穷举特征选择 ·································135 4.1.3 递归特征消除 ·································140 4.2 过滤器法 ····································144 4.3 嵌入法 ·····································149 第5 章 特征抽取 ···································154 5.1 无监督特征抽取··································154 5.1.1 主成分分析 ··································154 5.1.2 因子分析 ···································161 5.2 有监督特征抽取 ·································167 附录A Jupyter简介 ·································173 附录B NumPy简介 ··································176 附录C Pandas简介 ··································185 附录D Matplotlib简介 ································194 后记 ········································199