本书综合探讨了机器学习的理论基础，为读者提供了使用机器学习技术解决现实问题所需的知识。具体内容包括如何概念化问题、准确表示数据、选择和调整算法、解释和分析结果以及做出合理的决策，采用非严格意义的数学进行阐述，涵盖了一系列广泛的机器学习主题，并特别强调了一些有益的方法，如监督学习、统计学习、使用支持向量机（SVM）学习、使用神经网络（NN）学习、模糊推理系统、数据聚类、数据变换、决策树学习、商业智能、数据挖掘，等等。

PREFACE Over the past two decades, the field of Machine Learning has become one of the mainstays of information technology. Many successful machine learning applications have been developed, such as: machine vision (image processing) in the manufacturing industry for automation in assembly line, biometric recognition, handwriting recognition, medical diagnosis, speech recognition, text retrieval, natural language processing, and so on. Machine learning is so pervasive today that you probably use it several times a day, without knowing it. Examples of such “ubiquitous” or “invisible” usage include search engines, customer-adaptive web services, email managers (spam filters), computer network security, and so on. We are rethinking on everything we have been doing, with the aim of doing it differently using tools of machine learning for better success. Many organizations are routinely capturing huge volumes of historical data describing their operations, products, and customers. At the same time, scientists and engineers are capturing increasingly complex datasets. For example, banks are collecting huge volumes of customer data to analyze how people spend their money; hospitals are recording what treatments patients are on, for which periods (and how they respond to them); engine monitoring systems in cars are recording information about the engine in order to detect when it might fail; world’s observatories are storing incredibly high-resolution images of night sky; medical science is storing the outcomes of medical tests from measurements as diverse as Magnetic Resonance Imaging (MRI) scans and simple blood tests; bioinformatics is storing massive amounts of data with the ability to measure gene expression in DNA microarrays, and so on. The field of machine learning addresses the question of how best to use this historical data to discover general patterns and improve the process of making decisions. Terminology in the field of learning is exceptionally diverse, and very often similar concepts are variously named. In this book, the term machine learning has been mostly used to describe various concepts, though the terms: artificial intelligence, machine intelligence, pattern recognition, statistical learning, data mining, soft computing, data analytics (when applied in business contexts), also appear at various places. There have been important advances in the theory and algorithms that form the foundations of machine learning field. The goal of this text book is to present basic concepts of the theory, and a wide range of techniques (algorithms) that can be applied to a variety of problems. There are many machine learning algorithms not included in this book, that can be quite effective in specific situations. However, almost all of them are some adaptation of the algorithms included in this book. Self-learning will easily help to acquire the required knowledge. Basically, there are two approaches for understanding machine learning field. In one approach, we treat machine learning techniques as a ‘black box’, and focus on understanding the problems (tasks) of interest: matching these tasks to machine learning tools and assessing the quality of the output. This gives us hands-on experience with machine learning from practical case studies. Subsequently, we delve into the components of this black box by examining machine learning algorithms (a theoretical principle-driven exposition is necessary to be effective in machine learning). The second approach starts with the theory; this is then followed by hands-on experience. The approach into the field of machine learning taken in this book has been the second one. We have focussed on machine learning theory. For hands-on experience, we propose to provide a platform through self-study machine learning projects. In this book on “Applied Machine Learning”, the reader will get not only the theoretical underpinnings of learning, but also gain the practical know-how needed to quickly and powerfully apply these techniques to challenging problems: learning how to conceptualize a problem, knowing how to represent the data, selecting and tuning algorithms, being able to interpret results properly, doing an effective analysis of results to make strategic decisions. Recognizing that most ideas behind machine learning are wonderfully simple and straightforward, the book presents machine learning concepts and techniques in a non-rigorous mathematical setting, with emphasis on effective methodology for using machine learning to solve practical problems. It is a comprehensive textbook on the subject, covering broad array of topics with more emphasis on the techniques (algorithms) that have been profitably employed, thus exploiting the available knowledge base. Machine learning draws on concepts and techniques from many fields, including computational statistics (a discipline that aims at the design of algorithms for implementing statistical methods on computers), artificial intelligence, information theory, mathematical optimization, biology, cognitive science, and control theory. The primary goal of this book is to provide a broad-based single source introduction to the field. It introduces basic concepts from various fields as the need arises, focussing on just those concepts most relevant to machine learning. Though the required material has been given in the book, some experience with probability, statistics, and linear algebra will be useful. The first-generation machine learning algorithms covered in this book, have been demonstrated to be of significant value in a variety of real-world applications with numeric features. But these algorithms also have significant limitations, for example, although some learning algorithms are available to classify images, text or speech, we still lack effective algorithms for learning from data that is represented by a combination of these various media. Also most learning algorithms perform acceptably well on datasets with tens of thousands of training examples, but many important datasets are significantly larger. The volume and diversity (structured/unstructured) of data available on the Internet and corporate Intranets is extremely large and growing rapidly. Scaling to complex, extremely large datasets—the big data analytics—is probably the most debated current issue. Given these and other limitations, and the strong commercial interest despite them, we might well expect the next decade to produce an order of magnitude advancement in the state of the art. Deep learning algorithms are emerging as very powerful next-generation tools. Like most other areas of technology, data mining exists on a shifting landscape; not only is the old part of the landscape being redefined, but new areas of interest always loom ahead. All learning algorithms are explained so that the student can easily move from the equations in the book to computer programs. Proliferation of free software that makes machine learning easier to implement, will also be helpful in the project work. The diversity of machine learning libraries means that there is likely to be an option available of what language or environment a student uses. There are many machine learning websites that give information on available machine learning software. Some of the popular software sources are R, SAS, Python, Weka, MATLAB, Excel, and Tableau. This book does not promote any specific software. We have included a large number of examples, but we use illustrative datasets that are small enough to allow the reader to follow what is going on without the help of software. Real datasets are far too large to show this. Datasets in the book are chosen not to illustrate actual large-scale practical problems, but to help the reader understand what the different techniques do, how they work, and what their range of application is. This explains why a heavy focus on project work is a necessity. Each project must handle a large-scale practical problem. Use of domain knowledge to formulate the problem in machine learning setting, and interpretation of the results given by machine learning algorithms are important ingredients of training the students, in addition to the training on machine learning software. This book on ‘Applied Machine Learning’ provides necessary ingredients for practice—the concepts and the techniques —but the actual practice will follow through project work on real-life problems. In a university setting, this book provides an introductory course for undergraduate students in computer science and all engineering degree programs. Such an introductory course will require a properly selected subset of techniques covered in the book. The course design must have a heavy focus on project work, so that when a student has completed the course, he/she should be fully prepared to attack new problems using machine learning. Postgraduate students and Ph.D. research scholars will find in this book a useful initial exposure to the subject, before they go for highly theoretical depth in the specific areas of their research. The book is aimed at a large professional audience as well: engineers, scientists, and business managers; with machine learning and deep learning predicted to be the next ‘grand slam’ in technology, professionals in almost all fields will need to know at least the basics of machine learning. I hope that the reader will share my excitement on the subject of machine learning, and will find the book useful. M. Gopal mgopal.iitd@gmail.com

目录 1. Introduction 引言… ………………………………………………………………………………1 1.1 Towards Intelligent Machines 走向智能机器 ………………………………………………1 1.2 Well-Posed Machine Learning Problems 适定的机器学习问题 ……………………………5 1.3 Examples of Applications in Diverse Fields 不同领域的应用实例 …………………………7 1.4 Data Representation 数据表示 ………………………………………………………………12 1.4.1 Time Series Forecasting 时间序列预测 ………………………………………………15 1.4.2 Datasets for Toy (Unreastically Simple) and Realistic Problems 初级问题和现实问题数据集 …………………………………………………………17 1.5 Domain Knowledge for Productive use of Machine Learning 使机器学习有效应用的领域知识 ……………………………………………………………18 1.6 Diversity of Data: Structured/Unstructured 数据多样性：结构化/非结构化 ……………20 1.7 Forms of Learning 学习形式 …………………………………………………………………21 1.7.1 Supervised/Directed Learning 监督/指导学习 ……………………………………21 1.7.2 Unsupervised/Undirected Learning 非监督/无指导学习 …………………………22 1.7.3 Reinforcement Learning 强化学习 …………………………………………………22 1.7.4 Learning Based on Natural Processes: Evolution, Swarming, and Immune Systems 基于自然处理的学习：进化、集群和免疫系统………………………………………23 1.8 Machine Learning and Data Mining 机器学习和数据挖掘 …………………………………25 1.9 Basic Linear Algebra in Machine Learning Techniques 机器学习技术中的基础线性代数 ……………………………………………………………26 1.10 Relevant Resources for Machine Learning 机器学习的相关资源 …………………………34 2. Supervised Learning: Rationale and Basics 监督学习：基本原理和基础… ………………36 2.1 Learning from Observations 从观测中学习 …………………………………………………36 2.2 Bias and Variance 偏差和方差 ………………………………………………………………42 2.3 Why Learning Works: Computational Learning Theory 学习为什么有效：计算学习理论 ……………………………………………………………46 2.4 Occam’s Razor Principle and Overfitting Avoidance 奥卡姆剃刀原理和防止过拟合 ………………………………………………………………49 2.5 Heuristic Search in Inductive Learning 归纳学习中的启发式搜索 …………………………51 2.5.1 Search through Hypothesis Space 假设空间搜索 ……………………………………52 2.5.2 Ensemble Learning 集成学习 ………………………………………………………53 2.5.3 Evaluation of a Learning System 学习系统的评价 …………………………………55 2.6 Estimating Generalization Errors 估计泛化误差 ……………………………………………56 2.6.1 Holdout Method and Random Subsampling 留出法和随机下采样 …………………56 2.6.2 Cross-validation 交叉验证 ……………………………………………………………57 2.6.3 Bootstrapping 自助法 …………………………………………………………………58 2.7 Metrics for Assessing Regression (Numeric Prediction) Accuracy 评价回归（数值预测）精度的指标 …………………………………………………………59 2.7.1 Mean Square Error 均方误差 …………………………………………………………60 2.7.2 Mean Absolute Error 平均绝对误差 …………………………………………………60 2.8 Metrics for Assessing Classification (Pattern Recognition) Accuracy 评价分类（模式识别）精度的指标 …………………………………………………………61 2.8.1 Misclassification Error 误分类误差 …………………………………………………61 2.8.2 Confusion Matrix 混淆矩阵 …………………………………………………………62 2.8.3 Comparing Classifiers Based on ROC Curves 基于ROC曲线的分类器比较 ……66 2.9 An Overview of the Design Cycle and Issues in Machine Learning 机器学习中的设计周期和问题概述 …………………………………………………………68 3. Statistical Learning 统计学习… …………………………………………………………………73 3.1 Machine Learning and Inferential Statistical Analysis 机器学习与推断统计分析 …………73 3.2 Descriptive Statistics in Learning Techniques 学习技术中的描述统计学 …………………74 3.2.1 Representing Uncertainties in Data: Probability Distributions 表示数据中的不确定性：概率分布 …………………………………………………75 3.2.2 Descriptive Measures of Probability Distributions 概率分布的描述方法 …………80 3.2.3 Descriptive Measures from Data Sample 数据样本的描述方法 ……………………83 3.2.4 Normal Distributions 正态分布 ………………………………………………………84 3.2.5 Data Similarity 数据相似性 …………………………………………………………85 3.3 Bayesian Reasoning: A Probabilistic Approach to Inference 贝叶斯推理：推断的概率方法 ………………………………………………………………87 3.3.1 Bayes Theorem 贝叶斯定理 …………………………………………………………88 3.3.2 Naive Bayes Classifier 朴素贝叶斯分类器 …………………………………………93 3.3.3 Bayesian Belief Networks 贝叶斯信念网络 …………………………………………98 3.4 k-Nearest Neighbor (k-NN) Classifier k近邻（k-NN）分类器 …………………………102 3.5 Discriminant Functions and Regression Functions 判别函数和回归函数 ………………106 3.5.1 Classification and Discriminant Functions 分类和判别函数 ……………………107 3.5.2 Numeric Prediction and Regression Functions 数值预测和回归函数 ……………108 3.5.3 Practical Hypothesis Functions 实践应用中的假设函数 …………………………109 3.6 Linear Regression with Least Square Error Criterion 基于最小二乘误差准则的线性回归法 ……………………………………………………112 3.6.1 Minimal Sum-of-Error-Squares and the Pseudoinverse 最小误差平方和与伪逆 113 3.6.2 Gradient Descent Optimization Schemes 梯度下降法优化方案 …………………115 3.6.3 Least Mean Square (LMS) Algorithm 最小均方（LMS）算法 …………………115 3.7 Logistic Regression for Classification Tasks 分类任务的逻辑回归法 ……………………116 3.8 Fisher’s Linear Discriminant and Thresholding for Classification Fisher线性判别和分类阈值…………………………………………………………………120 3.8.1 Fisher’s Linear Discriminant Fisher线性判别式 …………………………………120 3.8.2 Thresholding 阈值 …………………………………………………………………125 3.9 Minimum Description Length Principle 最小描述长度原理 ……………………………126 3.9.1 Bayesian Perspective 贝叶斯角度 …………………………………………………127 3.9.2 Entropy and Information 熵和信息 ………………………………………………128 4. Learning With Support Vector Machines (SVM) 利用支持向量机（SVM）学习…………130 4.1 Introduction 简介 …………………………………………………………………………130 4.2 Linear Discriminant Functions for Binary Classification 二元分类的线性判别函数 …………………………………………………………………132 4.3 Perceptron Algorithm 感知机算法 …………………………………………………………136 4.4 Linear Maximal Margin Classifier for Linearly Separable Data 线性可分数据的最大边缘线性分类器 ……………………………………………………141 4.5 Linear Soft Margin Classifier for Overlapping Classes 重叠类的软边缘线性分类器 ………………………………………………………………152 4.6 Kernel-Induced Feature Spaces 核函数引导的特征空间 …………………………………158 4.7 Nonlinear Classifier 非线性分类器 ………………………………………………………162 4.8 Regression by Support Vector Machines 支持向量机回归 ………………………………167 4.8.1 Linear Regression 线性回归 ………………………………………………………169 4.8.2 Nonlinear Regression 非线性回归 …………………………………………………172 4.9 Decomposing Multiclass Classification Problem Into Binary Classification Tasks 将多类分类问题分解为二元分类任务 ……………………………………………………174 4.9.1 One-Against-All (OAA) 一对所有（OAA） ……………………………………175 4.9.2 One-Against-One (OAO) 一对一（OAO） ………………………………………176 4.10 Variants of Basic SVM Techniques 基础支持向量机技术的变体 ………………………177 5. Learning With Neural Networks (NN) 利用神经网络（NN）学习…………………………181 5.1 Towards Cognitive Machine 走向认知机器 ………………………………………………181 5.1.1 From Perceptrons to Deep Networks 从感知机到深度网络 ………………………182 5.2 Neuron Models 神经元模型 ………………………………………………………………184 5.2.1 Biological Neuron 生物神经元 ……………………………………………………184 5.2.2 Artificial Neuron 人工神经元 ……………………………………………………186 5.2.3 Mathmatical Model 数学模型 ……………………………………………………190 5.3 Network Architectures 网络架构 …………………………………………………………193 5.3.1 Feedforward Networks 前馈网络 …………………………………………………194 5.3.2 Recurrent Networks 循环网络 ……………………………………………………199 5.4 Perceptrons 感知机 …………………………………………………………………………200 5.4.1 Limitations of Perceptron Algorithm for Linear Classification Tasks 线性分类任务采用感知机算法的局限性 …………………………………………201 5.4.2 Linear Classification using Regression Techniques 使用回归技术的线性分类 …………………………………………………………201 5.4.3 Standard Gradient Descent Optimization Scheme: Steepest Descent 标准梯度下降法优化方案：最速下降法 …………………………………………203 5.5 Linear Neuron and the Widrow-Hoff Learning Rule 线性神经元与Widrow-Hoff学习规则 ……………………………………………………206 5.5.1 Stochastic Gradient Descent 随机梯度下降法 ……………………………………207 5.6 The Error-Correction Delta Rule 纠错Delta法则 …………………………………………208 5.6.1 Sigmoid Unit: Soft-Limiting Perceptron Sigmoid单元：软限制感知机 …………211 5.7 Multi-Layer Perceptron (MLP) Networks and the Error-Backpropagation Algorithm 多层感知机网络及误差反向传播算法 ……………………………………………………213 5.7.1 The Generalized Delta Rule 广义Delta法则 ……………………………………216 5.7.2 Convergence and Local Minima 收敛性和局部极小值 …………………………226 5.7.3 Adding Momentum to Gradient Descent 为梯度下降法加入动量 ………………227 5.7.4 Heuristic Aspects of the Error-backpropagation Algorithm 误差反向传播算法的启发性 ………………………………………………………228 5.8 Multi-Class Discrimination with MLP Networks 使用MLP网络的多类判别 …………232 5.9 Radial Basis Functions (RBF) Networks 径向基函数（RBF）网络 ……………………235 5.9.1 Training the RBF Network 训练RBF网络 ………………………………………239 5.10 Genetic-Neural Systems 遗传-神经系统 …………………………………………………241 6. Fuzzy Inference Systems 模糊推理系统… ……………………………………………………245 6.1 Introduction 简介 …………………………………………………………………………245 6.2 Cognitive Uncertainty and Fuzzy Rule-Base 认知不确定性和模糊规则库 ………………248 6.3 Fuzzy Quantification of Knowledge 知识的模糊量化 ……………………………………253 6.3.1 Fuzzy Logic 模糊逻辑 ……………………………………………………………253 6.3.2 Fuzzy Sets 模糊集 …………………………………………………………………257 6.3.3 Fuzzy Set Operations 模糊集运算 …………………………………………………267 6.3.4 Fuzzy Relations 模糊关系 …………………………………………………………268 6.4 Fuzzy Rule-Base and Approximate Reasoning 模糊规则库与近似推理 …………………277 6.4.1 Quantification of Rules via Fuzzy Relations 通过模糊关系量化规则 ……………281 6.4.2 Fuzzification of Input 输入的模糊化 ………………………………………………283 6.4.3 Inference Mechanism 推理机制 ……………………………………………………284 6.4.4 Defuzzification of Inferred Fuzzy Set 推理模糊集的解模糊 ……………………298 6.5 Mamdani Model for Fuzzy Inference Systems 模糊推理系统的Mamdani模型 …………301 6.5.1 Mobile Robot Navigation Among Moving Obstacles 移动障碍物环境下的移动机器人导航 ……………………………………………303 6.5.2 Mortgage Loan Assessment 抵押贷款评估 ………………………………………308 6.6 Takagi-Sugeno Fuzzy Model Takagi-Sugeno模糊模型 …………………………………311 6.7 Neuro-Fuzzy Inference Systems 神经模糊推理系统 ……………………………………317 6.7.1 ANFIS Architecture ANFIS结构 …………………………………………………318 6.7.2 How Does an ANFIS Learn? ANFIS如何学习 ……………………………………320 6.8 Gentic-Fuzzy Systems 遗传模糊系统 ……………………………………………………324 7. Data Clustering and Data Transformations 数据聚类和数据变换… ………………………328 7.1 Unsupervised Learning 非监督学习 ………………………………………………………328 7.1.1 Clustering 聚类 ……………………………………………………………………329 7.2 Engineering the Data 数据工程 ……………………………………………………………331 7.2.1 Exploratory Data Analysis: Learning about What is in the Data 探索数据分析：了解数据中的内容 ………………………………………………333 7.2.2 Cluster Analysis: Finding Similarities in the Data 聚类分析：发现数据间的相似性 …………………………………………………334 7.2.3 Data Transformations: Enhancing the Information Content of the Data 数据变换：提高数据的信息量 ……………………………………………………339 7.3 Overview of Basic Clustering Methods 基本聚类方法概述 ………………………………341 7.3.1 Partitional Clustering 划分聚类 ……………………………………………………341 7.3.2 Hierarchical Clustering 层次聚类 …………………………………………………344 7.3.3 Spectral Clustering 谱聚类 …………………………………………………………345 7.3.4 Clustering using Self-Organizing Maps 自组织映射聚类 ………………………349 7.4 K-Means Clustering K均值聚类 …………………………………………………………352 7.5 Fuzzy K-Means Clustering 模糊K均值聚类 ……………………………………………356 7.6 Expectation-Maximization (EM) Algorithm and Gaussian Mixtures Clustering 期望最大化（EM）算法和高斯混合聚类 …………………………………………………362 7.6.1 EM Algorithm EM算法 ……………………………………………………………362 7.6.2 Gaussian Mixture Models 高斯混合模型 …………………………………………365 7.7 Some Useful Data Transformations 一些有用的数据变换 ………………………………372 7.7.1 Data Cleansing 数据清洗 …………………………………………………………372 7.7.2 Derived Attributes 衍生属性 ………………………………………………………374 7.7.3 Discretizing Numeric Attributes 离散化数值属性 ………………………………375 7.7.4 Attribute Reduction Techniques 属性约简技术 ……………………………………377 7.8 Entropy-Based Method for Attribute Discretization 基于熵的属性离散化方法 …………377 7.9 Principal Components Analysis (PCA) for Attribute Reduction 用于属性约简的主成分分析（PCA） ……………………………………………………382 7.10 Rough Sets-Based Methods for Attribute Reduction 基于粗糙集的属性约简方法 ……390 7.10.1 Rough Set Preliminaries 粗糙集准备 ……………………………………………392 7.10.2 Analysis of Relevance of Attributes 属性相关性分析 ……………………………397 7.10.3 Reduction of Attributes 约简属性 …………………………………………………399 8. Decision Tree Learning 决策树学习……………………………………………………………404 8.1 Introduction 简介 …………………………………………………………………………404 8.2 Example of a Classification Decision Tree 分类决策树示例 ……………………………406 8.3 Measures of Impurity for Evaluating Splits in Decision Trees 评价决策树分裂的不纯度度量 ……………………………………………………………411 8.3.1 Information Gain/Entropy reduction 信息增益/熵减少 …………………………411 8.3.2 Gain Ratio 增益率 …………………………………………………………………416 8.3.3 Gini Index 基尼指数 ………………………………………………………………417 8.4 ID3, C4.5, and CART Decision Trees ID3, C4.5与CART决策树 ………………………418 8.5 Pruning the Tree 剪枝 ………………………………………………………………………427 8.6 Strengths and Weaknesses of Decision-Tree Approach 决策树方法的优缺点 …………429 8.7 Fuzzy Decision Trees 模糊决策树 …………………………………………………………433 9. Business Intelligence and Data Mining: Techniques and Applications 商业智能和数据挖掘：技术与应用… …………………………………………………………445 9.1 An Introduction to Analytics 分析简介 ……………………………………………………445 9.1.1 Machine Learning, Data Mining, and Predictive Analytics 机器学习、数据挖掘和预测分析 …………………………………………………448 9.1.2 Basic Analytics Techniques 基本分析技术 ………………………………………449 9.2 The CRISP-DM (Cross Industry Standard Process for Data Mining) Model CRISP-DM（数据挖掘跨行业标准流程）模型 ……………………………………………451 9.3 Data Warehousing and Online Analytical Processing 数据仓库和联机分析处理 ………456 9.3.1 Basic Concepts 基本概念 …………………………………………………………456 9.3.2 Databases 数据库 …………………………………………………………………458 9.3.3 Data Warehousing: A General Architecture, and OLAP Operations 数据仓库：通用架构和OLAP操作 ………………………………………………461 9.3.4 Data Mining in the Data Warehouse Environment 数据仓库环境中的数据挖掘 ………………………………………………………466 9.4 Mining Frequent Patterns and Association Rules 挖掘频繁模式和关联规则 ……………467 9.4.1 Basic Concepts 基本概念 …………………………………………………………469 9.4.2 Measures of Strength of Frequent Patterns and Association Rules 频繁模式和关联规则的强度的度量 ………………………………………………471 9.4.3 Frequent Item Set Mining Methods 频繁项集挖掘方法 …………………………473 9.4.4 Generating Association Rules from Frequent Itemsets 从频繁项集生成关联规则 …………………………………………………………477 9.5 Intelligent Information Retrieval Systems 智能信息检索系统 ……………………………479 9.5.1 Text Retrieval 文本检索 ……………………………………………………………483 9.5.2 Image Retrieval 图像检索 …………………………………………………………486 9.5.3 Audio Retrieval 音频检索 …………………………………………………………488 9.6 Applications and Trends 应用和趋势 ………………………………………………………490 9.6.1 Data Mining Applications 数据挖掘应用 …………………………………………490 9.6.2 Data Mining Trends 数据挖掘趋势 ………………………………………………495 9.7 Technologies for Big Data 大数据技术 ……………………………………………………498 9.7.1 Emerging Analytic Methods 新兴分析方法 ………………………………………500 9.7.2 Emerging Technologies for Higher Levels of Scalability 用于更高层次可扩展性的新兴技术 ………………………………………………503 Appendix A Genetic Algorithm (GA) For Search Optimization 搜索优化的遗传算法（GA）… …………………………………………………… 508 A.1 A Simple Overview of Genetics 遗传学的简单概述 ……………………………………510 A.2 Genetics on Computers 计算机遗传学 ……………………………………………………511 A.3 The Basic Genetic Algorithm 基本遗传算法 ……………………………………………515 A.4 Beyond the Basic Genetic Algorithm 超越基本遗传算法 ………………………………524 Appendix B Reinforcement Learning (RL) 强化学习（RL）… ……………………………527 B.1 Introduction 简介 …………………………………………………………………………527 B.2 Elements of Reinforcement Learning 强化学习的要素 …………………………………530 B.3 Basics of Dynamic Programming 动态编程基础 …………………………………………535 B.3.1 Finding Optimal Policies 寻找最优策略 …………………………………………538 B.3.2 Value Iteration 值迭代 ……………………………………………………………539 B.3.3 Policy Iteration 策略迭代 …………………………………………………………540 B.4 Temporal Difference Learning 时间差分学习 ……………………………………………542 B.4.1 Q-learning Q-学习 …………………………………………………………………544 B.4.2 Generalization 归纳 ………………………………………………………………546 B.4.3 Sarsa-learning SARSA学习 ………………………………………………………548 Datasets from Real-Life Applications for Machine Learning Experiments 机器学习实验的实际应用数据集………………………………………………………………… 549 Problems 习题… …………………………………………………………………………………567 References 参考文献………………………………………………………………………………613 Index 索引… ………………………………………………………………………………………623