本书是一本介绍深度学习核心算法的书籍。书中以轻松、直白的语言，生动、详细地介绍了与深度学习模型相关的基础知识，深入剖析了深度学习核心算法的原理与本质。同时，书中配有大量案例与源码，帮助读者切实体会深度学习的核心思想和精妙之处。此外，书中还介绍了深度学习在计算机视觉、自然语言处理和推荐系统领域的应用，从原理层面揭示其思想，帮助读者在这些领域中夯实技术基础。

不知不觉中，深度学习已经从一个新颖的概念变成了广为人知的实战利器。近十年来，深度学习的理论和技术都有了一定的发展，也创造了很多里程碑式的事件。无论是为人工智能业内熟知的事件（如 ImageNet 竞赛超越人类识别水平），还是广为人知的事件（如 AlphaGo 击败人类职业围棋选手），都使深度学习相关技术宛如一道绚丽的光芒，划破了人工智能的夜空，让人们看到了智能时代的无限可能。 近年来，很多智能设备走进了每一个人的生活。例如人脸识别技术、语音识别技术，这些技术让使用者感受到便捷，也有了从未有过的体验。此外，深度学习技术还在很多方向迈出了应用的脚步，在带来更好服务的同时，也让我们能够近距离接触这些有一定温度的技术。当然，与人工智能热潮同期到来的是学习人工智能的热潮。越来越多的人投入这个领域中，开始学习数学、计算机等相关知识，希望能够跟上这个时代的步伐，甚至成为这个时代的领跑者，引领人工智能的新技术不断发展前进。写作本书的目的就是帮助更多的人了解深度学习的相关概念和技术，理解深度学习的基本原理，同时对深度学习的一些基本技术进行实践应用。很多对深度学习领域感兴趣的读者都会遇到入门难的问题，即使是一个十分简单的深度学习模型，其中也包含了很多基础概念，这让入门学习变得十分困难。为了解决这些问题，本书前几章会介绍一个简单的深度学习模型的演化方式，当读者有了一定基础后，再为读者介绍更多全新的算法。 本书的主要内容介绍如下。 本书的第 1～9 章主要讲解基于图像分类问题的模型，同时介绍深度学习的基本概念。其中，第 1 章介绍机器学习、深度学习的基本概念；第 2 章介绍如何使用基本的神经网络解决一个小问题，引出神经网络结构、优化等基本概念；第 3 章介绍分类问题及其求解方法，引出分类损失函数等基本概念；第 4 章介绍卷积神经网络，引出卷积等相关操作的基本概念；第 5 章介绍深层神经网络的初始化方法，包括参数、输入初始化的基本思想和方法；第 6章介绍深层神经网络常见的优化方法，对优化算法的作用和特点进行分析；第 7 章介绍神经网络中一些有特点的网络结构，它们在网络训练中起到了很关键的作用；第 8 章介绍高级神经网络结构，同时介绍网络设计中的一些概念和思想；第 9 章介绍网络可视化及网络内部运行机制的知识和概念。至此，读者应该对深层神经网络都有了一定的了解。本书的第 10～14 章主要介绍深度学习在一些经典应用领域的核心算法。其中，第 10 章介绍物体检测问题的核心算法；第 11 章介绍词嵌入问题的核心算法；第 12 章介绍语言模型的核心算法，以及循环神经网络的基本原理；第13 章介绍机器翻译等自然语言处理问题的核心算法，以及 Transformer 网络的基本原理；第 14 章介绍推荐系统、广告点击预测等问题的核心算法，以及分解机模型的基本原理。至此，读者应该对深度学习在各领域的应用有了一定的了解。 以上就是本书的主要内容。希望读者能够通过阅读本书掌握这一系统知识，此后可以依靠更多的外部资源完成更加深入的学习，真正掌握深度学习的相关知识。 在编写本书的过程中，我体会到了求知的艰辛。获取知识的道路总是充满荆棘，除了自身不断地努力，更少不了身边人对我的支持与鼓励。感谢本书的编辑郑柳洁，她从本书立项开始就在出谋划策，对书中的每一个细节、每一句话的措辞都认真审核、校对，为本书付出了巨大的心血；感谢所有关心、支持我完成这项不易的工作的亲人、朋友。由于本人才疏学浅，行文间难免有所纰漏，望各位读者多多包涵，不吝赐教。

第 1 章 从生活走进深度学习 ················································.1 1.1 钞票面值问题·························································.1 1.2 机器学习的特征表示 ·················································.5 1.3 机器学习 ······························································.10 1.4 深度学习的逆袭 ······················································.14 1.5 总结与提问 ···························································.19 第 2 章 构建小型神经网络 ···················································.20 2.1 线性代数基础·························································.21 2.2 全连接层与非线性函数 ··············································.27 2.3 神经网络可视化 ······················································.30 2.4 反向传播法 ···························································.35 2.5 反向传播法的计算方法 ··············································.37 2.6 反向传播法在计算上的抽象 ·········································.40 2.7 反向传播法在批量数据上的推广 ····································.42 2.8 模型训练与结果可视化 ··············································.46 2.9 总结与提问 ···························································.48 第 3 章 多层网络与分类······················································.50 3.1 MNIST 数据集························································.50 3.2 概率论基础 ···························································.51 3.3 Softmax 函数··························································.57 3.4 交叉熵损失 ···························································.60 3.5 使用 PyTorch 实现模型构建与训练 ··································.67 3.6 模型结果分析·························································.72 3.7 总结与提问 ···························································.74 第 4 章 卷积神经网络 ························································.76 4.1 卷积操作 ······························································.76 4.2 卷积层汇总了什么····················································.83 4.3 卷积层的反向传播····················································.87 4.4 ReLU ··································································.93 4.5 Pooling 层 ·····························································.97 4.6 卷积神经网络实验····················································.101 4.7 卷积神经网络的感受野 ··············································.103 4.8 总结与提问 ···························································.112 第 5 章 网络初始化 ···························································.113 5.1 错误的初始化·························································.113 5.2 关于数值的初始化实验 ··············································.116 5.3 Xavier 初始化·························································.122 5.4 MSRA 初始化 ························································.128 5.5 ZCA 初始化···························································.132 5.6 总结与提问 ···························································.138 第 6 章 网络优化 ·····························································.140 6.1 梯度下降法 ···························································.140 6.2 动量法 ································································.145 6.3 随机梯度下降的变种算法 ············································.151 6.4 总结与提问 ···························································.164 第 7 章 进一步强化网络······················································.165 7.1 Dropout································································.165 7.2 Batch Normalization···················································.168 7.3 总结与提问 ···························································.176 第 8 章 高级网络结构 ························································.178 8.1 CIFAR10 数据集 ······················································.178 8.2 VGG 模型 ·····························································.179 8.3 ResNet·································································.183 8.4 Inception ······························································.195 8.5 通道分解的网络 ······················································.196 8.6 总结与提问 ···························································.202 第 9 章 网络可视化 ···························································.203 9.1 模型优化路径的简单可视化 ·········································.203 9.2 卷积神经网络的可视化 ··············································.206 9.3 图像风格转换·························································.211 9.4 总结与提问 ···························································.217 第 10 章 物体检测 ····························································.218 10.1 物体检测的评价指标 ·················································.218 10.2 YOLOv3：一阶段检测算法··········································.223 10.3 Faster RCNN：两阶段检测算法 ·····································.230 10.4 总结与提问 ···························································.235 第 11 章 词嵌入·······························································.237 11.1 One-Hot 编码的缺点··················································.237 11.2 分布式表征 ···························································.238 11.3 负采样 ································································.242 11.4 SGNS 实现243 11.5 tSNE···································································.247 11.6 总结与提问 ···························································.255 第 12 章 循环神经网络 ·······················································.256 12.1 语言模型与循环神经网络············································.256 12.2 RNN 实现·····························································.259 12.3 LSTM 网络 ···························································.262 12.4 语言模型实践·························································.266 12.5 LSTM 网络的可视化与分析 ·········································.272 12.6 RNN 的应用类型 ·····················································.274 12.7 CTC ···································································.276 12.8 总结与提问 ···························································.282 第 13 章 Transformer ························································.284 13.1 Transformer 模型的基本结构 ········································.286 13.2 模型训练与预测 ······················································.293 13.3 BERT 模型····························································.296 13.4 总结与提问 ···························································.303 第 14 章 深度分解模型 ·······················································.304 14.1 分解机 ································································.306 14.2 评价指标 AUC························································.310 14.3 DeepFM ·······························································.314 14.4 DeepFM 的改进方法 ·················································.317 14.5 总结与提问 ···························································.322