本书是容器与Kubernetes网络的基础和进阶书籍，旨在让更多人了解和学习云原生时代的底层网络模型与实现机制，指导企业在落地云原生时的网络方案选型。全书包括：容器网络虚拟化基础、Docker容器网络、Kubernetes网络和Istio网络4部分，共6章。第1章容器网络虚拟化基础将支撑容器网络的内核技术娓娓道来。第2章简单介绍了Docker原生的容器网络能力。Kubernetes网络分为3章，第3章介绍Kubernetes网络的基础概念和使用，第4章为读者剖析了Kubernetes网络的底层实现原理，第5章详解了业界主流的Kubernetes网络插件。Istio网络总共1章，重点解析Istio网络流量管控的背后机制。本书适合作为高等院校计算机相关专业云计算课程的参考资料，也适合云计算从业者，特别是希望对云原生网络技术有较深入了解并希望将其应用到日常工作中的所有读者阅读。

自序 缘起 这些年来，我观察到每次 Linux 世界的重大技术创新都发源于内核，经过一层层面向用户的抽象和封装，演化出应用层的森罗万象。正所谓万变不离其宗！ 我自认为是个“old school”（老派）的人，坚信维持这个世界运转的、最本质的那部分东西是不会轻易改变的。尽管新技术“乱花渐欲迷人眼”，但经历过时间锤炼的实用技术和工具总是历久弥新的。一个很好的例子便是传统的网络虚拟化和 BGP，它们就是在容器这个新瓶子里焕发第二春的。因此，当有时髦的新技术出现时，我的第一反应是这些新技术底层是怎么实现的，对那些看起来酷炫的功能反倒没有兴趣。正如 OpenStack 兴起的那些年，当大家都在谈论 nova、neutron 这类调度虚拟机和网络的组件时，我默默地翻读了内核虚拟机（KVM）的源码。至今，我对那种奇妙的感觉记忆犹新，恰如一个发烧级摄影爱好者走进暗房，欣喜若狂地亲自手洗一张可触摸的胶片影像。当我读懂了 KVM 时，再回过头来看 OpenStack 便有了一种“会当凌绝顶，一览众山小”的豁然贯通之感。 工作之余让心静下来，细细品味，认真思考技术的本质——相信这是所有有激情、有梦想的工程师的共同追求。虽然在软件版本快速迭代的高压面前，这种良好的追求有时也会变成一种奢求，但是我坚信工程师不能只当一个使用者，而一定要理解当前正在使用的技术的底层实现机制。因此，在我的《云原生分布式存储基石：etcd 深入解析》一书中，开篇只字未提全书主角 etcd，而耗费将近 80 页的笔墨，从分布式系统的基本理论一直讲到一致性协议 Raft。尽管内容看似与这个高速发展、追求快节奏的社会“格格不入”，但我仍希望能够通过出版技术书这种本身就慢节奏且带仪式感的行为沉淀自己的思考。如果能够跟有缘的读者碰撞出思想的火花，则将是我人生的一大幸事！ 我为什么写这本书 云计算的世界里，计算最基础，存储最重要，网络最复杂。在 Kubernetes 已经成为云原生代名词的今天，市面上介绍 Kubernetes 的书籍已经很多，然而限于篇幅或术业有专攻等诸多主客观因素，不少书籍对 Kubernetes 网络部分的讲解只是蜻蜓点水，甚至有些还存在专业性的错误。我经常在一些学习 Kubernetes 的论坛和群里看到有用户抱怨：传统网络架构都还没搞明白，又要理解容器网络。容器网络领域不但存在大量的术语，而且理解具体的方案需要不少前置知识，这无形中增加了学习的难度。计算机网络是我在大学里最喜欢，也是最擅长的一门课程。在我看来，计算机网络趣味性强，而且对逻辑性和动手能力要求较高。看到整个云原生网络领域正发生着激动人心的技术变革，意义不亚于上一次 SDN 兴起带来的冲击，我感觉这对传统网络工程师来说会是一次自我升级转型的契机。于是，我萌生了专门为 Docker、Kubernetes 的用户，以及传统网络工程师撰写一本云原生网络书籍的想法，破除他们学习过程中“不识庐山真面目，只缘身在此山中”的无力感。 关于本书 虽然书名是《Kubernetes 网络权威指南：基础、原理与实践》，但全书内容并不局限于Kubernetes。我对本书的定位是云原生领域的网络权威指南，企业落地方案的选型参考。按照我“old school”的思路，本书特别注重提供理解容器网络所必需的基础知识，会由浅入深地从架构、使用、实现原理等多方面展开，试图为读者呈现整个云原生网络的知识体系。 全书的脉络是：以 Linux 网络虚拟化基础作为“暖场嘉宾”，以 Docker 原生的容器网络“承前启后”，随后是主角 Kubernetes 网络“粉墨登场”，在各类 CNI 插件“沙场点兵”过后，以代表容器下半场的服务网格 Istio“谢幕”。 王安石在登上飞来峰后曾吟下“不畏浮云遮望眼，自缘身在最高层”这样的千古佳句。希望本书能够成为云计算 2.0 时代的弄潮儿们叩开网络大门的敲门砖，在解决各类场景下错综复杂的问题时能够做到“口中有粮、心中不慌”。不论是定位疑难杂症，还是技术选型，抑或是定制化开发都能轻松驾驭！ “人生不止眼前的苟且，还有诗和远方的田野”，愿更多的同路人加入。 杜军 2019 年 8 月于厦门鼓浪屿

目录 第 1 章 夯实基础：Linux 网络虚拟化 1 1.1 网络虚拟化基石：network namespace ············································.1 1.1.1 初识 network namespace ··················································.2 1.1.2 配置 network namespace ··················································.3 1.1.3 network namespace API 的使用 ···········································.6 1.1.4 小结 ·······································································.12 1.2 千呼万唤始出来：veth pair ·······················································.12 1.2.1 veth pair 内核实现·························································.14 1.2.2 容器与 host veth pair 的关系 ··············································.15 1.2.3 小结 ·······································································.17 1.3 连接你我他：Linux bridge ························································.17 1.3.1 Linux bridge 初体验 ·······················································.17 1.3.2 把 IP 让给 Linux bridge ···················································.21 1.3.3 将物理网卡添加到 Linux bridge ··········································.22 1.3.4 Linux bridge 在网络虚拟化中的应用 ·····································.25 1.3.5 网络接口的混杂模式 ·····················································.26 1.4 给用户态一个机会：tun/tap 设备·················································.28 1.4.1 tun/tap 设备的工作原理···················································.28 1.4.2 利用 tun 设备部署一个 VPN··············································.29 1.4.3 tun 设备编程 ······························································.31 1.5 iptables·············································································.34 1.5.1 祖师爷 netfilter ····························································.34 1.5.2 iptables 的三板斧：table、chain 和 rule ··································.36 1.5.3 iptables 的常规武器 ·······················································.39 1.6 初识 Linux 隧道：ipip ····························································.45 1.6.1 测试 ipip 隧道 ·····························································.46 1.6.2 ipip 隧道测试结果复盘 ···················································.49 1.6.3 小结 ·······································································.50 1.7 Linux 隧道网络的代表：VXLAN ················································.51 1.7.1 为什么需要 VXLAN ······················································.51 1.7.2 VXLAN 协议原理简介 ···················································.52 1.7.3 VXLAN 组网必要信息 ···················································.54 1.7.4 VXLAN 基本配置命令 ···················································.55 1.7.5 VXLAN 网络实践·························································.56 1.7.6 分布式控制中心 ··························································.63 1.7.7 自维护 VTEP 组 ··························································.63 1.7.8 小结 ·······································································.68 1.8 物理网卡的分身术：Macvlan·····················································.68 1.8.1 Macvlan 五大工作模式解析 ··············································.68 1.8.2 测试使用 Macvlan 设备 ···················································.72 1.8.3 Macvlan 的跨机通信 ······················································.73 1.8.4 Macvlan 与 overlay 对比 ··················································.74 1.8.5 小结 ·······································································.75 1.9 Macvlan 的救护员：IPvlan························································.75 1.9.1 IPvlan 简介 ································································.75 1.9.2 测试 IPvlan ································································.77 1.9.3 Docker IPvlan 网络 ························································.78 1.9.4 小结 ·······································································.78 第 2 章 饮水思源：Docker 网络模型简介 79 2.1 主角登场：Linux 容器 ····························································.79 2.1.1 容器是什么································································.79 2.1.2 容器与虚拟机对比 ························································.80 2.1.3 小结 ·······································································.81 2.2 打开万花筒：Docker 的四大网络模式 ···········································.81 2.2.1 bridge 模式 ································································.82 2.2.2 host 模式 ··································································.83 2.2.3 container 模式 ·····························································.84 2.2.4 none 模式··································································.85 2.3 最常用的 Docker 网络技巧 ·······················································.85 2.3.1 查看容器 IP ·······························································.85 2.3.2 端口映射 ··································································.86 2.3.3 访问外网 ··································································.87 2.3.4 DNS 和主机名 ····························································.87 2.3.5 自定义网络································································.88 2.3.6 发布服务 ··································································.90 2.3.7 docker link：两两互联 ····················································.91 2.4 容器网络的第一个标准：CNM···················································.93 2.4.1 CNM 标准 ·································································.93 2.4.2 体验 CNM 接口 ···························································.94 2.4.3 Libnetwork·································································.95 2.4.4 Libnetwork 扩展···························································.97 2.4.5 小结 ·······································································.98 2.5 天生不易：容器组网的挑战 ······················································.99 2.5.1 容器网络挑战综述 ························································.99 2.5.2 Docker 的解决方案 ·······················································.101 2.5.3 第三方容器网络插件 ·····················································.102 2.5.4 小结 ·······································································.103 2.6 如何做好技术选型：容器组网方案沙场点兵 ····································.103 2.6.1 隧道方案 ··································································.104 2.6.2 路由方案 ··································································.104 2.6.3 容器网络组网类型 ························································.106 2.6.4 关于容器网络标准接口···················································.107 2.6.5 小结 ·······································································.108 第 3 章 标准的胜利：Kubernetes 网络原理与实践 109 3.1 容器基础设施的代言人：Kubernetes·············································.109 3.1.1 Kubernetes 简介 ···························································.109 3.1.2 Kubernetes 能做什么 ······················································.111 3.1.3 如何用 Kubernetes ························································.113 3.1.4 Docker 在 Kubernetes 中的角色 ··········································.113 3.2 终于等到你：Kubernetes 网络 ····················································.114 3.2.1 Kubernetes 网络基础 ······················································.114 3.2.2 Kubernetes 网络架构综述·················································.115 3.2.3 Kubernetes 主机内组网模型 ··············································.117 3.2.4 Kubernetes 跨节点组网模型 ··············································.118 3.2.5 Pod 的 hosts 文件··························································.120 3.2.6 Pod 的 hostname ···························································.121 3.3 Pod 的核心：pause 容器 ··························································.124 3.4 打通 CNI 与 Kubernetes：Kubernetes 网络驱动··································.131 3.4.1 即将完成历史使命：Kubenet·············································.131 3.4.2 网络生态第一步：CNI ···················································.133 3.5 找到你并不容易：从集群内访问服务············································.139 3.5.1 Kubernetes Service 详解···················································.141 3.5.2 Service 的三个 port························································.145 3.5.3 你的服务适合哪种发布形式··············································.146 3.5.4 Kubernetes Service 发现···················································.150 3.5.5 特殊的无头 Service ·······················································.151 3.5.6 怎么访问本地服务 ························································.153 3.6 找到你并不容易：从集群外访问服务············································.154 3.6.1 Kubernetes Ingress·························································.155 3.6.2 小结 ·······································································.157 3.7 你的名字：通过域名访问服务 ···················································.158 3.7.1 DNS 服务基本框架 ·······················································.158 3.7.2 域名解析基本原理 ························································.159 3.7.3 DNS 使用··································································.161 3.7.4 调试 DNS··································································.166 3.8 Kubernetes 网络策略：为你的应用保驾护航 ····································.167 3.8.1 网络策略应用举例 ························································.168 3.8.2 小结 ·······································································.172 3.9 前方高能：Kubernetes 网络故障定位指南 ·······································.173 3.9.1 IP 转发和桥接·····························································.173 3.9.2 Pod CIDR 冲突 ····························································.175 3.9.3 hairpin ·····································································.176 3.9.4 查看 Pod IP 地址 ··························································.176 3.9.5 故障排查工具 ·····························································.178 3.9.6 为什么不推荐使用 SNAT ·················································.180 第 4 章 刨根问底：Kubernetes 网络实现机制 183 4.1 岂止 iptables：Kubernetes Service 官方实现细节探秘 ···························.183 4.1.1 userspace 模式·····························································.184 4.1.2 iptables 模式·······························································.186 4.1.3 IPVS 模式 ·································································.191 4.1.4 iptables VS. IPVS ··························································.198 4.1.5 conntrack ··································································.199 4.1.6 小结 ·······································································.200 4.2 Kubernetes 极客们的日常：DIY 一个 Ingress Controller·························.201 4.2.1 Ingress Controller 的通用框架 ············································.202 4.2.2 Nginx Ingress Controller 详解 ·············································.202 4.2.3 小结 ·······································································.209 4.3 沧海桑田：Kubernetes DNS 架构演进之路 ······································.209 4.3.1 Kube-dns 的工作原理 ·····················································.209 4.3.2 上位的 CoreDNS ··························································.212 4.3.3 Kube-dns VS. CoreDNS ···················································.217 4.3.4 小结 ·······································································.220 4.4 你的安全我负责：使用 Calico 提供 Kubernetes 网络策略·······················.220 4.4.1 部署一个带 Calico 的 Kubernetes 集群 ···································.221 4.4.2 测试 Calico 网络策略 ·····················································.225 第 5 章 百花齐放：Kubernetes 网络插件生态 228 5.1 从入门到放弃：Docker 原生网络的不足·········································.228 5.2 CNI 标准的胜出：从此江湖没有 CNM ··········································.229 5.2.1 CNI 与 CNM 的转换 ······················································.230 5.2.2 CNI 的工作原理···························································.231 5.2.3 为什么 Kubernetes 不使用 Libnetwork ···································.235 5.3 Kubernetes 网络插件鼻祖 flannel ·················································.238 5.3.1 flannel 简介································································.239 5.3.2 flannel 安装配置 ··························································.241 5.3.3 flannel backend 详解 ······················································.244 5.3.4 flannel 与 etcd ·····························································.256 5.3.5 小结 ·······································································.257 5.4 全能大三层网络插件：Calico ····················································.257 5.4.1 Calico 简介 ································································.258 5.4.2 Calico 的隧道模式 ························································.263 5.4.3 安装 Calico ································································.263 5.4.4 Calico 报文路径 ···························································.264 5.4.5 Calico 使用指南 ···························································.267 5.4.6 为什么 Calico 网络选择 BGP ·············································.272 5.4.7 小结 ·······································································.274 5.5 Weave：支持数据加密的网络插件 ···············································.276 5.5.1 Weave 简介································································.276 5.5.2 Weave 实现原理···························································.277 5.5.3 Weave 安装································································.278 5.5.4 Weave 网络通信模型 ·····················································.280 5.5.5 Weave 的应用示例 ························································.282 5.5.6 小结 ·······································································.288 5.6 Cilium：为微服务网络连接安全而生 ············································.288 5.6.1 为什么使用 Cilium ························································.289 5.6.2 以 API 为中心的微服务安全 ·············································.294 5.6.3 BPF 优化的数据平面性能 ················································.295 5.6.4 试用 Cilium：网络策略···················································.297 5.6.5 小结 ·······································································.299 5.7 Kubernetes 多网络的先行者：CNI-Genie ········································.299 5.7.1 为什么需要 CNI-Genie····················································.300 5.7.2 CNI-Genie 功能速递 ······················································.302 5.7.3 容器多 IP ··································································.303 第 6 章 Kubernetes 网络下半场：Istio 305 6.1 微服务架构的大地震：sidecar 模式 ··············································.305 6.1.1 你真的需要 Service Mesh 吗··············································.306 6.1.2 sidecar 模式 ·······························································.307 6.1.3 Service Mesh 与 sidecar ···················································.307 6.1.4 Kubernetes Service VS. Service Mesh ·····································.309 6.1.5 Service Mesh 典型实现之 Linkerd ········································.310 6.2 Istio：引领新一代微服务架构潮流···············································.312 6.2.1 Istio 简介 ··································································.312 6.2.2 Istio 安装 ··································································.313 6.2.3 Istio 路由规则的实现 ·····················································.317 6.3 一切尽在不言中：Istio sidecar 透明注入·········································.319 6.3.1 Init 容器 ···································································.319 6.3.2 sidecar 注入示例 ··························································.319 6.3.3 手工注入 sidecar ··························································.326 6.3.4 自动注入 sidecar ··························································.327 6.3.5 从应用容器到 sidecar 代理的通信········································.329 6.4 不再为 iptables 脚本所困：Istio CNI 插件 ·······································.330 6.5 除了微服务，Istio 还能做更多 ···················································.331