深入理解InfluxDB——时序数据库详解与实践

深入理解InfluxDB——时序数据库详解与实践"

作者:郑强、张伟、刘爽
ISBN:9787302630012
定价:¥65
字数:千字
页数:
出版时间:2023.04.01
开本:
版次:1-1
装帧:
出版社:清华大学出版社
简介

时序数据库是一种新型技术,主要用于工业互联网软件建设中。近年来,伴随着物联网技术在智能制造、交通、能源、智慧城市等领域的发展,时序数据库也发展迅速,成为搭建应用的必备数据库之一。《深入理解 InfluxDB》从 InfluxDB 的安装开始,一步步详细介绍 InfluxDB 的功能及原理,带领读者深入理解以 InfluxDB 为代表的时序数据库。

前言

据IDC预测,全球数据将从2018年的33ZB(泽字节)增长到2025年的175ZB,年均复合增长率为23%。到2025年,全球联网设备会有1500亿台,其中大多数设备将产生实时数据。届时,实时数据将超过其他类型的数据,成为全球第一大数据。

智能制造2025、工业互联网、工业4.0等政策推动了时序数据库的发展,全球对工业品的需求逐年上升。中国作为工业大国,2020年开始大力发展新基建,时序数据库作为智能制造的基础软件,其发展也同步受到关注。

和操作系统一样,数据库属于基础软件,更新迭代慢,一旦使用,可以在生产环境运行十几年,替换成本高,且企业替换意愿不强。企业初期选择数据库会非常慎重,使用后无论是管理层、执行层,还是技术人员都很难主动提出替换建议。

替换成本高、周期长、风险高造成了企业只相信头部数据库产品。毫无疑问,目前使用最多、技术生态最完善的时序数据库之一就是InfluxDB。InfluxDB是一个开源的、高性能的时序数据库,在时序数据库榜单DB-Engines Ranking上排名第一。正因为如此,熟练掌握InfluxDB已成为相关开发者非常重要的一项技能。本书将带领大家从InfluxDB的基础开始,一步一步理解InfluxDB,相信这些内容会对大家有帮助。

如何阅读本书

本书既是教程,又是参考指南,同时也适合作为高校相关专业教材。如果读者刚刚接触时序数据库开发,按照本书的章节顺序学习定会有所收获。

本书共14章,每章的内容简单介绍如下: 

第1章主要介绍了什么是时序数据、InfluxDB是什么,以及InfluxDB的发展历史。

第2章主要介绍了InfluxDB的安装及配置。

第3章主要介绍了InfluxDB的可视化UI工具。

第4章主要介绍了InfluxDB的基本写入、查询操作。

第5章主要介绍了InfluxDB的常用函数和运算。

第6章主要介绍了InfluxDB的连续查询。

第7章主要介绍了InfluxDB的数据保留策略。

第8章主要介绍了InfluxDB的数据安全策略。

第9章主要介绍了InfluxDB的性能评估。

第10章主要介绍了InfluxDB集群相关内容。

第11章主要介绍了InfluxDB的数据备份与恢复。

第12章主要介绍了InfluxDB与程序设计。

第13章主要介绍了InfluxDB数据处理语言FLux。

第14章主要介绍了InfluxDB存储引擎知识。

读者对象

在本书的编写过程中,尽可能做到通俗易懂、由浅入深,不仅适用于初学者学习,也适用于专业人员学习。

本书可作为高校相关专业教材,也适合软件工程师、软件架构师、数据库工程师等从业人员阅读。

读者交流与图书反馈

本书的读者还可以访问InfluxDB专栏补充学习。该专栏搭建了一个供开发者交流学习的在线平台,阅读过程中如有疑问,也可以在网站上向作者提问,期待能够得到你们的真挚反馈。

由于作者水平有限,编写时间仓促,书中难免会出现一些错误或表达不准确的地方,恳请读者批评指正。我们也会将书中的勘误发布在专栏中,供大家参考。

编写说明

本书编写团队由业界一线研发人员组成,其中郑强负责全书的框架搭建及第1~9章的撰写,字数约28万字;张伟负责第10~13章的撰写,字数约6万字;刘爽负责第14章的撰写,字数约2万字。

致谢

感谢清华大学出版社的编辑,因为你们的帮助,这本书才得以问世。最后要感谢的就是你,我亲爱的读者,感谢你拿起这本书,你的认可是我们最大的快乐。

作者

目录

第1章 时序数据库InfluxDB简介

1.1 数据简史 ······························································································2

1.1.1 什么是时序数据 ··················································································3

1.1.2 时序数据库的历史 ···············································································3

1.2 InfluxDB简介 ························································································4

1.2.1 什么是InfluxDB ··················································································4

1.2.2 InfluxDB的历史 ··················································································5

1.2.3 InfluxDB现状及未来 ············································································6

1.3 InfluxDB的基本概念 ···············································································7

1.4 InfluxDB的设计理念 ···············································································7

1.5 InfluxDB的核心特性 ···············································································8

1.5.1 内置REST接口 ··················································································8

1.5.2 数据Tag标记 ·····················································································8

1.5.3 类SQL的查询语句 ··············································································9

1.5.4 高性能 ······························································································9

1.5.5 开源 ·······························································································10

1.6 时序数据库的应用场景 ··········································································10

1.6.1 在工业环境监控中的应用 ····································································10

1.6.2 在物联网IoT设备采集存储中的应用 ·····················································11

1.6.3 互联网业务性能监控服务 ····································································11

1.6.4 在智能汽车中的应用 ··········································································12

1.7 小结 ··································································································12

第2章 InfluxDB的安装、配置、启动

2.1 在不同操作系统上安装InfluxDB································································14

2.1.1 硬件要求 ·························································································14

2.1.2 在Ubuntu和Debian系统中安装InfluxDB ···············································15

2.1.3 在RedHat和CentOS系统中安装InfluxDB ··············································20

2.1.4 在Windows系统中安装InfluxDB··························································21

2.1.5 在macOS系统中安装InfluxDB ·····························································22

2.1.6 使用Docker安装InfluxDB ···································································24

2.1.7 InfluxDB的端口设置 ··········································································25

2.1.8 InfluxDB 程序的使用 ··········································································25

2.1.9 InfluxDB的配置详解 ··········································································31

2.2 小结 ··································································································37

第3章InfluxDB UI数据可视化

3.1 InfluxDB UI ·························································································39

3.1.1 安装配置 ·························································································39

3.1.2 常用功能介绍 ···················································································41

3.2 Chronograf ·························································································47

3.2.1 安装配置 ·························································································48

3.2.2 常用功能介绍 ···················································································50

3.3 小结 ··································································································57

第4章InfluxDB基本操作写入与查询

4.1 客户端命令行方式 ················································································59

4.1.1 简介·······························································································59

4.1.2 使用InfluxDB ···················································································59

4.1.3 Influx基本命令·················································································61

4.2 数据样本 ····························································································62

4.2.1 空气传感器样本数据 ··········································································63

4.2.2 鸟类迁徙样本数据 ·············································································64

4.2.3 NOAA样本数据 ················································································64

4.2.4 美国地质勘探局地震数据 ····································································64

4.3 行协议 ·······························································································65

4.3.1 行协议案例 ······················································································65

4.3.2 行协议语法 ······················································································66

4.3.3 行协议要素分析 ················································································66

4.3.4 数据类型 ·························································································67

4.3.5 引号·······························································································68

4.3.6 特殊字符 ·························································································68

4.3.7 注释·······························································································69

4.3.8 重复数据 ·························································································69

4.4 桶操作 ·······························································································70

目 录

VII

4.5 写入操作 ····························································································71

4.5.1 写入数据 ·························································································71

4.5.2 文件数据导入 ···················································································72

4.6 查询操作 ····························································································73

4.6.1 select语句 ·······················································································74

4.6.2 类型转换 ·························································································77

4.6.3 where子句 ·······················································································78

4.6.4 函数 ·······························································································80

4.6.5 group by子句 ···················································································82

4.6.6 into子句 ··························································································85

4.6.7 排序 ·······························································································86

4.6.8 limit和slimit子句 ·············································································86

4.6.9 offset和soffset子句 ···········································································86

4.6.10 Time Zone子句 ················································································87

4.6.11 时间语法 ························································································87

4.7 小结 ··································································································88

第5章 InfluxDB的函数与运算

5.1 样本数据导入 ······················································································90

5.2 InfluxDB函数 ······················································································91

5.2.1 函数说明 ·························································································91

5.2.2 聚合类函数 ······················································································92

5.2.3 选择类函数 ······················································································94

5.2.4 转换类函数 ······················································································95

5.2.5 预测类函数 ······················································································98

5.3 数学运算 ····························································································99

5.3.1 常用运算符 ······················································································99

5.3.2 数学运算符的常见问题 ······································································101

5.4 小结 ································································································102

第6章 InfluxDB连续查询

6.1 连续查询 ··························································································104

6.2 样本数据导入 ····················································································104

6.3 创建连续查询 ····················································································106

6.3.1 <cq_query>语句 ···············································································106

6.3.2 连续查询运行时刻及查询的时间范围 ····················································107

6.3.3 连续查询举例 ··················································································107

6.4 复杂连续查询 ····················································································109

6.4.1 创建高级连续查询 ············································································109

6.4.2 高级连续查询的时间设置 ···································································109

6.4.3 GROUP BY time()、EVERY、FOR三者关系 ···········································111

6.4.4 高级连续查询举例 ············································································112

6.5 连续查询的管理 ··················································································113

6.5.1 查询所有连续查询 ············································································113

6.5.2 删除连续查询 ··················································································114

6.5.3 修改连续查询 ··················································································114

6.6 连续查询案例分析 ···············································································114

6.6.1 数据预处理 ·····················································································115

6.6.2 降低数据采样率 ···············································································115

6.6.3 HAVING功能 ··················································································116

6.6.4 替换嵌套函数 ··················································································116

6.7 小结 ·································································································117

第7章InfluxDB数据保留策略

7.1 保留策略 ···························································································119

7.2 创建保留策略 ·····················································································119

7.2.1 使用CREATE RETENTION POLICY创建保留策略 ························119

7.2.2 创建保留策略举例 ············································································121

7.3 查询保留策略 ····················································································121

7.4 修改保留策略 ····················································································122

7.5 删除保留策略 ····················································································123

7.6 综合实例 ··························································································123

7.6.1 样本数据 ························································································123

7.6.2 实验目标 ························································································124

7.6.3 实验过程 ························································································124

7.6.4 结论······························································································126

7.7 小结 ································································································126

第8章InfluxDB数据安全策略

8.1 认证技术 ··························································································128

8.1.1 简介······························································································128

8.1.2 认证方式 ························································································128

8.1.3 Token认证 ······················································································128

8.1.4 1.x版本兼容授权认证 ·······································································131

目 录

8.1.5 授权使用 ························································································137

8.2 权限类型 ··························································································142

8.3 用户管理 ··························································································143

8.4 权限控制实战 ····················································································146

8.4.1 创建与授权 ·····················································································146

8.4.2 测试效果 ························································································147

8.5 小结 ································································································149

第9章 InfluxDB性能评估

9.1 性能测试工具 ····················································································151

9.1.1 安装Go语言运行环境 ·······································································151

9.1.2 测试工具安装 ··················································································153

9.2 基准性能测试 ····················································································154

9.2.1 测试环境 ························································································154

9.2.2 写入性能测试 ··················································································154

9.3 性能优化 ··························································································157

9.3.1 数据写入优化方案 ············································································157

9.3.2 数据查询优化方案 ············································································157

9.4 性能报警 ··························································································159

9.4.1 创建检查 ························································································159

9.4.2 添加通知端点 ··················································································162

9.4.3 创建通知规则 ··················································································163

9.5 小结 ································································································164

第10章 InfluxDB集群

10.1 集群简介 ·························································································166

10.1.1 集群架构概述 ················································································166

10.1.2 数据所在的地方 ·············································································166

10.1.3 节点数量 ······················································································167

10.2 集群安装配置 ···················································································167

10.2.1 申请试用 ······················································································167

10.2.2 环境准备 ······················································································169

10.2.3 添加DNS条目 ···············································································170

10.2.4 Meta节点的安装配置 ·······································································171

10.2.5 Data节点的安装配置 ·······································································173

10.3 小结 ·······························································································176

第11章 备份管理

11.1 单机版备份管理 ················································································178

11.1.1 备份数据 ·······················································································178

11.1.2 恢复数据 ·······················································································179

11.2 集群版备份管理 ················································································180

11.2.1 备份数据 ·······················································································181

11.2.2 恢复数据 ·······················································································182

11.2.3 导出数据 ·······················································································184

11.2.4 导入数据 ·······················································································185

11.3 小结 ·······························································································185

第12章 InfluxDB与程序设计

12.1 Java SDK使用 ·················································································187

12.1.1 使用Java在InfluxDB中写入数据 ·······················································188

12.1.2 WriteApiBlocking ············································································192

12.1.3 使用Java查询InfluxDB中的数据 ·······················································193

12.1.4 queryApi ·······················································································195

12.1.5 使用Java为bucket添加权限 ·····························································196

12.1.6 连接InfluxDB的另一种方式—用户名和密码 ······································198

12.2 Python SDK使用 ··············································································200

12.2.1 使用Python在InfluxDB中写入数据 ····················································201

12.2.2 使用Python查询InfluxDB中的数据 ····················································202

12.3 小结 ·······························································································203

第13章 InfluxDB数据处理语言Flux

13.1 Flux概述 ························································································205

13.2 基本数据类型 ···················································································205

13.2.1 Boolean(布尔值) ·········································································206

13.2.2 Bytes(字节) ················································································206

13.2.3 Duration(持续时间) ······································································206

13.2.4 String(字符串类型) ······································································207

13.2.5 Time(时间类型) ··········································································207

13.2.6 Float(浮点类型) ··········································································208

13.2.7 Integer(整数类型) ········································································208

13.2.8 Null(空值) ·················································································209

13.3 查询数据源 ······················································································209

13.3.1 from()函数 ····················································································210

13.3.2 range()函数 ···················································································210

13.3.3 查询InfluxDB 1.x ············································································211

13.3.4 远程查询InfluxDB Cloud 或InfluxDB 2.x ··············································211

13.4 写入数据源 ······················································································212

13.5 小结 ·······························································································213

第14章 InfluxDB存储引擎

14.1 InfluxDB存储引擎历史 ·······································································215

14.2 LSM-Tree(LSM树)概述··································································215

14.2.1 LSM-Tree存储原理 ·········································································216

14.2.2 优势和问题 ···················································································220

14.3 InfluxDB数据格式 ·············································································220

14.4 TSM存储组件 ··················································································221

14.5 TSM file详解 ··················································································222

14.5.1 SeriesKey(时间序列关键字) ···························································223

14.5.2 Series Data Block ·············································································224

14.5.3 Series Index Block ············································································225

14.6 TSM数据写入 ··················································································227

14.6.1 写入总体框架 ················································································227

14.6.2 Shard路由 ·····················································································228

14.6.3 倒排索引引擎构建倒排索引 ······························································228

14.6.4 写入流程 ······················································································229

14.7 小结 ·······························································································230

作者简介

编辑推荐

《深入理解 InfluxDB》适合时序数据库工程师、物联网采集工程师、运维工程师阅读,也可供数据库爱好者、监控技术爱好者、工业互联网平台爱好者参考学习,还适合作为高校相关专业教材。

作者寄语

电子资料

www.luweidong.cn

下一个