本书力求与时俱进，将最新、最具代表性的虚拟现实系统及开发相关知识整理出来，从实践与应用的角度出发，在阐述虚拟现实技术特点、理论知识的基础上，介绍虚拟现实系统开发的实践方法及案例，使读者能够在较短时间内认识和掌握虚拟现实开发的相关技术，并具备开发具有沉浸交互、引人构想的虚拟现实系统的能力。本书共10 章，内容包括虚拟现实技术概述、虚拟现实的感知过程、虚拟现实系统介绍、用户界面与人机交互、系统输入及运动跟踪、系统输出系统、开发环境和开发过程、网络虚拟现实、增强现实。本书可作为高等院校计算机科学与技术、软件工程、人工智能、数字媒体等专业虚拟现实技术课程的教材，也可供从事虚拟现实技术行业的工程技术人员及虚拟现实爱好者参考。

前言 虚拟现实（Virtual Reality，VR）是目前最热门的技术之一，也是21 世纪重 要的发展学科及影响人们生活的重要技术之一。鉴于此，很多高等院校在计算机 科学与技术、软件工程、人工智能及数字媒体等专业开设了虚拟现实课程；2020 年，教育部新增审批本科专业名单中就包含“虚拟现实技术专业”。同时，社会也 需要大量的虚拟现实系统开发人员。本书正是为了有一定计算机基础的读者编写 的，是一本虚拟现实系统开发快速入门的图书，也能满足高校虚拟现实相关课程 的教学需要。 本书作者从2007 年开始就在重庆邮电大学计算机科学与技术学院从事虚拟 现实课程的教学工作，参与了大量的系统开发实践，十几年来一直在做虚拟现实 技术与应用的相关研究。目前，市面上的大部分图书是以翻译国外的研发过程和 基础概念性教材为主的，其中有一部分书籍发表的年代相对较早。在经历了2016 年即“虚拟现实元年”的爆发之后，虚拟现实系统应用与开发进入了一个相对较 为平静的发展时期，虚拟现实技术飞速发展，新方法、新的人机交互设备不断涌 现，各种软件开发工具包层出不穷，虚拟现实系统开发的许多新思维、新方法和 新例证引领着虚拟现实开发领域的不断变革。虚拟现实技术的发展日新月异，旧 的方法和系统被快速迭代，新的技术不断出现。 本书编写的目的是为读者提供一本既基础全面，又紧跟时代需求，还能满足 最新虚拟现实系统开发入门需求的图书。本书简明扼要地介绍了虚拟现实系统开 发的核心框架，帮助读者快速了解虚拟现实系统，掌握虚拟现实系统开发的基本 思维和方法；结合了实践和理论的相关知识，深入浅出地介绍了虚拟现实的相关 概念、感知循环和人机交互过程；通过实例介绍了虚拟现实系统的开发环境搭建 及漫游系统的开发过程。 本书共 10 章。第1 章主要介绍虚拟现实的核心内容，包括虚拟环境和虚拟现 实的概念定义及其发展演变的过程，虚拟现实系统区别于其他类似的多媒体技术 的主要特征，着重探讨沉浸和互动以及构想特性以及虚拟现实系统应具有的共性 和结构。第2 章主要介绍虚拟现实系统提供计算机生成的人工刺激来创造虚拟环 境中的存在感所需要的理解的人类感知过程；探讨人类最重要的感官的行为和生 理特征，以便更好地理解运行虚拟现实系统的约束条件。第3 章主要介绍主流虚 拟现实系统及常用组件。第4～6 章着重介绍虚拟现实系统的交互输入与输出循环 中的主要方法及原理。第7 章主要介绍现阶段下虚拟现实系统开发所需具备的网 络及软硬件环节，探讨5G 技术对VR 技术的影响，并着重介绍虚拟现实系统开发 下的主要软件环境及工具，探讨在相对简单的硬件和网络环境下如何开始虚拟现 实系统的开发工作。第8 章是本书的重点内容，通过一个虚拟现实漫游项目实例 介绍虚拟现实系统开发的基本使用流程，包括分工、原则、环境搭建、创建场景、 输入/输出操作、基本的物理碰撞、UI 显示和逻辑处理等，模拟实际的系统开发、 场景建设、最后发布调试的过程；还介绍了虚拟现实系统的主要应用开发类型及 应用领域。第9 章介绍了网络虚拟现实系统开发。第10 章是扩展章节，介绍了从 虚拟现实延展的增强现实系统的开发及未来可穿戴式应用的发展方向。 本书可作为高等院校计算机科学与技术、软件工程、人工智能、数字媒体等 专业虚拟现实技术课程的教材，也可供从事虚拟现实技术行业的工程技术人员及 虚拟现实爱好者参考。 Greengard Samuel 著的Virtual Reality 一书及Stanislav Stankovic 著的Virtual Reality and Virtual Environments in 10 Lectures 一书对作者完成本书起到了非常重 要的参考作用。另外，本书还引用了大量资料，在此向所有相关图书及文献的作 者致以衷心的感谢！ 由于作者水平有限，加之编写时间仓促，本书错误与疏漏之处在所难免，敬 请读者批评指正。

第1 章虚拟现实概述····················································································.1 1.1 什么是虚拟现实··················································································.1 1.2 虚拟现实发展历史···············································································.3 1.2.1 人的认知特征···················································································.3 1.2.2 沉浸感起源······················································································.3 1.2.3 视觉艺术中的真实感··········································································.4 1.2.4 摄影及动态图像的发展·······································································.5 1.2.5 立体视觉·························································································.6 1.2.6 计算设备发展历史·············································································.7 1.2.7 人机交互发展···················································································.7 1.2.8 虚拟现实的出现················································································.8 1.2.9 虚拟现实的现状················································································11 1.2.10 国内的虚拟现实现状········································································12 1.3 什么是虚拟现实系统···········································································13 1.3.1 人和真实环境之间的联系····································································14 1.3.2 虚拟现实技术如何改变人对真实世界的认知············································14 1.3.3 虚拟现实系统的目标··········································································15 1.3.4 虚拟现实系统的特征··········································································16 1.3.5 沉浸感····························································································17 1.3.6 多模态感知······················································································18 1.3.7 恐怖谷曲线······················································································18 1.3.8 交互性····························································································20 1.3.9 交互与沉浸的循环·············································································20 1.3.10 构想性··························································································21 1.4 虚拟现实系统的类型及应用·································································22 1.4.1 虚拟现实系统类型·············································································22 1.4.2 虚拟现实系统的应用··········································································23 1.4.3 虚拟现实影响的领域··········································································24 第 2 章虚拟现实的感知过程 ·········································································26 2.1 人类的感官·························································································26 2.1.1 感官的目的······················································································28 2.1.2 感知和认知过程················································································29 2.2 视觉····································································································30 2.2.1 颜色感知·························································································32 2.2.2 深度知觉·························································································32 2.2.3 人脑的模式识别················································································35 2.2.4 运动知觉·························································································36 2.3 立体听觉·····························································································37 2.4 平衡感································································································37 2.5 触觉····································································································38 2.6 本体感觉·····························································································39 2.7 综合感知·····························································································39 第 3 章虚拟现实系统····················································································40 3.1 虚拟现实系统基本要求········································································40 3.2 虚拟现实系统架构···············································································41 3.2.1 计算平台·························································································42 3.2.2 基于个人计算机的虚拟现实系统···························································43 3.2.3 GPU ·······························································································43 3.2.4 分布式虚拟现实系统··········································································44 3.2.5 移动设备·························································································44 3.2.6 标准I/O 设备···················································································45 3.2.7 虚拟现实专用I/O 设备·······································································45 3.3 CAVE 系统··························································································46 3.4 虚拟现实系统软件···············································································47 3.4.1 场景图····························································································47 3.4.2 三维渲染引擎···················································································49 3.4.3 物理引擎·························································································50 3.4.4 音频引擎·························································································50 第4 章用户界面与人机交互 ·········································································51 4.1 用户体验·····························································································51 4.1.1 人机交互·························································································52 4.1.2 评估用户体验···················································································54 4.2 框架设计·····························································································55 4.3 虚拟环境的用户界面设计····································································56 4.4 典型的三维交互任务···········································································57 4.5 导航····································································································58 4.5.1 参照·······························································································58 4.5.2 寻路·······························································································59 4.5.3 寻路帮助·························································································59 4.5.4 漫游·······························································································60 4.5.5 漫游任务·························································································61 4.6 选择和操作·························································································61 4.7 系统控制·····························································································63 4.8 图形菜单设计······················································································63 4.9 语音指令·····························································································64 4.10 手势指令···························································································65 4.11 交互工具···························································································65 4.12 多模式用户界面·················································································66 4.13 符号输入···························································································66 第 5 章输入系统及运动跟踪 ·········································································68 5.1 输入设备·····························································································68 5.1.1 输入反馈·························································································69 5.1.2 与人有关的问题················································································70 5.1.3 自由度····························································································70 5.1.4 数据手套·························································································71 5.2 运动跟踪·····························································································73 5.2.1 跟踪器属性······················································································73 5.2.2 跟踪技术·························································································73 5.2.3 机械跟踪器······················································································74 5.2.4 电磁跟踪器······················································································74 5.2.5 声学跟踪器······················································································76 5.2.6 光学跟踪器······················································································76 5.2.7 惯性跟踪器······················································································78 5.2.8 消费级产品······················································································78 5.3 运动捕捉·····························································································79 5.4 人脸识别·····························································································80 第 6 章输出系统···························································································82 6.1 输出设备·····························································································82 6.1.1 视频显示器的特性·············································································82 6.1.2 视频显示器的类型·············································································83 6.1.3 显示技术·························································································84 6.2 立体显示器·························································································85 6.2.1 立体显示系统的眼镜··········································································86 6.2.2 HMD······························································································89 6.2.6 CAVE 系统······················································································90 6.2.7 自动立体显示器················································································90 6.2.8 虚拟视网膜显示················································································92 6.3 音频显示·····························································································93 6.4 波场合成·····························································································93 6.5 触觉反馈·····························································································94 6.6 力反馈································································································95 第 7 章虚拟现实系统开发环境······································································97 7.1 5G 概述·······························································································97 7.2 云计算环境·························································································98 7.3 其他技术环境······················································································98 7.4 虚拟现实软件开发环境·····································································.101 7.4.1 输入/输出驱动接口·········································································.101 7.4.2 3D 引擎·······················································································.101 7.4.3 计算机图形程序接口·······································································.103 7.4.4 Web3D·························································································.105 7.5 虚拟现实系统开发的主要问题··························································.108 7.5.1 5G 问题·······················································································.108 7.5.2 用户体验问题················································································.108 7.5.3 眩晕感的影响因素··········································································.109 第8 章虚拟现实系统开发流程与实例 ························································.113 8.1 开发分工··························································································.113 8.1.1 阶段性计划···················································································.113 8.1.2 策划、程序和美术的协同·································································.114 8.2 虚拟现实系统设计············································································.117 8.2.1 虚拟现实的内容区域·······································································.117 8.2.2 虚拟现实的内容尺寸·······································································.119 8.2.3 代入感和舒适性·············································································.120 8.3 漫游系统开发实例············································································.123 8.3.1 搭建开发环境················································································.123 8.3.2 创建项目······················································································.124 8.3.3 设置StreamVR···············································································.124 8.3.4 处理输入······················································································.126 8.3.5 在物理对象上应用手柄····································································.129 8.3.6 制作一只激光笔·············································································.133 8.3.7 移动····························································································.136 8.4 应用类型··························································································.140 8.4.1 VR 游戏·······················································································.140 8.4.2 VR 旅游·······················································································.143 8.4.3 VR 购物·······················································································.144 8.4.4 VR 医疗·······················································································.145 8.4.5 VR 教育·······················································································.149 8.4.6 VR 行业应用·················································································.151 第9 章网络虚拟现实系统··········································································.152 9.1 网络虚拟环境···················································································.152 9.1.1 网络虚拟环境的应用·······································································.153 9.1.2 分布式虚拟环境的问题····································································.153 9.2 架构·································································································.154 9.2.1 无服务器架构················································································.154 9.2.2 单服务器架构················································································.155 9.2.3 多服务器架构················································································.156 9.2.4 协调多服务器架构··········································································.156 9.3 状态同步··························································································.157 9.3.1 共享数据库···················································································.157 9.3.2 状态更新······················································································.158 9.3.3 航位推算······················································································.158 9.4 多用户虚拟现实系统开发概述··························································.159 第 10 章增强现实······················································································.161 10.1 增强现实与虚拟现实·······································································.161 10.1.1 增强现实的历史···········································································.162 10.1.2 现实生活中的应用········································································.164 10.2 视觉增强现实·················································································.164 10.2.1 图像配准····················································································.164 10.2.2 基于HMD 的增强现实系统·····························································.167 10.2.3 基于显示器的增强现实系统与基于投影仪的增强现实系统·····················.169 10.2.4 移动增强现实··············································································.171 10.3 非视觉增强现实··············································································.172 10.4 增强现实系统开发··········································································.174 10.4.1 开发现状····················································································.174 10.4.2 开发步骤····················································································.174 10.4.3 主要开发环境··············································································.176 10.4.4 Unity 3D 与AR Foundation ······························································.177 10.5 增强现实与可穿戴计算···································································.179 附录A 相关词汇························································································.183 参考文献······································································································.189