414 lines
19 KiB
Markdown
414 lines
19 KiB
Markdown
# 客户端示例
|
||
|
||
<cite>
|
||
**本文档引用的文件**
|
||
- [client/public/index.html](file://client/public/index.html)
|
||
- [client/public/bidirectional/index.html](file://client/public/bidirectional/index.html)
|
||
- [client/public/bidirectional/js/main.js](file://client/public/bidirectional/js/main.js)
|
||
- [client/public/bidirectional/js/sendvideo.js](file://client/public/bidirectional/js/sendvideo.js)
|
||
- [client/public/receiver/index.html](file://client/public/receiver/index.html)
|
||
- [client/public/receiver/js/main.js](file://client/public/receiver/js/main.js)
|
||
- [client/public/multiplay/index.html](file://client/public/multiplay/index.html)
|
||
- [client/public/multiplay/js/main.js](file://client/public/multiplay/js/main.js)
|
||
- [client/public/onebyone/index.html](file://client/public/onebyone/index.html)
|
||
- [client/public/onebyone/main.js](file://client/public/onebyone/main.js)
|
||
- [client/public/onebyone/store.js](file://client/public/onebyone/store.js)
|
||
- [client/public/onebyone/renderer.js](file://client/public/onebyone/renderer.js)
|
||
- [client/public/onebyone/chatmessage.js](file://client/public/onebyone/chatmessage.js)
|
||
- [client/public/onebyone/utils.js](file://client/public/onebyone/utils.js)
|
||
- [client/public/videoplayer/index.html](file://client/public/videoplayer/index.html)
|
||
- [client/public/videoplayer/js/main.js](file://client/public/videoplayer/js/main.js)
|
||
- [client/public/videoplayer/js/video-player.js](file://client/public/videoplayer/js/video-player.js)
|
||
- [client/public/videoplayer/js/register-events.js](file://client/public/videoplayer/js/register-events.js)
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [client/public/js/stats.js](file://client/public/js/stats.js)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
- [client/src/sender.js](file://client/src/sender.js)
|
||
- [client/src/signaling.js](file://client/src/signaling.js)
|
||
- [client/src/peer.js](file://client/src/peer.js)
|
||
</cite>
|
||
|
||
## 目录
|
||
1. [简介](#简介)
|
||
2. [项目结构](#项目结构)
|
||
3. [核心组件](#核心组件)
|
||
4. [架构总览](#架构总览)
|
||
5. [详细组件分析](#详细组件分析)
|
||
6. [依赖关系分析](#依赖关系分析)
|
||
7. [性能考量](#性能考量)
|
||
8. [故障排查指南](#故障排查指南)
|
||
9. [结论](#结论)
|
||
10. [附录](#附录)
|
||
|
||
## 简介
|
||
本仓库提供一套基于 WebRTC 的客户端示例系统，涵盖以下典型场景：
|
||
- 双向视频通信：本地采集音视频，同时接收远端流，适合需要双向互动的场景。
|
||
- 单向视频接收（广播）：仅接收来自服务器的渲染流，适合“观看者”场景。
|
||
- 多客户端播放（Guest 模式）：多个客户端可共同观看同一渲染流。
|
||
- 视频播放器控制（WebBrowserInput）：在浏览器中接收 Unity 渲染的摄像头画面，并通过数据通道将输入事件回传给服务器。
|
||
|
||
这些示例统一采用模块化架构，通过信令层协调连接建立、媒体轨道处理与 UI 渲染，便于集成到自有项目中作为参考或基线。
|
||
|
||
## 项目结构
|
||
客户端示例主要分为两部分：
|
||
- public 目录：各示例页面及其资源（HTML/CSS/JS），每个示例独立运行。
|
||
- src 目录：通用的输入遥测、信令与 Peer 连接封装，供示例复用。
|
||
|
||
```mermaid
|
||
graph TB
|
||
subgraph "示例页面"
|
||
BI["双向示例<br/>bidirectional/index.html"]
|
||
RCV["接收示例<br/>receiver/index.html"]
|
||
MP["多播放示例<br/>multiplay/index.html"]
|
||
VP["播放器示例<br/>videoplayer/index.html"]
|
||
OB1["一对一通话示例<br/>onebyone/index.html"]
|
||
end
|
||
subgraph "公共脚本"
|
||
CFG["配置与ICE<br/>js/config.js, js/icesettings.js"]
|
||
STATS["统计信息<br/>js/stats.js"]
|
||
VID["视频播放器抽象<br/>js/videoplayer.js"]
|
||
end
|
||
subgraph "核心模块"
|
||
SIG["信令层<br/>src/signaling.js"]
|
||
PEER["Peer连接封装<br/>src/peer.js"]
|
||
SENDER["输入遥测发送器<br/>src/sender.js"]
|
||
end
|
||
BI --- CFG
|
||
RCV --- CFG
|
||
MP --- CFG
|
||
VP --- CFG
|
||
OB1 --- CFG
|
||
BI --- SIG
|
||
RCV --- SIG
|
||
MP --- SIG
|
||
VP --- SIG
|
||
OB1 --- SIG
|
||
BI --- PEER
|
||
RCV --- PEER
|
||
MP --- PEER
|
||
VP --- PEER
|
||
OB1 --- PEER
|
||
VP --- SENDER
|
||
VP --- VID
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/bidirectional/index.html:1-84](file://client/public/bidirectional/index.html#L1-L84)
|
||
- [client/public/receiver/index.html:1-54](file://client/public/receiver/index.html#L1-L54)
|
||
- [client/public/multiplay/index.html:1-54](file://client/public/multiplay/index.html#L1-L54)
|
||
- [client/public/videoplayer/index.html:1-38](file://client/public/videoplayer/index.html#L1-L38)
|
||
- [client/public/onebyone/index.html:1-615](file://client/public/onebyone/index.html#L1-L615)
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [client/public/js/stats.js](file://client/public/js/stats.js)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/sender.js:1-209](file://client/src/sender.js#L1-L209)
|
||
|
||
章节来源
|
||
- [client/public/index.html:1-78](file://client/public/index.html#L1-L78)
|
||
|
||
## 核心组件
|
||
- 配置与 ICE 管理：负责读取服务器配置、选择信令协议（HTTP 或 WebSocket）、管理 STUN/TURN 列表。
|
||
- 信令层：封装 HTTP 与 WebSocket 两种信令方式，统一发布/订阅事件，屏蔽连接细节。
|
||
- Peer 封装：对 RTCPeerConnection 的封装，处理 offer/answer、ICE 候选、轨道事件与统计。
|
||
- 输入遥测：将鼠标、键盘、手柄、触摸等输入事件序列化并通过数据通道发送至服务器。
|
||
- 视频播放器抽象：统一视频元素创建、缩放、全屏、输入事件注册等行为。
|
||
|
||
章节来源
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/sender.js:1-209](file://client/src/sender.js#L1-L209)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
|
||
## 架构总览
|
||
所有示例均遵循相同的控制流：初始化配置 → 选择信令 → 建立连接 → 添加媒体轨道 → 渲染远端流 → 统计与错误处理。
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant UI as "示例页面"
|
||
participant CFG as "配置/ICE"
|
||
participant SIG as "信令层(HTTP/WebSocket)"
|
||
participant PEER as "Peer封装"
|
||
participant REM as "远端Peer"
|
||
UI->>CFG : 读取服务器配置/ICE服务器
|
||
UI->>SIG : start()/连接信令
|
||
UI->>PEER : start()/createConnection(id)
|
||
PEER->>SIG : 发布offer/answer/candidate
|
||
SIG-->>PEER : 下发offer/answer/candidate
|
||
PEER->>PEER : setLocal/RemoteDescription
|
||
PEER->>PEER : addIceCandidate
|
||
PEER-->>UI : onTrack/onDataChannel
|
||
UI->>UI : 渲染远端流/注册输入事件
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/bidirectional/js/main.js:146-184](file://client/public/bidirectional/js/main.js#L146-L184)
|
||
- [client/public/receiver/js/main.js:75-88](file://client/public/receiver/js/main.js#L75-L88)
|
||
- [client/public/multiplay/js/main.js:82-95](file://client/public/multiplay/js/main.js#L82-L95)
|
||
- [client/src/signaling.js:30-97](file://client/src/signaling.js#L30-L97)
|
||
- [client/src/peer.js:57-82](file://client/src/peer.js#L57-L82)
|
||
|
||
## 详细组件分析
|
||
|
||
### 双向视频通信（Bidirectional）
|
||
- 功能要点
|
||
- 本地采集音视频，选择设备与分辨率，支持自定义宽高。
|
||
- 建立 WebRTC 连接，添加本地轨道为 sendonly，接收远端轨道并渲染。
|
||
- 支持编解码器偏好设置，按秒输出统计信息，显示本地/远端分辨率。
|
||
- 提示：Public 模式不工作，需在 Private 模式下运行。
|
||
- 关键流程
|
||
- 设备枚举与分辨率选择
|
||
- 连接建立与轨道添加
|
||
- 编解码器偏好设置
|
||
- 统计信息轮询与展示
|
||
- UI 交互
|
||
- 开始采集 → 设置连接 → 挂断清理
|
||
- 编解码器选择禁用/启用
|
||
- 本地/远端视频统计显示
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant UI as "双向页面"
|
||
participant SV as "SendVideo"
|
||
participant RS as "RenderStreaming"
|
||
participant SIG as "信令(HTTP/WebSocket)"
|
||
participant PC as "Peer封装"
|
||
UI->>SV : startLocalVideo(设备, 分辨率)
|
||
UI->>RS : start()/createConnection(id)
|
||
RS->>PC : onConnect回调
|
||
PC->>PC : addTransceiver(本地轨道, direction='sendonly')
|
||
PC->>SIG : 发布offer/answer/candidate
|
||
SIG-->>PC : 下发offer/answer/candidate
|
||
PC-->>UI : onTrack(远端轨道)
|
||
UI->>SV : addRemoteTrack(渲染)
|
||
UI->>PC : setCodecPreferences(可选)
|
||
UI->>UI : 每秒统计展示
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/bidirectional/js/main.js:112-184](file://client/public/bidirectional/js/main.js#L112-L184)
|
||
- [client/public/bidirectional/js/sendvideo.js](file://client/public/bidirectional/js/sendvideo.js)
|
||
- [client/src/signaling.js:152-292](file://client/src/signaling.js#L152-L292)
|
||
- [client/src/peer.js:108-130](file://client/src/peer.js#L108-L130)
|
||
|
||
章节来源
|
||
- [client/public/bidirectional/index.html:1-84](file://client/public/bidirectional/index.html#L1-L84)
|
||
- [client/public/bidirectional/js/main.js:1-383](file://client/public/bidirectional/js/main.js#L1-L383)
|
||
|
||
### 单向视频接收（Receiver）
|
||
- 功能要点
|
||
- 点击播放按钮后创建视频播放器，建立连接并接收渲染流。
|
||
- 支持编解码器偏好设置，提供光标锁定到播放器区域的选项。
|
||
- 断开连接时清理播放器与统计信息，恢复播放按钮。
|
||
- 关键流程
|
||
- 点击播放 → 创建播放器 → 建立连接 → onTrack 添加轨道 → 输入通道设置
|
||
- 断开时清理并重置 UI
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant UI as "接收页面"
|
||
participant VP as "VideoPlayer"
|
||
participant RS as "RenderStreaming"
|
||
participant SIG as "信令(HTTP/WebSocket)"
|
||
UI->>VP : createPlayer(容器, 锁定选项)
|
||
UI->>RS : start()/createConnection()
|
||
RS->>SIG : 发布/订阅信令
|
||
SIG-->>RS : offer/answer/candidate
|
||
RS-->>UI : onTrack -> VP.addTrack
|
||
UI->>VP : setupInput(数据通道)
|
||
UI->>UI : 统计展示/断开清理
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/receiver/js/main.js:67-108](file://client/public/receiver/js/main.js#L67-L108)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
- [client/src/signaling.js:30-97](file://client/src/signaling.js#L30-L97)
|
||
|
||
章节来源
|
||
- [client/public/receiver/index.html:1-54](file://client/public/receiver/index.html#L1-L54)
|
||
- [client/public/receiver/js/main.js:1-186](file://client/public/receiver/js/main.js#L1-L186)
|
||
|
||
### 多客户端播放（Multiplay）
|
||
- 功能要点
|
||
- 与接收示例类似，但额外创建名为 "multiplay" 的数据通道，用于多客户端标签同步等场景。
|
||
- 打开通道后延时发送一条包含类型与参数的消息，演示数据通道的使用。
|
||
- 关键流程
|
||
- 建立连接 → 创建 "input" 与 "multiplay" 数据通道 → onOpenMultiplayChannel 发送消息
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant UI as "多播放页面"
|
||
participant VP as "VideoPlayer"
|
||
participant RS as "RenderStreaming"
|
||
participant DC as "数据通道(input/multiplay)"
|
||
UI->>VP : createPlayer(容器, 锁定选项)
|
||
UI->>RS : start()/createConnection()
|
||
RS-->>UI : onConnect
|
||
UI->>DC : createDataChannel("input")
|
||
UI->>DC : createDataChannel("multiplay")
|
||
DC-->>UI : onopen
|
||
UI->>DC : send({type, argument})
|
||
UI->>UI : 统计展示/断开清理
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/multiplay/js/main.js:97-125](file://client/public/multiplay/js/main.js#L97-L125)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
- [client/src/signaling.js:243-290](file://client/src/signaling.js#L243-L290)
|
||
|
||
章节来源
|
||
- [client/public/multiplay/index.html:1-54](file://client/public/multiplay/index.html#L1-L54)
|
||
- [client/public/multiplay/js/main.js:1-204](file://client/public/multiplay/js/main.js#L1-L204)
|
||
|
||
### 视频播放器控制（WebBrowserInput）
|
||
- 功能要点
|
||
- 点击播放后创建主视频与缩略图元素，注册游戏手柄、键盘、鼠标事件。
|
||
- 通过数据通道将点击事件与输入事件发送至服务器，实现“在浏览器中操作 Unity 摄像头”的效果。
|
||
- 支持全屏切换与自动尺寸调整。
|
||
- 关键流程
|
||
- 点击播放 → 创建元素 → setupConnection → 注册事件 → 发送输入事件
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant UI as "播放器页面"
|
||
participant VP as "VideoPlayer"
|
||
participant SE as "输入遥测(sender)"
|
||
participant SIG as "信令(HTTP/WebSocket)"
|
||
UI->>VP : setupConnection(使用WebSocket?)
|
||
UI->>SE : registerGamepad/Keyboard/Mouse
|
||
SE->>SIG : 通过数据通道发送事件
|
||
UI->>UI : 全屏切换/尺寸调整
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/videoplayer/js/main.js:49-142](file://client/public/videoplayer/js/main.js#L49-L142)
|
||
- [client/public/videoplayer/js/video-player.js](file://client/public/videoplayer/js/video-player.js)
|
||
- [client/public/videoplayer/js/register-events.js](file://client/public/videoplayer/js/register-events.js)
|
||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||
|
||
章节来源
|
||
- [client/public/videoplayer/index.html:1-38](file://client/public/videoplayer/index.html#L1-L38)
|
||
- [client/public/videoplayer/js/main.js:1-157](file://client/public/videoplayer/js/main.js#L1-L157)
|
||
|
||
### 一对一视频通话（One-by-one）
|
||
- 功能要点
|
||
- 使用模块化架构：store 管理状态，renderer 负责 UI 渲染，chatmessage 管理聊天，utils 提供工具函数。
|
||
- 支持媒体状态切换（音频/视频/录制）、分辨率切换、侧边栏与通知、通话请求弹窗等。
|
||
- 通过 API 与 WebSocket 事件驱动 UI 更新与业务流程。
|
||
- 关键流程
|
||
- 页面加载 → 读取本地连接ID → joinCall → setUp → 绑定事件 → 渲染头部标题
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Page as "一对一页面"
|
||
participant Store as "状态管理(store)"
|
||
participant Renderer as "UI渲染(renderer)"
|
||
participant Chat as "聊天(chatmessage)"
|
||
participant Utils as "工具(utils)"
|
||
Page->>Store : joinCall(connectionId)
|
||
Page->>Store : setUp(connectionId)
|
||
Page->>Renderer : renderHeaderTitle()
|
||
Page->>Page : 绑定DOM事件/快捷键
|
||
Page->>Chat : bindMessageEvents()
|
||
Page->>Utils : showNotification(...)
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/onebyone/main.js:179-212](file://client/public/onebyone/main.js#L179-L212)
|
||
- [client/public/onebyone/store.js](file://client/public/onebyone/store.js)
|
||
- [client/public/onebyone/renderer.js](file://client/public/onebyone/renderer.js)
|
||
- [client/public/onebyone/chatmessage.js](file://client/public/onebyone/chatmessage.js)
|
||
- [client/public/onebyone/utils.js](file://client/public/onebyone/utils.js)
|
||
|
||
章节来源
|
||
- [client/public/onebyone/index.html:1-615](file://client/public/onebyone/index.html#L1-L615)
|
||
- [client/public/onebyone/main.js:1-216](file://client/public/onebyone/main.js#L1-L216)
|
||
|
||
## 依赖关系分析
|
||
- 示例页面依赖公共配置与 ICE 设置，以及核心模块（信令、Peer、输入遥测、视频播放器抽象）。
|
||
- 一对一通话示例采用分层模块（store/renderer/chat/utils），降低耦合度，提升可维护性。
|
||
- 信令层同时支持 HTTP 与 WebSocket，通过统一事件接口屏蔽差异。
|
||
|
||
```mermaid
|
||
graph LR
|
||
BI["双向示例"] --> CFG["配置/ICE"]
|
||
RCV["接收示例"] --> CFG
|
||
MP["多播放示例"] --> CFG
|
||
VP["播放器示例"] --> CFG
|
||
OB1["一对一示例"] --> CFG
|
||
BI --> SIG["信令(HTTP/WebSocket)"]
|
||
RCV --> SIG
|
||
MP --> SIG
|
||
VP --> SIG
|
||
OB1 --> SIG
|
||
BI --> PEER["Peer封装"]
|
||
RCV --> PEER
|
||
MP --> PEER
|
||
VP --> PEER
|
||
OB1 --> PEER
|
||
VP --> SENDER["输入遥测(sender)"]
|
||
VP --> VID["视频播放器抽象"]
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/sender.js:1-209](file://client/src/sender.js#L1-L209)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
|
||
章节来源
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/sender.js:1-209](file://client/src/sender.js#L1-L209)
|
||
|
||
## 性能考量
|
||
- 分辨率与编解码器
|
||
- 示例提供多种预设分辨率，支持自定义分辨率；可通过编解码器偏好减少带宽占用。
|
||
- 建议根据网络状况动态调整分辨率与编解码器，避免过高码率导致卡顿。
|
||
- 统计信息
|
||
- 每秒轮询统计信息，可用于监控丢包、延迟与分辨率变化，便于前端提示与自动调节。
|
||
- 输入事件
|
||
- 输入遥测通过数据通道发送，建议合并事件与节流发送，降低信令压力。
|
||
- 全屏与缩放
|
||
- 全屏切换与窗口 resize 时及时调整视频尺寸，避免重绘开销过大。
|
||
|
||
## 故障排查指南
|
||
- Public/Private 模式限制
|
||
- 双向示例在 Public 模式下不工作；接收/多播放示例在 Private 模式下不工作。请根据示例页面警告提示切换模式。
|
||
- 信令协议选择
|
||
- 若使用 WebSocket 信令，请确保服务端已正确配置；若使用 HTTP 信令，请确认会话创建与轮询接口可用。
|
||
- ICE 服务器
|
||
- 若无法建立连接，检查 STUN/TURN 服务器配置与网络策略；可在页面 ICE 配置区添加/移除/重置服务器。
|
||
- 编解码器不支持
|
||
- 部分浏览器不支持设置编解码器偏好；若出现相关提示，请降级处理或更换浏览器。
|
||
- 断线与重连
|
||
- ICE 失败会触发断开事件；示例会在断开时清理资源并提示消息。请检查网络稳定性与防火墙设置。
|
||
|
||
章节来源
|
||
- [client/public/bidirectional/js/main.js:99-105](file://client/public/bidirectional/js/main.js#L99-L105)
|
||
- [client/public/receiver/js/main.js:48-54](file://client/public/receiver/js/main.js#L48-L54)
|
||
- [client/public/multiplay/js/main.js:55-61](file://client/public/multiplay/js/main.js#L55-L61)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [client/src/signaling.js:30-97](file://client/src/signaling.js#L30-L97)
|
||
|
||
## 结论
|
||
该客户端示例系统提供了从基础接收、双向通信到复杂输入遥测与 UI 管理的完整参考。其模块化设计与统一的信令抽象，使得开发者可以快速将示例中的关键模式（如连接建立、媒体轨道处理、数据通道输入、UI 渲染与状态管理）集成到自有项目中，并根据业务需求扩展功能。
|
||
|
||
## 附录
|
||
- 快速开始
|
||
- 在浏览器中打开示例页面，根据页面提示选择模式（Public/Private）。
|
||
- 如需使用 TURN，请在 ICE 配置区添加 STUN/TURN 服务器。
|
||
- 对于需要输入控制的场景，确保数据通道可用且已注册相应事件。
|
||
- 集成建议
|
||
- 将公共配置与 ICE 设置抽取为独立模块，便于跨示例共享。
|
||
- 将信令层抽象为统一接口，支持 HTTP 与 WebSocket 两种实现。
|
||
- 将 Peer 封装与视频播放器抽象作为通用组件，按需组合到不同示例中。
|
||
- 对于复杂 UI（如一对一通话），采用分层模块（store/renderer/chat/utils）组织代码，提升可维护性。 |