361 lines
16 KiB
Markdown
361 lines
16 KiB
Markdown
# WebRTC 实现
|
||
|
||
<cite>
|
||
**本文引用的文件**
|
||
- [src/index.ts](file://src/index.ts)
|
||
- [src/signaling.ts](file://src/signaling.ts)
|
||
- [src/class/websockethandler.ts](file://src/class/websockethandler.ts)
|
||
- [src/class/httphandler.ts](file://src/class/httphandler.ts)
|
||
- [client/src/peer.js](file://client/src/peer.js)
|
||
- [client/src/renderstreaming.js](file://client/src/renderstreaming.js)
|
||
- [client/src/signaling.js](file://client/src/signaling.js)
|
||
- [client/src/inputremoting.js](file://client/src/inputremoting.js)
|
||
- [client/src/inputdevice.js](file://client/src/inputdevice.js)
|
||
- [client/src/sender.js](file://client/src/sender.js)
|
||
- [client/src/logger.js](file://client/src/logger.js)
|
||
- [src/log.ts](file://src/log.ts)
|
||
- [package.json](file://package.json)
|
||
- [client/test/peerconnection.test.js](file://client/test/peerconnection.test.js)
|
||
- [client/test/inputremoting.test.js](file://client/test/inputremoting.test.js)
|
||
</cite>
|
||
|
||
## 目录
|
||
1. [简介](#简介)
|
||
2. [项目结构](#项目结构)
|
||
3. [核心组件](#核心组件)
|
||
4. [架构总览](#架构总览)
|
||
5. [详细组件分析](#详细组件分析)
|
||
6. [依赖关系分析](#依赖关系分析)
|
||
7. [性能考虑](#性能考虑)
|
||
8. [故障排查指南](#故障排查指南)
|
||
9. [结论](#结论)
|
||
10. [附录](#附录)
|
||
|
||
## 简介
|
||
本项目是一个基于 WebRTC 的远程渲染与输入控制解决方案，涵盖以下能力：
|
||
- PeerConnection 管理与控制：连接建立、媒体协商、连接状态监控
|
||
- 输入设备远程控制：鼠标、键盘、触摸屏、手柄事件的采集、编码与转发
|
||
- 渲染流处理：媒体流接收、解码与显示
|
||
- 信令客户端：HTTP 轮询与 WebSocket 两种信令协议，消息路由与广播
|
||
- 性能优化与最佳实践：连接稳定性、资源管理、事件驱动架构
|
||
- 测试与调试：单元测试覆盖、日志与调试开关
|
||
|
||
## 项目结构
|
||
后端采用 TypeScript + Express + WebSocket，前端采用原生 JavaScript，核心模块分布如下：
|
||
- 后端入口与配置：启动参数解析、HTTPS/HTTP 选择、WebSocket 信令服务初始化
|
||
- 信令路由：HTTP 轮询接口与 WebSocket 通道
|
||
- 信令处理器：WebSocket 与 HTTP 两类处理器，负责消息分发、连接组管理、广播
|
||
- 前端渲染与信令：RenderStreaming 控制器、PeerConnection 管理、输入远程控制链路
|
||
- 输入系统：设备抽象、事件状态编码、消息打包与发送观察者
|
||
|
||
```mermaid
|
||
graph TB
|
||
subgraph "后端"
|
||
A["Express 应用<br/>src/index.ts"]
|
||
B["HTTP 信令路由<br/>src/signaling.ts"]
|
||
C["WebSocket 处理器<br/>src/class/websockethandler.ts"]
|
||
D["HTTP 处理器<br/>src/class/httphandler.ts"]
|
||
end
|
||
subgraph "前端"
|
||
E["RenderStreaming 控制器<br/>client/src/renderstreaming.js"]
|
||
F["PeerConnection 管理<br/>client/src/peer.js"]
|
||
G["信令客户端(HTTP/WebSocket)<br/>client/src/signaling.js"]
|
||
H["输入远程控制链路<br/>client/src/sender.js + inputremoting.js + inputdevice.js"]
|
||
end
|
||
A --> B
|
||
A --> C
|
||
A --> D
|
||
E --> F
|
||
E --> G
|
||
E --> H
|
||
```
|
||
|
||
**图表来源**
|
||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||
- [src/class/websockethandler.ts:1-479](file://src/class/websockethandler.ts#L1-L479)
|
||
- [src/class/httphandler.ts:1-800](file://src/class/httphandler.ts#L1-L800)
|
||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||
|
||
**章节来源**
|
||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||
- [src/class/websockethandler.ts:1-479](file://src/class/websockethandler.ts#L1-L479)
|
||
- [src/class/httphandler.ts:1-800](file://src/class/httphandler.ts#L1-L800)
|
||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||
|
||
## 核心组件
|
||
- 后端启动与配置：命令行参数解析、HTTPS/HTTP 选择、WebSocket 信令服务启动
|
||
- 信令路由与中间件：HTTP 路由定义、会话校验中间件、消息聚合接口
|
||
- WebSocket 与 HTTP 处理器：连接组管理、消息广播、参与者加入/离开、心跳与超时
|
||
- 前端渲染控制器：连接生命周期管理、Offer/Answer/Candidate 分发、统计与数据通道
|
||
- PeerConnection 管理：SDP 描述协商、ICE 候选收集与注入、事件派发
|
||
- 输入远程控制：设备事件采集、状态编码、消息打包与发送观察者
|
||
- 日志系统：前后端日志级别控制与格式化输出
|
||
|
||
**章节来源**
|
||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||
- [src/signaling.ts:6-24](file://src/signaling.ts#L6-L24)
|
||
- [src/class/websockethandler.ts:10-137](file://src/class/websockethandler.ts#L10-L137)
|
||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||
|
||
## 架构总览
|
||
系统采用“前端渲染 + 后端信令 + 媒体直连”的架构：
|
||
- 前端通过 HTTP 或 WebSocket 与后端进行信令交互，完成连接建立与媒体协商
|
||
- 媒体流在浏览器间直接传输，信令仅传递 SDP 与 ICE 候选
|
||
- 输入事件在前端采集后编码并通过 RTCDataChannel 或自定义消息通道发送至远端
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Host as "主机(Host)"
|
||
participant Server as "信令服务器"
|
||
participant Participant as "参与者(Participant)"
|
||
Host->>Server : "创建会话/连接"
|
||
Note over Host,Server : "HTTP/WebSocket 信令握手"
|
||
Host->>Server : "发送 Offer"
|
||
Server-->>Participant : "转发 Offer"
|
||
Participant->>Server : "发送 Answer"
|
||
Server-->>Host : "转发 Answer"
|
||
Host->>Server : "发送 ICE Candidate"
|
||
Server-->>Participant : "转发 Candidate"
|
||
Participant->>Server : "发送 ICE Candidate"
|
||
Server-->>Host : "转发 Candidate"
|
||
Note over Host,Participant : "媒体流直连传输"
|
||
```
|
||
|
||
**图表来源**
|
||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||
- [client/src/renderstreaming.js:72-147](file://client/src/renderstreaming.js#L72-L147)
|
||
|
||
**章节来源**
|
||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||
- [client/src/renderstreaming.js:72-147](file://client/src/renderstreaming.js#L72-L147)
|
||
|
||
## 详细组件分析
|
||
|
||
### PeerConnection 管理与控制
|
||
- 事件绑定：track、datachannel、icecandidate、negotiationneeded、signalingstatechange、iceconnectionstatechange、icegatheringstatechange
|
||
- 协商流程：在 negotiationneeded 触发时设置本地描述并派发 offer；收到远端 offer 后设置远端描述并生成 answer
|
||
- ICE 候选：收集到候选时派发候选事件，等待远端注入
|
||
- 状态监控：失败时派发 disconnect 事件，便于上层重连或降级
|
||
- 统计与控制：支持 getStats、addTrack/addTransceiver、createDataChannel 等
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant Peer as "Peer(前端)"
|
||
participant Signaling as "信令客户端"
|
||
participant Remote as "远端Peer"
|
||
Peer->>Peer : "negotiationneeded 触发"
|
||
Peer->>Peer : "setLocalDescription()"
|
||
Peer-->>Signaling : "派发 sendoffer"
|
||
Signaling-->>Remote : "发送 Offer"
|
||
Remote->>Remote : "setRemoteDescription(Offer)"
|
||
Remote->>Remote : "setLocalDescription(Answer)"
|
||
Remote-->>Signaling : "派发 sendanswer"
|
||
Signaling-->>Peer : "发送 Answer"
|
||
Peer->>Peer : "setRemoteDescription(Answer)"
|
||
Peer-->>Peer : "触发 negotiated 事件"
|
||
```
|
||
|
||
**图表来源**
|
||
- [client/src/peer.js:37-173](file://client/src/peer.js#L37-L173)
|
||
- [client/src/renderstreaming.js:72-130](file://client/src/renderstreaming.js#L72-L130)
|
||
|
||
**章节来源**
|
||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||
- [client/src/renderstreaming.js:72-130](file://client/src/renderstreaming.js#L72-L130)
|
||
- [client/test/peerconnection.test.js:1-251](file://client/test/peerconnection.test.js#L1-L251)
|
||
|
||
### 输入设备远程控制
|
||
- 设备抽象：Mouse、Keyboard、Touchscreen、Gamepad 四类设备，统一状态接口
|
||
- 事件采集：鼠标/键盘/触摸/手柄事件监听与状态更新
|
||
- 状态编码：各设备状态转为二进制缓冲区，包含格式标识、尺寸、数据
|
||
- 消息打包：StateEvent/TextEvent 包装基础事件与状态数据
|
||
- 发送链路：InputRemoting 订阅观察者，将消息序列化后通过 RTCDataChannel 或自定义通道发送
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Start(["用户输入事件"]) --> Queue["设备队列更新状态"]
|
||
Queue --> Encode["编码为 IInputState 缓冲区"]
|
||
Encode --> Wrap["包装为 StateEvent/TextEvent"]
|
||
Wrap --> Pack["封装为 Message(含 participant_id/type/length/data)"]
|
||
Pack --> Send["通过观察者发送(如 RTCDataChannel)"]
|
||
Send --> End(["远端接收并应用"])
|
||
```
|
||
|
||
**图表来源**
|
||
- [client/src/inputdevice.js:91-719](file://client/src/inputdevice.js#L91-L719)
|
||
- [client/src/inputremoting.js:63-300](file://client/src/inputremoting.js#L63-L300)
|
||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||
|
||
**章节来源**
|
||
- [client/src/inputdevice.js:91-719](file://client/src/inputdevice.js#L91-L719)
|
||
- [client/src/inputremoting.js:63-300](file://client/src/inputremoting.js#L63-L300)
|
||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||
- [client/test/inputremoting.test.js:1-132](file://client/test/inputremoting.test.js#L1-L132)
|
||
|
||
### 渲染流处理
|
||
- 渲染控制器：根据角色(host/participant)创建/复用 Peer，订阅 trackevent/adddatachannel/onmessage 等事件
|
||
- 媒体流接收：trackevent 中获取远端媒体轨道，绑定到视频元素进行播放
|
||
- 数据通道：adddatachannel 事件用于建立输入控制通道或其他应用通道
|
||
- 统计与诊断：getStats 接口用于网络质量与传输指标观测
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant RS as "RenderStreaming"
|
||
participant Peer as "Peer"
|
||
participant Video as "视频元素"
|
||
RS->>Peer : "创建/复用 Peer 并绑定事件"
|
||
Peer-->>RS : "trackevent(媒体轨道)"
|
||
RS->>Video : "设置 srcObject 为媒体流"
|
||
Peer-->>RS : "adddatachannel(数据通道)"
|
||
RS->>RS : "getStats() 收集统计信息"
|
||
```
|
||
|
||
**图表来源**
|
||
- [client/src/renderstreaming.js:212-250](file://client/src/renderstreaming.js#L212-L250)
|
||
- [client/src/peer.js:21-28](file://client/src/peer.js#L21-L28)
|
||
|
||
**章节来源**
|
||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||
- [client/src/peer.js:21-28](file://client/src/peer.js#L21-L28)
|
||
|
||
### 信令客户端实现
|
||
- HTTP 轮询：定时拉取消息，解析类型并派发事件
|
||
- WebSocket：长连接，消息类型丰富，支持广播与参与者路由
|
||
- 会话管理：创建/删除会话、连接管理、消息聚合
|
||
- 参与者模型：私有模式下 host 与多个 participants，支持按 participantId 路由
|
||
|
||
```mermaid
|
||
classDiagram
|
||
class Signaling {
|
||
+start()
|
||
+stop()
|
||
+createConnection()
|
||
+deleteConnection()
|
||
+sendOffer()
|
||
+sendAnswer()
|
||
+sendCandidate()
|
||
+getAll()
|
||
}
|
||
class WebSocketSignaling {
|
||
+start()
|
||
+stop()
|
||
+createConnection()
|
||
+deleteConnection()
|
||
+sendOffer()
|
||
+sendAnswer()
|
||
+sendCandidate()
|
||
+sendMessage()
|
||
}
|
||
Signaling <|-- WebSocketSignaling
|
||
```
|
||
|
||
**图表来源**
|
||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||
|
||
**章节来源**
|
||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||
|
||
## 依赖关系分析
|
||
- 后端依赖：Express、ws、uuid、swagger-ui-express 等
|
||
- 前端依赖：浏览器原生 WebRTC API、RTCDataChannel、EventTarget、Fetch/WebSocket
|
||
- 日志：前后端独立日志模块，支持级别控制与格式化输出
|
||
|
||
```mermaid
|
||
graph LR
|
||
Pkg["package.json 依赖声明"] --> Express["express"]
|
||
Pkg --> WS["ws"]
|
||
Pkg --> UUID["uuid"]
|
||
Pkg --> Swagger["swagger-ui-express"]
|
||
Front["前端模块"] --> WebRTC["WebRTC API"]
|
||
Front --> FetchWS["Fetch/WebSocket"]
|
||
LogB["后端日志(src/log.ts)"] --> ConsoleB["控制台输出"]
|
||
LogF["前端日志(client/src/logger.js)"] --> ConsoleF["控制台输出"]
|
||
```
|
||
|
||
**图表来源**
|
||
- [package.json:14-46](file://package.json#L14-L46)
|
||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||
|
||
**章节来源**
|
||
- [package.json:14-46](file://package.json#L14-L46)
|
||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||
|
||
## 性能考虑
|
||
- 连接稳定性
|
||
- 使用轮询重发 Offer 以应对网络抖动，避免长时间无响应
|
||
- ICE 候选收集完成后及时注入，减少连接建立延迟
|
||
- 媒体协商
|
||
- 优先使用匹配编解码器，减少转码开销
|
||
- 合理设置带宽限制与回退策略
|
||
- 输入事件
|
||
- 状态事件合并发送，降低消息频率
|
||
- 使用二进制消息格式，减少序列化成本
|
||
- 信令
|
||
- WebSocket 模式优于 HTTP 轮询，降低延迟与 CPU 开销
|
||
- 私有模式下按 participantId 路由，避免广播风暴
|
||
- 资源管理
|
||
- 及时关闭 PeerConnection 与数据通道
|
||
- 监控统计指标，动态调整分辨率与帧率
|
||
|
||
[本节为通用指导，无需具体文件分析]
|
||
|
||
## 故障排查指南
|
||
- 日志启用
|
||
- 前端：调用日志模块的 enable/disable 控制输出
|
||
- 后端：通过命令行参数设置日志级别
|
||
- 常见问题定位
|
||
- 连接失败：检查 signaling 事件派发与路由，确认 Offer/Answer/Candidate 是否正确传递
|
||
- ICE 失败：查看 iceconnectionstatechange/disconnect 事件，确认候选注入与防火墙/NAT 配置
|
||
- 输入无响应：确认 RTCDataChannel 已打开，消息格式与 participantId 正确
|
||
- 单元测试
|
||
- PeerConnection 行为验证：添加/移除轨道、发送 Offer/Answer、ICE 候选注入
|
||
- 输入系统：设备消息打包、事件编码、观察者发送
|
||
|
||
**章节来源**
|
||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||
- [client/test/peerconnection.test.js:1-251](file://client/test/peerconnection.test.js#L1-L251)
|
||
- [client/test/inputremoting.test.js:1-132](file://client/test/inputremoting.test.js#L1-L132)
|
||
|
||
## 结论
|
||
本实现提供了完整的 WebRTC 远程渲染与输入控制方案，具备清晰的模块划分与可扩展性。通过事件驱动与观察者模式，前端渲染控制器与输入系统能够高效协作；后端信令处理器支持私有与公共两种模式，满足不同场景需求。配合完善的日志与测试体系，便于维护与优化。
|
||
|
||
[本节为总结，无需具体文件分析]
|
||
|
||
## 附录
|
||
- 启动参数与脚本
|
||
- 开发模式：ts-node 启动，支持 HTTPS/HTTP 与私有/公共模式切换
|
||
- 生产模式：构建后运行，支持证书文件与端口配置
|
||
- API 与消息类型
|
||
- HTTP 轮询：/signaling/* 接口，支持会话管理与消息聚合
|
||
- WebSocket：connect/disconnect/offer/answer/candidate/on-message/participant-joined/participant-left/broadcast 等
|
||
|
||
**章节来源**
|
||
- [src/index.ts:16-82](file://src/index.ts#L16-L82)
|
||
- [package.json:5-12](file://package.json#L5-L12)
|
||
- [src/signaling.ts:6-24](file://src/signaling.ts#L6-L24)
|
||
- [src/class/websockethandler.ts:145-473](file://src/class/websockethandler.ts#L145-L473) |