11
This commit is contained in:
361
.qoder/repowiki/zh/content/WebRTC 实现/WebRTC 实现.md
Normal file
361
.qoder/repowiki/zh/content/WebRTC 实现/WebRTC 实现.md
Normal file
@@ -0,0 +1,361 @@
|
||||
# WebRTC 实现
|
||||
|
||||
<cite>
|
||||
**本文引用的文件**
|
||||
- [src/index.ts](file://src/index.ts)
|
||||
- [src/signaling.ts](file://src/signaling.ts)
|
||||
- [src/class/websockethandler.ts](file://src/class/websockethandler.ts)
|
||||
- [src/class/httphandler.ts](file://src/class/httphandler.ts)
|
||||
- [client/src/peer.js](file://client/src/peer.js)
|
||||
- [client/src/renderstreaming.js](file://client/src/renderstreaming.js)
|
||||
- [client/src/signaling.js](file://client/src/signaling.js)
|
||||
- [client/src/inputremoting.js](file://client/src/inputremoting.js)
|
||||
- [client/src/inputdevice.js](file://client/src/inputdevice.js)
|
||||
- [client/src/sender.js](file://client/src/sender.js)
|
||||
- [client/src/logger.js](file://client/src/logger.js)
|
||||
- [src/log.ts](file://src/log.ts)
|
||||
- [package.json](file://package.json)
|
||||
- [client/test/peerconnection.test.js](file://client/test/peerconnection.test.js)
|
||||
- [client/test/inputremoting.test.js](file://client/test/inputremoting.test.js)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [简介](#简介)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考虑](#性能考虑)
|
||||
8. [故障排查指南](#故障排查指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 简介
|
||||
本项目是一个基于 WebRTC 的远程渲染与输入控制解决方案,涵盖以下能力:
|
||||
- PeerConnection 管理与控制:连接建立、媒体协商、连接状态监控
|
||||
- 输入设备远程控制:鼠标、键盘、触摸屏、手柄事件的采集、编码与转发
|
||||
- 渲染流处理:媒体流接收、解码与显示
|
||||
- 信令客户端:HTTP 轮询与 WebSocket 两种信令协议,消息路由与广播
|
||||
- 性能优化与最佳实践:连接稳定性、资源管理、事件驱动架构
|
||||
- 测试与调试:单元测试覆盖、日志与调试开关
|
||||
|
||||
## 项目结构
|
||||
后端采用 TypeScript + Express + WebSocket,前端采用原生 JavaScript,核心模块分布如下:
|
||||
- 后端入口与配置:启动参数解析、HTTPS/HTTP 选择、WebSocket 信令服务初始化
|
||||
- 信令路由:HTTP 轮询接口与 WebSocket 通道
|
||||
- 信令处理器:WebSocket 与 HTTP 两类处理器,负责消息分发、连接组管理、广播
|
||||
- 前端渲染与信令:RenderStreaming 控制器、PeerConnection 管理、输入远程控制链路
|
||||
- 输入系统:设备抽象、事件状态编码、消息打包与发送观察者
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "后端"
|
||||
A["Express 应用<br/>src/index.ts"]
|
||||
B["HTTP 信令路由<br/>src/signaling.ts"]
|
||||
C["WebSocket 处理器<br/>src/class/websockethandler.ts"]
|
||||
D["HTTP 处理器<br/>src/class/httphandler.ts"]
|
||||
end
|
||||
subgraph "前端"
|
||||
E["RenderStreaming 控制器<br/>client/src/renderstreaming.js"]
|
||||
F["PeerConnection 管理<br/>client/src/peer.js"]
|
||||
G["信令客户端(HTTP/WebSocket)<br/>client/src/signaling.js"]
|
||||
H["输入远程控制链路<br/>client/src/sender.js + inputremoting.js + inputdevice.js"]
|
||||
end
|
||||
A --> B
|
||||
A --> C
|
||||
A --> D
|
||||
E --> F
|
||||
E --> G
|
||||
E --> H
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||||
- [src/class/websockethandler.ts:1-479](file://src/class/websockethandler.ts#L1-L479)
|
||||
- [src/class/httphandler.ts:1-800](file://src/class/httphandler.ts#L1-L800)
|
||||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||||
- [src/class/websockethandler.ts:1-479](file://src/class/websockethandler.ts#L1-L479)
|
||||
- [src/class/httphandler.ts:1-800](file://src/class/httphandler.ts#L1-L800)
|
||||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||||
|
||||
## 核心组件
|
||||
- 后端启动与配置:命令行参数解析、HTTPS/HTTP 选择、WebSocket 信令服务启动
|
||||
- 信令路由与中间件:HTTP 路由定义、会话校验中间件、消息聚合接口
|
||||
- WebSocket 与 HTTP 处理器:连接组管理、消息广播、参与者加入/离开、心跳与超时
|
||||
- 前端渲染控制器:连接生命周期管理、Offer/Answer/Candidate 分发、统计与数据通道
|
||||
- PeerConnection 管理:SDP 描述协商、ICE 候选收集与注入、事件派发
|
||||
- 输入远程控制:设备事件采集、状态编码、消息打包与发送观察者
|
||||
- 日志系统:前后端日志级别控制与格式化输出
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||||
- [src/signaling.ts:6-24](file://src/signaling.ts#L6-L24)
|
||||
- [src/class/websockethandler.ts:10-137](file://src/class/websockethandler.ts#L10-L137)
|
||||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
- [client/src/inputremoting.js:9-300](file://client/src/inputremoting.js#L9-L300)
|
||||
- [client/src/inputdevice.js:40-719](file://client/src/inputdevice.js#L40-L719)
|
||||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||||
|
||||
## 架构总览
|
||||
系统采用“前端渲染 + 后端信令 + 媒体直连”的架构:
|
||||
- 前端通过 HTTP 或 WebSocket 与后端进行信令交互,完成连接建立与媒体协商
|
||||
- 媒体流在浏览器间直接传输,信令仅传递 SDP 与 ICE 候选
|
||||
- 输入事件在前端采集后编码并通过 RTCDataChannel 或自定义消息通道发送至远端
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Host as "主机(Host)"
|
||||
participant Server as "信令服务器"
|
||||
participant Participant as "参与者(Participant)"
|
||||
Host->>Server : "创建会话/连接"
|
||||
Note over Host,Server : "HTTP/WebSocket 信令握手"
|
||||
Host->>Server : "发送 Offer"
|
||||
Server-->>Participant : "转发 Offer"
|
||||
Participant->>Server : "发送 Answer"
|
||||
Server-->>Host : "转发 Answer"
|
||||
Host->>Server : "发送 ICE Candidate"
|
||||
Server-->>Participant : "转发 Candidate"
|
||||
Participant->>Server : "发送 ICE Candidate"
|
||||
Server-->>Host : "转发 Candidate"
|
||||
Note over Host,Participant : "媒体流直连传输"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||||
- [client/src/renderstreaming.js:72-147](file://client/src/renderstreaming.js#L72-L147)
|
||||
|
||||
**章节来源**
|
||||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||||
- [client/src/renderstreaming.js:72-147](file://client/src/renderstreaming.js#L72-L147)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### PeerConnection 管理与控制
|
||||
- 事件绑定:track、datachannel、icecandidate、negotiationneeded、signalingstatechange、iceconnectionstatechange、icegatheringstatechange
|
||||
- 协商流程:在 negotiationneeded 触发时设置本地描述并派发 offer;收到远端 offer 后设置远端描述并生成 answer
|
||||
- ICE 候选:收集到候选时派发候选事件,等待远端注入
|
||||
- 状态监控:失败时派发 disconnect 事件,便于上层重连或降级
|
||||
- 统计与控制:支持 getStats、addTrack/addTransceiver、createDataChannel 等
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Peer as "Peer(前端)"
|
||||
participant Signaling as "信令客户端"
|
||||
participant Remote as "远端Peer"
|
||||
Peer->>Peer : "negotiationneeded 触发"
|
||||
Peer->>Peer : "setLocalDescription()"
|
||||
Peer-->>Signaling : "派发 sendoffer"
|
||||
Signaling-->>Remote : "发送 Offer"
|
||||
Remote->>Remote : "setRemoteDescription(Offer)"
|
||||
Remote->>Remote : "setLocalDescription(Answer)"
|
||||
Remote-->>Signaling : "派发 sendanswer"
|
||||
Signaling-->>Peer : "发送 Answer"
|
||||
Peer->>Peer : "setRemoteDescription(Answer)"
|
||||
Peer-->>Peer : "触发 negotiated 事件"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [client/src/peer.js:37-173](file://client/src/peer.js#L37-L173)
|
||||
- [client/src/renderstreaming.js:72-130](file://client/src/renderstreaming.js#L72-L130)
|
||||
|
||||
**章节来源**
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
- [client/src/renderstreaming.js:72-130](file://client/src/renderstreaming.js#L72-L130)
|
||||
- [client/test/peerconnection.test.js:1-251](file://client/test/peerconnection.test.js#L1-L251)
|
||||
|
||||
### 输入设备远程控制
|
||||
- 设备抽象:Mouse、Keyboard、Touchscreen、Gamepad 四类设备,统一状态接口
|
||||
- 事件采集:鼠标/键盘/触摸/手柄事件监听与状态更新
|
||||
- 状态编码:各设备状态转为二进制缓冲区,包含格式标识、尺寸、数据
|
||||
- 消息打包:StateEvent/TextEvent 包装基础事件与状态数据
|
||||
- 发送链路:InputRemoting 订阅观察者,将消息序列化后通过 RTCDataChannel 或自定义通道发送
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["用户输入事件"]) --> Queue["设备队列更新状态"]
|
||||
Queue --> Encode["编码为 IInputState 缓冲区"]
|
||||
Encode --> Wrap["包装为 StateEvent/TextEvent"]
|
||||
Wrap --> Pack["封装为 Message(含 participant_id/type/length/data)"]
|
||||
Pack --> Send["通过观察者发送(如 RTCDataChannel)"]
|
||||
Send --> End(["远端接收并应用"])
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [client/src/inputdevice.js:91-719](file://client/src/inputdevice.js#L91-L719)
|
||||
- [client/src/inputremoting.js:63-300](file://client/src/inputremoting.js#L63-L300)
|
||||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||||
|
||||
**章节来源**
|
||||
- [client/src/inputdevice.js:91-719](file://client/src/inputdevice.js#L91-L719)
|
||||
- [client/src/inputremoting.js:63-300](file://client/src/inputremoting.js#L63-L300)
|
||||
- [client/src/sender.js:14-209](file://client/src/sender.js#L14-L209)
|
||||
- [client/test/inputremoting.test.js:1-132](file://client/test/inputremoting.test.js#L1-L132)
|
||||
|
||||
### 渲染流处理
|
||||
- 渲染控制器:根据角色(host/participant)创建/复用 Peer,订阅 trackevent/adddatachannel/onmessage 等事件
|
||||
- 媒体流接收:trackevent 中获取远端媒体轨道,绑定到视频元素进行播放
|
||||
- 数据通道:adddatachannel 事件用于建立输入控制通道或其他应用通道
|
||||
- 统计与诊断:getStats 接口用于网络质量与传输指标观测
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant RS as "RenderStreaming"
|
||||
participant Peer as "Peer"
|
||||
participant Video as "视频元素"
|
||||
RS->>Peer : "创建/复用 Peer 并绑定事件"
|
||||
Peer-->>RS : "trackevent(媒体轨道)"
|
||||
RS->>Video : "设置 srcObject 为媒体流"
|
||||
Peer-->>RS : "adddatachannel(数据通道)"
|
||||
RS->>RS : "getStats() 收集统计信息"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [client/src/renderstreaming.js:212-250](file://client/src/renderstreaming.js#L212-L250)
|
||||
- [client/src/peer.js:21-28](file://client/src/peer.js#L21-L28)
|
||||
|
||||
**章节来源**
|
||||
- [client/src/renderstreaming.js:11-317](file://client/src/renderstreaming.js#L11-L317)
|
||||
- [client/src/peer.js:21-28](file://client/src/peer.js#L21-L28)
|
||||
|
||||
### 信令客户端实现
|
||||
- HTTP 轮询:定时拉取消息,解析类型并派发事件
|
||||
- WebSocket:长连接,消息类型丰富,支持广播与参与者路由
|
||||
- 会话管理:创建/删除会话、连接管理、消息聚合
|
||||
- 参与者模型:私有模式下 host 与多个 participants,支持按 participantId 路由
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class Signaling {
|
||||
+start()
|
||||
+stop()
|
||||
+createConnection()
|
||||
+deleteConnection()
|
||||
+sendOffer()
|
||||
+sendAnswer()
|
||||
+sendCandidate()
|
||||
+getAll()
|
||||
}
|
||||
class WebSocketSignaling {
|
||||
+start()
|
||||
+stop()
|
||||
+createConnection()
|
||||
+deleteConnection()
|
||||
+sendOffer()
|
||||
+sendAnswer()
|
||||
+sendCandidate()
|
||||
+sendMessage()
|
||||
}
|
||||
Signaling <|-- WebSocketSignaling
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
|
||||
**章节来源**
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
- [src/class/httphandler.ts:492-641](file://src/class/httphandler.ts#L492-L641)
|
||||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||||
|
||||
## 依赖关系分析
|
||||
- 后端依赖:Express、ws、uuid、swagger-ui-express 等
|
||||
- 前端依赖:浏览器原生 WebRTC API、RTCDataChannel、EventTarget、Fetch/WebSocket
|
||||
- 日志:前后端独立日志模块,支持级别控制与格式化输出
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
Pkg["package.json 依赖声明"] --> Express["express"]
|
||||
Pkg --> WS["ws"]
|
||||
Pkg --> UUID["uuid"]
|
||||
Pkg --> Swagger["swagger-ui-express"]
|
||||
Front["前端模块"] --> WebRTC["WebRTC API"]
|
||||
Front --> FetchWS["Fetch/WebSocket"]
|
||||
LogB["后端日志(src/log.ts)"] --> ConsoleB["控制台输出"]
|
||||
LogF["前端日志(client/src/logger.js)"] --> ConsoleF["控制台输出"]
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [package.json:14-46](file://package.json#L14-L46)
|
||||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||||
|
||||
**章节来源**
|
||||
- [package.json:14-46](file://package.json#L14-L46)
|
||||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||||
|
||||
## 性能考虑
|
||||
- 连接稳定性
|
||||
- 使用轮询重发 Offer 以应对网络抖动,避免长时间无响应
|
||||
- ICE 候选收集完成后及时注入,减少连接建立延迟
|
||||
- 媒体协商
|
||||
- 优先使用匹配编解码器,减少转码开销
|
||||
- 合理设置带宽限制与回退策略
|
||||
- 输入事件
|
||||
- 状态事件合并发送,降低消息频率
|
||||
- 使用二进制消息格式,减少序列化成本
|
||||
- 信令
|
||||
- WebSocket 模式优于 HTTP 轮询,降低延迟与 CPU 开销
|
||||
- 私有模式下按 participantId 路由,避免广播风暴
|
||||
- 资源管理
|
||||
- 及时关闭 PeerConnection 与数据通道
|
||||
- 监控统计指标,动态调整分辨率与帧率
|
||||
|
||||
[本节为通用指导,无需具体文件分析]
|
||||
|
||||
## 故障排查指南
|
||||
- 日志启用
|
||||
- 前端:调用日志模块的 enable/disable 控制输出
|
||||
- 后端:通过命令行参数设置日志级别
|
||||
- 常见问题定位
|
||||
- 连接失败:检查 signaling 事件派发与路由,确认 Offer/Answer/Candidate 是否正确传递
|
||||
- ICE 失败:查看 iceconnectionstatechange/disconnect 事件,确认候选注入与防火墙/NAT 配置
|
||||
- 输入无响应:确认 RTCDataChannel 已打开,消息格式与 participantId 正确
|
||||
- 单元测试
|
||||
- PeerConnection 行为验证:添加/移除轨道、发送 Offer/Answer、ICE 候选注入
|
||||
- 输入系统:设备消息打包、事件编码、观察者发送
|
||||
|
||||
**章节来源**
|
||||
- [client/src/logger.js:1-30](file://client/src/logger.js#L1-L30)
|
||||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||||
- [client/test/peerconnection.test.js:1-251](file://client/test/peerconnection.test.js#L1-L251)
|
||||
- [client/test/inputremoting.test.js:1-132](file://client/test/inputremoting.test.js#L1-L132)
|
||||
|
||||
## 结论
|
||||
本实现提供了完整的 WebRTC 远程渲染与输入控制方案,具备清晰的模块划分与可扩展性。通过事件驱动与观察者模式,前端渲染控制器与输入系统能够高效协作;后端信令处理器支持私有与公共两种模式,满足不同场景需求。配合完善的日志与测试体系,便于维护与优化。
|
||||
|
||||
[本节为总结,无需具体文件分析]
|
||||
|
||||
## 附录
|
||||
- 启动参数与脚本
|
||||
- 开发模式:ts-node 启动,支持 HTTPS/HTTP 与私有/公共模式切换
|
||||
- 生产模式:构建后运行,支持证书文件与端口配置
|
||||
- API 与消息类型
|
||||
- HTTP 轮询:/signaling/* 接口,支持会话管理与消息聚合
|
||||
- WebSocket:connect/disconnect/offer/answer/candidate/on-message/participant-joined/participant-left/broadcast 等
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:16-82](file://src/index.ts#L16-L82)
|
||||
- [package.json:5-12](file://package.json#L5-L12)
|
||||
- [src/signaling.ts:6-24](file://src/signaling.ts#L6-L24)
|
||||
- [src/class/websockethandler.ts:145-473](file://src/class/websockethandler.ts#L145-L473)
|
||||
428
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/游戏手柄输入处理.md
Normal file
428
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/游戏手柄输入处理.md
Normal file
@@ -0,0 +1,428 @@
|
||||
# 游戏手柄输入处理
|
||||
|
||||
<cite>
|
||||
**本文档引用的文件**
|
||||
- [gamepadhandler.js](file://client/src/gamepadhandler.js)
|
||||
- [inputdevice.js](file://client/src/inputdevice.js)
|
||||
- [inputremoting.js](file://client/src/inputremoting.js)
|
||||
- [gamepadbutton.js](file://client/src/gamepadbutton.js)
|
||||
- [memoryhelper.js](file://client/src/memoryhelper.js)
|
||||
- [keymap.js](file://client/src/keymap.js)
|
||||
- [mousebutton.js](file://client/src/mousebutton.js)
|
||||
- [touchflags.js](file://client/src/touchflags.js)
|
||||
- [touchphase.js](file://client/src/touchphase.js)
|
||||
- [gamepadEvents.js](file://client/public/videoplayer/js/gamepadEvents.js)
|
||||
- [register-events.js](file://client/public/videoplayer/js/register-events.js)
|
||||
- [index.html](file://client/public/videoplayer/index.html)
|
||||
- [inputdevice.test.js](file://client/test/inputdevice.test.js)
|
||||
- [inputremoting.test.js](file://client/test/inputremoting.test.js)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [简介](#简介)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考虑](#性能考虑)
|
||||
8. [故障排除指南](#故障排除指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 简介
|
||||
本文件面向需要在浏览器中实现高质量游戏手柄输入处理的开发者,系统性阐述从设备枚举、事件捕获、状态转换到远端传输的完整链路。文档重点覆盖以下方面:
|
||||
- 游戏手柄事件处理机制:摇杆值、按钮状态、触发器事件的捕获与转发
|
||||
- 设备识别、枚举与状态同步
|
||||
- 输入校准、死区处理与输入平滑算法
|
||||
- 不同厂商手柄差异的兼容策略
|
||||
- 连接状态监控与输入延迟优化
|
||||
|
||||
## 项目结构
|
||||
客户端采用模块化设计,围绕输入设备抽象、状态序列化与消息封装展开,并在视频播放示例中集成手柄事件监听与发送。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "输入设备层"
|
||||
ID["InputDevice 抽象类"]
|
||||
GP["Gamepad 手柄设备"]
|
||||
KB["Keyboard 键盘设备"]
|
||||
MS["Mouse 鼠标设备"]
|
||||
TS["Touchscreen 触摸屏设备"]
|
||||
end
|
||||
subgraph "状态模型层"
|
||||
GS["GamepadState 状态"]
|
||||
KS["KeyboardState 状态"]
|
||||
MS["MouseState 状态"]
|
||||
TSS["TouchscreenState 状态"]
|
||||
end
|
||||
subgraph "消息与传输层"
|
||||
SE["StateEvent 状态事件"]
|
||||
NE["NewEventsMsg 新事件消息"]
|
||||
ND["NewDeviceMsg 新设备消息"]
|
||||
RM["InputRemoting 远程输入管理"]
|
||||
end
|
||||
subgraph "浏览器接口层"
|
||||
GH["GamepadHandler 扫描器"]
|
||||
GE["gamepadEvents.js 事件桥接"]
|
||||
RE["register-events.js 事件注册"]
|
||||
end
|
||||
ID --> GP
|
||||
ID --> KB
|
||||
ID --> MS
|
||||
ID --> TS
|
||||
GP --> GS
|
||||
KB --> KS
|
||||
MS --> MS
|
||||
TS --> TSS
|
||||
GS --> SE
|
||||
SE --> NE
|
||||
ID --> ND
|
||||
RM --> NE
|
||||
RM --> ND
|
||||
GH --> GP
|
||||
GE --> GP
|
||||
RE --> GE
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputdevice.js:40-127](file://client/src/inputdevice.js#L40-L127)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [gamepadhandler.js:1-45](file://client/src/gamepadhandler.js#L1-L45)
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:1-719](file://client/src/inputdevice.js#L1-L719)
|
||||
- [inputremoting.js:1-300](file://client/src/inputremoting.js#L1-L300)
|
||||
- [gamepadhandler.js:1-45](file://client/src/gamepadhandler.js#L1-L45)
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
|
||||
## 核心组件
|
||||
- GamepadHandler:基于 requestAnimationFrame 的周期性扫描与分发,维护控制器索引映射并派发自定义 gamepadupdated 事件。
|
||||
- Gamepad/InputDevice:输入设备抽象与手柄状态队列,将事件转换为 GamepadState。
|
||||
- GamepadState:将摇杆、触发器与按钮状态打包为二进制缓冲区,包含固定格式标识与字段布局。
|
||||
- MemoryHelper:位级写入工具,用于将布尔型按钮状态压缩到字节数组。
|
||||
- InputRemoting:订阅本地输入事件,生成设备与事件消息并通过观察者推送。
|
||||
- NewEventsMsg/NewDeviceMsg:消息封装,承载设备描述与状态数据。
|
||||
- 浏览器事件桥接:gamepadEvents.js 与 register-events.js 将原生 Gamepad API 事件转换为统一格式并发送。
|
||||
|
||||
**章节来源**
|
||||
- [gamepadhandler.js:1-45](file://client/src/gamepadhandler.js#L1-L45)
|
||||
- [inputdevice.js:120-127](file://client/src/inputdevice.js#L120-L127)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [inputremoting.js:234-256](file://client/src/inputremoting.js#L234-L256)
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
|
||||
## 架构总览
|
||||
下图展示了从浏览器手柄事件到远端传输的关键流程,包括事件桥接、状态序列化与消息封装。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant Browser as "浏览器"
|
||||
participant GH as "GamepadHandler"
|
||||
participant GE as "gamepadEvents.js"
|
||||
participant RE as "register-events.js"
|
||||
participant ID as "InputDevice/Gamepad"
|
||||
participant GS as "GamepadState"
|
||||
participant RM as "InputRemoting"
|
||||
participant NE as "NewEventsMsg"
|
||||
Browser->>GH : "requestAnimationFrame 循环"
|
||||
GH->>GH : "_scanGamepad() 枚举并更新控制器"
|
||||
GH-->>Browser : "gamepadupdated 事件"
|
||||
Browser->>GE : "原生 Gamepad 事件"
|
||||
GE->>RE : "统一格式事件按钮/轴"
|
||||
RE->>ID : "queueEvent(event)"
|
||||
ID->>GS : "构造 GamepadState"
|
||||
GS-->>RM : "StateEvent.from(device)"
|
||||
RM->>NE : "NewEventsMsg.create(state)"
|
||||
NE-->>Browser : "通过观察者推送"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadhandler.js:22-42](file://client/src/gamepadhandler.js#L22-L42)
|
||||
- [gamepadEvents.js:68-94](file://client/public/videoplayer/js/gamepadEvents.js#L68-L94)
|
||||
- [register-events.js:42-101](file://client/public/videoplayer/js/register-events.js#L42-L101)
|
||||
- [inputdevice.js:120-127](file://client/src/inputdevice.js#L120-L127)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### 组件一:GamepadHandler(设备扫描与事件派发)
|
||||
- 职责:维护控制器映射表,周期性调用 _scanGamepad 更新控制器快照,遍历控制器并派发 gamepadupdated 事件。
|
||||
- 关键点:
|
||||
- 使用 window.requestAnimationFrame 驱动循环,避免阻塞主线程。
|
||||
- 通过 navigator.getGamepads() 获取实时控制器集合,按索引覆盖旧状态。
|
||||
- 自定义事件类型 gamepadupdated,便于上层统一处理。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["进入 _updateStatus"]) --> Scan["_scanGamepad() 枚举控制器"]
|
||||
Scan --> Loop{"遍历控制器"}
|
||||
Loop --> |存在| Dispatch["派发 gamepadupdated 事件"]
|
||||
Loop --> |不存在| NextFrame["requestAnimationFrame 下次循环"]
|
||||
Dispatch --> NextFrame
|
||||
NextFrame --> Start
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadhandler.js:22-42](file://client/src/gamepadhandler.js#L22-L42)
|
||||
|
||||
**章节来源**
|
||||
- [gamepadhandler.js:1-45](file://client/src/gamepadhandler.js#L1-L45)
|
||||
|
||||
### 组件二:Gamepad 与 GamepadState(状态建模与序列化)
|
||||
- Gamepad:继承 InputDevice,将事件队列转换为 GamepadState。
|
||||
- GamepadState:
|
||||
- 字段:buttons(4字节位图)、leftStick、rightStick(每项浮点)、leftTrigger、rightTrigger(浮点)。
|
||||
- 使用 MemoryHelper 将布尔型按钮状态写入位图,紧凑存储。
|
||||
- buffer 输出固定长度字节块,配合 FourCC 格式标识('GPAD')。
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class InputDevice {
|
||||
+name
|
||||
+layout
|
||||
+deviceId
|
||||
+usages
|
||||
+description
|
||||
+queueEvent(event)
|
||||
+currentState
|
||||
}
|
||||
class Gamepad {
|
||||
+queueEvent(event)
|
||||
}
|
||||
class IInputState {
|
||||
+buffer
|
||||
+format
|
||||
}
|
||||
class GamepadState {
|
||||
+buttons
|
||||
+leftStick
|
||||
+rightStick
|
||||
+leftTrigger
|
||||
+rightTrigger
|
||||
+buffer
|
||||
+format
|
||||
}
|
||||
InputDevice <|-- Gamepad
|
||||
IInputState <|-- GamepadState
|
||||
Gamepad --> GamepadState : "构造状态"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputdevice.js:40-127](file://client/src/inputdevice.js#L40-L127)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:120-127](file://client/src/inputdevice.js#L120-L127)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
|
||||
### 组件三:事件桥接与消息封装(浏览器到远端)
|
||||
- gamepadEvents.js:
|
||||
- 存储每个手柄的按钮与轴历史状态,计算差分以检测变化。
|
||||
- 死区阈值 _e 控制轴值是否上报;当从移动回到静止时强制发送归零事件,保证状态一致性。
|
||||
- 通过自定义事件(gamepadButtonDown/Up/Pressed、gamepadAxis)向上游广播。
|
||||
- register-events.js:
|
||||
- 注册原生 gamepadconnected/disconnected 事件,委托给 gamepadHandler。
|
||||
- 将统一格式的按钮/轴事件序列化为二进制消息,通过 VideoPlayer 发送。
|
||||
- InputRemoting/NewEventsMsg/NewDeviceMsg:
|
||||
- 订阅本地事件,生成 StateEvent 或设备消息,通过观察者推送。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant NW as "navigator.getGamepads()"
|
||||
participant GE as "gamepadEvents.js"
|
||||
participant RE as "register-events.js"
|
||||
participant RM as "InputRemoting"
|
||||
participant NE as "NewEventsMsg"
|
||||
NW-->>GE : "返回控制器数组"
|
||||
GE->>GE : "比较按钮/轴前后状态<br/>应用死区与差分"
|
||||
GE-->>RE : "自定义事件按钮/轴"
|
||||
RE->>RM : "sendMsg(二进制消息)"
|
||||
RM->>NE : "NewEventsMsg.create(state)"
|
||||
NE-->>RE : "完成发送"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadEvents.js:43-94](file://client/public/videoplayer/js/gamepadEvents.js#L43-L94)
|
||||
- [register-events.js:42-101](file://client/public/videoplayer/js/register-events.js#L42-L101)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
|
||||
**章节来源**
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
|
||||
### 组件四:输入校准、死区与平滑(算法与实现要点)
|
||||
- 死区处理:
|
||||
- 轴值绝对值小于阈值 _e 时不发送,避免漂移误触。
|
||||
- 当从移动回到静止时强制发送 (0,0),确保接收端状态收敛。
|
||||
- 平滑与差分:
|
||||
- 通过存储上一帧按钮/轴状态,仅在变化或持续按压时发送,降低冗余。
|
||||
- 坐标与方向:
|
||||
- 左右摇杆 Y 轴乘以 _axisYInverted 实现符合预期的朝向。
|
||||
- 触发器值直接取用 buttons[n].value,映射到浮点范围。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
S(["开始轮询"]) --> Axes["遍历摇杆轴对"]
|
||||
Axes --> AbsCheck{"|x|>|_e| 或 |y|>|_e| ?"}
|
||||
AbsCheck --> |是| SendMove["发送轴事件(x,y)"]
|
||||
AbsCheck --> |否| PrevAbs{"之前是否移动?"}
|
||||
PrevAbs --> |是| SendZero["发送轴事件(0,0)"]
|
||||
PrevAbs --> |否| Skip["跳过"]
|
||||
SendMove --> Store["保存当前状态"]
|
||||
SendZero --> Store
|
||||
Skip --> Store
|
||||
Store --> Next["下一轴对"]
|
||||
Next --> |完成| S
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadEvents.js:43-66](file://client/public/videoplayer/js/gamepadEvents.js#L43-L66)
|
||||
|
||||
**章节来源**
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
|
||||
### 组件五:设备识别、枚举与状态同步
|
||||
- 设备识别:
|
||||
- 使用 gamepad.id 作为唯一标识,去除空格后作为 Cookie 键,持久化连接时间戳,确保跨刷新一致。
|
||||
- 枚举与同步:
|
||||
- GamepadHandler 在每次循环中调用 _scanGamepad,将已知索引的控制器替换为最新快照,保证状态同步。
|
||||
- 连接状态监控:
|
||||
- 监听 gamepadconnected/disconnected,分别初始化/清理状态缓存与轮询定时器。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant W as "Window"
|
||||
participant GE as "gamepadEvents.js"
|
||||
participant GH as "GamepadHandler"
|
||||
participant DOC as "Document"
|
||||
W->>DOC : "gamepadconnected/disconnected"
|
||||
DOC->>GE : "gamepadHandler(connecting)"
|
||||
GE->>GE : "初始化/清理状态缓存与定时器"
|
||||
GH->>GH : "_scanGamepad() 同步控制器快照"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadEvents.js:112-146](file://client/public/videoplayer/js/gamepadEvents.js#L112-L146)
|
||||
- [gamepadhandler.js:35-42](file://client/src/gamepadhandler.js#L35-L42)
|
||||
|
||||
**章节来源**
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [gamepadhandler.js:1-45](file://client/src/gamepadhandler.js#L1-L45)
|
||||
|
||||
### 组件六:不同厂商手柄差异与兼容策略
|
||||
- 按键与轴数量差异:
|
||||
- 代码假设 buttons 数组长度为 16,axes 数组长度为 4;若实际设备不满足,需在构造 GamepadState 前进行边界检查与默认填充。
|
||||
- 按键语义映射:
|
||||
- 通过 GamepadButton 常量将不同厂商的按键映射到统一编号,减少上游分支逻辑。
|
||||
- 触发器与肩键:
|
||||
- 触发器值通过 buttons[6]/buttons[7] 获取,注意部分设备可能不支持压力值,应以 pressed/value 双重判断。
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
- [gamepadbutton.js:1-26](file://client/src/gamepadbutton.js#L1-L26)
|
||||
|
||||
## 依赖关系分析
|
||||
- 模块内聚与耦合:
|
||||
- InputDevice/GamepadState 与 MemoryHelper 解耦,通过位操作实现紧凑存储。
|
||||
- InputRemoting 通过事件总线与消息封装与输入层解耦。
|
||||
- 外部依赖:
|
||||
- 浏览器 Gamepad API 提供原始输入;gamepadEvents.js 与 register-events.js 作为桥接层。
|
||||
- 潜在循环依赖:
|
||||
- 当前结构未见循环导入;建议保持事件桥接层独立于输入设备层。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
GE["gamepadEvents.js"] --> RE["register-events.js"]
|
||||
RE --> RM["InputRemoting"]
|
||||
RM --> NE["NewEventsMsg"]
|
||||
ID["InputDevice"] --> GS["GamepadState"]
|
||||
GS --> SE["StateEvent"]
|
||||
SE --> NE
|
||||
MH["MemoryHelper"] --> GS
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:1-719](file://client/src/inputdevice.js#L1-L719)
|
||||
- [inputremoting.js:1-300](file://client/src/inputremoting.js#L1-L300)
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [register-events.js:1-101](file://client/public/videoplayer/js/register-events.js#L1-L101)
|
||||
|
||||
## 性能考虑
|
||||
- 采样频率与帧驱动:
|
||||
- 使用 requestAnimationFrame 驱动扫描,避免固定间隔造成的卡顿与能耗。
|
||||
- 事件去抖与差分:
|
||||
- 仅在按钮/轴状态变化或持续按压时发送,显著降低带宽占用。
|
||||
- 死区与归零:
|
||||
- 通过死区阈值与“从移动到静止”的强制归零,减少无效数据与状态抖动。
|
||||
- 序列化开销:
|
||||
- 位图存储按钮状态,单个状态包固定大小,便于批量传输与解析。
|
||||
|
||||
[本节为通用性能讨论,无需特定文件来源]
|
||||
|
||||
## 故障排除指南
|
||||
- 手柄未被识别:
|
||||
- 确认浏览器已启用 Gamepad API,且手柄正确连接;检查 gamepadconnected 事件是否触发。
|
||||
- 摇杆无响应或漂移:
|
||||
- 调整死区阈值 _e;确认左右摇杆 Y 轴方向是否正确(_axisYInverted)。
|
||||
- 按键状态异常:
|
||||
- 检查按钮索引映射与设备实际按键布局;必要时在构造 GamepadState 前进行边界保护。
|
||||
- 事件丢失或延迟:
|
||||
- 确保在 _updateStatus 中持续调用 _scanGamepad;检查 requestAnimationFrame 是否被页面隐藏中断。
|
||||
|
||||
**章节来源**
|
||||
- [gamepadhandler.js:22-42](file://client/src/gamepadhandler.js#L22-L42)
|
||||
- [gamepadEvents.js:1-147](file://client/public/videoplayer/js/gamepadEvents.js#L1-L147)
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
|
||||
## 结论
|
||||
该系统通过清晰的分层设计实现了从浏览器手柄输入到远端传输的完整链路:GamepadHandler 负责设备扫描与事件派发,gamepadEvents.js 与 register-events.js 完成事件桥接与消息封装,InputDevice/GamepadState 提供稳定的输入状态模型,InputRemoting 则负责消息生成与推送。通过死区、差分与归零等策略有效降低了冗余与延迟,具备良好的可扩展性与兼容性。
|
||||
|
||||
[本节为总结性内容,无需特定文件来源]
|
||||
|
||||
## 附录
|
||||
|
||||
### A. 数据模型与字段布局(GamepadState)
|
||||
- 固定格式标识:FourCC('GPAD')
|
||||
- 字段布局(字节偏移):
|
||||
- buttons:0~3(4字节位图)
|
||||
- leftStick.x:4
|
||||
- leftStick.y:8
|
||||
- rightStick.x:12
|
||||
- rightStick.y:16
|
||||
- leftTrigger:20
|
||||
- rightTrigger:24
|
||||
- 总长度:28 字节
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:540-618](file://client/src/inputdevice.js#L540-L618)
|
||||
|
||||
### B. 测试要点(验证输入序列化与消息封装)
|
||||
- GamepadState 格式与缓冲区长度验证
|
||||
- StateEvent 与 NewEventsMsg 的二进制输出结构
|
||||
- 设备消息 NewDeviceMsg 的 JSON 描述序列化
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.test.js:102-121](file://client/test/inputdevice.test.js#L102-L121)
|
||||
- [inputremoting.test.js:107-121](file://client/test/inputremoting.test.js#L107-L121)
|
||||
- [inputremoting.test.js:50-57](file://client/test/inputremoting.test.js#L50-L57)
|
||||
326
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/输入事件同步.md
Normal file
326
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/输入事件同步.md
Normal file
@@ -0,0 +1,326 @@
|
||||
# 输入事件同步
|
||||
|
||||
<cite>
|
||||
**本文引用的文件**
|
||||
- [src/index.ts](file://src/index.ts)
|
||||
- [src/websocket.ts](file://src/websocket.ts)
|
||||
- [client/src/inputremoting.js](file://client/src/inputremoting.js)
|
||||
- [client/src/sender.js](file://client/src/sender.js)
|
||||
- [client/src/inputdevice.js](file://client/src/inputdevice.js)
|
||||
- [client/src/memoryhelper.js](file://client/src/memoryhelper.js)
|
||||
- [client/src/pointercorrect.js](file://client/src/pointercorrect.js)
|
||||
- [client/src/signaling.js](file://client/src/signaling.js)
|
||||
- [client/src/peer.js](file://client/src/peer.js)
|
||||
- [client/test/inputremoting.test.js](file://client/test/inputremoting.test.js)
|
||||
- [client/test/inputdevice.test.js](file://client/test/inputdevice.test.js)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [引言](#引言)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考量](#性能考量)
|
||||
8. [故障排查指南](#故障排查指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 引言
|
||||
本技术文档聚焦于输入事件同步机制,围绕以下目标展开:时间同步、延迟补偿、状态一致性保证;事件队列管理、时间戳处理与网络传输优化;缓冲策略、丢包处理与重排序;以及在多设备间实现高精度时间同步与输入一致性。文档基于仓库中的客户端输入遥测与信令通道实现,结合服务端 WebSocket 信令处理,系统性阐述从事件采集、编码、传输到渲染的一体化流程。
|
||||
|
||||
## 项目结构
|
||||
该项目采用前后端分离的结构:
|
||||
- 客户端包含输入设备抽象、事件打包、信令交互与 WebRTC 数据通道发送等模块;
|
||||
- 服务端提供基于 WebSocket 的信令通道,负责连接管理与消息转发。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "客户端"
|
||||
A["输入设备抽象<br/>inputdevice.js"]
|
||||
B["本地输入管理器<br/>inputremoting.js"]
|
||||
C["发送器与事件队列<br/>sender.js"]
|
||||
D["指针坐标矫正<br/>pointercorrect.js"]
|
||||
E["信令(HTTP/WebSocket)<br/>signaling.js"]
|
||||
F["WebRTC Peer 连接<br/>peer.js"]
|
||||
end
|
||||
subgraph "服务端"
|
||||
G["WebSocket 信令服务<br/>websocket.ts"]
|
||||
end
|
||||
A --> B
|
||||
B --> C
|
||||
C --> F
|
||||
C --> E
|
||||
F --> G
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/inputdevice.js:40-89](file://client/src/inputdevice.js#L40-L89)
|
||||
- [client/src/inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [client/src/sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [client/src/pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
- [src/websocket.ts:6-118](file://src/websocket.ts#L6-L118)
|
||||
|
||||
章节来源
|
||||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||||
- [src/websocket.ts:6-118](file://src/websocket.ts#L6-L118)
|
||||
|
||||
## 核心组件
|
||||
- 输入设备与状态模型:鼠标、键盘、触摸屏、手柄的状态与事件定义,统一以二进制帧格式封装,便于跨网络传输与解码。
|
||||
- 本地输入管理器:负责事件采集、时间戳注入、初始布局与设备信息广播、订阅者分发。
|
||||
- 发送器:将 DOM 事件转换为输入状态,生成状态事件与文本事件,并通过观察者模式推送至数据通道或信令通道。
|
||||
- 指针矫正器:根据视频显示比例与留边(letterbox)计算屏幕坐标到视频坐标的映射,确保跨设备一致的输入位置。
|
||||
- 信令层:支持 HTTP 轮询与 WebSocket 两种模式,用于连接建立、SDP 交换与自定义消息广播。
|
||||
- WebRTC Peer:封装 ICE、Offer/Answer、数据通道创建与状态机,保障媒体与控制面协同。
|
||||
|
||||
章节来源
|
||||
- [client/src/inputdevice.js:129-178](file://client/src/inputdevice.js#L129-L178)
|
||||
- [client/src/inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [client/src/sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [client/src/pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [client/src/signaling.js:3-292](file://client/src/signaling.js#L3-L292)
|
||||
- [client/src/peer.js:3-188](file://client/src/peer.js#L3-L188)
|
||||
|
||||
## 架构总览
|
||||
下图展示了从用户输入到远端渲染的关键路径:DOM 事件经由发送器转换为状态事件,通过 RTC 数据通道或信令通道发送,服务端 WebSocket 信令进行路由,最终到达对端渲染端。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as "用户"
|
||||
participant DOM as "DOM 事件"
|
||||
participant S as "Sender"
|
||||
participant IR as "InputRemoting"
|
||||
participant DC as "RTC 数据通道"
|
||||
participant WS as "WebSocket 信令"
|
||||
participant P as "Peer(对端)"
|
||||
participant R as "渲染端"
|
||||
U->>DOM : 触发输入事件
|
||||
DOM->>S : 鼠标/键盘/触摸/手柄事件
|
||||
S->>S : 生成状态事件与文本事件
|
||||
S->>IR : 分发事件
|
||||
IR->>DC : 通过观察者发送二进制消息
|
||||
IR->>WS : 通过信令通道发送自定义消息
|
||||
WS-->>P : 转发消息
|
||||
P->>P : 解析并应用状态
|
||||
P->>R : 渲染一致状态
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
- [client/src/inputremoting.js:138-168](file://client/src/inputremoting.js#L138-L168)
|
||||
- [client/src/signaling.js:140-146](file://client/src/signaling.js#L140-L146)
|
||||
- [src/websocket.ts:44-114](file://src/websocket.ts#L44-L114)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### 时间同步与时间戳处理
|
||||
- 启动时间注入:本地输入管理器在开始发送时记录启动时间,后续所有事件携带相对时间戳,确保跨设备时间基准一致。
|
||||
- 事件时间戳:输入事件与状态事件均包含时间字段,用于对端按时间顺序重建状态。
|
||||
- 坐标映射与时间耦合:指针矫正器在事件生成时使用当前相对时间,避免因时间漂移导致的错位。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["开始发送"]) --> SetStart["记录启动时间"]
|
||||
SetStart --> OnEvent["捕获输入事件"]
|
||||
OnEvent --> InjectTime["注入相对时间戳"]
|
||||
InjectTime --> BuildState["构建状态事件/文本事件"]
|
||||
BuildState --> Send["发送至观察者/信令通道"]
|
||||
Send --> End(["完成一次事件发送"])
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/inputremoting.js:73-91](file://client/src/inputremoting.js#L73-L91)
|
||||
- [client/src/sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
- [client/src/pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
|
||||
章节来源
|
||||
- [client/src/inputremoting.js:32-48](file://client/src/inputremoting.js#L32-L48)
|
||||
- [client/src/sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
- [client/src/inputdevice.js:149-163](file://client/src/inputdevice.js#L149-L163)
|
||||
|
||||
### 事件队列管理与去抖动
|
||||
- 事件聚合:触摸事件在一次 TouchEvent 中可能包含多个 changedTouches,发送器逐个拆分并生成对应状态事件,保证每个触点独立同步。
|
||||
- 文本事件与按键事件分离:按键事件区分 keydown/keyup 与重复按下(repeat),仅在非重复场景生成状态事件,避免冗余。
|
||||
- 去抖动策略:通过事件类型判断与状态缓存(如键盘按键位图、触摸起始时间与起点),减少无效更新。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
TStart(["TouchEvent 到达"]) --> Split["拆分 changedTouches"]
|
||||
Split --> ForEach["遍历每个触点"]
|
||||
ForEach --> MapPos["指针矫正器映射坐标"]
|
||||
MapPos --> BuildTouchState["构建 TouchState"]
|
||||
BuildTouchState --> Emit["生成状态事件并派发"]
|
||||
Emit --> TEnd(["结束"])
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/sender.js:144-151](file://client/src/sender.js#L144-L151)
|
||||
- [client/src/inputdevice.js:478-517](file://client/src/inputdevice.js#L478-L517)
|
||||
- [client/src/pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
|
||||
章节来源
|
||||
- [client/src/sender.js:130-143](file://client/src/sender.js#L130-L143)
|
||||
- [client/src/inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
|
||||
### 状态一致性保证机制
|
||||
- 设备描述与布局广播:首次连接时发送设备列表与布局信息,确保对端具备相同的输入模型。
|
||||
- 设备变更通知:新增/移除/使用变化等设备状态变更通过专用消息通知,对端据此调整输入映射。
|
||||
- 状态事件结构:状态事件包含基础输入事件头、状态格式与状态数据,对端按格式解析并应用,保证状态一致性。
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class InputEvent {
|
||||
+number type
|
||||
+number sizeInBytes
|
||||
+number deviceId
|
||||
+number time
|
||||
+number eventId
|
||||
+buffer()
|
||||
}
|
||||
class StateEvent {
|
||||
+InputEvent baseEvent
|
||||
+number stateFormat
|
||||
+ArrayBuffer stateData
|
||||
+buffer()
|
||||
}
|
||||
class IInputState {
|
||||
+buffer()
|
||||
+format
|
||||
}
|
||||
class MouseState
|
||||
class KeyboardState
|
||||
class TouchscreenState
|
||||
class GamepadState
|
||||
StateEvent --> InputEvent : "组合"
|
||||
IInputState <|-- MouseState
|
||||
IInputState <|-- KeyboardState
|
||||
IInputState <|-- TouchscreenState
|
||||
IInputState <|-- GamepadState
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/inputdevice.js:129-178](file://client/src/inputdevice.js#L129-L178)
|
||||
- [client/src/inputdevice.js:662-718](file://client/src/inputdevice.js#L662-L718)
|
||||
- [client/src/inputdevice.js:180-193](file://client/src/inputdevice.js#L180-L193)
|
||||
|
||||
章节来源
|
||||
- [client/src/inputremoting.js:108-136](file://client/src/inputremoting.js#L108-L136)
|
||||
- [client/src/inputremoting.js:143-162](file://client/src/inputremoting.js#L143-L162)
|
||||
- [client/src/inputdevice.js:662-718](file://client/src/inputdevice.js#L662-L718)
|
||||
|
||||
### 网络传输优化与消息封装
|
||||
- 自定义消息头:消息包含参与者 ID、类型与长度,数据部分为二进制帧,紧凑且跨语言友好。
|
||||
- 二进制帧格式:输入事件与状态事件均以 DataView/TypedArray 写入,保证小端序与定长字段,降低解析开销。
|
||||
- 信令通道:支持 HTTP 轮询与 WebSocket,前者适合受限环境,后者提供低延迟与广播能力。
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class Message {
|
||||
+number participant_id
|
||||
+number type
|
||||
+number length
|
||||
+ArrayBuffer data
|
||||
+buffer()
|
||||
}
|
||||
class NewDeviceMsg {
|
||||
+create(device)
|
||||
}
|
||||
class NewEventsMsg {
|
||||
+create(event)
|
||||
+createStateEvent(device)
|
||||
}
|
||||
class RemoveDeviceMsg {
|
||||
+create(device)
|
||||
}
|
||||
Message <.. NewDeviceMsg
|
||||
Message <.. NewEventsMsg
|
||||
Message <.. RemoveDeviceMsg
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
- [client/src/inputremoting.js:234-256](file://client/src/inputremoting.js#L234-L256)
|
||||
- [client/src/inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [client/src/inputremoting.js:279-291](file://client/src/inputremoting.js#L279-L291)
|
||||
|
||||
章节来源
|
||||
- [client/src/inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
- [client/src/signaling.js:140-146](file://client/src/signaling.js#L140-L146)
|
||||
- [src/websocket.ts:44-114](file://src/websocket.ts#L44-L114)
|
||||
|
||||
### 缓冲策略、丢包处理与重排序
|
||||
- 缓冲策略:事件以“所见即所得”的方式即时生成并发送,避免大缓冲导致的时延累积;触摸事件按触点拆分,降低单帧体积。
|
||||
- 丢包处理:当前实现未显式实现重传/确认机制;建议在数据通道上启用可靠传输或在应用层引入序列号与请求重发。
|
||||
- 重排序:事件按时间戳顺序应用,若出现乱序,可在对端按时间戳队列进行重排与延迟投递,确保状态一致性。
|
||||
|
||||
章节来源
|
||||
- [client/src/sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
- [client/src/inputdevice.js:149-163](file://client/src/inputdevice.js#L149-L163)
|
||||
|
||||
### 跨设备输入一致性保证
|
||||
- 指针矫正:根据视频宽高与元素显示区域计算 letterbox,将屏幕坐标映射到视频坐标,消除不同设备显示差异。
|
||||
- 设备描述:通过设备消息告知对端设备布局与变体,避免按键映射不一致。
|
||||
- 时间基准:统一使用相对时间戳,结合启动时间校准,减少网络抖动带来的累计误差。
|
||||
|
||||
章节来源
|
||||
- [client/src/pointercorrect.js:78-119](file://client/src/pointercorrect.js#L78-L119)
|
||||
- [client/src/inputremoting.js:117-136](file://client/src/inputremoting.js#L117-L136)
|
||||
|
||||
## 依赖关系分析
|
||||
- 客户端模块内聚度高:输入设备抽象与状态事件解耦,发送器仅负责事件到状态的转换与派发。
|
||||
- 信令与传输解耦:信令层负责连接与消息路由,数据通道负责高吞吐事件传输,职责清晰。
|
||||
- 服务端依赖:WebSocket 信令服务集中处理连接生命周期与消息分发,简化客户端复杂度。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
IR["InputRemoting"] --> S["Sender"]
|
||||
S --> ID["InputDevice/IInputState"]
|
||||
S --> PC["PointerCorrector"]
|
||||
S --> OBS["Observer(RTC 数据通道)"]
|
||||
IR --> WS["WebSocket 信令"]
|
||||
WS --> H["WebSocket 处理器"]
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [client/src/inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [client/src/sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [client/src/inputdevice.js:40-89](file://client/src/inputdevice.js#L40-L89)
|
||||
- [client/src/pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [client/src/signaling.js:152-292](file://client/src/signaling.js#L152-L292)
|
||||
- [src/websocket.ts:6-118](file://src/websocket.ts#L6-L118)
|
||||
|
||||
章节来源
|
||||
- [client/src/signaling.js:152-292](file://client/src/signaling.js#L152-L292)
|
||||
- [src/websocket.ts:6-118](file://src/websocket.ts#L6-L118)
|
||||
|
||||
## 性能考量
|
||||
- 二进制帧优先:事件以 TypedArray 写入,减少 JSON 序列化成本。
|
||||
- 小步快跑:事件即时发送,避免批量堆积;触摸事件拆分,降低单帧大小。
|
||||
- 显示适配:指针矫正一次性计算,避免每帧重复昂贵运算。
|
||||
- 传输层选择:在低延迟要求下优先使用 WebSocket 信令与 RTC 数据通道可靠传输。
|
||||
|
||||
## 故障排查指南
|
||||
- 事件未到达对端:检查 RTC 数据通道状态与 readyState,确认观察者是否正确注册。
|
||||
- 事件乱序或错位:核对时间戳注入逻辑与对端按时间戳重排策略;检查指针矫正参数(视频宽高、元素尺寸)。
|
||||
- 设备映射异常:确认设备消息已发送且对端已更新设备布局;核对按键映射表与手柄按钮位图写入。
|
||||
- 信令不通:验证 WebSocket 信令服务运行状态与消息类型路由;检查 HTTP 轮询会话 ID 与轮询间隔。
|
||||
|
||||
章节来源
|
||||
- [client/src/sender.js:202-208](file://client/src/sender.js#L202-L208)
|
||||
- [client/src/signaling.js:152-292](file://client/src/signaling.js#L152-L292)
|
||||
- [client/test/inputremoting.test.js:35-47](file://client/test/inputremoting.test.js#L35-L47)
|
||||
- [client/test/inputdevice.test.js:123-144](file://client/test/inputdevice.test.js#L123-L144)
|
||||
|
||||
## 结论
|
||||
该输入事件同步方案通过“事件即时生成 + 二进制帧封装 + 相对时间戳 + 指针矫正 + 信令路由”的组合,在保证低延迟的同时实现了跨设备的一致性与可扩展性。为进一步提升鲁棒性,建议在应用层引入序列号、确认与重传机制,并对触摸与键盘事件实施更细粒度的去抖与合并策略。
|
||||
|
||||
## 附录
|
||||
- 测试用例覆盖了消息创建与事件缓冲的基本行为,可作为集成测试基线。
|
||||
- 建议在生产环境中增加心跳、超时与重连策略,以应对网络波动。
|
||||
|
||||
章节来源
|
||||
- [client/test/inputremoting.test.js:50-132](file://client/test/inputremoting.test.js#L50-L132)
|
||||
- [client/test/inputdevice.test.js:16-173](file://client/test/inputdevice.test.js#L16-L173)
|
||||
370
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/鼠标输入处理.md
Normal file
370
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/鼠标输入处理.md
Normal file
@@ -0,0 +1,370 @@
|
||||
# 鼠标输入处理
|
||||
|
||||
<cite>
|
||||
**本文档引用的文件**
|
||||
- [inputremoting.js](file://client/src/inputremoting.js)
|
||||
- [inputdevice.js](file://client/src/inputdevice.js)
|
||||
- [mousebutton.js](file://client/src/mousebutton.js)
|
||||
- [pointercorrect.js](file://client/src/pointercorrect.js)
|
||||
- [sender.js](file://client/src/sender.js)
|
||||
- [memoryhelper.js](file://client/src/memoryhelper.js)
|
||||
- [register-events.js](file://client/public/videoplayer/js/register-events.js)
|
||||
- [main.js(视频播放器)](file://client/public/videoplayer/js/main.js)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [简介](#简介)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考量](#性能考量)
|
||||
8. [故障排查指南](#故障排查指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 简介
|
||||
本文件聚焦于鼠标输入处理的技术文档,涵盖鼠标事件的捕获、转换与转发机制,包括鼠标移动、点击、滚轮等事件类型。文档详细说明鼠标坐标系统、按钮状态映射与事件数据结构,并给出时间戳同步、坐标变换与延迟补偿策略。同时提供具体代码路径示例,展示如何捕获鼠标事件、序列化为二进制消息并通过网络传输,在远端设备上进行重放。
|
||||
|
||||
## 项目结构
|
||||
该仓库包含客户端与服务端两部分。与鼠标输入直接相关的核心代码位于 client/src 与 client/public/videoplayer/js 目录中:
|
||||
- 输入设备建模与事件序列化:client/src/inputdevice.js
|
||||
- 输入远程转发与消息封装:client/src/inputremoting.js
|
||||
- 鼠标按钮常量定义:client/src/mousebutton.js
|
||||
- 坐标矫正与视频适配:client/src/pointercorrect.js
|
||||
- 本地事件监听与状态队列:client/src/sender.js
|
||||
- 内存位写入工具:client/src/memoryhelper.js
|
||||
- 前端事件注册与消息发送(另一套实现):client/public/videoplayer/js/register-events.js
|
||||
- 视频播放器入口与事件注册:client/public/videoplayer/js/main.js
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "客户端"
|
||||
A["事件监听<br/>sender.js"]
|
||||
B["输入设备模型<br/>inputdevice.js"]
|
||||
C["远程转发与消息封装<br/>inputremoting.js"]
|
||||
D["坐标矫正<br/>pointercorrect.js"]
|
||||
E["内存位写入<br/>memoryhelper.js"]
|
||||
F["鼠标按钮常量<br/>mousebutton.js"]
|
||||
G["前端事件注册(替代实现)<br/>register-events.js"]
|
||||
end
|
||||
A --> B
|
||||
A --> D
|
||||
A --> C
|
||||
B --> E
|
||||
B --> F
|
||||
G --> |"注册鼠标/滚轮/触摸事件"| A
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [inputdevice.js:91-272](file://client/src/inputdevice.js#L91-L272)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
|
||||
章节来源
|
||||
- [sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [inputdevice.js:91-272](file://client/src/inputdevice.js#L91-L272)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
|
||||
## 核心组件
|
||||
- LocalInputManager:本地输入管理基类,负责事件分发与时间基准。
|
||||
- Sender:具体实现,注册鼠标/键盘/手柄/触摸事件,构建输入设备状态,派发 StateEvent/TextEvent。
|
||||
- Mouse/MouseState:鼠标设备与状态,包含位置、相对位移、滚轮、按钮位图等字段。
|
||||
- InputRemoting:将本地事件序列化为消息并转发给订阅者(如 RTC 数据通道)。
|
||||
- Message/NewEventsMsg/NewDeviceMsg:消息封装与序列化。
|
||||
- PointerCorrector:将页面坐标映射到视频画布坐标,处理留边距与缩放。
|
||||
- MemoryHelper:按位写入布尔状态,用于按钮位图。
|
||||
- MouseButton:按钮枚举。
|
||||
|
||||
章节来源
|
||||
- [inputremoting.js:9-49](file://client/src/inputremoting.js#L9-L49)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [inputdevice.js:91-272](file://client/src/inputdevice.js#L91-L272)
|
||||
- [sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
|
||||
## 架构总览
|
||||
鼠标事件从 DOM 事件开始,经由 Sender 注册监听,转换为 MouseState,再包装为 StateEvent,通过 InputRemoting 序列化为 Message 并发送至远端。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as "用户"
|
||||
participant DOM as "DOM事件"
|
||||
participant S as "Sender"
|
||||
participant M as "Mouse/MouseState"
|
||||
participant PC as "PointerCorrector"
|
||||
participant IR as "InputRemoting"
|
||||
participant MSG as "Message/NewEventsMsg"
|
||||
participant NET as "网络/数据通道"
|
||||
U->>DOM : "鼠标移动/点击/滚轮"
|
||||
DOM->>S : "事件回调"
|
||||
S->>M : "queueEvent(event)"
|
||||
M-->>S : "currentState"
|
||||
S->>PC : "map(position)"
|
||||
PC-->>S : "矫正后坐标"
|
||||
S->>IR : "派发 StateEvent"
|
||||
IR->>MSG : "NewEventsMsg.create(stateEvent)"
|
||||
MSG-->>IR : "Message"
|
||||
IR->>NET : "发送 buffer"
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [sender.js:121-129](file://client/src/sender.js#L121-L129)
|
||||
- [inputdevice.js:91-98](file://client/src/inputdevice.js#L91-L98)
|
||||
- [inputdevice.js:195-272](file://client/src/inputdevice.js#L195-L272)
|
||||
- [pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### 鼠标事件捕获与转换
|
||||
- 事件监听:在视频元素上注册 click/mousedown/mouseup/mousemove/wheel 回调。
|
||||
- 状态构建:将 MouseEvent/WheelEvent 转换为 MouseState,记录绝对坐标、相对位移、滚轮增量与按钮位图。
|
||||
- 按钮映射:使用位图存储左右中及其他扩展按钮状态,通过 MemoryHelper.writeSingleBit 写入。
|
||||
- 坐标矫正:使用 PointerCorrector 将页面坐标(clientX/Y)映射到视频画布坐标,考虑留边距与缩放。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["事件到达"]) --> Type{"事件类型"}
|
||||
Type --> |mousemove| BuildMove["构建 MouseState<br/>position/delta/buttons"]
|
||||
Type --> |click/mousedown/mouseup| BuildBtn["构建 MouseState<br/>position/buttons"]
|
||||
Type --> |wheel| BuildWheel["构建 MouseState<br/>scroll"]
|
||||
BuildMove --> BtnMap["按钮位图写入"]
|
||||
BuildBtn --> BtnMap
|
||||
BuildWheel --> BtnMap
|
||||
BtnMap --> Correct["PointerCorrector.map(position)"]
|
||||
Correct --> Queue["派发 StateEvent"]
|
||||
Queue --> End(["完成"])
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [sender.js:121-129](file://client/src/sender.js#L121-L129)
|
||||
- [inputdevice.js:195-272](file://client/src/inputdevice.js#L195-L272)
|
||||
- [memoryhelper.js:7-20](file://client/src/memoryhelper.js#L7-L20)
|
||||
- [pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
|
||||
章节来源
|
||||
- [sender.js:31-49](file://client/src/sender.js#L31-L49)
|
||||
- [sender.js:121-129](file://client/src/sender.js#L121-L129)
|
||||
- [inputdevice.js:91-98](file://client/src/inputdevice.js#L91-L98)
|
||||
- [inputdevice.js:195-272](file://client/src/inputdevice.js#L195-L272)
|
||||
- [memoryhelper.js:7-20](file://client/src/memoryhelper.js#L7-L20)
|
||||
- [pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
|
||||
### 鼠标坐标系统与按钮状态
|
||||
- 坐标系统
|
||||
- 绝对坐标:clientX/clientY 来源于 DOM 事件。
|
||||
- 相对位移:movementX/movementY 由浏览器提供;Y 方向取负以匹配常见坐标系。
|
||||
- 留边距与缩放:通过 PointerCorrector 的 contentRect 与 video 尺寸计算映射。
|
||||
- 按钮状态
|
||||
- 使用 16 字节位图存储多按钮状态,左/右/中/前进/后退分别对应不同位。
|
||||
- 通过 MemoryHelper.writeSingleBit 写入位图。
|
||||
|
||||
章节来源
|
||||
- [inputdevice.js:222-244](file://client/src/inputdevice.js#L222-L244)
|
||||
- [inputdevice.js:249-264](file://client/src/inputdevice.js#L249-L264)
|
||||
- [memoryhelper.js:7-20](file://client/src/memoryhelper.js#L7-L20)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
|
||||
### 事件数据结构与时序
|
||||
- InputEvent:统一事件头,包含类型、大小、设备 ID、时间戳。
|
||||
- StateEvent:封装 IInputState,携带状态格式与状态数据。
|
||||
- Message:封装 participant_id、type、length 与 data,作为网络传输单元。
|
||||
- 时间戳:LocalInputManager 提供基于秒的时间基准,Sender 在派发时使用 timeSinceStartup。
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class InputEvent {
|
||||
+number type
|
||||
+number sizeInBytes
|
||||
+number deviceId
|
||||
+number time
|
||||
+buffer
|
||||
}
|
||||
class IInputState {
|
||||
+buffer
|
||||
+format
|
||||
}
|
||||
class MouseState {
|
||||
+float32 position[2]
|
||||
+float32 delta[2]
|
||||
+float32 scroll[2]
|
||||
+uint16 buttons
|
||||
+uint16 displayIndex
|
||||
+uint16 clickCount
|
||||
+buffer
|
||||
+format
|
||||
}
|
||||
class StateEvent {
|
||||
+baseEvent : InputEvent
|
||||
+stateFormat : number
|
||||
+stateData : ArrayBuffer
|
||||
+buffer
|
||||
}
|
||||
class Message {
|
||||
+number participant_id
|
||||
+number type
|
||||
+number length
|
||||
+ArrayBuffer data
|
||||
+buffer
|
||||
}
|
||||
MouseState ..|> IInputState
|
||||
StateEvent --> InputEvent : "组合"
|
||||
Message --> StateEvent : "承载"
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [inputdevice.js:129-178](file://client/src/inputdevice.js#L129-L178)
|
||||
- [inputdevice.js:180-193](file://client/src/inputdevice.js#L180-L193)
|
||||
- [inputdevice.js:195-272](file://client/src/inputdevice.js#L195-L272)
|
||||
- [inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
|
||||
章节来源
|
||||
- [inputdevice.js:129-178](file://client/src/inputdevice.js#L129-L178)
|
||||
- [inputdevice.js:180-193](file://client/src/inputdevice.js#L180-L193)
|
||||
- [inputdevice.js:195-272](file://client/src/inputdevice.js#L195-L272)
|
||||
- [inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
|
||||
### 网络传输与远端重放
|
||||
- 发送侧
|
||||
- InputRemoting 订阅 LocalInputManager 的事件,将 StateEvent 包装为 NewEventsMsg,再封装为 Message。
|
||||
- 通过订阅者(如 RTC 数据通道)发送二进制 buffer。
|
||||
- 接收侧
|
||||
- 远端需解析 Message,识别事件类型与状态格式,重建 MouseState 并应用到目标设备或渲染层。
|
||||
- 由于未在仓库中发现接收与重放的具体实现,建议遵循以下原则:
|
||||
- 使用与发送端一致的格式标识与字段顺序。
|
||||
- 使用时间戳进行本地重放缓冲与插值(见“性能考量”)。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant L as "本地 Sender"
|
||||
participant IR as "InputRemoting"
|
||||
participant CH as "数据通道"
|
||||
participant R as "远端接收器"
|
||||
L->>IR : "StateEvent"
|
||||
IR->>IR : "NewEventsMsg.create()"
|
||||
IR->>CH : "Message.buffer"
|
||||
CH-->>R : "二进制数据"
|
||||
R->>R : "解析格式/重建状态"
|
||||
R-->>R : "重放到目标设备"
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
|
||||
章节来源
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
- [sender.js:170-182](file://client/src/sender.js#L170-L182)
|
||||
|
||||
### 坐标变换与延迟补偿策略
|
||||
- 坐标变换
|
||||
- 将 clientX/Y 减去元素边界矩形偏移,反转 Y 轴,减去留边距,再按内容矩形与视频尺寸比例映射。
|
||||
- 通过 PointerCorrector 的 contentRect 动态计算留边类型(水平/垂直)与尺寸。
|
||||
- 延迟补偿
|
||||
- 使用 timeSinceStartup 作为事件时间戳,远端可据此进行缓冲与插值,减少抖动。
|
||||
- 若网络存在抖动,可在接收端维护一个环形缓冲队列,按时间戳排序后重放。
|
||||
|
||||
章节来源
|
||||
- [pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
- [pointercorrect.js:78-119](file://client/src/pointercorrect.js#L78-L119)
|
||||
- [sender.js:39-41](file://client/src/sender.js#L39-L41)
|
||||
- [sender.js:170-172](file://client/src/sender.js#L170-L172)
|
||||
|
||||
### 另一套前端事件注册实现(对比参考)
|
||||
- register-events.js 提供了另一种前端事件注册方式,同样处理鼠标、滚轮与触摸事件,并将坐标与按钮状态打包为二进制消息发送。
|
||||
- 该实现与 sender.js 的差异在于:
|
||||
- 事件监听对象不同(视频元素 vs 文档/窗口)。
|
||||
- 坐标映射逻辑略有差异(考虑 videoOriginX/Y 与 videoScale)。
|
||||
- 未使用 InputRemoting/Message 体系,而是直接构造 DataView 发送。
|
||||
|
||||
章节来源
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [main.js(视频播放器):132-142](file://client/public/videoplayer/js/main.js#L132-L142)
|
||||
|
||||
## 依赖关系分析
|
||||
- Sender 依赖 Mouse/MouseState、PointerCorrector、LocalInputManager、StateEvent。
|
||||
- MouseState 依赖 MemoryHelper 与 MouseButton。
|
||||
- InputRemoting 依赖 Message/NewEventsMsg/RemoveDeviceMsg/ChangeUsageMsg 与订阅者。
|
||||
- register-events.js 与 main.js 形成独立的事件注册与发送链路。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
Sender["sender.js"] --> Mouse["inputdevice.js: Mouse/MouseState"]
|
||||
Sender --> PC["pointercorrect.js"]
|
||||
Sender --> LIM["inputremoting.js: LocalInputManager"]
|
||||
Sender --> SE["inputdevice.js: StateEvent"]
|
||||
Mouse --> MH["memoryhelper.js"]
|
||||
Mouse --> MB["mousebutton.js"]
|
||||
IR["inputremoting.js"] --> MSG["inputremoting.js: Message/NewEventsMsg"]
|
||||
RE["register-events.js"] --> |"事件注册/发送"| Sender
|
||||
```
|
||||
|
||||
图表来源
|
||||
- [sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [inputdevice.js:91-272](file://client/src/inputdevice.js#L91-L272)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
|
||||
章节来源
|
||||
- [sender.js:14-188](file://client/src/sender.js#L14-L188)
|
||||
- [inputdevice.js:91-272](file://client/src/inputdevice.js#L91-L272)
|
||||
- [inputremoting.js:63-169](file://client/src/inputremoting.js#L63-L169)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
- [mousebutton.js:1-7](file://client/src/mousebutton.js#L1-L7)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
|
||||
## 性能考量
|
||||
- 事件采样与节流
|
||||
- 对高频 mousemove 可采用节流/降采样策略,避免带宽与 CPU 压力。
|
||||
- 时间戳与缓冲
|
||||
- 使用 timeSinceStartup 作为单调递增时间戳,远端按时间戳排序与重放。
|
||||
- 建议在接收端维护环形缓冲,按目标帧率进行插值。
|
||||
- 坐标计算优化
|
||||
- 将 contentRect 缓存并在 resize 时更新,避免重复计算。
|
||||
- 位图写入
|
||||
- 使用 MemoryHelper.writeSingleBit 批量写入按钮状态,减少内存拷贝。
|
||||
|
||||
## 故障排查指南
|
||||
- 事件未触发
|
||||
- 检查事件监听是否绑定到正确的元素(视频元素)。
|
||||
- 确认 PointerCorrector 的 videoWidth/videoHeight 已正确初始化。
|
||||
- 坐标错位
|
||||
- 核对 contentRect 与留边距计算,确认 videoScale 与 originX/Y 设置。
|
||||
- 按钮状态异常
|
||||
- 检查按钮位图写入逻辑,确认 MouseButton 枚举与位索引一致。
|
||||
- 网络发送失败
|
||||
- 确认订阅者 readyState 为 open,Message.buffer 构造无误。
|
||||
|
||||
章节来源
|
||||
- [sender.js:114-120](file://client/src/sender.js#L114-L120)
|
||||
- [pointercorrect.js:71-76](file://client/src/pointercorrect.js#L71-L76)
|
||||
- [memoryhelper.js:7-20](file://client/src/memoryhelper.js#L7-L20)
|
||||
- [inputremoting.js:202-208](file://client/src/inputremoting.js#L202-L208)
|
||||
|
||||
## 结论
|
||||
本项目提供了完整的鼠标输入捕获、转换与转发链路:DOM 事件 → MouseState → StateEvent → Message → 网络传输。通过 PointerCorrector 实现跨设备坐标一致性,MemoryHelper 提供高效的按钮位图写入。若需在远端重放,应严格遵循格式标识与字段顺序,并结合时间戳进行缓冲与插值,以获得流畅体验。
|
||||
|
||||
## 附录
|
||||
- 代码路径示例(不展示具体代码内容)
|
||||
- 鼠标事件监听与状态派发:[sender.js:121-129](file://client/src/sender.js#L121-L129)
|
||||
- MouseState 构建与位图写入:[inputdevice.js:222-244](file://client/src/inputdevice.js#L222-L244)
|
||||
- 坐标矫正:[pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)
|
||||
- 事件序列化为 Message:[inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- 前端事件注册(对比实现):[register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
447
.qoder/repowiki/zh/content/信令系统/信令系统.md
Normal file
447
.qoder/repowiki/zh/content/信令系统/信令系统.md
Normal file
@@ -0,0 +1,447 @@
|
||||
# 信令系统
|
||||
|
||||
<cite>
|
||||
**本文引用的文件**
|
||||
- [src/index.ts](file://src/index.ts)
|
||||
- [src/server.ts](file://src/server.ts)
|
||||
- [src/signaling.ts](file://src/signaling.ts)
|
||||
- [src/websocket.ts](file://src/websocket.ts)
|
||||
- [src/class/websockethandler.ts](file://src/class/websockethandler.ts)
|
||||
- [src/class/httphandler.ts](file://src/class/httphandler.ts)
|
||||
- [src/class/offer.ts](file://src/class/offer.ts)
|
||||
- [src/class/answer.ts](file://src/class/answer.ts)
|
||||
- [src/class/candidate.ts](file://src/class/candidate.ts)
|
||||
- [src/class/options.ts](file://src/class/options.ts)
|
||||
- [src/log.ts](file://src/log.ts)
|
||||
- [src/服务端接口与WebSocket消息类型.md](file://src/服务端接口与WebSocket消息类型.md)
|
||||
- [test/websockethandler.test.ts](file://test/websockethandler.test.ts)
|
||||
- [test/httphandler.test.ts](file://test/httphandler.test.ts)
|
||||
- [package.json](file://package.json)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [简介](#简介)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考虑](#性能考虑)
|
||||
8. [故障排查指南](#故障排查指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 简介
|
||||
本项目是一个支持 HTTP 轮询与 WebSocket 双协议的 WebRTC 信令服务器,提供 Offer/Answer SDP 协商与 ICE 候选者交换能力,并内置公共模式与私有模式两种通信拓扑。系统通过会话(Session)与连接(Connection)抽象实现消息隔离与路由,支持增量拉取、心跳检测与房间广播等特性。
|
||||
|
||||
## 项目结构
|
||||
- 服务启动与配置:应用入口负责解析命令行参数、创建 HTTP/HTTPS 服务器、选择信令协议与通信模式,并初始化 WebSocket 信令服务。
|
||||
- 服务器装配:Express 应用注册日志中间件、CORS、静态资源、Swagger 文档,并挂载 /signaling 路由。
|
||||
- 信令路由:HTTP 路由统一转发至 HTTP 处理器;WebSocket 服务器监听连接事件并分派消息到 WebSocket 处理器。
|
||||
- 数据模型:Offer/Answer/Candidate 作为不可变数据载体,承载 SDP 与 ICE 候选者元数据。
|
||||
- 测试与文档:配套单元测试覆盖公共/私有模式行为与会话超时清理;Markdown 提供完整接口与消息类型规范。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
A["应用入口<br/>src/index.ts"] --> B["Express 服务器<br/>src/server.ts"]
|
||||
B --> C["HTTP 路由<br/>src/signaling.ts"]
|
||||
B --> D["WebSocket 服务器<br/>src/websocket.ts"]
|
||||
C --> E["HTTP 处理器<br/>src/class/httphandler.ts"]
|
||||
D --> F["WebSocket 处理器<br/>src/class/websockethandler.ts"]
|
||||
E --> G["数据模型<br/>offer.ts / answer.ts / candidate.ts"]
|
||||
F --> G
|
||||
A --> H["日志工具<br/>src/log.ts"]
|
||||
A --> I["配置选项<br/>src/class/options.ts"]
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/index.ts:1-109](file://src/index.ts#L1-L109)
|
||||
- [src/server.ts:14-89](file://src/server.ts#L14-L89)
|
||||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||||
- [src/websocket.ts:1-118](file://src/websocket.ts#L1-L118)
|
||||
- [src/class/httphandler.ts:1-120](file://src/class/httphandler.ts#L1-L120)
|
||||
- [src/class/websockethandler.ts:1-66](file://src/class/websockethandler.ts#L1-L66)
|
||||
- [src/class/offer.ts:1-11](file://src/class/offer.ts#L1-L11)
|
||||
- [src/class/answer.ts:1-8](file://src/class/answer.ts#L1-L8)
|
||||
- [src/class/candidate.ts:1-12](file://src/class/candidate.ts#L1-L12)
|
||||
- [src/log.ts:1-51](file://src/log.ts#L1-L51)
|
||||
- [src/class/options.ts:1-10](file://src/class/options.ts#L1-L10)
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:13-109](file://src/index.ts#L13-L109)
|
||||
- [src/server.ts:14-89](file://src/server.ts#L14-L89)
|
||||
- [src/signaling.ts:1-25](file://src/signaling.ts#L1-L25)
|
||||
- [src/websocket.ts:6-118](file://src/websocket.ts#L6-L118)
|
||||
- [src/class/httphandler.ts:1-120](file://src/class/httphandler.ts#L1-L120)
|
||||
- [src/class/websockethandler.ts:1-66](file://src/class/websockethandler.ts#L1-L66)
|
||||
|
||||
## 核心组件
|
||||
- 应用入口与配置
|
||||
- 解析端口、HTTPS、协议类型(websocket/http)、通信模式(public/private)、日志级别等参数。
|
||||
- 根据配置创建 HTTP/HTTPS 服务器,并在 websocket 模式下初始化 WebSocket 信令服务。
|
||||
- Express 服务器
|
||||
- 注册 Morgan 日志、CORS、JSON/URL 编码中间件。
|
||||
- 提供 /config、/signaling 路由、静态资源与 Swagger 文档。
|
||||
- HTTP 信令处理器
|
||||
- 会话管理:创建/删除会话、检查会话 ID、超时清理。
|
||||
- 连接管理:创建/删除连接、维护连接对映射。
|
||||
- SDP 与 ICE:存储 offer/answer/candidate,按模式进行路由与过滤。
|
||||
- 增量拉取:支持 fromtime 参数按时间窗口获取消息。
|
||||
- WebSocket 信令处理器
|
||||
- 会话与连接组:支持公共模式全连通与私有模式 1 对多房间。
|
||||
- 消息路由:connect/disconnect/offer/answer/candidate/broadcast/on-message/call-request/ping/pong。
|
||||
- 心跳检测:可选的 ping/pong 心跳(默认未启用)。
|
||||
- 数据模型
|
||||
- Offer:SDP、时间戳、polite 标志。
|
||||
- Answer:SDP、时间戳。
|
||||
- Candidate:ICE 候选者字符串、sdpMLineIndex、sdpMid、时间戳。
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:14-109](file://src/index.ts#L14-L109)
|
||||
- [src/server.ts:14-89](file://src/server.ts#L14-L89)
|
||||
- [src/class/httphandler.ts:31-120](file://src/class/httphandler.ts#L31-L120)
|
||||
- [src/class/websockethandler.ts:10-66](file://src/class/websockethandler.ts#L10-L66)
|
||||
- [src/class/offer.ts:1-11](file://src/class/offer.ts#L1-L11)
|
||||
- [src/class/answer.ts:1-8](file://src/class/answer.ts#L1-L8)
|
||||
- [src/class/candidate.ts:1-12](file://src/class/candidate.ts#L1-L12)
|
||||
|
||||
## 架构总览
|
||||
系统采用“协议无关”的信令处理层设计:HTTP 与 WebSocket 分别通过各自的处理器对接同一套业务逻辑与数据模型,从而实现公共/私有模式的统一语义。
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "客户端"
|
||||
WS["WebSocket 客户端"]
|
||||
HTTP["HTTP 客户端"]
|
||||
end
|
||||
subgraph "服务器"
|
||||
Srv["Express 服务器"]
|
||||
WSrv["WebSocket 服务器"]
|
||||
HR["HTTP 处理器"]
|
||||
WR["WebSocket 处理器"]
|
||||
DM["数据模型"]
|
||||
end
|
||||
WS --> WSrv --> WR
|
||||
HTTP --> Srv --> HR
|
||||
WR --> DM
|
||||
HR --> DM
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/server.ts:14-29](file://src/server.ts#L14-L29)
|
||||
- [src/websocket.ts:15-118](file://src/websocket.ts#L15-L118)
|
||||
- [src/class/httphandler.ts:107-120](file://src/class/httphandler.ts#L107-L120)
|
||||
- [src/class/websockethandler.ts:63-66](file://src/class/websockethandler.ts#L63-L66)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### WebSocket 信令处理器
|
||||
- 会话与连接组
|
||||
- 使用 Map<WebSocket, Set<string>> 维护每个连接的连接 ID 集合。
|
||||
- 使用 Map<string, ConnectionGroup> 维护连接组:host 与 participants。
|
||||
- 模式差异
|
||||
- 公共模式:任意客户端可向其他所有客户端广播消息。
|
||||
- 私有模式:host 与 participants 之间双向路由,支持单播/广播。
|
||||
- 关键消息处理
|
||||
- connect/disconnect:建立/断开连接,维护连接组与角色(host/participant)。
|
||||
- offer/answer/candidate:根据模式与 participantId 进行路由。
|
||||
- broadcast/on-message:组内广播与点对点消息传递。
|
||||
- ping/pong:心跳检测(可选)。
|
||||
- 广播与路由
|
||||
- host → 所有 participants;participant → host。
|
||||
- 私有模式支持按 participantId 单播。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant C1 as "客户端1"
|
||||
participant C2 as "客户端2"
|
||||
participant WS as "WebSocket服务器"
|
||||
participant WH as "WebSocket处理器"
|
||||
C1->>WS : "connect {connectionId}"
|
||||
WS->>WH : "onConnect(ws, connectionId)"
|
||||
WH-->>C1 : "{type : 'connect', polite : false, role : 'host', participantId}"
|
||||
C2->>WS : "connect {connectionId}"
|
||||
WS->>WH : "onConnect(ws, connectionId)"
|
||||
WH-->>C2 : "{type : 'connect', polite : true, role : 'participant', participantId}"
|
||||
C1->>WS : "offer {connectionId, sdp}"
|
||||
WS->>WH : "onOffer(ws, data)"
|
||||
WH-->>C2 : "{type : 'offer', data : {sdp, connectionId, participantId}, participantId}"
|
||||
C2->>WS : "answer {connectionId, sdp}"
|
||||
WS->>WH : "onAnswer(ws, data)"
|
||||
WH-->>C1 : "{type : 'answer', data : {sdp, connectionId, participantId}, participantId}"
|
||||
C1->>WS : "candidate {connectionId, candidate, sdpMLineIndex, sdpMid}"
|
||||
WS->>WH : "onCandidate(ws, data)"
|
||||
WH-->>C2 : "{type : 'candidate', data : {candidate, sdpMLineIndex, sdpMid, connectionId, participantId}, participantId}"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/websocket.ts:27-115](file://src/websocket.ts#L27-L115)
|
||||
- [src/class/websockethandler.ts:145-338](file://src/class/websockethandler.ts#L145-L338)
|
||||
|
||||
**章节来源**
|
||||
- [src/class/websockethandler.ts:139-338](file://src/class/websockethandler.ts#L139-L338)
|
||||
- [src/websocket.ts:15-118](file://src/websocket.ts#L15-L118)
|
||||
|
||||
### HTTP 信令处理器
|
||||
- 会话与连接
|
||||
- 会话 ID 通过请求头 Session-Id 传递;处理器内部维护 clients、connectionPair、offers、answers、candidates、disconnections 映射。
|
||||
- 私有模式下通过 connectionPair 实现 1 对 1 或 1 对多配对。
|
||||
- 超时与清理
|
||||
- lastRequestedTime 记录会话最后请求时间;超过阈值(10 秒)自动清理会话及其关联资源。
|
||||
- SDP 与 ICE
|
||||
- postOffer/postAnswer/postCandidate 存储消息;getOffer/getAnswer/getCandidate 支持 fromtime 增量拉取。
|
||||
- answer 到来时更新对应连接的 candidate 时间戳,确保时序一致性。
|
||||
- 房间与连接 ID
|
||||
- 提供 /rooms、/connection-ids 等辅助接口,便于监控与调试。
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["HTTP 请求进入"]) --> CheckSession["校验 Session-Id"]
|
||||
CheckSession --> Route{"路由到哪类操作?"}
|
||||
Route --> |创建会话| CreateSession["PUT / -> createSession"]
|
||||
Route --> |删除会话| DeleteSession["DELETE / -> deleteSession"]
|
||||
Route --> |创建连接| CreateConn["PUT /connection -> createConnection"]
|
||||
Route --> |删除连接| DeleteConn["DELETE /connection -> deleteConnection"]
|
||||
Route --> |发送SDP| PostMsg["POST /offer|answer|candidate -> 存储消息"]
|
||||
Route --> |拉取消息| GetMsg["GET /offer|answer|candidate| -> 增量返回"]
|
||||
CreateSession --> End(["返回 sessionId"])
|
||||
DeleteSession --> End
|
||||
CreateConn --> End
|
||||
DeleteConn --> End
|
||||
PostMsg --> End
|
||||
GetMsg --> End
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||||
- [src/class/httphandler.ts:661-696](file://src/class/httphandler.ts#L661-L696)
|
||||
- [src/class/httphandler.ts:739-783](file://src/class/httphandler.ts#L739-L783)
|
||||
- [src/class/httphandler.ts:815-828](file://src/class/httphandler.ts#L815-L828)
|
||||
- [src/class/httphandler.ts:855-886](file://src/class/httphandler.ts#L855-L886)
|
||||
- [src/class/httphandler.ts:913-952](file://src/class/httphandler.ts#L913-L952)
|
||||
- [src/class/httphandler.ts:985-998](file://src/class/httphandler.ts#L985-L998)
|
||||
|
||||
**章节来源**
|
||||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||||
- [src/class/httphandler.ts:218-232](file://src/class/httphandler.ts#L218-L232)
|
||||
- [src/class/httphandler.ts:274-297](file://src/class/httphandler.ts#L274-L297)
|
||||
- [src/class/httphandler.ts:305-318](file://src/class/httphandler.ts#L305-L318)
|
||||
- [src/class/httphandler.ts:326-356](file://src/class/httphandler.ts#L326-L356)
|
||||
- [src/class/httphandler.ts:661-696](file://src/class/httphandler.ts#L661-L696)
|
||||
- [src/class/httphandler.ts:739-783](file://src/class/httphandler.ts#L739-L783)
|
||||
- [src/class/httphandler.ts:815-828](file://src/class/httphandler.ts#L815-L828)
|
||||
- [src/class/httphandler.ts:855-886](file://src/class/httphandler.ts#L855-L886)
|
||||
- [src/class/httphandler.ts:913-952](file://src/class/httphandler.ts#L913-L952)
|
||||
- [src/class/httphandler.ts:985-998](file://src/class/httphandler.ts#L985-L998)
|
||||
|
||||
### Offer/Answer SDP 协商与 ICE 候选者交换
|
||||
- Offer/Answer
|
||||
- 公共模式:Peer → 所有其他 Peers。
|
||||
- 私有模式:Host ↔ Participants;Host 可单播给特定 participant 或广播给所有 participants。
|
||||
- Candidate
|
||||
- 与 answer 路由规则一致,支持按 participantId 单播。
|
||||
- Polite 标志
|
||||
- 私有模式下:host 为 `polite=false`,participants 为 `polite=true`,避免并发 offer 冲突。
|
||||
- 增量拉取
|
||||
- HTTP 模式支持 fromtime 参数,客户端可增量获取消息,减少网络与存储压力。
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant P1 as "参与者1"
|
||||
participant P2 as "参与者2"
|
||||
participant S as "信令服务器"
|
||||
P1->>S : "POST /signaling/offer {connectionId, sdp}"
|
||||
S-->>P2 : "GET /signaling/offer?fromtime=... -> {offers : [{connectionId,sdp,polite,...}]}"
|
||||
P2->>S : "POST /signaling/answer {connectionId, sdp}"
|
||||
S-->>P1 : "GET /signaling/answer?fromtime=... -> {answers : [{connectionId,sdp,...}]}"
|
||||
loop "ICE 候选者收集"
|
||||
P1->>S : "POST /signaling/candidate {connectionId, candidate, sdpMLineIndex, sdpMid}"
|
||||
S-->>P2 : "GET /signaling/candidate?fromtime=... -> {candidates : [{connectionId,candidate,sdpMLineIndex,sdpMid,...}]}"
|
||||
P2->>S : "POST /signaling/candidate {connectionId, candidate, sdpMLineIndex, sdpMid}"
|
||||
S-->>P1 : "GET /signaling/candidate?fromtime=... -> {candidates : [{connectionId,candidate,sdpMLineIndex,sdpMid,...}]}"
|
||||
end
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/class/httphandler.ts:274-297](file://src/class/httphandler.ts#L274-L297)
|
||||
- [src/class/httphandler.ts:305-318](file://src/class/httphandler.ts#L305-L318)
|
||||
- [src/class/httphandler.ts:326-356](file://src/class/httphandler.ts#L326-L356)
|
||||
- [src/class/httphandler.ts:855-886](file://src/class/httphandler.ts#L855-L886)
|
||||
- [src/class/httphandler.ts:913-952](file://src/class/httphandler.ts#L913-L952)
|
||||
- [src/class/httphandler.ts:985-998](file://src/class/httphandler.ts#L985-L998)
|
||||
|
||||
**章节来源**
|
||||
- [src/class/httphandler.ts:274-356](file://src/class/httphandler.ts#L274-L356)
|
||||
- [src/class/httphandler.ts:855-998](file://src/class/httphandler.ts#L855-L998)
|
||||
|
||||
### 连接组管理机制(公共模式 vs 私有模式)
|
||||
- 公共模式
|
||||
- 任意客户端可向其他所有客户端广播消息。
|
||||
- 适合全连通场景,消息广播范围广但复杂度低。
|
||||
- 私有模式
|
||||
- 以 connectionId 为房间标识,host 与 participants 之间双向路由。
|
||||
- 支持单播与广播;host 离开房间即解散,participant 离开房间保留。
|
||||
- 通过 participantId 区分发送者身份,便于路由与 UI 展示。
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class WebSocket处理器 {
|
||||
+reset(mode)
|
||||
+add(ws)
|
||||
+remove(ws)
|
||||
+onConnect(ws, connectionId)
|
||||
+onDisconnect(ws, connectionId)
|
||||
+onOffer(ws, message)
|
||||
+onAnswer(ws, message)
|
||||
+onCandidate(ws, message)
|
||||
+onBroadcast(ws, message)
|
||||
+onMessage(ws, message)
|
||||
}
|
||||
class 连接组 {
|
||||
+host : WebSocket
|
||||
+participants : Set~WebSocket~
|
||||
}
|
||||
WebSocket处理器 --> 连接组 : "维护/路由"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/class/websockethandler.ts:63-66](file://src/class/websockethandler.ts#L63-L66)
|
||||
- [src/class/websockethandler.ts:27-37](file://src/class/websockethandler.ts#L27-L37)
|
||||
|
||||
**章节来源**
|
||||
- [src/class/websockethandler.ts:139-338](file://src/class/websockethandler.ts#L139-L338)
|
||||
|
||||
### 信令消息格式规范
|
||||
- HTTP REST API
|
||||
- /signaling、/signaling/offer、/signaling/answer、/signaling/candidate、/signaling/connection、/signaling/connection-ids、/signaling/rooms 等接口的请求与响应结构详见接口文档。
|
||||
- WebSocket 消息类型
|
||||
- connect/disconnect/participant-joined/participant-left:连接生命周期消息。
|
||||
- offer/answer/candidate:SDP 与 ICE 候选者消息,支持 participantId 与单播路由。
|
||||
- on-message:通用消息传递,支持文本/对象。
|
||||
- broadcast:组内广播或全局广播。
|
||||
- call-request:发起呼叫请求。
|
||||
- ping/pong:心跳检测(可选)。
|
||||
|
||||
**章节来源**
|
||||
- [src/服务端接口与WebSocket消息类型.md:35-260](file://src/服务端接口与WebSocket消息类型.md#L35-L260)
|
||||
- [src/服务端接口与WebSocket消息类型.md:262-487](file://src/服务端接口与WebSocket消息类型.md#L262-L487)
|
||||
|
||||
### 错误处理策略与重连机制
|
||||
- 会话超时
|
||||
- HTTP 模式下,若 10 秒内无请求,自动清理会话及其资源,避免内存泄漏。
|
||||
- 会话校验
|
||||
- HTTP 路由中间件检查 Session-Id,不存在则返回 404。
|
||||
- 私有模式约束
|
||||
- 连接 ID 在私有模式下需唯一配对;重复使用会返回 400。
|
||||
- 重连建议
|
||||
- 客户端应在断开后重新创建会话(HTTP)或重新建立 WebSocket 连接(WebSocket),并再次执行 connect 流程。
|
||||
- 增量拉取 fromtime 可帮助客户端恢复丢失的消息。
|
||||
|
||||
**章节来源**
|
||||
- [src/class/httphandler.ts:218-232](file://src/class/httphandler.ts#L218-L232)
|
||||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||||
- [src/class/httphandler.ts:362-368](file://src/class/httphandler.ts#L362-L368)
|
||||
|
||||
### 实际信令流程示例与调试技巧
|
||||
- 公共模式典型流程
|
||||
- 客户端 A/B 同时 connect 同一 connectionId,服务器广播 offer/answer/candidate。
|
||||
- 增量拉取 fromtime 可避免重复处理。
|
||||
- 私有模式典型流程
|
||||
- 客户端 A 作为 host,B/C 作为 participants;A 发送 offer 给 B,B 回答后 A 再发送 answer 给 B;ICE 候选者双向交换。
|
||||
- 调试技巧
|
||||
- 使用 /config 检查服务器配置(是否启用 WebSocket、启动模式、日志级别)。
|
||||
- 使用 /signaling/connection-ids 查看活跃连接 ID。
|
||||
- 使用 /signaling/rooms 查看房间与用户状态。
|
||||
- 使用 /signaling?fromtime=... 增量拉取消息,定位异常时间窗。
|
||||
- 开启更详细日志级别以观察消息流转。
|
||||
|
||||
**章节来源**
|
||||
- [src/server.ts:25-41](file://src/server.ts#L25-L41)
|
||||
- [src/class/httphandler.ts:1041-1076](file://src/class/httphandler.ts#L1041-L1076)
|
||||
- [src/class/httphandler.ts:1103-1108](file://src/class/httphandler.ts#L1103-L1108)
|
||||
- [src/服务端接口与WebSocket消息类型.md:508-542](file://src/服务端接口与WebSocket消息类型.md#L508-L542)
|
||||
|
||||
## 依赖关系分析
|
||||
- 外部依赖
|
||||
- Express、ws、cors、morgan、multer、uuid、swagger 等。
|
||||
- 内部模块
|
||||
- index.ts 依赖 server.ts、websocket.ts、websockethandler.ts、httphandler.ts。
|
||||
- server.ts 依赖 signaling.ts、httphandler.ts。
|
||||
- signaling.ts 依赖 httphandler.ts。
|
||||
- websockethandler.ts 依赖 offer.ts、answer.ts、candidate.ts、log.ts。
|
||||
- httphandler.ts 依赖 offer.ts、answer.ts、candidate.ts、log.ts。
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
IDX["index.ts"] --> SRV["server.ts"]
|
||||
IDX --> WSS["websocket.ts"]
|
||||
SRV --> SIG["signaling.ts"]
|
||||
SIG --> HTH["httphandler.ts"]
|
||||
WSS --> WSH["websockethandler.ts"]
|
||||
WSH --> OFF["offer.ts"]
|
||||
WSH --> ANS["answer.ts"]
|
||||
WSH --> CAN["candidate.ts"]
|
||||
HTH --> OFF
|
||||
HTH --> ANS
|
||||
HTH --> CAN
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [src/index.ts:7-10](file://src/index.ts#L7-L10)
|
||||
- [src/server.ts:1-12](file://src/server.ts#L1-L12)
|
||||
- [src/signaling.ts:1-3](file://src/signaling.ts#L1-L3)
|
||||
- [src/websocket.ts:1-4](file://src/websocket.ts#L1-L4)
|
||||
- [src/class/websockethandler.ts:5-8](file://src/class/websockethandler.ts#L5-L8)
|
||||
- [src/class/httphandler.ts:5-11](file://src/class/httphandler.ts#L5-L11)
|
||||
|
||||
**章节来源**
|
||||
- [package.json:14-27](file://package.json#L14-L27)
|
||||
- [src/index.ts:1-12](file://src/index.ts#L1-L12)
|
||||
- [src/server.ts:1-12](file://src/server.ts#L1-L12)
|
||||
- [src/signaling.ts:1-3](file://src/signaling.ts#L1-L3)
|
||||
- [src/websocket.ts:1-4](file://src/websocket.ts#L1-L4)
|
||||
- [src/class/websockethandler.ts:5-8](file://src/class/websockethandler.ts#L5-L8)
|
||||
- [src/class/httphandler.ts:5-11](file://src/class/httphandler.ts#L5-L11)
|
||||
|
||||
## 性能考虑
|
||||
- HTTP 轮询
|
||||
- fromtime 增量拉取减少无效数据传输。
|
||||
- 会话超时清理避免长期占用内存。
|
||||
- WebSocket
|
||||
- 按组广播与单播路由降低广播风暴。
|
||||
- 可选心跳检测防止长连接空闲断开。
|
||||
- 存储与序列化
|
||||
- 使用轻量级对象承载 SDP 与 ICE 元数据,避免冗余字段。
|
||||
- 建议客户端侧缓存最近消息,减少重复请求。
|
||||
|
||||
## 故障排查指南
|
||||
- 无法获取会话
|
||||
- 确认请求头是否包含正确的 Session-Id。
|
||||
- 检查会话是否因超时被清理。
|
||||
- 私有模式连接冲突
|
||||
- 确认 connectionId 未被重复使用。
|
||||
- 检查 host/participant 角色与 polite 标志。
|
||||
- 消息未到达
|
||||
- 使用 /signaling?fromtime=... 增量拉取确认消息是否已存储。
|
||||
- 检查模式配置(public/private)与路由规则。
|
||||
- 日志与监控
|
||||
- 提升日志级别以观察消息流转。
|
||||
- 使用 /signaling/rooms 与 /signaling/connection-ids 快速定位问题。
|
||||
|
||||
**章节来源**
|
||||
- [src/class/httphandler.ts:128-145](file://src/class/httphandler.ts#L128-L145)
|
||||
- [src/class/httphandler.ts:218-232](file://src/class/httphandler.ts#L218-L232)
|
||||
- [src/class/httphandler.ts:362-368](file://src/class/httphandler.ts#L362-L368)
|
||||
- [src/log.ts:15-24](file://src/log.ts#L15-L24)
|
||||
|
||||
## 结论
|
||||
本信令系统通过统一的数据模型与路由策略,在 HTTP 与 WebSocket 两种协议下实现了公共/私有两种通信模式的一致语义。Offer/Answer 与 ICE 候选者交换流程清晰,配合增量拉取与会话超时清理,满足实时性与稳定性需求。建议在生产环境中结合日志与监控接口进行持续观测与优化。
|
||||
|
||||
## 附录
|
||||
- 启动参数与脚本
|
||||
- 通过命令行参数或环境变量配置端口、HTTPS、协议类型、通信模式与日志级别。
|
||||
- 提供 npm scripts 用于开发与打包。
|
||||
|
||||
**章节来源**
|
||||
- [src/index.ts:20-42](file://src/index.ts#L20-L42)
|
||||
- [package.json:5-13](file://package.json#L5-L13)
|
||||
File diff suppressed because one or more lines are too long
Reference in New Issue
Block a user