348 lines
16 KiB
Markdown
348 lines
16 KiB
Markdown
# 接收端示例
|
||
|
||
<cite>
|
||
**本文引用的文件**
|
||
- [client/public/receiver/js/main.js](file://client/public/receiver/js/main.js)
|
||
- [client/public/receiver/index.html](file://client/public/receiver/index.html)
|
||
- [client/public/receiver/css/style.css](file://client/public/receiver/css/style.css)
|
||
- [client/src/renderstreaming.js](file://client/src/renderstreaming.js)
|
||
- [client/src/peer.js](file://client/src/peer.js)
|
||
- [client/src/signaling.js](file://client/src/signaling.js)
|
||
- [client/public/js/config.js](file://client/public/js/config.js)
|
||
- [client/public/js/videoplayer.js](file://client/public/js/videoplayer.js)
|
||
- [client/public/js/stats.js](file://client/public/js/stats.js)
|
||
- [client/public/js/icesettings.js](file://client/public/js/icesettings.js)
|
||
- [package.json](file://package.json)
|
||
</cite>
|
||
|
||
## 目录
|
||
1. [简介](#简介)
|
||
2. [项目结构](#项目结构)
|
||
3. [核心组件](#核心组件)
|
||
4. [架构总览](#架构总览)
|
||
5. [详细组件分析](#详细组件分析)
|
||
6. [依赖关系分析](#依赖关系分析)
|
||
7. [性能考虑](#性能考虑)
|
||
8. [故障排查指南](#故障排查指南)
|
||
9. [结论](#结论)
|
||
10. [附录](#附录)
|
||
|
||
## 简介
|
||
本指南面向使用 Unity Render Streaming 的接收端示例，帮助您在浏览器中完成以下目标：
|
||
- 连接到服务器，建立 WebRTC 媒体会话
|
||
- 接收并解码视频/音频流
|
||
- 将媒体轨道渲染到页面中的视频元素
|
||
- 处理输入通道（鼠标/键盘/触摸/手柄）以实现远程输入遥测
|
||
- 实时查看统计信息，辅助调试与性能评估
|
||
- 提供性能优化建议与兼容性注意事项
|
||
- 给出典型使用场景与配置示例
|
||
|
||
## 项目结构
|
||
接收端示例位于 client/public/receiver 目录，前端采用模块化 JavaScript，配合自定义的 RenderStreaming、Peer、Signaling 模块实现 WebRTC 信令与媒体处理。
|
||
|
||
```mermaid
|
||
graph TB
|
||
subgraph "接收端页面"
|
||
HTML["index.html"]
|
||
CSS["css/style.css"]
|
||
JSMain["receiver/js/main.js"]
|
||
JSConfig["public/js/config.js"]
|
||
JSStats["public/js/stats.js"]
|
||
JSVideo["public/js/videoplayer.js"]
|
||
end
|
||
subgraph "核心模块"
|
||
RRS["src/renderstreaming.js"]
|
||
PEER["src/peer.js"]
|
||
SIG["src/signaling.js"]
|
||
end
|
||
HTML --> JSMain
|
||
HTML --> CSS
|
||
JSMain --> JSConfig
|
||
JSMain --> JSStats
|
||
JSMain --> JSVideo
|
||
JSMain --> RRS
|
||
RRS --> PEER
|
||
RRS --> SIG
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/receiver/index.html:1-54](file://client/public/receiver/index.html#L1-L54)
|
||
- [client/public/receiver/js/main.js:1-186](file://client/public/receiver/js/main.js#L1-L186)
|
||
- [client/public/receiver/css/style.css:1-43](file://client/public/receiver/css/style.css#L1-L43)
|
||
- [client/src/renderstreaming.js:1-317](file://client/src/renderstreaming.js#L1-L317)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
|
||
章节来源
|
||
- [client/public/receiver/index.html:1-54](file://client/public/receiver/index.html#L1-L54)
|
||
- [client/public/receiver/js/main.js:1-186](file://client/public/receiver/js/main.js#L1-L186)
|
||
- [client/public/receiver/css/style.css:1-43](file://client/public/receiver/css/style.css#L1-L43)
|
||
- [client/src/renderstreaming.js:1-317](file://client/src/renderstreaming.js#L1-L317)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
|
||
## 核心组件
|
||
- 页面与入口
|
||
- index.html：页面骨架、样式与脚本加载入口
|
||
- receiver/js/main.js：应用主流程，负责初始化、信令选择、RTCPeerConnection 配置、媒体轨道接入与统计展示
|
||
- 媒体播放器
|
||
- public/js/videoplayer.js：封装视频元素创建、全屏切换、输入通道绑定与媒体轨道追加
|
||
- 信令层
|
||
- src/signaling.js：HTTP 与 WebSocket 两种信令实现，负责连接生命周期、offer/answer/candidate 转发
|
||
- 会话与媒体
|
||
- src/renderstreaming.js：高层封装，管理连接、多 peer、事件路由、统计查询、数据通道创建
|
||
- src/peer.js：底层 RTCPeerConnection 管理，包含协商、ICE、统计查询、轨道与数据通道操作
|
||
- 配置与工具
|
||
- public/js/config.js：读取服务器配置、生成 RTC 配置（含 STUN/TURN、音频增强）
|
||
- public/js/stats.js：从 RTCStatsReport 生成可读字符串数组
|
||
- public/js/icesettings.js：STUN/TURN 配置持久化与读取
|
||
|
||
章节来源
|
||
- [client/public/receiver/index.html:1-54](file://client/public/receiver/index.html#L1-L54)
|
||
- [client/public/receiver/js/main.js:1-186](file://client/public/receiver/js/main.js#L1-L186)
|
||
- [client/public/js/videoplayer.js:1-213](file://client/public/js/videoplayer.js#L1-L213)
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
- [client/src/renderstreaming.js:1-317](file://client/src/renderstreaming.js#L1-L317)
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
- [client/public/js/config.js:1-39](file://client/public/js/config.js#L1-L39)
|
||
- [client/public/js/stats.js:1-91](file://client/public/js/stats.js#L1-L91)
|
||
- [client/public/js/icesettings.js:1-104](file://client/public/js/icesettings.js#L1-L104)
|
||
|
||
## 架构总览
|
||
下图展示了从用户点击“开始”到媒体渲染的关键交互路径，以及输入通道的建立流程。
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant U as "用户"
|
||
participant Page as "页面(main.js)"
|
||
participant RS as "RenderStreaming"
|
||
participant Sig as "Signaling(HTTP/WebSocket)"
|
||
participant P as "Peer(RTCPeerConnection)"
|
||
participant VP as "VideoPlayer"
|
||
U->>Page : 点击“开始”
|
||
Page->>VP : 创建视频容器与播放器
|
||
Page->>Sig : 选择信令类型并创建连接
|
||
Page->>RS : 初始化并启动
|
||
RS->>P : 创建/准备 Peer
|
||
P-->>RS : 触发 onnegotiationneeded
|
||
RS->>Sig : 发送 offer
|
||
Sig-->>RS : 下行 offer
|
||
RS->>P : 设置远端 offer 并生成 answer
|
||
RS->>Sig : 发送 answer
|
||
Sig-->>RS : 下行 answer
|
||
P-->>RS : 触发 ontrack
|
||
RS-->>Page : 分发 track 事件
|
||
Page->>VP : 追加轨道到视频元素
|
||
Page->>VP : 建立输入数据通道并绑定
|
||
Page->>Page : 定时获取统计并展示
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/receiver/js/main.js:67-108](file://client/public/receiver/js/main.js#L67-L108)
|
||
- [client/src/renderstreaming.js:182-250](file://client/src/renderstreaming.js#L182-L250)
|
||
- [client/src/peer.js:57-173](file://client/src/peer.js#L57-L173)
|
||
- [client/src/signaling.js:30-149](file://client/src/signaling.js#L30-L149)
|
||
- [client/public/js/videoplayer.js:19-41](file://client/public/js/videoplayer.js#L19-L41)
|
||
|
||
## 详细组件分析
|
||
|
||
### 页面与入口：main.js
|
||
- 初始化与页面控制
|
||
- 读取服务器配置，决定是否使用 WebSocket 信令
|
||
- 显示警告（如私有模式限制）、编解码器选择下拉框、播放按钮
|
||
- 点击播放后创建视频播放器并初始化 RenderStreaming
|
||
- RTCPeerConnection 配置
|
||
- 通过 config.js 获取 RTC 配置，包含 sdpSemantics、iceServers、媒体约束与音频增强参数
|
||
- 事件与生命周期
|
||
- onConnect：创建输入数据通道，绑定 VideoPlayer 输入遥测
|
||
- onDisconnect：清理统计、重置 UI、重建播放器与编解码器选择
|
||
- onTrackEvent：将轨道追加到视频元素
|
||
- onGotOffer：根据浏览器能力设置编解码器偏好
|
||
- 统计展示
|
||
- 定时调用 RenderStreaming.getStats，使用 stats.js 生成可读字符串并显示
|
||
|
||
章节来源
|
||
- [client/public/receiver/js/main.js:1-186](file://client/public/receiver/js/main.js#L1-L186)
|
||
- [client/public/js/config.js:1-39](file://client/public/js/config.js#L1-L39)
|
||
- [client/public/js/stats.js:1-91](file://client/public/js/stats.js#L1-L91)
|
||
|
||
### 媒体播放器：VideoPlayer
|
||
- 视频元素与布局
|
||
- 创建 video 元素，设置 playsInline、空 MediaStream，监听元数据加载完成自动播放
|
||
- 提供全屏按钮与全屏状态切换逻辑，支持指针锁定（可选）
|
||
- 轨道与尺寸
|
||
- addTrack：向 video.srcObject 追加媒体轨道
|
||
- resizeVideo：计算缩放与偏移，适配容器尺寸
|
||
- 输入遥测
|
||
- setupInput：创建 Sender/InputRemoting，订阅 RTCDataChannel，打开后开始发送输入事件
|
||
- 支持鼠标、键盘、触摸屏、手柄输入的转发
|
||
|
||
章节来源
|
||
- [client/public/js/videoplayer.js:1-213](file://client/public/js/videoplayer.js#L1-L213)
|
||
|
||
### 信令层：Signaling（HTTP 与 WebSocket）
|
||
- HTTP 信令
|
||
- start：轮询获取消息队列，分发 connect/disconnect/offer/answer/candidate/on-message
|
||
- createConnection/deleteConnection：创建/删除连接
|
||
- sendOffer/sendAnswer/sendCandidate/sendMessage：发送 SDP 与消息
|
||
- WebSocket 信令
|
||
- onopen/onmessage/onclose：解析消息类型并分发事件
|
||
- createConnection/deleteConnection/sendOffer/sendAnswer/sendCandidate/sendMessage：与 HTTP 版本对应
|
||
|
||
章节来源
|
||
- [client/src/signaling.js:1-292](file://client/src/signaling.js#L1-L292)
|
||
|
||
### 会话与媒体：RenderStreaming
|
||
- 生命周期与角色
|
||
- 支持 Host/Participant 双端模式；Participant 端默认单 peer，Host 端按参与者维护多 peer
|
||
- 事件路由
|
||
- 将 Peer 的 track/adddatachannel/offer/answer/candidate 等事件上抛，并携带 participantId
|
||
- 关键方法
|
||
- createConnection/start/stop：连接生命周期管理
|
||
- getStats/createDataChannel/addTrack/addTransceiver/getTransceivers：媒体与统计接口
|
||
- 与 Signaling 事件绑定，完成 offer/answer/candidate 的收发
|
||
|
||
章节来源
|
||
- [client/src/renderstreaming.js:1-317](file://client/src/renderstreaming.js#L1-L317)
|
||
|
||
### 底层会话：Peer（RTCPeerConnection）
|
||
- 协商与 ICE
|
||
- onnegotiationneeded：生成本地 offer 并通过事件上报
|
||
- onGotDescription：设置远端描述，若为 offer 则生成 answer 并上报
|
||
- onIceCandidate：上报候选
|
||
- 状态与错误
|
||
- iceConnectionState=failed 时触发 disconnect 事件
|
||
- 工具方法
|
||
- getTransceivers/addTrack/addTransceiver/createDataChannel/getStats/close：媒体与统计操作
|
||
|
||
章节来源
|
||
- [client/src/peer.js:1-188](file://client/src/peer.js#L1-L188)
|
||
|
||
### 配置与工具
|
||
- config.js
|
||
- getServerConfig：读取服务器配置（含 useWebSocket）
|
||
- getRTCConfiguration：统一计划 SDP、ICE 服务器、媒体约束与音频增强参数
|
||
- icesettings.js
|
||
- STUN/TURN 服务器的增删改查与本地存储
|
||
- stats.js
|
||
- 从 inbound/outbound RTP 统计生成可读字符串，包含编解码器、分辨率、帧率、比特率等
|
||
|
||
章节来源
|
||
- [client/public/js/config.js:1-39](file://client/public/js/config.js#L1-L39)
|
||
- [client/public/js/icesettings.js:1-104](file://client/public/js/icesettings.js#L1-L104)
|
||
- [client/public/js/stats.js:1-91](file://client/public/js/stats.js#L1-L91)
|
||
|
||
## 依赖关系分析
|
||
- 模块耦合
|
||
- main.js 依赖 renderstreaming.js、videoplayer.js、config.js、stats.js
|
||
- renderstreaming.js 依赖 peer.js 与 signaling.js
|
||
- peer.js 仅依赖 logger.js（内部日志）
|
||
- 外部依赖
|
||
- 浏览器原生 WebRTC API（RTCPeerConnection、MediaStream、RTCIceServer 等）
|
||
- 第三方 polyfill/适配库（页面引入了 webrtc-adapter）
|
||
|
||
```mermaid
|
||
graph LR
|
||
MAIN["main.js"] --> CFG["config.js"]
|
||
MAIN --> STATS["stats.js"]
|
||
MAIN --> VP["videoplayer.js"]
|
||
MAIN --> RRS["renderstreaming.js"]
|
||
RRS --> PEER["peer.js"]
|
||
RRS --> SIG["signaling.js"]
|
||
VP --> INPUT["inputremoting.js(sender.js)"]
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/receiver/js/main.js:1-10](file://client/public/receiver/js/main.js#L1-L10)
|
||
- [client/src/renderstreaming.js:1-30](file://client/src/renderstreaming.js#L1-L30)
|
||
- [client/src/peer.js:1-10](file://client/src/peer.js#L1-L10)
|
||
- [client/src/signaling.js:1-10](file://client/src/signaling.js#L1-L10)
|
||
- [client/public/js/videoplayer.js:1-3](file://client/public/js/videoplayer.js#L1-L3)
|
||
|
||
章节来源
|
||
- [client/public/receiver/js/main.js:1-10](file://client/public/receiver/js/main.js#L1-L10)
|
||
- [client/src/renderstreaming.js:1-30](file://client/src/renderstreaming.js#L1-L30)
|
||
- [client/src/peer.js:1-10](file://client/src/peer.js#L1-L10)
|
||
- [client/src/signaling.js:1-10](file://client/src/signaling.js#L1-L10)
|
||
- [client/public/js/videoplayer.js:1-3](file://client/public/js/videoplayer.js#L1-L3)
|
||
|
||
## 性能考虑
|
||
- 编解码器选择
|
||
- 若浏览器支持 setCodecPreferences，可在 offer 生成前设置偏好，减少不必要解码开销
|
||
- 统计监控
|
||
- 使用 stats.js 输出的比特率、分辨率、帧率等指标，结合网络状况动态调整
|
||
- 媒体约束与音频增强
|
||
- 合理配置音频回声消除、降噪、自动增益等参数，降低 CPU 占用与提升音质
|
||
- 渲染与布局
|
||
- 使用 videoplayer.js 的 resizeVideo 计算缩放，避免不必要的重排
|
||
- 信令选择
|
||
- WebSocket 信令延迟更低，适合低延迟场景；HTTP 轮询更通用，兼容性更好
|
||
|
||
[本节为通用指导，无需列出具体文件来源]
|
||
|
||
## 故障排查指南
|
||
- 私有模式限制
|
||
- 页面会在私有模式下提示警告，导致部分功能不可用
|
||
- 无法播放或黑屏
|
||
- 检查 onConnect 是否触发、track 事件是否到达、VideoPlayer 是否成功追加轨道
|
||
- 确认浏览器是否支持所选编解码器，必要时切换默认或手动选择
|
||
- 无声音
|
||
- 确认音频媒体轨道已加入，检查音频增强参数与浏览器静音策略
|
||
- 输入无响应
|
||
- 确认数据通道已创建并打开，VideoPlayer.setupInput 是否被调用
|
||
- 连接失败或频繁断开
|
||
- 查看 ICE 候选是否正常收发，STUN/TURN 配置是否正确
|
||
- 关注 Peer 的 iceConnectionState 与 signalingState 变化
|
||
|
||
章节来源
|
||
- [client/public/receiver/js/main.js:48-54](file://client/public/receiver/js/main.js#L48-L54)
|
||
- [client/public/js/videoplayer.js:194-212](file://client/public/js/videoplayer.js#L194-L212)
|
||
- [client/src/peer.js:43-48](file://client/src/peer.js#L43-L48)
|
||
- [client/public/js/icesettings.js:94-104](file://client/public/js/icesettings.js#L94-L104)
|
||
|
||
## 结论
|
||
该接收端示例通过清晰的模块划分与事件驱动设计，实现了从信令、协商、媒体轨道接入到播放器渲染与输入遥测的完整链路。借助统计模块与编解码器偏好设置，开发者可以快速定位问题并优化体验。建议在生产环境中优先使用 WebSocket 信令、合理配置 ICE 服务器与音频增强，并持续监控统计指标以保障质量。
|
||
|
||
[本节为总结性内容，无需列出具体文件来源]
|
||
|
||
## 附录
|
||
|
||
### 使用步骤（从零到运行）
|
||
- 启动服务端
|
||
- 使用 package.json 中的脚本启动服务端，默认端口与证书参数可参考脚本配置
|
||
- 打开接收端页面
|
||
- 在浏览器中访问接收端页面，点击“开始”
|
||
- 选择信令方式
|
||
- 若服务器返回 useWebSocket=true，则使用 WebSocket 信令；否则使用 HTTP 信令
|
||
- 选择编解码器（可选）
|
||
- 若浏览器支持 setCodecPreferences，可在下拉框中选择偏好编解码器
|
||
- 观察统计与播放
|
||
- 页面会定时显示统计信息；视频元素加载完成后自动播放
|
||
- 输入遥测（可选）
|
||
- 勾选“锁定光标到播放器”，进入全屏后可进行指针锁定与输入转发
|
||
|
||
章节来源
|
||
- [package.json:9-10](file://package.json#L9-L10)
|
||
- [client/public/receiver/js/main.js:40-88](file://client/public/receiver/js/main.js#L40-L88)
|
||
- [client/public/js/config.js:3-7](file://client/public/js/config.js#L3-L7)
|
||
- [client/public/js/stats.js:7-91](file://client/public/js/stats.js#L7-L91)
|
||
|
||
### 关键流程图：编解码器偏好设置
|
||
```mermaid
|
||
flowchart TD
|
||
Start(["收到 Offer"]) --> CheckSupport{"浏览器支持<br/>setCodecPreferences?"}
|
||
CheckSupport --> |否| Skip["跳过设置偏好"]
|
||
CheckSupport --> |是| ReadOptions["读取下拉框选中项"]
|
||
ReadOptions --> Parse["解析 mimeType 与 sdpFmtpLine"]
|
||
Parse --> Find["在 RTCRtpSender.getCapabilities 中查找匹配编解码器"]
|
||
Find --> Found{"找到匹配项?"}
|
||
Found --> |否| Skip
|
||
Found --> |是| GetTransceivers["获取视频接收端点列表"]
|
||
GetTransceivers --> Apply["对每个视频接收端点调用 setCodecPreferences"]
|
||
Apply --> End(["完成"])
|
||
Skip --> End
|
||
```
|
||
|
||
图表来源
|
||
- [client/public/receiver/js/main.js:110-131](file://client/public/receiver/js/main.js#L110-L131)
|
||
- [client/src/renderstreaming.js:284-290](file://client/src/renderstreaming.js#L284-L290) |