修改
This commit is contained in:
385
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/触摸输入处理.md
Normal file
385
.qoder/repowiki/zh/content/WebRTC 实现/输入设备控制/触摸输入处理.md
Normal file
@@ -0,0 +1,385 @@
|
||||
# 触摸输入处理
|
||||
|
||||
<cite>
|
||||
**本文引用的文件**
|
||||
- [touchflags.js](file://client/src/touchflags.js)
|
||||
- [touchphase.js](file://client/src/touchphase.js)
|
||||
- [inputdevice.js](file://client/src/inputdevice.js)
|
||||
- [inputremoting.js](file://client/src/inputremoting.js)
|
||||
- [memoryhelper.js](file://client/src/memoryhelper.js)
|
||||
- [pointercorrect.js](file://client/src/pointercorrect.js)
|
||||
- [register-events.js](file://client/public/videoplayer/js/register-events.js)
|
||||
- [video-player.js](file://client/public/videoplayer/js/video-player.js)
|
||||
- [main.js](file://client/public/videoplayer/js/main.js)
|
||||
- [inputdevice.test.js](file://client/test/inputdevice.test.js)
|
||||
- [inputremoting.test.js](file://client/test/inputremoting.test.js)
|
||||
</cite>
|
||||
|
||||
## 目录
|
||||
1. [简介](#简介)
|
||||
2. [项目结构](#项目结构)
|
||||
3. [核心组件](#核心组件)
|
||||
4. [架构总览](#架构总览)
|
||||
5. [详细组件分析](#详细组件分析)
|
||||
6. [依赖关系分析](#依赖关系分析)
|
||||
7. [性能考量](#性能考量)
|
||||
8. [故障排查指南](#故障排查指南)
|
||||
9. [结论](#结论)
|
||||
10. [附录](#附录)
|
||||
|
||||
## 简介
|
||||
本文件聚焦于移动端触摸输入的处理机制，涵盖触摸事件的捕获与转发、触摸点标识、多点触控支持、手势识别基础、坐标系统映射、压力感应与触摸相位管理。文档基于仓库中的触摸输入相关源码，结合测试用例与客户端示例，给出可操作的实现路径与最佳实践。
|
||||
|
||||
## 项目结构
|
||||
触摸输入处理涉及三层：
|
||||
- 输入设备建模层：定义触摸状态、事件格式与序列化协议
|
||||
- 输入远程转发层：将本地输入事件打包并通过数据通道发送
|
||||
- 客户端事件绑定层：在页面元素上注册触摸事件监听并进行坐标映射
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
subgraph "输入设备建模层"
|
||||
A["touchphase.js<br/>触摸相位常量"]
|
||||
B["touchflags.js<br/>触摸标志位"]
|
||||
C["inputdevice.js<br/>Touchscreen/ToucnState/TouchnscreenState<br/>MemoryHelper"]
|
||||
end
|
||||
subgraph "输入远程转发层"
|
||||
D["inputremoting.js<br/>InputRemoting/NewEventsMsg/Message"]
|
||||
end
|
||||
subgraph "客户端事件绑定层"
|
||||
E["register-events.js<br/>注册触摸事件/坐标映射"]
|
||||
F["video-player.js<br/>计算视频区域偏移与缩放"]
|
||||
G["main.js<br/>设置元素 touch-action:none"]
|
||||
end
|
||||
A --> C
|
||||
B --> C
|
||||
C --> D
|
||||
D --> E
|
||||
F --> E
|
||||
G --> E
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)
|
||||
- [touchflags.js:1-8](file://client/src/touchflags.js#L1-L8)
|
||||
- [inputdevice.js:111-118](file://client/src/inputdevice.js#L111-L118)
|
||||
- [inputremoting.js:63-91](file://client/src/inputremoting.js#L63-L91)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- [main.js:58-65](file://client/public/videoplayer/js/main.js#L58-L65)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:111-118](file://client/src/inputdevice.js#L111-L118)
|
||||
- [inputremoting.js:63-91](file://client/src/inputremoting.js#L63-L91)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- [main.js:58-65](file://client/public/videoplayer/js/main.js#L58-L65)
|
||||
|
||||
## 核心组件
|
||||
- 触摸相位与标志
|
||||
- 相位：None、Began、Moved、Ended、Canceled、Stationary
|
||||
- 标志：IndirectTouch、PrimaryTouch、Tap、OrphanedPrimaryTouch
|
||||
- 触摸状态建模
|
||||
- TouchState：包含触摸点唯一ID、位置、增量、压力、半径、相位、点击次数、显示索引、标志、起始时间与起始位置等字段
|
||||
- TouchscreenState：聚合多个TouchState，负责将TouchEvent转换为内部状态数组
|
||||
- 输入事件序列化
|
||||
- TouchState.buffer与TouchscreenState.buffer将状态编码为二进制帧
|
||||
- 输入远程转发
|
||||
- InputRemoting监听本地输入事件，通过NewEventsMsg封装后经订阅者发送
|
||||
- 客户端事件绑定与坐标映射
|
||||
- 在播放器元素上注册touchstart/move/end/cancel事件
|
||||
- 使用PointerCorrector或VideoPlayer提供的缩放与偏移参数进行坐标映射
|
||||
|
||||
**章节来源**
|
||||
- [touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)
|
||||
- [touchflags.js:1-8](file://client/src/touchflags.js#L1-L8)
|
||||
- [inputdevice.js:324-453](file://client/src/inputdevice.js#L324-L453)
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
|
||||
## 架构总览
|
||||
触摸事件从浏览器到远端的典型流程如下：
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant U as "用户"
|
||||
participant V as "VideoPlayer<br/>video-player.js"
|
||||
participant R as "注册函数<br/>register-events.js"
|
||||
participant S as "序列化<br/>inputdevice.js"
|
||||
participant M as "远程转发<br/>inputremoting.js"
|
||||
U->>R : "触摸事件 : touchstart/move/end/cancel"
|
||||
R->>R : "收集changedTouches与touches"
|
||||
R->>V : "读取videoScale/videoOriginX/Y"
|
||||
R->>R : "坐标映射与相位判定"
|
||||
R->>S : "构造TouchState/TouchnscreenState"
|
||||
S-->>R : "buffer(二进制)"
|
||||
R->>M : "NewEventsMsg.create(state)"
|
||||
M-->>M : "Message封装"
|
||||
M-->>远端 : "发送二进制帧"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
|
||||
## 详细组件分析
|
||||
|
||||
### 触摸相位与标志
|
||||
- 相位常量用于统一表示触摸生命周期各阶段，便于跨平台一致性处理
|
||||
- 标志位用于表达触摸属性（如主触摸、点击标记等），便于后续手势合成与策略判断
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class TouchPhase {
|
||||
+None
|
||||
+Began
|
||||
+Moved
|
||||
+Ended
|
||||
+Canceled
|
||||
+Stationary
|
||||
}
|
||||
class TouchFlags {
|
||||
+IndirectTouch
|
||||
+PrimaryTouch
|
||||
+Tap
|
||||
+OrphanedPrimaryTouch
|
||||
}
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)
|
||||
- [touchflags.js:1-8](file://client/src/touchflags.js#L1-L8)
|
||||
|
||||
**章节来源**
|
||||
- [touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)
|
||||
- [touchflags.js:1-8](file://client/src/touchflags.js#L1-L8)
|
||||
|
||||
### 触摸状态建模与序列化
|
||||
- TouchState字段覆盖位置、增量、压力、半径、相位、起始时间与起始位置等，满足轨迹预测与手势识别需求
|
||||
- TouchscreenState聚合多点触摸，将Web TouchEvent转换为内部状态数组
|
||||
- MemoryHelper提供按位写入能力，支撑布尔型标志位的紧凑存储
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class TouchState {
|
||||
+touchId : number
|
||||
+position : number[]
|
||||
+delta : number[]
|
||||
+pressure : number
|
||||
+radius : number[]
|
||||
+phaseId : number
|
||||
+tapCount : number
|
||||
+displayIndex : number
|
||||
+flags : number
|
||||
+padding : number
|
||||
+startTime : number
|
||||
+startPosition : number[]
|
||||
+buffer() ArrayBuffer
|
||||
+format() number
|
||||
}
|
||||
class TouchscreenState {
|
||||
+touchData : TouchState[]
|
||||
+buffer() ArrayBuffer
|
||||
+format() number
|
||||
+convertPhaseId(type) number
|
||||
}
|
||||
class MemoryHelper {
|
||||
+writeSingleBit(buffer, bitOffset, value) void
|
||||
+sizeOfInt : number
|
||||
}
|
||||
TouchscreenState --> TouchState : "聚合"
|
||||
TouchState --> MemoryHelper : "使用"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputdevice.js:324-453](file://client/src/inputdevice.js#L324-L453)
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:324-453](file://client/src/inputdevice.js#L324-L453)
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- [memoryhelper.js:1-28](file://client/src/memoryhelper.js#L1-L28)
|
||||
|
||||
### 客户端触摸事件绑定与坐标映射
|
||||
- 在播放器元素上注册触摸事件监听，区分Began/Moved/Ended/Canceled
|
||||
- 通过VideoPlayer提供的videoScale、videoOriginX/Y与videoHeight进行坐标映射，适配letterbox与CSS布局
|
||||
- 对于Unity坐标系，需要对Y轴做翻转处理
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
Start(["触摸事件触发"]) --> Collect["收集changedTouches与touches"]
|
||||
Collect --> Phase["根据是否存在于touches判定Stationary或对应相位"]
|
||||
Phase --> Map["读取videoScale/videoOriginX/Y与videoHeight"]
|
||||
Map --> Scale["坐标映射: (x-origin)/scale 或 (height-(y-origin))/scale"]
|
||||
Scale --> Pack["构造二进制帧: TouchState/TouchnscreenState"]
|
||||
Pack --> End(["发送至InputRemoting"])
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
|
||||
**章节来源**
|
||||
- [register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- [main.js:58-65](file://client/public/videoplayer/js/main.js#L58-L65)
|
||||
|
||||
### 输入远程转发与消息封装
|
||||
- InputRemoting监听本地输入事件，调用NewEventsMsg.create将状态事件封装为Message
|
||||
- Message包含参与者ID、消息类型与数据载荷，供订阅者发送
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
participant L as "LocalInputManager"
|
||||
participant IR as "InputRemoting"
|
||||
participant NE as "NewEventsMsg"
|
||||
participant MSG as "Message"
|
||||
L->>IR : "onEvent : event"
|
||||
IR->>NE : "create(state)"
|
||||
NE-->>IR : "Message"
|
||||
IR-->>订阅者 : "onNext(Message)"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputremoting.js:79-91](file://client/src/inputremoting.js#L79-L91)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
|
||||
**章节来源**
|
||||
- [inputremoting.js:63-91](file://client/src/inputremoting.js#L63-L91)
|
||||
- [inputremoting.js:138-141](file://client/src/inputremoting.js#L138-L141)
|
||||
- [inputremoting.js:258-277](file://client/src/inputremoting.js#L258-L277)
|
||||
- [inputremoting.js:184-232](file://client/src/inputremoting.js#L184-L232)
|
||||
|
||||
### 坐标系统与Letterbox处理
|
||||
- PointerCorrector提供从元素坐标到视频坐标的映射，考虑letterbox偏移与缩放
|
||||
- VideoPlayer在resizeVideo时计算videoScale与videoOriginX/Y，确保触摸坐标与渲染区域一致
|
||||
|
||||
```mermaid
|
||||
classDiagram
|
||||
class PointerCorrector {
|
||||
+map(position) number[]
|
||||
+reset(w,h,elem) void
|
||||
+contentRect() Object
|
||||
+letterBoxType() number
|
||||
+letterBoxSize() number
|
||||
}
|
||||
class VideoPlayer {
|
||||
+resizeVideo() void
|
||||
+videoScale : number
|
||||
+videoOriginX : number
|
||||
+videoOriginY : number
|
||||
}
|
||||
PointerCorrector --> VideoPlayer : "配合使用"
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
|
||||
**章节来源**
|
||||
- [pointercorrect.js:6-124](file://client/src/pointercorrect.js#L6-L124)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
|
||||
### 多点触控与手势识别基础
|
||||
- TouchscreenState支持最多10个触摸点，通过TouchState.prevTouches缓存上一帧状态，便于计算delta与轨迹
|
||||
- 测试用例验证TouchscreenState能正确生成buffer，表明多点数据可被序列化传输
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
TStart(["touchstart"]) --> Cache["缓存TouchState(prevTouches)"]
|
||||
TM["touchmove"] --> Delta["计算delta与轨迹"]
|
||||
TEnd(["touchend"]) --> Tap["标记点击/结束状态"]
|
||||
Cache --> Delta
|
||||
Delta --> TEnd
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputdevice.js:494-516](file://client/src/inputdevice.js#L494-L516)
|
||||
- [inputdevice.test.js:69-100](file://client/test/inputdevice.test.js#L69-L100)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- [inputdevice.test.js:69-100](file://client/test/inputdevice.test.js#L69-L100)
|
||||
|
||||
### 压力感应与触摸相位管理
|
||||
- TouchState记录pressure与radius，支持压力与接触面信息
|
||||
- 相位管理由TouchscreenState.convertPhaseId统一转换，保证与TouchPhase一致
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:324-453](file://client/src/inputdevice.js#L324-L453)
|
||||
- [inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
|
||||
### 具体代码示例路径
|
||||
- 如何处理移动端触摸事件：[register-events.js:161-281](file://client/public/videoplayer/js/register-events.js#L161-L281)
|
||||
- 触摸轨迹预测与多点触控：[inputdevice.js:455-538](file://client/src/inputdevice.js#L455-L538)
|
||||
- 坐标映射与letterbox处理：[pointercorrect.js:20-40](file://client/src/pointercorrect.js#L20-L40)，[video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- 触摸相位与标志：[touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)，[touchflags.js:1-8](file://client/src/touchflags.js#L1-L8)
|
||||
|
||||
## 依赖关系分析
|
||||
- TouchscreenState依赖TouchState与TouchPhase、TouchFlags
|
||||
- TouchState依赖MemoryHelper进行位级写入
|
||||
- InputRemoting依赖NewEventsMsg与Message进行事件封装与发送
|
||||
- 客户端事件绑定依赖VideoPlayer提供的缩放与偏移参数
|
||||
|
||||
```mermaid
|
||||
graph LR
|
||||
TP["touchphase.js"] --> IDS["inputdevice.js"]
|
||||
TF["touchflags.js"] --> IDS
|
||||
MH["memoryhelper.js"] --> IDS
|
||||
IDS --> IR["inputremoting.js"]
|
||||
VP["video-player.js"] --> RE["register-events.js"]
|
||||
IR --> RE
|
||||
```
|
||||
|
||||
**图表来源**
|
||||
- [inputdevice.js:9-10](file://client/src/inputdevice.js#L9-L10)
|
||||
- [inputremoting.js:1-8](file://client/src/inputremoting.js#L1-L8)
|
||||
- [register-events.js:1-4](file://client/public/videoplayer/js/register-events.js#L1-L4)
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.js:9-10](file://client/src/inputdevice.js#L9-L10)
|
||||
- [inputremoting.js:1-8](file://client/src/inputremoting.js#L1-L8)
|
||||
- [register-events.js:1-4](file://client/public/videoplayer/js/register-events.js#L1-L4)
|
||||
|
||||
## 性能考量
|
||||
- 二进制序列化：TouchState/TouchnscreenState采用定长字段与紧凑布局，减少序列化开销
|
||||
- 缓存策略：TouchState.prevTouches缓存上一帧状态，避免重复计算与查找
|
||||
- 事件节流：在高频touchmove场景中，建议按需采样或合并更新，降低发送频率
|
||||
- 坐标映射：在resize频繁的场景，应避免在每帧都重建映射对象，复用VideoPlayer计算结果
|
||||
|
||||
## 故障排查指南
|
||||
- 触摸事件未生效
|
||||
- 检查播放器元素是否设置了touch-action:none，防止默认滚动行为干扰
|
||||
- 确认已注册touchstart/move/end/cancel事件监听
|
||||
- 坐标错位
|
||||
- 确保在resize后调用VideoPlayer.resizeVideo重新计算videoScale与videoOriginX/Y
|
||||
- 检查letterbox类型与偏移是否正确
|
||||
- 压力或半径无效
|
||||
- 确认设备与浏览器支持force/radius属性；若不支持，pressure/radius可能为0
|
||||
- 手势识别异常
|
||||
- 检查TouchState.prevTouches缓存是否正确更新
|
||||
- 确认相位转换逻辑与TouchPhase一致
|
||||
|
||||
**章节来源**
|
||||
- [main.js:58-65](file://client/public/videoplayer/js/main.js#L58-L65)
|
||||
- [video-player.js:157-213](file://client/public/videoplayer/js/video-player.js#L157-L213)
|
||||
- [inputdevice.js:494-516](file://client/src/inputdevice.js#L494-L516)
|
||||
- [touchphase.js:1-9](file://client/src/touchphase.js#L1-L9)
|
||||
|
||||
## 结论
|
||||
该触摸输入处理方案通过清晰的状态建模、稳定的相位与标志体系、高效的二进制序列化以及可靠的客户端坐标映射，实现了移动端触摸事件的可靠捕获与转发。结合多点触控与压力信息，可为进一步的手势识别与轨迹预测奠定基础。建议在高刷场景中引入采样与合并策略，以平衡交互流畅性与网络开销。
|
||||
|
||||
## 附录
|
||||
- 测试用例验证了TouchscreenState的buffer生成与格式正确性，可作为集成测试参考
|
||||
- 客户端示例展示了如何在播放器元素上注册触摸事件并进行坐标映射
|
||||
|
||||
**章节来源**
|
||||
- [inputdevice.test.js:69-100](file://client/test/inputdevice.test.js#L69-L100)
|
||||
- [inputremoting.test.js:59-106](file://client/test/inputremoting.test.js#L59-L106)
|
||||
Reference in New Issue
Block a user