chore(push-gateway): ai02 module updates - config, hub, ws, kafka, health, docs
This commit is contained in:
@@ -1,32 +1,42 @@
|
||||
# push-gateway 推送网关服务
|
||||
|
||||
> 版本:0.1(骨架)
|
||||
> 日期:2026-07-07
|
||||
> 状态:P5 待交付
|
||||
> 关联文档:[架构影响地图](../../architecture/004_architecture_impact_map.md)
|
||||
> 版本:1.0(仲裁回写)
|
||||
> 日期:2026-07-10
|
||||
> 状态:P5 已交付
|
||||
> 关联文档:
|
||||
>
|
||||
> - [架构影响地图](../../architecture/004_architecture_impact_map.md)
|
||||
> - [02 架构设计](../../../services/push-gateway/docs/02-architecture-design.md)
|
||||
> - [president-final-rulings](../../architecture/president-final-rulings.md)
|
||||
|
||||
---
|
||||
|
||||
## 1. 模块职责
|
||||
|
||||
push-gateway 是实时推送基础设施服务(Go 实现),管理 WebSocket/SSE 长连接。
|
||||
接收 msg 服务的推送请求,维护用户在线连接池,将消息实时投递到浏览器/移动端。
|
||||
支持连接鉴权、心跳保活、断线重连、多设备同步、连接数监控。
|
||||
push-gateway 是实时推送基础设施服务(Go 实现),管理 WebSocket 长连接(**仅 WebSocket,不支持 SSE** — ISSUE-001)。
|
||||
接收 msg 服务的推送请求(HTTP `/internal/*` + Kafka 事件),维护用户在线连接池,将消息实时投递到浏览器/移动端。
|
||||
支持连接鉴权(JWT RS256)、RFC 6455 控制帧心跳、断线重连、多设备同步、连接数监控、跨实例 Pub/Sub 广播。
|
||||
|
||||
**无 DB**(ISSUE-005):消息审计、离线落库、重试队列均由 msg 服务负责。
|
||||
|
||||
---
|
||||
|
||||
## 2. 技术栈
|
||||
|
||||
| 类别 | 技术 | 版本 | 用途 |
|
||||
|------|------|------|------|
|
||||
| 语言 | Go | 1.22+ | 高并发长连接 |
|
||||
| Web 框架 | Gin | 1.10+ | HTTP 升级 WebSocket |
|
||||
| WebSocket | gorilla/websocket | 1.5+ | WebSocket 协议 |
|
||||
| 缓存 | Redis | 7.x | 在线连接表、pub/sub |
|
||||
| 配置 | viper | 1.18+ | 配置注入 |
|
||||
| 日志 | zap | 1.27+ | 结构化日志 |
|
||||
| 追踪 | OpenTelemetry | 1.24+ | 分布式追踪 |
|
||||
| 编排 | Kubernetes | 1.28+ | 生产部署 |
|
||||
| 类别 | 技术 | 版本 | 用途 |
|
||||
| --------- | ------------------------ | ------ | -------------------------------------------------- |
|
||||
| 语言 | Go | 1.25+ | 高并发长连接 |
|
||||
| Web 框架 | Gin | 1.12+ | HTTP 升级 WebSocket + 内部 API |
|
||||
| WebSocket | gorilla/websocket | 1.5+ | WebSocket 协议 |
|
||||
| 缓存 | Redis | 7.x | 在线连接表、pub/sub 跨实例 fanout |
|
||||
| Kafka | segmentio/kafka-go | 0.4+ | 消费 `edu.notification.requested`(纯 Go,无 cgo) |
|
||||
| JWT | golang-jwt/jwt/v5 | 5.2+ | RS256 token 校验(via shared-go/jwks) |
|
||||
| 配置 | shared-go/env | — | 环境变量加载(替代 viper) |
|
||||
| 日志 | log/slog | stdlib | 结构化日志(JSON prod / text dev) |
|
||||
| 追踪 | OpenTelemetry | 1.44+ | 分布式追踪(via shared-go/tracer) |
|
||||
| 指标 | prometheus/client_golang | 1.23+ | Prometheus 指标 |
|
||||
| 共享包 | shared-go | — | tracer/logger/env/jwks 复用(ARB-015 §17.5) |
|
||||
| 编排 | Kubernetes | 1.28+ | 生产部署 |
|
||||
|
||||
---
|
||||
|
||||
@@ -34,17 +44,20 @@ push-gateway 是实时推送基础设施服务(Go 实现),管理 WebSocket
|
||||
|
||||
```mermaid
|
||||
graph TB
|
||||
MSG[msg 服务] -->|gRPC Push| PG[push-gateway]
|
||||
PG --> HUB[ConnectionHub<br/>连接池管理]
|
||||
MSG[msg 服务] -->|HTTP /internal/push| PG[push-gateway]
|
||||
MSG -->|Kafka edu.notification.requested| PG
|
||||
PG --> HUB[Hub<br/>连接池管理<br/>map userID -> connID -> Connection]
|
||||
HUB --> C1[用户 1 连接]
|
||||
HUB --> C2[用户 2 连接]
|
||||
HUB --> CN[用户 N 连接]
|
||||
PG -->[(Redis<br/>在线连接表 + pub/sub)]
|
||||
MFE[微前端] -.WebSocket.-> PG
|
||||
PG --> HEART[心跳保活<br/>30s ping/pong]
|
||||
PG --> REDIS[(Redis<br/>在线 SET + pub/sub)]
|
||||
MFE[微前端] -.WebSocket /ws?token=JWT.-> PG
|
||||
PG --> HEART[心跳保活<br/>RFC 6455 Ping/Pong 30s]
|
||||
PG --> JWKS[iam JWKS<br/>RS256 公钥 5min 缓存]
|
||||
PG --> METRICS[/metrics<br/>9 个 Prometheus 指标]
|
||||
```
|
||||
|
||||
> 待 P5 交付时补充完整的分层架构图与多副本连接同步图。
|
||||
> 完整分层图、跨实例同步图、失败模式矩阵见 [02 架构设计](../../../services/push-gateway/docs/02-architecture-design.md) §1 / §8 / §16。
|
||||
|
||||
---
|
||||
|
||||
@@ -57,58 +70,125 @@ sequenceDiagram
|
||||
participant R as Redis
|
||||
participant MSG as msg 服务
|
||||
|
||||
U->>PG: WebSocket 连接 + JWT 鉴权
|
||||
PG->>R: 注册在线连接(userId → nodeId)
|
||||
U->>PG: WebSocket /ws?token=JWT
|
||||
PG->>PG: JWT RS256 校验(via JWKS)
|
||||
PG->>HUB: Register(userID, conn)
|
||||
HUB->>R: SADD edu:push:online:<userID> + EXPIRE 60s
|
||||
PG-->>U: 连接建立
|
||||
Note over PG: 心跳保活 30s ping/pong
|
||||
Note over PG: RFC 6455 Ping/Pong 30s;60s 无帧断开
|
||||
|
||||
MSG->>PG: gRPC Push(userId, payload)
|
||||
PG->>R: 查询 userId 在线 nodeId
|
||||
alt 本节点在线
|
||||
PG->>U: WebSocket 推送消息
|
||||
else 跨节点在线
|
||||
PG->>R: pub/sub 转发到目标节点
|
||||
R->>PG: 目标节点订阅收到
|
||||
PG->>U: 推送消息
|
||||
MSG->>PG: POST /internal/push {user_id, event, data}
|
||||
PG->>HUB: SendToUser(userID, msg)
|
||||
alt 本实例有此用户
|
||||
PG->>U: WebSocket 推送
|
||||
PG-->>MSG: {delivered:true, online:true}
|
||||
else 本实例无此用户
|
||||
PG->>R: SMEMBERS edu:push:online:<userID>
|
||||
alt 用户在线(其他实例)
|
||||
PG->>R: PUBLISH edu:push:channel:user.<userID>
|
||||
R-->>PG: 持有实例订阅后投递
|
||||
PG-->>MSG: {delivered:true, online:true}
|
||||
else 用户离线
|
||||
PG-->>MSG: {delivered:false, online:false}
|
||||
Note over MSG: msg 走离线推送(SMS/邮件)
|
||||
end
|
||||
end
|
||||
```
|
||||
|
||||
> 待 P5 交付时补充:断线重连流程、多设备同步流程、连接数监控流程。
|
||||
|
||||
---
|
||||
|
||||
## 5. 对外契约
|
||||
|
||||
- **gRPC 服务**:PushService(Push、Broadcast、GetOnlineStatus)(待 P5 定义 protobuf)
|
||||
- **HTTP 端点**:`/ws`(WebSocket 升级)、`/healthz`、`/readyz`、`/metrics`
|
||||
- **事件**:不发布也不消费 Kafka 事件,仅接收 msg 的 gRPC 推送调用
|
||||
- **Redis 协议**:在线连接表(HASH)、跨节点推送(pub/sub channel `push:{nodeId}`)
|
||||
### 5.1 HTTP 端点
|
||||
|
||||
> 待 P5 交付时补充完整 protobuf 定义与 WebSocket 消息协议。
|
||||
| 方法 | 路径 | 鉴权 | 用途 |
|
||||
| ---- | --------------------------- | ------------------------------------------------------- | ----------------------------------- |
|
||||
| GET | `/ws` | JWT RS256(query `?token=` 或 `Authorization: Bearer`) | WebSocket 升级 |
|
||||
| POST | `/internal/push` | `X-Internal-Token` 头(ISSUE-002) | 定向推送 |
|
||||
| POST | `/internal/broadcast` | `X-Internal-Token` 头 | 广播 |
|
||||
| GET | `/internal/online/<userID>` | `X-Internal-Token` 头 | 查在线状态 |
|
||||
| GET | `/healthz` | 无 | liveness 探针 |
|
||||
| GET | `/readyz` | 无 | readiness 探针(软失败,ISSUE-058) |
|
||||
| GET | `/metrics` | 无 | Prometheus 指标 |
|
||||
|
||||
### 5.2 Kafka 消费
|
||||
|
||||
| Topic | Consumer Group | Message | 说明 |
|
||||
| ---------------------------- | -------------- | ----------------------- | ------------------------------------------------ |
|
||||
| `edu.notification.requested` | `push-gateway` | `NotificationRequested` | ISSUE-053 重命名(原 `edu.notification.events`) |
|
||||
|
||||
- 至少一次 + Redis SETNX 幂等去重(event_id,TTL 24h)
|
||||
- 失败重试 3 次后入 DLQ(`edu.notification.requested.dlq`)
|
||||
|
||||
### 5.3 Redis 数据结构
|
||||
|
||||
| Key 模式 | 类型 | TTL | 用途 |
|
||||
| -------------------------------- | ----------------- | --- | -------------------- |
|
||||
| `edu:push:online:<userID>` | SET(instanceID) | 60s | 在线用户所在实例集合 |
|
||||
| `edu:push:channel:user:<userID>` | Pub/Sub | — | 跨实例定向推送 |
|
||||
| `edu:push:channel:broadcast` | Pub/Sub | — | 跨实例广播 |
|
||||
| `edu:push:idempotent:<event_id>` | SETNX | 24h | Kafka 幂等去重 |
|
||||
|
||||
> **无 gRPC 服务**(ISSUE-007):push-gateway 仅暴露 HTTP + WebSocket,不实现 gRPC 服务,`packages/shared-proto/proto/` 无 push-gateway.proto。
|
||||
|
||||
---
|
||||
|
||||
## 6. 依赖关系
|
||||
|
||||
- **上游**:msg 服务(gRPC Push 调用)
|
||||
- **下游**:Redis(在线连接表、pub/sub 跨节点同步)
|
||||
- **客户端**:teacher-portal、student-portal、parent-portal(WebSocket 连接)
|
||||
- **共享包**:shared-proto、shared-go
|
||||
- **上游**:msg 服务(HTTP `/internal/*` + Kafka 事件)
|
||||
- **下游**:Redis(在线 SET + pub/sub 跨实例同步)、iam(JWKS 公钥)
|
||||
- **客户端**:teacher-portal、student-portal、parent-portal、admin-portal(WebSocket 连接)
|
||||
- **共享包**:shared-go(tracer/logger/env/jwks)
|
||||
|
||||
---
|
||||
|
||||
## 7. 架构约束
|
||||
|
||||
1. 无业务逻辑,仅做连接管理与消息转发
|
||||
2. 多副本部署下,通过 Redis pub/sub 同步跨节点推送
|
||||
3. 连接鉴权必须校验 JWT,禁止未认证连接
|
||||
4. 心跳超时 60s 无响应则断开连接
|
||||
5. 单节点最大连接数 50k,超出时拒绝新连接
|
||||
|
||||
> 待 P5 交付时补充完整约束清单。
|
||||
2. **无 DB**(ISSUE-005):审计/落库/重试由 msg 负责
|
||||
3. **仅 WebSocket**(ISSUE-001):不支持 SSE
|
||||
4. **无 gRPC**(ISSUE-007):仅 HTTP + WebSocket
|
||||
5. 连接鉴权必须校验 JWT RS256(DevMode 接受 `dev-token`)
|
||||
6. RFC 6455 控制帧心跳(Ping/Pong 30s,60s 无帧断开)
|
||||
7. 单用户连接数上限 5(默认,可配 `MAX_CONNS_PER_USER`)
|
||||
8. **单实例容量 10w+**(ISSUE-003,原 50k 已提升)
|
||||
9. Redis 故障软失败(ISSUE-058/006):`/readyz` 返 200 + `degraded:true`,不断本地 WebSocket
|
||||
10. Redis online SET 启动重建(ISSUE-058):先 SREM 旧 instanceID,再基于内存重建
|
||||
|
||||
---
|
||||
|
||||
## 8. 架构决策
|
||||
## 8. 关键 ADR(完整见 02 §14)
|
||||
|
||||
> 待 P5 交付时补充 ADR 记录,预计包括:Go+gorilla/websocket 选型、Redis pub/sub 跨节点方案、连接数上限策略、心跳间隔选型。
|
||||
| ADR | 决策 | 来源 |
|
||||
| ------- | ---------------------------------------- | ---------------- |
|
||||
| ADR-001 | 仅 WebSocket,不支持 SSE | ISSUE-001 |
|
||||
| ADR-002 | `X-Internal-Token` 命名 | ISSUE-002 |
|
||||
| ADR-003 | 单实例 10w+ 连接 | ISSUE-003 |
|
||||
| ADR-004 | 无 DB,审计由 msg 负责 | ISSUE-005 |
|
||||
| ADR-005 | Kafka topic `edu.notification.requested` | ISSUE-053 |
|
||||
| ADR-006 | `/readyz` 软失败 | ISSUE-058/006 |
|
||||
| ADR-007 | Redis SET 启动重建 | ISSUE-058 |
|
||||
| ADR-008 | Redis 必需但软失败 | ISSUE-055 vs 058 |
|
||||
| ADR-009 | 纯 HTTP + WebSocket,无 gRPC | ISSUE-007 |
|
||||
| ADR-010 | shared-go 复用 | ARB-015 §17.5 |
|
||||
|
||||
---
|
||||
|
||||
## 9. 环境变量
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| ----------------------------- | ---------------------------------------------- | ---------------------------------------- |
|
||||
| `PUSH_GATEWAY_PORT` | 8081 | HTTP 监听端口 |
|
||||
| `DEV_MODE` | false | 开发模式(跳过内部鉴权,接受 dev-token) |
|
||||
| `JWT_SECRET` | (DevMode 有默认) | JWT 密钥(生产必填) |
|
||||
| `JWKS_URL` | `http://localhost:50052/.well-known/jwks.json` | iam JWKS 端点 |
|
||||
| `INTERNAL_API_TOKEN` | (生产必填) | `/internal/*` 鉴权 token(ISSUE-002) |
|
||||
| `REDIS_URL` | `redis://localhost:6379/0` | Redis 连接 URL |
|
||||
| `KAFKA_BROKERS` | `localhost:9092` | Kafka broker 列表(逗号分隔) |
|
||||
| `KAFKA_NOTIFICATION_TOPIC` | `edu.notification.requested` | 消费 topic(ISSUE-053) |
|
||||
| `KAFKA_CONSUMER_GROUP` | `push-gateway` | consumer group |
|
||||
| `WS_ALLOWED_ORIGINS` | (空) | WebSocket Origin 白名单(逗号分隔) |
|
||||
| `MAX_CONNS_PER_USER` | 5 | 单用户最大连接数 |
|
||||
| `HEARTBEAT_INTERVAL_SECONDS` | 30 | 心跳间隔 |
|
||||
| `INSTANCE_ID` | (hostname) | 实例标识(Redis SET 成员) |
|
||||
| `OTEL_EXPORTER_OTLP_ENDPOINT` | `localhost:4318` | OTLP 端点 |
|
||||
|
||||
Reference in New Issue
Block a user