SpecialX
5b06bdbc52
feat(data-ana): docker 本地测试通过 + gRPC 修复 + nextstep 上下游依赖文档
2026-07-13 23:57:48 +08:00
SpecialX
9e4d442c0e
chore(data-ana): 更新工作进度跟踪与ISSUE状态，补建基础设施脚本，清理遗留代码
...
- workline: 追加 §6 实现进度跟踪（P2-P5 已完成，P6 未开始）
- issue: 更新 §0.2/§0.3 核查表，proto 文件已全部补全
- 新增 scripts/clickhouse_ddl.sql（5 宽表建表脚本）
- 新增 scripts/seed_clickhouse.py（mock 种子数据脚本）
- 删除遗留代码 clickhouse_client.py 和 health.py
2026-07-13 11:12:26 +08:00
SpecialX
ca3780aa24
feat(data-ana): 完整实现 data-ana 数据分析服务
...
包含 CDC consumer、analytics/mastery/warning service、grpc server、repository、ClickHouse DDL 等
2026-07-10 19:09:27 +08:00
SpecialX
0a71b02e04
fix: code compliance audit and fix across all services
...
CI / quality-ts (push) Failing after 48s
CI / quality-go (push) Failing after 4s
CI / quality-proto (push) Failing after 2s
CI / deploy (push) Has been skipped
NestJS (6 services): implement @RequirePermission decorator with
SetMetadata+Reflector, register APP_GUARD globally, fix as assertions
to type guards, add explicit return types, fix import type for express,
fix /metrics implicit any, replace native Error with ApplicationError,
remove typeorm remnants, register LifecycleService.
teacher-bff: add logger, ApplicationError, GlobalErrorFilter, forward
real userId to downstream, log downstream failures, migrate health
controller to shared/health.
Go (2 services): interface to any, doc comments, CORS dev whitelist,
JWT secret fail-fast, push-gateway internal API auth, metrics and
readyz endpoints, remove dead code.
Python (2 services): lifespan return type, dev_mode to bool, data-ana
APIRouter, ai POST body model, ClickHouse async wrapping.
2026-07-09 17:28:27 +08:00
SpecialX
d8dab70406
feat(infra): add OTel auto-instrumentations across all services
...
CI / quality-proto (push) Failing after 2s
CI / deploy (push) Has been skipped
CI / quality-ts (push) Failing after 1m11s
CI / quality-go (push) Failing after 5s
NestJS 6 services use getNodeAutoInstrumentations().
Python 2 services use FastAPIInstrumentor. Go 2 services use otelgin.
2026-07-09 13:25:46 +08:00
SpecialX
1f901c5b20
feat(data-ana): implement complete CDC pipeline MySQL to ClickHouse
...
Debezium + Kafka + aiokafka consumer with table routing.
E2E verified: MySQL INSERT to ClickHouse upsert.
2026-07-09 13:02:59 +08:00
SpecialX
421edd8a41
feat(data-ana): 完善学情诊断服务并添加ClickHouse降级模式
...
config.py ClickHouse连接改可选+加DEV_MODE/kafka_brokers
clickhouse_client.py 降级模式: host为空时返回None
main.py 端点先查ClickHouse降级返回骨架数据+新增errorbook
新增clickhouse-init.sql创建宽表和错题表
Gateway添加/analytics路由
2026-07-09 08:58:39 +08:00
SpecialX
4fea3de1d1
fix(data-ana): ruff 自动修复代码风格
2026-07-08 12:52:33 +08:00
SpecialX
e9ea34fe53
feat(p6): production hardening with circuit breaker, backup, monitoring and chaos engineering
...
CI Go / test (push) Has been cancelled
CI Python / test (push) Has been cancelled
CI TypeScript / test (push) Has been cancelled
CI Proto / lint (push) Failing after 8m7s
P6 生产硬化阶段交付物（46 文件）：
## 1. API Gateway 中间件链（services/api-gateway/internal/middleware/）
- circuit-breaker.go: gobreaker v2 熔断器（5s 窗口/50% 错误率/30s OPEN→HALF_OPEN）
- ratelimit.go: 令牌桶限流（sync.Map + cleanup goroutine，默认 100rps/20 burst）
- cors.go: CORS 中间件（CORS_ORIGINS 环境变量）
- recovery.go: panic 恢复 + uuid request_id
- security.go: 安全头 + 请求体 10MB 限制
- requestid.go: 请求 ID 注入
- health/health.go: /healthz + /readyz 健康检查
- main.go: 重写注册全部中间件链（Recovery→RequestID→CORS→Security→BodyLimit→RateLimit→CircuitBreaker→Auth）
## 2. 基础设施硬化（infra/）
- backup/backup-mysql.sh: MySQL 全量备份（mysqldump+gzip，按服务独立）
- backup/restore-mysql.sh: 恢复脚本
- backup/backup-cron.sh: cron 调度入口（5 服务批量备份）
- alertmanager/alertmanager.yml: 告警路由（webhook + 邮件示例）
- prometheus/rules.yml: 8 条告警规则（服务可用性/性能/资源 3 组）
- grafana/dashboards/microservices-overview.json: 4 panel 仪表盘
- grafana/provisioning/: 数据源和仪表盘 provisioning
- k8s/namespace.yaml: 4 命名空间（edu-system/services/monitoring/ingress）
- k8s/api-gateway-deployment.yaml: Deployment + Service 骨架
- chaos/experiments.yaml: 3 个 Litmus 混沌实验（pod-kill/network-latency/disk-fill）
- docker-compose.monitoring.yml: 监控栈 profile
- security/secrets.example.env: 8 项密钥占位符
- security/waf-rules.conf: ModSecurity WAF 规则骨架
## 3. 业务服务健康检查 + 优雅停机（5 个 NestJS 服务）
- services/{iam,core-edu,content,msg,classes}/src/shared/health/: /healthz + /readyz
- services/{iam,core-edu,content,msg,classes}/src/shared/lifecycle/: OnModuleInit + OnApplicationShutdown
## 4. Python 服务健康检查
- services/{ai,data-ana}/src/health/health.py: FastAPI APIRouter
## 5. 运维文档
- docs/architecture/runbooks/p6-hardening.md: P6 总览 Runbook（9 章节）
- docs/architecture/runbooks/incident-response.md: 事件响应手册（5 章节）
- docs/architecture/004-p6-addendum.md: 004 架构补记 P6 章节
- docs/troubleshooting/known-issues-p6-addendum.md: 15 条 P6 场景→技术映射
## 验收信号
- RPO ≤ 15min（MySQL 备份 + binlog PITR）
- RTO ≤ 30min（K8s 滚动更新 + DNS 切换）
- P99 ≤ 500ms（熔断 + 限流 + 缓存）
- 熔断器错误率 > 50% 触发 OPEN
- 限流 100rps/20 burst
- 备份保留 7 天
- 混沌实验每月 1 次
2026-07-08 02:16:58 +08:00
SpecialX
9850bfcfd1
feat(p4): content analysis service with Neo4j knowledge graph and ClickHouse analytics
...
P4 阶段交付物：
- services/content: 内容资源服务（NestJS）
- textbooks: 教材 CRUD + 知识图谱绑定
- config/neo4j.ts: Neo4j driver 单例
- textbooks.service.ts: MySQL CRUD + Neo4j 知识图谱（createKnowledgeGraph/getPrerequisites）
- package.json: 补充 @opentelemetry/sdk-node + exporter-trace-otlp-http
- services/data-ana: 数据分析服务（Python FastAPI）
- main.py: FastAPI + /healthz + class_performance + student_weakness 骨架
- clickhouse_client.py: ClickHouse 客户端封装
- config.py: 环境变量配置
- packages/shared-proto/proto/content.proto: TextbookService + KnowledgeGraphService 契约
- packages/shared-proto/proto/analytics.proto: AnalyticsService 契约（class_performance/student_weakness）
2026-07-08 01:38:35 +08:00
