Commit Graph

6 Commits

Author SHA1 Message Date
SpecialX
9db7fd917e feat(data-ana): v2 P6 硬化完成 + 6 新 RPC + Prometheus 监控
P6 硬化(5 项全部完成):

- CDC 多实例水平扩展: _INSTANCE_ID + get_lag() 真实 lag 计算

- ExamCache Redis 化: key data_ana:exam:{exam_id}, TTL 30 天 + 内存 LRU fallback

- ClickHouse TTL 归档: 5 表均加 TTL(1-3 年),分区级删除

- Prometheus 监控: 18 个指标(CDC/CH/ExamCache/DataScope/gRPC/业务)

- readyz 深度硬化: 4 依赖超时检查(CH 1s/Redis 200ms/iam 2s/CDC lag<1000)

v2 新增 6 个 RPC(analytics.proto 扩展为 18 RPC):

- GetStudentGrowth / GetAssignmentAnalysis / GetMasterySummary

- ListDiagnosticReports(占位,待 ai 服务)/ ListErrorBookItems / GetErrorBookStats

监控与可观测性: lifespan 预热 + gRPC ServerInterceptor + CDC 消费者指标

Docker 本地测试 19 项全部通过(healthz/readyz/metrics + 11 HTTP + 10 gRPC + ruff)

nextstep-v2.md: 上游需求对齐 + 下游要求(iam/core-edu/content/ai/SRE)
2026-07-14 18:07:17 +08:00
SpecialX
ca3780aa24 feat(data-ana): 完整实现 data-ana 数据分析服务
包含 CDC consumer、analytics/mastery/warning service、grpc server、repository、ClickHouse DDL 等
2026-07-10 19:09:27 +08:00
SpecialX
0a71b02e04 fix: code compliance audit and fix across all services
Some checks failed
CI / quality-ts (push) Failing after 48s
CI / quality-go (push) Failing after 4s
CI / quality-proto (push) Failing after 2s
CI / deploy (push) Has been skipped
NestJS (6 services): implement @RequirePermission decorator with
SetMetadata+Reflector, register APP_GUARD globally, fix as assertions
to type guards, add explicit return types, fix import type for express,
fix /metrics implicit any, replace native Error with ApplicationError,
remove typeorm remnants, register LifecycleService.

teacher-bff: add logger, ApplicationError, GlobalErrorFilter, forward
real userId to downstream, log downstream failures, migrate health
controller to shared/health.

Go (2 services): interface to any, doc comments, CORS dev whitelist,
JWT secret fail-fast, push-gateway internal API auth, metrics and
readyz endpoints, remove dead code.

Python (2 services): lifespan return type, dev_mode to bool, data-ana
APIRouter, ai POST body model, ClickHouse async wrapping.
2026-07-09 17:28:27 +08:00
SpecialX
1f901c5b20 feat(data-ana): implement complete CDC pipeline MySQL to ClickHouse
Debezium + Kafka + aiokafka consumer with table routing.

E2E verified: MySQL INSERT to ClickHouse upsert.
2026-07-09 13:02:59 +08:00
SpecialX
421edd8a41 feat(data-ana): 完善学情诊断服务并添加ClickHouse降级模式
config.py ClickHouse连接改可选+加DEV_MODE/kafka_brokers

clickhouse_client.py 降级模式: host为空时返回None

main.py 端点先查ClickHouse降级返回骨架数据+新增errorbook

新增clickhouse-init.sql创建宽表和错题表

Gateway添加/analytics路由
2026-07-09 08:58:39 +08:00
SpecialX
9850bfcfd1 feat(p4): content analysis service with Neo4j knowledge graph and ClickHouse analytics
P4 阶段交付物:
- services/content: 内容资源服务(NestJS)
  - textbooks: 教材 CRUD + 知识图谱绑定
  - config/neo4j.ts: Neo4j driver 单例
  - textbooks.service.ts: MySQL CRUD + Neo4j 知识图谱(createKnowledgeGraph/getPrerequisites)
  - package.json: 补充 @opentelemetry/sdk-node + exporter-trace-otlp-http
- services/data-ana: 数据分析服务(Python FastAPI)
  - main.py: FastAPI + /healthz + class_performance + student_weakness 骨架
  - clickhouse_client.py: ClickHouse 客户端封装
  - config.py: 环境变量配置
- packages/shared-proto/proto/content.proto: TextbookService + KnowledgeGraphService 契约
- packages/shared-proto/proto/analytics.proto: AnalyticsService 契约(class_performance/student_weakness)
2026-07-08 01:38:35 +08:00