feat(data-ana): implement complete CDC pipeline MySQL to ClickHouse

Debezium + Kafka + aiokafka consumer with table routing.

E2E verified: MySQL INSERT to ClickHouse upsert.

docs/troubleshooting/known-issues.md

@@ -27,7 +27,7 @@
 ### 1.2 Docker Compose 基础设施
 
 | 场景                      | 技术/规则                                                                                                               |
-| ---------------- | ----------------------------------------------------------------------------------------------- |
+| ------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
 | 日常开发启动              | 用 `docker-compose.minimal.yml` 仅起 MySQL+Redis                                                                        |
 | 全量启动内存不足          | 按 `profiles` 分阶段启用（full/kafka/cdc/analytics/graph/search/config/observability）                                  |
 | 每服务 mem_limit          | 避免单服务吃满内存：MySQL 512m、Redis 128m、Kafka 512m、ClickHouse 1g                                                   |
@@ -36,6 +36,11 @@
 | healthcheck               | MySQL 用 `mysqladmin ping`，Redis 用 `redis-cli ping`                                                                   |
 | Windows 下卷挂载          | init-sql 用绝对路径或确保相对路径正确                                                                                   |
 | 容器名固定                | `container_name: edu-mysql` 便于服务连接配置                                                                            |
+| Kafka 双 listener         | INSIDE (kafka:29092) 容器间互访 + OUTSIDE (localhost:9092) 主机访问，避免 Debezium 拿到 localhost metadata 后切回连不上 |
+| ClickHouse 远程访问       | 默认 default-user.xml 限制 127.0.0.1/::1 无密码，挂载 `clickhouse/users.d/custom-users.xml` 覆盖密码+任意 IP            |
+| Debezium Connect 镜像源   | daocloud 禁用 debezium/*，用 `quay.io/debezium/connect:2.7` 替代                                                        |
+| Debezium 跨网络访问 MySQL | MySQL 容器在 edu-minimal_default 时，`docker network connect edu-full_default edu-mysql` 让 Debezium 同时可达           |
+| CDC 注册 connector        | POST `:8083/connectors`，配置 `topic.prefix`/`database.include.list`/`schema.history.internal.kafka.topic`              |
 
 ### 1.3 protobuf + buf 契约
 
@@ -256,12 +261,18 @@
 ### 2.6 data-ana（Python/FastAPI，P4）
 
 | 场景                 | 技术/规则                                                                                                           |
-| ------------ | ------------------------------------------------------------------------------ |
+| -------------------- | ------------------------------------------------------------------------------------------------------------------- |
 | 学情诊断             | ClickHouse 宽表查询，5s 内返回                                                                                      |
-| CDC 链路     | Debezium 监听 MySQL binlog → Kafka（`mysql.cdc.*`）→ DataAna 消费写 ClickHouse |
+| CDC 链路             | Debezium 监听 MySQL binlog → Kafka（`edu-cdc.next_edu_cloud.<table>`）→ DataAna 消费写 ClickHouse                   |
 | CDC 延迟监控         | Debezium 暴露 lag metrics，超阈值告警                                                                               |
 | 双轨读策略           | 实时查 MySQL 主库（刚提交的成绩），聚合查 CH 宽表（延迟 1-5s 可接受）                                               |
 | 幂等消费             | 所有事件消费者必须幂等（基于 event_id 去重）                                                                        |
+| CDC 消费者实现       | `cdc_consumer.py` 用 aiokafka AIOKafkaConsumer，lifespan 启动 asyncio.create_task 后台运行                          |
+| ClickHouse 写入      | `clickhouse_client.upsert_student_dashboard()` 用 client.insert() 写宽表，client 为 None 时降级返回 False           |
+| Debezium 事件解析    | `before/after/source/op/ts_ms` 五字段，op=r(快照)/c(新增)/u(更新)/d(删除)                                           |
+| 多表关联缓存         | 内存 ExamCache 缓存 exam_id→class_id 映射（来自 core_edu_exams CDC 事件），grades 事件触发时查缓存填充宽表 class_id |
+| Consumer offset 重置 | `kafka-consumer-groups --reset-offsets --to-earliest --execute` 需先停消费者让 group 处于 Empty 状态                |
+| structlog API        | 24.x 用 `make_filtering_bound_logger(level)`，旧版 `make_filtering_logger` 已废弃                                   |
 
 ### 2.7 messaging（TS/NestJS，P5）
 
@@ -338,8 +349,9 @@
 > 按时间倒序，50 条上限。AI 发现更好方案时可更新本节。
 
 | 日期       | 时间 | 模块                            | 做了什么 + 学到什么                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                      |
-| ---------- | ---- | ------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| ---------- | ---- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
 | 2026-07-09 | 下午 | 全局                            | **P6 硬化：可观测性 + 部署 + CI 硬化**：(1) 可观测性栈完善：5 个 NestJS 服务 main.ts 添加 `/metrics` Prometheus 端点（用 `app.getHttpAdapter().get('/metrics', ...)` 绕过 DI 容器 get 方法）；prometheus.yml 从 2 个目标扩展到 8 个应用服务 + MySQL/Redis + node-exporter + prometheus 自身 + rule_files + alertmanager 关联；monitoring compose 用 Loki + Promtail 替换未配置的 blackbox-exporter；Grafana datasource 新增 Loki；新建 promtail/config.yml 用 docker_sd_configs 仅采集 `edu-*` 容器日志。(2) 部署 compose 扩展：docker-compose.deploy.yml 从 3 服务扩展到 11 服务（+ iam/teacher-bff/core-edu/content/msg/ai/data-ana/push-gateway），每个服务带 healthcheck + depends_on 条件 + edu-net/edu-shared 双网络；deploy.env.example 补全 Neo4j/ES/ClickHouse/LLM/Kafka 可选依赖配置。(3) teacher-bff 补 health.controller.ts（原缺失 /healthz 导致 deploy depends_on service_healthy 失败）。(4) CI 硬化：移除 lint 步骤的 continue-on-error（ESLint 9 flat config 已配置完成），test 保留 continue-on-error（部分服务无 test 脚本）。**学到**：NestJS `app.get('/metrics')` 会被解析为 DI 容器 `get(typeOrToken)`，必须用 `app.getHttpAdapter().get()` 才能注册 Express 路由；Promtail docker_sd_configs 通过 relabel_configs 的 `regex: '/(edu-.*).*'` 过滤容器名前缀；docker-compose.depends_on.condition: service_healthy 要求被依赖服务必须有 healthcheck 配置，否则启动失败。                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           |
+| 2026-07-09 | 下午 | data-ana/infra                  | **CDC 完整链路实现**：MySQL binlog → Debezium Connect → Kafka → data-ana 消费者 → ClickHouse 宽表。(1) MySQL binlog 配置：log_bin=ON, binlog_format=ROW, binlog_row_image=FULL, server_id=1；用 root 创建 `debezium` 用户授予 REPLICATION SLAVE + REPLICATION CLIENT。(2) Debezium Connect 容器：daocloud 禁用 debezium 镜像改用 `quay.io/debezium/connect:2.7`；MySQL 容器在 edu-minimal_default 网络，需 `docker network connect edu-full_default edu-mysql` 让 Debezium 同时可达；Kafka 必须配置双 listener（INSIDE:kafka:29092 + OUTSIDE:localhost:9092），否则 Debezium 拿到 advertised.listeners 中的 localhost metadata 后切换失败；Debezium 2.x 容器环境变量名用 BOOTSTRAP_SERVERS（不带 KAFKA_ 前缀），通过 envsubst 替换到 connect-distributed.properties。(3) 注册 connector：POST :8083/connectors，配置 topic.prefix=edu-cdc, database.include.list=next_edu_cloud, snapshot.mode=initial，4 张表（core_edu_grades/exams/classes/iam_users）成功产生快照事件。(4) data-ana 消费者实现：新建 cdc_consumer.py 用 aiokafka AIOKafkaConsumer，lifespan 中 asyncio.create_task 后台运行；按 source.table 路由（exams→内存缓存 exam_id→class_id 映射，grades→查缓存填 class_id 后 upsert ClickHouse）；readyz 端点附加 cdc_consumer 状态。(5) ClickHouse 远程访问：默认 default-user.xml 限制 127.0.0.1/::1 无密码，挂载 `clickhouse/users.d/custom-users.xml` 覆盖密码+任意 IP。(6) structlog 24.x API：`make_filtering_bound_logger(level)` 替代废弃的 `make_filtering_logger`。(7) E2E 验证：MySQL INSERT 成绩 → Debezium op=c 事件 → Kafka → 消费者写 ClickHouse 宽表（class_id 通过 exam 缓存正确填充）→ /readyz cdc_consumer=running → /analytics/student/student-002/weakness 返回实时 92 分数据。**学到**：Debezium 2.x 容器 bootstrap.servers 默认值是 0.0.0.0:9092 必须显式覆盖；Kafka 单 listener 配置 localhost 会让容器间通信的客户端拿到 metadata 后切换失败，必须用双 listener；ClickHouse users_xml 存储是 readonly 不能用 ALTER USER 修改密码，必须挂载 users.d 配置文件覆盖；消费者 offset 重置必须先停消费者让 group 处于 Empty 状态才能执行 --reset-offsets。 |
 | 2026-07-09 | 下午 | 全局                            | **P6 硬化：ESLint 9 flat config 配置**：(1) 根目录创建 `eslint.config.js`（ESLint 9 flat config 格式）：用 `typescript-eslint` recommended 规则集 + `@eslint/js` recommended + `eslint-config-prettier` 禁用冲突规则；自定义规则：`no-explicit-any` warn + `no-unused-vars` 允许下划线前缀 + 测试文件放宽。(2) 6 个 TS 服务 package.json lint 脚本从 `eslint src --ext .ts` 改为 `eslint src`（flat config 不需要 --ext）。(3) `lint-staged.config.js` 恢复 `eslint --fix`。(4) 验证：classes/content/msg/core-edu 四服务 lint 全部零错误零警告通过。**学到**：ESLint 9 flat config 用 `tseslint.config()` 工厂函数组装配置数组；`--ext` 参数在 flat config 模式下被移除，ESLint 自动根据 `eslint.config.js` 中的 `files` 匹配；`@typescript-eslint/consistent-type-assertions` 规则选项格式在 v8 中变化（`objectLiteralType` → `objectLiteralTypeAssertions`），配置时需查最新文档。                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
 | 2026-07-09 | 中午 | msg/push-gateway/ai/api-gateway | **P5 沟通与 AI 阶段三服务完善**：(1) msg 服务修复：database.ts 导出 db 常量；env.ts JWT_SECRET/ES_URL 改 optional 加 DEV_MODE/PUSH_GATEWAY_URL；elasticsearch.ts ES 降级（esClient=null 时 safeIndex/safeSearch 跳过）；notifications.service.ts 加 createBatch + listByUserWithPagination + Push Gateway 推送调用（try/catch 降级）；新建 msg-init.sql 2 张表。(2) push-gateway 完善：hub.go 重写用 send chan + 单写协程模式修复 gorilla/websocket 并发写竞争；handler.go 加 DEV_MODE dev-token 支持 + broadcast 端点；config.go 加 DevMode/RedisURL。(3) ai 服务完善：config.py 加 openai_api_key/base_url/dev_mode；新建 llm_client.py（httpx 异步调 OpenAI REST API）；main.py 加 /ai 前缀 + 降级模式（无 key 返回骨架 + degraded: true）+ /readyz 端点。(4) Gateway 路由扩展：/notifications → msg，/ai → ai 服务。**学到**：gorilla/websocket 不支持并发写，必须用 send chan 串行化所有写入；FastAPI APIRouter prefix 与 Gateway 代理路径要协调（ai 服务加 /ai 前缀，Gateway 代理 /ai/*path）；LLM 降级策略统一返回 degraded 标记，调用方据此判断是否路由流量。                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                    |
 | 2026-07-09 | 上午 | content/api-gateway             | **P4 内容分析服务端到端打通**：(1) content 服务系统性修复：database.ts 导出 db 常量；env.ts JWT_SECRET/ES_URL/NEO4J_URL/NEO4J_PASSWORD 改 optional 加 DEV_MODE；neo4j.ts driver 惰性创建+try/catch+connectionTimeout:3000；health/lifecycle 改用 Drizzle；global-error.filter 移除 @types/express 依赖；textbooks.schema 修复 integer→int + 导出 NewTextbook/NewChapter 类型；textbooks.controller 移除 body as any + 加 PUT/DELETE。(2) 新建 3 模块：chapters（CRUD + 按 textbook 查询）、knowledge-points（CRUD + Neo4j 前置依赖图非阻塞查询）、questions（CRUD + 4 种题型校验）。(3) Gateway 路由扩展：textbooks/chapters/knowledge-points/questions 四组路由。(4) 数据库：content-init.sql 4 张表。(5) E2E 验证：POST /textbooks 201 → POST /chapters 201（字段用 order 非 orderNum）→ POST /knowledge-points 201（Neo4j 不可用 MySQL 正常写入）→ POST /questions 201 → GET 各列表 200。**学到**：Drizzle schema TS 字段名与 DB 列名解耦（order→order_num），API 请求体用 TS 字段名；Neo4j 不可用时必须 driver=null（不设 NEO4J_URL），否则每次请求尝试连接拖慢响应；neo4j-driver safeCreateNode 用 try/catch 非阻塞，MySQL 数据始终先落库。                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                         |

infra/clickhouse/users.d/custom-users.xml

@@ -0,0 +1,14 @@
+<clickhouse>
+  <!-- 覆盖 default-user.xml 的本地限制，允许 default 用户从任意 IP 用密码访问 -->
+  <users>
+    <default>
+      <password>clickhouse</password>
+      <networks>
+        <ip>::/0</ip>
+      </networks>
+      <profile>default</profile>
+      <quota>default</quota>
+      <access_management>1</access_management>
+    </default>
+  </users>
+</clickhouse>

infra/docker-compose.yml

@@ -44,7 +44,13 @@ services:
     environment:
       KAFKA_BROKER_ID: 1
       KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
-      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
+      # 双 listener：INSIDE 容器间互访（kafka:29092），OUTSIDE 主机访问（localhost:9092）
+      # 必须用 INSIDE 作为 inter.broker.listener.name，否则 Debezium Connect 拿到 metadata
+      # 后会切回 advertised.listeners 中的 localhost，导致连接失败
+      KAFKA_LISTENERS: INSIDE://:29092,OUTSIDE://:9092
+      KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:29092,OUTSIDE://localhost:9092
+      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
+      KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
       KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
       KAFKA_AUTO_CREATE_TOPICS_ENABLE: "true"
     ports:
@@ -76,6 +82,8 @@ services:
       - "9000:9000"
     volumes:
       - clickhouse_data:/var/lib/clickhouse
+      # 覆盖默认 default-user.xml 限制（默认仅允许 127.0.0.1/::1 无密码访问）
+      - ./clickhouse/users.d/custom-users.xml:/etc/clickhouse-server/users.d/custom-users.xml:ro
     healthcheck:
       test: ["CMD", "wget", "--spider", "-q", "http://localhost:8123/ping"]
       interval: 10s
@@ -127,6 +135,50 @@ services:
     ports:
       - "16686:16686"
       - "4318:4318"
+  # ============================================================
+  # Debezium Connect - CDC 链路核心
+  # 监听 MySQL binlog → 写入 Kafka topic
+  # topic 命名约定：<prefix>.<database>.<table>（如 edu-cdc.next_edu_cloud.grades）
+  # ============================================================
+  debezium-connect:
+    image: quay.io/debezium/connect:2.7
+    container_name: edu-debezium
+    profiles: ["p4", "p5", "p6"]
+    restart: unless-stopped
+    depends_on:
+      kafka:
+        condition: service_healthy
+    environment:
+      # Kafka Connect 基础配置（Debezium 2.x 容器映射规则：环境变量名大写 → connect 配置项）
+      # 必须用 INSIDE listener (kafka:29092)，否则会拿到 OUTSIDE 的 localhost metadata 导致连不上
+      BOOTSTRAP_SERVERS: kafka:29092
+      GROUP_ID: edu-debezium
+      CONFIG_STORAGE_TOPIC: edu-connect-configs
+      OFFSET_STORAGE_TOPIC: edu-connect-offsets
+      STATUS_STORAGE_TOPIC: edu-connect-status
+      # 内部 converter 配置（必须与 Debezium 事件格式一致）
+      CONFIG_STORAGE_REPLICATION_FACTOR: "1"
+      OFFSET_STORAGE_REPLICATION_FACTOR: "1"
+      STATUS_STORAGE_REPLICATION_FACTOR: "1"
+      KEY_CONVERTER: org.apache.kafka.connect.json.JsonConverter
+      VALUE_CONVERTER: org.apache.kafka.connect.json.JsonConverter
+      KEY_CONVERTER_SCHEMAS_ENABLE: "false"
+      VALUE_CONVERTER_SCHEMAS_ENABLE: "false"
+      # 监听端口
+      REST_PORT: 8083
+      REST_ADVERTISED_HOST_NAME: debezium-connect
+      # 日志级别
+      LOG_LEVEL: INFO
+    ports:
+      - "8083:8083"
+    healthcheck:
+      test: ["CMD", "curl", "-f", "http://localhost:8083/connectors"]
+      interval: 15s
+      timeout: 5s
+      start_period: 30s
+      retries: 10
+    networks:
+      - default
   prometheus:
     image: docker.m.daocloud.io/prom/prometheus:v2.51.0
     container_name: edu-prometheus

scripts/debezium-register.json

@@ -0,0 +1,22 @@
+{
+  "name": "edu-mysql-source",
+  "config": {
+    "connector.class": "io.debezium.connector.mysql.MySqlConnector",
+    "database.hostname": "edu-mysql",
+    "database.port": "3306",
+    "database.user": "debezium",
+    "database.password": "debezium-pwd",
+    "database.allowPublicKeyRetrieval": "true",
+    "database.server.id": "184054",
+    "topic.prefix": "edu-cdc",
+    "database.include.list": "next_edu_cloud",
+    "table.include.list": "next_edu_cloud.core_edu_grades,next_edu_cloud.core_edu_exams,next_edu_cloud.classes,next_edu_cloud.iam_users",
+    "schema.history.internal.kafka.bootstrap.servers": "kafka:29092",
+    "schema.history.internal.kafka.topic": "edu-connect-schema-history",
+    "snapshot.mode": "initial",
+    "key.converter": "org.apache.kafka.connect.json.JsonConverter",
+    "value.converter": "org.apache.kafka.connect.json.JsonConverter",
+    "key.converter.schemas.enable": "false",
+    "value.converter.schemas.enable": "false"
+  }
+}

services/data-ana/pyproject.toml

@@ -15,6 +15,8 @@ dependencies = [
     "opentelemetry-instrumentation-fastapi>=0.48b0",
     "prometheus-client>=0.20.0",
     "structlog>=24.4.0",
+    # CDC 链路：消费 Debezium 写入 Kafka 的 MySQL binlog 变更事件
+    "aiokafka>=0.11.0",
 ]
 
 [tool.ruff]

services/data-ana/src/data_ana/cdc_consumer.py

@@ -0,0 +1,233 @@
+"""CDC 消费者（Debezium MySQL binlog → ClickHouse 宽表）.
+
+链路：
+  MySQL binlog → Debezium Connect → Kafka topic
+    edu-cdc.next_edu_cloud.<table>
+  → 本消费者 → 解析 Debezium 事件 → 写入 ClickHouse student_dashboard_view
+
+设计要点：
+- 监听多张表，按 source.table 路由
+- 内存缓存 exam_id → (class_id, subject_id) 映射（来自 core_edu_exams 快照+流）
+- 监听 core_edu_grades 时用缓存扩展为宽表记录写入 ClickHouse
+- 幂等性：依赖 ClickHouse ReplacingMergeTree 引擎按 ORDER BY 去重
+  （schema 需用 ReplacingMergeTree(last_updated)，当前为简化版 MergeTree）
+- op 类型：r(快照读)、c(新增)、u(更新)、d(删除)；d 时 after 为 null
+"""
+
+import asyncio
+import contextlib
+import json
+from datetime import UTC, datetime
+from typing import Any
+
+import structlog
+
+from .clickhouse_client import upsert_student_dashboard
+from .config import settings
+
+logger = structlog.get_logger(__name__)
+
+
+def _parse_ts(ts_ms: int | None) -> datetime:
+    """Debezium ts_ms（毫秒）→ datetime."""
+    if ts_ms is None:
+        return datetime.now(UTC)
+    return datetime.fromtimestamp(ts_ms / 1000, tz=UTC)
+
+
+def _safe_float(value: Any) -> float:
+    """安全转 float（Debezium 数值字段可能是字符串）."""
+    if value is None:
+        return 0.0
+    try:
+        return float(value)
+    except (TypeError, ValueError):
+        return 0.0
+
+
+class ExamCache:
+    """内存缓存 exam_id → (class_id, subject_id).
+
+    从 core_edu_exams 表的 CDC 事件构建。subject_id 在 exams 表中暂无字段，
+    这里占位为空字符串，后续扩展 schema 时再补充。
+    """
+
+    def __init__(self) -> None:
+        self._data: dict[str, tuple[str, str]] = {}
+
+    def upsert(self, exam_id: str, class_id: str, subject_id: str = "") -> None:
+        if exam_id:
+            self._data[exam_id] = (class_id, subject_id)
+
+    def get(self, exam_id: str) -> tuple[str, str] | None:
+        return self._data.get(exam_id) if exam_id else None
+
+
+# 全局缓存（进程级单例）
+_exam_cache = ExamCache()
+
+
+async def _handle_exams_event(after: dict[str, Any] | None) -> None:
+    """处理 core_edu_exams 表事件."""
+    if after is None:
+        return
+    exam_id = after.get("id")
+    class_id = after.get("class_id", "")
+    if exam_id:
+        _exam_cache.upsert(exam_id, class_id)
+        logger.info("exam_cache_updated", exam_id=exam_id, class_id=class_id)
+
+
+async def _handle_grades_event(
+    after: dict[str, Any] | None,
+    op: str,
+    ts_ms: int | None,
+) -> None:
+    """处理 core_edu_grades 表事件 → 写入 ClickHouse 宽表.
+
+    - op=r/c/u：after 为新数据，写入宽表
+    - op=d：after 为 null，暂不处理（宽表保留历史记录）
+    """
+    if after is None:
+        return
+
+    student_id = after.get("student_id", "")
+    exam_id = after.get("exam_id", "")
+    score = _safe_float(after.get("score"))
+
+    # 从缓存拿 class_id
+    class_id = ""
+    if exam_id:
+        cached = _exam_cache.get(exam_id)
+        if cached:
+            class_id = cached[0]
+
+    last_updated = _parse_ts(ts_ms)
+    if after.get("updated_at"):
+        # 优先用 MySQL 的 updated_at 字段
+        with contextlib.suppress(ValueError, AttributeError):
+            last_updated = datetime.fromisoformat(after["updated_at"].replace("Z", "+00:00"))
+
+    # 简化：rank/kp/mastery/error_count 暂用默认值
+    # 真实场景应通过其他 CDC 事件或聚合计算得到
+    await upsert_student_dashboard(
+        student_id=student_id,
+        class_id=class_id,
+        exam_id=exam_id,
+        subject_id="",  # 占位
+        score=score,
+        rank_in_class=0,
+        knowledge_point_id="",  # 占位
+        mastery_level=score / 100.0,  # 简化：用分数百分比作为掌握度
+        error_count=0,
+        last_updated=last_updated,
+    )
+
+
+async def _process_message(topic: str, value: bytes | str) -> None:
+    """处理单条 Kafka 消息.
+
+    Debezium 事件格式（简化后，schemas.enable=false）：
+    {
+      "before": {...} | null,
+      "after": {...} | null,
+      "source": {"table": "...", "db": "...", ...},
+      "op": "r|c|u|d",
+      "ts_ms": 1783572350928
+    }
+    """
+    try:
+        value_str = value.decode("utf-8") if isinstance(value, bytes) else value
+        event = json.loads(value_str)
+    except (json.JSONDecodeError, UnicodeDecodeError) as exc:
+        logger.warning("cdc_message_decode_failed", error=str(exc), topic=topic)
+        return
+
+    source = event.get("source") or {}
+    table = source.get("table", "")
+    op = event.get("op", "")
+    ts_ms = event.get("ts_ms")
+    after = event.get("after")
+
+    logger.info(
+        "cdc_event_received",
+        topic=topic,
+        table=table,
+        op=op,
+        ts_ms=ts_ms,
+    )
+
+    if table == "core_edu_exams":
+        await _handle_exams_event(after)
+    elif table == "core_edu_grades":
+        await _handle_grades_event(after, op, ts_ms)
+    else:
+        # 其他表暂不处理，仅记录
+        logger.debug("cdc_event_skipped", table=table)
+
+
+async def run_consumer() -> None:
+    """CDC 消费者主循环（lifespan 启动）.
+
+    - kafka_brokers 未配置：直接返回，不启动消费者（降级模式）
+    - 启动失败：仅记录错误，不阻塞 FastAPI 主流程
+    """
+    if not settings.kafka_brokers:
+        logger.info("cdc_consumer_disabled_no_kafka_brokers")
+        return
+
+    try:
+        from aiokafka import AIOKafkaConsumer
+    except ImportError:
+        logger.warning("cdc_consumer_aiokafka_not_installed")
+        return
+
+    topics = [t.strip() for t in settings.kafka_cdc_topics.split(",") if t.strip()]
+    if not topics:
+        logger.warning("cdc_consumer_no_topics_configured")
+        return
+
+    brokers = [b.strip() for b in settings.kafka_brokers.split(",") if b.strip()]
+
+    consumer = AIOKafkaConsumer(
+        *topics,
+        bootstrap_servers=brokers,
+        group_id=settings.kafka_group_id,
+        auto_offset_reset=settings.kafka_auto_offset_reset,
+        enable_auto_commit=True,
+        value_deserializer=lambda v: v,  # 保留原始 bytes，由 _process_message 解码
+    )
+
+    try:
+        await consumer.start()
+        logger.info(
+            "cdc_consumer_started",
+            brokers=brokers,
+            topics=topics,
+            group_id=settings.kafka_group_id,
+        )
+    except Exception as exc:  # noqa: BLE001
+        logger.error("cdc_consumer_start_failed", error=str(exc))
+        return
+
+    try:
+        async for msg in consumer:
+            try:
+                await _process_message(msg.topic, msg.value)
+            except Exception as exc:  # noqa: BLE001
+                logger.error(
+                    "cdc_message_process_failed",
+                    error=str(exc),
+                    topic=msg.topic,
+                    partition=msg.partition,
+                    offset=msg.offset,
+                )
+    except asyncio.CancelledError:
+        logger.info("cdc_consumer_cancelled")
+        raise
+    finally:
+        try:
+            await consumer.stop()
+            logger.info("cdc_consumer_stopped")
+        except Exception as exc:  # noqa: BLE001
+            logger.warning("cdc_consumer_stop_failed", error=str(exc))

services/data-ana/src/data_ana/clickhouse_client.py

@@ -5,6 +5,7 @@
 保证服务在 ClickHouse 不可用时仍可启动并响应骨架数据。
 """
 
+from datetime import datetime
 from typing import Any
 
 import structlog
@@ -216,3 +217,127 @@ async def ping() -> bool:
     except Exception as exc:  # noqa: BLE001
         logger.warning("clickhouse_ping_failed", error=str(exc))
         return False
+
+
+async def upsert_student_dashboard(
+    student_id: str,
+    class_id: str,
+    exam_id: str,
+    subject_id: str,
+    score: float,
+    rank_in_class: int,
+    knowledge_point_id: str,
+    mastery_level: float,
+    error_count: int,
+    last_updated: datetime,
+) -> bool:
+    """写入/更新学生学情宽表（CDC 消费专用）.
+
+    使用 ReplacingMergeTree 语义：按 ORDER BY 字段去重，保留 last_updated 最大版本。
+    返回 True 表示成功；False 表示降级模式或写入失败。
+    """
+    client = get_client()
+    if client is None:
+        return False
+
+    try:
+        client.insert(
+            "student_dashboard_view",
+            [
+                [
+                    student_id,
+                    class_id,
+                    exam_id,
+                    subject_id,
+                    score,
+                    rank_in_class,
+                    knowledge_point_id,
+                    mastery_level,
+                    error_count,
+                    last_updated,
+                ]
+            ],
+            column_names=[
+                "student_id",
+                "class_id",
+                "exam_id",
+                "subject_id",
+                "score",
+                "rank_in_class",
+                "knowledge_point_id",
+                "mastery_level",
+                "error_count",
+                "last_updated",
+            ],
+        )
+        logger.info(
+            "student_dashboard_upserted",
+            student_id=student_id,
+            class_id=class_id,
+            exam_id=exam_id,
+            score=score,
+        )
+        return True
+    except Exception as exc:  # noqa: BLE001
+        logger.warning(
+            "student_dashboard_upsert_failed_degraded",
+            error=str(exc),
+            student_id=student_id,
+            exam_id=exam_id,
+        )
+        return False
+
+
+async def upsert_student_error(
+    student_id: str,
+    question_id: str,
+    knowledge_point_id: str,
+    error_count: int,
+    last_error_time: datetime,
+    content: str,
+) -> bool:
+    """写入/更新学生错题本（CDC 消费专用）.
+
+    返回 True 表示成功；False 表示降级模式或写入失败。
+    """
+    client = get_client()
+    if client is None:
+        return False
+
+    try:
+        client.insert(
+            "student_errors",
+            [
+                [
+                    student_id,
+                    question_id,
+                    knowledge_point_id,
+                    error_count,
+                    last_error_time,
+                    content,
+                ]
+            ],
+            column_names=[
+                "student_id",
+                "question_id",
+                "knowledge_point_id",
+                "error_count",
+                "last_error_time",
+                "content",
+            ],
+        )
+        logger.info(
+            "student_error_upserted",
+            student_id=student_id,
+            question_id=question_id,
+            error_count=error_count,
+        )
+        return True
+    except Exception as exc:  # noqa: BLE001
+        logger.warning(
+            "student_error_upsert_failed_degraded",
+            error=str(exc),
+            student_id=student_id,
+            question_id=question_id,
+        )
+        return False

services/data-ana/src/data_ana/config.py

@@ -8,6 +8,9 @@ class Settings(BaseSettings):
 
     ClickHouse 连接参数为可选：当 clickhouse_host 为空字符串时，
     服务进入降级模式（查询方法返回 None / 空数据），保证服务可启动。
+
+    Kafka 连接参数为可选：当 kafka_brokers 为空字符串时，
+    CDC 消费者不启动（降级模式），保证服务可启动。
     """
 
     port: int = 3006
@@ -22,8 +25,20 @@ class Settings(BaseSettings):
     log_level: str = "info"
     # 开发模式开关（"true"/"false"）
     dev_mode: str = "false"
-    # Kafka brokers（CDC 消费预留，暂不实现）
+    # Kafka brokers（CDC 消费；留空则不启动消费者）
-    kafka_brokers: str = "localhost:9092"
+    # 主机访问用 localhost:9092，容器内访问用 kafka:29092
+    kafka_brokers: str = ""
+    # CDC 消费组 id
+    kafka_group_id: str = "data-ana-cdc-consumer"
+    # 要消费的 CDC topic（Debezium 默认命名：<prefix>.<database>.<table>）
+    # 用逗号分隔多个 topic
+    kafka_cdc_topics: str = (
+        "edu-cdc.next_edu_cloud.core_edu_grades,"
+        "edu-cdc.next_edu_cloud.core_edu_exams,"
+        "edu-cdc.next_edu_cloud.classes"
+    )
+    # 消费者自动偏移重置策略（earliest / latest）
+    kafka_auto_offset_reset: str = "earliest"
 
     model_config = {"env_file": ".env", "env_prefix": ""}
 

services/data-ana/src/data_ana/main.py

@@ -2,8 +2,13 @@
 
 支持 ClickHouse 降级模式：当 CLICKHOUSE_HOST 未配置或不可达时，
 查询端点返回骨架数据，服务仍可启动与响应。
+
+支持 CDC 消费者：当 KAFKA_BROKERS 配置时，
+后台启动 aiokafka 消费者，监听 Debezium CDC 事件写入 ClickHouse。
 """
 
+import asyncio
+import contextlib
 from contextlib import asynccontextmanager
 from datetime import UTC, datetime
 
@@ -15,6 +20,7 @@ from opentelemetry.sdk.trace import TracerProvider
 from opentelemetry.sdk.trace.export import BatchSpanProcessor
 from prometheus_client import make_asgi_app
 
+from .cdc_consumer import run_consumer as run_cdc_consumer
 from .clickhouse_client import (
     close_client,
     query_class_performance,
@@ -27,6 +33,9 @@ from .config import settings
 _logger: structlog.stdlib.BoundLogger | None = None
 tracer = trace.get_tracer(__name__)
 
+# CDC 消费者后台任务句柄
+_cdc_task: asyncio.Task | None = None
+
 # 日志级别映射
 _LOG_LEVELS: dict[str, int] = {
     "DEBUG": 10,
@@ -45,7 +54,7 @@ def init_logger() -> structlog.stdlib.BoundLogger:
     global _logger
     level = _LOG_LEVELS.get(settings.log_level.upper(), 20)
     structlog.configure(
-        wrapper_class=structlog.make_filtering_logger(level),
+        wrapper_class=structlog.make_filtering_bound_logger(level),
         processors=[
             structlog.contextvars.merge_contextvars,
             structlog.processors.add_log_level,
@@ -85,8 +94,10 @@ async def lifespan(app: FastAPI):
     1. 初始化 logger（structlog）
     2. 初始化 OTel tracer（endpoint 从 config 读）
     3. 触发 ClickHouse 客户端惰性初始化（不阻塞启动，失败进入降级模式）
-    4. 关闭时释放 ClickHouse 客户端
+    4. 若配置了 kafka_brokers，后台启动 CDC 消费者任务
+    5. 关闭时停止 CDC 任务并释放 ClickHouse 客户端
     """
+    global _cdc_task
     logger = init_logger()
     init_tracer()
     logger.info(
@@ -95,9 +106,17 @@ async def lifespan(app: FastAPI):
         dev_mode=settings.dev_mode,
         clickhouse_configured=bool(settings.clickhouse_host),
         kafka_brokers=settings.kafka_brokers,
+        kafka_cdc_topics=settings.kafka_cdc_topics,
     )
+    # 启动 CDC 消费者后台任务（若未配置 kafka_brokers，run_consumer 内部直接返回）
+    _cdc_task = asyncio.create_task(run_cdc_consumer())
     yield
     logger.info("data_ana_service_stopping")
+    # 取消 CDC 任务
+    if _cdc_task is not None and not _cdc_task.done():
+        _cdc_task.cancel()
+        with contextlib.suppress(asyncio.CancelledError):
+            await _cdc_task
     await close_client()
 
 
@@ -128,7 +147,19 @@ async def readyz() -> dict:
     - 已配置且可达：ready=true
     - 未配置：ready=true，degraded=true（降级模式仍可服务）
     - 已配置但不可达：ready=false
+
+    CDC 消费者状态附加在响应中：
+    - cdc_consumer: running / disabled / failed
     """
+    cdc_status = "disabled"
+    if _cdc_task is not None:
+        if _cdc_task.done():
+            cdc_status = "failed"
+        elif not settings.kafka_brokers:
+            cdc_status = "disabled"
+        else:
+            cdc_status = "running"
+
     if not settings.clickhouse_host:
         return {
             "status": "ok",
@@ -136,6 +167,8 @@ async def readyz() -> dict:
             "ready": True,
             "degraded": True,
             "clickhouse": "not_configured",
+            "cdc_consumer": cdc_status,
+            "kafka_brokers": settings.kafka_brokers or None,
             "timestamp": datetime.now(UTC).isoformat(),
         }
 
@@ -146,6 +179,8 @@ async def readyz() -> dict:
         "ready": ch_ok,
         "degraded": not ch_ok,
         "clickhouse": "ok" if ch_ok else "unreachable",
+        "cdc_consumer": cdc_status,
+        "kafka_brokers": settings.kafka_brokers or None,
         "timestamp": datetime.now(UTC).isoformat(),
     }
 

uv.lock

@@ -42,6 +42,37 @@ requires-dist = [
     { name = "uvicorn", extras = ["standard"], specifier = ">=0.30.0" },
 ]
 
+[[package]]
+name = "aiokafka"
+version = "0.14.0"
+source = { registry = "https://pypi.org/simple" }
+dependencies = [
+    { name = "async-timeout" },
+    { name = "packaging" },
+    { name = "typing-extensions" },
+]
+sdist = { url = "https://files.pythonhosted.org/packages/89/5f/dfc1180fd22d1acdc91949ec36e97199c43742dacb057cb8efed3679ed04/aiokafka-0.14.0.tar.gz", hash = "sha256:8ffdc945798ba4d3d132b705d4244d0a1f493925efb57c637a2ca88ee82794e1", size = 601374, upload-time = "2026-04-29T10:43:03.574Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/f9/9d/3441db94829f9feb802a2f4052df61c0d1a01272accd174c351d7e9e1f6a/aiokafka-0.14.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:284a90d617584d7e42688a181aaa8c2a909d9c658ab9b69c6cf92f4df5c4b320", size = 348458, upload-time = "2026-04-29T10:42:37.243Z" },
+    { url = "https://files.pythonhosted.org/packages/a4/10/7297589aac95654596af13301b31da2c9502c80e7e308530ee7a9bd5b9f1/aiokafka-0.14.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:b4f211d9e03a1fc83871a37eefcf307bc0943ee99adae25aa39bd1722e70747b", size = 351057, upload-time = "2026-04-29T10:42:38.69Z" },
+    { url = "https://files.pythonhosted.org/packages/26/4e/5c0aa8db717fff0ffb8f3e16deece8f98ded6ca17c6a543b6b20cc9a7f84/aiokafka-0.14.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:be517b9b9513eba43ba19961dd770a6e26d08325743093feb47182770d235dd9", size = 1142238, upload-time = "2026-04-29T10:42:39.96Z" },
+    { url = "https://files.pythonhosted.org/packages/88/78/322f797b9593a4cc8afd647342fa66b9ad732ee55098e5e084188c6202aa/aiokafka-0.14.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:219d2dc66b97b1aaea100697c928024b6a0348b7baa370b824900054bf86916e", size = 1131567, upload-time = "2026-04-29T10:42:41.542Z" },
+    { url = "https://files.pythonhosted.org/packages/8d/0a/a45320778385142299a7fc3ae402152ec1f383537130b8aa8e8587742fad/aiokafka-0.14.0-cp312-cp312-win32.whl", hash = "sha256:1086b470f6c452471603a2d9c8d6933739230c75758d777d8d113ff8112bad68", size = 312160, upload-time = "2026-04-29T10:42:42.811Z" },
+    { url = "https://files.pythonhosted.org/packages/a3/fb/7802a0ed69200e3e8e8791df06bd6daf9b00523839d045662de4ff061b18/aiokafka-0.14.0-cp312-cp312-win_amd64.whl", hash = "sha256:bcf3a8f6592d73f45965ca0750bfdfccf2555c8625358175c92f75f2cce1261a", size = 331897, upload-time = "2026-04-29T10:42:43.984Z" },
+    { url = "https://files.pythonhosted.org/packages/30/b0/c9384541b2e4cc52a16402fc53fb9d44af0d78d37954cf8c7271c376ad47/aiokafka-0.14.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:db16e43fac4c1c5006131046c1bf370c580d6ac4495a10ac7778245710943179", size = 345859, upload-time = "2026-04-29T10:42:45.449Z" },
+    { url = "https://files.pythonhosted.org/packages/7f/d1/fc266d9f4ffba4f197356c6ffdfbb0fe32e7cb874e240f299935d058ac06/aiokafka-0.14.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:32a8e91d88cf3ccf0778927715610d6579888c5f4748db4c2022cda25d628a48", size = 348284, upload-time = "2026-04-29T10:42:47.104Z" },
+    { url = "https://files.pythonhosted.org/packages/b2/8e/0c4c270786dac79f3fca74c6166c3a25b61b0d26132be0d69f0d7f206f0a/aiokafka-0.14.0-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:aad4a575a506e7784e25e430f27026fe2f4378560b21b7f4e8c9a54f0d06eaee", size = 1117867, upload-time = "2026-04-29T10:42:48.394Z" },
+    { url = "https://files.pythonhosted.org/packages/9d/7f/3b89fbd0a3be9edfd5b51e20bb5cd695c851219b63c501c051cf84367fa9/aiokafka-0.14.0-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:75e4a003502c9c3b5c705fa7c00d634ba146bf38fa5d525b80bb6ff6e3e779fe", size = 1108860, upload-time = "2026-04-29T10:42:50.249Z" },
+    { url = "https://files.pythonhosted.org/packages/b3/59/849aba75cff93277bf6bf8b630de79e902949ff7ec48e4b12a64e6e32cae/aiokafka-0.14.0-cp313-cp313-win32.whl", hash = "sha256:a128e213cbc2bce0ea3db65a68920e52cebeeb8209bf001ac7aa022a8bd54d7d", size = 310889, upload-time = "2026-04-29T10:42:52.038Z" },
+    { url = "https://files.pythonhosted.org/packages/c4/e5/52eab8f8515d23da7b5d90e2c5ba10eab9494a0314f749e3f73e003f4a50/aiokafka-0.14.0-cp313-cp313-win_amd64.whl", hash = "sha256:d6fa16bef3544be87bd1a7a8317b9d85e3da59f3202326d9ff22735ed052746e", size = 329470, upload-time = "2026-04-29T10:42:53.536Z" },
+    { url = "https://files.pythonhosted.org/packages/50/9d/984803315fe2b883ea6e08b1d9c8a752bd5c16e966d8714bacc67c72c417/aiokafka-0.14.0-cp314-cp314-macosx_10_15_x86_64.whl", hash = "sha256:5d70615d1530ad19d0c4da8d87abaec0a12b9fdaabffdcd4e400efa0c50ef80c", size = 346672, upload-time = "2026-04-29T10:42:55.267Z" },
+    { url = "https://files.pythonhosted.org/packages/49/df/da314966b7f3c3117bd78b082563cb03dbe3007848cb8f4b0932faf390a0/aiokafka-0.14.0-cp314-cp314-macosx_11_0_arm64.whl", hash = "sha256:7e2392360c370b1ba6564c57d2889e154ecdb43157a8f7b7d7afe5e3c02fcc1a", size = 349594, upload-time = "2026-04-29T10:42:56.565Z" },
+    { url = "https://files.pythonhosted.org/packages/57/7a/160516944ea0e0f68ea78e38f944c52f5248c7c7df26cba22a40b9f25709/aiokafka-0.14.0-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:201e38ecc595f9f65a945f1ef9085157ddf28f25cd2e482fd9efa1fcf4638213", size = 1114112, upload-time = "2026-04-29T10:42:57.869Z" },
+    { url = "https://files.pythonhosted.org/packages/68/c4/9841118a2157e913e8ebfbc0a2b58f7b60f1f7202040c3e1df8925ed1184/aiokafka-0.14.0-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1cd651e1f56571baae306fdd0b5509047ab9625797a24cd75902e139c5a20318", size = 1098571, upload-time = "2026-04-29T10:42:59.356Z" },
+    { url = "https://files.pythonhosted.org/packages/d6/a1/0af8a37849a4108ae227f46c4c62f6beab31863cf66ba318fb73b0be5b26/aiokafka-0.14.0-cp314-cp314-win32.whl", hash = "sha256:128127eb96dab98150b636bb5f480c80e15f02f82a118eec206a521c8cf7cf7c", size = 314107, upload-time = "2026-04-29T10:43:01.111Z" },
+    { url = "https://files.pythonhosted.org/packages/fa/18/fb46c65f758900c71d0f1c73b7802720f99cabcb1f4a11676573f9bc1b8f/aiokafka-0.14.0-cp314-cp314-win_amd64.whl", hash = "sha256:aa385039aa9b235359319bbdcf48c9c86a75d81c9c547d645056d00361238903", size = 333320, upload-time = "2026-04-29T10:43:02.424Z" },
+]
+
 [[package]]
 name = "annotated-doc"
 version = "0.0.4"
@@ -82,6 +113,15 @@ wheels = [
     { url = "https://files.pythonhosted.org/packages/5c/0a/a72d10ed65068e115044937873362e6e32fab1b7dce0046aeb224682c989/asgiref-3.11.1-py3-none-any.whl", hash = "sha256:e8667a091e69529631969fd45dc268fa79b99c92c5fcdda727757e52146ec133", size = 24345, upload-time = "2026-02-03T13:30:13.039Z" },
 ]
 
+[[package]]
+name = "async-timeout"
+version = "5.0.1"
+source = { registry = "https://pypi.org/simple" }
+sdist = { url = "https://files.pythonhosted.org/packages/a5/ae/136395dfbfe00dfc94da3f3e136d0b13f394cba8f4841120e34226265780/async_timeout-5.0.1.tar.gz", hash = "sha256:d9321a7a3d5a6a5e187e824d2fa0793ce379a202935782d555d6e9d2735677d3", size = 9274, upload-time = "2024-11-06T16:41:39.6Z" }
+wheels = [
+    { url = "https://files.pythonhosted.org/packages/fe/ba/e2081de779ca30d473f21f5b30e0e737c438205440784c7dfc81efc2b029/async_timeout-5.0.1-py3-none-any.whl", hash = "sha256:39e3809566ff85354557ec2398b55e096c8364bacac9405a7a1fa429e77fe76c", size = 6233, upload-time = "2024-11-06T16:41:37.9Z" },
+]
+
 [[package]]
 name = "certifi"
 version = "2026.6.17"
@@ -229,6 +269,7 @@ name = "data-ana-service"
 version = "0.1.0"
 source = { virtual = "services/data-ana" }
 dependencies = [
+    { name = "aiokafka" },
     { name = "clickhouse-connect" },
     { name = "fastapi" },
     { name = "opentelemetry-api" },
@@ -244,6 +285,7 @@ dependencies = [
 
 [package.metadata]
 requires-dist = [
+    { name = "aiokafka", specifier = ">=0.11.0" },
     { name = "clickhouse-connect", specifier = ">=0.7.0" },
     { name = "fastapi", specifier = ">=0.115.0" },
     { name = "opentelemetry-api", specifier = ">=1.27.0" },