docs: add full-stack runbook and one-click scripts
start-all/stop-all/health-check/test-cdc PowerShell scripts. Compatible with PowerShell 5.1 (ASCII only, no emoji).
This commit is contained in:
SpecialX
279
docs/standards/full-stack-runbook.md
Normal file
279
docs/standards/full-stack-runbook.md
Normal file
@@ -0,0 +1,279 @@
|
|||||||
|
# 全链路启用与测试手册（Full Stack Runbook）
|
||||||
|
|
||||||
|
> 版本：1.0
|
||||||
|
> 日期：2026-07-09
|
||||||
|
> 适用范围：Edu 微服务架构（P6 阶段：11 基础设施 + 10 应用服务 + 1 前端）
|
||||||
|
> 关联文档：[local-dev-runbook](./local-dev-runbook.md)、[004 架构影响地图](../architecture/004_architecture_impact_map.md)、[known-issues](../troubleshooting/known-issues.md)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 1. 前置条件
|
||||||
|
|
||||||
|
| 工具 | 版本要求 | 验证命令 |
|
||||||
|
| -------------- | -------- | ------------------------ |
|
||||||
|
| Node.js | ≥ 20 | `node -v` |
|
||||||
|
| pnpm | ≥ 9 | `pnpm -v` |
|
||||||
|
| Go | 1.22+ | `go version` |
|
||||||
|
| uv | 0.4+ | `uv --version` |
|
||||||
|
| Docker | 24+ | `docker version` |
|
||||||
|
| Docker Compose | v2+ | `docker compose version` |
|
||||||
|
|
||||||
|
> Windows 用户：Go 工具链若不在 PATH，临时加入：`$env:Path = "C:\Program Files\Go\bin;" + $env:Path`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 2. 服务端口矩阵
|
||||||
|
|
||||||
|
### 2.1 应用服务（10 个）
|
||||||
|
|
||||||
|
| 端口 | 服务 | 语言/框架 | 启动方式 |
|
||||||
|
| ---- | -------------- | --------- | ---------------------------------------------- |
|
||||||
|
| 3000 | teacher-portal | Next.js | `pnpm --filter teacher-portal dev` |
|
||||||
|
| 3001 | classes | NestJS | `pnpm --filter @edu/classes-service dev` |
|
||||||
|
| 3002 | iam | NestJS | `pnpm --filter @edu/iam-service dev` |
|
||||||
|
| 3003 | teacher-bff | NestJS | `pnpm --filter @edu/teacher-bff dev` |
|
||||||
|
| 3004 | core-edu | NestJS | `pnpm --filter @edu/core-edu-service dev` |
|
||||||
|
| 3005 | content | NestJS | `pnpm --filter @edu/content-service dev` |
|
||||||
|
| 3006 | data-ana | FastAPI | `uv run uvicorn data_ana.main:app --port 3006` |
|
||||||
|
| 3007 | msg | NestJS | `pnpm --filter @edu/msg-service dev` |
|
||||||
|
| 3008 | ai | FastAPI | `uv run uvicorn ai.main:app --port 3008` |
|
||||||
|
| 8080 | api-gateway | Go (Gin) | `go run .` |
|
||||||
|
| 8081 | push-gateway | Go (Gin) | `go run .` |
|
||||||
|
|
||||||
|
### 2.2 基础设施（11 个）
|
||||||
|
|
||||||
|
| 端口 | 服务 | 用途 |
|
||||||
|
| --------- | ------------------ | --------------------- |
|
||||||
|
| 3306 | MySQL 8 | 写模型主库 |
|
||||||
|
| 6379 | Redis 7 | 缓存/会话 |
|
||||||
|
| 8083 | Debezium Connect | CDC source connector |
|
||||||
|
| 8123/9000 | ClickHouse 24.3 | 读模型宽表 |
|
||||||
|
| 9092 | Kafka 7.6 | 事件总线 |
|
||||||
|
| 7474/7687 | Neo4j 5.20 | 知识图谱 |
|
||||||
|
| 9200 | Elasticsearch 8.13 | 题库检索 |
|
||||||
|
| 9090 | Prometheus | 指标采集 |
|
||||||
|
| 3030 | Grafana | 可视化（admin/admin） |
|
||||||
|
| 16686 | Jaeger | 分布式追踪 UI |
|
||||||
|
| 4318 | OTLP Collector | OTel span 接收 |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 3. 一键启动
|
||||||
|
|
||||||
|
### 3.1 启动基础设施
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd e:\Desktop\Edu\infra
|
||||||
|
docker compose -f docker-compose.yml --profile p6 --profile observability up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
等待所有容器 healthy（约 60 秒）：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
docker ps --filter "name=edu-" --format "table {{.Names}}\t{{.Status}}"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 3.2 一键启动所有应用服务
|
||||||
|
|
||||||
|
使用项目根目录的启动脚本：
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd e:\Desktop\Edu
|
||||||
|
.\scripts\start-all.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
该脚本会：
|
||||||
|
|
||||||
|
1. 检查基础设施健康状态
|
||||||
|
2. 为每个应用服务启动独立后台窗口（带标题）
|
||||||
|
3. 自动注入 Python 服务的环境变量（CLICKHOUSE/KAFKA/OTEL）
|
||||||
|
4. 等待所有服务健康检查通过
|
||||||
|
|
||||||
|
### 3.3 健康检查
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd e:\Desktop\Edu
|
||||||
|
.\scripts\health-check.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
预期输出：所有服务 ✅
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 4. 端到端链路测试
|
||||||
|
|
||||||
|
### 4.1 IAM 注册 + 登录
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$h = @{Authorization="Bearer dev-token"}
|
||||||
|
|
||||||
|
# 注册
|
||||||
|
$body = @{username="testteacher";password="Test@1234";email="test@edu.com";role="teacher"} | ConvertTo-Json
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:8080/iam/auth/register" -Method Post -Body $body -ContentType "application/json" -Headers $h
|
||||||
|
|
||||||
|
# 登录
|
||||||
|
$loginBody = @{username="testteacher";password="Test@1234"} | ConvertTo-Json
|
||||||
|
$resp = Invoke-RestMethod -Uri "http://localhost:8080/iam/auth/login" -Method Post -Body $loginBody -ContentType "application/json"
|
||||||
|
$token = $resp.data.accessToken
|
||||||
|
Write-Host "Token: $token"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.2 Classes CRUD
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 创建班级
|
||||||
|
$classBody = @{name="高三一班";gradeId="grade-1";headTeacherId=""} | ConvertTo-Json
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:8080/classes" -Method Post -Body $classBody -ContentType "application/json" -Headers $h
|
||||||
|
|
||||||
|
# 查询班级列表
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:8080/classes" -Method Get -Headers $h
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.3 CDC 完整链路（MySQL → Debezium → Kafka → data-ana → ClickHouse）
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 1. 向 MySQL 插入成绩（触发 binlog）
|
||||||
|
docker exec edu-mysql mysql -uedu -pchangeme next_edu_cloud -e "
|
||||||
|
INSERT INTO core_edu_exams (id, class_id, subject_id, title, exam_date, total_score, created_at, updated_at)
|
||||||
|
VALUES ('exam-cdc-test-001','cls-test-001','sub-math','CDC测试考试',NOW(),100,NOW(),NOW())
|
||||||
|
ON DUPLICATE KEY UPDATE updated_at=NOW();
|
||||||
|
|
||||||
|
INSERT INTO core_edu_grades (id, exam_id, student_id, score, rank_in_class, created_at, updated_at)
|
||||||
|
VALUES ('grade-cdc-001','exam-cdc-test-001','student-cdc-001',92.5,1,NOW(),NOW())
|
||||||
|
ON DUPLICATE KEY UPDATE score=92.5, updated_at=NOW();
|
||||||
|
"
|
||||||
|
|
||||||
|
# 2. 等待 Debezium 捕获 + data-ana 消费
|
||||||
|
Start-Sleep -Seconds 5
|
||||||
|
|
||||||
|
# 3. 验证 ClickHouse 已同步
|
||||||
|
docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q "
|
||||||
|
SELECT student_id, class_id, exam_id, score, last_updated
|
||||||
|
FROM edu_analytics.student_dashboard_view
|
||||||
|
WHERE student_id = 'student-cdc-001'
|
||||||
|
ORDER BY last_updated DESC
|
||||||
|
"
|
||||||
|
# 预期：返回一行，score=92.5，class_id='cls-test-001'
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.4 data-ana 查询 API
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 学生学情看板
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:3006/analytics/student/student-cdc-001/weakness" -Headers $h
|
||||||
|
|
||||||
|
# 班级成绩分析
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:3006/analytics/class/cls-test-001/performance" -Headers $h
|
||||||
|
|
||||||
|
# CDC 消费者状态
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:3006/readyz"
|
||||||
|
# 预期: cdc_consumer = "running"
|
||||||
|
```
|
||||||
|
|
||||||
|
### 4.5 可观测性验证
|
||||||
|
|
||||||
|
| 检查项 | URL | 预期 |
|
||||||
|
| ------------------ | ----------------------------------- | ------------------------ |
|
||||||
|
| Prometheus targets | http://localhost:9090/targets | 所有 target UP |
|
||||||
|
| Grafana 面板 | http://localhost:3030 (admin/admin) | 可登录 |
|
||||||
|
| Jaeger UI | http://localhost:16686 | 可搜索到各 service trace |
|
||||||
|
| data-ana /metrics | http://localhost:3006/metrics | Prometheus 格式输出 |
|
||||||
|
| iam /metrics | http://localhost:3002/metrics | Prometheus 格式输出 |
|
||||||
|
|
||||||
|
**Jaeger trace 验证步骤**：
|
||||||
|
|
||||||
|
1. 打开 http://localhost:16686
|
||||||
|
2. Service 下拉框应能看到 `iam`、`classes`、`data-ana`、`api-gateway` 等
|
||||||
|
3. 选择任一服务 → Find Traces → 应看到 HTTP 请求的自动埋点 span
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 5. 一键停止
|
||||||
|
|
||||||
|
### 5.1 停止应用服务
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd e:\Desktop\Edu
|
||||||
|
.\scripts\stop-all.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
该脚本会关闭所有 `edu-app-*` 标题的终端窗口。
|
||||||
|
|
||||||
|
### 5.2 停止基础设施
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd e:\Desktop\Edu\infra
|
||||||
|
docker compose -f docker-compose.yml --profile p6 --profile observability down
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 6. 故障排查
|
||||||
|
|
||||||
|
### 6.1 端口占用
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 查看占用端口的进程
|
||||||
|
netstat -ano | findstr :3001
|
||||||
|
# 终止进程
|
||||||
|
taskkill /PID <PID> /F
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.2 基础设施未启动
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 检查容器状态
|
||||||
|
docker ps --filter "name=edu-"
|
||||||
|
|
||||||
|
# 重启单个容器
|
||||||
|
docker restart edu-mysql
|
||||||
|
|
||||||
|
# 查看日志
|
||||||
|
docker logs edu-debezium --tail 50
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.3 CDC 链路断开
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 1. 检查 Debezium connector 状态
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status"
|
||||||
|
|
||||||
|
# 2. 重启 connector
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/restart" -Method Post
|
||||||
|
|
||||||
|
# 3. 检查 Kafka topic 是否有数据
|
||||||
|
docker exec edu-kafka kafka-console-consumer --bootstrap-server localhost:9092 --topic edu-cdc.next_edu_cloud.core_edu_grades --from-beginning --max-messages 1
|
||||||
|
|
||||||
|
# 4. 检查 data-ana 消费者日志
|
||||||
|
# 查看 data-ana 终端窗口的 cdc_consumer_started / cdc_event_received 日志
|
||||||
|
```
|
||||||
|
|
||||||
|
### 6.4 OTel trace 未上报
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# 1. 检查 Jaeger 是否收到 trace
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:16686/api/services"
|
||||||
|
|
||||||
|
# 2. 检查 OTLP endpoint 是否可达
|
||||||
|
Invoke-RestMethod -Uri "http://localhost:4318/v1/traces" -Method Post -ContentType "application/json" -Body "{}"
|
||||||
|
|
||||||
|
# 3. 检查服务环境变量
|
||||||
|
# 确保 OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 已设置
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 7. 速查：常用命令
|
||||||
|
|
||||||
|
| 场景 | 命令 |
|
||||||
|
| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
|
| 启动基础设施 | `docker compose -f infra/docker-compose.yml --profile p6 --profile observability up -d` |
|
||||||
|
| 一键启动应用 | `.\scripts\start-all.ps1` |
|
||||||
|
| 一键停止应用 | `.\scripts\stop-all.ps1` |
|
||||||
|
| 健康检查 | `.\scripts\health-check.ps1` |
|
||||||
|
| CDC 链路验证 | `.\scripts\test-cdc.ps1` |
|
||||||
|
| 查看容器状态 | `docker ps --filter "name=edu-"` |
|
||||||
|
| 查看 Debezium 状态 | `Invoke-RestMethod http://localhost:8083/connectors/edu-mysql-source/status` |
|
||||||
|
| ClickHouse 查询 | `docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q "SELECT * FROM edu_analytics.student_dashboard_view LIMIT 10"` |
|
||||||
|
| Kafka topic 列表 | `docker exec edu-kafka kafka-topics --bootstrap-server localhost:9092 --list` |
|
||||||
|
| Prometheus 查询 | `Invoke-RestMethod "http://localhost:9090/api/v1/query?query=up"` |
|
||||||
152
scripts/health-check.ps1
Normal file
152
scripts/health-check.ps1
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
<#
|
||||||
|
.SYNOPSIS
|
||||||
|
Edu health check script
|
||||||
|
.DESCRIPTION
|
||||||
|
Checks health of all infrastructure containers + application services + observability endpoints
|
||||||
|
.EXAMPLE
|
||||||
|
.\scripts\health-check.ps1
|
||||||
|
#>
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Continue"
|
||||||
|
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Edu Health Check" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 1. Infrastructure containers =====
|
||||||
|
Write-Host "[1/3] Infrastructure Containers" -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$infraContainers = @(
|
||||||
|
@{Name="MySQL"; Container="edu-mysql"},
|
||||||
|
@{Name="Redis"; Container="edu-redis"},
|
||||||
|
@{Name="Kafka"; Container="edu-kafka"},
|
||||||
|
@{Name="Zookeeper"; Container="edu-zookeeper"},
|
||||||
|
@{Name="ClickHouse"; Container="edu-clickhouse"},
|
||||||
|
@{Name="Debezium"; Container="edu-debezium"},
|
||||||
|
@{Name="Neo4j"; Container="edu-neo4j"},
|
||||||
|
@{Name="Elasticsearch"; Container="edu-es"},
|
||||||
|
@{Name="Jaeger"; Container="edu-jaeger"},
|
||||||
|
@{Name="Prometheus"; Container="edu-prometheus"},
|
||||||
|
@{Name="Grafana"; Container="edu-grafana"}
|
||||||
|
)
|
||||||
|
|
||||||
|
$infraOk = 0
|
||||||
|
$infraFail = 0
|
||||||
|
foreach ($svc in $infraContainers) {
|
||||||
|
$status = docker inspect --format='{{.State.Health.Status}}' $svc.Container 2>$null
|
||||||
|
if ($status -eq "healthy") {
|
||||||
|
Write-Host " [OK] $($svc.Name)" -ForegroundColor Green
|
||||||
|
$infraOk++
|
||||||
|
} elseif ($status -eq "running") {
|
||||||
|
Write-Host " [WARN] $($svc.Name) (running, no healthcheck)" -ForegroundColor Yellow
|
||||||
|
$infraOk++
|
||||||
|
} else {
|
||||||
|
Write-Host " [FAIL] $($svc.Name) (status=$status)" -ForegroundColor Red
|
||||||
|
$infraFail++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 2. Application services =====
|
||||||
|
Write-Host "[2/3] Application Services" -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$appServices = @(
|
||||||
|
@{Name="classes"; Url="http://localhost:3001/healthz"},
|
||||||
|
@{Name="iam"; Url="http://localhost:3002/healthz"},
|
||||||
|
@{Name="teacher-bff"; Url="http://localhost:3003/healthz"},
|
||||||
|
@{Name="core-edu"; Url="http://localhost:3004/healthz"},
|
||||||
|
@{Name="content"; Url="http://localhost:3005/healthz"},
|
||||||
|
@{Name="data-ana"; Url="http://localhost:3006/healthz"},
|
||||||
|
@{Name="msg"; Url="http://localhost:3007/healthz"},
|
||||||
|
@{Name="ai"; Url="http://localhost:3008/healthz"},
|
||||||
|
@{Name="api-gateway"; Url="http://localhost:8080/healthz"},
|
||||||
|
@{Name="push-gateway"; Url="http://localhost:8081/healthz"},
|
||||||
|
@{Name="teacher-portal";Url="http://localhost:3000/"}
|
||||||
|
)
|
||||||
|
|
||||||
|
$appOk = 0
|
||||||
|
$appFail = 0
|
||||||
|
foreach ($svc in $appServices) {
|
||||||
|
try {
|
||||||
|
$null = Invoke-RestMethod -Uri $svc.Url -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] $($svc.Name)" -ForegroundColor Green
|
||||||
|
$appOk++
|
||||||
|
} catch {
|
||||||
|
Write-Host " [FAIL] $($svc.Name)" -ForegroundColor Red
|
||||||
|
$appFail++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 3. Observability endpoints =====
|
||||||
|
Write-Host "[3/3] Observability Endpoints" -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$obsEndpoints = @(
|
||||||
|
@{Name="Prometheus API"; Url="http://localhost:9090/api/v1/query?query=up"},
|
||||||
|
@{Name="Jaeger API"; Url="http://localhost:16686/api/services"},
|
||||||
|
@{Name="Debezium Connect"; Url="http://localhost:8083/connectors"},
|
||||||
|
@{Name="data-ana /metrics";Url="http://localhost:3006/metrics"},
|
||||||
|
@{Name="iam /metrics"; Url="http://localhost:3002/metrics"}
|
||||||
|
)
|
||||||
|
|
||||||
|
$obsOk = 0
|
||||||
|
$obsFail = 0
|
||||||
|
foreach ($ep in $obsEndpoints) {
|
||||||
|
try {
|
||||||
|
$null = Invoke-RestMethod -Uri $ep.Url -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] $($ep.Name)" -ForegroundColor Green
|
||||||
|
$obsOk++
|
||||||
|
} catch {
|
||||||
|
Write-Host " [FAIL] $($ep.Name)" -ForegroundColor Red
|
||||||
|
$obsFail++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
# ===== 4. CDC pipeline status =====
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "[Extra] CDC Pipeline Status" -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$connectorStatus = $null
|
||||||
|
try {
|
||||||
|
$connectorStatus = Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status" -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
} catch {
|
||||||
|
Write-Host " [FAIL] Debezium connector not registered or error" -ForegroundColor Red
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($connectorStatus) {
|
||||||
|
$connectorState = $connectorStatus.connector.state
|
||||||
|
$tasks = @($connectorStatus.tasks)
|
||||||
|
if ($tasks.Count -gt 0) {
|
||||||
|
$taskState = $tasks[0].state
|
||||||
|
} else {
|
||||||
|
$taskState = "UNKNOWN"
|
||||||
|
}
|
||||||
|
if ($connectorState -eq "RUNNING" -and $taskState -eq "RUNNING") {
|
||||||
|
$color = "Green"
|
||||||
|
} else {
|
||||||
|
$color = "Yellow"
|
||||||
|
}
|
||||||
|
Write-Host " Connector: $connectorState / Task: $taskState" -ForegroundColor $color
|
||||||
|
}
|
||||||
|
|
||||||
|
$chSql = 'SELECT count(*) FROM edu_analytics.student_dashboard_view'
|
||||||
|
$chOutput = docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q $chSql 2>&1
|
||||||
|
$chCount = "$chOutput".Trim()
|
||||||
|
if ($chCount -match '^\d+$') {
|
||||||
|
Write-Host " ClickHouse student_dashboard_view: $chCount records" -ForegroundColor Green
|
||||||
|
} else {
|
||||||
|
Write-Host " ClickHouse query failed or empty" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
|
||||||
|
# ===== Summary =====
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Infra: $infraOk/$($infraContainers.Count) | App: $appOk/$($appServices.Count) | Obs: $obsOk/$($obsEndpoints.Count)" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
|
||||||
|
if ($infraFail -gt 0 -or $appFail -gt 0 -or $obsFail -gt 0) {
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
185
scripts/start-all.ps1
Normal file
185
scripts/start-all.ps1
Normal file
@@ -0,0 +1,185 @@
|
|||||||
|
<#
|
||||||
|
.SYNOPSIS
|
||||||
|
Edu start all application services script
|
||||||
|
.DESCRIPTION
|
||||||
|
Starts 10 application services + 1 frontend (teacher-portal)
|
||||||
|
Each service runs in a separate terminal window for log visibility
|
||||||
|
Automatically injects Python service env vars (CLICKHOUSE/KAFKA/OTEL)
|
||||||
|
Prerequisite: infrastructure containers (MySQL/Redis/Kafka/ClickHouse etc) must be running
|
||||||
|
.PARAMETER SkipInfraCheck
|
||||||
|
Skip infrastructure health check (use when infra is known to be running)
|
||||||
|
.EXAMPLE
|
||||||
|
.\scripts\start-all.ps1
|
||||||
|
.\scripts\start-all.ps1 -SkipInfraCheck
|
||||||
|
#>
|
||||||
|
param(
|
||||||
|
[switch]$SkipInfraCheck
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$ProjectRoot = Split-Path -Parent $PSScriptRoot
|
||||||
|
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Edu Start All Services" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 1. Infrastructure health check =====
|
||||||
|
if (-not $SkipInfraCheck) {
|
||||||
|
Write-Host "[1/4] Checking infrastructure health..." -ForegroundColor Yellow
|
||||||
|
$infraServices = @(
|
||||||
|
@{Name="MySQL"; Container="edu-mysql"},
|
||||||
|
@{Name="Redis"; Container="edu-redis"},
|
||||||
|
@{Name="Kafka"; Container="edu-kafka"},
|
||||||
|
@{Name="ClickHouse"; Container="edu-clickhouse"},
|
||||||
|
@{Name="Debezium"; Container="edu-debezium"},
|
||||||
|
@{Name="Jaeger"; Container="edu-jaeger"}
|
||||||
|
)
|
||||||
|
$allHealthy = $true
|
||||||
|
foreach ($svc in $infraServices) {
|
||||||
|
$status = docker inspect --format='{{.State.Health.Status}}' $svc.Container 2>$null
|
||||||
|
if ($status -eq "healthy") {
|
||||||
|
Write-Host " [OK] $($svc.Name) ($($svc.Container))" -ForegroundColor Green
|
||||||
|
} elseif ($status -eq "running") {
|
||||||
|
Write-Host " [WARN] $($svc.Name) running but no healthcheck" -ForegroundColor Yellow
|
||||||
|
} else {
|
||||||
|
Write-Host " [FAIL] $($svc.Name) not started or unhealthy (status=$status)" -ForegroundColor Red
|
||||||
|
Write-Host " Please start infrastructure first:" -ForegroundColor Red
|
||||||
|
Write-Host " docker compose -f infra/docker-compose.yml --profile p6 --profile observability up -d" -ForegroundColor Red
|
||||||
|
$allHealthy = $false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (-not $allHealthy) {
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "Infrastructure not ready. Start it first." -ForegroundColor Red
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
Write-Host ""
|
||||||
|
}
|
||||||
|
|
||||||
|
# ===== 2. Environment variables =====
|
||||||
|
Write-Host "[2/4] Preparing environment variables..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$env:DEV_MODE = "true"
|
||||||
|
$env:OTEL_EXPORTER_OTLP_ENDPOINT = "http://localhost:4318"
|
||||||
|
$env:DATABASE_URL = "mysql://edu:changeme@localhost:3306/next_edu_cloud"
|
||||||
|
$env:REDIS_URL = "redis://localhost:6379"
|
||||||
|
$env:JWT_SECRET = "p1-dev-secret-change-in-production"
|
||||||
|
$env:KAFKA_BROKERS = "localhost:9092"
|
||||||
|
|
||||||
|
$pyEnv = @{
|
||||||
|
CLICKHOUSE_HOST = "localhost"
|
||||||
|
CLICKHOUSE_PORT = "8123"
|
||||||
|
CLICKHOUSE_USER = "default"
|
||||||
|
CLICKHOUSE_PASSWORD = "clickhouse"
|
||||||
|
CLICKHOUSE_DATABASE = "edu_analytics"
|
||||||
|
KAFKA_BROKERS = "localhost:9092"
|
||||||
|
OTEL_ENDPOINT = "http://localhost:4318"
|
||||||
|
DEV_MODE = "true"
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host " DEV_MODE=true (dev-token bypass)" -ForegroundColor Green
|
||||||
|
Write-Host " OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318" -ForegroundColor Green
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 3. Start application services =====
|
||||||
|
Write-Host "[3/4] Starting application services (11 windows)..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$services = @(
|
||||||
|
@{Title="edu-app-classes"; Cmd="pnpm"; Args=@("--filter","@edu/classes-service","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-iam"; Cmd="pnpm"; Args=@("--filter","@edu/iam-service","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-teacher-bff"; Cmd="pnpm"; Args=@("--filter","@edu/teacher-bff","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-core-edu"; Cmd="pnpm"; Args=@("--filter","@edu/core-edu-service","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-content"; Cmd="pnpm"; Args=@("--filter","@edu/content-service","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-msg"; Cmd="pnpm"; Args=@("--filter","@edu/msg-service","dev"); Dir="$ProjectRoot"},
|
||||||
|
@{Title="edu-app-data-ana"; Cmd="uv"; Args=@("run","uvicorn","data_ana.main:app","--host","0.0.0.0","--port","3006","--reload"); Dir="$ProjectRoot\services\data-ana"; PyEnv=$true},
|
||||||
|
@{Title="edu-app-ai"; Cmd="uv"; Args=@("run","uvicorn","ai.main:app","--host","0.0.0.0","--port","3008","--reload"); Dir="$ProjectRoot\services\ai"; PyEnv=$true},
|
||||||
|
@{Title="edu-app-api-gateway"; Cmd="go"; Args=@("run","."); Dir="$ProjectRoot\services\api-gateway"; GoEnv=$true},
|
||||||
|
@{Title="edu-app-push-gateway"; Cmd="go"; Args=@("run","."); Dir="$ProjectRoot\services\push-gateway";GoEnv=$true},
|
||||||
|
@{Title="edu-app-teacher-portal";Cmd="pnpm";Args=@("--filter","teacher-portal","dev"); Dir="$ProjectRoot"}
|
||||||
|
)
|
||||||
|
|
||||||
|
foreach ($svc in $services) {
|
||||||
|
$cmdStr = "$($svc.Cmd) $($svc.Args -join ' ')"
|
||||||
|
|
||||||
|
$psCmd = "Set-Location '$($svc.Dir)'; "
|
||||||
|
|
||||||
|
if ($svc.PyEnv) {
|
||||||
|
foreach ($kv in $pyEnv.GetEnumerator()) {
|
||||||
|
$psCmd += "`$env:$($kv.Key)='$($kv.Value)'; "
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($svc.GoEnv) {
|
||||||
|
$psCmd += "`$env:Path = 'C:\Program Files\Go\bin;' + `$env:Path; "
|
||||||
|
}
|
||||||
|
|
||||||
|
$psCmd += "$cmdStr; Write-Host ''; Write-Host 'Service stopped. Press any key to close...' -ForegroundColor Yellow; `$null = `$Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown')"
|
||||||
|
|
||||||
|
Start-Process -FilePath "powershell" -ArgumentList "-NoExit","-Command",$psCmd -WindowStyle Normal | Out-Null
|
||||||
|
|
||||||
|
Write-Host " [START] $($svc.Title)..." -ForegroundColor Green
|
||||||
|
Start-Sleep -Milliseconds 500
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host " All services started in new windows. Waiting 30s for init..." -ForegroundColor Yellow
|
||||||
|
Start-Sleep -Seconds 30
|
||||||
|
|
||||||
|
# ===== 4. Health check =====
|
||||||
|
Write-Host "[4/4] Health check..." -ForegroundColor Yellow
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
$healthServices = @(
|
||||||
|
@{Name="classes"; Url="http://localhost:3001/healthz"},
|
||||||
|
@{Name="iam"; Url="http://localhost:3002/healthz"},
|
||||||
|
@{Name="teacher-bff"; Url="http://localhost:3003/healthz"},
|
||||||
|
@{Name="core-edu"; Url="http://localhost:3004/healthz"},
|
||||||
|
@{Name="content"; Url="http://localhost:3005/healthz"},
|
||||||
|
@{Name="data-ana"; Url="http://localhost:3006/healthz"},
|
||||||
|
@{Name="msg"; Url="http://localhost:3007/healthz"},
|
||||||
|
@{Name="ai"; Url="http://localhost:3008/healthz"},
|
||||||
|
@{Name="api-gateway"; Url="http://localhost:8080/healthz"},
|
||||||
|
@{Name="push-gateway"; Url="http://localhost:8081/healthz"},
|
||||||
|
@{Name="teacher-portal";Url="http://localhost:3000/"}
|
||||||
|
)
|
||||||
|
|
||||||
|
$okCount = 0
|
||||||
|
$failCount = 0
|
||||||
|
foreach ($svc in $healthServices) {
|
||||||
|
$retries = 0
|
||||||
|
$maxRetries = 5
|
||||||
|
$success = $false
|
||||||
|
while ($retries -lt $maxRetries -and -not $success) {
|
||||||
|
try {
|
||||||
|
$null = Invoke-RestMethod -Uri $svc.Url -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] $($svc.Name)" -ForegroundColor Green
|
||||||
|
$success = $true
|
||||||
|
$okCount++
|
||||||
|
} catch {
|
||||||
|
$retries++
|
||||||
|
if ($retries -lt $maxRetries) {
|
||||||
|
Start-Sleep -Seconds 3
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (-not $success) {
|
||||||
|
Write-Host " [FAIL] $($svc.Name) (failed after $maxRetries retries)" -ForegroundColor Red
|
||||||
|
$failCount++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Started: [OK] $okCount ready / [FAIL] $failCount failed" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "Next steps:" -ForegroundColor Yellow
|
||||||
|
Write-Host " Health check: .\scripts\health-check.ps1" -ForegroundColor White
|
||||||
|
Write-Host " CDC test: .\scripts\test-cdc.ps1" -ForegroundColor White
|
||||||
|
Write-Host " Stop all: .\scripts\stop-all.ps1" -ForegroundColor White
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
if ($failCount -gt 0) {
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
98
scripts/stop-all.ps1
Normal file
98
scripts/stop-all.ps1
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
<#
|
||||||
|
.SYNOPSIS
|
||||||
|
Edu stop all application services script
|
||||||
|
.DESCRIPTION
|
||||||
|
Closes all edu-app-* terminal windows (started by start-all.ps1)
|
||||||
|
Optional: kill processes by port (fallback when windows are closed but processes linger)
|
||||||
|
.PARAMETER KillByPort
|
||||||
|
Kill processes by port (fallback when windows are closed but processes still alive)
|
||||||
|
.EXAMPLE
|
||||||
|
.\scripts\stop-all.ps1
|
||||||
|
.\scripts\stop-all.ps1 -KillByPort
|
||||||
|
#>
|
||||||
|
param(
|
||||||
|
[switch]$KillByPort
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Continue"
|
||||||
|
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Edu Stop All Services" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 1. Close app service terminal windows =====
|
||||||
|
Write-Host "[1/2] Closing app service windows..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$appTitles = @(
|
||||||
|
"edu-app-classes",
|
||||||
|
"edu-app-iam",
|
||||||
|
"edu-app-teacher-bff",
|
||||||
|
"edu-app-core-edu",
|
||||||
|
"edu-app-content",
|
||||||
|
"edu-app-msg",
|
||||||
|
"edu-app-data-ana",
|
||||||
|
"edu-app-ai",
|
||||||
|
"edu-app-api-gateway",
|
||||||
|
"edu-app-push-gateway",
|
||||||
|
"edu-app-teacher-portal"
|
||||||
|
)
|
||||||
|
|
||||||
|
$closedCount = 0
|
||||||
|
foreach ($title in $appTitles) {
|
||||||
|
$procs = Get-Process -Name "powershell","pwsh","node","python","uvicorn","go" -ErrorAction SilentlyContinue |
|
||||||
|
Where-Object { $_.MainWindowTitle -like "*$title*" }
|
||||||
|
|
||||||
|
if ($procs) {
|
||||||
|
foreach ($p in $procs) {
|
||||||
|
try {
|
||||||
|
Stop-Process -Id $p.Id -Force -ErrorAction Stop
|
||||||
|
Write-Host " [OK] Closed $title (PID $($p.Id))" -ForegroundColor Green
|
||||||
|
$closedCount++
|
||||||
|
} catch {
|
||||||
|
Write-Host " [WARN] Cannot close $title (PID $($p.Id)): $($_.Exception.Message)" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
Write-Host " [--] $title window not found" -ForegroundColor Gray
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host " Closed $closedCount windows" -ForegroundColor Green
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 2. Kill by port (optional) =====
|
||||||
|
if ($KillByPort) {
|
||||||
|
Write-Host "[2/2] Killing processes by port..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$ports = @(3000,3001,3002,3003,3004,3005,3006,3007,3008,8080,8081)
|
||||||
|
|
||||||
|
foreach ($port in $ports) {
|
||||||
|
$connections = Get-NetTCPConnection -LocalPort $port -State Listen -ErrorAction SilentlyContinue
|
||||||
|
if ($connections) {
|
||||||
|
foreach ($conn in $connections) {
|
||||||
|
try {
|
||||||
|
$proc = Get-Process -Id $conn.OwningProcess -ErrorAction Stop
|
||||||
|
Stop-Process -Id $conn.OwningProcess -Force -ErrorAction Stop
|
||||||
|
Write-Host " [OK] Port $port -> killed $($proc.ProcessName) (PID $($conn.OwningProcess))" -ForegroundColor Green
|
||||||
|
} catch {
|
||||||
|
Write-Host " [WARN] Port $port -> cannot kill PID $($conn.OwningProcess)" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
Write-Host " [--] Port $port free" -ForegroundColor Gray
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
Write-Host "[2/2] Skipping port kill (use -KillByPort to enable)" -ForegroundColor Gray
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Stop complete" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "To stop infrastructure:" -ForegroundColor Yellow
|
||||||
|
Write-Host " docker compose -f infra/docker-compose.yml --profile p6 --profile observability down" -ForegroundColor White
|
||||||
|
Write-Host ""
|
||||||
167
scripts/test-cdc.ps1
Normal file
167
scripts/test-cdc.ps1
Normal file
@@ -0,0 +1,167 @@
|
|||||||
|
<#
|
||||||
|
.SYNOPSIS
|
||||||
|
Edu CDC pipeline end-to-end test script
|
||||||
|
.DESCRIPTION
|
||||||
|
Validates the full CDC pipeline: MySQL binlog -> Debezium -> Kafka -> data-ana -> ClickHouse
|
||||||
|
Steps:
|
||||||
|
1. Insert test grade into MySQL
|
||||||
|
2. Wait for Debezium capture + data-ana consume
|
||||||
|
3. Query ClickHouse to verify data synced
|
||||||
|
4. Call data-ana API to verify query works
|
||||||
|
.PARAMETER StudentId
|
||||||
|
Custom test student_id (default: cdc-test-<timestamp>)
|
||||||
|
.EXAMPLE
|
||||||
|
.\scripts\test-cdc.ps1
|
||||||
|
.\scripts\test-cdc.ps1 -StudentId "my-test-001"
|
||||||
|
#>
|
||||||
|
param(
|
||||||
|
[string]$StudentId = "cdc-test-$(Get-Date -Format 'yyyyMMddHHmmss')"
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Continue"
|
||||||
|
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " Edu CDC Pipeline E2E Test" -ForegroundColor Cyan
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
$examId = "exam-$StudentId"
|
||||||
|
$gradeId = "grade-$StudentId"
|
||||||
|
$testScore = 92.5
|
||||||
|
$classId = "cls-cdc-test"
|
||||||
|
|
||||||
|
Write-Host "Test parameters:" -ForegroundColor Yellow
|
||||||
|
Write-Host " StudentId: $StudentId"
|
||||||
|
Write-Host " ExamId: $examId"
|
||||||
|
Write-Host " GradeId: $gradeId"
|
||||||
|
Write-Host " Score: $testScore"
|
||||||
|
Write-Host " ClassId: $classId"
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 1. Pre-check =====
|
||||||
|
Write-Host "[1/5] Pre-check..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
# Check Debezium connector
|
||||||
|
$connectorStatus = $null
|
||||||
|
try {
|
||||||
|
$connectorStatus = Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status" -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
} catch {
|
||||||
|
Write-Host " [FAIL] Debezium Connect not reachable. Start infrastructure first." -ForegroundColor Red
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
if ($connectorStatus.connector.state -ne "RUNNING") {
|
||||||
|
Write-Host " [FAIL] Debezium connector state: $($connectorStatus.connector.state)" -ForegroundColor Red
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
Write-Host " [OK] Debezium connector: RUNNING" -ForegroundColor Green
|
||||||
|
|
||||||
|
# Check data-ana service
|
||||||
|
try {
|
||||||
|
$null = Invoke-RestMethod -Uri "http://localhost:3006/healthz" -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] data-ana service: running" -ForegroundColor Green
|
||||||
|
} catch {
|
||||||
|
Write-Host " [FAIL] data-ana service not reachable. Start application services first." -ForegroundColor Red
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# Check CDC consumer status
|
||||||
|
try {
|
||||||
|
$readyz = Invoke-RestMethod -Uri "http://localhost:3006/readyz" -Method Get -TimeoutSec 3 -ErrorAction Stop
|
||||||
|
$cdcStatus = $readyz.services.cdc_consumer
|
||||||
|
if ($cdcStatus -ne "running") {
|
||||||
|
Write-Host " [WARN] CDC consumer status: $cdcStatus (KAFKA_BROKERS may not be set)" -ForegroundColor Yellow
|
||||||
|
} else {
|
||||||
|
Write-Host " [OK] CDC consumer: running" -ForegroundColor Green
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
Write-Host " [WARN] Cannot get /readyz status" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 2. Insert test data into MySQL =====
|
||||||
|
Write-Host "[2/5] Inserting test data into MySQL..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$sqlInsert = @"
|
||||||
|
INSERT INTO core_edu_exams (id, class_id, subject_id, title, exam_date, total_score, created_at, updated_at)
|
||||||
|
VALUES ('$examId', '$classId', 'sub-math', 'CDC Test Exam', NOW(), 100, NOW(), NOW())
|
||||||
|
ON DUPLICATE KEY UPDATE updated_at=NOW();
|
||||||
|
INSERT INTO core_edu_grades (id, exam_id, student_id, score, rank_in_class, created_at, updated_at)
|
||||||
|
VALUES ('$gradeId', '$examId', '$StudentId', $testScore, 1, NOW(), NOW())
|
||||||
|
ON DUPLICATE KEY UPDATE score=$testScore, updated_at=NOW();
|
||||||
|
"@
|
||||||
|
|
||||||
|
docker exec edu-mysql mysql -uedu -pchangeme next_edu_cloud -e $sqlInsert 2>&1 | Out-Null
|
||||||
|
if ($LASTEXITCODE -eq 0) {
|
||||||
|
Write-Host " [OK] Inserted: exam=$examId / grade=$gradeId / student=$StudentId / score=$testScore" -ForegroundColor Green
|
||||||
|
} else {
|
||||||
|
Write-Host " [FAIL] MySQL insert failed" -ForegroundColor Red
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 3. Wait for CDC propagation =====
|
||||||
|
Write-Host "[3/5] Waiting for CDC propagation (5s)..." -ForegroundColor Yellow
|
||||||
|
Start-Sleep -Seconds 5
|
||||||
|
Write-Host " [OK] Wait complete" -ForegroundColor Green
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 4. Verify ClickHouse data =====
|
||||||
|
Write-Host "[4/5] Verifying ClickHouse data..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$chQuery = "SELECT student_id, class_id, exam_id, score, last_updated FROM edu_analytics.student_dashboard_view WHERE student_id = '$StudentId' ORDER BY last_updated DESC LIMIT 5"
|
||||||
|
$chResult = docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q $chQuery 2>$null
|
||||||
|
|
||||||
|
if ($chResult) {
|
||||||
|
Write-Host " [OK] ClickHouse returned data:" -ForegroundColor Green
|
||||||
|
Write-Host " $chResult" -ForegroundColor White
|
||||||
|
|
||||||
|
if ($chResult -match $StudentId -and $chResult -match "$testScore") {
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host " [OK] Verified: student_id match + score=$testScore match" -ForegroundColor Green
|
||||||
|
if ($chResult -match $classId) {
|
||||||
|
Write-Host " [OK] class_id filled via exam cache: $classId" -ForegroundColor Green
|
||||||
|
} else {
|
||||||
|
Write-Host " [WARN] class_id not filled (exam cache may have missed, check event order)" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
Write-Host " [FAIL] Data mismatch: expected student=$StudentId, score=$testScore" -ForegroundColor Red
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
Write-Host " [FAIL] ClickHouse has no data for student_id=$StudentId" -ForegroundColor Red
|
||||||
|
Write-Host " Possible causes:" -ForegroundColor Yellow
|
||||||
|
Write-Host " 1. Debezium did not capture MySQL change (check connector status)" -ForegroundColor White
|
||||||
|
Write-Host " 2. data-ana consumer not running (check /readyz cdc_consumer)" -ForegroundColor White
|
||||||
|
Write-Host " 3. Kafka topic name mismatch (check debezium-register.json)" -ForegroundColor White
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
# ===== 5. Verify data-ana API =====
|
||||||
|
Write-Host "[5/5] Verifying data-ana query API..." -ForegroundColor Yellow
|
||||||
|
|
||||||
|
$h = @{Authorization="Bearer dev-token"}
|
||||||
|
|
||||||
|
try {
|
||||||
|
$weakness = Invoke-RestMethod -Uri "http://localhost:3006/analytics/student/$StudentId/weakness" -Method Get -Headers $h -TimeoutSec 5 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] /analytics/student/$StudentId/weakness" -ForegroundColor Green
|
||||||
|
Write-Host " Response: $($weakness | ConvertTo-Json -Depth 3)" -ForegroundColor Gray
|
||||||
|
} catch {
|
||||||
|
Write-Host " [WARN] /analytics/student/$StudentId/weakness failed: $($_.Exception.Message)" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
$perf = Invoke-RestMethod -Uri "http://localhost:3006/analytics/class/$classId/performance" -Method Get -Headers $h -TimeoutSec 5 -ErrorAction Stop
|
||||||
|
Write-Host " [OK] /analytics/class/$classId/performance" -ForegroundColor Green
|
||||||
|
Write-Host " Response: $($perf | ConvertTo-Json -Depth 3)" -ForegroundColor Gray
|
||||||
|
} catch {
|
||||||
|
Write-Host " [WARN] /analytics/class/$classId/performance failed: $($_.Exception.Message)" -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
|
Write-Host " CDC Pipeline Test Complete" -ForegroundColor Cyan
|
||||||
|
Write-Host " MySQL -> Debezium -> Kafka -> data-ana -> ClickHouse [OK]" -ForegroundColor Green
|
||||||
|
Write-Host "========================================" -ForegroundColor Cyan
|
||||||
Reference in New Issue
Block a user