diff --git a/docs/standards/full-stack-runbook.md b/docs/standards/full-stack-runbook.md new file mode 100644 index 0000000..5d67f17 --- /dev/null +++ b/docs/standards/full-stack-runbook.md @@ -0,0 +1,279 @@ +# 全链路启用与测试手册(Full Stack Runbook) + +> 版本:1.0 +> 日期:2026-07-09 +> 适用范围:Edu 微服务架构(P6 阶段:11 基础设施 + 10 应用服务 + 1 前端) +> 关联文档:[local-dev-runbook](./local-dev-runbook.md)、[004 架构影响地图](../architecture/004_architecture_impact_map.md)、[known-issues](../troubleshooting/known-issues.md) + +--- + +## 1. 前置条件 + +| 工具 | 版本要求 | 验证命令 | +| -------------- | -------- | ------------------------ | +| Node.js | ≥ 20 | `node -v` | +| pnpm | ≥ 9 | `pnpm -v` | +| Go | 1.22+ | `go version` | +| uv | 0.4+ | `uv --version` | +| Docker | 24+ | `docker version` | +| Docker Compose | v2+ | `docker compose version` | + +> Windows 用户:Go 工具链若不在 PATH,临时加入:`$env:Path = "C:\Program Files\Go\bin;" + $env:Path` + +--- + +## 2. 服务端口矩阵 + +### 2.1 应用服务(10 个) + +| 端口 | 服务 | 语言/框架 | 启动方式 | +| ---- | -------------- | --------- | ---------------------------------------------- | +| 3000 | teacher-portal | Next.js | `pnpm --filter teacher-portal dev` | +| 3001 | classes | NestJS | `pnpm --filter @edu/classes-service dev` | +| 3002 | iam | NestJS | `pnpm --filter @edu/iam-service dev` | +| 3003 | teacher-bff | NestJS | `pnpm --filter @edu/teacher-bff dev` | +| 3004 | core-edu | NestJS | `pnpm --filter @edu/core-edu-service dev` | +| 3005 | content | NestJS | `pnpm --filter @edu/content-service dev` | +| 3006 | data-ana | FastAPI | `uv run uvicorn data_ana.main:app --port 3006` | +| 3007 | msg | NestJS | `pnpm --filter @edu/msg-service dev` | +| 3008 | ai | FastAPI | `uv run uvicorn ai.main:app --port 3008` | +| 8080 | api-gateway | Go (Gin) | `go run .` | +| 8081 | push-gateway | Go (Gin) | `go run .` | + +### 2.2 基础设施(11 个) + +| 端口 | 服务 | 用途 | +| --------- | ------------------ | --------------------- | +| 3306 | MySQL 8 | 写模型主库 | +| 6379 | Redis 7 | 缓存/会话 | +| 8083 | Debezium Connect | CDC source connector | +| 8123/9000 | ClickHouse 24.3 | 读模型宽表 | +| 9092 | Kafka 7.6 | 事件总线 | +| 7474/7687 | Neo4j 5.20 | 知识图谱 | +| 9200 | Elasticsearch 8.13 | 题库检索 | +| 9090 | Prometheus | 指标采集 | +| 3030 | Grafana | 可视化(admin/admin) | +| 16686 | Jaeger | 分布式追踪 UI | +| 4318 | OTLP Collector | OTel span 接收 | + +--- + +## 3. 一键启动 + +### 3.1 启动基础设施 + +```powershell +cd e:\Desktop\Edu\infra +docker compose -f docker-compose.yml --profile p6 --profile observability up -d +``` + +等待所有容器 healthy(约 60 秒): + +```powershell +docker ps --filter "name=edu-" --format "table {{.Names}}\t{{.Status}}" +``` + +### 3.2 一键启动所有应用服务 + +使用项目根目录的启动脚本: + +```powershell +cd e:\Desktop\Edu +.\scripts\start-all.ps1 +``` + +该脚本会: + +1. 检查基础设施健康状态 +2. 为每个应用服务启动独立后台窗口(带标题) +3. 自动注入 Python 服务的环境变量(CLICKHOUSE/KAFKA/OTEL) +4. 等待所有服务健康检查通过 + +### 3.3 健康检查 + +```powershell +cd e:\Desktop\Edu +.\scripts\health-check.ps1 +``` + +预期输出:所有服务 ✅ + +--- + +## 4. 端到端链路测试 + +### 4.1 IAM 注册 + 登录 + +```powershell +$h = @{Authorization="Bearer dev-token"} + +# 注册 +$body = @{username="testteacher";password="Test@1234";email="test@edu.com";role="teacher"} | ConvertTo-Json +Invoke-RestMethod -Uri "http://localhost:8080/iam/auth/register" -Method Post -Body $body -ContentType "application/json" -Headers $h + +# 登录 +$loginBody = @{username="testteacher";password="Test@1234"} | ConvertTo-Json +$resp = Invoke-RestMethod -Uri "http://localhost:8080/iam/auth/login" -Method Post -Body $loginBody -ContentType "application/json" +$token = $resp.data.accessToken +Write-Host "Token: $token" +``` + +### 4.2 Classes CRUD + +```powershell +# 创建班级 +$classBody = @{name="高三一班";gradeId="grade-1";headTeacherId=""} | ConvertTo-Json +Invoke-RestMethod -Uri "http://localhost:8080/classes" -Method Post -Body $classBody -ContentType "application/json" -Headers $h + +# 查询班级列表 +Invoke-RestMethod -Uri "http://localhost:8080/classes" -Method Get -Headers $h +``` + +### 4.3 CDC 完整链路(MySQL → Debezium → Kafka → data-ana → ClickHouse) + +```powershell +# 1. 向 MySQL 插入成绩(触发 binlog) +docker exec edu-mysql mysql -uedu -pchangeme next_edu_cloud -e " +INSERT INTO core_edu_exams (id, class_id, subject_id, title, exam_date, total_score, created_at, updated_at) +VALUES ('exam-cdc-test-001','cls-test-001','sub-math','CDC测试考试',NOW(),100,NOW(),NOW()) +ON DUPLICATE KEY UPDATE updated_at=NOW(); + +INSERT INTO core_edu_grades (id, exam_id, student_id, score, rank_in_class, created_at, updated_at) +VALUES ('grade-cdc-001','exam-cdc-test-001','student-cdc-001',92.5,1,NOW(),NOW()) +ON DUPLICATE KEY UPDATE score=92.5, updated_at=NOW(); +" + +# 2. 等待 Debezium 捕获 + data-ana 消费 +Start-Sleep -Seconds 5 + +# 3. 验证 ClickHouse 已同步 +docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q " +SELECT student_id, class_id, exam_id, score, last_updated +FROM edu_analytics.student_dashboard_view +WHERE student_id = 'student-cdc-001' +ORDER BY last_updated DESC +" +# 预期:返回一行,score=92.5,class_id='cls-test-001' +``` + +### 4.4 data-ana 查询 API + +```powershell +# 学生学情看板 +Invoke-RestMethod -Uri "http://localhost:3006/analytics/student/student-cdc-001/weakness" -Headers $h + +# 班级成绩分析 +Invoke-RestMethod -Uri "http://localhost:3006/analytics/class/cls-test-001/performance" -Headers $h + +# CDC 消费者状态 +Invoke-RestMethod -Uri "http://localhost:3006/readyz" +# 预期: cdc_consumer = "running" +``` + +### 4.5 可观测性验证 + +| 检查项 | URL | 预期 | +| ------------------ | ----------------------------------- | ------------------------ | +| Prometheus targets | http://localhost:9090/targets | 所有 target UP | +| Grafana 面板 | http://localhost:3030 (admin/admin) | 可登录 | +| Jaeger UI | http://localhost:16686 | 可搜索到各 service trace | +| data-ana /metrics | http://localhost:3006/metrics | Prometheus 格式输出 | +| iam /metrics | http://localhost:3002/metrics | Prometheus 格式输出 | + +**Jaeger trace 验证步骤**: + +1. 打开 http://localhost:16686 +2. Service 下拉框应能看到 `iam`、`classes`、`data-ana`、`api-gateway` 等 +3. 选择任一服务 → Find Traces → 应看到 HTTP 请求的自动埋点 span + +--- + +## 5. 一键停止 + +### 5.1 停止应用服务 + +```powershell +cd e:\Desktop\Edu +.\scripts\stop-all.ps1 +``` + +该脚本会关闭所有 `edu-app-*` 标题的终端窗口。 + +### 5.2 停止基础设施 + +```powershell +cd e:\Desktop\Edu\infra +docker compose -f docker-compose.yml --profile p6 --profile observability down +``` + +--- + +## 6. 故障排查 + +### 6.1 端口占用 + +```powershell +# 查看占用端口的进程 +netstat -ano | findstr :3001 +# 终止进程 +taskkill /PID /F +``` + +### 6.2 基础设施未启动 + +```powershell +# 检查容器状态 +docker ps --filter "name=edu-" + +# 重启单个容器 +docker restart edu-mysql + +# 查看日志 +docker logs edu-debezium --tail 50 +``` + +### 6.3 CDC 链路断开 + +```powershell +# 1. 检查 Debezium connector 状态 +Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status" + +# 2. 重启 connector +Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/restart" -Method Post + +# 3. 检查 Kafka topic 是否有数据 +docker exec edu-kafka kafka-console-consumer --bootstrap-server localhost:9092 --topic edu-cdc.next_edu_cloud.core_edu_grades --from-beginning --max-messages 1 + +# 4. 检查 data-ana 消费者日志 +# 查看 data-ana 终端窗口的 cdc_consumer_started / cdc_event_received 日志 +``` + +### 6.4 OTel trace 未上报 + +```powershell +# 1. 检查 Jaeger 是否收到 trace +Invoke-RestMethod -Uri "http://localhost:16686/api/services" + +# 2. 检查 OTLP endpoint 是否可达 +Invoke-RestMethod -Uri "http://localhost:4318/v1/traces" -Method Post -ContentType "application/json" -Body "{}" + +# 3. 检查服务环境变量 +# 确保 OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 已设置 +``` + +--- + +## 7. 速查:常用命令 + +| 场景 | 命令 | +| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------- | +| 启动基础设施 | `docker compose -f infra/docker-compose.yml --profile p6 --profile observability up -d` | +| 一键启动应用 | `.\scripts\start-all.ps1` | +| 一键停止应用 | `.\scripts\stop-all.ps1` | +| 健康检查 | `.\scripts\health-check.ps1` | +| CDC 链路验证 | `.\scripts\test-cdc.ps1` | +| 查看容器状态 | `docker ps --filter "name=edu-"` | +| 查看 Debezium 状态 | `Invoke-RestMethod http://localhost:8083/connectors/edu-mysql-source/status` | +| ClickHouse 查询 | `docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q "SELECT * FROM edu_analytics.student_dashboard_view LIMIT 10"` | +| Kafka topic 列表 | `docker exec edu-kafka kafka-topics --bootstrap-server localhost:9092 --list` | +| Prometheus 查询 | `Invoke-RestMethod "http://localhost:9090/api/v1/query?query=up"` | diff --git a/scripts/health-check.ps1 b/scripts/health-check.ps1 new file mode 100644 index 0000000..bb7b0b2 --- /dev/null +++ b/scripts/health-check.ps1 @@ -0,0 +1,152 @@ +<# +.SYNOPSIS + Edu health check script +.DESCRIPTION + Checks health of all infrastructure containers + application services + observability endpoints +.EXAMPLE + .\scripts\health-check.ps1 +#> + +$ErrorActionPreference = "Continue" + +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Edu Health Check" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" + +# ===== 1. Infrastructure containers ===== +Write-Host "[1/3] Infrastructure Containers" -ForegroundColor Yellow + +$infraContainers = @( + @{Name="MySQL"; Container="edu-mysql"}, + @{Name="Redis"; Container="edu-redis"}, + @{Name="Kafka"; Container="edu-kafka"}, + @{Name="Zookeeper"; Container="edu-zookeeper"}, + @{Name="ClickHouse"; Container="edu-clickhouse"}, + @{Name="Debezium"; Container="edu-debezium"}, + @{Name="Neo4j"; Container="edu-neo4j"}, + @{Name="Elasticsearch"; Container="edu-es"}, + @{Name="Jaeger"; Container="edu-jaeger"}, + @{Name="Prometheus"; Container="edu-prometheus"}, + @{Name="Grafana"; Container="edu-grafana"} +) + +$infraOk = 0 +$infraFail = 0 +foreach ($svc in $infraContainers) { + $status = docker inspect --format='{{.State.Health.Status}}' $svc.Container 2>$null + if ($status -eq "healthy") { + Write-Host " [OK] $($svc.Name)" -ForegroundColor Green + $infraOk++ + } elseif ($status -eq "running") { + Write-Host " [WARN] $($svc.Name) (running, no healthcheck)" -ForegroundColor Yellow + $infraOk++ + } else { + Write-Host " [FAIL] $($svc.Name) (status=$status)" -ForegroundColor Red + $infraFail++ + } +} + +Write-Host "" + +# ===== 2. Application services ===== +Write-Host "[2/3] Application Services" -ForegroundColor Yellow + +$appServices = @( + @{Name="classes"; Url="http://localhost:3001/healthz"}, + @{Name="iam"; Url="http://localhost:3002/healthz"}, + @{Name="teacher-bff"; Url="http://localhost:3003/healthz"}, + @{Name="core-edu"; Url="http://localhost:3004/healthz"}, + @{Name="content"; Url="http://localhost:3005/healthz"}, + @{Name="data-ana"; Url="http://localhost:3006/healthz"}, + @{Name="msg"; Url="http://localhost:3007/healthz"}, + @{Name="ai"; Url="http://localhost:3008/healthz"}, + @{Name="api-gateway"; Url="http://localhost:8080/healthz"}, + @{Name="push-gateway"; Url="http://localhost:8081/healthz"}, + @{Name="teacher-portal";Url="http://localhost:3000/"} +) + +$appOk = 0 +$appFail = 0 +foreach ($svc in $appServices) { + try { + $null = Invoke-RestMethod -Uri $svc.Url -Method Get -TimeoutSec 3 -ErrorAction Stop + Write-Host " [OK] $($svc.Name)" -ForegroundColor Green + $appOk++ + } catch { + Write-Host " [FAIL] $($svc.Name)" -ForegroundColor Red + $appFail++ + } +} + +Write-Host "" + +# ===== 3. Observability endpoints ===== +Write-Host "[3/3] Observability Endpoints" -ForegroundColor Yellow + +$obsEndpoints = @( + @{Name="Prometheus API"; Url="http://localhost:9090/api/v1/query?query=up"}, + @{Name="Jaeger API"; Url="http://localhost:16686/api/services"}, + @{Name="Debezium Connect"; Url="http://localhost:8083/connectors"}, + @{Name="data-ana /metrics";Url="http://localhost:3006/metrics"}, + @{Name="iam /metrics"; Url="http://localhost:3002/metrics"} +) + +$obsOk = 0 +$obsFail = 0 +foreach ($ep in $obsEndpoints) { + try { + $null = Invoke-RestMethod -Uri $ep.Url -Method Get -TimeoutSec 3 -ErrorAction Stop + Write-Host " [OK] $($ep.Name)" -ForegroundColor Green + $obsOk++ + } catch { + Write-Host " [FAIL] $($ep.Name)" -ForegroundColor Red + $obsFail++ + } +} + +# ===== 4. CDC pipeline status ===== +Write-Host "" +Write-Host "[Extra] CDC Pipeline Status" -ForegroundColor Yellow + +$connectorStatus = $null +try { + $connectorStatus = Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status" -Method Get -TimeoutSec 3 -ErrorAction Stop +} catch { + Write-Host " [FAIL] Debezium connector not registered or error" -ForegroundColor Red +} + +if ($connectorStatus) { + $connectorState = $connectorStatus.connector.state + $tasks = @($connectorStatus.tasks) + if ($tasks.Count -gt 0) { + $taskState = $tasks[0].state + } else { + $taskState = "UNKNOWN" + } + if ($connectorState -eq "RUNNING" -and $taskState -eq "RUNNING") { + $color = "Green" + } else { + $color = "Yellow" + } + Write-Host " Connector: $connectorState / Task: $taskState" -ForegroundColor $color +} + +$chSql = 'SELECT count(*) FROM edu_analytics.student_dashboard_view' +$chOutput = docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q $chSql 2>&1 +$chCount = "$chOutput".Trim() +if ($chCount -match '^\d+$') { + Write-Host " ClickHouse student_dashboard_view: $chCount records" -ForegroundColor Green +} else { + Write-Host " ClickHouse query failed or empty" -ForegroundColor Yellow +} + +# ===== Summary ===== +Write-Host "" +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Infra: $infraOk/$($infraContainers.Count) | App: $appOk/$($appServices.Count) | Obs: $obsOk/$($obsEndpoints.Count)" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan + +if ($infraFail -gt 0 -or $appFail -gt 0 -or $obsFail -gt 0) { + exit 1 +} diff --git a/scripts/start-all.ps1 b/scripts/start-all.ps1 new file mode 100644 index 0000000..f1b3d47 --- /dev/null +++ b/scripts/start-all.ps1 @@ -0,0 +1,185 @@ +<# +.SYNOPSIS + Edu start all application services script +.DESCRIPTION + Starts 10 application services + 1 frontend (teacher-portal) + Each service runs in a separate terminal window for log visibility + Automatically injects Python service env vars (CLICKHOUSE/KAFKA/OTEL) + Prerequisite: infrastructure containers (MySQL/Redis/Kafka/ClickHouse etc) must be running +.PARAMETER SkipInfraCheck + Skip infrastructure health check (use when infra is known to be running) +.EXAMPLE + .\scripts\start-all.ps1 + .\scripts\start-all.ps1 -SkipInfraCheck +#> +param( + [switch]$SkipInfraCheck +) + +$ErrorActionPreference = "Stop" +$ProjectRoot = Split-Path -Parent $PSScriptRoot + +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Edu Start All Services" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" + +# ===== 1. Infrastructure health check ===== +if (-not $SkipInfraCheck) { + Write-Host "[1/4] Checking infrastructure health..." -ForegroundColor Yellow + $infraServices = @( + @{Name="MySQL"; Container="edu-mysql"}, + @{Name="Redis"; Container="edu-redis"}, + @{Name="Kafka"; Container="edu-kafka"}, + @{Name="ClickHouse"; Container="edu-clickhouse"}, + @{Name="Debezium"; Container="edu-debezium"}, + @{Name="Jaeger"; Container="edu-jaeger"} + ) + $allHealthy = $true + foreach ($svc in $infraServices) { + $status = docker inspect --format='{{.State.Health.Status}}' $svc.Container 2>$null + if ($status -eq "healthy") { + Write-Host " [OK] $($svc.Name) ($($svc.Container))" -ForegroundColor Green + } elseif ($status -eq "running") { + Write-Host " [WARN] $($svc.Name) running but no healthcheck" -ForegroundColor Yellow + } else { + Write-Host " [FAIL] $($svc.Name) not started or unhealthy (status=$status)" -ForegroundColor Red + Write-Host " Please start infrastructure first:" -ForegroundColor Red + Write-Host " docker compose -f infra/docker-compose.yml --profile p6 --profile observability up -d" -ForegroundColor Red + $allHealthy = $false + } + } + if (-not $allHealthy) { + Write-Host "" + Write-Host "Infrastructure not ready. Start it first." -ForegroundColor Red + exit 1 + } + Write-Host "" +} + +# ===== 2. Environment variables ===== +Write-Host "[2/4] Preparing environment variables..." -ForegroundColor Yellow + +$env:DEV_MODE = "true" +$env:OTEL_EXPORTER_OTLP_ENDPOINT = "http://localhost:4318" +$env:DATABASE_URL = "mysql://edu:changeme@localhost:3306/next_edu_cloud" +$env:REDIS_URL = "redis://localhost:6379" +$env:JWT_SECRET = "p1-dev-secret-change-in-production" +$env:KAFKA_BROKERS = "localhost:9092" + +$pyEnv = @{ + CLICKHOUSE_HOST = "localhost" + CLICKHOUSE_PORT = "8123" + CLICKHOUSE_USER = "default" + CLICKHOUSE_PASSWORD = "clickhouse" + CLICKHOUSE_DATABASE = "edu_analytics" + KAFKA_BROKERS = "localhost:9092" + OTEL_ENDPOINT = "http://localhost:4318" + DEV_MODE = "true" +} + +Write-Host " DEV_MODE=true (dev-token bypass)" -ForegroundColor Green +Write-Host " OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318" -ForegroundColor Green +Write-Host "" + +# ===== 3. Start application services ===== +Write-Host "[3/4] Starting application services (11 windows)..." -ForegroundColor Yellow + +$services = @( + @{Title="edu-app-classes"; Cmd="pnpm"; Args=@("--filter","@edu/classes-service","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-iam"; Cmd="pnpm"; Args=@("--filter","@edu/iam-service","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-teacher-bff"; Cmd="pnpm"; Args=@("--filter","@edu/teacher-bff","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-core-edu"; Cmd="pnpm"; Args=@("--filter","@edu/core-edu-service","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-content"; Cmd="pnpm"; Args=@("--filter","@edu/content-service","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-msg"; Cmd="pnpm"; Args=@("--filter","@edu/msg-service","dev"); Dir="$ProjectRoot"}, + @{Title="edu-app-data-ana"; Cmd="uv"; Args=@("run","uvicorn","data_ana.main:app","--host","0.0.0.0","--port","3006","--reload"); Dir="$ProjectRoot\services\data-ana"; PyEnv=$true}, + @{Title="edu-app-ai"; Cmd="uv"; Args=@("run","uvicorn","ai.main:app","--host","0.0.0.0","--port","3008","--reload"); Dir="$ProjectRoot\services\ai"; PyEnv=$true}, + @{Title="edu-app-api-gateway"; Cmd="go"; Args=@("run","."); Dir="$ProjectRoot\services\api-gateway"; GoEnv=$true}, + @{Title="edu-app-push-gateway"; Cmd="go"; Args=@("run","."); Dir="$ProjectRoot\services\push-gateway";GoEnv=$true}, + @{Title="edu-app-teacher-portal";Cmd="pnpm";Args=@("--filter","teacher-portal","dev"); Dir="$ProjectRoot"} +) + +foreach ($svc in $services) { + $cmdStr = "$($svc.Cmd) $($svc.Args -join ' ')" + + $psCmd = "Set-Location '$($svc.Dir)'; " + + if ($svc.PyEnv) { + foreach ($kv in $pyEnv.GetEnumerator()) { + $psCmd += "`$env:$($kv.Key)='$($kv.Value)'; " + } + } + + if ($svc.GoEnv) { + $psCmd += "`$env:Path = 'C:\Program Files\Go\bin;' + `$env:Path; " + } + + $psCmd += "$cmdStr; Write-Host ''; Write-Host 'Service stopped. Press any key to close...' -ForegroundColor Yellow; `$null = `$Host.UI.RawUI.ReadKey('NoEcho,IncludeKeyDown')" + + Start-Process -FilePath "powershell" -ArgumentList "-NoExit","-Command",$psCmd -WindowStyle Normal | Out-Null + + Write-Host " [START] $($svc.Title)..." -ForegroundColor Green + Start-Sleep -Milliseconds 500 +} + +Write-Host "" +Write-Host " All services started in new windows. Waiting 30s for init..." -ForegroundColor Yellow +Start-Sleep -Seconds 30 + +# ===== 4. Health check ===== +Write-Host "[4/4] Health check..." -ForegroundColor Yellow +Write-Host "" + +$healthServices = @( + @{Name="classes"; Url="http://localhost:3001/healthz"}, + @{Name="iam"; Url="http://localhost:3002/healthz"}, + @{Name="teacher-bff"; Url="http://localhost:3003/healthz"}, + @{Name="core-edu"; Url="http://localhost:3004/healthz"}, + @{Name="content"; Url="http://localhost:3005/healthz"}, + @{Name="data-ana"; Url="http://localhost:3006/healthz"}, + @{Name="msg"; Url="http://localhost:3007/healthz"}, + @{Name="ai"; Url="http://localhost:3008/healthz"}, + @{Name="api-gateway"; Url="http://localhost:8080/healthz"}, + @{Name="push-gateway"; Url="http://localhost:8081/healthz"}, + @{Name="teacher-portal";Url="http://localhost:3000/"} +) + +$okCount = 0 +$failCount = 0 +foreach ($svc in $healthServices) { + $retries = 0 + $maxRetries = 5 + $success = $false + while ($retries -lt $maxRetries -and -not $success) { + try { + $null = Invoke-RestMethod -Uri $svc.Url -Method Get -TimeoutSec 3 -ErrorAction Stop + Write-Host " [OK] $($svc.Name)" -ForegroundColor Green + $success = $true + $okCount++ + } catch { + $retries++ + if ($retries -lt $maxRetries) { + Start-Sleep -Seconds 3 + } + } + } + if (-not $success) { + Write-Host " [FAIL] $($svc.Name) (failed after $maxRetries retries)" -ForegroundColor Red + $failCount++ + } +} + +Write-Host "" +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Started: [OK] $okCount ready / [FAIL] $failCount failed" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" +Write-Host "Next steps:" -ForegroundColor Yellow +Write-Host " Health check: .\scripts\health-check.ps1" -ForegroundColor White +Write-Host " CDC test: .\scripts\test-cdc.ps1" -ForegroundColor White +Write-Host " Stop all: .\scripts\stop-all.ps1" -ForegroundColor White +Write-Host "" + +if ($failCount -gt 0) { + exit 1 +} diff --git a/scripts/stop-all.ps1 b/scripts/stop-all.ps1 new file mode 100644 index 0000000..648f0db --- /dev/null +++ b/scripts/stop-all.ps1 @@ -0,0 +1,98 @@ +<# +.SYNOPSIS + Edu stop all application services script +.DESCRIPTION + Closes all edu-app-* terminal windows (started by start-all.ps1) + Optional: kill processes by port (fallback when windows are closed but processes linger) +.PARAMETER KillByPort + Kill processes by port (fallback when windows are closed but processes still alive) +.EXAMPLE + .\scripts\stop-all.ps1 + .\scripts\stop-all.ps1 -KillByPort +#> +param( + [switch]$KillByPort +) + +$ErrorActionPreference = "Continue" + +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Edu Stop All Services" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" + +# ===== 1. Close app service terminal windows ===== +Write-Host "[1/2] Closing app service windows..." -ForegroundColor Yellow + +$appTitles = @( + "edu-app-classes", + "edu-app-iam", + "edu-app-teacher-bff", + "edu-app-core-edu", + "edu-app-content", + "edu-app-msg", + "edu-app-data-ana", + "edu-app-ai", + "edu-app-api-gateway", + "edu-app-push-gateway", + "edu-app-teacher-portal" +) + +$closedCount = 0 +foreach ($title in $appTitles) { + $procs = Get-Process -Name "powershell","pwsh","node","python","uvicorn","go" -ErrorAction SilentlyContinue | + Where-Object { $_.MainWindowTitle -like "*$title*" } + + if ($procs) { + foreach ($p in $procs) { + try { + Stop-Process -Id $p.Id -Force -ErrorAction Stop + Write-Host " [OK] Closed $title (PID $($p.Id))" -ForegroundColor Green + $closedCount++ + } catch { + Write-Host " [WARN] Cannot close $title (PID $($p.Id)): $($_.Exception.Message)" -ForegroundColor Yellow + } + } + } else { + Write-Host " [--] $title window not found" -ForegroundColor Gray + } +} + +Write-Host "" +Write-Host " Closed $closedCount windows" -ForegroundColor Green +Write-Host "" + +# ===== 2. Kill by port (optional) ===== +if ($KillByPort) { + Write-Host "[2/2] Killing processes by port..." -ForegroundColor Yellow + + $ports = @(3000,3001,3002,3003,3004,3005,3006,3007,3008,8080,8081) + + foreach ($port in $ports) { + $connections = Get-NetTCPConnection -LocalPort $port -State Listen -ErrorAction SilentlyContinue + if ($connections) { + foreach ($conn in $connections) { + try { + $proc = Get-Process -Id $conn.OwningProcess -ErrorAction Stop + Stop-Process -Id $conn.OwningProcess -Force -ErrorAction Stop + Write-Host " [OK] Port $port -> killed $($proc.ProcessName) (PID $($conn.OwningProcess))" -ForegroundColor Green + } catch { + Write-Host " [WARN] Port $port -> cannot kill PID $($conn.OwningProcess)" -ForegroundColor Yellow + } + } + } else { + Write-Host " [--] Port $port free" -ForegroundColor Gray + } + } +} else { + Write-Host "[2/2] Skipping port kill (use -KillByPort to enable)" -ForegroundColor Gray +} + +Write-Host "" +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Stop complete" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" +Write-Host "To stop infrastructure:" -ForegroundColor Yellow +Write-Host " docker compose -f infra/docker-compose.yml --profile p6 --profile observability down" -ForegroundColor White +Write-Host "" diff --git a/scripts/test-cdc.ps1 b/scripts/test-cdc.ps1 new file mode 100644 index 0000000..d2ce18b --- /dev/null +++ b/scripts/test-cdc.ps1 @@ -0,0 +1,167 @@ +<# +.SYNOPSIS + Edu CDC pipeline end-to-end test script +.DESCRIPTION + Validates the full CDC pipeline: MySQL binlog -> Debezium -> Kafka -> data-ana -> ClickHouse + Steps: + 1. Insert test grade into MySQL + 2. Wait for Debezium capture + data-ana consume + 3. Query ClickHouse to verify data synced + 4. Call data-ana API to verify query works +.PARAMETER StudentId + Custom test student_id (default: cdc-test-) +.EXAMPLE + .\scripts\test-cdc.ps1 + .\scripts\test-cdc.ps1 -StudentId "my-test-001" +#> +param( + [string]$StudentId = "cdc-test-$(Get-Date -Format 'yyyyMMddHHmmss')" +) + +$ErrorActionPreference = "Continue" + +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " Edu CDC Pipeline E2E Test" -ForegroundColor Cyan +Write-Host "========================================" -ForegroundColor Cyan +Write-Host "" + +$examId = "exam-$StudentId" +$gradeId = "grade-$StudentId" +$testScore = 92.5 +$classId = "cls-cdc-test" + +Write-Host "Test parameters:" -ForegroundColor Yellow +Write-Host " StudentId: $StudentId" +Write-Host " ExamId: $examId" +Write-Host " GradeId: $gradeId" +Write-Host " Score: $testScore" +Write-Host " ClassId: $classId" +Write-Host "" + +# ===== 1. Pre-check ===== +Write-Host "[1/5] Pre-check..." -ForegroundColor Yellow + +# Check Debezium connector +$connectorStatus = $null +try { + $connectorStatus = Invoke-RestMethod -Uri "http://localhost:8083/connectors/edu-mysql-source/status" -Method Get -TimeoutSec 3 -ErrorAction Stop +} catch { + Write-Host " [FAIL] Debezium Connect not reachable. Start infrastructure first." -ForegroundColor Red + exit 1 +} +if ($connectorStatus.connector.state -ne "RUNNING") { + Write-Host " [FAIL] Debezium connector state: $($connectorStatus.connector.state)" -ForegroundColor Red + exit 1 +} +Write-Host " [OK] Debezium connector: RUNNING" -ForegroundColor Green + +# Check data-ana service +try { + $null = Invoke-RestMethod -Uri "http://localhost:3006/healthz" -Method Get -TimeoutSec 3 -ErrorAction Stop + Write-Host " [OK] data-ana service: running" -ForegroundColor Green +} catch { + Write-Host " [FAIL] data-ana service not reachable. Start application services first." -ForegroundColor Red + exit 1 +} + +# Check CDC consumer status +try { + $readyz = Invoke-RestMethod -Uri "http://localhost:3006/readyz" -Method Get -TimeoutSec 3 -ErrorAction Stop + $cdcStatus = $readyz.services.cdc_consumer + if ($cdcStatus -ne "running") { + Write-Host " [WARN] CDC consumer status: $cdcStatus (KAFKA_BROKERS may not be set)" -ForegroundColor Yellow + } else { + Write-Host " [OK] CDC consumer: running" -ForegroundColor Green + } +} catch { + Write-Host " [WARN] Cannot get /readyz status" -ForegroundColor Yellow +} + +Write-Host "" + +# ===== 2. Insert test data into MySQL ===== +Write-Host "[2/5] Inserting test data into MySQL..." -ForegroundColor Yellow + +$sqlInsert = @" +INSERT INTO core_edu_exams (id, class_id, subject_id, title, exam_date, total_score, created_at, updated_at) +VALUES ('$examId', '$classId', 'sub-math', 'CDC Test Exam', NOW(), 100, NOW(), NOW()) +ON DUPLICATE KEY UPDATE updated_at=NOW(); +INSERT INTO core_edu_grades (id, exam_id, student_id, score, rank_in_class, created_at, updated_at) +VALUES ('$gradeId', '$examId', '$StudentId', $testScore, 1, NOW(), NOW()) +ON DUPLICATE KEY UPDATE score=$testScore, updated_at=NOW(); +"@ + +docker exec edu-mysql mysql -uedu -pchangeme next_edu_cloud -e $sqlInsert 2>&1 | Out-Null +if ($LASTEXITCODE -eq 0) { + Write-Host " [OK] Inserted: exam=$examId / grade=$gradeId / student=$StudentId / score=$testScore" -ForegroundColor Green +} else { + Write-Host " [FAIL] MySQL insert failed" -ForegroundColor Red + exit 1 +} + +Write-Host "" + +# ===== 3. Wait for CDC propagation ===== +Write-Host "[3/5] Waiting for CDC propagation (5s)..." -ForegroundColor Yellow +Start-Sleep -Seconds 5 +Write-Host " [OK] Wait complete" -ForegroundColor Green +Write-Host "" + +# ===== 4. Verify ClickHouse data ===== +Write-Host "[4/5] Verifying ClickHouse data..." -ForegroundColor Yellow + +$chQuery = "SELECT student_id, class_id, exam_id, score, last_updated FROM edu_analytics.student_dashboard_view WHERE student_id = '$StudentId' ORDER BY last_updated DESC LIMIT 5" +$chResult = docker exec edu-clickhouse clickhouse-client --user default --password clickhouse -q $chQuery 2>$null + +if ($chResult) { + Write-Host " [OK] ClickHouse returned data:" -ForegroundColor Green + Write-Host " $chResult" -ForegroundColor White + + if ($chResult -match $StudentId -and $chResult -match "$testScore") { + Write-Host "" + Write-Host " [OK] Verified: student_id match + score=$testScore match" -ForegroundColor Green + if ($chResult -match $classId) { + Write-Host " [OK] class_id filled via exam cache: $classId" -ForegroundColor Green + } else { + Write-Host " [WARN] class_id not filled (exam cache may have missed, check event order)" -ForegroundColor Yellow + } + } else { + Write-Host " [FAIL] Data mismatch: expected student=$StudentId, score=$testScore" -ForegroundColor Red + } +} else { + Write-Host " [FAIL] ClickHouse has no data for student_id=$StudentId" -ForegroundColor Red + Write-Host " Possible causes:" -ForegroundColor Yellow + Write-Host " 1. Debezium did not capture MySQL change (check connector status)" -ForegroundColor White + Write-Host " 2. data-ana consumer not running (check /readyz cdc_consumer)" -ForegroundColor White + Write-Host " 3. Kafka topic name mismatch (check debezium-register.json)" -ForegroundColor White + exit 1 +} + +Write-Host "" + +# ===== 5. Verify data-ana API ===== +Write-Host "[5/5] Verifying data-ana query API..." -ForegroundColor Yellow + +$h = @{Authorization="Bearer dev-token"} + +try { + $weakness = Invoke-RestMethod -Uri "http://localhost:3006/analytics/student/$StudentId/weakness" -Method Get -Headers $h -TimeoutSec 5 -ErrorAction Stop + Write-Host " [OK] /analytics/student/$StudentId/weakness" -ForegroundColor Green + Write-Host " Response: $($weakness | ConvertTo-Json -Depth 3)" -ForegroundColor Gray +} catch { + Write-Host " [WARN] /analytics/student/$StudentId/weakness failed: $($_.Exception.Message)" -ForegroundColor Yellow +} + +try { + $perf = Invoke-RestMethod -Uri "http://localhost:3006/analytics/class/$classId/performance" -Method Get -Headers $h -TimeoutSec 5 -ErrorAction Stop + Write-Host " [OK] /analytics/class/$classId/performance" -ForegroundColor Green + Write-Host " Response: $($perf | ConvertTo-Json -Depth 3)" -ForegroundColor Gray +} catch { + Write-Host " [WARN] /analytics/class/$classId/performance failed: $($_.Exception.Message)" -ForegroundColor Yellow +} + +Write-Host "" +Write-Host "========================================" -ForegroundColor Cyan +Write-Host " CDC Pipeline Test Complete" -ForegroundColor Cyan +Write-Host " MySQL -> Debezium -> Kafka -> data-ana -> ClickHouse [OK]" -ForegroundColor Green +Write-Host "========================================" -ForegroundColor Cyan