Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4b0d6ee58a | ||
|
|
45497d2112 | ||
|
|
66f0844798 | ||
|
|
317a5e338a | ||
|
|
7a5c695b89 | ||
|
|
6bdb1f8ae6 | ||
|
|
80616cf459 | ||
|
|
a24017eb69 | ||
|
|
3056a95ef4 |
@@ -40,7 +40,6 @@ LLM_TIMEOUT=60
|
|||||||
|
|
||||||
# ---- 数据源 ----
|
# ---- 数据源 ----
|
||||||
BZZOIRO_KEY=
|
BZZOIRO_KEY=
|
||||||
API_FOOTBALL_KEY=
|
|
||||||
|
|
||||||
# ---- CORS ----
|
# ---- CORS ----
|
||||||
CORS_ORIGINS=http://localhost:5173,http://localhost:3000
|
CORS_ORIGINS=http://localhost:5173,http://localhost:3000
|
||||||
|
|||||||
@@ -12,28 +12,30 @@
|
|||||||
│ REST API
|
│ REST API
|
||||||
┌──────────────────────▼──────────────────────────────┐
|
┌──────────────────────▼──────────────────────────────┐
|
||||||
│ FastAPI │
|
│ FastAPI │
|
||||||
│ ├── /api/v1/matches 比赛查询 │
|
│ ├── /api/v1/matches 比赛查询(公开只读) │
|
||||||
│ ├── /api/v1/predict LLM 预测 (单/多 Agent) │
|
│ ├── /api/v1/predict LLM 预测 (单/多 Agent) │
|
||||||
│ ├── /api/v1/ingest/* 数据采集 │
|
│ ├── /api/v1/ingest/* 数据采集(需管理员) │
|
||||||
│ ├── /api/v1/eval/* 评估回填 │
|
│ ├── /api/v1/eval/* 评估回填(需管理员) │
|
||||||
│ └── /api/v1/backtest 回测 │
|
│ └── /api/v1/backtest 回测(需管理员) │
|
||||||
└──────────┬─────────────────────────────┬────────────┘
|
└──────────┬─────────────────────────────┬────────────┘
|
||||||
│ │
|
│ │
|
||||||
┌──────────▼──────────┐ ┌─────────────▼────────────┐
|
┌──────────▼──────────┐ ┌─────────────▼────────────┐
|
||||||
│ PostgreSQL │ │ LLM (OpenAI-compatible) │
|
│ PostgreSQL │ │ LLM (OpenAI-compatible) │
|
||||||
│ 6 张表 │ │ OpenAI / Deepseek / │
|
│ 12 张表 │ │ OpenAI / Deepseek / │
|
||||||
│ leagues/teams/ │ │ Ollama / 任意网关 │
|
│ leagues/teams/ │ │ Ollama / 任意网关 │
|
||||||
│ matches/match_ │ └──────────────────────────┘
|
│ matches/match_ │ └──────────────────────────┘
|
||||||
│ stats/predictions/ │
|
│ stats/standings/ │
|
||||||
│ injuries │
|
│ predictions/ │
|
||||||
|
│ app_settings/ │
|
||||||
|
│ schedules + │
|
||||||
|
│ raw_events 等 4 张 │
|
||||||
|
│ 数据治理表 │
|
||||||
└─────────────────────┘
|
└─────────────────────┘
|
||||||
▲
|
▲
|
||||||
│ 采集
|
│ 采集
|
||||||
┌──────────┴─────────────────────────────────────────┐
|
┌──────────┴─────────────────────────────────────────┐
|
||||||
│ 数据源 (DataSource 协议 + 注册表) │
|
│ 数据源 (DataSource 协议 + 注册表) │
|
||||||
│ ├── bzzoiro 比分 / 统计 / xG │
|
│ └── bzzoiro 比分 / 赛程 / 统计 / 积分榜 │
|
||||||
│ ├── understat xG 回填 │
|
|
||||||
│ └── injuries 伤停数据 (api-football) │
|
|
||||||
└────────────────────────────────────────────────────┘
|
└────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -55,7 +57,7 @@ API Route → Application Service → Repository → UnitOfWork → DB
|
|||||||
比赛数据 → 切片 ─┬─→ A 近期状态专家 ─┐
|
比赛数据 → 切片 ─┬─→ A 近期状态专家 ─┐
|
||||||
├─→ B 攻防数据专家 ─┤
|
├─→ B 攻防数据专家 ─┤
|
||||||
├─→ C 主客因素专家 ─┼─→ 终裁专家 ─→ 最终预测
|
├─→ C 主客因素专家 ─┼─→ 终裁专家 ─→ 最终预测
|
||||||
├─→ D 阵容完整专家 ─┤
|
├─→ D 联赛排名专家 ─┤
|
||||||
└─→ E 历史交锋专家 ─┘
|
└─→ E 历史交锋专家 ─┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -66,8 +68,7 @@ API Route → Application Service → Repository → UnitOfWork → DB
|
|||||||
|
|
||||||
### 数据正确性保障
|
### 数据正确性保障
|
||||||
|
|
||||||
- **Cutoff 机制**: 回测时只使用 `cutoff_at` 之前已采集的数据
|
- **Cutoff 机制**: 回测时只使用 `cutoff_at` 之前已采集的数据(近况/交锋/统计/积分榜切片统一生效)
|
||||||
- **Injury 防泄漏**: 伤停查询强制 `retrieved_at <= cutoff`
|
|
||||||
- **LLM 输出校验**: Pydantic 严格校验 + 语义一致性检查
|
- **LLM 输出校验**: Pydantic 严格校验 + 语义一致性检查
|
||||||
- **数据库约束**: CHECK 约束作为最后一道防线
|
- **数据库约束**: CHECK 约束作为最后一道防线
|
||||||
|
|
||||||
@@ -130,6 +131,21 @@ cd frontend && npm install && npm run dev
|
|||||||
|
|
||||||
后端运行在 `http://localhost:8000`,前端在 `http://localhost:5173`。
|
后端运行在 `http://localhost:8000`,前端在 `http://localhost:5173`。
|
||||||
|
|
||||||
|
## 生产上线检查清单
|
||||||
|
|
||||||
|
公网部署前逐项确认(第 1–4 项由启动校验强制,不满足拒绝启动;详见 [docs/06-deployment.md](docs/06-deployment.md#生产上线检查清单)):
|
||||||
|
|
||||||
|
- [ ] `APP_ENV=production`(安全校验 / Cookie `Secure` / 管理端点 fail-closed 的总开关)
|
||||||
|
- [ ] `SECRET_KEY` 强随机:`openssl rand -base64 32`,禁止弱值
|
||||||
|
- [ ] `ADMIN_PASSWORD` 或 `ADMIN_API_KEY` 至少配置其一
|
||||||
|
- [ ] 数据库强密码,禁止 `football:football` 等示例弱密码
|
||||||
|
- [ ] HTTPS(反代终结 TLS;production 下会话 Cookie 自动 `Secure`)
|
||||||
|
- [ ] 反代后设 `TRUST_PROXY_HEADERS=True`,仅可信反代可达 API,并配置 `X-Forwarded-For` / `X-Real-IP`
|
||||||
|
- [ ] 限流前置到 Nginx `limit_req`;应用内限流与 KeyRing 仅单进程有效,多 worker 会放大配额
|
||||||
|
- [ ] uvicorn 单 worker(默认);需扩容先网关统一限流再起多实例
|
||||||
|
- [ ] 启动后验证 `/health` 与 `/health/ready` 均 200
|
||||||
|
- [ ] 数据库迁移已内置:compose/Dockerfile 启动即执行 `alembic upgrade head`
|
||||||
|
|
||||||
## 安全与限流
|
## 安全与限流
|
||||||
|
|
||||||
- `/api/v1/predict`: 内存滑动窗口限流(10 次/分钟/IP),多 worker 时每进程独立计数
|
- `/api/v1/predict`: 内存滑动窗口限流(10 次/分钟/IP),多 worker 时每进程独立计数
|
||||||
@@ -141,20 +157,29 @@ cd frontend && npm install && npm run dev
|
|||||||
|
|
||||||
## API 概览
|
## API 概览
|
||||||
|
|
||||||
|
**公开只读**(无需登录;`predict` 带内存限流):
|
||||||
|
|
||||||
| 方法 | 路径 | 说明 |
|
| 方法 | 路径 | 说明 |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| GET | `/api/v1/matches` | 比赛查询(筛选/分页) |
|
| GET | `/api/v1/leagues` | 联赛列表(仅 id/code/name/country) |
|
||||||
| GET | `/api/v1/leagues` | 联赛列表 |
|
| GET | `/api/v1/matches` | 比赛查询(筛选/游标分页) |
|
||||||
| POST | `/api/v1/predict` | LLM 预测 (`mode=single`/`multi`) |
|
| GET | `/api/v1/matches/{id}` | 比赛详情(含统计与最近预测) |
|
||||||
| GET | `/api/v1/predictions` | 预测历史 |
|
| GET | `/api/v1/matches/{id}/context` | 比赛上下文(双方近况 + 历史交锋) |
|
||||||
| POST | `/api/v1/ingest/bzzoiro` | 采集比分/统计 |
|
| GET | `/api/v1/standings` | 联赛积分榜 |
|
||||||
| POST | `/api/v1/ingest/understat` | 回填 xG |
|
| POST | `/api/v1/predict` | LLM 预测 (`mode=single`/`multi`/`baseline`) |
|
||||||
| POST | `/api/v1/ingest/injuries` | 采集伤停 |
|
| GET | `/health`、`/health/ready` | 存活 / 就绪检查(含 DB) |
|
||||||
|
|
||||||
|
**需管理员**(Cookie 会话或 `X-API-Key`):
|
||||||
|
|
||||||
|
| 方法 | 路径 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| POST | `/api/v1/ingest/bzzoiro` | 采集赛果/赛程/统计/积分榜 |
|
||||||
|
| GET | `/api/v1/predictions` | 预测历史(列表) |
|
||||||
|
| GET | `/api/v1/predictions/{id}` | 单条预测详情 |
|
||||||
| POST | `/api/v1/eval/settle` | 回填实际结果 |
|
| POST | `/api/v1/eval/settle` | 回填实际结果 |
|
||||||
| GET | `/api/v1/eval/summary` | 准确率汇总 |
|
| GET | `/api/v1/eval/summary` | 准确率汇总 |
|
||||||
| POST | `/api/v1/backtest` | 历史回测 |
|
| POST | `/api/v1/backtest` | 历史回测 |
|
||||||
| GET | `/health` | 存活检查 |
|
| `/api/v1/admin/**` | 配置/采集状态/日志/定时任务/死信等 | 管理后台(router 级鉴权) |
|
||||||
| GET | `/health/ready` | 就绪检查(含 DB) |
|
|
||||||
|
|
||||||
## 项目结构
|
## 项目结构
|
||||||
|
|
||||||
@@ -163,36 +188,46 @@ Profeto/
|
|||||||
├── src/
|
├── src/
|
||||||
│ ├── api/ # FastAPI 路由层
|
│ ├── api/ # FastAPI 路由层
|
||||||
│ │ ├── app.py # 应用工厂 + lifespan
|
│ │ ├── app.py # 应用工厂 + lifespan
|
||||||
|
│ │ ├── deps.py # 依赖注入:鉴权 / 限流
|
||||||
│ │ ├── schemas.py # Pydantic 请求/响应模型
|
│ │ ├── schemas.py # Pydantic 请求/响应模型
|
||||||
│ │ └── routes/
|
│ │ └── routes/
|
||||||
│ │ ├── matches.py # 比赛查询
|
│ │ ├── matches.py # 比赛查询(公开只读)
|
||||||
│ │ ├── predict.py # 预测入口
|
│ │ ├── predict.py # 预测入口 + 预测历史
|
||||||
│ │ ├── ingest.py # 数据采集
|
│ │ ├── ingest.py # 数据采集(需管理员)
|
||||||
│ │ ├── eval.py # 评估回填
|
│ │ ├── eval.py # 评估回填(需管理员)
|
||||||
│ │ └── backtest.py # 回测
|
│ │ ├── backtest.py # 回测(需管理员)
|
||||||
|
│ │ ├── auth.py # 登录/登出/改密
|
||||||
|
│ │ ├── admin_settings.py # /admin/** 配置/日志/数据质量(router 级鉴权)
|
||||||
|
│ │ └── schedules.py # 定时任务 + 死信重试(router 级鉴权)
|
||||||
│ ├── core/ # 基础设施
|
│ ├── core/ # 基础设施
|
||||||
│ │ ├── config.py # pydantic-settings 配置
|
│ │ ├── config.py # pydantic-settings 配置
|
||||||
|
│ │ ├── crypto.py # 加密/哈希
|
||||||
│ │ ├── http_client.py # 共享 httpx 客户端
|
│ │ ├── http_client.py # 共享 httpx 客户端
|
||||||
│ │ └── retry.py # 重试工具(指数退避)
|
│ │ ├── log_buffer.py # 内存日志缓冲(admin 日志页)
|
||||||
|
│ │ ├── runtime_config.py # DB 配置覆盖(.env → app_settings)
|
||||||
|
│ │ ├── scheduler.py # 进程内 cron 调度器
|
||||||
|
│ │ └── security_check.py # 启动安全校验
|
||||||
│ ├── data/ # 数据层
|
│ ├── data/ # 数据层
|
||||||
│ │ ├── sources.py # DataSource 协议 + 注册表
|
│ │ ├── sources.py # DataSource 协议 + 注册表
|
||||||
|
│ │ ├── bzzoiro.py # bzzoiro 数据源(events/standings/stats)
|
||||||
│ │ ├── normalize.py # 数据规范化契约
|
│ │ ├── normalize.py # 数据规范化契约
|
||||||
│ │ ├── bzzoiro.py # bzzoiro 数据源
|
|
||||||
│ │ ├── understat.py # understat xG 数据源
|
|
||||||
│ │ ├── injuries.py # 伤停数据
|
|
||||||
│ │ ├── config.py # 联赛映射常量
|
│ │ ├── config.py # 联赛映射常量
|
||||||
│ │ └── team_names.py # 队名归一化
|
│ │ ├── key_ring.py # API Key 轮换环(429 冷却)
|
||||||
|
│ │ ├── team_names.py # 队名归一化
|
||||||
|
│ │ └── team_names_zh.py # 队名中文映射
|
||||||
│ ├── db/ # 数据库
|
│ ├── db/ # 数据库
|
||||||
│ │ ├── base.py # SQLAlchemy async engine
|
│ │ ├── base.py # SQLAlchemy async engine
|
||||||
│ │ ├── models.py # ORM 模型 (6 表)
|
│ │ ├── models.py # ORM 模型 (12 表)
|
||||||
│ │ ├── unit_of_work.py # UnitOfWork 事务封装
|
│ │ ├── unit_of_work.py # UnitOfWork 事务封装
|
||||||
│ │ └── repositories.py # Repository 数据访问
|
│ │ └── repositories.py # Repository 数据访问
|
||||||
│ └── llm/ # LLM 预测核心
|
│ └── llm/ # LLM 预测核心
|
||||||
│ ├── predict.py # 预测服务 (缓存 + 单/多模式)
|
│ ├── predict.py # 预测服务 (缓存 + 单/多/基线模式)
|
||||||
│ ├── context_builder.py # 数据切片 + 上下文拼接
|
│ ├── context_builder.py # 数据切片 + 上下文拼接
|
||||||
|
│ ├── baseline.py # 基线预测(均值模型)
|
||||||
│ ├── eval.py # 评估统计
|
│ ├── eval.py # 评估统计
|
||||||
│ ├── backtest.py # 回测框架
|
│ ├── backtest.py # 回测框架
|
||||||
│ ├── provider.py # 多提供商 LLM 抽象
|
│ ├── provider.py # 多提供商 LLM 抽象
|
||||||
|
│ ├── utils.py # LLM 工具函数
|
||||||
│ ├── validation.py # LLM 输出校验
|
│ ├── validation.py # LLM 输出校验
|
||||||
│ ├── agents/
|
│ ├── agents/
|
||||||
│ │ ├── base.py # Agent 基础设施 + 解析
|
│ │ ├── base.py # Agent 基础设施 + 解析
|
||||||
@@ -200,6 +235,11 @@ Profeto/
|
|||||||
│ └── prompts/ # Prompt 模板
|
│ └── prompts/ # Prompt 模板
|
||||||
├── alembic/ # 数据库迁移
|
├── alembic/ # 数据库迁移
|
||||||
├── frontend/ # React 前端
|
├── frontend/ # React 前端
|
||||||
|
│ └── src/
|
||||||
|
│ ├── pages/ # 公开站(赛程 Matches + 积分榜 Standings)
|
||||||
|
│ ├── admin/ # 管理后台(布局/页面/数据访问层 dal.ts)
|
||||||
|
│ ├── components/ # 共享组件
|
||||||
|
│ └── lib/http.ts # 唯一 HTTP 实现(带凭据/超时/错误处理)
|
||||||
├── docs/ # 详细文档
|
├── docs/ # 详细文档
|
||||||
├── tests/ # 单元测试
|
├── tests/ # 单元测试
|
||||||
├── docker-compose.yml
|
├── docker-compose.yml
|
||||||
@@ -234,7 +274,6 @@ Profeto/
|
|||||||
| `LLM_SPECIALIST_MODEL` | 专家模型 (空=回落 LLM_MODEL) | |
|
| `LLM_SPECIALIST_MODEL` | 专家模型 (空=回落 LLM_MODEL) | |
|
||||||
| `LLM_AGGREGATOR_MODEL` | 终裁模型 (空=回落 LLM_MODEL) | |
|
| `LLM_AGGREGATOR_MODEL` | 终裁模型 (空=回落 LLM_MODEL) | |
|
||||||
| `BZZOIRO_KEY` | bzzoiro API Key | *(必填)* |
|
| `BZZOIRO_KEY` | bzzoiro API Key | *(必填)* |
|
||||||
| `API_FOOTBALL_KEY` | api-football Key (伤停) | |
|
|
||||||
| `CORS_ORIGINS` | 允许的跨域来源 | `http://localhost:5173` |
|
| `CORS_ORIGINS` | 允许的跨域来源 | `http://localhost:5173` |
|
||||||
|
|
||||||
## 测试
|
## 测试
|
||||||
|
|||||||
+36
-28
@@ -13,16 +13,15 @@
|
|||||||
│ │
|
│ │
|
||||||
│ 数据查询 预测编排 采集(手动/cron 触发) │
|
│ 数据查询 预测编排 采集(手动/cron 触发) │
|
||||||
│ ┌──────┐ ┌────────────┐ ┌───────────────────┐ │
|
│ ┌──────┐ ┌────────────┐ ┌───────────────────┐ │
|
||||||
│ │matches│ │ orchestrator│ │ bzzoiro (赛果) │ │
|
│ │matches│ │ orchestrator│ │ bzzoiro (唯一源) │ │
|
||||||
│ │leagues│ │ ┌─ 5 专家并行(便宜模型) │ │
|
│ │leagues│ │ ┌─ 5 专家并行(便宜模型) │ │
|
||||||
│ └──┬───┘ │ │ h2h / form / stats / │ │
|
│ └──┬───┘ │ │ h2h / form / stats / │ │
|
||||||
│ │ │ │ home_away / injuries │ │
|
│ │ │ │ home_away / standings │ │
|
||||||
│ │ │ └─ aggregator 终裁(强模型) │ │
|
│ │ │ └─ aggregator 终裁(强模型) │ │
|
||||||
│ │ └────────────┘ └───────────────────┘ │
|
│ │ └────────────┘ │ events / standings │ │
|
||||||
│ │ │ └ understat (xG) │
|
│ ┌──┴──────────────┴──┐ │ /stats 三条管线 │ │
|
||||||
│ ┌──┴──────────────┴──┐ └ injuries (伤停) │
|
│ │ PostgreSQL (12 张表)│ └───────────────────┘ │
|
||||||
│ │ PostgreSQL (6 张表) │ httpx → 外部 API │
|
│ └────────────────────┘ httpx → 外部 API │
|
||||||
│ └────────────────────┘ │
|
|
||||||
└─────────────────────────────────────────────────────┘
|
└─────────────────────────────────────────────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -31,7 +30,7 @@
|
|||||||
1. `POST /predict {match_id}` → orchestrator
|
1. `POST /predict {match_id}` → orchestrator
|
||||||
2. `load_match_header`: 查比赛 + 双方 + 联赛(一次 eager load)
|
2. `load_match_header`: 查比赛 + 双方 + 联赛(一次 eager load)
|
||||||
3. **5 个专家 agent 并行**(`asyncio.gather`),每个:
|
3. **5 个专家 agent 并行**(`asyncio.gather`),每个:
|
||||||
- 各自的数据切片函数查库(近况/交锋/积分榜 SQL 聚合/伤停/xG)
|
- 各自的数据切片函数查库(近况/交锋/积分榜聚合/射门控球/xG)
|
||||||
- 切片无数据 → **跳过 LLM**,直接 `no_data` stub(省 token、防幻觉)
|
- 切片无数据 → **跳过 LLM**,直接 `no_data` stub(省 token、防幻觉)
|
||||||
- 有数据 → 专属 prompt(专家模型,便宜快)→ 结构化 JSON 报告(`home_edge` 方向性评分 + 证据)
|
- 有数据 → 专属 prompt(专家模型,便宜快)→ 结构化 JSON 报告(`home_edge` 方向性评分 + 证据)
|
||||||
4. **终裁 agent**:5 份报告 + 比赛信息 → 权衡采信度(`agent_weights`)→ 最终预测 JSON
|
4. **终裁 agent**:5 份报告 + 比赛信息 → 权衡采信度(`agent_weights`)→ 最终预测 JSON
|
||||||
@@ -44,7 +43,7 @@
|
|||||||
|---|---|
|
|---|---|
|
||||||
| **多专家并行而非单次大 prompt** | 每维度独立迭代 prompt;报告可归因(哪个维度分析错了);总延迟 ≈ 2 次串行调用 |
|
| **多专家并行而非单次大 prompt** | 每维度独立迭代 prompt;报告可归因(哪个维度分析错了);总延迟 ≈ 2 次串行调用 |
|
||||||
| **专家/终裁模型分档** | 专家用便宜模型快速分析,终裁用强模型汇总决策,成本与质量平衡(`LLM_SPECIALIST_MODEL` / `LLM_AGGREGATOR_MODEL`) |
|
| **专家/终裁模型分档** | 专家用便宜模型快速分析,终裁用强模型汇总决策,成本与质量平衡(`LLM_SPECIALIST_MODEL` / `LLM_AGGREGATOR_MODEL`) |
|
||||||
| **no_data 门控** | 无数据维度(如伤停未接入)不调 LLM,终裁知道维度缺失,不编造 |
|
| **no_data 门控** | 无数据维度(如积分榜未采集)不调 LLM,终裁知道维度缺失,不编造 |
|
||||||
| **fail-open** | 单个专家失败只标记 `status=error`,其余照常;研究场景可用性优先 |
|
| **fail-open** | 单个专家失败只标记 `status=error`,其余照常;研究场景可用性优先 |
|
||||||
| **`match_date_date` 天级去重** | 不同源时间精度不同,秒级匹配会产生重复行;天级 + 数据库唯一约束 |
|
| **`match_date_date` 天级去重** | 不同源时间精度不同,秒级匹配会产生重复行;天级 + 数据库唯一约束 |
|
||||||
| **积分榜 SQL 聚合 + season 过滤** | `UNION ALL` 主客双视角 + `GROUP BY` 在库内算,只算当前赛季(修复过跨赛季 bug) |
|
| **积分榜 SQL 聚合 + season 过滤** | `UNION ALL` 主客双视角 + `GROUP BY` 在库内算,只算当前赛季(修复过跨赛季 bug) |
|
||||||
@@ -57,33 +56,38 @@
|
|||||||
Profeto/
|
Profeto/
|
||||||
├── src/
|
├── src/
|
||||||
│ ├── api/
|
│ ├── api/
|
||||||
│ │ ├── app.py # FastAPI 工厂(lifespan 仅验证 DB 连接,不建表)
|
│ │ ├── app.py # FastAPI 工厂(lifespan:迁移校验/定时任务/生产限流提醒)
|
||||||
│ │ ├── deps.py # 依赖:管理接口鉴权(X-API-Key)
|
│ │ ├── deps.py # 依赖:管理接口鉴权(Cookie/X-API-Key)+ 限流
|
||||||
│ │ ├── schemas.py # Pydantic v2 请求/响应
|
│ │ ├── schemas.py # Pydantic v2 请求/响应
|
||||||
│ │ └── routes/
|
│ │ └── routes/
|
||||||
│ │ ├── matches.py # 联赛/比赛查询(游标分页)
|
│ │ ├── matches.py # 联赛/比赛/上下文/积分榜(公开只读)
|
||||||
│ │ ├── predict.py # 预测 + 预测历史
|
│ │ ├── predict.py # 预测(限流)+ 预测历史(需鉴权)
|
||||||
│ │ ├── ingest.py # 采集触发(自管 session,需鉴权)
|
│ │ ├── ingest.py # 采集触发(需鉴权)
|
||||||
│ │ ├── eval.py # 赛后回填 + 准确率汇总
|
│ │ ├── eval.py # 赛后回填 + 准确率汇总(需鉴权)
|
||||||
│ │ └── backtest.py # 历史回测(需鉴权)
|
│ │ ├── backtest.py # 历史回测(需鉴权)
|
||||||
|
│ │ ├── auth.py # 登录/登出/改密
|
||||||
|
│ │ ├── admin_settings.py # /admin/** 配置/日志/数据质量(router 级鉴权)
|
||||||
|
│ │ └── schedules.py # 定时任务 + 死信重试(router 级鉴权)
|
||||||
│ ├── db/
|
│ ├── db/
|
||||||
│ │ ├── base.py # async engine + get_db/get_db_read
|
│ │ ├── base.py # async engine + get_db/get_db_read
|
||||||
│ │ ├── models.py # 6 张表 ORM
|
│ │ ├── models.py # 12 张表 ORM
|
||||||
│ │ ├── repositories.py # 仓储层
|
│ │ ├── repositories.py # 仓储层
|
||||||
│ │ └── unit_of_work.py # 事务边界
|
│ │ └── unit_of_work.py # 事务边界
|
||||||
│ ├── data/
|
│ ├── data/
|
||||||
│ │ ├── bzzoiro.py # 赛果采集 + 幂等入库
|
│ │ ├── bzzoiro.py # 唯一数据源:events/standings/stats 三管线 + Bronze 层
|
||||||
│ │ ├── understat.py # xG 回填
|
|
||||||
│ │ ├── injuries.py # 伤停采集(带文件缓存)
|
|
||||||
│ │ ├── normalize.py # NormalizedMatch 清洗契约
|
│ │ ├── normalize.py # NormalizedMatch 清洗契约
|
||||||
│ │ ├── team_names.py # 队名归一映射
|
│ │ ├── team_names.py # 队名归一映射
|
||||||
|
│ │ ├── team_names_zh.py # 队名中文名映射
|
||||||
|
│ │ ├── key_ring.py # 多 key 轮换(429 冷却,进程内)
|
||||||
│ │ ├── sources.py # 数据源注册表
|
│ │ ├── sources.py # 数据源注册表
|
||||||
│ │ └── config.py # 联赛代码映射
|
│ │ └── config.py # 联赛代码映射
|
||||||
│ ├── llm/
|
│ ├── llm/
|
||||||
│ │ ├── provider.py # OpenAI-compatible 抽象(共享连接池/JSON 兜底解析)
|
│ │ ├── provider.py # OpenAI-compatible 抽象(共享连接池/JSON 兜底解析)
|
||||||
│ │ ├── context_builder.py # 数据切片(h2h/form/stats/home_away/injuries)+ 单 agent 拼接
|
│ │ ├── context_builder.py # 数据切片(h2h/form/stats/home_away/standings)+ 单 agent 拼接
|
||||||
│ │ ├── predict.py # 预测入口(mode 分派 + 缓存)
|
│ │ ├── predict.py # 预测入口(mode 分派 + 缓存)
|
||||||
|
│ │ ├── baseline.py # 基线预测(均值模型,mode=baseline)
|
||||||
│ │ ├── eval.py # 准确率统计
|
│ │ ├── eval.py # 准确率统计
|
||||||
|
│ │ ├── utils.py # LLM 工具函数
|
||||||
│ │ ├── validation.py # LLM 输出严格校验(Pydantic)
|
│ │ ├── validation.py # LLM 输出严格校验(Pydantic)
|
||||||
│ │ ├── backtest.py # 回测执行
|
│ │ ├── backtest.py # 回测执行
|
||||||
│ │ ├── agents/
|
│ │ ├── agents/
|
||||||
@@ -91,16 +95,20 @@ Profeto/
|
|||||||
│ │ │ └── orchestrator.py # 并行专家 → 终裁 → 存库
|
│ │ │ └── orchestrator.py # 并行专家 → 终裁 → 存库
|
||||||
│ │ └── prompts/
|
│ │ └── prompts/
|
||||||
│ │ ├── match_prediction_v1/v2.md # 单 agent 模板
|
│ │ ├── match_prediction_v1/v2.md # 单 agent 模板
|
||||||
│ │ └── agents/{h2h,form,stats,home_away,injuries,aggregator}_v1.md
|
│ │ └── agents/{form,stats,home_away,standings,h2h,aggregator}_v1.md
|
||||||
│ └── core/
|
│ └── core/
|
||||||
│ ├── config.py # pydantic-settings
|
│ ├── config.py # pydantic-settings
|
||||||
|
│ ├── crypto.py # 加密/哈希
|
||||||
│ ├── http_client.py # 共享 httpx 客户端
|
│ ├── http_client.py # 共享 httpx 客户端
|
||||||
│ └── retry.py # 重试工具
|
│ ├── log_buffer.py # 内存日志缓冲(admin 日志页)
|
||||||
├── alembic/versions/ # 0001~0006(0001 建表 → 0006 漂移清理)
|
│ ├── runtime_config.py # DB 配置覆盖(app_settings)
|
||||||
├── frontend/src/pages/Matches.tsx # 单页(预测面板 + 专家报告折叠区 + 游标分页)
|
│ ├── scheduler.py # 进程内 cron 调度器
|
||||||
├── tests/ # 核心 + agent 测试
|
│ └── security_check.py # 启动安全校验
|
||||||
|
├── alembic/versions/ # 0001~0018(建表 → Bronze 层 → 单一数据源 → 基线模式等)
|
||||||
|
├── frontend/src/ # pages/(公开站) + admin/(管理后台) + lib/http.ts(唯一 HTTP 实现)
|
||||||
|
├── tests/ # 核心 + agent 测试(250+ 项,自包含)
|
||||||
├── docker-compose.yml # api + postgres 两容器
|
├── docker-compose.yml # api + postgres 两容器
|
||||||
└── docs/ # 本文档
|
└── docs/ # 本文档
|
||||||
```
|
```
|
||||||
|
|
||||||
## 技术栈
|
## 技术栈
|
||||||
@@ -113,5 +121,5 @@ Profeto/
|
|||||||
| HTTP | httpx(共享连接池)/ urllib(bzzoiro 同步限速) |
|
| HTTP | httpx(共享连接池)/ urllib(bzzoiro 同步限速) |
|
||||||
| LLM | OpenAI-compatible 接口(openai/deepseek/ollama 等任一) |
|
| LLM | OpenAI-compatible 接口(openai/deepseek/ollama 等任一) |
|
||||||
| 前端 | Vite + React 18 + TypeScript + Tailwind |
|
| 前端 | Vite + React 18 + TypeScript + Tailwind |
|
||||||
| 测试 | pytest + pytest-asyncio(33 项,自包含) |
|
| 测试 | pytest + pytest-asyncio(250+ 项,自包含) |
|
||||||
| 部署 | Docker Compose(api + postgres) |
|
| 部署 | Docker Compose(api + postgres) |
|
||||||
|
|||||||
+13
-4
@@ -66,6 +66,9 @@ curl -X POST http://localhost:8000/api/v1/ingest/bzzoiro \
|
|||||||
-d '{"leagues":["E0"],"date_from":"2026-08-01","date_to":"2026-09-08"}'
|
-d '{"leagues":["E0"],"date_from":"2026-08-01","date_to":"2026-09-08"}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
> 注:采集/评估端点需管理员凭据。本地开发环境(未配置鉴权、非 production)默认放行;
|
||||||
|
> 生产环境需先 `POST /auth/login` 取 Cookie,或带 `X-API-Key` 头。
|
||||||
|
|
||||||
数据量大时**直接拉整赛季**(约 380 场,含近几个赛季更好,近况/交锋/积分榜都需要历史):
|
数据量大时**直接拉整赛季**(约 380 场,含近几个赛季更好,近况/交锋/积分榜都需要历史):
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -74,14 +77,20 @@ curl -X POST http://localhost:8000/api/v1/ingest/bzzoiro \
|
|||||||
-d '{"leagues":["E0"],"date_from":"2025-08-01","date_to":"2026-09-08"}'
|
-d '{"leagues":["E0"],"date_from":"2025-08-01","date_to":"2026-09-08"}'
|
||||||
```
|
```
|
||||||
|
|
||||||
### 回填 xG(可选,让攻防数据 agent 有数据)
|
### 回填积分榜与统计(让攻防/排名专家有数据)
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
curl -X POST http://localhost:8000/api/v1/ingest/understat \
|
curl -X POST http://localhost:8000/api/v1/ingest/bzzoiro \
|
||||||
-H "Content-Type: application/json" \
|
-H "Content-Type: application/json" \
|
||||||
-d '{"league":"E0","season":2025}'
|
-d '{"task":"standings"}'
|
||||||
|
|
||||||
|
curl -X POST http://localhost:8000/api/v1/ingest/bzzoiro \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-d '{"task":"stats","leagues":["E0"],"limit":300}'
|
||||||
```
|
```
|
||||||
|
|
||||||
|
`task=stats` 只补空字段(xG/射门/控球等),不创建比赛。
|
||||||
|
|
||||||
### 查比赛
|
### 查比赛
|
||||||
|
|
||||||
浏览器打开 http://localhost:5173 ,选"英超 / 未开赛";
|
浏览器打开 http://localhost:5173 ,选"英超 / 未开赛";
|
||||||
@@ -126,4 +135,4 @@ curl http://localhost:8000/api/v1/eval/summary
|
|||||||
| predict 返回 502 | 看 uvicorn 日志的 LLM error;确认 `LLM_BASE_URL`/`LLM_API_KEY`;`response_format` 不兼容的网关会报错(改用支持 json mode 的模型) |
|
| predict 返回 502 | 看 uvicorn 日志的 LLM error;确认 `LLM_BASE_URL`/`LLM_API_KEY`;`response_format` 不兼容的网关会报错(改用支持 json mode 的模型) |
|
||||||
| 采集 0 场 | bzzoiro Key 失效或联赛代码写错;先 `GET /api/v1/leagues` 看库里有没有联赛 |
|
| 采集 0 场 | bzzoiro Key 失效或联赛代码写错;先 `GET /api/v1/leagues` 看库里有没有联赛 |
|
||||||
| 专家报告全是 no_data | 历史数据不够 —— 近况需要每队近 5 场、积分榜需要本赛季已完赛比赛,多拉几周数据 |
|
| 专家报告全是 no_data | 历史数据不够 —— 近况需要每队近 5 场、积分榜需要本赛季已完赛比赛,多拉几周数据 |
|
||||||
| xg agent 报无 xG 数据 | 先跑 understat 回填;注意 understat 只有五大联赛 |
|
| stats 专家报无 xG/统计 | 先跑 `task=stats` 回填(bzzoiro 统计管线,只补空字段) |
|
||||||
|
|||||||
+80
-34
@@ -4,18 +4,30 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
|
|
||||||
所有数据端点返回 JSON。错误统一为 `{"detail": "<message>"}` + 对应 HTTP 状态码。
|
所有数据端点返回 JSON。错误统一为 `{"detail": "<message>"}` + 对应 HTTP 状态码。
|
||||||
|
|
||||||
|
## 鉴权模型
|
||||||
|
|
||||||
|
| 级别 | 端点 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| **公开只读** | `GET /leagues`、`GET /matches`、`GET /matches/{id}`、`GET /matches/{id}/context`、`GET /standings`、`GET /health*` | 无需任何凭据;公开站直接调用 |
|
||||||
|
| **公开 + 限流** | `POST /predict` | 内存滑动窗口限流(10 次/分钟/IP) |
|
||||||
|
| **需管理员** | `GET /predictions*`、`POST /ingest/bzzoiro`、`/eval/*`、`POST /backtest`、`/admin/**` | Cookie 会话(`POST /auth/login` 颁发)或 `X-API-Key` 头 |
|
||||||
|
|
||||||
|
管理端点在生产环境未配置鉴权时 fail-closed(503),不会静默放行。
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 数据查询
|
## 数据查询(公开只读)
|
||||||
|
|
||||||
### `GET /api/v1/leagues`
|
### `GET /api/v1/leagues`
|
||||||
|
|
||||||
列出已入库联赛。
|
列出已入库联赛(P1-3: 公开站联赛筛选动态加载来源)。
|
||||||
|
|
||||||
```json
|
```json
|
||||||
[{"id": 1, "code": "E0", "name": "Premier League", "country": "England"}]
|
[{"id": 1, "code": "E0", "name": "Premier League", "country": "England"}]
|
||||||
```
|
```
|
||||||
|
|
||||||
|
仅返回 `id/code/name/country` 四个展示字段,不含任何配置或密钥信息。
|
||||||
|
|
||||||
### `GET /api/v1/matches`
|
### `GET /api/v1/matches`
|
||||||
|
|
||||||
比赛列表,游标分页。
|
比赛列表,游标分页。
|
||||||
@@ -47,7 +59,44 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
|
|
||||||
### `GET /api/v1/matches/{id}`
|
### `GET /api/v1/matches/{id}`
|
||||||
|
|
||||||
单场比赛详情,字段同上。
|
单场比赛详情,字段同上,另含 `stats`(统计)与 `recent_predictions`(最近 5 条预测摘要)。
|
||||||
|
|
||||||
|
### `GET /api/v1/matches/{id}/context`
|
||||||
|
|
||||||
|
比赛上下文(公开只读,P1-2: 公开站详情页「近况/交锋」数据来源;不触发 LLM):
|
||||||
|
|
||||||
|
- `home_recent`: 主队最近 5 场已完赛
|
||||||
|
- `away_recent`: 客队最近 5 场已完赛
|
||||||
|
- `h2h`: 双方最近 5 次交手
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"home_recent": [
|
||||||
|
{"match_date": "2026-09-12T14:00:00+00:00", "home_team": "阿森纳",
|
||||||
|
"away_team": "切尔西", "home_goals": 2, "away_goals": 1}
|
||||||
|
],
|
||||||
|
"away_recent": [],
|
||||||
|
"h2h": []
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
数据不足时对应列表为空(前端展示空态)。比赛不存在返回 404。
|
||||||
|
|
||||||
|
### `GET /api/v1/standings`
|
||||||
|
|
||||||
|
联赛积分榜(公开只读)。参数:`league`(联赛代码,空 = 全部)、`season`(空 = 各联赛最新赛季)。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{"leagues": [
|
||||||
|
{"league_code": "E0", "league_name": "Premier League", "season": "2026-2027",
|
||||||
|
"retrieved_at": "2026-09-20T08:00:00+00:00",
|
||||||
|
"rows": [{"position": 1, "team": "阿森纳", "team_en": "Arsenal",
|
||||||
|
"played": 5, "won": 4, "drawn": 1, "lost": 0,
|
||||||
|
"goals_for": 11, "goals_against": 3, "goal_diff": 8,
|
||||||
|
"points": 13, "xg_for": 9.8, "xg_against": 3.9,
|
||||||
|
"form": "WWWDW", "zone": "UEFA Champions League"}]}
|
||||||
|
]}
|
||||||
|
```
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -96,13 +145,13 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
"exp_home_goals": null, "exp_away_goals": null, "probable_score": null,
|
"exp_home_goals": null, "exp_away_goals": null, "probable_score": null,
|
||||||
"model": "gpt-4o-mini", "latency_ms": 2100,
|
"model": "gpt-4o-mini", "latency_ms": 2100,
|
||||||
"prompt_tokens": 380, "completion_tokens": 120},
|
"prompt_tokens": 380, "completion_tokens": 120},
|
||||||
{"agent": "injuries", "status": "no_data", "data_sufficiency": "none",
|
{"agent": "standings", "status": "no_data", "data_sufficiency": "none",
|
||||||
"analysis": "该维度无数据,跳过分析。", "home_edge": null, "subjective_confidence": null,
|
"analysis": "该维度无数据,跳过分析。", "home_edge": null, "subjective_confidence": null,
|
||||||
"key_evidence": [], "exp_home_goals": null, "exp_away_goals": null,
|
"key_evidence": [], "exp_home_goals": null, "exp_away_goals": null,
|
||||||
"probable_score": null, "model": "", "latency_ms": null,
|
"probable_score": null, "model": "", "latency_ms": null,
|
||||||
"prompt_tokens": null, "completion_tokens": null}
|
"prompt_tokens": null, "completion_tokens": null}
|
||||||
],
|
],
|
||||||
"agent_weights": {"form": 0.9, "stats": 0.8, "home_away": 0.7, "injuries": 0.0, "h2h": 0.8},
|
"agent_weights": {"form": 0.9, "stats": 0.8, "home_away": 0.7, "standings": 0.8, "h2h": 0.8},
|
||||||
"context": "[5 份报告的 JSON 串]",
|
"context": "[5 份报告的 JSON 串]",
|
||||||
"latency_ms": 9800
|
"latency_ms": 9800
|
||||||
}
|
}
|
||||||
@@ -112,11 +161,11 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
|
|
||||||
错误:404 比赛不存在;502 LLM 调用失败(终裁失败时整体失败,专家失败不会)。
|
错误:404 比赛不存在;502 LLM 调用失败(终裁失败时整体失败,专家失败不会)。
|
||||||
|
|
||||||
### `GET /api/v1/predictions?match_id=&limit=`
|
### `GET /api/v1/predictions?match_id=&limit=`(需管理员)
|
||||||
|
|
||||||
预测历史(倒序),含 `settled` 与实际比分回填状态。
|
预测历史(倒序),含 `settled` 与实际比分回填状态。
|
||||||
|
|
||||||
### `GET /api/v1/predictions/{id}`
|
### `GET /api/v1/predictions/{id}`(需管理员)
|
||||||
|
|
||||||
单条预测详情(含完整 `agent_outputs`)。
|
单条预测详情(含完整 `agent_outputs`)。
|
||||||
|
|
||||||
@@ -124,40 +173,34 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
|
|
||||||
## 数据采集
|
## 数据采集
|
||||||
|
|
||||||
### `POST /api/v1/ingest/bzzoiro`
|
### `POST /api/v1/ingest/bzzoiro`(需管理员)
|
||||||
|
|
||||||
从 bzzoiro 采集赛果/赛程并入库(幂等,重复跑安全)。
|
从 bzzoiro(唯一数据源)采集数据并入库(幂等,重复跑安全)。任务在后台异步执行,请求立即返回。
|
||||||
|
|
||||||
```json
|
```json
|
||||||
{"leagues": ["E0", "SP1"], "date_from": "2025-08-01", "date_to": "2026-09-08", "status": "finished"}
|
{"task": "all", "leagues": ["E0", "SP1"], "date_from": "2025-08-01", "date_to": "2026-09-08", "status": "finished"}
|
||||||
```
|
```
|
||||||
|
|
||||||
- `status` 还可传 `scheduled` 拉未来赛程
|
| 字段 | 默认 | 说明 |
|
||||||
|
|---|---|---|
|
||||||
|
| `task` | `events` | 采集任务:`events`(比赛)/ `standings`(积分榜)/ `stats`(统计回填,含 xG)/ `all` |
|
||||||
|
| `leagues` | 全部已知联赛 | 联赛代码列表,如 `["E0", "SP1"]` |
|
||||||
|
| `date_from` / `date_to` | 空 | 日期范围(`YYYY-MM-DD`) |
|
||||||
|
| `status` | 空(两者都采) | `finished`(已完赛)/ `scheduled`(未来赛程) |
|
||||||
|
| `season` | 当前赛季 | standings 赛季,如 `"2026-2027"` |
|
||||||
|
| `limit` | 100 | stats 回填单次最大比赛数(1–500) |
|
||||||
|
|
||||||
- 响应含每联赛 `inserted`/`updated`/`errors` 统计
|
- 响应含每联赛 `inserted`/`updated`/`errors` 统计
|
||||||
|
- `task=stats` 只补空字段、不创建比赛(xG/射门/控球等统计回填)
|
||||||
|
|
||||||
### `POST /api/v1/ingest/understat`
|
> 历史版本曾有独立的 understat(xG)与 injuries(伤停)采集端点,
|
||||||
|
> 已随数据源收敛为 bzzoiro 唯一来源而移除。
|
||||||
回填 xG(只补空字段,不创建比赛):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{"league": "E0", "season": 2025}
|
|
||||||
```
|
|
||||||
|
|
||||||
`season=2025` 表示 2025-2026 赛季。仅支持五大联赛。
|
|
||||||
|
|
||||||
### `POST /api/v1/ingest/injuries`
|
|
||||||
|
|
||||||
采集伤停(需 `API_FOOTBALL_KEY`,当前只返回计数,尚未接入 context):
|
|
||||||
|
|
||||||
```json
|
|
||||||
{"date": "2026-09-10"}
|
|
||||||
```
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 评估
|
## 评估
|
||||||
|
|
||||||
### `POST /api/v1/eval/settle`
|
### `POST /api/v1/eval/settle`(需管理员)
|
||||||
|
|
||||||
赛后回填实际比分:
|
赛后回填实际比分:
|
||||||
|
|
||||||
@@ -165,7 +208,7 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
{"prediction_id": 7, "home_goals": 2, "away_goals": 1}
|
{"prediction_id": 7, "home_goals": 2, "away_goals": 1}
|
||||||
```
|
```
|
||||||
|
|
||||||
### `GET /api/v1/eval/summary`
|
### `GET /api/v1/eval/summary`(需管理员)
|
||||||
|
|
||||||
按 `provider × model` 聚合已结算预测:
|
按 `provider × model` 聚合已结算预测:
|
||||||
|
|
||||||
@@ -183,7 +226,10 @@ Base URL: `http://localhost:8000` · 交互式文档: `/docs`(Swagger)与 `/redo
|
|||||||
|
|
||||||
## 基础
|
## 基础
|
||||||
|
|
||||||
| 端点 | 说明 |
|
| 端点 | 权限 | 说明 |
|
||||||
|---|---|
|
|---|---|---|
|
||||||
| `GET /health` | 存活检查 |
|
| `GET /health` | 公开 | 存活检查 |
|
||||||
| `GET /docs` | Swagger UI |
|
| `GET /docs`、`GET /redoc` | 公开 | Swagger UI / ReDoc |
|
||||||
|
| `POST /auth/login`、`POST /auth/logout`、`GET /auth/me` | 公开 | 管理员 Cookie 会话登录/登出/当前用户 |
|
||||||
|
| `POST /auth/change-password` | 需管理员 | 修改管理员密码 |
|
||||||
|
| `POST /backtest` | 需管理员 | 历史回测(对已完赛比赛批量预测并评估) |
|
||||||
|
|||||||
+7
-7
@@ -10,7 +10,7 @@ Profeto 的核心预测路径是 **5 个领域专家 Agent 并行分析 + 1 个
|
|||||||
| `form` 近期状态 | 分析比分与关键事件,判断近期走势 | 两队近 N 场赛果(含 xG) | `home_edge` + 走势判断 |
|
| `form` 近期状态 | 分析比分与关键事件,判断近期走势 | 两队近 N 场赛果(含 xG) | `home_edge` + 走势判断 |
|
||||||
| `stats` 攻防数据 | 评估进球、射门与控球,量化攻防强度 | 近 N 场进球/射门/控球/xG 统计 | `home_edge` + 攻防强度 |
|
| `stats` 攻防数据 | 评估进球、射门与控球,量化攻防强度 | 近 N 场进球/射门/控球/xG 统计 | `home_edge` + 攻防强度 |
|
||||||
| `home_away` 主客因素 | 对比主场与客场表现,评估地理优势影响 | 主队主场战绩 + 客队客场战绩 | `home_edge` + 地理优势 |
|
| `home_away` 主客因素 | 对比主场与客场表现,评估地理优势影响 | 主队主场战绩 + 客队客场战绩 | `home_edge` + 地理优势 |
|
||||||
| `injuries` 阵容完整性 | 汇总伤停与停赛名单,评估战力缺失程度 | 伤停数据(当前无源 → no_data 门控) | `home_edge` 或 `no_data` |
|
| `standings` 联赛排名 | 结合积分榜排名、积分与分区(欧冠/欧联/降级),评估双方竞争位置 | 两队当前赛季积分榜行(排名/积分/分区/近期战绩) | `home_edge` 或 `no_data` |
|
||||||
| `h2h` 历史交锋 | 分析过去数年以及近期的交手数据,提取交手规律 | 近 N 次交锋(含主客方向 + 总计统计) | `home_edge` + 交手规律 |
|
| `h2h` 历史交锋 | 分析过去数年以及近期的交手数据,提取交手规律 | 近 N 次交锋(含主客方向 + 总计统计) | `home_edge` + 交手规律 |
|
||||||
| `aggregator` 终裁 | 权衡 5 份报告 → 最终结论 | 5 份结构化报告 + 比赛头信息 | 最终预测 + 各报告采信度 |
|
| `aggregator` 终裁 | 权衡 5 份报告 → 最终结论 | 5 份结构化报告 + 比赛头信息 | 最终预测 + 各报告采信度 |
|
||||||
|
|
||||||
@@ -25,7 +25,7 @@ POST /predict {match_id, mode: "multi"}
|
|||||||
│ ├─ form agent ─┐
|
│ ├─ form agent ─┐
|
||||||
│ ├─ stats agent │ 每个 agent 拿到专属数据切片
|
│ ├─ stats agent │ 每个 agent 拿到专属数据切片
|
||||||
│ ├─ home_away agent │ → no_data 门控 → 调 LLM → 输出 JSON 报告
|
│ ├─ home_away agent │ → no_data 门控 → 调 LLM → 输出 JSON 报告
|
||||||
│ ├─ injuries agent │ (无数据 → 跳过 LLM,返回 stub)
|
│ ├─ standings agent │ (无数据 → 跳过 LLM,返回 stub)
|
||||||
│ └─ h2h agent ─┘
|
│ └─ h2h agent ─┘
|
||||||
│
|
│
|
||||||
├─ aggregator agent(5 份报告 + 比赛头 → 最终 JSON)
|
├─ aggregator agent(5 份报告 + 比赛头 → 最终 JSON)
|
||||||
@@ -39,12 +39,12 @@ POST /predict {match_id, mode: "multi"}
|
|||||||
5 个专家通过 `asyncio.gather` 并发,总延迟 ≈ `max(专家延迟) + 终裁延迟` ≈ 2 次串行 LLM 调用。
|
5 个专家通过 `asyncio.gather` 并发,总延迟 ≈ `max(专家延迟) + 终裁延迟` ≈ 2 次串行 LLM 调用。
|
||||||
|
|
||||||
### 2. no_data 门控(省 token、防幻觉)
|
### 2. no_data 门控(省 token、防幻觉)
|
||||||
数据切片为空时(如伤停数据源未接入),**跳过 LLM 调用**,直接返回:
|
数据切片为空时(如该场比赛的积分榜尚未采集),**跳过 LLM 调用**,直接返回:
|
||||||
```json
|
```json
|
||||||
{"agent": "injuries", "status": "no_data", "data_sufficiency": "none",
|
{"agent": "standings", "status": "no_data", "data_sufficiency": "none",
|
||||||
"analysis": "该维度无数据,跳过分析。"}
|
"analysis": "该维度无数据,跳过分析。"}
|
||||||
```
|
```
|
||||||
终裁 Agent 会看到这个 `no_data` 状态,不会编造伤停分析。
|
终裁 Agent 会看到这个 `no_data` 状态,不会编造积分榜分析。
|
||||||
|
|
||||||
### 3. fail-open(单专家失败不阻断)
|
### 3. fail-open(单专家失败不阻断)
|
||||||
单个专家 LLM 调用失败 → 其报告标记 `status: error`,其余 4 份 + 终裁照常执行。
|
单个专家 LLM 调用失败 → 其报告标记 `status: error`,其余 4 份 + 终裁照常执行。
|
||||||
@@ -89,7 +89,7 @@ POST /predict {match_id, mode: "multi"}
|
|||||||
"1x2": "1",
|
"1x2": "1",
|
||||||
"subjective_confidence": 0.68,
|
"subjective_confidence": 0.68,
|
||||||
"reasoning": "综合 stats 报告的攻防强度与 form 报告的三连胜势头……",
|
"reasoning": "综合 stats 报告的攻防强度与 form 报告的三连胜势头……",
|
||||||
"agent_weights": {"form": 0.9, "stats": 0.8, "home_away": 0.7, "injuries": 0.0, "h2h": 0.8}
|
"agent_weights": {"form": 0.9, "stats": 0.8, "home_away": 0.7, "standings": 0.8, "h2h": 0.8}
|
||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -120,7 +120,7 @@ agents/
|
|||||||
├── form_v1.md # 近期状态专家
|
├── form_v1.md # 近期状态专家
|
||||||
├── stats_v1.md # 攻防数据专家
|
├── stats_v1.md # 攻防数据专家
|
||||||
├── home_away_v1.md # 主客因素专家
|
├── home_away_v1.md # 主客因素专家
|
||||||
├── injuries_v1.md # 阵容完整性专家
|
├── standings_v1.md # 联赛排名专家
|
||||||
├── h2h_v1.md # 历史交锋专家
|
├── h2h_v1.md # 历史交锋专家
|
||||||
└── aggregator_v1.md # 终裁
|
└── aggregator_v1.md # 终裁
|
||||||
```
|
```
|
||||||
|
|||||||
+56
-14
@@ -4,9 +4,10 @@
|
|||||||
|
|
||||||
| 数据源 | 用途 | 必需 Key | 说明 |
|
| 数据源 | 用途 | 必需 Key | 说明 |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| bzzoiro | 赛果/赛程(主源) | `BZZOIRO_KEY` | 五大联赛历史 + 实时 |
|
| bzzoiro(唯一) | 赛果/赛程/积分榜/统计(xG、射门、控球等) | `BZZOIRO_KEY` | 五大联赛 + 欧战,历史 + 实时 |
|
||||||
| understat | xG 回填 | 无(公开) | 仅五大联赛,补 `match_stats.xg` |
|
|
||||||
| api-football | 伤停 | `API_FOOTBALL_KEY` | 当前只采集计数,未接入 context |
|
> 历史版本曾有 understat(xG 回填)与 api-football(伤停)两个辅助源,
|
||||||
|
> 现已移除:数据源收敛为 bzzoiro 唯一来源,统计与积分榜均由 bzzoiro 管线采集。
|
||||||
|
|
||||||
### bzzoiro
|
### bzzoiro
|
||||||
|
|
||||||
@@ -28,12 +29,6 @@
|
|||||||
> - 黄牌: `home_yellow_cards` / `away_yellow_cards`
|
> - 黄牌: `home_yellow_cards` / `away_yellow_cards`
|
||||||
> - 红牌: `home_red_cards` / `away_red_cards`
|
> - 红牌: `home_red_cards` / `away_red_cards`
|
||||||
|
|
||||||
### understat
|
|
||||||
|
|
||||||
- 端点:`/getLeagueData/{league}/{season}`,返回 JS 包裹的 JSON(需正则提取)
|
|
||||||
- 只回填 xG(`match_stats.home_xg`/`away_xg`),**不创建新比赛**
|
|
||||||
- 通过"天级日期 + 队名归一"匹配已有比赛
|
|
||||||
|
|
||||||
## 数据清洗契约
|
## 数据清洗契约
|
||||||
|
|
||||||
所有数据源统一清洗为 `NormalizedMatch`(`src/data/normalize.py`),字段:
|
所有数据源统一清洗为 `NormalizedMatch`(`src/data/normalize.py`),字段:
|
||||||
@@ -68,9 +63,39 @@
|
|||||||
`src/data/team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
|
`src/data/team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
|
||||||
归一前先做 Unicode NFKD 去重音。
|
归一前先做 Unicode NFKD 去重音。
|
||||||
|
|
||||||
|
**唯一键是归一后英文名**:`teams.name` 带 `UNIQUE` 约束,所有入库路径均经 `TeamRepository.get_or_create` 收敛归一化
|
||||||
|
(events / standings 管线在调用前归一,仓库层再做一次幂等归一作为兜底)。创建新 Team 时打 `info` 日志记录「原始名 → 归一后名」。
|
||||||
|
|
||||||
|
> ⚠️ **`normalize` 当前大小写敏感**:仅当入参大小写与 `NORMALIZE_MAP` 键完全匹配时才触发映射
|
||||||
|
>(如 `"Man City"` → `"Manchester City"`,但 `"man city"` 原样保留)。上游 bzzoiro 返回的队名首字母大写,
|
||||||
|
>实际命中无问题;若新增数据源返回全小写/全大写队名,需先 `title()` 再归一,否则会绕过映射产生重复 Team。
|
||||||
|
|
||||||
|
**改名 / 合并流程**(人工):
|
||||||
|
|
||||||
|
当发现两个 `teams` 行实际是同一球队(如 `Manchester City` 与 `Man City` 因历史数据大小写差异各占一行):
|
||||||
|
|
||||||
|
1. 确定**保留行**(通常选归一后规范名、且被更多 Match 引用的那行)。
|
||||||
|
2. 将被删行的所有引用指向保留行(`UPDATE matches SET home_team_id = 保留id WHERE home_team_id = 删行id`,客场同理;
|
||||||
|
`standings` / `match_stats` 按 `team_id` 同理)。
|
||||||
|
3. 删掉多余行:`DELETE FROM teams WHERE id = 删行id`。
|
||||||
|
|
||||||
|
> 此过程引入外键约束风险,务必在事务中执行并先 `BEGIN; ... ` 验证行数后再 `COMMIT`。
|
||||||
|
> 暂不做自动合并(避免误合相似名),仅通过下方 Admin 接口列出「近似重名」候选,由人工判定。
|
||||||
|
|
||||||
|
## Admin:近似重名候选
|
||||||
|
|
||||||
|
`GET /api/v1/admin/team-name-duplicates` 只读列出启发式相似候选(大小写差异、子串包含、前缀碰撞),不做自动合并。
|
||||||
|
典型用途:定期巡检,发现候选后走上方人工 SQL 合并。启发式规则:
|
||||||
|
|
||||||
|
- **大小写变体**:`lower(name)` 相同但 `name` 不同(如 `Arsenal FC` / `arsenal fc`)。
|
||||||
|
- **子串包含**:A 是 B 的子串且长度 ≥ 5(如 `Manchester` / `Manchester City`)。
|
||||||
|
- **前缀碰撞**:前 8 个字符相同的两队。
|
||||||
|
|
||||||
|
命中任一规则即列为候选,按相似度分组返回。
|
||||||
|
|
||||||
## 数据库 Schema
|
## 数据库 Schema
|
||||||
|
|
||||||
6 张表:
|
12 张表:核心业务表 5 张见下方 DDL,其余 7 张(积分榜/配置/调度/治理)见后文表格。
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
-- 联赛
|
-- 联赛
|
||||||
@@ -144,12 +169,25 @@ CREATE TABLE predictions (
|
|||||||
reasoning TEXT,
|
reasoning TEXT,
|
||||||
raw_response JSONB, -- LLM 完整原始响应
|
raw_response JSONB, -- LLM 完整原始响应
|
||||||
agent_outputs JSONB, -- multi 模式: 5 份专家报告
|
agent_outputs JSONB, -- multi 模式: 5 份专家报告
|
||||||
|
agent_weights JSONB, -- multi 模式: 终裁给出的各专家权重
|
||||||
created_at TIMESTAMPTZ,
|
created_at TIMESTAMPTZ,
|
||||||
actual_home_goals INT, actual_away_goals INT, -- 赛后回填
|
actual_home_goals INT, actual_away_goals INT, -- 赛后回填
|
||||||
settled BOOLEAN DEFAULT FALSE
|
settled BOOLEAN DEFAULT FALSE
|
||||||
);
|
);
|
||||||
```
|
```
|
||||||
|
|
||||||
|
其余 7 张表(DDL 略,详见 `src/db/models.py` 与 alembic 迁移):
|
||||||
|
|
||||||
|
| 表 | 状态 | 用途 |
|
||||||
|
|---|---|---|
|
||||||
|
| `standings` | 已启用 | 联赛积分榜快照,按 `(league_id, season, team_id)` upsert,同联赛同赛季只保留最新快照;含排名/战绩/进失球/积分/分区(zone) |
|
||||||
|
| `app_settings` | 已启用 | 后台运行时设置(如数据源 API Key),读取时优先于 `.env` 默认值 |
|
||||||
|
| `schedules` | 已启用 | 定时采集任务配置(task/cron/leagues/enabled),供内置调度器执行 |
|
||||||
|
| `raw_events` | 预留未启用 | Bronze 层原始事件存档;规划中用于重放与审计 |
|
||||||
|
| `ingest_failures` | 已启用 | 采集失败死信:bzzoiro 三条管线(events/standings/stats)抓取失败时写入,admin 后台可查看与重试 |
|
||||||
|
| `data_quality_checks` | 预留未启用 | 数据质量检查结果;规划中定时检查比赛/统计/积分榜完整性 |
|
||||||
|
| `data_lineage` | 预留未启用 | ETL 血缘追踪;规划中记录源记录到目标表的映射 |
|
||||||
|
|
||||||
### 关键设计点
|
### 关键设计点
|
||||||
|
|
||||||
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
|
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
|
||||||
@@ -170,7 +208,9 @@ CREATE TABLE predictions (
|
|||||||
- 状态:只允许单向升级(`scheduled` → `finished`),防止完赛行被覆盖成赛程
|
- 状态:只允许单向升级(`scheduled` → `finished`),防止完赛行被覆盖成赛程
|
||||||
- stats:只补空(`home_xg` 已有值时不覆盖)
|
- stats:只补空(`home_xg` 已有值时不覆盖)
|
||||||
|
|
||||||
`ingest_understat` 只回填 xG(也只补空),不创建比赛。
|
`task=stats` 只回填统计(xG/射门/控球等,也只补空),不创建比赛。
|
||||||
|
|
||||||
|
`task=standings` 按 `(league_id, season, team_id)` upsert 积分榜快照,同一联赛同一赛季只保留最新一份。
|
||||||
|
|
||||||
## 采集建议
|
## 采集建议
|
||||||
|
|
||||||
@@ -183,9 +223,11 @@ curl -X POST /api/v1/ingest/bzzoiro \
|
|||||||
curl -X POST /api/v1/ingest/bzzoiro \
|
curl -X POST /api/v1/ingest/bzzoiro \
|
||||||
-d '{"leagues":["E0"],"date_from":"2026-09-01","date_to":"2026-09-08"}'
|
-d '{"leagues":["E0"],"date_from":"2026-09-01","date_to":"2026-09-08"}'
|
||||||
|
|
||||||
# 3. xG 回填(可选,提升 xg agent 质量)
|
# 3. 积分榜 + 统计回填(xG/射门/控球,提升 stats/standings 专家质量)
|
||||||
curl -X POST /api/v1/ingest/understat -d '{"league":"E0","season":2025}'
|
curl -X POST /api/v1/ingest/bzzoiro -d '{"task":"standings"}'
|
||||||
curl -X POST /api/v1/ingest/understat -d '{"league":"E0","season":2026}'
|
curl -X POST /api/v1/ingest/bzzoiro -d '{"task":"stats","leagues":["E0"],"limit":300}'
|
||||||
|
|
||||||
|
# 注:采集端点需管理员凭据(Cookie 会话或 X-API-Key 头),下同
|
||||||
```
|
```
|
||||||
|
|
||||||
建议用外部 cron(如系统 crontab)定时触发,不引入 worker/redis。
|
建议用外部 cron(如系统 crontab)定时触发,不引入 worker/redis。
|
||||||
|
|||||||
+39
-2
@@ -33,6 +33,25 @@ curl http://localhost:8000/health
|
|||||||
> **注意**: `api` 服务启动时会先执行 `alembic upgrade head` 迁移数据库,再启动 uvicorn。
|
> **注意**: `api` 服务启动时会先执行 `alembic upgrade head` 迁移数据库,再启动 uvicorn。
|
||||||
> 容器内数据库连接自动使用 `postgres` 服务名(通过 compose `environment` 覆盖 `.env` 中的 `DB_HOST`)。
|
> 容器内数据库连接自动使用 `postgres` 服务名(通过 compose `environment` 覆盖 `.env` 中的 `DB_HOST`)。
|
||||||
|
|
||||||
|
## 生产上线检查清单
|
||||||
|
|
||||||
|
公网上线前逐项勾选。第 1–4 项由 `src/core/security_check.py` 在 `APP_ENV=production`
|
||||||
|
启动时**强制校验**,不满足直接拒绝启动(开发环境仅告警);管理鉴权另有请求期 fail-closed(503)。
|
||||||
|
|
||||||
|
- [ ] **1. `APP_ENV=production`** — 安全校验、Cookie `Secure`、管理端点 fail-closed 均以它为总开关
|
||||||
|
- [ ] **2. `SECRET_KEY` 强随机** — 用 `openssl rand -base64 32` 生成;禁止弱值/短值(弱值黑名单会拒绝启动,含把生成指令原样粘进去的情况)
|
||||||
|
- [ ] **3. 管理鉴权至少其一** — `ADMIN_PASSWORD` 或 `ADMIN_API_KEY`(后台改过密码后以数据库哈希优先);两者皆空时管理接口 503
|
||||||
|
- [ ] **4. 数据库强密码** — 禁止 `football:football` 等示例弱密码(启动校验会拒绝);compose 的 `POSTGRES_PASSWORD` 必填,缺失时容器拒绝启动
|
||||||
|
- [ ] **5. HTTPS** — 由反代(Nginx/Caddy)终结 TLS;`APP_ENV=production` 下会话 Cookie 自动 `Secure`(且 HttpOnly + SameSite=Lax)
|
||||||
|
- [ ] **6. 反代信任头** — `TRUST_PROXY_HEADERS=True`,且**仅可信反代可达 API**;反代需设置 `X-Forwarded-For`(`$proxy_add_x_forwarded_for`)与 `X-Real-IP`,否则限流/日志按反代 IP 计数
|
||||||
|
- [ ] **7. 限流前置到网关** — 推荐 Nginx `limit_req`(配置见[安全与限流](#安全与限流));应用内限流与 KeyRing 为**单进程内存实现**,多 worker 各自独立计数会把实际配额放大 N 倍(启动时会打印一次性告警)
|
||||||
|
- [ ] **8. uvicorn 单 worker** — compose/Dockerfile 默认单 worker,保持即可;需横向扩容时先在网关统一限流,再起多实例(每实例仍单 worker)
|
||||||
|
|
||||||
|
> ⚠️ **多 worker 陷阱**:应用内限流(`_RateLimiter`)与 KeyRing 均为**进程内纯内存状态**,多 worker 部署(如 `uvicorn --workers 4`)时各进程**各自独立计数、互不共享**——实际限流配额会被放大 N 倍、KeyRing 限流状态也不同步。
|
||||||
|
> 若确需多 worker,必须前置 Nginx/网关做**全局限流**(见[安全与限流](#安全与限流)),并设环境变量 `STRICT_SINGLE_WORKER=True`(见下)在启动期强制拒绝多 worker,避免静默配额漂移。
|
||||||
|
- [ ] **9. 启动后健康检查** — `curl /health` 返回 200(存活);`curl /health/ready` 返回 200(就绪,校验数据库连通,不可达时 503)
|
||||||
|
- [ ] **10. 数据库迁移** — compose/Dockerfile 启动命令已内置 `alembic upgrade head && uvicorn …`,升级镜像重启即自动迁移,无需手动执行
|
||||||
|
|
||||||
## 本地开发部署
|
## 本地开发部署
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
@@ -81,13 +100,31 @@ cd frontend && npm install && npm run dev
|
|||||||
| `LLM_TIMEOUT` | ❌ | `60` | 单次调用超时(秒) |
|
| `LLM_TIMEOUT` | ❌ | `60` | 单次调用超时(秒) |
|
||||||
| `LLM_SPECIALIST_MODEL` | ❌ | — | 专家模型(回落 `LLM_MODEL`) |
|
| `LLM_SPECIALIST_MODEL` | ❌ | — | 专家模型(回落 `LLM_MODEL`) |
|
||||||
| `LLM_AGGREGATOR_MODEL` | ❌ | — | 终裁模型(回落 `LLM_MODEL`) |
|
| `LLM_AGGREGATOR_MODEL` | ❌ | — | 终裁模型(回落 `LLM_MODEL`) |
|
||||||
| `BZZOIRO_KEY` | ✅ | — | bzzoiro 数据源 Key |
|
| `BZZOIRO_KEY` | ✅ | — | bzzoiro 数据源 Key(唯一数据源) |
|
||||||
| `API_FOOTBALL_KEY` | ❌ | — | 伤停数据源 Key |
|
|
||||||
| `CORS_ORIGINS` | ❌ | `http://localhost:5173,...` | 允许的跨域来源 |
|
| `CORS_ORIGINS` | ❌ | `http://localhost:5173,...` | 允许的跨域来源 |
|
||||||
|
| `STRICT_SINGLE_WORKER` | ❌ | `False` | `True` 时若以多 worker 启动则拒绝(防限流配额漂移) |
|
||||||
| `SECRET_KEY` | ❌ | — | 加密主密钥(生产环境必填) |
|
| `SECRET_KEY` | ❌ | — | 加密主密钥(生产环境必填) |
|
||||||
| `ADMIN_PASSWORD` | ❌ | — | 管理后台密码(留空=不启用) |
|
| `ADMIN_PASSWORD` | ❌ | — | 管理后台密码(留空=不启用) |
|
||||||
| `ADMIN_API_KEY` | ❌ | — | 机器/脚本调用的 API Key |
|
| `ADMIN_API_KEY` | ❌ | — | 机器/脚本调用的 API Key |
|
||||||
|
|
||||||
|
## 同站部署 vs 跨站 CSRF
|
||||||
|
|
||||||
|
Profeto 管理鉴权使用 **HttpOnly Cookie 会话**(登录后服务端写入),`allow_credentials=True` 的 CORS 配置允许浏览器跨域携带 Cookie——这也引入了 CSRF 面。部署拓扑决定风险等级:
|
||||||
|
|
||||||
|
**同站部署(推荐)**: 前端与 API 同域(反代把 `/` 与 `/api` 都转发到同一后端,或同源端口)。
|
||||||
|
- 浏览器视为 **same-origin**,CORS 不触发;`SameSite=Lax` 会话 Cookie 天然阻断跨站请求携带。
|
||||||
|
- 风险最低。`CORS_ORIGINS` 可设为空或同域来源,仅作兜底。
|
||||||
|
|
||||||
|
**跨站部署**: 前端与 API 不同域(如前端 `app.example.com`、API `api.example.com`,或开发时 `localhost:3000` → `localhost:8000`)。
|
||||||
|
- 必须把 API 域名列入 `CORS_ORIGINS`,且 `allow_credentials=True` 才能携带 Cookie。
|
||||||
|
- 此时任何被允许域下的页面都能构造带 Cookie 的请求 → **CSRF 面**:
|
||||||
|
- 状态变更接口(采集/回测/改密等写操作)要求**管理员 Cookie + 同域**,攻击者无法从第三方站点读取 Cookie,但可构造跨域表单/请求——`SameSite=Lax` 会阻断跨站 POST 表单提交(顶级导航 GET 仍放行),这是当前主要防线。
|
||||||
|
- `GET /api/v1/admin/*` 只读接口受 `SameSite=Lax` 下顶级导航可能被利用,但攻击者无法读取响应(CORS 不匹配时浏览器拦截)。
|
||||||
|
- **加固建议**:
|
||||||
|
1. 反代层加 `Origin`/`Referer` 校验,仅放行 `CORS_ORIGINS` 列表中的来源(即便 FastAPI CORS 已通过,反代校验是多一层纵深)。
|
||||||
|
2. 写操作要求自定义请求头(如 `X-Requested-With: XMLHttpRequest`),第三方站点无法在无预检下添加自定义头,天然阻断简单跨站 POST。
|
||||||
|
3. 生产强制 HTTPS(`APP_ENV=production` 下 Cookie 自动 `Secure`),防中间人窃 Cookie。
|
||||||
|
|
||||||
## LLM 提供商配置示例
|
## LLM 提供商配置示例
|
||||||
|
|
||||||
### OpenAI
|
### OpenAI
|
||||||
|
|||||||
+11
-7
@@ -83,16 +83,16 @@ Profeto/
|
|||||||
│ │ └── app.py # FastAPI 工厂
|
│ │ └── app.py # FastAPI 工厂
|
||||||
│ ├── db/
|
│ ├── db/
|
||||||
│ │ ├── base.py # SQLAlchemy async engine + session
|
│ │ ├── base.py # SQLAlchemy async engine + session
|
||||||
│ │ ├── models.py # 6 张表 ORM
|
│ │ ├── models.py # 12 张表 ORM
|
||||||
│ │ ├── repositories.py # 仓储层(查询封装)
|
│ │ ├── repositories.py # 仓储层(查询封装)
|
||||||
│ │ └── unit_of_work.py # 事务边界
|
│ │ └── unit_of_work.py # 事务边界
|
||||||
│ ├── data/
|
│ ├── data/
|
||||||
│ │ ├── bzzoiro.py # bzzoiro 采集 + 入库
|
│ │ ├── bzzoiro.py # bzzoiro 采集 + 入库(唯一数据源)
|
||||||
│ │ ├── understat.py # understat xG 回填
|
|
||||||
│ │ ├── injuries.py # 伤停采集
|
|
||||||
│ │ ├── normalize.py # 数据清洗契约
|
│ │ ├── normalize.py # 数据清洗契约
|
||||||
│ │ ├── team_names.py # 队名归一化映射
|
│ │ ├── team_names.py # 队名归一化映射
|
||||||
|
│ │ ├── team_names_zh.py # 队名中文名映射
|
||||||
│ │ ├── sources.py # 数据源注册表
|
│ │ ├── sources.py # 数据源注册表
|
||||||
|
│ │ ├── key_ring.py # 数据源 Key 读取(DB 设置优先于 env)
|
||||||
│ │ └── config.py # 联赛映射常量
|
│ │ └── config.py # 联赛映射常量
|
||||||
│ ├── llm/
|
│ ├── llm/
|
||||||
│ │ ├── provider.py # LLM 提供商抽象(OpenAI-compatible)
|
│ │ ├── provider.py # LLM 提供商抽象(OpenAI-compatible)
|
||||||
@@ -110,13 +110,17 @@ Profeto/
|
|||||||
│ │ ├── form_v1.md
|
│ │ ├── form_v1.md
|
||||||
│ │ ├── stats_v1.md
|
│ │ ├── stats_v1.md
|
||||||
│ │ ├── home_away_v1.md
|
│ │ ├── home_away_v1.md
|
||||||
│ │ ├── injuries_v1.md
|
│ │ ├── standings_v1.md
|
||||||
│ │ ├── h2h_v1.md
|
│ │ ├── h2h_v1.md
|
||||||
│ │ └── aggregator_v1.md
|
│ │ └── aggregator_v1.md
|
||||||
│ └── core/
|
│ └── core/
|
||||||
│ ├── config.py # pydantic-settings 配置
|
│ ├── config.py # pydantic-settings 配置
|
||||||
│ ├── http_client.py # 共享 httpx 客户端
|
│ ├── http_client.py # 共享 httpx 客户端
|
||||||
│ └── retry.py # 重试工具
|
│ ├── crypto.py # 对称加密(Fernet)与密码哈希
|
||||||
|
│ ├── log_buffer.py # 内存日志缓冲(admin「系统日志」页)
|
||||||
|
│ ├── runtime_config.py # 运行时配置(数据库优先,回落 .env)
|
||||||
|
│ ├── scheduler.py # 定时任务调度器(cron 触发采集)
|
||||||
|
│ └── security_check.py # 生产启动安全校验(缺配置拒绝启动)
|
||||||
├── frontend/ # React 单页前端
|
├── frontend/ # React 单页前端
|
||||||
├── alembic/ # 数据库迁移
|
├── alembic/ # 数据库迁移
|
||||||
│ └── versions/
|
│ └── versions/
|
||||||
@@ -194,7 +198,7 @@ cp src/llm/prompts/agents/h2h_v1.md src/llm/prompts/agents/h2h_v2.md
|
|||||||
|
|
||||||
### 3. 新增数据源
|
### 3. 新增数据源
|
||||||
|
|
||||||
1. 在 `src/data/` 写采集模块(参考 `understat.py`)
|
1. 在 `src/data/` 写采集模块(参考 `bzzoiro.py`)
|
||||||
2. 在 `normalize.py` 加清洗函数
|
2. 在 `normalize.py` 加清洗函数
|
||||||
3. 在 `context_builder.py` 加切片函数
|
3. 在 `context_builder.py` 加切片函数
|
||||||
4. 在 `api/routes/ingest.py` 加端点
|
4. 在 `api/routes/ingest.py` 加端点
|
||||||
|
|||||||
@@ -1,3 +1,9 @@
|
|||||||
|
> ⚠️ **历史文档(已过时)**:伤停(injuries)数据源与 api-football 集成已移除,
|
||||||
|
> 数据源收敛为 bzzoiro 唯一来源。本文仅作历史决策记录保留,
|
||||||
|
> 现状请见 [05-data.md](05-data.md) 与 [01-architecture.md](01-architecture.md)。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
"""检查 injuries 数据源 api-football 的响应结构。"""
|
"""检查 injuries 数据源 api-football 的响应结构。"""
|
||||||
API_FOOTBALL_INJURY_RESPONSE_EXAMPLE = """
|
API_FOOTBALL_INJURY_RESPONSE_EXAMPLE = """
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -15,13 +15,16 @@ import { fetchMatchDetail, fetchMatchContext } from '../admin/dal'
|
|||||||
import type { MatchDetailOut, MatchContextOut } from '../admin/types'
|
import type { MatchDetailOut, MatchContextOut } from '../admin/types'
|
||||||
import { useMatchesList } from './matches/hooks/useMatchesList'
|
import { useMatchesList } from './matches/hooks/useMatchesList'
|
||||||
import { useMatchPredict } from './matches/hooks/useMatchPredict'
|
import { useMatchPredict } from './matches/hooks/useMatchPredict'
|
||||||
|
import { useLeagues } from './matches/hooks/useLeagues'
|
||||||
import { PredictModal } from './matches/components/MatchPredictPanel'
|
import { PredictModal } from './matches/components/MatchPredictPanel'
|
||||||
import { MatchRow } from './matches/components/MatchDetailSection'
|
import { MatchRow } from './matches/components/MatchDetailSection'
|
||||||
import { Spinner, SkeletonRows, Switch, formatDateHeader, groupByDate, withinNext3Days } from './matches/ui'
|
import { Spinner, SkeletonRows, Switch, formatDateHeader, groupByDate, withinNext3Days } from './matches/ui'
|
||||||
import { LEAGUES, type Match } from './matches/types'
|
import { type Match } from './matches/types'
|
||||||
|
|
||||||
export default function Matches() {
|
export default function Matches() {
|
||||||
const [error, setError] = useState<string | null>(null) // 列表与预测共用(拆分前即如此)
|
const [error, setError] = useState<string | null>(null) // 列表与预测共用(拆分前即如此)
|
||||||
|
// P1-3: 联赛列表优先请求 /api/v1/leagues,失败/空回退本地五大联赛常量
|
||||||
|
const leagues = useLeagues()
|
||||||
const {
|
const {
|
||||||
league, setLeague,
|
league, setLeague,
|
||||||
status, setStatus,
|
status, setStatus,
|
||||||
@@ -58,7 +61,7 @@ export default function Matches() {
|
|||||||
}, [])
|
}, [])
|
||||||
const scrollToTop = () => window.scrollTo({ top: 0, behavior: 'smooth' })
|
const scrollToTop = () => window.scrollTo({ top: 0, behavior: 'smooth' })
|
||||||
|
|
||||||
const leagueName = LEAGUES.find(l => l.code === league)?.name ?? league
|
const leagueName = leagues.find(l => l.code === league)?.name ?? league
|
||||||
|
|
||||||
/** 未开赛默认仅展示未来 3 天;其余状态展示全部。showAllUpcoming=true 时展开全部。 */
|
/** 未开赛默认仅展示未来 3 天;其余状态展示全部。showAllUpcoming=true 时展开全部。 */
|
||||||
const isScheduledView = status === 'scheduled'
|
const isScheduledView = status === 'scheduled'
|
||||||
@@ -91,7 +94,7 @@ export default function Matches() {
|
|||||||
<div className="space-y-5">
|
<div className="space-y-5">
|
||||||
{/* ── 联赛版面切换 ── */}
|
{/* ── 联赛版面切换 ── */}
|
||||||
<nav className="flex items-center gap-6 overflow-x-auto border-b border-ink-900" aria-label="联赛">
|
<nav className="flex items-center gap-6 overflow-x-auto border-b border-ink-900" aria-label="联赛">
|
||||||
{LEAGUES.map(l => (
|
{leagues.map(l => (
|
||||||
<button
|
<button
|
||||||
key={l.code}
|
key={l.code}
|
||||||
onClick={() => setLeague(l.code)}
|
onClick={() => setLeague(l.code)}
|
||||||
|
|||||||
@@ -0,0 +1,171 @@
|
|||||||
|
/**
|
||||||
|
* AgentsPanel: 五路专家意见 —— 可折叠 + 状态摘要 + 权重条形图 + 单路详情。
|
||||||
|
*
|
||||||
|
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
|
||||||
|
*/
|
||||||
|
import { useState } from 'react'
|
||||||
|
import type { AgentReport, Prediction } from '../../types'
|
||||||
|
import { AGENT_LABELS, CN_NUM } from '../../types'
|
||||||
|
|
||||||
|
const STATUS_BADGE: Record<string, { label: string; cls: string }> = {
|
||||||
|
ok: { label: '正常', cls: 'text-ink-500' },
|
||||||
|
no_data: { label: '无数据', cls: 'text-ink-400' },
|
||||||
|
error: { label: '调用失败', cls: 'text-press' },
|
||||||
|
parse_error: { label: '解析失败', cls: 'text-press' },
|
||||||
|
}
|
||||||
|
|
||||||
|
const SUFFICIENCY_LABEL: Record<string, string> = {
|
||||||
|
high: '充分',
|
||||||
|
medium: '一般',
|
||||||
|
low: '偏少',
|
||||||
|
none: '无',
|
||||||
|
}
|
||||||
|
|
||||||
|
/** home_edge(-1~1,正=利主队)的可视化:以中线为原点的双向细条 */
|
||||||
|
function EdgeBar({ value }: { value: number }) {
|
||||||
|
const v = Math.max(-1, Math.min(1, value))
|
||||||
|
const half = Math.abs(v) * 50
|
||||||
|
return (
|
||||||
|
<div className="relative h-px w-full bg-ink-200" role="presentation">
|
||||||
|
<span className="absolute left-1/2 top-1/2 h-2 w-px -translate-x-1/2 -translate-y-1/2 bg-ink-400" />
|
||||||
|
<span
|
||||||
|
className={`absolute top-0 h-px transition-all duration-500 ${v >= 0 ? 'bg-press' : 'bg-ink-600'}`}
|
||||||
|
style={
|
||||||
|
v >= 0
|
||||||
|
? { left: '50%', width: `${half}%` }
|
||||||
|
: { right: '50%', width: `${half}%` }
|
||||||
|
}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 单路专家意见:汉字编号 + 细线行 */
|
||||||
|
function AgentCard({ report: r, no }: { report: AgentReport; no: string }) {
|
||||||
|
const badge = STATUS_BADGE[r.status] ?? { label: r.status, cls: 'text-ink-400' }
|
||||||
|
const inactive = r.status !== 'ok'
|
||||||
|
|
||||||
|
return (
|
||||||
|
<details className="group border-b border-ink-200">
|
||||||
|
<summary className="flex cursor-pointer list-none items-baseline gap-2.5 px-1 py-3">
|
||||||
|
<span className="font-serif text-sm text-ink-400">{no}</span>
|
||||||
|
<span className="text-sm font-medium text-ink-900">{AGENT_LABELS[r.agent] ?? r.agent}</span>
|
||||||
|
<span className={`text-2xs ${badge.cls}`}>{badge.label}</span>
|
||||||
|
|
||||||
|
<span className="ml-auto flex items-baseline gap-3 text-2xs tabular-nums text-ink-500">
|
||||||
|
{r.status === 'ok' && r.subjective_confidence !== null && (
|
||||||
|
<span>信心 {Math.round(r.subjective_confidence * 100)}%</span>
|
||||||
|
)}
|
||||||
|
{r.status === 'ok' && r.probable_score && (
|
||||||
|
<span className="font-serif font-bold text-ink-800">{r.probable_score}</span>
|
||||||
|
)}
|
||||||
|
<svg viewBox="0 0 20 20" className="h-3 w-3 self-center text-ink-300 transition-transform group-open:rotate-90" fill="currentColor" aria-hidden="true">
|
||||||
|
<path d="M7.3 5.3a1 1 0 011.4 0l4 4a1 1 0 010 1.4l-4 4a1 1 0 01-1.4-1.4L10.6 10 7.3 6.7a1 1 0 010-1.4z" />
|
||||||
|
</svg>
|
||||||
|
</span>
|
||||||
|
</summary>
|
||||||
|
|
||||||
|
<div className="space-y-3 px-1 pb-4 pl-7">
|
||||||
|
{inactive && (
|
||||||
|
<p className="text-xs leading-relaxed text-ink-500">
|
||||||
|
{r.status === 'no_data' && '该维度没有可用数据,已跳过 LLM 分析以节省额度(不影响其他专家)。'}
|
||||||
|
{r.status === 'error' && '该专家调用失败,本次结论未纳入其视角(fail-open 设计,不阻断整体预测)。'}
|
||||||
|
{r.status === 'parse_error' && '模型输出未通过格式校验,该报告已丢弃。'}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{!inactive && r.home_edge !== null && (
|
||||||
|
<div>
|
||||||
|
<div className="mb-1.5 flex items-baseline justify-between text-2xs">
|
||||||
|
<span className="text-ink-500">主队优势</span>
|
||||||
|
<span className={`font-semibold tabular-nums ${r.home_edge > 0 ? 'text-press' : r.home_edge < 0 ? 'text-ink-700' : 'text-ink-500'}`}>
|
||||||
|
{r.home_edge > 0 ? '+' : ''}{r.home_edge.toFixed(2)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<EdgeBar value={r.home_edge} />
|
||||||
|
<div className="mt-1 flex justify-between text-2xs text-ink-400">
|
||||||
|
<span>利客队</span>
|
||||||
|
<span>利主队</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{r.analysis && (
|
||||||
|
<p className="font-serif text-sm leading-loose text-ink-700">{r.analysis}</p>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{r.key_evidence.length > 0 && (
|
||||||
|
<ul className="space-y-1.5">
|
||||||
|
{r.key_evidence.map((e, i) => (
|
||||||
|
<li key={i} className="flex gap-2 text-xs leading-relaxed text-ink-600">
|
||||||
|
<span className="flex-shrink-0 text-ink-300" aria-hidden="true">—</span>
|
||||||
|
<span>{e}</span>
|
||||||
|
</li>
|
||||||
|
))}
|
||||||
|
</ul>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{r.exp_home_goals !== null && r.exp_away_goals !== null && (
|
||||||
|
<p className="text-xs text-ink-500">
|
||||||
|
进球期望 <span className="font-serif font-bold tabular-nums text-ink-900">{r.exp_home_goals.toFixed(1)} - {r.exp_away_goals.toFixed(1)}</span>
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{!inactive && (
|
||||||
|
<p className="border-t border-ink-100 pt-2.5 text-2xs text-ink-400">
|
||||||
|
数据充分度 {SUFFICIENCY_LABEL[r.data_sufficiency] ?? r.data_sufficiency}
|
||||||
|
<span className="mx-2 text-ink-200">|</span>
|
||||||
|
<span className="font-mono">{r.model}</span>
|
||||||
|
{r.latency_ms !== null && <span className="ml-2 tabular-nums">{r.latency_ms}ms</span>}
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</details>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export function AgentsPanel({ prediction }: { prediction: Prediction }) {
|
||||||
|
const [expertsOpen, setExpertsOpen] = useState(false)
|
||||||
|
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
|
||||||
|
const reports = prediction.agent_outputs ?? []
|
||||||
|
const okReports = reports.filter(r => r.status === 'ok')
|
||||||
|
|
||||||
|
if (reports.length === 0) return null
|
||||||
|
|
||||||
|
return (
|
||||||
|
<section>
|
||||||
|
<button
|
||||||
|
onClick={() => setExpertsOpen(o => !o)}
|
||||||
|
className="flex w-full items-center justify-between border-b border-ink-200 pb-2 text-left"
|
||||||
|
>
|
||||||
|
<span className="section-head mb-0">五路专家意见({okReports.length}/{reports.length} 路有效)</span>
|
||||||
|
<span className="text-2xs text-ink-400">{expertsOpen ? '收起' : '展开'}</span>
|
||||||
|
</button>
|
||||||
|
|
||||||
|
{!degraded && prediction.agent_weights && Object.keys(prediction.agent_weights).length > 0 && (
|
||||||
|
<div className="mt-3 space-y-1.5">
|
||||||
|
<span className="text-2xs text-ink-500">终裁权重分布</span>
|
||||||
|
{Object.entries(prediction.agent_weights)
|
||||||
|
.sort((a, b) => b[1] - a[1])
|
||||||
|
.map(([k, v]) => (
|
||||||
|
<div key={k} className="grid grid-cols-[96px_minmax(0,1fr)_40px] items-center gap-2">
|
||||||
|
<span className="truncate text-2xs text-ink-500">{AGENT_LABELS[k] ?? k}</span>
|
||||||
|
<div className="h-1.5 bg-paper-100">
|
||||||
|
<div className="h-full bg-press" style={{ width: `${Math.round(v * 100)}%` }} />
|
||||||
|
</div>
|
||||||
|
<span className="text-right text-2xs tabular-nums text-ink-500">{Math.round(v * 100)}%</span>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{expertsOpen && (
|
||||||
|
<div className="mt-2">
|
||||||
|
{reports.map((r, i) => (
|
||||||
|
<AgentCard key={r.agent} report={r} no={CN_NUM[i] ?? String(i + 1)} />
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</section>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -6,37 +6,11 @@
|
|||||||
*/
|
*/
|
||||||
import { useEffect, useState } from 'react'
|
import { useEffect, useState } from 'react'
|
||||||
import TeamSideTag from '../../../components/TeamSideTag'
|
import TeamSideTag from '../../../components/TeamSideTag'
|
||||||
import type { AgentReport, Match, Prediction } from '../types'
|
import type { Match, Prediction } from '../types'
|
||||||
import { AGENT_LABELS, CN_NUM, OUTCOME_LABEL } from '../types'
|
import { AGENT_LABELS } from '../types'
|
||||||
|
import { AgentsPanel } from './AgentsPanel'
|
||||||
/** 置信度细线:0~1 数值的低调可视化 */
|
import { OutcomePanel } from './OutcomePanel'
|
||||||
function Meter({ value }: { value: number }) {
|
import { ReasoningPanel } from './ReasoningPanel'
|
||||||
const pct = Math.max(0, Math.min(100, Math.round(value * 100)))
|
|
||||||
return (
|
|
||||||
<div className="h-px w-full bg-ink-200" role="presentation">
|
|
||||||
<div className="h-px bg-press transition-[width] duration-500" style={{ width: `${pct}%` }} />
|
|
||||||
</div>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** home_edge(-1~1,正=利主队)的可视化:以中线为原点的双向细条 */
|
|
||||||
function EdgeBar({ value }: { value: number }) {
|
|
||||||
const v = Math.max(-1, Math.min(1, value))
|
|
||||||
const half = Math.abs(v) * 50
|
|
||||||
return (
|
|
||||||
<div className="relative h-px w-full bg-ink-200" role="presentation">
|
|
||||||
<span className="absolute left-1/2 top-1/2 h-2 w-px -translate-x-1/2 -translate-y-1/2 bg-ink-400" />
|
|
||||||
<span
|
|
||||||
className={`absolute top-0 h-px transition-all duration-500 ${v >= 0 ? 'bg-press' : 'bg-ink-600'}`}
|
|
||||||
style={
|
|
||||||
v >= 0
|
|
||||||
? { left: '50%', width: `${half}%` }
|
|
||||||
: { right: '50%', width: `${half}%` }
|
|
||||||
}
|
|
||||||
/>
|
|
||||||
</div>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
function Spinner({ className = '' }: { className?: string }) {
|
function Spinner({ className = '' }: { className?: string }) {
|
||||||
return (
|
return (
|
||||||
@@ -53,77 +27,73 @@ function Spinner({ className = '' }: { className?: string }) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/** 胜平负一行文字:选中的红字加方块标记,未选中的退灰 */
|
/** 胜平负一行文字:选中的红字加方块标记,未选中的退灰 */
|
||||||
function OutcomeLine({
|
.**
|
||||||
pick,
|
* P3-1:PredictionPanel 不再自绘,改为组合三个子组件:
|
||||||
confidence,
|
* OutcomePanel(比分/胜平负/成本) / AgentsPanel(专家意见) / ReasoningPanel(终裁/降级)。
|
||||||
|
* 渲染输出与拆分前完全一致(仅降级警示 + 报头 + 元信息仍在此处)。
|
||||||
|
*/
|
||||||
|
function PredictionPanel({
|
||||||
|
prediction,
|
||||||
|
match,
|
||||||
|
embedded = false,
|
||||||
}: {
|
}: {
|
||||||
pick: string | null
|
prediction: Prediction
|
||||||
confidence: number | null
|
match: Match
|
||||||
|
/** 弹窗嵌入模式:弹窗已提供报头,这里省略自带版头 */
|
||||||
|
embedded?: boolean
|
||||||
}) {
|
}) {
|
||||||
const options = ['1', 'X', '2'] as const
|
const homeName = match.home_team_zh || match.home_team
|
||||||
|
const awayName = match.away_team_zh || match.away_team
|
||||||
|
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
|
||||||
|
const reports = prediction.agent_outputs ?? []
|
||||||
|
const okReports = reports.filter(r => r.status === 'ok')
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div>
|
<article className={embedded ? 'bg-paper-50' : 'border border-ink-900 bg-paper-50'}>
|
||||||
<div className="flex items-baseline justify-center gap-6 sm:gap-10">
|
{!embedded && (
|
||||||
{options.map(o => {
|
<div className="flex flex-wrap items-baseline justify-between gap-2 border-b border-ink-900 bg-paper-100 px-4 py-2.5 sm:px-5">
|
||||||
const on = pick === o
|
<h3 className="flex flex-wrap items-center gap-1.5 font-serif text-sm font-bold text-ink-900">
|
||||||
return (
|
预测版 ·
|
||||||
<div key={o} className="flex flex-col items-center gap-1">
|
<TeamSideTag side="home" />
|
||||||
<span className={`flex items-center gap-1.5 text-sm ${on ? 'font-semibold text-press' : 'text-ink-400'}`}>
|
{homeName}
|
||||||
{on && <span className="inline-block h-2 w-2 bg-press" aria-hidden="true" />}
|
<span>对</span>
|
||||||
{OUTCOME_LABEL[o]}
|
<TeamSideTag side="away" />
|
||||||
</span>
|
{awayName}
|
||||||
{on && confidence !== null && (
|
</h3>
|
||||||
<span className="text-2xs tabular-nums text-ink-500">
|
<span className="text-2xs tabular-nums text-ink-500">
|
||||||
置信 {Math.round(confidence * 100)}%
|
{prediction.provider} / {prediction.model}
|
||||||
</span>
|
{prediction.latency_ms !== null && ` · ${(prediction.latency_ms / 1000).toFixed(1)}s`}
|
||||||
)}
|
</span>
|
||||||
</div>
|
|
||||||
)
|
|
||||||
})}
|
|
||||||
</div>
|
</div>
|
||||||
{pick && confidence !== null && (
|
|
||||||
<div className="mx-auto mt-3 max-w-xs">
|
|
||||||
<Meter value={confidence} />
|
|
||||||
<p className="mt-1 text-center text-2xs text-ink-400">主观置信度,非统计概率</p>
|
|
||||||
</div>
|
|
||||||
)}
|
)}
|
||||||
</div>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 预测成本展示:耗时 + token + 限流余量 */
|
<div className="space-y-7 px-4 py-6 sm:px-5">
|
||||||
function PredictionCost({ prediction }: { prediction: Prediction }) {
|
{degraded && (
|
||||||
const latency = prediction.latency_ms != null ? `${(prediction.latency_ms / 1000).toFixed(1)}s` : null
|
<div className="border-l-2 border-press bg-press-wash/40 px-4 py-3">
|
||||||
const tokens = prediction.prompt_tokens != null || prediction.completion_tokens != null
|
<p className="font-serif text-sm font-bold text-press-dark">
|
||||||
? `${prediction.prompt_tokens ?? '?'}/${prediction.completion_tokens ?? '?'}`
|
{prediction.status === 'failed' ? '预测失败' : '预测降级(degraded)'}
|
||||||
: null
|
</p>
|
||||||
|
<p className="mt-1.5 whitespace-pre-wrap text-xs leading-relaxed text-ink-600">
|
||||||
|
{prediction.reasoning || '所有专家均无有效数据或调用失败,无法生成可靠比分。'}
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
if (!latency && !tokens && prediction.rate_limit_remaining == null) return null
|
{!degraded && <OutcomePanel prediction={prediction} match={match} />}
|
||||||
|
|
||||||
return (
|
<p className="text-center text-2xs text-ink-500">
|
||||||
<div className="border-t border-ink-200 pt-3 text-2xs text-ink-500">
|
`多专家模式 · ${okReports.length}/${reports.length} 路有效`
|
||||||
<div className="flex flex-wrap items-center justify-center gap-x-4 gap-y-1">
|
{prediction.prompt_version && ` · prompt ${prediction.prompt_version}`}
|
||||||
{latency && (
|
</p>
|
||||||
<span className="inline-flex items-center gap-1">
|
|
||||||
<span aria-hidden="true" className="opacity-60">⏱</span>耗时 {latency}
|
<AgentsPanel prediction={prediction} />
|
||||||
</span>
|
|
||||||
)}
|
<ReasoningPanel prediction={prediction} />
|
||||||
{tokens && (
|
|
||||||
<span className="inline-flex items-center gap-1">
|
|
||||||
<span aria-hidden="true" className="opacity-60">Tok</span>prompt/completion: {tokens}
|
|
||||||
</span>
|
|
||||||
)}
|
|
||||||
{prediction.rate_limit_remaining != null && prediction.rate_limit_remaining <= 3 && (
|
|
||||||
<span className="text-press" title="每分钟最多 10 次预测">
|
|
||||||
剩余配额: {prediction.rate_limit_remaining}/10(分钟)
|
|
||||||
</span>
|
|
||||||
)}
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</article>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
/** 预测过程阶段(按时长模拟;结果到达即跳到完成) */
|
|
||||||
function PredictProgress() {
|
function PredictProgress() {
|
||||||
const [elapsed, setElapsed] = useState(0)
|
const [elapsed, setElapsed] = useState(0)
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
@@ -199,256 +169,6 @@ function PredictProgress() {
|
|||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
const STATUS_BADGE: Record<string, { label: string; cls: string }> = {
|
|
||||||
ok: { label: '正常', cls: 'text-ink-500' },
|
|
||||||
no_data: { label: '无数据', cls: 'text-ink-400' },
|
|
||||||
error: { label: '调用失败', cls: 'text-press' },
|
|
||||||
parse_error: { label: '解析失败', cls: 'text-press' },
|
|
||||||
}
|
|
||||||
|
|
||||||
const SUFFICIENCY_LABEL: Record<string, string> = {
|
|
||||||
high: '充分',
|
|
||||||
medium: '一般',
|
|
||||||
low: '偏少',
|
|
||||||
none: '无',
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 单路专家意见:汉字编号 + 细线行 */
|
|
||||||
function AgentCard({ report: r, no }: { report: AgentReport; no: string }) {
|
|
||||||
const badge = STATUS_BADGE[r.status] ?? { label: r.status, cls: 'text-ink-400' }
|
|
||||||
const inactive = r.status !== 'ok'
|
|
||||||
|
|
||||||
return (
|
|
||||||
<details className="group border-b border-ink-200">
|
|
||||||
<summary className="flex cursor-pointer list-none items-baseline gap-2.5 px-1 py-3">
|
|
||||||
<span className="font-serif text-sm text-ink-400">{no}</span>
|
|
||||||
<span className="text-sm font-medium text-ink-900">{AGENT_LABELS[r.agent] ?? r.agent}</span>
|
|
||||||
<span className={`text-2xs ${badge.cls}`}>{badge.label}</span>
|
|
||||||
|
|
||||||
<span className="ml-auto flex items-baseline gap-3 text-2xs tabular-nums text-ink-500">
|
|
||||||
{r.status === 'ok' && r.subjective_confidence !== null && (
|
|
||||||
<span>信心 {Math.round(r.subjective_confidence * 100)}%</span>
|
|
||||||
)}
|
|
||||||
{r.status === 'ok' && r.probable_score && (
|
|
||||||
<span className="font-serif font-bold text-ink-800">{r.probable_score}</span>
|
|
||||||
)}
|
|
||||||
<svg viewBox="0 0 20 20" className="h-3 w-3 self-center text-ink-300 transition-transform group-open:rotate-90" fill="currentColor" aria-hidden="true">
|
|
||||||
<path d="M7.3 5.3a1 1 0 011.4 0l4 4a1 1 0 010 1.4l-4 4a1 1 0 01-1.4-1.4L10.6 10 7.3 6.7a1 1 0 010-1.4z" />
|
|
||||||
</svg>
|
|
||||||
</span>
|
|
||||||
</summary>
|
|
||||||
|
|
||||||
<div className="space-y-3 px-1 pb-4 pl-7">
|
|
||||||
{/* 无数据 / 失败时给出明确说明,避免用户以为是空白 bug */}
|
|
||||||
{inactive && (
|
|
||||||
<p className="text-xs leading-relaxed text-ink-500">
|
|
||||||
{r.status === 'no_data' && '该维度没有可用数据,已跳过 LLM 分析以节省额度(不影响其他专家)。'}
|
|
||||||
{r.status === 'error' && '该专家调用失败,本次结论未纳入其视角(fail-open 设计,不阻断整体预测)。'}
|
|
||||||
{r.status === 'parse_error' && '模型输出未通过格式校验,该报告已丢弃。'}
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{!inactive && r.home_edge !== null && (
|
|
||||||
<div>
|
|
||||||
<div className="mb-1.5 flex items-baseline justify-between text-2xs">
|
|
||||||
<span className="text-ink-500">主队优势</span>
|
|
||||||
<span className={`font-semibold tabular-nums ${r.home_edge > 0 ? 'text-press' : r.home_edge < 0 ? 'text-ink-700' : 'text-ink-500'}`}>
|
|
||||||
{r.home_edge > 0 ? '+' : ''}{r.home_edge.toFixed(2)}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
<EdgeBar value={r.home_edge} />
|
|
||||||
<div className="mt-1 flex justify-between text-2xs text-ink-400">
|
|
||||||
<span>利客队</span>
|
|
||||||
<span>利主队</span>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{r.analysis && (
|
|
||||||
<p className="font-serif text-sm leading-loose text-ink-700">{r.analysis}</p>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{r.key_evidence.length > 0 && (
|
|
||||||
<ul className="space-y-1.5">
|
|
||||||
{r.key_evidence.map((e, i) => (
|
|
||||||
<li key={i} className="flex gap-2 text-xs leading-relaxed text-ink-600">
|
|
||||||
<span className="flex-shrink-0 text-ink-300" aria-hidden="true">—</span>
|
|
||||||
<span>{e}</span>
|
|
||||||
</li>
|
|
||||||
))}
|
|
||||||
</ul>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{r.exp_home_goals !== null && r.exp_away_goals !== null && (
|
|
||||||
<p className="text-xs text-ink-500">
|
|
||||||
进球期望 <span className="font-serif font-bold tabular-nums text-ink-900">{r.exp_home_goals.toFixed(1)} - {r.exp_away_goals.toFixed(1)}</span>
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{!inactive && (
|
|
||||||
<p className="border-t border-ink-100 pt-2.5 text-2xs text-ink-400">
|
|
||||||
数据充分度 {SUFFICIENCY_LABEL[r.data_sufficiency] ?? r.data_sufficiency}
|
|
||||||
<span className="mx-2 text-ink-200">|</span>
|
|
||||||
<span className="font-mono">{r.model}</span>
|
|
||||||
{r.latency_ms !== null && <span className="ml-2 tabular-nums">{r.latency_ms}ms</span>}
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
</details>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
function PredictionPanel({
|
|
||||||
prediction,
|
|
||||||
match,
|
|
||||||
embedded = false,
|
|
||||||
}: {
|
|
||||||
prediction: Prediction
|
|
||||||
match: Match
|
|
||||||
/** 弹窗嵌入模式:弹窗已提供报头,这里省略自带版头 */
|
|
||||||
embedded?: boolean
|
|
||||||
}) {
|
|
||||||
const homeName = match.home_team_zh || match.home_team
|
|
||||||
const [expertsOpen, setExpertsOpen] = useState(false)
|
|
||||||
const awayName = match.away_team_zh || match.away_team
|
|
||||||
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
|
|
||||||
const reports = prediction.agent_outputs ?? []
|
|
||||||
const okReports = reports.filter(r => r.status === 'ok')
|
|
||||||
|
|
||||||
return (
|
|
||||||
<article className={embedded ? 'bg-paper-50' : 'border border-ink-900 bg-paper-50'}>
|
|
||||||
{!embedded && (
|
|
||||||
<div className="flex flex-wrap items-baseline justify-between gap-2 border-b border-ink-900 bg-paper-100 px-4 py-2.5 sm:px-5">
|
|
||||||
<h3 className="flex flex-wrap items-center gap-1.5 font-serif text-sm font-bold text-ink-900">
|
|
||||||
预测版 ·
|
|
||||||
<TeamSideTag side="home" />
|
|
||||||
{homeName}
|
|
||||||
<span>对</span>
|
|
||||||
<TeamSideTag side="away" />
|
|
||||||
{awayName}
|
|
||||||
</h3>
|
|
||||||
<span className="text-2xs tabular-nums text-ink-500">
|
|
||||||
{prediction.provider} / {prediction.model}
|
|
||||||
{prediction.latency_ms !== null && ` · ${(prediction.latency_ms / 1000).toFixed(1)}s`}
|
|
||||||
</span>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
<div className="space-y-7 px-4 py-6 sm:px-5">
|
|
||||||
{/* ── degraded / failed 态:醒目警示 + 原因,不展示虚假比分 ── */}
|
|
||||||
{degraded && (
|
|
||||||
<div className="border-l-2 border-press bg-press-wash/40 px-4 py-3">
|
|
||||||
<p className="font-serif text-sm font-bold text-press-dark">
|
|
||||||
{prediction.status === 'failed' ? '预测失败' : '预测降级(degraded)'}
|
|
||||||
</p>
|
|
||||||
<p className="mt-1.5 whitespace-pre-wrap text-xs leading-relaxed text-ink-600">
|
|
||||||
{prediction.reasoning || '所有专家均无有效数据或调用失败,无法生成可靠比分。'}
|
|
||||||
</p>
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* ── 主结论(仅 success 展示) ── */}
|
|
||||||
{!degraded && (
|
|
||||||
<>
|
|
||||||
<div className="text-center">
|
|
||||||
<p className="font-serif text-5xl font-bold tabular-nums leading-none text-ink-900 sm:text-6xl">
|
|
||||||
{prediction.pred_home_goals ?? '-'}
|
|
||||||
<span className="mx-3 font-normal text-ink-300">:</span>
|
|
||||||
{prediction.pred_away_goals ?? '-'}
|
|
||||||
</p>
|
|
||||||
<p className="mt-3 text-2xs tracking-[0.5em] text-ink-400">预测比分</p>
|
|
||||||
{prediction.alt_pred_home_goals != null && prediction.alt_pred_away_goals != null && (
|
|
||||||
<p className="mt-2 text-2xs tabular-nums text-ink-400">
|
|
||||||
备选{' '}
|
|
||||||
<span className="font-serif text-sm font-bold tabular-nums text-ink-600">
|
|
||||||
{prediction.alt_pred_home_goals}<span className="mx-0.5 font-normal text-ink-300">:</span>{prediction.alt_pred_away_goals}
|
|
||||||
</span>
|
|
||||||
</p>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<div className="border-y border-ink-200 py-4">
|
|
||||||
<OutcomeLine pick={prediction.pred_1x2} confidence={prediction.subjective_confidence} />
|
|
||||||
</div>
|
|
||||||
</>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* ── 成本信息(耗时 + token + 限流余量) ── */}
|
|
||||||
{!degraded && (
|
|
||||||
<PredictionCost prediction={prediction} />
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* ── 元信息 ── */}
|
|
||||||
<p className="text-center text-2xs text-ink-500">
|
|
||||||
`多专家模式 · ${okReports.length}/${reports.length} 路有效`
|
|
||||||
{prediction.prompt_version && ` · prompt ${prediction.prompt_version}`}
|
|
||||||
</p>
|
|
||||||
|
|
||||||
{/* ── 终裁/降级说明意见 ── */}
|
|
||||||
{prediction.reasoning && degraded && (
|
|
||||||
<section>
|
|
||||||
<h4 className="section-head mb-2">降级原因</h4>
|
|
||||||
<blockquote className="border-l-2 border-press pl-4">
|
|
||||||
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
|
|
||||||
</blockquote>
|
|
||||||
</section>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* ── 专家意见(多模式):可折叠 + 状态摘要 + 权重条形图 ── */}
|
|
||||||
{reports.length > 0 && (
|
|
||||||
<section>
|
|
||||||
<button
|
|
||||||
onClick={() => setExpertsOpen(o => !o)}
|
|
||||||
className="flex w-full items-center justify-between border-b border-ink-200 pb-2 text-left"
|
|
||||||
>
|
|
||||||
<span className="section-head mb-0">五路专家意见({okReports.length}/{reports.length} 路有效)</span>
|
|
||||||
<span className="text-2xs text-ink-400">{expertsOpen ? '收起' : '展开'}</span>
|
|
||||||
</button>
|
|
||||||
|
|
||||||
{/* 权重条形图(仅 success 且有权重时显示) */}
|
|
||||||
{!degraded && prediction.agent_weights && Object.keys(prediction.agent_weights).length > 0 && (
|
|
||||||
<div className="mt-3 space-y-1.5">
|
|
||||||
<span className="text-2xs text-ink-500">终裁权重分布</span>
|
|
||||||
{Object.entries(prediction.agent_weights)
|
|
||||||
.sort((a, b) => b[1] - a[1])
|
|
||||||
.map(([k, v]) => (
|
|
||||||
<div key={k} className="grid grid-cols-[96px_minmax(0,1fr)_40px] items-center gap-2">
|
|
||||||
<span className="truncate text-2xs text-ink-500">{AGENT_LABELS[k] ?? k}</span>
|
|
||||||
<div className="h-1.5 bg-paper-100">
|
|
||||||
<div className="h-full bg-press" style={{ width: `${Math.round(v * 100)}%` }} />
|
|
||||||
</div>
|
|
||||||
<span className="text-right text-2xs tabular-nums text-ink-500">{Math.round(v * 100)}%</span>
|
|
||||||
</div>
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{expertsOpen && (
|
|
||||||
<div className="mt-2">
|
|
||||||
{reports.map((r, i) => (
|
|
||||||
<AgentCard key={r.agent} report={r} no={CN_NUM[i] ?? String(i + 1)} />
|
|
||||||
))}
|
|
||||||
</div>
|
|
||||||
)}
|
|
||||||
</section>
|
|
||||||
)}
|
|
||||||
|
|
||||||
{/* ── 终裁意见(success) ── */}
|
|
||||||
{prediction.reasoning && !degraded && (
|
|
||||||
<section>
|
|
||||||
<h4 className="section-head mb-3">终裁意见</h4>
|
|
||||||
<blockquote className="border-l-2 border-press pl-4">
|
|
||||||
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
|
|
||||||
</blockquote>
|
|
||||||
</section>
|
|
||||||
)}
|
|
||||||
</div>
|
|
||||||
</article>
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
/** 预测弹窗:进行中显示过程可视化,完成后显示预测版,失败显示原因 */
|
|
||||||
export function PredictModal({
|
export function PredictModal({
|
||||||
match,
|
match,
|
||||||
predicting,
|
predicting,
|
||||||
|
|||||||
@@ -0,0 +1,126 @@
|
|||||||
|
/**
|
||||||
|
* OutcomePanel: 预测主结论 —— 比分 / 胜平负 / 置信度 / 成本。
|
||||||
|
*
|
||||||
|
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
|
||||||
|
*/
|
||||||
|
import TeamSideTag from '../../../../components/TeamSideTag'
|
||||||
|
import type { Match, Prediction } from '../../types'
|
||||||
|
import { OUTCOME_LABEL } from '../../types'
|
||||||
|
|
||||||
|
/** 置信度细线:0~1 数值的低调可视化 */
|
||||||
|
function Meter({ value }: { value: number }) {
|
||||||
|
const pct = Math.max(0, Math.min(100, Math.round(value * 100)))
|
||||||
|
return (
|
||||||
|
<div className="h-px w-full bg-ink-200" role="presentation">
|
||||||
|
<div className="h-px bg-press transition-[width] duration-500" style={{ width: `${pct}%` }} />
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 胜平负一行文字:选中的红字加方块标记,未选中的退灰 */
|
||||||
|
function OutcomeLine({
|
||||||
|
pick,
|
||||||
|
confidence,
|
||||||
|
}: {
|
||||||
|
pick: string | null
|
||||||
|
confidence: number | null
|
||||||
|
}) {
|
||||||
|
const options = ['1', 'X', '2'] as const
|
||||||
|
return (
|
||||||
|
<div>
|
||||||
|
<div className="flex items-baseline justify-center gap-6 sm:gap-10">
|
||||||
|
{options.map(o => {
|
||||||
|
const on = pick === o
|
||||||
|
return (
|
||||||
|
<div key={o} className="flex flex-col items-center gap-1">
|
||||||
|
<span className={`flex items-center gap-1.5 text-sm ${on ? 'font-semibold text-press' : 'text-ink-400'}`}>
|
||||||
|
{on && <span className="inline-block h-2 w-2 bg-press" aria-hidden="true" />}
|
||||||
|
{OUTCOME_LABEL[o]}
|
||||||
|
</span>
|
||||||
|
{on && confidence !== null && (
|
||||||
|
<span className="text-2xs tabular-nums text-ink-500">
|
||||||
|
置信 {Math.round(confidence * 100)}%
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
})}
|
||||||
|
</div>
|
||||||
|
{pick && confidence !== null && (
|
||||||
|
<div className="mx-auto mt-3 max-w-xs">
|
||||||
|
<Meter value={confidence} />
|
||||||
|
<p className="mt-1 text-center text-2xs text-ink-400">主观置信度,非统计概率</p>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 预测成本展示:耗时 + token + 限流余量 */
|
||||||
|
function PredictionCost({ prediction }: { prediction: Prediction }) {
|
||||||
|
const latency = prediction.latency_ms != null ? `${(prediction.latency_ms / 1000).toFixed(1)}s` : null
|
||||||
|
const tokens = prediction.prompt_tokens != null || prediction.completion_tokens != null
|
||||||
|
? `${prediction.prompt_tokens ?? '?'}/${prediction.completion_tokens ?? '?'}`
|
||||||
|
: null
|
||||||
|
|
||||||
|
if (!latency && !tokens && prediction.rate_limit_remaining == null) return null
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="border-t border-ink-200 pt-3 text-2xs text-ink-500">
|
||||||
|
<div className="flex flex-wrap items-center justify-center gap-x-4 gap-y-1">
|
||||||
|
{latency && (
|
||||||
|
<span className="inline-flex items-center gap-1">
|
||||||
|
<span aria-hidden="true" className="opacity-60">⏱</span>耗时 {latency}
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
{tokens && (
|
||||||
|
<span className="inline-flex items-center gap-1">
|
||||||
|
<span aria-hidden="true" className="opacity-60">Tok</span>prompt/completion: {tokens}
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
{prediction.rate_limit_remaining != null && prediction.rate_limit_remaining <= 3 && (
|
||||||
|
<span className="text-press" title="每分钟最多 10 次预测">
|
||||||
|
剩余配额: {prediction.rate_limit_remaining}/10(分钟)
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export function OutcomePanel({ prediction, match }: { prediction: Prediction; match: Match }) {
|
||||||
|
const homeName = match.home_team_zh || match.home_team
|
||||||
|
const awayName = match.away_team_zh || match.away_team
|
||||||
|
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
|
||||||
|
|
||||||
|
return (
|
||||||
|
<>
|
||||||
|
{!degraded && (
|
||||||
|
<div className="text-center">
|
||||||
|
<p className="font-serif text-5xl font-bold tabular-nums leading-none text-ink-900 sm:text-6xl">
|
||||||
|
{prediction.pred_home_goals ?? '-'}
|
||||||
|
<span className="mx-3 font-normal text-ink-300">:</span>
|
||||||
|
{prediction.pred_away_goals ?? '-'}
|
||||||
|
</p>
|
||||||
|
<p className="mt-3 text-2xs tracking-[0.5em] text-ink-400">预测比分</p>
|
||||||
|
{prediction.alt_pred_home_goals != null && prediction.alt_pred_away_goals != null && (
|
||||||
|
<p className="mt-2 text-2xs tabular-nums text-ink-400">
|
||||||
|
备选{' '}
|
||||||
|
<span className="font-serif text-sm font-bold tabular-nums text-ink-600">
|
||||||
|
{prediction.alt_pred_home_goals}<span className="mx-0.5 font-normal text-ink-300">:</span>{prediction.alt_pred_away_goals}
|
||||||
|
</span>
|
||||||
|
</p>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{!degraded && (
|
||||||
|
<div className="border-y border-ink-200 py-4">
|
||||||
|
<OutcomeLine pick={prediction.pred_1x2} confidence={prediction.subjective_confidence} />
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
{!degraded && <PredictionCost prediction={prediction} />}
|
||||||
|
</>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,34 @@
|
|||||||
|
/**
|
||||||
|
* ReasoningPanel: 终裁意见 / 降级原因 —— 预测的文本解释。
|
||||||
|
*
|
||||||
|
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
|
||||||
|
*/
|
||||||
|
import type { Prediction } from '../../types'
|
||||||
|
|
||||||
|
export function ReasoningPanel({ prediction }: { prediction: Prediction }) {
|
||||||
|
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
|
||||||
|
|
||||||
|
if (!prediction.reasoning) return null
|
||||||
|
|
||||||
|
// 降级态:reasoning 展示为「降级原因」
|
||||||
|
if (degraded) {
|
||||||
|
return (
|
||||||
|
<section>
|
||||||
|
<h4 className="section-head mb-2">降级原因</h4>
|
||||||
|
<blockquote className="border-l-2 border-press pl-4">
|
||||||
|
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
|
||||||
|
</blockquote>
|
||||||
|
</section>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// 成功态:reasoning 展示为「终裁意见」
|
||||||
|
return (
|
||||||
|
<section>
|
||||||
|
<h4 className="section-head mb-3">终裁意见</h4>
|
||||||
|
<blockquote className="border-l-2 border-press pl-4">
|
||||||
|
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
|
||||||
|
</blockquote>
|
||||||
|
</section>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
/**
|
||||||
|
* 公开站联赛列表(P1-3):优先请求 GET /api/v1/leagues,
|
||||||
|
* 失败或返回空数组则回退本地五大联赛常量(LEAGUES)。
|
||||||
|
*
|
||||||
|
* 显示名规则:常量里已有的 code 沿用中文标签(保持现有 UI 语言不变),
|
||||||
|
* 新增联赛用 API 返回的 name;排序按常量顺序优先、新联赛按 API 返回序追加。
|
||||||
|
* API 仅返回 {id, code, name, country},无敏感配置字段。
|
||||||
|
*/
|
||||||
|
import { useEffect, useState } from 'react'
|
||||||
|
import { fetchLeagues } from '../../../admin/dal'
|
||||||
|
import { LEAGUES } from '../types'
|
||||||
|
|
||||||
|
export function useLeagues(): { code: string; name: string }[] {
|
||||||
|
const [leagues, setLeagues] = useState(LEAGUES)
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
let alive = true
|
||||||
|
;(async () => {
|
||||||
|
// dal.fetchLeagues 已兜底:网络/权限异常时返回 []
|
||||||
|
const rows = await fetchLeagues()
|
||||||
|
if (!alive || rows.length === 0) return
|
||||||
|
const zhName = new Map(LEAGUES.map(l => [l.code, l.name] as const))
|
||||||
|
const rank = new Map(LEAGUES.map((l, i) => [l.code, i] as const))
|
||||||
|
const merged = rows
|
||||||
|
.slice()
|
||||||
|
.sort(
|
||||||
|
(a, b) => (rank.get(a.code) ?? LEAGUES.length) - (rank.get(b.code) ?? LEAGUES.length),
|
||||||
|
)
|
||||||
|
.map(l => ({ code: l.code, name: zhName.get(l.code) ?? l.name }))
|
||||||
|
setLeagues(merged)
|
||||||
|
})()
|
||||||
|
return () => {
|
||||||
|
alive = false
|
||||||
|
}
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
return leagues
|
||||||
|
}
|
||||||
@@ -2,6 +2,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
import logging
|
||||||
|
import os
|
||||||
from collections.abc import AsyncIterator
|
from collections.abc import AsyncIterator
|
||||||
from contextlib import asynccontextmanager
|
from contextlib import asynccontextmanager
|
||||||
|
|
||||||
@@ -40,6 +41,24 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
|
|||||||
"多 worker 部署请将限流前置到 Nginx/网关,或以单 worker 运行"
|
"多 worker 部署请将限流前置到 Nginx/网关,或以单 worker 运行"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
# P3-3:STRICT_SINGLE_WORKER 启动期强制校验,拒绝多 worker 静默配额漂移。
|
||||||
|
# uvicorn 通过 --workers 传入;此处以环境变量 UVICORN_WORKERS 或启动参数判定。
|
||||||
|
# 为避免耦合 uvicorn 内部,仅校验一个显式传入的标记:当 STRICT_SINGLE_WORKER=True 时,
|
||||||
|
# 要求环境变量 UVICORN_WORKERS 不为空且 <=1,否则拒绝启动。
|
||||||
|
if settings.STRICT_SINGLE_WORKER:
|
||||||
|
workers = os.environ.get("UVICORN_WORKERS", "1")
|
||||||
|
try:
|
||||||
|
n_workers = int(workers)
|
||||||
|
except ValueError:
|
||||||
|
n_workers = 1
|
||||||
|
if n_workers > 1:
|
||||||
|
raise RuntimeError(
|
||||||
|
f"STRICT_SINGLE_WORKER=True 但以 {n_workers} worker 启动会被拒绝 "
|
||||||
|
f"(应用内限流/KeyRing 多 worker 下各自独立计数,配额放大 {n_workers} 倍)。"
|
||||||
|
f"请前置 Nginx/网关全局限流后再启用多 worker,或保持单 worker。"
|
||||||
|
)
|
||||||
|
logger.info("STRICT_SINGLE_WORKER=True:已确认单 worker 启动,限流配额不会漂移")
|
||||||
|
|
||||||
# 注册默认定时任务(如果数据库中没有)
|
# 注册默认定时任务(如果数据库中没有)
|
||||||
from src.db.base import AsyncSessionLocal
|
from src.db.base import AsyncSessionLocal
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
|
|||||||
@@ -0,0 +1,88 @@
|
|||||||
|
"""后台管理:配置项 CRUD(settings)与运行日志查询。
|
||||||
|
|
||||||
|
所有接口需管理员鉴权(require_admin)。路由前缀 /api/v1/admin。
|
||||||
|
配置项白名单见 src/core/runtime_config.py SETTING_DEFS,之外的 key 一律拒绝。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||||
|
from pydantic import BaseModel
|
||||||
|
|
||||||
|
from src.api.deps import require_admin
|
||||||
|
from src.core.log_buffer import get_entries
|
||||||
|
from src.core.runtime_config import (
|
||||||
|
SETTING_DEFS,
|
||||||
|
clear_runtime_value,
|
||||||
|
get_setting_origin,
|
||||||
|
mask_value,
|
||||||
|
set_runtime_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
|
||||||
|
|
||||||
|
|
||||||
|
class SettingUpdateIn(BaseModel):
|
||||||
|
value: str
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/settings")
|
||||||
|
async def list_settings():
|
||||||
|
"""全部可配置项(脱敏),供后台各配置页渲染。"""
|
||||||
|
out = []
|
||||||
|
for key, defn in SETTING_DEFS.items():
|
||||||
|
origin, value = await get_setting_origin(key)
|
||||||
|
out.append(
|
||||||
|
{
|
||||||
|
"key": key,
|
||||||
|
"label": defn.label,
|
||||||
|
"description": defn.description,
|
||||||
|
"sensitive": defn.sensitive,
|
||||||
|
"configured": origin != "none",
|
||||||
|
"masked": mask_value(value, defn.sensitive),
|
||||||
|
"origin": origin,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/logs")
|
||||||
|
async def read_logs(
|
||||||
|
level: str | None = Query(None, description="最低级别: DEBUG/INFO/WARNING/ERROR"),
|
||||||
|
keyword: str | None = Query(None, description="消息或 logger 关键字"),
|
||||||
|
limit: int = Query(200, ge=1, le=1000),
|
||||||
|
):
|
||||||
|
"""查询应用运行日志(内存环形缓冲,最新在前;进程重启后清零)。"""
|
||||||
|
entries = get_entries(level, keyword, limit)
|
||||||
|
return {"entries": entries, "count": len(entries)}
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/settings/{key}")
|
||||||
|
async def update_setting(key: str, body: SettingUpdateIn):
|
||||||
|
"""更新配置项(写入 app_settings 覆盖 .env)。传空值请改用 DELETE。"""
|
||||||
|
if key not in SETTING_DEFS:
|
||||||
|
raise HTTPException(404, f"不支持的配置项: {key}")
|
||||||
|
value = body.value.strip()
|
||||||
|
if not value:
|
||||||
|
raise HTTPException(400, "值不能为空;如需回落 .env 请调用清除接口")
|
||||||
|
await set_runtime_value(key, value)
|
||||||
|
defn = SETTING_DEFS[key]
|
||||||
|
return {"key": key, "masked": mask_value(value, defn.sensitive), "origin": "db"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.delete("/settings/{key}")
|
||||||
|
async def clear_setting(key: str):
|
||||||
|
"""清除 DB 覆盖值,回落 .env 默认。"""
|
||||||
|
if key not in SETTING_DEFS:
|
||||||
|
raise HTTPException(404, f"不支持的配置项: {key}")
|
||||||
|
await clear_runtime_value(key)
|
||||||
|
origin, value = await get_setting_origin(key)
|
||||||
|
defn = SETTING_DEFS[key]
|
||||||
|
return {
|
||||||
|
"key": key,
|
||||||
|
"masked": mask_value(value, defn.sensitive),
|
||||||
|
"origin": origin,
|
||||||
|
}
|
||||||
@@ -0,0 +1,228 @@
|
|||||||
|
"""后台管理:数据源列表/连通性测试、KeyRing 状态、采集健康概览。
|
||||||
|
|
||||||
|
所有接口需管理员鉴权(require_admin)。路由前缀 /api/v1/admin。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import time
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, HTTPException
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
|
||||||
|
from src.api.deps import require_admin
|
||||||
|
from src.core.config import settings
|
||||||
|
from src.core.http_client import get_client
|
||||||
|
from src.core.runtime_config import (
|
||||||
|
SETTING_DEFS,
|
||||||
|
get_runtime_value,
|
||||||
|
get_setting_origin,
|
||||||
|
mask_value,
|
||||||
|
)
|
||||||
|
from src.db.base import AsyncSession, get_db_read
|
||||||
|
from src.db.models import Match, MatchStats, Standing
|
||||||
|
from src.data.key_ring import get_key_ring
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
|
||||||
|
|
||||||
|
# ── 数据源元数据(bzzoiro 单一数据源) ────────────────────────────
|
||||||
|
|
||||||
|
_SOURCES: list[dict] = [
|
||||||
|
{
|
||||||
|
"name": "bzzoiro",
|
||||||
|
"label": "Bzzoiro",
|
||||||
|
"description": "唯一数据源:赛程比分 + 积分榜 + 比赛详细统计(xG/射门/控球等)",
|
||||||
|
"setting_keys": ["BZZOIRO_KEY", "BZZOIRO_BASE"],
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
|
||||||
|
async def _last_ingestion(db: AsyncSession, source: str) -> datetime | None:
|
||||||
|
"""源最近一次采集时间(取自数据血缘字段,无记录返回 None)。"""
|
||||||
|
return (
|
||||||
|
await db.execute(
|
||||||
|
select(func.max(MatchStats.retrieved_at)).where(MatchStats.source == source)
|
||||||
|
)
|
||||||
|
).scalar()
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/datasources")
|
||||||
|
async def list_datasources(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""数据源列表:各配置项的脱敏值、来源(db/env/none)与最近采集时间。"""
|
||||||
|
result = []
|
||||||
|
for src in _SOURCES:
|
||||||
|
settings_out = []
|
||||||
|
for key in src["setting_keys"]:
|
||||||
|
origin, value = await get_setting_origin(key)
|
||||||
|
defn = SETTING_DEFS[key]
|
||||||
|
settings_out.append(
|
||||||
|
{
|
||||||
|
"key": key,
|
||||||
|
"label": defn.label,
|
||||||
|
"description": defn.description,
|
||||||
|
"sensitive": defn.sensitive,
|
||||||
|
"configured": origin != "none",
|
||||||
|
"masked": mask_value(value, defn.sensitive),
|
||||||
|
"origin": origin,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
key_configured = all(s["configured"] for s in settings_out) if settings_out else True
|
||||||
|
last = await _last_ingestion(db, src["name"])
|
||||||
|
result.append(
|
||||||
|
{
|
||||||
|
"name": src["name"],
|
||||||
|
"label": src["label"],
|
||||||
|
"description": src["description"],
|
||||||
|
"key_configured": key_configured,
|
||||||
|
"last_ingestion": last.isoformat() if last else None,
|
||||||
|
"settings": settings_out,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
# ── 连通性测试 ──────────────────────────────────────────────────
|
||||||
|
|
||||||
|
_TEST_TIMEOUT = 15
|
||||||
|
|
||||||
|
|
||||||
|
async def _probe(url: str, headers: dict | None = None, params: dict | None = None) -> dict:
|
||||||
|
"""单次 HTTP 探测,返回 (ok, status, latency_ms, detail)。不重试。"""
|
||||||
|
client = get_client()
|
||||||
|
start = time.monotonic()
|
||||||
|
try:
|
||||||
|
resp = await client.get(url, headers=headers, params=params, timeout=_TEST_TIMEOUT)
|
||||||
|
except Exception as e:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"status": None,
|
||||||
|
"latency_ms": int((time.monotonic() - start) * 1000),
|
||||||
|
"detail": f"无法连接: {e}",
|
||||||
|
}
|
||||||
|
latency = int((time.monotonic() - start) * 1000)
|
||||||
|
status = resp.status_code
|
||||||
|
if status == 200:
|
||||||
|
detail = "连接成功"
|
||||||
|
elif status in (401, 403):
|
||||||
|
detail = "服务可达,但密钥无效或无权限"
|
||||||
|
else:
|
||||||
|
detail = f"服务返回 HTTP {status}"
|
||||||
|
return {"ok": status == 200, "status": status, "latency_ms": latency, "detail": detail}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/datasources/{name}/test")
|
||||||
|
async def test_datasource(name: str):
|
||||||
|
"""轻量连通性测试:真实请求上游一次,不触发任何入库。"""
|
||||||
|
src = next((s for s in _SOURCES if s["name"] == name), None)
|
||||||
|
if src is None:
|
||||||
|
raise HTTPException(404, f"未知数据源: {name}")
|
||||||
|
|
||||||
|
if name == "bzzoiro":
|
||||||
|
key = await get_runtime_value("BZZOIRO_KEY")
|
||||||
|
if not key:
|
||||||
|
return {"ok": False, "status": None, "latency_ms": 0, "detail": "BZZOIRO_KEY 未配置"}
|
||||||
|
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
||||||
|
today = date.today().isoformat()
|
||||||
|
return await _probe(
|
||||||
|
f"{base}/events/",
|
||||||
|
headers={"Authorization": f"Token {key}", "Accept": "application/json"},
|
||||||
|
params={"date_from": today, "date_to": today},
|
||||||
|
)
|
||||||
|
|
||||||
|
raise HTTPException(404, f"未知数据源: {name}")
|
||||||
|
|
||||||
|
|
||||||
|
# ── 数据源健康/最近采集状态(只读,不触发采集) ──────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/ingest/status")
|
||||||
|
async def ingest_status(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""数据源采集健康概览(bzzoiro 单源;只读,不触发任何采集)。"""
|
||||||
|
bzzoiro_key = await get_runtime_value("BZZOIRO_KEY")
|
||||||
|
bzzoiro_base = await get_runtime_value("BZZOIRO_BASE")
|
||||||
|
|
||||||
|
# 比赛覆盖
|
||||||
|
match_row = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("cnt"),
|
||||||
|
func.max(Match.match_date).label("latest_match_date"),
|
||||||
|
func.max(Match.created_at).label("latest_row_at"),
|
||||||
|
).where(Match.match_status == "finished")
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
# 统计覆盖(精确 retrieved_at)
|
||||||
|
stats_row = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("cnt"),
|
||||||
|
func.max(MatchStats.retrieved_at).label("latest_retrieved"),
|
||||||
|
).where(MatchStats.source == "bzzoiro")
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
# 积分榜覆盖
|
||||||
|
standings_row = (
|
||||||
|
await db.execute(select(func.count()).select_from(Standing))
|
||||||
|
).scalar()
|
||||||
|
|
||||||
|
bzzoiro = {
|
||||||
|
"name": "bzzoiro",
|
||||||
|
"label": "Bzzoiro",
|
||||||
|
"key_configured": bool(bzzoiro_key),
|
||||||
|
"base_url": (bzzoiro_base.rstrip("/") if bzzoiro_base else None) or settings.BZZOIRO_BASE,
|
||||||
|
"reachable": None, # 不主动探测
|
||||||
|
"last_success_at": (stats_row.latest_retrieved or match_row.latest_row_at),
|
||||||
|
"last_success_at_iso": (
|
||||||
|
stats_row.latest_retrieved or match_row.latest_row_at
|
||||||
|
).isoformat() if (stats_row.latest_retrieved or match_row.latest_row_at) else None,
|
||||||
|
"latest_match_date": match_row.latest_match_date.isoformat() if match_row.latest_match_date else None,
|
||||||
|
"recent_count": match_row.cnt or 0,
|
||||||
|
"stats_count": stats_row.cnt or 0,
|
||||||
|
"standings_count": standings_row or 0,
|
||||||
|
"note": "last_success_at 取 match_stats.retrieved_at(统计回填)与 matches.created_at(比赛行)的较大者",
|
||||||
|
"last_failure": _last_failure_log("bzzoiro"),
|
||||||
|
}
|
||||||
|
|
||||||
|
return {"sources": [bzzoiro]}
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/keyring/status")
|
||||||
|
async def keyring_status():
|
||||||
|
"""KeyRing 运行状态:当前使用的 key、冷却状态、轮转信息(供管理后台展示)。"""
|
||||||
|
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
||||||
|
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
||||||
|
ring = get_key_ring(base, raw_keys)
|
||||||
|
st = ring.stats()
|
||||||
|
st["base_url"] = base
|
||||||
|
st["cooldown_seconds"] = ring._cooldown
|
||||||
|
st["has_multiple"] = ring.has_multiple
|
||||||
|
st["active_key"] = ring.active_key
|
||||||
|
return st
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/keyring/cooldown/reset")
|
||||||
|
async def keyring_reset_cooldown():
|
||||||
|
"""手动重置所有 key 的冷却状态(用于紧急恢复)。"""
|
||||||
|
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
||||||
|
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
||||||
|
ring = get_key_ring(base, raw_keys)
|
||||||
|
ring._blocked_until.clear()
|
||||||
|
return {"ok": True, "message": "已重置所有 key 冷却状态", "stats": ring.stats()}
|
||||||
|
|
||||||
|
|
||||||
|
def _last_failure_log(source: str) -> dict | None:
|
||||||
|
"""从系统日志缓冲中查找某数据源的最近一次错误(仅作参考,非专用失败表)。"""
|
||||||
|
from src.core.log_buffer import get_entries
|
||||||
|
entries = get_entries(min_level="ERROR", keyword=source, limit=5)
|
||||||
|
if not entries:
|
||||||
|
return None
|
||||||
|
e = entries[0]
|
||||||
|
return {
|
||||||
|
"at": datetime.fromtimestamp(e["ts"]).isoformat(),
|
||||||
|
"logger": e["logger"],
|
||||||
|
"detail": e["message"][:200],
|
||||||
|
"note": "approx:来自内存日志缓冲,非专用采集失败表;进程重启后清零",
|
||||||
|
}
|
||||||
@@ -0,0 +1,126 @@
|
|||||||
|
"""后台管理:LLM 专家/终裁配置、可用模型探测、连通性测试。
|
||||||
|
|
||||||
|
所有接口需管理员鉴权(require_admin)。路由前缀 /api/v1/admin。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
import time
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
|
||||||
|
from src.api.deps import require_admin
|
||||||
|
from src.core.config import settings
|
||||||
|
from src.core.http_client import get_client
|
||||||
|
from src.core.runtime_config import (
|
||||||
|
AGENT_META,
|
||||||
|
SETTING_DEFS,
|
||||||
|
get_runtime_value,
|
||||||
|
get_setting_origin,
|
||||||
|
mask_value,
|
||||||
|
)
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/llm/agents")
|
||||||
|
async def list_llm_agents():
|
||||||
|
"""各专家/终裁的独立 LLM 配置状态(含当前生效模型的解析结果)。"""
|
||||||
|
out = []
|
||||||
|
for agent in AGENT_META:
|
||||||
|
aid = agent["id"].upper()
|
||||||
|
pfx = f"AGENT_{aid}_"
|
||||||
|
fields = {}
|
||||||
|
for suffix in ("MODEL", "BASE_URL", "API_KEY"):
|
||||||
|
origin, value = await get_setting_origin(f"{pfx}{suffix}")
|
||||||
|
defn = SETTING_DEFS[f"{pfx}{suffix}"]
|
||||||
|
fields[suffix.lower()] = {
|
||||||
|
"configured": origin != "none",
|
||||||
|
"masked": mask_value(value, defn.sensitive),
|
||||||
|
"origin": origin,
|
||||||
|
}
|
||||||
|
# 生效模型 = 覆盖 → 层级默认(专家/终裁 env) → 全局 LLM_MODEL
|
||||||
|
tier_default = (
|
||||||
|
settings.LLM_AGGREGATOR_MODEL if agent["id"] == "aggregator" else settings.LLM_SPECIALIST_MODEL
|
||||||
|
)
|
||||||
|
effective_model = (
|
||||||
|
fields["model"]["masked"]
|
||||||
|
if fields["model"]["configured"]
|
||||||
|
else (tier_default or await get_runtime_value("LLM_MODEL"))
|
||||||
|
)
|
||||||
|
out.append(
|
||||||
|
{
|
||||||
|
"id": agent["id"],
|
||||||
|
"label": agent["label"],
|
||||||
|
"fields": fields,
|
||||||
|
"effective_model": effective_model,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/llm/models")
|
||||||
|
async def list_llm_models():
|
||||||
|
"""探测当前 LLM 服务可用的模型列表(OpenAI 兼容 GET /models)。
|
||||||
|
|
||||||
|
只读探测,不产生费用;配置缺失或服务不可达时返回 ok=false 与原因。
|
||||||
|
"""
|
||||||
|
base_url = (await get_runtime_value("LLM_BASE_URL")).rstrip("/")
|
||||||
|
api_key = await get_runtime_value("LLM_API_KEY")
|
||||||
|
if not base_url or not api_key:
|
||||||
|
return {"ok": False, "models": [], "detail": "LLM_BASE_URL 或 LLM_API_KEY 未配置"}
|
||||||
|
|
||||||
|
client = get_client()
|
||||||
|
start = time.monotonic()
|
||||||
|
try:
|
||||||
|
resp = await client.get(
|
||||||
|
f"{base_url}/models",
|
||||||
|
headers={"Authorization": f"Bearer {api_key}"},
|
||||||
|
timeout=httpx.Timeout(connect=10.0, read=20.0, write=10.0, pool=10.0),
|
||||||
|
)
|
||||||
|
except Exception as e:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"models": [],
|
||||||
|
"latency_ms": int((time.monotonic() - start) * 1000),
|
||||||
|
"detail": f"无法连接 LLM 服务: {e}",
|
||||||
|
}
|
||||||
|
|
||||||
|
latency = int((time.monotonic() - start) * 1000)
|
||||||
|
if resp.status_code in (401, 403):
|
||||||
|
return {"ok": False, "models": [], "latency_ms": latency, "detail": "密钥无效或无权限(HTTP 401/403)"}
|
||||||
|
if resp.status_code != 200:
|
||||||
|
return {"ok": False, "models": [], "latency_ms": latency, "detail": f"服务返回 HTTP {resp.status_code}"}
|
||||||
|
|
||||||
|
try:
|
||||||
|
data = resp.json()
|
||||||
|
except Exception:
|
||||||
|
return {"ok": False, "models": [], "latency_ms": latency, "detail": "响应不是合法 JSON"}
|
||||||
|
|
||||||
|
models: list[str] = []
|
||||||
|
items = data.get("data") if isinstance(data, dict) else None
|
||||||
|
if isinstance(items, list):
|
||||||
|
models = sorted(
|
||||||
|
str(m.get("id")) for m in items if isinstance(m, dict) and m.get("id")
|
||||||
|
)
|
||||||
|
if not models:
|
||||||
|
return {"ok": False, "models": [], "latency_ms": latency, "detail": "服务未返回模型列表"}
|
||||||
|
return {"ok": True, "models": models, "latency_ms": latency, "detail": f"共 {len(models)} 个可用模型"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/llm/ping")
|
||||||
|
async def llm_ping():
|
||||||
|
"""LLM 连通性测试(不依赖比赛)。只发一次 chat 请求验证配置。"""
|
||||||
|
from src.llm.provider import get_default_provider
|
||||||
|
p = await get_default_provider()
|
||||||
|
resp = await p.chat(
|
||||||
|
system="你是测试助手。",
|
||||||
|
user="ping",
|
||||||
|
max_tokens=10,
|
||||||
|
)
|
||||||
|
if resp.error:
|
||||||
|
return {"ok": False, "message": resp.error}
|
||||||
|
return {"ok": True, "message": "LLM 连接正常", "model": p.model}
|
||||||
@@ -0,0 +1,343 @@
|
|||||||
|
"""后台管理:管理区统计、数据完整性分析、数据质量检查。
|
||||||
|
|
||||||
|
所有接口需管理员鉴权(require_admin)。路由前缀 /api/v1/admin。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
from sqlalchemy import func, select
|
||||||
|
|
||||||
|
from src.api.deps import require_admin
|
||||||
|
from src.db.base import AsyncSession, get_db_read
|
||||||
|
from src.db.models import DataQualityCheck, IngestFailure, League, Match, MatchStats, Prediction, Standing
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/stats")
|
||||||
|
async def admin_stats(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""管理区统计(只读):预测次数 + 比赛覆盖。轻量聚合,无 LLM 调用。"""
|
||||||
|
day_ago = datetime.now(timezone.utc) - timedelta(days=1)
|
||||||
|
week_ago = datetime.now(timezone.utc) - timedelta(days=7)
|
||||||
|
r = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("total"),
|
||||||
|
func.count().filter(Prediction.created_at >= day_ago).label("last_24h"),
|
||||||
|
func.count().filter(Prediction.created_at >= week_ago).label("last_7d"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
# F3 修复: 补充真实比赛计数(非 limit=100 近似)
|
||||||
|
match_cnt = (await db.execute(select(func.count()).select_from(Match))).scalar() or 0
|
||||||
|
finished_cnt = (await db.execute(select(func.count()).where(Match.match_status == "finished"))).scalar() or 0
|
||||||
|
stats_cnt = (await db.execute(select(func.count()).select_from(MatchStats))).scalar() or 0
|
||||||
|
standings_cnt = (await db.execute(select(func.count()).select_from(Standing))).scalar() or 0
|
||||||
|
return {
|
||||||
|
"predictions": {"total": r.total, "last_24h": r.last_24h, "last_7d": r.last_7d},
|
||||||
|
"matches": {"total": match_cnt, "finished": finished_cnt},
|
||||||
|
"stats": {"total": stats_cnt},
|
||||||
|
"standings": {"total": standings_cnt},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── 数据完整性分析(可视化数据源) ────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/data-completeness")
|
||||||
|
async def data_completeness(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""按联赛统计数据完整性:比赛覆盖、字段覆盖、积分榜覆盖。
|
||||||
|
|
||||||
|
前端「数据完整性」页据此渲染,回答三个问题:
|
||||||
|
1. 数据是否齐全(各联赛比赛/统计/积分榜量级)
|
||||||
|
2. 字段是否齐全(每张统计表各字段非空率)
|
||||||
|
3. 覆盖是否新鲜(最近一场/最近一次采集)
|
||||||
|
"""
|
||||||
|
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_NAMES, LEAGUE_COUNTRIES
|
||||||
|
|
||||||
|
out_leagues: list[dict] = []
|
||||||
|
for code, bzz_id in BZZOIRO_LEAGUE_IDS.items():
|
||||||
|
# 比赛覆盖
|
||||||
|
m = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("total"),
|
||||||
|
func.count().filter(Match.match_status == "finished").label("finished"),
|
||||||
|
func.count().filter(Match.match_status == "scheduled").label("scheduled"),
|
||||||
|
func.count().filter(Match.source_event_id.is_not(None)).label("with_source_id"),
|
||||||
|
func.max(Match.match_date).label("latest_match"),
|
||||||
|
func.min(Match.match_date).label("earliest_match"),
|
||||||
|
)
|
||||||
|
.select_from(Match)
|
||||||
|
.join(League, League.id == Match.league_id)
|
||||||
|
.where(League.code == code)
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
# 统计字段覆盖(联表 matches)
|
||||||
|
s = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("rows"),
|
||||||
|
func.count(MatchStats.home_xg).label("xg"),
|
||||||
|
func.count(MatchStats.home_shots).label("shots"),
|
||||||
|
func.count(MatchStats.home_possession).label("possession"),
|
||||||
|
func.count(MatchStats.home_corners).label("corners"),
|
||||||
|
func.count(MatchStats.home_fouls).label("fouls"),
|
||||||
|
func.count(MatchStats.home_big_chances).label("big_chances"),
|
||||||
|
func.count(MatchStats.home_yellow_cards).label("cards"),
|
||||||
|
)
|
||||||
|
.select_from(MatchStats)
|
||||||
|
.join(Match, Match.id == MatchStats.match_id)
|
||||||
|
.join(League, League.id == Match.league_id)
|
||||||
|
.where(League.code == code)
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
# 积分榜覆盖
|
||||||
|
st = (
|
||||||
|
await db.execute(
|
||||||
|
select(
|
||||||
|
func.count().label("rows"),
|
||||||
|
func.max(Standing.retrieved_at).label("latest_retrieved"),
|
||||||
|
)
|
||||||
|
.select_from(Standing)
|
||||||
|
.join(League, League.id == Standing.league_id)
|
||||||
|
.where(League.code == code)
|
||||||
|
)
|
||||||
|
).one()
|
||||||
|
|
||||||
|
stats_rows = s.rows or 0
|
||||||
|
pct = lambda n: round(n / stats_rows * 100, 1) if stats_rows else 0.0 # noqa: E731
|
||||||
|
out_leagues.append(
|
||||||
|
{
|
||||||
|
"code": code,
|
||||||
|
"name": LEAGUE_NAMES.get(code, code),
|
||||||
|
"country": LEAGUE_COUNTRIES.get(code),
|
||||||
|
"matches": {
|
||||||
|
"total": m.total or 0,
|
||||||
|
"finished": m.finished or 0,
|
||||||
|
"scheduled": m.scheduled or 0,
|
||||||
|
"with_source_id": m.with_source_id or 0,
|
||||||
|
"earliest_match": m.earliest_match.isoformat() if m.earliest_match else None,
|
||||||
|
"latest_match": m.latest_match.isoformat() if m.latest_match else None,
|
||||||
|
},
|
||||||
|
"stats": {
|
||||||
|
"rows": stats_rows,
|
||||||
|
"fields": {
|
||||||
|
"xg": {"count": s.xg or 0, "pct": pct(s.xg or 0)},
|
||||||
|
"shots": {"count": s.shots or 0, "pct": pct(s.shots or 0)},
|
||||||
|
"possession": {"count": s.possession or 0, "pct": pct(s.possession or 0)},
|
||||||
|
"corners": {"count": s.corners or 0, "pct": pct(s.corners or 0)},
|
||||||
|
"fouls": {"count": s.fouls or 0, "pct": pct(s.fouls or 0)},
|
||||||
|
"big_chances": {"count": s.big_chances or 0, "pct": pct(s.big_chances or 0)},
|
||||||
|
"cards": {"count": s.cards or 0, "pct": pct(s.cards or 0)},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"standings": {
|
||||||
|
"rows": st.rows or 0,
|
||||||
|
"latest_retrieved": st.latest_retrieved.isoformat() if st.latest_retrieved else None,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
# 整体健康信号
|
||||||
|
total_finished = sum(l["matches"]["finished"] for l in out_leagues)
|
||||||
|
total_stats = sum(l["stats"]["rows"] for l in out_leagues)
|
||||||
|
stats_coverage = round(total_stats / total_finished * 100, 1) if total_finished else 0.0
|
||||||
|
issues: list[str] = []
|
||||||
|
for l in out_leagues:
|
||||||
|
if l["matches"]["finished"] == 0:
|
||||||
|
issues.append(f"{l['name']}: 无已完赛比赛,请先运行「比赛数据」采集")
|
||||||
|
elif l["stats"]["rows"] == 0:
|
||||||
|
issues.append(f"{l['name']}: 已完赛 {l['matches']['finished']} 场但无统计回填,请运行「统计回填」采集")
|
||||||
|
elif stats_coverage < 80:
|
||||||
|
issues.append(f"{l['name']}: 统计覆盖率仅 {stats_coverage}%,建议增量回填")
|
||||||
|
if l["standings"]["rows"] == 0:
|
||||||
|
issues.append(f"{l['name']}: 无积分榜数据,请运行「积分榜」采集")
|
||||||
|
if not issues:
|
||||||
|
issues.append("各联赛数据完整度良好")
|
||||||
|
|
||||||
|
return {
|
||||||
|
"generated_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
"leagues": out_leagues,
|
||||||
|
"totals": {
|
||||||
|
"finished_matches": total_finished,
|
||||||
|
"stats_rows": total_stats,
|
||||||
|
"stats_coverage_pct": stats_coverage,
|
||||||
|
},
|
||||||
|
"issues": issues,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ── 数据质量检查 API ────────────────────────────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/data-quality")
|
||||||
|
async def data_quality_checks(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""数据质量检查结果(只读)。"""
|
||||||
|
# 最近的失败记录
|
||||||
|
failures = (
|
||||||
|
await db.execute(
|
||||||
|
select(IngestFailure)
|
||||||
|
.where(IngestFailure.status.in_(["pending", "retrying"]))
|
||||||
|
.order_by(IngestFailure.created_at.desc())
|
||||||
|
.limit(20)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
# 最近的质量检查
|
||||||
|
checks = (
|
||||||
|
await db.execute(
|
||||||
|
select(DataQualityCheck)
|
||||||
|
.order_by(DataQualityCheck.checked_at.desc())
|
||||||
|
.limit(20)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
return {
|
||||||
|
"failures": [
|
||||||
|
{
|
||||||
|
"id": f.id,
|
||||||
|
"source": f.source_system,
|
||||||
|
"entity_type": f.entity_type,
|
||||||
|
"source_record_id": f.source_record_id,
|
||||||
|
"error_type": f.error_type,
|
||||||
|
"error_detail": f.error_detail,
|
||||||
|
"retry_count": f.retry_count,
|
||||||
|
"status": f.status,
|
||||||
|
"created_at": f.created_at.isoformat() if f.created_at else None,
|
||||||
|
}
|
||||||
|
for f in failures
|
||||||
|
],
|
||||||
|
"checks": [
|
||||||
|
{
|
||||||
|
"id": c.id,
|
||||||
|
"check_name": c.check_name,
|
||||||
|
"entity_type": c.entity_type,
|
||||||
|
"passed": c.passed,
|
||||||
|
"severity": c.severity,
|
||||||
|
"detail": c.detail,
|
||||||
|
"checked_at": c.checked_at.isoformat() if c.checked_at else None,
|
||||||
|
}
|
||||||
|
for c in checks
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/data-quality/run")
|
||||||
|
async def run_data_quality_check(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""手动触发一次数据质量检查。"""
|
||||||
|
checks = []
|
||||||
|
|
||||||
|
# 检查1: 已完赛但无统计的比赛
|
||||||
|
finished_no_stats = (
|
||||||
|
await db.execute(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(Match)
|
||||||
|
.outerjoin(MatchStats, Match.id == MatchStats.match_id)
|
||||||
|
.where(Match.match_status == "finished")
|
||||||
|
.where(MatchStats.id.is_(None))
|
||||||
|
)
|
||||||
|
).scalar() or 0
|
||||||
|
|
||||||
|
checks.append(DataQualityCheck(
|
||||||
|
check_name="finished_without_stats",
|
||||||
|
entity_type="match",
|
||||||
|
actual_value=float(finished_no_stats),
|
||||||
|
passed=finished_no_stats == 0,
|
||||||
|
severity="warning" if finished_no_stats > 0 else "info",
|
||||||
|
detail={"message": f"{finished_no_stats} 场已完赛比赛缺少统计数据"},
|
||||||
|
))
|
||||||
|
|
||||||
|
# 检查2: 积分榜缺失的联赛
|
||||||
|
leagues_without_standings = (
|
||||||
|
await db.execute(
|
||||||
|
select(func.count())
|
||||||
|
.select_from(League)
|
||||||
|
.outerjoin(Standing, League.id == Standing.league_id)
|
||||||
|
.where(Standing.id.is_(None))
|
||||||
|
)
|
||||||
|
).scalar() or 0
|
||||||
|
|
||||||
|
checks.append(DataQualityCheck(
|
||||||
|
check_name="league_without_standings",
|
||||||
|
entity_type="league",
|
||||||
|
actual_value=float(leagues_without_standings),
|
||||||
|
passed=leagues_without_standings == 0,
|
||||||
|
severity="warning" if leagues_without_standings > 0 else "info",
|
||||||
|
detail={"message": f"{leagues_without_standings} 个联赛缺少积分榜"},
|
||||||
|
))
|
||||||
|
|
||||||
|
for c in checks:
|
||||||
|
db.add(c)
|
||||||
|
await db.commit()
|
||||||
|
|
||||||
|
return {"ok": True, "checks": [{"name": c.check_name, "passed": c.passed} for c in checks]}
|
||||||
|
|
||||||
|
|
||||||
|
# ── 近似重名候选(只读,启发式,不做自动合并) ──────────────────────
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/team-name-duplicates")
|
||||||
|
async def team_name_duplicates(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""只读列出近似重名候选(大小写变体/子串包含/前缀碰撞)。
|
||||||
|
|
||||||
|
启发式规则(命中任一即列为候选):
|
||||||
|
- 大小写变体: lower(name) 相同但 name 不同
|
||||||
|
- 子串包含: A 是 B 的子串且 len(A) ≥ 5
|
||||||
|
- 前缀碰撞: 前 8 字符相同(忽略大小写)
|
||||||
|
|
||||||
|
仅作排查参考,合并需走人工 SQL(见 docs/05-data.md)。
|
||||||
|
"""
|
||||||
|
teams = (await db.execute(select(Team.id, Team.name))).all()
|
||||||
|
by_lower: dict[str, list[dict]] = {}
|
||||||
|
for t in teams:
|
||||||
|
key = (t.name or "").lower()
|
||||||
|
by_lower.setdefault(key, []).append({"id": t.id, "name": t.name})
|
||||||
|
|
||||||
|
groups: list[dict] = []
|
||||||
|
|
||||||
|
# 规则1: 大小写变体(lower 相同但原名不同)
|
||||||
|
for key, members in by_lower.items():
|
||||||
|
if len(members) > 1:
|
||||||
|
groups.append({
|
||||||
|
"rule": "case_variant",
|
||||||
|
"key": key,
|
||||||
|
"members": members,
|
||||||
|
})
|
||||||
|
|
||||||
|
# 规则2 & 3: 子串包含 / 前缀碰撞(仅在 lower 名不同的组间比较)
|
||||||
|
distinct = [m for members in by_lower.values() for m in members]
|
||||||
|
seen_pairs: set[tuple[int, int]] = set()
|
||||||
|
for i, a in enumerate(distinct):
|
||||||
|
na = (a["name"] or "").lower()
|
||||||
|
for b in distinct[i + 1:]:
|
||||||
|
nb = (b["name"] or "").lower()
|
||||||
|
if na == nb:
|
||||||
|
continue # 已被规则1覆盖
|
||||||
|
pair = (min(a["id"], b["id"]), max(a["id"], b["id"]))
|
||||||
|
if pair in seen_pairs:
|
||||||
|
continue
|
||||||
|
hit = None
|
||||||
|
if len(na) >= 5 and na in nb:
|
||||||
|
hit = "substring"
|
||||||
|
elif len(nb) >= 5 and nb in na:
|
||||||
|
hit = "substring"
|
||||||
|
elif len(na) >= 8 and len(nb) >= 8 and na[:8] == nb[:8]:
|
||||||
|
hit = "prefix"
|
||||||
|
if hit:
|
||||||
|
seen_pairs.add(pair)
|
||||||
|
groups.append({
|
||||||
|
"rule": hit,
|
||||||
|
"members": [a, b],
|
||||||
|
})
|
||||||
|
|
||||||
|
return {
|
||||||
|
"count": len(groups),
|
||||||
|
"hint": "命中任一启发式仅表示'可疑',合并前请人工确认是否同一球队",
|
||||||
|
"groups": groups,
|
||||||
|
}
|
||||||
@@ -1,668 +1,25 @@
|
|||||||
"""后台管理路由:数据源配置的查看、修改与连通性测试。
|
"""后台管理路由聚合入口:按职责拆分为四个子模块,统一挂载。
|
||||||
|
|
||||||
所有接口需管理员鉴权(require_admin)。配置项白名单见
|
所有路由仍挂在 /api/v1/admin,且均带 dependencies=[Depends(require_admin)]
|
||||||
src/core/runtime_config.py SETTING_DEFS,之外的 key 一律拒绝。
|
(鉴权由各子路由器声明,行为与拆分前完全一致)。
|
||||||
|
|
||||||
|
子模块:
|
||||||
|
- admin_datasources 数据源列表/连通性测试、KeyRing、采集健康概览
|
||||||
|
- admin_config settings CRUD、运行日志
|
||||||
|
- admin_llm LLM agents/models/ping
|
||||||
|
- admin_quality stats、data-completeness、data-quality
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import logging
|
from fastapi import APIRouter
|
||||||
import time
|
|
||||||
from datetime import date, datetime, timedelta, timezone
|
from src.api.routes.admin_config import router as admin_config_router
|
||||||
|
from src.api.routes.admin_datasources import router as admin_datasources_router
|
||||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
from src.api.routes.admin_llm import router as admin_llm_router
|
||||||
from pydantic import BaseModel
|
from src.api.routes.admin_quality import router as admin_quality_router
|
||||||
from sqlalchemy import func, select
|
|
||||||
import httpx
|
router = APIRouter()
|
||||||
|
router.include_router(admin_datasources_router)
|
||||||
from src.api.deps import require_admin
|
router.include_router(admin_config_router)
|
||||||
from src.core.config import settings
|
router.include_router(admin_llm_router)
|
||||||
from src.core.http_client import get_client
|
router.include_router(admin_quality_router)
|
||||||
from src.core.log_buffer import get_entries
|
|
||||||
from src.core.runtime_config import (
|
|
||||||
AGENT_META,
|
|
||||||
SETTING_DEFS,
|
|
||||||
clear_runtime_value,
|
|
||||||
get_runtime_value,
|
|
||||||
get_setting_origin,
|
|
||||||
mask_value,
|
|
||||||
set_runtime_value,
|
|
||||||
)
|
|
||||||
from src.db.base import AsyncSession, get_db_read
|
|
||||||
from src.db.models import League, Match, MatchStats, Standing
|
|
||||||
from src.data.key_ring import get_key_ring, parse_keys
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
|
|
||||||
|
|
||||||
# ── 数据源元数据(bzzoiro 单一数据源) ────────────────────────────
|
|
||||||
|
|
||||||
_SOURCES: list[dict] = [
|
|
||||||
{
|
|
||||||
"name": "bzzoiro",
|
|
||||||
"label": "Bzzoiro",
|
|
||||||
"description": "唯一数据源:赛程比分 + 积分榜 + 比赛详细统计(xG/射门/控球等)",
|
|
||||||
"setting_keys": ["BZZOIRO_KEY", "BZZOIRO_BASE"],
|
|
||||||
},
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
class SettingUpdateIn(BaseModel):
|
|
||||||
value: str
|
|
||||||
|
|
||||||
|
|
||||||
async def _last_ingestion(db: AsyncSession, source: str) -> datetime | None:
|
|
||||||
"""源最近一次采集时间(取自数据血缘字段,无记录返回 None)。"""
|
|
||||||
return (
|
|
||||||
await db.execute(
|
|
||||||
select(func.max(MatchStats.retrieved_at)).where(MatchStats.source == source)
|
|
||||||
)
|
|
||||||
).scalar()
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/datasources")
|
|
||||||
async def list_datasources(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""数据源列表:各配置项的脱敏值、来源(db/env/none)与最近采集时间。"""
|
|
||||||
result = []
|
|
||||||
for src in _SOURCES:
|
|
||||||
settings_out = []
|
|
||||||
for key in src["setting_keys"]:
|
|
||||||
origin, value = await get_setting_origin(key)
|
|
||||||
defn = SETTING_DEFS[key]
|
|
||||||
settings_out.append(
|
|
||||||
{
|
|
||||||
"key": key,
|
|
||||||
"label": defn.label,
|
|
||||||
"description": defn.description,
|
|
||||||
"sensitive": defn.sensitive,
|
|
||||||
"configured": origin != "none",
|
|
||||||
"masked": mask_value(value, defn.sensitive),
|
|
||||||
"origin": origin,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
key_configured = all(s["configured"] for s in settings_out) if settings_out else True
|
|
||||||
last = await _last_ingestion(db, src["name"])
|
|
||||||
result.append(
|
|
||||||
{
|
|
||||||
"name": src["name"],
|
|
||||||
"label": src["label"],
|
|
||||||
"description": src["description"],
|
|
||||||
"key_configured": key_configured,
|
|
||||||
"last_ingestion": last.isoformat() if last else None,
|
|
||||||
"settings": settings_out,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/settings")
|
|
||||||
async def list_settings():
|
|
||||||
"""全部可配置项(脱敏),供后台各配置页渲染。"""
|
|
||||||
out = []
|
|
||||||
for key, defn in SETTING_DEFS.items():
|
|
||||||
origin, value = await get_setting_origin(key)
|
|
||||||
out.append(
|
|
||||||
{
|
|
||||||
"key": key,
|
|
||||||
"label": defn.label,
|
|
||||||
"description": defn.description,
|
|
||||||
"sensitive": defn.sensitive,
|
|
||||||
"configured": origin != "none",
|
|
||||||
"masked": mask_value(value, defn.sensitive),
|
|
||||||
"origin": origin,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
# ── LLM 可用模型检测 ────────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/logs")
|
|
||||||
async def read_logs(
|
|
||||||
level: str | None = Query(None, description="最低级别: DEBUG/INFO/WARNING/ERROR"),
|
|
||||||
keyword: str | None = Query(None, description="消息或 logger 关键字"),
|
|
||||||
limit: int = Query(200, ge=1, le=1000),
|
|
||||||
):
|
|
||||||
"""查询应用运行日志(内存环形缓冲,最新在前;进程重启后清零)。"""
|
|
||||||
entries = get_entries(level, keyword, limit)
|
|
||||||
return {"entries": entries, "count": len(entries)}
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/llm/agents")
|
|
||||||
async def list_llm_agents():
|
|
||||||
"""各专家/终裁的独立 LLM 配置状态(含当前生效模型的解析结果)。"""
|
|
||||||
out = []
|
|
||||||
for agent in AGENT_META:
|
|
||||||
aid = agent["id"].upper()
|
|
||||||
pfx = f"AGENT_{aid}_"
|
|
||||||
fields = {}
|
|
||||||
for suffix in ("MODEL", "BASE_URL", "API_KEY"):
|
|
||||||
origin, value = await get_setting_origin(f"{pfx}{suffix}")
|
|
||||||
defn = SETTING_DEFS[f"{pfx}{suffix}"]
|
|
||||||
fields[suffix.lower()] = {
|
|
||||||
"configured": origin != "none",
|
|
||||||
"masked": mask_value(value, defn.sensitive),
|
|
||||||
"origin": origin,
|
|
||||||
}
|
|
||||||
# 生效模型 = 覆盖 → 层级默认(专家/终裁 env) → 全局 LLM_MODEL
|
|
||||||
tier_default = (
|
|
||||||
settings.LLM_AGGREGATOR_MODEL if agent["id"] == "aggregator" else settings.LLM_SPECIALIST_MODEL
|
|
||||||
)
|
|
||||||
effective_model = (
|
|
||||||
fields["model"]["masked"]
|
|
||||||
if fields["model"]["configured"]
|
|
||||||
else (tier_default or await get_runtime_value("LLM_MODEL"))
|
|
||||||
)
|
|
||||||
out.append(
|
|
||||||
{
|
|
||||||
"id": agent["id"],
|
|
||||||
"label": agent["label"],
|
|
||||||
"fields": fields,
|
|
||||||
"effective_model": effective_model,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/llm/models")
|
|
||||||
async def list_llm_models():
|
|
||||||
"""探测当前 LLM 服务可用的模型列表(OpenAI 兼容 GET /models)。
|
|
||||||
|
|
||||||
只读探测,不产生费用;配置缺失或服务不可达时返回 ok=false 与原因。
|
|
||||||
"""
|
|
||||||
base_url = (await get_runtime_value("LLM_BASE_URL")).rstrip("/")
|
|
||||||
api_key = await get_runtime_value("LLM_API_KEY")
|
|
||||||
if not base_url or not api_key:
|
|
||||||
return {"ok": False, "models": [], "detail": "LLM_BASE_URL 或 LLM_API_KEY 未配置"}
|
|
||||||
|
|
||||||
client = get_client()
|
|
||||||
start = time.monotonic()
|
|
||||||
try:
|
|
||||||
resp = await client.get(
|
|
||||||
f"{base_url}/models",
|
|
||||||
headers={"Authorization": f"Bearer {api_key}"},
|
|
||||||
timeout=httpx.Timeout(connect=10.0, read=20.0, write=10.0, pool=10.0),
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
return {
|
|
||||||
"ok": False,
|
|
||||||
"models": [],
|
|
||||||
"latency_ms": int((time.monotonic() - start) * 1000),
|
|
||||||
"detail": f"无法连接 LLM 服务: {e}",
|
|
||||||
}
|
|
||||||
|
|
||||||
latency = int((time.monotonic() - start) * 1000)
|
|
||||||
if resp.status_code in (401, 403):
|
|
||||||
return {"ok": False, "models": [], "latency_ms": latency, "detail": "密钥无效或无权限(HTTP 401/403)"}
|
|
||||||
if resp.status_code != 200:
|
|
||||||
return {"ok": False, "models": [], "latency_ms": latency, "detail": f"服务返回 HTTP {resp.status_code}"}
|
|
||||||
|
|
||||||
try:
|
|
||||||
data = resp.json()
|
|
||||||
except Exception:
|
|
||||||
return {"ok": False, "models": [], "latency_ms": latency, "detail": "响应不是合法 JSON"}
|
|
||||||
|
|
||||||
models: list[str] = []
|
|
||||||
items = data.get("data") if isinstance(data, dict) else None
|
|
||||||
if isinstance(items, list):
|
|
||||||
models = sorted(
|
|
||||||
str(m.get("id")) for m in items if isinstance(m, dict) and m.get("id")
|
|
||||||
)
|
|
||||||
if not models:
|
|
||||||
return {"ok": False, "models": [], "latency_ms": latency, "detail": "服务未返回模型列表"}
|
|
||||||
return {"ok": True, "models": models, "latency_ms": latency, "detail": f"共 {len(models)} 个可用模型"}
|
|
||||||
|
|
||||||
|
|
||||||
@router.put("/settings/{key}")
|
|
||||||
async def update_setting(key: str, body: SettingUpdateIn):
|
|
||||||
"""更新配置项(写入 app_settings 覆盖 .env)。传空值请改用 DELETE。"""
|
|
||||||
if key not in SETTING_DEFS:
|
|
||||||
raise HTTPException(404, f"不支持的配置项: {key}")
|
|
||||||
value = body.value.strip()
|
|
||||||
if not value:
|
|
||||||
raise HTTPException(400, "值不能为空;如需回落 .env 请调用清除接口")
|
|
||||||
await set_runtime_value(key, value)
|
|
||||||
defn = SETTING_DEFS[key]
|
|
||||||
return {"key": key, "masked": mask_value(value, defn.sensitive), "origin": "db"}
|
|
||||||
|
|
||||||
|
|
||||||
@router.delete("/settings/{key}")
|
|
||||||
async def clear_setting(key: str):
|
|
||||||
"""清除 DB 覆盖值,回落 .env 默认。"""
|
|
||||||
if key not in SETTING_DEFS:
|
|
||||||
raise HTTPException(404, f"不支持的配置项: {key}")
|
|
||||||
await clear_runtime_value(key)
|
|
||||||
origin, value = await get_setting_origin(key)
|
|
||||||
defn = SETTING_DEFS[key]
|
|
||||||
return {
|
|
||||||
"key": key,
|
|
||||||
"masked": mask_value(value, defn.sensitive),
|
|
||||||
"origin": origin,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ── 连通性测试 ──────────────────────────────────────────────────
|
|
||||||
|
|
||||||
_TEST_TIMEOUT = 15
|
|
||||||
|
|
||||||
|
|
||||||
async def _probe(url: str, headers: dict | None = None, params: dict | None = None) -> dict:
|
|
||||||
"""单次 HTTP 探测,返回 (ok, status, latency_ms, detail)。不重试。"""
|
|
||||||
client = get_client()
|
|
||||||
start = time.monotonic()
|
|
||||||
try:
|
|
||||||
resp = await client.get(url, headers=headers, params=params, timeout=_TEST_TIMEOUT)
|
|
||||||
except Exception as e:
|
|
||||||
return {
|
|
||||||
"ok": False,
|
|
||||||
"status": None,
|
|
||||||
"latency_ms": int((time.monotonic() - start) * 1000),
|
|
||||||
"detail": f"无法连接: {e}",
|
|
||||||
}
|
|
||||||
latency = int((time.monotonic() - start) * 1000)
|
|
||||||
status = resp.status_code
|
|
||||||
if status == 200:
|
|
||||||
detail = "连接成功"
|
|
||||||
elif status in (401, 403):
|
|
||||||
detail = "服务可达,但密钥无效或无权限"
|
|
||||||
else:
|
|
||||||
detail = f"服务返回 HTTP {status}"
|
|
||||||
return {"ok": status == 200, "status": status, "latency_ms": latency, "detail": detail}
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/datasources/{name}/test")
|
|
||||||
async def test_datasource(name: str):
|
|
||||||
"""轻量连通性测试:真实请求上游一次,不触发任何入库。"""
|
|
||||||
src = next((s for s in _SOURCES if s["name"] == name), None)
|
|
||||||
if src is None:
|
|
||||||
raise HTTPException(404, f"未知数据源: {name}")
|
|
||||||
|
|
||||||
if name == "bzzoiro":
|
|
||||||
key = await get_runtime_value("BZZOIRO_KEY")
|
|
||||||
if not key:
|
|
||||||
return {"ok": False, "status": None, "latency_ms": 0, "detail": "BZZOIRO_KEY 未配置"}
|
|
||||||
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
|
||||||
today = date.today().isoformat()
|
|
||||||
return await _probe(
|
|
||||||
f"{base}/events/",
|
|
||||||
headers={"Authorization": f"Token {key}", "Accept": "application/json"},
|
|
||||||
params={"date_from": today, "date_to": today},
|
|
||||||
)
|
|
||||||
|
|
||||||
raise HTTPException(404, f"未知数据源: {name}")
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/llm/ping")
|
|
||||||
async def llm_ping():
|
|
||||||
"""LLM 连通性测试(不依赖比赛)。只发一次 chat 请求验证配置。"""
|
|
||||||
from src.llm.provider import get_default_provider
|
|
||||||
p = await get_default_provider()
|
|
||||||
resp = await p.chat(
|
|
||||||
system="你是测试助手。",
|
|
||||||
user="ping",
|
|
||||||
max_tokens=10,
|
|
||||||
)
|
|
||||||
if resp.error:
|
|
||||||
return {"ok": False, "message": resp.error}
|
|
||||||
return {"ok": True, "message": "LLM 连接正常", "model": p.model}
|
|
||||||
|
|
||||||
|
|
||||||
# ── 数据源健康/最近采集状态(只读,不触发采集) ──────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/ingest/status")
|
|
||||||
async def ingest_status(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""数据源采集健康概览(bzzoiro 单源;只读,不触发任何采集)。"""
|
|
||||||
bzzoiro_key = await get_runtime_value("BZZOIRO_KEY")
|
|
||||||
bzzoiro_base = await get_runtime_value("BZZOIRO_BASE")
|
|
||||||
|
|
||||||
# 比赛覆盖
|
|
||||||
match_row = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("cnt"),
|
|
||||||
func.max(Match.match_date).label("latest_match_date"),
|
|
||||||
func.max(Match.created_at).label("latest_row_at"),
|
|
||||||
).where(Match.match_status == "finished")
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
# 统计覆盖(精确 retrieved_at)
|
|
||||||
stats_row = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("cnt"),
|
|
||||||
func.max(MatchStats.retrieved_at).label("latest_retrieved"),
|
|
||||||
).where(MatchStats.source == "bzzoiro")
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
# 积分榜覆盖
|
|
||||||
standings_row = (
|
|
||||||
await db.execute(select(func.count()).select_from(Standing))
|
|
||||||
).scalar()
|
|
||||||
|
|
||||||
bzzoiro = {
|
|
||||||
"name": "bzzoiro",
|
|
||||||
"label": "Bzzoiro",
|
|
||||||
"key_configured": bool(bzzoiro_key),
|
|
||||||
"base_url": (bzzoiro_base.rstrip("/") if bzzoiro_base else None) or settings.BZZOIRO_BASE,
|
|
||||||
"reachable": None, # 不主动探测
|
|
||||||
"last_success_at": (stats_row.latest_retrieved or match_row.latest_row_at),
|
|
||||||
"last_success_at_iso": (
|
|
||||||
stats_row.latest_retrieved or match_row.latest_row_at
|
|
||||||
).isoformat() if (stats_row.latest_retrieved or match_row.latest_row_at) else None,
|
|
||||||
"latest_match_date": match_row.latest_match_date.isoformat() if match_row.latest_match_date else None,
|
|
||||||
"recent_count": match_row.cnt or 0,
|
|
||||||
"stats_count": stats_row.cnt or 0,
|
|
||||||
"standings_count": standings_row or 0,
|
|
||||||
"note": "last_success_at 取 match_stats.retrieved_at(统计回填)与 matches.created_at(比赛行)的较大者",
|
|
||||||
"last_failure": _last_failure_log("bzzoiro"),
|
|
||||||
}
|
|
||||||
|
|
||||||
return {"sources": [bzzoiro]}
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/keyring/status")
|
|
||||||
async def keyring_status():
|
|
||||||
"""KeyRing 运行状态:当前使用的 key、冷却状态、轮转信息(供管理后台展示)。"""
|
|
||||||
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
|
||||||
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
|
||||||
ring = get_key_ring(base, raw_keys)
|
|
||||||
st = ring.stats()
|
|
||||||
st["base_url"] = base
|
|
||||||
st["cooldown_seconds"] = ring._cooldown
|
|
||||||
st["has_multiple"] = ring.has_multiple
|
|
||||||
st["active_key"] = ring.active_key
|
|
||||||
return st
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/keyring/cooldown/reset")
|
|
||||||
async def keyring_reset_cooldown():
|
|
||||||
"""手动重置所有 key 的冷却状态(用于紧急恢复)。"""
|
|
||||||
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
|
||||||
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
|
||||||
ring = get_key_ring(base, raw_keys)
|
|
||||||
ring._blocked_until.clear()
|
|
||||||
return {"ok": True, "message": "已重置所有 key 冷却状态", "stats": ring.stats()}
|
|
||||||
|
|
||||||
|
|
||||||
def _last_failure_log(source: str) -> dict | None:
|
|
||||||
"""从系统日志缓冲中查找某数据源的最近一次错误(仅作参考,非专用失败表)。"""
|
|
||||||
entries = get_entries(min_level="ERROR", keyword=source, limit=5)
|
|
||||||
if not entries:
|
|
||||||
return None
|
|
||||||
e = entries[0]
|
|
||||||
return {
|
|
||||||
"at": datetime.fromtimestamp(e["ts"]).isoformat(),
|
|
||||||
"logger": e["logger"],
|
|
||||||
"detail": e["message"][:200],
|
|
||||||
"note": "approx:来自内存日志缓冲,非专用采集失败表;进程重启后清零",
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/stats")
|
|
||||||
async def admin_stats(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""管理区统计(只读):预测次数 + 比赛覆盖。轻量聚合,无 LLM 调用。"""
|
|
||||||
from sqlalchemy import func, text
|
|
||||||
from src.db.models import Prediction, Match, MatchStats, Standing
|
|
||||||
day_ago = datetime.now(timezone.utc) - timedelta(days=1)
|
|
||||||
week_ago = datetime.now(timezone.utc) - timedelta(days=7)
|
|
||||||
r = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("total"),
|
|
||||||
func.count().filter(Prediction.created_at >= day_ago).label("last_24h"),
|
|
||||||
func.count().filter(Prediction.created_at >= week_ago).label("last_7d"),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
# F3 修复: 补充真实比赛计数(非 limit=100 近似)
|
|
||||||
match_cnt = (await db.execute(select(func.count()).select_from(Match))).scalar() or 0
|
|
||||||
finished_cnt = (await db.execute(select(func.count()).where(Match.match_status == "finished"))).scalar() or 0
|
|
||||||
stats_cnt = (await db.execute(select(func.count()).select_from(MatchStats))).scalar() or 0
|
|
||||||
standings_cnt = (await db.execute(select(func.count()).select_from(Standing))).scalar() or 0
|
|
||||||
return {
|
|
||||||
"predictions": {"total": r.total, "last_24h": r.last_24h, "last_7d": r.last_7d},
|
|
||||||
"matches": {"total": match_cnt, "finished": finished_cnt},
|
|
||||||
"stats": {"total": stats_cnt},
|
|
||||||
"standings": {"total": standings_cnt},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ── 数据完整性分析(可视化数据源) ────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/data-completeness")
|
|
||||||
async def data_completeness(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""按联赛统计数据完整性:比赛覆盖、字段覆盖、积分榜覆盖。
|
|
||||||
|
|
||||||
前端「数据完整性」页据此渲染,回答三个问题:
|
|
||||||
1. 数据是否齐全(各联赛比赛/统计/积分榜量级)
|
|
||||||
2. 字段是否齐全(每张统计表各字段非空率)
|
|
||||||
3. 覆盖是否新鲜(最近一场/最近一次采集)
|
|
||||||
"""
|
|
||||||
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_NAMES, LEAGUE_COUNTRIES
|
|
||||||
|
|
||||||
out_leagues: list[dict] = []
|
|
||||||
for code, bzz_id in BZZOIRO_LEAGUE_IDS.items():
|
|
||||||
# 比赛覆盖
|
|
||||||
m = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("total"),
|
|
||||||
func.count().filter(Match.match_status == "finished").label("finished"),
|
|
||||||
func.count().filter(Match.match_status == "scheduled").label("scheduled"),
|
|
||||||
func.count().filter(Match.source_event_id.is_not(None)).label("with_source_id"),
|
|
||||||
func.max(Match.match_date).label("latest_match"),
|
|
||||||
func.min(Match.match_date).label("earliest_match"),
|
|
||||||
)
|
|
||||||
.select_from(Match)
|
|
||||||
.join(League, League.id == Match.league_id)
|
|
||||||
.where(League.code == code)
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
# 统计字段覆盖(联表 matches)
|
|
||||||
s = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("rows"),
|
|
||||||
func.count(MatchStats.home_xg).label("xg"),
|
|
||||||
func.count(MatchStats.home_shots).label("shots"),
|
|
||||||
func.count(MatchStats.home_possession).label("possession"),
|
|
||||||
func.count(MatchStats.home_corners).label("corners"),
|
|
||||||
func.count(MatchStats.home_fouls).label("fouls"),
|
|
||||||
func.count(MatchStats.home_big_chances).label("big_chances"),
|
|
||||||
func.count(MatchStats.home_yellow_cards).label("cards"),
|
|
||||||
)
|
|
||||||
.select_from(MatchStats)
|
|
||||||
.join(Match, Match.id == MatchStats.match_id)
|
|
||||||
.join(League, League.id == Match.league_id)
|
|
||||||
.where(League.code == code)
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
# 积分榜覆盖
|
|
||||||
st = (
|
|
||||||
await db.execute(
|
|
||||||
select(
|
|
||||||
func.count().label("rows"),
|
|
||||||
func.max(Standing.retrieved_at).label("latest_retrieved"),
|
|
||||||
)
|
|
||||||
.select_from(Standing)
|
|
||||||
.join(League, League.id == Standing.league_id)
|
|
||||||
.where(League.code == code)
|
|
||||||
)
|
|
||||||
).one()
|
|
||||||
|
|
||||||
stats_rows = s.rows or 0
|
|
||||||
pct = lambda n: round(n / stats_rows * 100, 1) if stats_rows else 0.0 # noqa: E731
|
|
||||||
out_leagues.append(
|
|
||||||
{
|
|
||||||
"code": code,
|
|
||||||
"name": LEAGUE_NAMES.get(code, code),
|
|
||||||
"country": LEAGUE_COUNTRIES.get(code),
|
|
||||||
"matches": {
|
|
||||||
"total": m.total or 0,
|
|
||||||
"finished": m.finished or 0,
|
|
||||||
"scheduled": m.scheduled or 0,
|
|
||||||
"with_source_id": m.with_source_id or 0,
|
|
||||||
"earliest_match": m.earliest_match.isoformat() if m.earliest_match else None,
|
|
||||||
"latest_match": m.latest_match.isoformat() if m.latest_match else None,
|
|
||||||
},
|
|
||||||
"stats": {
|
|
||||||
"rows": stats_rows,
|
|
||||||
"fields": {
|
|
||||||
"xg": {"count": s.xg or 0, "pct": pct(s.xg or 0)},
|
|
||||||
"shots": {"count": s.shots or 0, "pct": pct(s.shots or 0)},
|
|
||||||
"possession": {"count": s.possession or 0, "pct": pct(s.possession or 0)},
|
|
||||||
"corners": {"count": s.corners or 0, "pct": pct(s.corners or 0)},
|
|
||||||
"fouls": {"count": s.fouls or 0, "pct": pct(s.fouls or 0)},
|
|
||||||
"big_chances": {"count": s.big_chances or 0, "pct": pct(s.big_chances or 0)},
|
|
||||||
"cards": {"count": s.cards or 0, "pct": pct(s.cards or 0)},
|
|
||||||
},
|
|
||||||
},
|
|
||||||
"standings": {
|
|
||||||
"rows": st.rows or 0,
|
|
||||||
"latest_retrieved": st.latest_retrieved.isoformat() if st.latest_retrieved else None,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
# 整体健康信号
|
|
||||||
total_finished = sum(l["matches"]["finished"] for l in out_leagues)
|
|
||||||
total_stats = sum(l["stats"]["rows"] for l in out_leagues)
|
|
||||||
stats_coverage = round(total_stats / total_finished * 100, 1) if total_finished else 0.0
|
|
||||||
issues: list[str] = []
|
|
||||||
for l in out_leagues:
|
|
||||||
if l["matches"]["finished"] == 0:
|
|
||||||
issues.append(f"{l['name']}: 无已完赛比赛,请先运行「比赛数据」采集")
|
|
||||||
elif l["stats"]["rows"] == 0:
|
|
||||||
issues.append(f"{l['name']}: 已完赛 {l['matches']['finished']} 场但无统计回填,请运行「统计回填」采集")
|
|
||||||
elif stats_coverage < 80:
|
|
||||||
issues.append(f"{l['name']}: 统计覆盖率仅 {stats_coverage}%,建议增量回填")
|
|
||||||
if l["standings"]["rows"] == 0:
|
|
||||||
issues.append(f"{l['name']}: 无积分榜数据,请运行「积分榜」采集")
|
|
||||||
if not issues:
|
|
||||||
issues.append("各联赛数据完整度良好")
|
|
||||||
|
|
||||||
return {
|
|
||||||
"generated_at": datetime.now(timezone.utc).isoformat(),
|
|
||||||
"leagues": out_leagues,
|
|
||||||
"totals": {
|
|
||||||
"finished_matches": total_finished,
|
|
||||||
"stats_rows": total_stats,
|
|
||||||
"stats_coverage_pct": stats_coverage,
|
|
||||||
},
|
|
||||||
"issues": issues,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
# ── 数据质量检查 API ────────────────────────────────────────────
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/data-quality")
|
|
||||||
async def data_quality_checks(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""数据质量检查结果(只读)。"""
|
|
||||||
from src.db.models import IngestFailure, DataQualityCheck
|
|
||||||
from sqlalchemy import func
|
|
||||||
|
|
||||||
# 最近的失败记录
|
|
||||||
failures = (
|
|
||||||
await db.execute(
|
|
||||||
select(IngestFailure)
|
|
||||||
.where(IngestFailure.status.in_(["pending", "retrying"]))
|
|
||||||
.order_by(IngestFailure.created_at.desc())
|
|
||||||
.limit(20)
|
|
||||||
)
|
|
||||||
).scalars().all()
|
|
||||||
|
|
||||||
# 最近的质量检查
|
|
||||||
checks = (
|
|
||||||
await db.execute(
|
|
||||||
select(DataQualityCheck)
|
|
||||||
.order_by(DataQualityCheck.checked_at.desc())
|
|
||||||
.limit(20)
|
|
||||||
)
|
|
||||||
).scalars().all()
|
|
||||||
|
|
||||||
return {
|
|
||||||
"failures": [
|
|
||||||
{
|
|
||||||
"id": f.id,
|
|
||||||
"source": f.source_system,
|
|
||||||
"entity_type": f.entity_type,
|
|
||||||
"source_record_id": f.source_record_id,
|
|
||||||
"error_type": f.error_type,
|
|
||||||
"error_detail": f.error_detail,
|
|
||||||
"retry_count": f.retry_count,
|
|
||||||
"status": f.status,
|
|
||||||
"created_at": f.created_at.isoformat() if f.created_at else None,
|
|
||||||
}
|
|
||||||
for f in failures
|
|
||||||
],
|
|
||||||
"checks": [
|
|
||||||
{
|
|
||||||
"id": c.id,
|
|
||||||
"check_name": c.check_name,
|
|
||||||
"entity_type": c.entity_type,
|
|
||||||
"passed": c.passed,
|
|
||||||
"severity": c.severity,
|
|
||||||
"detail": c.detail,
|
|
||||||
"checked_at": c.checked_at.isoformat() if c.checked_at else None,
|
|
||||||
}
|
|
||||||
for c in checks
|
|
||||||
],
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
@router.post("/data-quality/run")
|
|
||||||
async def run_data_quality_check(db: AsyncSession = Depends(get_db_read)):
|
|
||||||
"""手动触发一次数据质量检查。"""
|
|
||||||
from src.db.models import DataQualityCheck, Match, MatchStats, Standing, League
|
|
||||||
from sqlalchemy import func
|
|
||||||
|
|
||||||
checks = []
|
|
||||||
|
|
||||||
# 检查1: 已完赛但无统计的比赛
|
|
||||||
finished_no_stats = (
|
|
||||||
await db.execute(
|
|
||||||
select(func.count())
|
|
||||||
.select_from(Match)
|
|
||||||
.outerjoin(MatchStats, Match.id == MatchStats.match_id)
|
|
||||||
.where(Match.match_status == "finished")
|
|
||||||
.where(MatchStats.id.is_(None))
|
|
||||||
)
|
|
||||||
).scalar() or 0
|
|
||||||
|
|
||||||
checks.append(DataQualityCheck(
|
|
||||||
check_name="finished_without_stats",
|
|
||||||
entity_type="match",
|
|
||||||
actual_value=float(finished_no_stats),
|
|
||||||
passed=finished_no_stats == 0,
|
|
||||||
severity="warning" if finished_no_stats > 0 else "info",
|
|
||||||
detail={"message": f"{finished_no_stats} 场已完赛比赛缺少统计数据"},
|
|
||||||
))
|
|
||||||
|
|
||||||
# 检查2: 积分榜缺失的联赛
|
|
||||||
leagues_without_standings = (
|
|
||||||
await db.execute(
|
|
||||||
select(func.count())
|
|
||||||
.select_from(League)
|
|
||||||
.outerjoin(Standing, League.id == Standing.league_id)
|
|
||||||
.where(Standing.id.is_(None))
|
|
||||||
)
|
|
||||||
).scalar() or 0
|
|
||||||
|
|
||||||
checks.append(DataQualityCheck(
|
|
||||||
check_name="league_without_standings",
|
|
||||||
entity_type="league",
|
|
||||||
actual_value=float(leagues_without_standings),
|
|
||||||
passed=leagues_without_standings == 0,
|
|
||||||
severity="warning" if leagues_without_standings > 0 else "info",
|
|
||||||
detail={"message": f"{leagues_without_standings} 个联赛缺少积分榜"},
|
|
||||||
))
|
|
||||||
|
|
||||||
for c in checks:
|
|
||||||
db.add(c)
|
|
||||||
await db.commit()
|
|
||||||
|
|
||||||
return {"ok": True, "checks": [{"name": c.check_name, "passed": c.passed} for c in checks]}
|
|
||||||
|
|||||||
@@ -16,7 +16,8 @@ from fastapi import APIRouter, Depends, HTTPException
|
|||||||
from src.api.deps import require_admin
|
from src.api.deps import require_admin
|
||||||
from src.api.schemas import IngestBzzoiroRequest
|
from src.api.schemas import IngestBzzoiroRequest
|
||||||
from src.data.config import BZZOIRO_LEAGUE_IDS
|
from src.data.config import BZZOIRO_LEAGUE_IDS
|
||||||
from src.data.bzzoiro import ingest_bzzoiro_event_stats, ingest_bzzoiro_standings
|
from src.data.bzzoiro_standings import ingest_bzzoiro_standings
|
||||||
|
from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats
|
||||||
from src.data.sources import get_source
|
from src.data.sources import get_source
|
||||||
from src.db.unit_of_work import get_uow
|
from src.db.unit_of_work import get_uow
|
||||||
|
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ from fastapi import APIRouter, Depends, HTTPException, Query
|
|||||||
from sqlalchemy import func, or_, select
|
from sqlalchemy import func, or_, select
|
||||||
from sqlalchemy.orm import selectinload
|
from sqlalchemy.orm import selectinload
|
||||||
|
|
||||||
from src.api.deps import require_admin
|
|
||||||
from src.api.schemas import MatchListOut, MatchOut, PredictionOut
|
from src.api.schemas import MatchListOut, MatchOut, PredictionOut
|
||||||
from src.db.base import AsyncSession, get_db_read
|
from src.db.base import AsyncSession, get_db_read
|
||||||
from src.db.models import League, Match, Prediction, Standing
|
from src.db.models import League, Match, Prediction, Standing
|
||||||
@@ -32,8 +31,9 @@ def _stats_dict(stats) -> dict | None:
|
|||||||
}
|
}
|
||||||
|
|
||||||
|
|
||||||
@router.get("/leagues", response_model=list[dict], dependencies=[Depends(require_admin)])
|
@router.get("/leagues", response_model=list[dict])
|
||||||
async def list_leagues(db: AsyncSession = Depends(get_db_read)):
|
async def list_leagues(db: AsyncSession = Depends(get_db_read)):
|
||||||
|
"""联赛列表(公开只读,P1-3: 公开站联赛筛选需要;仅返回展示字段)。"""
|
||||||
stmt = select(League).order_by(League.name)
|
stmt = select(League).order_by(League.name)
|
||||||
result = await db.execute(stmt)
|
result = await db.execute(stmt)
|
||||||
leagues = result.scalars().all()
|
leagues = result.scalars().all()
|
||||||
@@ -190,9 +190,9 @@ async def get_match(match_id: int, db: AsyncSession = Depends(get_db_read)):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@router.get("/matches/{match_id}/context", dependencies=[Depends(require_admin)])
|
@router.get("/matches/{match_id}/context")
|
||||||
async def match_context(match_id: int, db: AsyncSession = Depends(get_db_read)):
|
async def match_context(match_id: int, db: AsyncSession = Depends(get_db_read)):
|
||||||
"""比赛上下文(只读,不触发 LLM):双方近况 + 历史交锋。
|
"""比赛上下文(公开只读,P1-2: 公开站详情页需要;不触发 LLM):双方近况 + 历史交锋。
|
||||||
|
|
||||||
全部基于现有数据聚合:
|
全部基于现有数据聚合:
|
||||||
- recent_home / recent客队:该队最近 5 场已完赛(进球/结果)
|
- recent_home / recent客队:该队最近 5 场已完赛(进球/结果)
|
||||||
|
|||||||
@@ -64,11 +64,9 @@ async def predict(req: PredictRequest, request: Request):
|
|||||||
raise HTTPException(500, "预测失败,请查看服务器日志")
|
raise HTTPException(500, "预测失败,请查看服务器日志")
|
||||||
|
|
||||||
# D2: 三种模式统一返回 PredictResult —— 字段映射单一化,无 dict 分支。
|
# D2: 三种模式统一返回 PredictResult —— 字段映射单一化,无 dict 分支。
|
||||||
# 仅 baseline 的 prediction_id 需要在此落库补齐(服务层不落库)。
|
# P3-2:baseline 已在服务层(predict_baseline)落库并回填真实 prediction_id,
|
||||||
if req.mode == "baseline":
|
# 路由层不再需要特殊的 _persist_baseline,与 single/multi 路径统一。
|
||||||
prediction_id = await _persist_baseline(req.match_id, result)
|
prediction_id = result.prediction_id
|
||||||
else:
|
|
||||||
prediction_id = result.prediction_id
|
|
||||||
|
|
||||||
# 3. 结果映射(无 DB 访问)
|
# 3. 结果映射(无 DB 访问)
|
||||||
logger.info(
|
logger.info(
|
||||||
@@ -101,36 +99,6 @@ async def predict(req: PredictRequest, request: Request):
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
async def _persist_baseline(match_id: int, baseline: PredictResult) -> int:
|
|
||||||
"""将基线预测结果写入 prediction 表,复用 upsert 语义。"""
|
|
||||||
from src.db.unit_of_work import get_uow
|
|
||||||
from src.llm.predict import _upsert_prediction
|
|
||||||
|
|
||||||
async with get_uow() as session:
|
|
||||||
pred = await _upsert_prediction(
|
|
||||||
session,
|
|
||||||
match_id=match_id,
|
|
||||||
provider_name="baseline",
|
|
||||||
model="baseline",
|
|
||||||
mode="baseline",
|
|
||||||
run_type="live", # baseline 是 live 预测的变体,符合 ck_run_type_enum
|
|
||||||
values={
|
|
||||||
"prompt_version": baseline.prompt_version,
|
|
||||||
"prompt_tokens": baseline.prompt_tokens or 0,
|
|
||||||
"completion_tokens": baseline.completion_tokens or 0,
|
|
||||||
"latency_ms": baseline.latency_ms or 0,
|
|
||||||
"pred_home_goals": baseline.pred_home_goals,
|
|
||||||
"pred_away_goals": baseline.pred_away_goals,
|
|
||||||
"pred_1x2": baseline.pred_1x2,
|
|
||||||
"subjective_confidence": baseline.subjective_confidence,
|
|
||||||
"reasoning": baseline.reasoning,
|
|
||||||
"raw_response": baseline.raw,
|
|
||||||
"status": "success",
|
|
||||||
},
|
|
||||||
)
|
|
||||||
return pred.id
|
|
||||||
|
|
||||||
|
|
||||||
@router.get("/predictions", response_model=list[PredictionOut], dependencies=[Depends(require_admin)])
|
@router.get("/predictions", response_model=list[PredictionOut], dependencies=[Depends(require_admin)])
|
||||||
async def list_predictions(
|
async def list_predictions(
|
||||||
match_id: int | None = None,
|
match_id: int | None = None,
|
||||||
|
|||||||
@@ -10,7 +10,8 @@ from sqlalchemy import select, delete
|
|||||||
from src.api.deps import require_admin
|
from src.api.deps import require_admin
|
||||||
from src.api.schemas import ScheduleIn, ScheduleUpdate, ScheduleOut
|
from src.api.schemas import ScheduleIn, ScheduleUpdate, ScheduleOut
|
||||||
from src.core.scheduler import scheduler
|
from src.core.scheduler import scheduler
|
||||||
from src.data.bzzoiro import ingest_bzzoiro_event_stats, ingest_bzzoiro_standings
|
from src.data.bzzoiro_standings import ingest_bzzoiro_standings
|
||||||
|
from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats
|
||||||
from src.data.sources import get_source
|
from src.data.sources import get_source
|
||||||
from src.data.config import BZZOIRO_LEAGUE_IDS
|
from src.data.config import BZZOIRO_LEAGUE_IDS
|
||||||
from src.db.base import AsyncSession, get_db_read
|
from src.db.base import AsyncSession, get_db_read
|
||||||
|
|||||||
@@ -11,6 +11,10 @@ class Settings(BaseSettings):
|
|||||||
# --- app ---
|
# --- app ---
|
||||||
APP_ENV: str = "development"
|
APP_ENV: str = "development"
|
||||||
LOG_LEVEL: str = "INFO"
|
LOG_LEVEL: str = "INFO"
|
||||||
|
# P3-3:多 worker 时应用内限流与 KeyRing 各自独立计数(配额放大 N 倍)。
|
||||||
|
# 设为 True 时若以多 worker 启动 uvicorn 则拒绝启动,避免静默配额漂移。
|
||||||
|
# 仅在你已前置 Nginx/网关做全局限流、确认不需要此守护时留空/False。
|
||||||
|
STRICT_SINGLE_WORKER: bool = False
|
||||||
# 生产环境强制要求管理鉴权配置,即使 APP_ENV=production 也生效。
|
# 生产环境强制要求管理鉴权配置,即使 APP_ENV=production 也生效。
|
||||||
# True 时若 auth_configured() 为 False 则拒绝(503),development 保持 fail-open。
|
# True 时若 auth_configured() 为 False 则拒绝(503),development 保持 fail-open。
|
||||||
REQUIRE_ADMIN_AUTH: bool = False
|
REQUIRE_ADMIN_AUTH: bool = False
|
||||||
|
|||||||
+65
-787
@@ -1,793 +1,71 @@
|
|||||||
"""Bzzoiro 数据源:抓取 + 入库(单一数据源)。
|
"""Bzzoiro 数据源:抓取 + 入库(单一数据源)—— 聚合门面。
|
||||||
|
|
||||||
三条管线:
|
实现按管线拆分(单文件 → 多模块),本模块只做再导出,保持两个不变量:
|
||||||
1. events — 比赛日程/比分(/events/),含 source_event_id 血缘
|
1. sources._load_sources() 仍从本模块导入 BzzoiroSource(注册表入口不变);
|
||||||
2. standings— 联赛积分榜快照(/leagues/{id}/standings/)
|
2. 测试与脚本对 `bz.<名称>` 的 monkeypatch 语义不变 —— 子模块在运行期
|
||||||
3. stats — 已完赛比赛详细统计回填(/events/{id}/stats/)
|
经本门面解析可替换协作者(抓取函数 / Bronze 写入助手 / REQUEST_INTERVAL),
|
||||||
|
与拆分前的单文件行为一致。
|
||||||
|
|
||||||
|
三条管线(各自模块):
|
||||||
|
1. events — 比赛日程/比分(/events/),含 source_event_id 血缘 → bzzoiro_events.py
|
||||||
|
2. standings— 联赛积分榜快照(/leagues/{id}/standings/) → bzzoiro_standings.py
|
||||||
|
3. stats — 已完赛比赛详细统计回填(/events/{id}/stats/) → bzzoiro_stats.py
|
||||||
|
|
||||||
|
共享基础:HTTP 抓取(多 key 轮换)与字段转换 → bzzoiro_common.py;
|
||||||
|
Bronze 基础设施(RawEvent/IngestFailure/DataLineage)→ pipeline_write.py。
|
||||||
|
|
||||||
D4(工程债): Team/League/Match 的查找/创建经 Repository 层(src/db/repositories.py),
|
D4(工程债): Team/League/Match 的查找/创建经 Repository 层(src/db/repositories.py),
|
||||||
本模块不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制,这里只 flush)。
|
各管线不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制,只 flush)。
|
||||||
Standing/RawEvent/Lineage 等管线内私有读写仍在本模块内实现,不强行 Repository 化。
|
Standing/RawEvent/Lineage 等管线内私有读写仍不强行 Repository 化。
|
||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
# ── 配置常量(原文件即从 config 再导出,维持 bz.REQUEST_INTERVAL 等引用) ──
|
||||||
import logging
|
from src.data.config import ( # noqa: F401
|
||||||
import random
|
BZZOIRO_LEAGUE_IDS,
|
||||||
from collections.abc import Iterable
|
LEAGUE_COUNTRIES,
|
||||||
from datetime import datetime, timedelta, timezone
|
LEAGUE_NAMES,
|
||||||
|
REQUEST_INTERVAL,
|
||||||
from sqlalchemy import select
|
)
|
||||||
|
from src.data.key_ring import _mask # noqa: F401 (R1 测试引用 bz._mask)
|
||||||
import httpx
|
from src.data.normalize import normalize_bzzoiro # noqa: F401
|
||||||
|
|
||||||
from src.core.runtime_config import get_runtime_value
|
# ── 共享原语:HTTP 抓取 + 宽松字段转换 ──
|
||||||
from src.core.http_client import get_client
|
from src.data.bzzoiro_common import ( # noqa: F401
|
||||||
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES, REQUEST_INTERVAL
|
_fetch_json_async,
|
||||||
from src.data.key_ring import _mask, get_key_ring
|
_match_key,
|
||||||
from src.data.normalize import normalize_bzzoiro
|
_to_date,
|
||||||
from src.data.team_names_zh import zh_name
|
_to_float_or_none,
|
||||||
from src.data.sources import register
|
_to_int_or_none,
|
||||||
from src.db.models import Match, MatchStats, Standing, Team, RawEvent, IngestFailure, DataLineage
|
)
|
||||||
from src.db.repositories import LeagueRepository, MatchRepository, TeamRepository
|
|
||||||
|
# ── 管线基础设施:RawEvent / IngestFailure / DataLineage ──
|
||||||
logger = logging.getLogger(__name__)
|
from src.data.pipeline_write import ( # noqa: F401
|
||||||
|
_safe_write_ingest_failure,
|
||||||
|
_write_ingest_failure,
|
||||||
def _to_date(value):
|
_write_lineage,
|
||||||
"""把 datetime / date / str 统一成 `date`。"""
|
_write_raw_event,
|
||||||
if value is None:
|
)
|
||||||
return None
|
|
||||||
if hasattr(value, "date") and callable(value.date):
|
# ── events 管线:BzzoiroSource(注册表入口)+ 抓取/入库 ──
|
||||||
return value.date()
|
from src.data.bzzoiro_events import ( # noqa: F401
|
||||||
return value
|
BzzoiroSource,
|
||||||
|
_events_record_id,
|
||||||
|
_write_events_bronze,
|
||||||
def _to_int_or_none(value) -> int | None:
|
fetch_bzzoiro_events,
|
||||||
"""宽松转 int(用于上游 ID 解析,失败返回 None 不抛错)。"""
|
)
|
||||||
if value is None:
|
|
||||||
return None
|
# ── standings 管线 ──
|
||||||
try:
|
from src.data.bzzoiro_standings import ( # noqa: F401
|
||||||
return int(str(value).strip())
|
_season_label_from_dates,
|
||||||
except (TypeError, ValueError):
|
_write_standings_bronze,
|
||||||
return None
|
fetch_bzzoiro_standings,
|
||||||
|
ingest_bzzoiro_standings,
|
||||||
|
)
|
||||||
def _match_key(home_team_id: int, away_team_id: int, match_date) -> tuple[int, int, str]:
|
|
||||||
"""比赛去重键:(主队, 客队, 天级日期 ISO 字符串)。
|
# ── stats 回填管线 ──
|
||||||
|
from src.data.bzzoiro_stats import ( # noqa: F401
|
||||||
统一在这里构造,避免"预加载时用 str(date)、写入时用 isoformat()"这类
|
_pick,
|
||||||
隐式格式依赖 —— 两者当前恰好相等,但一旦有人改动其一就会静默失配,
|
_stats_from_payload,
|
||||||
导致所有比赛被判为不存在而重复插入。
|
ingest_bzzoiro_event_stats,
|
||||||
"""
|
)
|
||||||
d = _to_date(match_date)
|
|
||||||
return (home_team_id, away_team_id, d.isoformat() if d is not None else "")
|
|
||||||
|
|
||||||
|
|
||||||
async def _fetch_json_async(path: str, params: dict | None = None, max_retries: int = 3) -> dict | list:
|
|
||||||
"""异步 HTTP(bzzoiro 使用 httpx,不再阻塞事件循环线程池)。
|
|
||||||
|
|
||||||
多 key 轮换:遇到 429 自动切换到下一个 key;全部 key 冷却时等待最早恢复。
|
|
||||||
"""
|
|
||||||
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
|
||||||
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
|
||||||
ring = get_key_ring(base, raw_keys)
|
|
||||||
|
|
||||||
url = f"{base}/{path.lstrip('/')}"
|
|
||||||
key = ring.get()
|
|
||||||
if not key:
|
|
||||||
raise RuntimeError("BZZOIRO_KEY 未设置")
|
|
||||||
|
|
||||||
last_exc: Exception | None = None
|
|
||||||
for attempt in range(max_retries):
|
|
||||||
headers = {
|
|
||||||
"Authorization": f"Token {key}",
|
|
||||||
"Accept": "application/json",
|
|
||||||
}
|
|
||||||
try:
|
|
||||||
client = get_client()
|
|
||||||
# 整请求兜底: httpx 无 total 超时,用 wait_for 防「滴水式」限速挂死
|
|
||||||
resp = await asyncio.wait_for(
|
|
||||||
client.get(
|
|
||||||
url, headers=headers, params=params,
|
|
||||||
timeout=httpx.Timeout(connect=10.0, read=30.0, write=10.0, pool=10.0),
|
|
||||||
),
|
|
||||||
timeout=60.0,
|
|
||||||
)
|
|
||||||
resp.raise_for_status()
|
|
||||||
return resp.json()
|
|
||||||
except Exception as e:
|
|
||||||
last_exc = e
|
|
||||||
status = getattr(getattr(e, "response", None), "status_code", None)
|
|
||||||
if status == 429:
|
|
||||||
# 限流:标记当前 key 冷却,切换到下一个
|
|
||||||
new_key = ring.report_rate_limited(key)
|
|
||||||
if new_key and new_key != key:
|
|
||||||
logger.info("bzzoiro 429 → 切换 key: %s → %s,立即重试", _mask(key), _mask(new_key))
|
|
||||||
key = new_key
|
|
||||||
continue # 立即重试,不等待
|
|
||||||
# 单 key 或全部冷却:等待最早恢复的 key
|
|
||||||
wait = ring.wait_if_all_blocked()
|
|
||||||
if wait > 0:
|
|
||||||
logger.warning("bzzoiro 全部 key 冷却,等待 %.1fs 后重试", wait)
|
|
||||||
await asyncio.sleep(min(wait, 30.0))
|
|
||||||
else:
|
|
||||||
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
|
||||||
logger.warning("bzzoiro 429, retry %d in %.1fs", attempt + 1, delay)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
key = ring.get() or key
|
|
||||||
continue
|
|
||||||
if 500 <= (status or 0) < 600:
|
|
||||||
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
|
||||||
logger.warning("bzzoiro %d, retry %d in %.1fs", status, attempt + 1, delay)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
continue
|
|
||||||
# 网络错误(连接失败/超时)也退避重试
|
|
||||||
if isinstance(e, (TimeoutError, ConnectionError, OSError)):
|
|
||||||
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
|
||||||
logger.warning("bzzoiro network error, retry %d in %.1fs: %s", attempt + 1, delay, e)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
continue
|
|
||||||
raise
|
|
||||||
raise RuntimeError(f"bzzoiro request failed after {max_retries} attempts: {last_exc}")
|
|
||||||
|
|
||||||
|
|
||||||
async def fetch_bzzoiro_events(
|
|
||||||
league_code: str,
|
|
||||||
*,
|
|
||||||
status: str = "finished",
|
|
||||||
date_from: str | None = None,
|
|
||||||
date_to: str | None = None,
|
|
||||||
limit: int = 200,
|
|
||||||
) -> list[dict]:
|
|
||||||
"""抓取 bzzoiro 原始事件(纯异步,无需 run_in_executor)。"""
|
|
||||||
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
|
|
||||||
if league_id is None:
|
|
||||||
raise ValueError(f"未知联赛代码: {league_code}")
|
|
||||||
|
|
||||||
rows: list[dict] = []
|
|
||||||
offset = 0
|
|
||||||
while True:
|
|
||||||
params: dict = {
|
|
||||||
"league_id": league_id,
|
|
||||||
"status": status,
|
|
||||||
"limit": limit,
|
|
||||||
"offset": offset,
|
|
||||||
}
|
|
||||||
if date_from:
|
|
||||||
params["date_from"] = str(date_from)[:10]
|
|
||||||
if date_to:
|
|
||||||
params["date_to"] = str(date_to)[:10]
|
|
||||||
payload = await _fetch_json_async("/events/", params)
|
|
||||||
batch = payload.get("results") or []
|
|
||||||
if not batch:
|
|
||||||
break
|
|
||||||
rows.extend(batch)
|
|
||||||
total = payload.get("total")
|
|
||||||
offset += limit
|
|
||||||
if total is not None and offset >= total:
|
|
||||||
break
|
|
||||||
if len(batch) < limit:
|
|
||||||
break
|
|
||||||
await asyncio.sleep(REQUEST_INTERVAL)
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
@register
|
|
||||||
class BzzoiroSource:
|
|
||||||
"""bzzoiro 数据源(实现 DataSource 协议)。"""
|
|
||||||
|
|
||||||
name = "bzzoiro"
|
|
||||||
|
|
||||||
async def ingest(
|
|
||||||
self,
|
|
||||||
db,
|
|
||||||
*,
|
|
||||||
leagues: Iterable[str],
|
|
||||||
date_from: str | None = None,
|
|
||||||
date_to: str | None = None,
|
|
||||||
status: str = "finished",
|
|
||||||
) -> dict:
|
|
||||||
"""采集 bzzoiro → 入库。返回统计。
|
|
||||||
|
|
||||||
注意: 本方法不控制事务(commit/rollback),由调用方通过 UnitOfWork 控制。
|
|
||||||
"""
|
|
||||||
result: dict = {"leagues": {}, "total_inserted": 0, "total_updated": 0, "errors": []}
|
|
||||||
|
|
||||||
for code in leagues:
|
|
||||||
league_r: dict = {"inserted": 0, "updated": 0, "errors": []}
|
|
||||||
try:
|
|
||||||
raw_events = await fetch_bzzoiro_events(code, status=status, date_from=date_from, date_to=date_to)
|
|
||||||
except Exception as e:
|
|
||||||
# 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批
|
|
||||||
logger.exception("bzzoiro fetch failed for %s", code)
|
|
||||||
league_r["errors"].append(f"fetch failed: {e}")
|
|
||||||
await _safe_write_ingest_failure(
|
|
||||||
db,
|
|
||||||
entity_type="events",
|
|
||||||
source_record_id=None,
|
|
||||||
error=e,
|
|
||||||
raw_payload={"league": code, "status": status, "date_from": date_from, "date_to": date_to},
|
|
||||||
)
|
|
||||||
result["leagues"][code] = league_r
|
|
||||||
continue
|
|
||||||
|
|
||||||
# D4: 联赛查找/创建经 LeagueRepository(事务仍由调用方 UoW 提交)
|
|
||||||
league = await LeagueRepository(db).get_or_create(
|
|
||||||
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
|
|
||||||
)
|
|
||||||
team_r = TeamRepository(db)
|
|
||||||
match_r = MatchRepository(db)
|
|
||||||
|
|
||||||
# === 批量优化: 预加载球队和已有比赛到内存 ===
|
|
||||||
team_name_to_id: dict[str, int] = {}
|
|
||||||
existing_matches: dict[tuple[int, int, str], Match] = {} # 完整对象,避免重复查询
|
|
||||||
# (NormalizedMatch, 原始 event) 成对保存:后续写 source_event_id 时
|
|
||||||
# 必须用配对的那条 event,不能依赖外层循环变量残留值。
|
|
||||||
normalized_matches: list[tuple] = []
|
|
||||||
|
|
||||||
if raw_events:
|
|
||||||
# 一次遍历: 收集球队名 + 规范化
|
|
||||||
all_team_names = set()
|
|
||||||
for raw in raw_events:
|
|
||||||
nm = normalize_bzzoiro(raw, code)
|
|
||||||
if nm is not None:
|
|
||||||
try:
|
|
||||||
nm.validate()
|
|
||||||
except Exception as e:
|
|
||||||
# P1-3: 统一使用 warning,不追加到 errors(仅运行时错误入 errors)
|
|
||||||
logger.warning("normalize skip: %s", e)
|
|
||||||
continue
|
|
||||||
normalized_matches.append((nm, raw))
|
|
||||||
all_team_names.add(nm.home_team)
|
|
||||||
all_team_names.add(nm.away_team)
|
|
||||||
|
|
||||||
if all_team_names:
|
|
||||||
team_name_to_id = {
|
|
||||||
name: t.id
|
|
||||||
for name, t in (await team_r.get_all_by_names(list(all_team_names))).items()
|
|
||||||
}
|
|
||||||
|
|
||||||
# P1-2: 按需加载,只加载 raw_events 涉及日期范围的比赛(加 30 天缓冲)
|
|
||||||
# 避免加载联赛全部历史比赛到内存(多赛季采集时内存溢出)
|
|
||||||
if normalized_matches:
|
|
||||||
# normalized_matches 存的是 (nm, raw) 元组,遍历需解包
|
|
||||||
dates = [nm.date for nm, _raw in normalized_matches if nm.date is not None]
|
|
||||||
if dates:
|
|
||||||
min_dt = min(dates) - timedelta(days=30)
|
|
||||||
max_dt = max(dates) + timedelta(days=30)
|
|
||||||
matches_in_range = await match_r.find_by_league_and_date_range(
|
|
||||||
league.id, min_dt, max_dt
|
|
||||||
)
|
|
||||||
existing_matches = {
|
|
||||||
_match_key(m.home_team_id, m.away_team_id, m.match_date_date): m
|
|
||||||
for m in matches_in_range
|
|
||||||
}
|
|
||||||
# else: existing_matches 保持空 dict(全量新比赛)
|
|
||||||
|
|
||||||
# D1: Bronze 层批次信息(每联赛每批次一个 batch_id;seen 防同批重复写入)
|
|
||||||
now = datetime.now(timezone.utc)
|
|
||||||
bronze_batch_id = f"bzzoiro-events-{code}-{now:%Y%m%d%H%M%S}"
|
|
||||||
bronze_written: set[str] = set()
|
|
||||||
|
|
||||||
for nm, raw in normalized_matches:
|
|
||||||
# D1: RawEvent 幂等键(上游 id 或合成键),插入/变更更新共用
|
|
||||||
record_id = _events_record_id(code, nm, raw)
|
|
||||||
|
|
||||||
# 球队: 内存查找 + 按需创建(D4: 经 TeamRepository)
|
|
||||||
home_team_id = team_name_to_id.get(nm.home_team)
|
|
||||||
if home_team_id is None:
|
|
||||||
home = await team_r.get_or_create(nm.home_team, name_zh=zh_name(nm.home_team))
|
|
||||||
home_team_id = home.id
|
|
||||||
team_name_to_id[nm.home_team] = home_team_id
|
|
||||||
|
|
||||||
away_team_id = team_name_to_id.get(nm.away_team)
|
|
||||||
if away_team_id is None:
|
|
||||||
away = await team_r.get_or_create(nm.away_team, name_zh=zh_name(nm.away_team))
|
|
||||||
away_team_id = away.id
|
|
||||||
team_name_to_id[nm.away_team] = away_team_id
|
|
||||||
|
|
||||||
# 查找已有比赛: 内存查找
|
|
||||||
match_key = _match_key(home_team_id, away_team_id, nm.date)
|
|
||||||
existing_match = existing_matches.get(match_key)
|
|
||||||
|
|
||||||
if existing_match is None:
|
|
||||||
m = Match(
|
|
||||||
league_id=league.id,
|
|
||||||
season=nm.season_label or None,
|
|
||||||
home_team_id=home_team_id,
|
|
||||||
away_team_id=away_team_id,
|
|
||||||
match_date=nm.date,
|
|
||||||
match_date_date=_to_date(nm.date),
|
|
||||||
match_status=nm.match_status,
|
|
||||||
home_goals=nm.home_goals,
|
|
||||||
away_goals=nm.away_goals,
|
|
||||||
home_ht_goals=nm.home_ht_goals,
|
|
||||||
away_ht_goals=nm.away_ht_goals,
|
|
||||||
match_stage=nm.match_stage,
|
|
||||||
source_event_id=_to_int_or_none(raw.get("id")),
|
|
||||||
)
|
|
||||||
db.add(m)
|
|
||||||
await db.flush()
|
|
||||||
existing_matches[match_key] = m # 防止同批重复
|
|
||||||
# 统计字段不在 /events/ 载荷中(单独由 stats 管线回填),
|
|
||||||
# 此处不再创建 MatchStats。
|
|
||||||
league_r["inserted"] += 1
|
|
||||||
# D1: 成功插入 → 补写 Bronze 层(原始载荷 + 血缘)
|
|
||||||
if record_id not in bronze_written:
|
|
||||||
bronze_written.add(record_id)
|
|
||||||
await _write_events_bronze(
|
|
||||||
db,
|
|
||||||
source_record_id=record_id,
|
|
||||||
raw_payload=raw,
|
|
||||||
target_match_id=m.id,
|
|
||||||
league_code=code,
|
|
||||||
match_status=nm.match_status,
|
|
||||||
batch_id=bronze_batch_id,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
# 已有比赛: 直接从内存获取对象更新(无需再查询)
|
|
||||||
changed = False
|
|
||||||
if existing_match.match_status != nm.match_status and nm.match_status == "finished":
|
|
||||||
existing_match.match_status = nm.match_status
|
|
||||||
changed = True
|
|
||||||
if existing_match.home_goals is None and nm.home_goals is not None:
|
|
||||||
existing_match.home_goals = nm.home_goals
|
|
||||||
existing_match.away_goals = nm.away_goals
|
|
||||||
existing_match.home_ht_goals = nm.home_ht_goals
|
|
||||||
existing_match.away_ht_goals = nm.away_ht_goals
|
|
||||||
changed = True
|
|
||||||
if existing_match.match_stage is None and nm.match_stage:
|
|
||||||
existing_match.match_stage = nm.match_stage
|
|
||||||
changed = True
|
|
||||||
if existing_match.source_event_id is None:
|
|
||||||
eid = _to_int_or_none(raw.get("id"))
|
|
||||||
if eid is not None:
|
|
||||||
existing_match.source_event_id = eid
|
|
||||||
changed = True
|
|
||||||
if changed:
|
|
||||||
league_r["updated"] += 1
|
|
||||||
# D1: 变更更新 → 补写血缘(RawEvent 幂等键不变,重复采集自动跳过)
|
|
||||||
if record_id not in bronze_written:
|
|
||||||
bronze_written.add(record_id)
|
|
||||||
await _write_events_bronze(
|
|
||||||
db,
|
|
||||||
source_record_id=record_id,
|
|
||||||
raw_payload=raw,
|
|
||||||
target_match_id=existing_match.id,
|
|
||||||
league_code=code,
|
|
||||||
match_status=nm.match_status,
|
|
||||||
batch_id=bronze_batch_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
# 注意: 不在此处 commit,由调用方 UnitOfWork 控制事务
|
|
||||||
result["leagues"][code] = league_r
|
|
||||||
result["total_inserted"] += league_r["inserted"]
|
|
||||||
result["total_updated"] += league_r["updated"]
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
# ============================================================
|
|
||||||
# 管线基础设施:RawEvent / IngestFailure / DataLineage
|
|
||||||
# ============================================================
|
|
||||||
|
|
||||||
|
|
||||||
async def _write_raw_event(db, source_system: str, source_record_id: str, raw_payload: dict, batch_id: str | None = None) -> None:
|
|
||||||
"""写入 Bronze 层原始事件(幂等:同 source_record_id 跳过)。"""
|
|
||||||
from sqlalchemy import select as _select
|
|
||||||
stmt = _select(RawEvent).where(
|
|
||||||
RawEvent.source_system == source_system,
|
|
||||||
RawEvent.source_record_id == source_record_id,
|
|
||||||
)
|
|
||||||
existing = (await db.execute(stmt)).scalar_one_or_none()
|
|
||||||
if existing is None:
|
|
||||||
db.add(RawEvent(
|
|
||||||
source_system=source_system,
|
|
||||||
source_record_id=source_record_id,
|
|
||||||
raw_payload=raw_payload,
|
|
||||||
ingest_batch_id=batch_id,
|
|
||||||
))
|
|
||||||
|
|
||||||
|
|
||||||
async def _write_ingest_failure(db, source_system: str, entity_type: str, source_record_id: str | None, error_type: str, error_detail: str | None, raw_payload: dict | None = None) -> None:
|
|
||||||
"""写入采集失败死信。"""
|
|
||||||
db.add(IngestFailure(
|
|
||||||
source_system=source_system,
|
|
||||||
entity_type=entity_type,
|
|
||||||
source_record_id=source_record_id,
|
|
||||||
error_type=error_type,
|
|
||||||
error_detail=error_detail,
|
|
||||||
raw_payload=raw_payload,
|
|
||||||
))
|
|
||||||
|
|
||||||
|
|
||||||
async def _safe_write_ingest_failure(
|
|
||||||
db,
|
|
||||||
*,
|
|
||||||
entity_type: str,
|
|
||||||
source_record_id: str | None,
|
|
||||||
error: Exception,
|
|
||||||
raw_payload: dict | None = None,
|
|
||||||
) -> None:
|
|
||||||
"""抓取失败时尽力写入死信表(失败不影响主流程)。
|
|
||||||
|
|
||||||
死信是「可观测性」基础设施,与 RawEvent/Lineage 同级:写入失败只记
|
|
||||||
warning,绝不能让原始抓取错误之外的新异常打断采集循环。
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
await _write_ingest_failure(
|
|
||||||
db, "bzzoiro", entity_type, source_record_id,
|
|
||||||
"fetch_error", str(error), raw_payload,
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
logger.warning(
|
|
||||||
"写入 ingest_failures 死信失败(entity=%s, record=%s): %s",
|
|
||||||
entity_type, source_record_id, error, exc_info=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _write_lineage(db, source_system: str, source_record_id: str, target_table: str, target_id: int | None, transform_name: str, transform_detail: dict | None = None, batch_id: str | None = None) -> None:
|
|
||||||
"""写入 ETL 血缘追踪。"""
|
|
||||||
db.add(DataLineage(
|
|
||||||
source_system=source_system,
|
|
||||||
source_record_id=source_record_id,
|
|
||||||
target_table=target_table,
|
|
||||||
target_id=target_id,
|
|
||||||
transform_name=transform_name,
|
|
||||||
transform_detail=transform_detail,
|
|
||||||
batch_id=batch_id,
|
|
||||||
))
|
|
||||||
|
|
||||||
|
|
||||||
def _events_record_id(league_code: str, nm, raw: dict) -> str:
|
|
||||||
"""events 载荷的 RawEvent 幂等键。
|
|
||||||
|
|
||||||
优先用上游 event id;缺失时用 (league:home:away:date) 合成稳定键 ——
|
|
||||||
取 normalize 后的队名与天级日期(与 _match_key 同口径),不依赖 DB 自增 id,
|
|
||||||
保证同一来源比赛重复采集时命中同一条 RawEvent,不产生重复原始载荷。
|
|
||||||
"""
|
|
||||||
eid = _to_int_or_none(raw.get("id"))
|
|
||||||
if eid is not None:
|
|
||||||
return str(eid)
|
|
||||||
d = _to_date(nm.date)
|
|
||||||
date_part = d.isoformat() if d is not None else "na"
|
|
||||||
return f"{league_code}:{nm.home_team}:{nm.away_team}:{date_part}"
|
|
||||||
|
|
||||||
|
|
||||||
async def _write_events_bronze(
|
|
||||||
db,
|
|
||||||
*,
|
|
||||||
source_record_id: str,
|
|
||||||
raw_payload: dict,
|
|
||||||
target_match_id: int | None,
|
|
||||||
league_code: str,
|
|
||||||
match_status: str | None,
|
|
||||||
batch_id: str,
|
|
||||||
) -> None:
|
|
||||||
"""events 成功插入/更新单场比赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
|
|
||||||
|
|
||||||
D1(工程债):此前只有 stats 回填写 RawEvent/Lineage,events 管线作为比赛
|
|
||||||
主数据的唯一入口反而不留溯源记录。幂等性由 _write_raw_event 的
|
|
||||||
source_record_id 查重保证;best-effort:基础设施写入失败只记 warning,
|
|
||||||
绝不拖垮采集主流程(与 _safe_write_ingest_failure 同级约束)。
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
|
|
||||||
await _write_lineage(
|
|
||||||
db, "bzzoiro", source_record_id,
|
|
||||||
"matches", target_match_id, "events_ingest",
|
|
||||||
{"league": league_code, "match_status": match_status},
|
|
||||||
batch_id,
|
|
||||||
)
|
|
||||||
except Exception:
|
|
||||||
logger.warning(
|
|
||||||
"events Bronze 写入失败(record=%s, match=%s),不影响采集主流程",
|
|
||||||
source_record_id, target_match_id, exc_info=True,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ============================================================
|
|
||||||
# 积分榜管线:/leagues/{id}/standings/ → standings 表
|
|
||||||
# ============================================================
|
|
||||||
|
|
||||||
async def fetch_bzzoiro_standings(league_code: str, season: str | None = None) -> dict:
|
|
||||||
"""抓取联赛积分榜(纯抓取,不入库)。season 为 None 时取当前赛季。"""
|
|
||||||
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
|
|
||||||
if league_id is None:
|
|
||||||
raise ValueError(f"未知联赛代码: {league_code}")
|
|
||||||
params: dict = {}
|
|
||||||
if season:
|
|
||||||
params["season"] = season
|
|
||||||
return await _fetch_json_async(f"/leagues/{league_id}/standings/", params)
|
|
||||||
|
|
||||||
|
|
||||||
def _season_label_from_dates(start_date, end_date) -> str:
|
|
||||||
"""从赛季起止日期推导赛季标签(与 derive_season_label 语义一致)。"""
|
|
||||||
try:
|
|
||||||
if isinstance(start_date, str):
|
|
||||||
start = datetime.fromisoformat(start_date[:10])
|
|
||||||
else:
|
|
||||||
start = start_date
|
|
||||||
y = start.year
|
|
||||||
return f"{y}-{y + 1}" if start.month >= 8 else f"{y - 1}-{y}"
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return "?"
|
|
||||||
|
|
||||||
|
|
||||||
async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str | None = None) -> dict:
|
|
||||||
"""采集积分榜 → upsert standings 表。
|
|
||||||
|
|
||||||
season 为 None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)。
|
|
||||||
球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配。
|
|
||||||
"""
|
|
||||||
from src.data.team_names import normalize as normalize_name
|
|
||||||
|
|
||||||
result: dict = {"leagues": {}, "total_upserted": 0, "errors": []}
|
|
||||||
for code in leagues:
|
|
||||||
league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []}
|
|
||||||
try:
|
|
||||||
payload = await fetch_bzzoiro_standings(code, season=season)
|
|
||||||
except Exception as e:
|
|
||||||
logger.exception("bzzoiro standings fetch failed for %s", code)
|
|
||||||
league_r["errors"].append(str(e))
|
|
||||||
await _safe_write_ingest_failure(
|
|
||||||
db,
|
|
||||||
entity_type="standings",
|
|
||||||
source_record_id=None,
|
|
||||||
error=e,
|
|
||||||
raw_payload={"league": code, "season": season},
|
|
||||||
)
|
|
||||||
result["leagues"][code] = league_r
|
|
||||||
result["errors"].append(f"{code}: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
rows = payload.get("standings") or []
|
|
||||||
if not rows:
|
|
||||||
result["leagues"][code] = {"error": "无积分榜数据(赛季未开始或未提供)"}
|
|
||||||
result["errors"].append(f"{code}: 无积分榜数据")
|
|
||||||
continue
|
|
||||||
|
|
||||||
# 联赛(get-or-create,D4: 经 LeagueRepository)
|
|
||||||
league = await LeagueRepository(db).get_or_create(
|
|
||||||
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
|
|
||||||
)
|
|
||||||
team_r = TeamRepository(db)
|
|
||||||
|
|
||||||
# 赛季标签:优先用返回的 season 对象推导
|
|
||||||
season_obj = payload.get("season") or {}
|
|
||||||
season_label = _season_label_from_dates(
|
|
||||||
season_obj.get("start_date"), season_obj.get("end_date")
|
|
||||||
)
|
|
||||||
if season_label == "?":
|
|
||||||
season_label = season or ""
|
|
||||||
|
|
||||||
# 批量预载球队(与 events 管线使用同一 normalize 规则,保证 Team 匹配)
|
|
||||||
names = {normalize_name(str(r.get("team_name", ""))) for r in rows}
|
|
||||||
names.discard("")
|
|
||||||
team_map: dict[str, Team] = await team_r.get_all_by_names(list(names))
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
|
||||||
for r in rows:
|
|
||||||
team_name = normalize_name(str(r.get("team_name", "")))
|
|
||||||
if not team_name:
|
|
||||||
continue
|
|
||||||
team = team_map.get(team_name)
|
|
||||||
if team is None:
|
|
||||||
team = await team_r.get_or_create(team_name, name_zh=zh_name(team_name))
|
|
||||||
team_map[team_name] = team
|
|
||||||
league_r["teams_created"] += 1
|
|
||||||
|
|
||||||
zone = r.get("zone") or {}
|
|
||||||
values = dict(
|
|
||||||
position=_to_int_or_none(r.get("position")) or 0,
|
|
||||||
played=_to_int_or_none(r.get("played")) or 0,
|
|
||||||
won=_to_int_or_none(r.get("won")) or 0,
|
|
||||||
drawn=_to_int_or_none(r.get("drawn")) or 0,
|
|
||||||
lost=_to_int_or_none(r.get("lost")) or 0,
|
|
||||||
goals_for=_to_int_or_none(r.get("gf")) or 0,
|
|
||||||
goals_against=_to_int_or_none(r.get("ga")) or 0,
|
|
||||||
goal_diff=_to_int_or_none(r.get("gd")) or 0,
|
|
||||||
points=_to_int_or_none(r.get("pts")) or 0,
|
|
||||||
xg_for=_to_float_or_none(r.get("xgf")),
|
|
||||||
xg_against=_to_float_or_none(r.get("xga")),
|
|
||||||
form=r.get("form") or None,
|
|
||||||
zone=zone.get("label") or zone.get("key") or None,
|
|
||||||
updated_at=now,
|
|
||||||
retrieved_at=now,
|
|
||||||
)
|
|
||||||
|
|
||||||
# 同一联赛同一赛季只保留最新快照:按 (league, season, team) upsert
|
|
||||||
stmt = select(Standing).where(
|
|
||||||
Standing.league_id == league.id,
|
|
||||||
Standing.season == season_label,
|
|
||||||
Standing.team_id == team.id,
|
|
||||||
)
|
|
||||||
standing = (await db.execute(stmt)).scalar_one_or_none()
|
|
||||||
if standing is None:
|
|
||||||
standing = Standing(
|
|
||||||
league_id=league.id, season=season_label, team_id=team.id, **values
|
|
||||||
)
|
|
||||||
db.add(standing)
|
|
||||||
else:
|
|
||||||
for k, v in values.items():
|
|
||||||
setattr(standing, k, v)
|
|
||||||
league_r["upserted"] += 1
|
|
||||||
|
|
||||||
league_r["rows"] = len(rows)
|
|
||||||
result["leagues"][code] = league_r
|
|
||||||
result["total_upserted"] += league_r["upserted"]
|
|
||||||
logger.info(
|
|
||||||
"bzzoiro standings 采集完成: %s 赛季 %s, upsert %d/%d",
|
|
||||||
code, season_label, league_r["upserted"], league_r["rows"],
|
|
||||||
)
|
|
||||||
return result
|
|
||||||
|
|
||||||
|
|
||||||
# ============================================================
|
|
||||||
# 统计回填管线:/events/{id}/stats/ → match_stats 表
|
|
||||||
# ============================================================
|
|
||||||
|
|
||||||
# bzzoiro stats 字段 → MatchStats 字段映射(stats.home / stats.away 下)
|
|
||||||
_STATS_FIELD_MAP = {
|
|
||||||
"xg": ("home_xg", "away_xg"), # 回退 expected_goals
|
|
||||||
"ball_possession": ("home_possession", None), # 只取主队值,客队=100-home
|
|
||||||
"total_shots": ("home_shots", "away_shots"),
|
|
||||||
"shots_on_target": ("home_shots_on_target", "away_shots_on_target"),
|
|
||||||
"corner_kicks": ("home_corners", "away_corners"),
|
|
||||||
"yellow_cards": ("home_yellow_cards", "away_yellow_cards"),
|
|
||||||
"red_cards": ("home_red_cards", "away_red_cards"),
|
|
||||||
"big_chances": ("home_big_chances", "away_big_chances"),
|
|
||||||
"fouls": ("home_fouls", "away_fouls"),
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _pick(d: dict, *keys):
|
|
||||||
"""按优先级取第一个非空字段值。"""
|
|
||||||
for k in keys:
|
|
||||||
v = d.get(k)
|
|
||||||
if v is not None:
|
|
||||||
return v
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _stats_from_payload(payload: dict) -> dict:
|
|
||||||
"""把 /events/{id}/stats/ 响应映射成 MatchStats 字段 dict。
|
|
||||||
|
|
||||||
响应结构: {"event_id": ..., "stats": {"home": {...}, "away": {...}}}
|
|
||||||
"""
|
|
||||||
stats = (payload or {}).get("stats") or {}
|
|
||||||
home = stats.get("home") or {}
|
|
||||||
away = stats.get("away") or {}
|
|
||||||
out: dict = {}
|
|
||||||
|
|
||||||
xg_h = _pick(home, "xg", "expected_goals")
|
|
||||||
xg_a = _pick(away, "xg", "expected_goals")
|
|
||||||
if xg_h is not None:
|
|
||||||
out["home_xg"] = _to_float_or_none(xg_h)
|
|
||||||
if xg_a is not None:
|
|
||||||
out["away_xg"] = _to_float_or_none(xg_a)
|
|
||||||
|
|
||||||
poss = home.get("ball_possession")
|
|
||||||
if poss is not None:
|
|
||||||
p = _to_float_or_none(poss)
|
|
||||||
if p is not None:
|
|
||||||
out["home_possession"] = p
|
|
||||||
|
|
||||||
for src, (h_fld, a_fld) in _STATS_FIELD_MAP.items():
|
|
||||||
if src in ("xg", "ball_possession"):
|
|
||||||
continue # 已处理
|
|
||||||
hv = home.get(src)
|
|
||||||
av = away.get(src)
|
|
||||||
if hv is not None and h_fld:
|
|
||||||
out[h_fld] = _to_int_or_none(hv)
|
|
||||||
if av is not None and a_fld:
|
|
||||||
out[a_fld] = _to_int_or_none(av)
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def _to_float_or_none(value) -> float | None:
|
|
||||||
if value is None:
|
|
||||||
return None
|
|
||||||
try:
|
|
||||||
return float(str(value).strip())
|
|
||||||
except (TypeError, ValueError):
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
async def ingest_bzzoiro_event_stats(
|
|
||||||
db,
|
|
||||||
*,
|
|
||||||
leagues: Iterable[str],
|
|
||||||
limit: int = 100,
|
|
||||||
only_missing: bool = True,
|
|
||||||
) -> dict:
|
|
||||||
"""回填已完赛比赛的详细统计(逐场调 /events/{id}/stats/)。
|
|
||||||
|
|
||||||
筛选条件: match_status=finished 且 source_event_id 非空。
|
|
||||||
only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新。
|
|
||||||
limit 控制单次最多处理的比赛数(上游限速 1.2s/请求,大批量需分次触发)。
|
|
||||||
"""
|
|
||||||
result: dict = {"fetched": 0, "created": 0, "updated": 0, "skipped": 0, "errors": []}
|
|
||||||
|
|
||||||
league_ids = [BZZOIRO_LEAGUE_IDS[c] for c in leagues if c in BZZOIRO_LEAGUE_IDS]
|
|
||||||
if not league_ids:
|
|
||||||
result["errors"].append("无有效联赛代码")
|
|
||||||
return result
|
|
||||||
|
|
||||||
# D4: 候选比赛查询经 MatchRepository(含 stats 预加载,筛选/排序/limit 语义不变)
|
|
||||||
matches = await MatchRepository(db).find_finished_with_stats(
|
|
||||||
league_ids, limit=limit * 3 if only_missing else limit
|
|
||||||
)
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc)
|
|
||||||
processed = 0
|
|
||||||
for m in matches:
|
|
||||||
if processed >= limit:
|
|
||||||
break
|
|
||||||
if only_missing and m.stats is not None and m.stats.home_shots is not None:
|
|
||||||
result["skipped"] += 1
|
|
||||||
continue
|
|
||||||
processed += 1
|
|
||||||
try:
|
|
||||||
payload = await _fetch_json_async(f"/events/{m.source_event_id}/stats/")
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e)
|
|
||||||
result["errors"].append(f"match {m.id}: {e}")
|
|
||||||
await _safe_write_ingest_failure(
|
|
||||||
db,
|
|
||||||
entity_type="match_stats",
|
|
||||||
source_record_id=str(m.source_event_id),
|
|
||||||
error=e,
|
|
||||||
raw_payload={"match_id": m.id},
|
|
||||||
)
|
|
||||||
await asyncio.sleep(REQUEST_INTERVAL)
|
|
||||||
continue
|
|
||||||
|
|
||||||
result["fetched"] += 1
|
|
||||||
fields = _stats_from_payload(payload)
|
|
||||||
if not fields:
|
|
||||||
result["skipped"] += 1
|
|
||||||
await asyncio.sleep(REQUEST_INTERVAL)
|
|
||||||
continue
|
|
||||||
|
|
||||||
if m.stats is None:
|
|
||||||
available_at = m.match_date + timedelta(hours=2) if m.match_date else now
|
|
||||||
m.stats = MatchStats(
|
|
||||||
match_id=m.id,
|
|
||||||
source="bzzoiro",
|
|
||||||
source_record_id=str(m.source_event_id),
|
|
||||||
retrieved_at=now,
|
|
||||||
available_at=available_at,
|
|
||||||
)
|
|
||||||
db.add(m.stats)
|
|
||||||
result["created"] += 1
|
|
||||||
else:
|
|
||||||
result["updated"] += 1
|
|
||||||
if m.stats.source is None:
|
|
||||||
m.stats.source = "bzzoiro"
|
|
||||||
m.stats.source_record_id = str(m.source_event_id)
|
|
||||||
if m.stats.retrieved_at is None:
|
|
||||||
m.stats.retrieved_at = now
|
|
||||||
if m.stats.available_at is None and m.match_date:
|
|
||||||
m.stats.available_at = m.match_date + timedelta(hours=2)
|
|
||||||
|
|
||||||
for fld, v in fields.items():
|
|
||||||
if hasattr(m.stats, fld):
|
|
||||||
setattr(m.stats, fld, v)
|
|
||||||
|
|
||||||
# 管线基础设施:写入 RawEvent + DataLineage
|
|
||||||
batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}"
|
|
||||||
try:
|
|
||||||
await _write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id)
|
|
||||||
await _write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id)
|
|
||||||
except Exception:
|
|
||||||
pass # 基础设施写入失败不影响主流程
|
|
||||||
|
|
||||||
await asyncio.sleep(REQUEST_INTERVAL)
|
|
||||||
|
|
||||||
logger.info(
|
|
||||||
"bzzoiro stats 回填完成: 抓取 %d, 新建 %d, 更新 %d, 跳过 %d, 错误 %d",
|
|
||||||
result["fetched"], result["created"], result["updated"],
|
|
||||||
result["skipped"], len(result["errors"]),
|
|
||||||
)
|
|
||||||
return result
|
|
||||||
|
|||||||
@@ -0,0 +1,123 @@
|
|||||||
|
"""bzzoiro 管线共享原语:HTTP 抓取(多 key 轮换)与宽松字段转换。
|
||||||
|
|
||||||
|
从 bzzoiro.py 拆出(单文件 → 多模块):仅放无业务语义的共享基础,
|
||||||
|
三条管线(events/standings/stats)与聚合门面见 bzzoiro.py。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
import random
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from src.core.http_client import get_client
|
||||||
|
from src.core.runtime_config import get_runtime_value
|
||||||
|
from src.data.key_ring import _mask, get_key_ring
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
def _to_date(value):
|
||||||
|
"""把 datetime / date / str 统一成 `date`。"""
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
if hasattr(value, "date") and callable(value.date):
|
||||||
|
return value.date()
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _to_int_or_none(value) -> int | None:
|
||||||
|
"""宽松转 int(用于上游 ID 解析,失败返回 None 不抛错)。"""
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return int(str(value).strip())
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _to_float_or_none(value) -> float | None:
|
||||||
|
try:
|
||||||
|
return float(str(value).strip())
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _match_key(home_team_id: int, away_team_id: int, match_date) -> tuple[int, int, str]:
|
||||||
|
"""比赛去重键:(主队, 客队, 天级日期 ISO 字符串)。
|
||||||
|
|
||||||
|
统一在这里构造,避免"预加载时用 str(date)、写入时用 isoformat()"这类
|
||||||
|
隐式格式依赖 —— 两者当前恰好相等,但一旦有人改动其一就会静默失配,
|
||||||
|
导致所有比赛被判为不存在而重复插入。
|
||||||
|
"""
|
||||||
|
d = _to_date(match_date)
|
||||||
|
return (home_team_id, away_team_id, d.isoformat() if d is not None else "")
|
||||||
|
|
||||||
|
|
||||||
|
async def _fetch_json_async(path: str, params: dict | None = None, max_retries: int = 3) -> dict | list:
|
||||||
|
"""异步 HTTP(bzzoiro 使用 httpx,不再阻塞事件循环线程池)。
|
||||||
|
|
||||||
|
多 key 轮换:遇到 429 自动切换到下一个 key;全部 key 冷却时等待最早恢复。
|
||||||
|
"""
|
||||||
|
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
|
||||||
|
raw_keys = await get_runtime_value("BZZOIRO_KEY")
|
||||||
|
ring = get_key_ring(base, raw_keys)
|
||||||
|
|
||||||
|
url = f"{base}/{path.lstrip('/')}"
|
||||||
|
key = ring.get()
|
||||||
|
if not key:
|
||||||
|
raise RuntimeError("BZZOIRO_KEY 未设置")
|
||||||
|
|
||||||
|
last_exc: Exception | None = None
|
||||||
|
for attempt in range(max_retries):
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Token {key}",
|
||||||
|
"Accept": "application/json",
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
client = get_client()
|
||||||
|
# 整请求兜底: httpx 无 total 超时,用 wait_for 防「滴水式」限速挂死
|
||||||
|
resp = await asyncio.wait_for(
|
||||||
|
client.get(
|
||||||
|
url, headers=headers, params=params,
|
||||||
|
timeout=httpx.Timeout(connect=10.0, read=30.0, write=10.0, pool=10.0),
|
||||||
|
),
|
||||||
|
timeout=60.0,
|
||||||
|
)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return resp.json()
|
||||||
|
except Exception as e:
|
||||||
|
last_exc = e
|
||||||
|
status = getattr(getattr(e, "response", None), "status_code", None)
|
||||||
|
if status == 429:
|
||||||
|
# 限流:标记当前 key 冷却,切换到下一个
|
||||||
|
new_key = ring.report_rate_limited(key)
|
||||||
|
if new_key and new_key != key:
|
||||||
|
logger.info("bzzoiro 429 → 切换 key: %s → %s,立即重试", _mask(key), _mask(new_key))
|
||||||
|
key = new_key
|
||||||
|
continue # 立即重试,不等待
|
||||||
|
# 单 key 或全部冷却:等待最早恢复的 key
|
||||||
|
wait = ring.wait_if_all_blocked()
|
||||||
|
if wait > 0:
|
||||||
|
logger.warning("bzzoiro 全部 key 冷却,等待 %.1fs 后重试", wait)
|
||||||
|
await asyncio.sleep(min(wait, 30.0))
|
||||||
|
else:
|
||||||
|
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
||||||
|
logger.warning("bzzoiro 429, retry %d in %.1fs", attempt + 1, delay)
|
||||||
|
await asyncio.sleep(delay)
|
||||||
|
key = ring.get() or key
|
||||||
|
continue
|
||||||
|
if 500 <= (status or 0) < 600:
|
||||||
|
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
||||||
|
logger.warning("bzzoiro %d, retry %d in %.1fs", status, attempt + 1, delay)
|
||||||
|
await asyncio.sleep(delay)
|
||||||
|
continue
|
||||||
|
# 网络错误(连接失败/超时)也退避重试
|
||||||
|
if isinstance(e, (TimeoutError, ConnectionError, OSError)):
|
||||||
|
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
|
||||||
|
logger.warning("bzzoiro network error, retry %d in %.1fs: %s", attempt + 1, delay, e)
|
||||||
|
await asyncio.sleep(delay)
|
||||||
|
continue
|
||||||
|
raise
|
||||||
|
raise RuntimeError(f"bzzoiro request failed after {max_retries} attempts: {last_exc}")
|
||||||
@@ -0,0 +1,310 @@
|
|||||||
|
"""bzzoiro events 管线:比赛日程/比分抓取(/events/)与入库(matches 表)。
|
||||||
|
|
||||||
|
从 bzzoiro.py 拆出。比赛主数据唯一入口;Team/League/Match 查找/创建经
|
||||||
|
Repository 层,事务由调用方 UnitOfWork 控制(分批事务约定不变)。
|
||||||
|
|
||||||
|
可替换协作者(抓取函数 / Bronze 写入助手 / REQUEST_INTERVAL)在运行期
|
||||||
|
经聚合门面 src.data.bzzoiro 解析 —— 与拆分前的单文件 monkeypatch 语义一致。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
from src.data.bzzoiro_common import _match_key, _to_date, _to_int_or_none
|
||||||
|
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
|
||||||
|
from src.data.normalize import normalize_bzzoiro
|
||||||
|
from src.data.sources import register
|
||||||
|
from src.data.team_names_zh import zh_name
|
||||||
|
from src.db.models import Match
|
||||||
|
from src.db.repositories import LeagueRepository, MatchRepository, TeamRepository
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def fetch_bzzoiro_events(
|
||||||
|
league_code: str,
|
||||||
|
*,
|
||||||
|
status: str = "finished",
|
||||||
|
date_from: str | None = None,
|
||||||
|
date_to: str | None = None,
|
||||||
|
limit: int = 200,
|
||||||
|
) -> list[dict]:
|
||||||
|
"""抓取 bzzoiro 原始事件(纯异步,无需 run_in_executor)。"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
|
||||||
|
if league_id is None:
|
||||||
|
raise ValueError(f"未知联赛代码: {league_code}")
|
||||||
|
|
||||||
|
rows: list[dict] = []
|
||||||
|
offset = 0
|
||||||
|
while True:
|
||||||
|
params: dict = {
|
||||||
|
"league_id": league_id,
|
||||||
|
"status": status,
|
||||||
|
"limit": limit,
|
||||||
|
"offset": offset,
|
||||||
|
}
|
||||||
|
if date_from:
|
||||||
|
params["date_from"] = str(date_from)[:10]
|
||||||
|
if date_to:
|
||||||
|
params["date_to"] = str(date_to)[:10]
|
||||||
|
payload = await bz._fetch_json_async("/events/", params)
|
||||||
|
batch = payload.get("results") or []
|
||||||
|
if not batch:
|
||||||
|
break
|
||||||
|
rows.extend(batch)
|
||||||
|
total = payload.get("total")
|
||||||
|
offset += limit
|
||||||
|
if total is not None and offset >= total:
|
||||||
|
break
|
||||||
|
if len(batch) < limit:
|
||||||
|
break
|
||||||
|
await asyncio.sleep(bz.REQUEST_INTERVAL)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
@register
|
||||||
|
class BzzoiroSource:
|
||||||
|
"""bzzoiro 数据源(实现 DataSource 协议)。"""
|
||||||
|
|
||||||
|
name = "bzzoiro"
|
||||||
|
|
||||||
|
async def ingest(
|
||||||
|
self,
|
||||||
|
db,
|
||||||
|
*,
|
||||||
|
leagues: Iterable[str],
|
||||||
|
date_from: str | None = None,
|
||||||
|
date_to: str | None = None,
|
||||||
|
status: str = "finished",
|
||||||
|
) -> dict:
|
||||||
|
"""采集 bzzoiro → 入库。返回统计。
|
||||||
|
|
||||||
|
注意: 本方法不控制事务(commit/rollback),由调用方通过 UnitOfWork 控制。
|
||||||
|
"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
result: dict = {"leagues": {}, "total_inserted": 0, "total_updated": 0, "errors": []}
|
||||||
|
|
||||||
|
for code in leagues:
|
||||||
|
league_r: dict = {"inserted": 0, "updated": 0, "errors": []}
|
||||||
|
try:
|
||||||
|
raw_events = await bz.fetch_bzzoiro_events(code, status=status, date_from=date_from, date_to=date_to)
|
||||||
|
except Exception as e:
|
||||||
|
# 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批
|
||||||
|
logger.exception("bzzoiro fetch failed for %s", code)
|
||||||
|
league_r["errors"].append(f"fetch failed: {e}")
|
||||||
|
await bz._safe_write_ingest_failure(
|
||||||
|
db,
|
||||||
|
entity_type="events",
|
||||||
|
source_record_id=None,
|
||||||
|
error=e,
|
||||||
|
raw_payload={"league": code, "status": status, "date_from": date_from, "date_to": date_to},
|
||||||
|
)
|
||||||
|
result["leagues"][code] = league_r
|
||||||
|
continue
|
||||||
|
|
||||||
|
# D4: 联赛查找/创建经 LeagueRepository(事务仍由调用方 UoW 提交)
|
||||||
|
league = await LeagueRepository(db).get_or_create(
|
||||||
|
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
|
||||||
|
)
|
||||||
|
team_r = TeamRepository(db)
|
||||||
|
match_r = MatchRepository(db)
|
||||||
|
|
||||||
|
# === 批量优化: 预加载球队和已有比赛到内存 ===
|
||||||
|
team_name_to_id: dict[str, int] = {}
|
||||||
|
existing_matches: dict[tuple[int, int, str], Match] = {} # 完整对象,避免重复查询
|
||||||
|
# (NormalizedMatch, 原始 event) 成对保存:后续写 source_event_id 时
|
||||||
|
# 必须用配对的那条 event,不能依赖外层循环变量残留值。
|
||||||
|
normalized_matches: list[tuple] = []
|
||||||
|
|
||||||
|
if raw_events:
|
||||||
|
# 一次遍历: 收集球队名 + 规范化
|
||||||
|
all_team_names = set()
|
||||||
|
for raw in raw_events:
|
||||||
|
nm = normalize_bzzoiro(raw, code)
|
||||||
|
if nm is not None:
|
||||||
|
try:
|
||||||
|
nm.validate()
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
normalized_matches.append((nm, raw))
|
||||||
|
all_team_names.add(nm.home_team)
|
||||||
|
all_team_names.add(nm.away_team)
|
||||||
|
|
||||||
|
if all_team_names:
|
||||||
|
team_name_to_id = {
|
||||||
|
name: t.id
|
||||||
|
for name, t in (await team_r.get_all_by_names(list(all_team_names))).items()
|
||||||
|
}
|
||||||
|
|
||||||
|
# P1-2: 按需加载,只加载 raw_events 涉及日期范围的比赛(加 30 天缓冲)
|
||||||
|
# 避免加载联赛全部历史比赛到内存(多赛季采集时内存溢出)
|
||||||
|
if normalized_matches:
|
||||||
|
# normalized_matches 存的是 (nm, raw) 元组,遍历需解包
|
||||||
|
dates = [nm.date for nm, _raw in normalized_matches if nm.date is not None]
|
||||||
|
if dates:
|
||||||
|
min_dt = min(dates) - timedelta(days=30)
|
||||||
|
max_dt = max(dates) + timedelta(days=30)
|
||||||
|
matches_in_range = await match_r.find_by_league_and_date_range(
|
||||||
|
league.id, min_dt, max_dt
|
||||||
|
)
|
||||||
|
existing_matches = {
|
||||||
|
_match_key(m.home_team_id, m.away_team_id, m.match_date_date): m
|
||||||
|
for m in matches_in_range
|
||||||
|
}
|
||||||
|
# else: existing_matches 保持空 dict(全量新比赛)
|
||||||
|
|
||||||
|
# D1: Bronze 层批次信息(每联赛每批次一个 batch_id;seen 防同批重复写入)
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
bronze_batch_id = f"bzzoiro-events-{code}-{now:%Y%m%d%H%M%S}"
|
||||||
|
bronze_written: set[str] = set()
|
||||||
|
|
||||||
|
for nm, raw in normalized_matches:
|
||||||
|
# D1: RawEvent 幂等键(上游 id 或合成键),插入/变更更新共用
|
||||||
|
record_id = _events_record_id(code, nm, raw)
|
||||||
|
|
||||||
|
# 球队: 内存查找 + 按需创建(D4: 经 TeamRepository)
|
||||||
|
home_team_id = team_name_to_id.get(nm.home_team)
|
||||||
|
if home_team_id is None:
|
||||||
|
home = await team_r.get_or_create(nm.home_team, name_zh=zh_name(nm.home_team))
|
||||||
|
home_team_id = home.id
|
||||||
|
team_name_to_id[nm.home_team] = home_team_id
|
||||||
|
|
||||||
|
away_team_id = team_name_to_id.get(nm.away_team)
|
||||||
|
if away_team_id is None:
|
||||||
|
away = await team_r.get_or_create(nm.away_team, name_zh=zh_name(nm.away_team))
|
||||||
|
away_team_id = away.id
|
||||||
|
team_name_to_id[nm.away_team] = away_team_id
|
||||||
|
|
||||||
|
# 查找已有比赛: 内存查找
|
||||||
|
match_key = _match_key(home_team_id, away_team_id, nm.date)
|
||||||
|
existing_match = existing_matches.get(match_key)
|
||||||
|
|
||||||
|
if existing_match is None:
|
||||||
|
m = Match(
|
||||||
|
league_id=league.id,
|
||||||
|
season=nm.season_label or None,
|
||||||
|
home_team_id=home_team_id,
|
||||||
|
away_team_id=away_team_id,
|
||||||
|
match_date=nm.date,
|
||||||
|
match_date_date=_to_date(nm.date),
|
||||||
|
match_status=nm.match_status,
|
||||||
|
home_goals=nm.home_goals,
|
||||||
|
away_goals=nm.away_goals,
|
||||||
|
home_ht_goals=nm.home_ht_goals,
|
||||||
|
away_ht_goals=nm.away_ht_goals,
|
||||||
|
match_stage=nm.match_stage,
|
||||||
|
source_event_id=_to_int_or_none(raw.get("id")),
|
||||||
|
)
|
||||||
|
db.add(m)
|
||||||
|
await db.flush()
|
||||||
|
existing_matches[match_key] = m # 防止同批重复
|
||||||
|
# 统计字段不在 /events/ 载荷中(单独由 stats 管线回填),
|
||||||
|
# 此处不再创建 MatchStats。
|
||||||
|
league_r["inserted"] += 1
|
||||||
|
# D1: 成功插入 → 补写 Bronze 层(原始载荷 + 血缘)
|
||||||
|
if record_id not in bronze_written:
|
||||||
|
bronze_written.add(record_id)
|
||||||
|
await _write_events_bronze(
|
||||||
|
db,
|
||||||
|
source_record_id=record_id,
|
||||||
|
raw_payload=raw,
|
||||||
|
target_match_id=m.id,
|
||||||
|
league_code=code,
|
||||||
|
match_status=nm.match_status,
|
||||||
|
batch_id=bronze_batch_id,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
# 已有比赛: 直接从内存获取对象更新(无需再查询)
|
||||||
|
changed = False
|
||||||
|
if existing_match.match_status != nm.match_status and nm.match_status == "finished":
|
||||||
|
existing_match.match_status = nm.match_status
|
||||||
|
changed = True
|
||||||
|
if existing_match.home_goals is None and nm.home_goals is not None:
|
||||||
|
existing_match.home_goals = nm.home_goals
|
||||||
|
existing_match.away_goals = nm.away_goals
|
||||||
|
existing_match.home_ht_goals = nm.home_ht_goals
|
||||||
|
existing_match.away_ht_goals = nm.away_ht_goals
|
||||||
|
changed = True
|
||||||
|
if existing_match.match_stage is None and nm.match_stage:
|
||||||
|
existing_match.match_stage = nm.match_stage
|
||||||
|
changed = True
|
||||||
|
if existing_match.source_event_id is None:
|
||||||
|
eid = _to_int_or_none(raw.get("id"))
|
||||||
|
if eid is not None:
|
||||||
|
existing_match.source_event_id = eid
|
||||||
|
changed = True
|
||||||
|
if changed:
|
||||||
|
league_r["updated"] += 1
|
||||||
|
# D1: 变更更新 → 补写血缘(RawEvent 幂等键不变,重复采集自动跳过)
|
||||||
|
if record_id not in bronze_written:
|
||||||
|
bronze_written.add(record_id)
|
||||||
|
await _write_events_bronze(
|
||||||
|
db,
|
||||||
|
source_record_id=record_id,
|
||||||
|
raw_payload=raw,
|
||||||
|
target_match_id=existing_match.id,
|
||||||
|
league_code=code,
|
||||||
|
match_status=nm.match_status,
|
||||||
|
batch_id=bronze_batch_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 注意: 不在此处 commit,由调用方 UnitOfWork 控制事务
|
||||||
|
result["leagues"][code] = league_r
|
||||||
|
result["total_inserted"] += league_r["inserted"]
|
||||||
|
result["total_updated"] += league_r["updated"]
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def _events_record_id(league_code: str, nm, raw: dict) -> str:
|
||||||
|
"""events 载荷的 RawEvent 幂等键。
|
||||||
|
|
||||||
|
优先用上游 event id;缺失时用 (league:home:away:date) 合成稳定键 ——
|
||||||
|
取 normalize 后的队名与天级日期(与 _match_key 同口径),不依赖 DB 自增 id,
|
||||||
|
保证同一来源比赛重复采集时命中同一条 RawEvent,不产生重复原始载荷。
|
||||||
|
"""
|
||||||
|
eid = _to_int_or_none(raw.get("id"))
|
||||||
|
if eid is not None:
|
||||||
|
return str(eid)
|
||||||
|
d = _to_date(nm.date)
|
||||||
|
date_part = d.isoformat() if d is not None else "na"
|
||||||
|
return f"{league_code}:{nm.home_team}:{nm.away_team}:{date_part}"
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_events_bronze(
|
||||||
|
db,
|
||||||
|
*,
|
||||||
|
source_record_id: str,
|
||||||
|
raw_payload: dict,
|
||||||
|
target_match_id: int | None,
|
||||||
|
league_code: str,
|
||||||
|
match_status: str | None,
|
||||||
|
batch_id: str,
|
||||||
|
) -> None:
|
||||||
|
"""events 成功插入/更新单场比赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
|
||||||
|
|
||||||
|
D1(工程债):此前只有 stats 回填写 RawEvent/Lineage,events 管线作为比赛
|
||||||
|
主数据的唯一入口反而不留溯源记录。幂等性由 _write_raw_event 的
|
||||||
|
source_record_id 查重保证;best-effort:基础设施写入失败只记 warning,
|
||||||
|
绝不拖垮采集主流程(与 _safe_write_ingest_failure 同级约束)。
|
||||||
|
"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
try:
|
||||||
|
await bz._write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
|
||||||
|
await bz._write_lineage(
|
||||||
|
db, "bzzoiro", source_record_id,
|
||||||
|
"matches", target_match_id, "events_ingest",
|
||||||
|
{"league": league_code, "match_status": match_status},
|
||||||
|
batch_id,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"events Bronze 写入失败(record=%s, match=%s),不影响采集主流程",
|
||||||
|
source_record_id, target_match_id, exc_info=True,
|
||||||
|
)
|
||||||
@@ -0,0 +1,215 @@
|
|||||||
|
"""bzzoiro standings 管线:联赛积分榜快照(/leagues/{id}/standings/)→ standings 表。
|
||||||
|
|
||||||
|
从 bzzoiro.py 拆出。同一联赛同一赛季只保留最新快照(按 (league, season, team)
|
||||||
|
upsert);球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配。
|
||||||
|
|
||||||
|
可替换协作者(抓取函数 / Bronze 写入助手)在运行期经聚合门面
|
||||||
|
src.data.bzzoiro 解析 —— 与拆分前的单文件 monkeypatch 语义一致。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
from sqlalchemy import select
|
||||||
|
|
||||||
|
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
|
||||||
|
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
|
||||||
|
from src.data.team_names_zh import zh_name
|
||||||
|
from src.db.models import Standing, Team
|
||||||
|
from src.db.repositories import LeagueRepository, TeamRepository
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def fetch_bzzoiro_standings(league_code: str, season: str | None = None) -> dict:
|
||||||
|
"""抓取联赛积分榜(纯抓取,不入库)。season 为 None 时取当前赛季。"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
|
||||||
|
if league_id is None:
|
||||||
|
raise ValueError(f"未知联赛代码: {league_code}")
|
||||||
|
params: dict = {}
|
||||||
|
if season:
|
||||||
|
params["season"] = season
|
||||||
|
return await bz._fetch_json_async(f"/leagues/{league_id}/standings/", params)
|
||||||
|
|
||||||
|
|
||||||
|
def _season_label_from_dates(start_date, end_date) -> str:
|
||||||
|
"""从赛季起止日期推导赛季标签(与 derive_season_label 语义一致)。"""
|
||||||
|
try:
|
||||||
|
if isinstance(start_date, str):
|
||||||
|
start = datetime.fromisoformat(start_date[:10])
|
||||||
|
else:
|
||||||
|
start = start_date
|
||||||
|
if start is None:
|
||||||
|
return "?"
|
||||||
|
y = start.year
|
||||||
|
return f"{y}-{y + 1}" if start.month >= 8 else f"{y - 1}-{y}"
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return "?"
|
||||||
|
|
||||||
|
|
||||||
|
async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str | None = None) -> dict:
|
||||||
|
"""采集积分榜 → upsert standings 表。
|
||||||
|
|
||||||
|
season 为 None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)。
|
||||||
|
球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配。
|
||||||
|
"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
from src.data.team_names import normalize as normalize_name
|
||||||
|
|
||||||
|
result: dict = {"leagues": {}, "total_upserted": 0, "errors": []}
|
||||||
|
for code in leagues:
|
||||||
|
league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []}
|
||||||
|
try:
|
||||||
|
payload = await bz.fetch_bzzoiro_standings(code, season=season)
|
||||||
|
except Exception as e:
|
||||||
|
logger.exception("bzzoiro standings fetch failed for %s", code)
|
||||||
|
league_r["errors"].append(str(e))
|
||||||
|
await bz._safe_write_ingest_failure(
|
||||||
|
db,
|
||||||
|
entity_type="standings",
|
||||||
|
source_record_id=None,
|
||||||
|
error=e,
|
||||||
|
raw_payload={"league": code, "season": season},
|
||||||
|
)
|
||||||
|
result["leagues"][code] = league_r
|
||||||
|
result["errors"].append(f"{code}: {e}")
|
||||||
|
continue
|
||||||
|
|
||||||
|
rows = payload.get("standings") or []
|
||||||
|
if not rows:
|
||||||
|
result["leagues"][code] = {"error": "无积分榜数据(赛季未开始或未提供)"}
|
||||||
|
result["errors"].append(f"{code}: 无积分榜数据")
|
||||||
|
continue
|
||||||
|
|
||||||
|
# 联赛(get-or-create,D4: 经 LeagueRepository)
|
||||||
|
league = await LeagueRepository(db).get_or_create(
|
||||||
|
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
|
||||||
|
)
|
||||||
|
team_r = TeamRepository(db)
|
||||||
|
|
||||||
|
# 赛季标签:优先用返回的 season 对象推导
|
||||||
|
season_obj = payload.get("season") or {}
|
||||||
|
season_label = _season_label_from_dates(
|
||||||
|
season_obj.get("start_date"), season_obj.get("end_date")
|
||||||
|
)
|
||||||
|
if season_label == "?":
|
||||||
|
season_label = season or ""
|
||||||
|
|
||||||
|
# 批量预载球队(与 events 管线使用同一 normalize 规则,保证 Team 匹配)
|
||||||
|
names = {normalize_name(str(r.get("team_name", ""))) for r in rows}
|
||||||
|
names.discard("")
|
||||||
|
team_map: dict[str, Team] = await team_r.get_all_by_names(list(names))
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
for r in rows:
|
||||||
|
team_name = normalize_name(str(r.get("team_name", "")))
|
||||||
|
if not team_name:
|
||||||
|
continue
|
||||||
|
team = team_map.get(team_name)
|
||||||
|
if team is None:
|
||||||
|
team = await team_r.get_or_create(team_name, name_zh=zh_name(team_name))
|
||||||
|
team_map[team_name] = team
|
||||||
|
league_r["teams_created"] += 1
|
||||||
|
|
||||||
|
zone = r.get("zone") or {}
|
||||||
|
values = dict(
|
||||||
|
position=_to_int_or_none(r.get("position")) or 0,
|
||||||
|
played=_to_int_or_none(r.get("played")) or 0,
|
||||||
|
won=_to_int_or_none(r.get("won")) or 0,
|
||||||
|
drawn=_to_int_or_none(r.get("drawn")) or 0,
|
||||||
|
lost=_to_int_or_none(r.get("lost")) or 0,
|
||||||
|
goals_for=_to_int_or_none(r.get("gf")) or 0,
|
||||||
|
goals_against=_to_int_or_none(r.get("ga")) or 0,
|
||||||
|
goal_diff=_to_int_or_none(r.get("gd")) or 0,
|
||||||
|
points=_to_int_or_none(r.get("pts")) or 0,
|
||||||
|
xg_for=_to_float_or_none(r.get("xgf")),
|
||||||
|
xg_against=_to_float_or_none(r.get("xga")),
|
||||||
|
form=r.get("form") or None,
|
||||||
|
zone=zone.get("label") or zone.get("key") or None,
|
||||||
|
updated_at=now,
|
||||||
|
retrieved_at=now,
|
||||||
|
)
|
||||||
|
|
||||||
|
# 同一联赛同一赛季只保留最新快照:按 (league, season, team) upsert
|
||||||
|
stmt = select(Standing).where(
|
||||||
|
Standing.league_id == league.id,
|
||||||
|
Standing.season == season_label,
|
||||||
|
Standing.team_id == team.id,
|
||||||
|
)
|
||||||
|
standing = (await db.execute(stmt)).scalar_one_or_none()
|
||||||
|
if standing is None:
|
||||||
|
standing = Standing(
|
||||||
|
league_id=league.id, season=season_label, team_id=team.id, **values
|
||||||
|
)
|
||||||
|
db.add(standing)
|
||||||
|
else:
|
||||||
|
for k, v in values.items():
|
||||||
|
setattr(standing, k, v)
|
||||||
|
league_r["upserted"] += 1
|
||||||
|
|
||||||
|
league_r["rows"] = len(rows)
|
||||||
|
|
||||||
|
# D1(对称 events/stats 管线): 联赛成功 upsert → 补写 Bronze 层。
|
||||||
|
# 幂等键 standings:{league}:{season}:积分榜是联赛级快照,一次成功
|
||||||
|
# 采集写一条 RawEvent(整份原始载荷)+ 一条血缘。season 用实际入库的
|
||||||
|
# 标签(由载荷推导,与 Standing.season 同口径),不依赖调用方传参,
|
||||||
|
# 保证不同调用方(season=None 或显式传参)对同一赛季命中同一条 RawEvent。
|
||||||
|
if league_r["upserted"] > 0:
|
||||||
|
bronze_batch_id = f"bzzoiro-standings-{code}-{now:%Y%m%d%H%M%S}"
|
||||||
|
await _write_standings_bronze(
|
||||||
|
db,
|
||||||
|
source_record_id=f"standings:{code}:{season_label}",
|
||||||
|
raw_payload=payload,
|
||||||
|
league_id=league.id,
|
||||||
|
league_code=code,
|
||||||
|
season_label=season_label,
|
||||||
|
rows_upserted=league_r["upserted"],
|
||||||
|
batch_id=bronze_batch_id,
|
||||||
|
)
|
||||||
|
|
||||||
|
result["leagues"][code] = league_r
|
||||||
|
result["total_upserted"] += league_r["upserted"]
|
||||||
|
logger.info(
|
||||||
|
"bzzoiro standings 采集完成: %s 赛季 %s, upsert %d/%d",
|
||||||
|
code, season_label, league_r["upserted"], league_r["rows"],
|
||||||
|
)
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_standings_bronze(
|
||||||
|
db,
|
||||||
|
*,
|
||||||
|
source_record_id: str,
|
||||||
|
raw_payload: dict,
|
||||||
|
league_id: int | None,
|
||||||
|
league_code: str,
|
||||||
|
season_label: str,
|
||||||
|
rows_upserted: int,
|
||||||
|
batch_id: str,
|
||||||
|
) -> None:
|
||||||
|
"""standings 成功 upsert 一个联赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
|
||||||
|
|
||||||
|
与 _write_events_bronze 同级约束:幂等性由 _write_raw_event 的
|
||||||
|
source_record_id 查重保证(积分榜是联赛级快照,同联赛同赛季重复采集
|
||||||
|
命中同一条 RawEvent);best-effort:基础设施写入失败只记 warning,
|
||||||
|
绝不拖垮采集主流程。
|
||||||
|
"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
try:
|
||||||
|
await bz._write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
|
||||||
|
await bz._write_lineage(
|
||||||
|
db, "bzzoiro", source_record_id,
|
||||||
|
"standings", league_id, "standings_ingest",
|
||||||
|
{"league": league_code, "season": season_label, "rows_upserted": rows_upserted},
|
||||||
|
batch_id,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"standings Bronze 写入失败(record=%s, league=%s),不影响采集主流程",
|
||||||
|
source_record_id, league_code, exc_info=True,
|
||||||
|
)
|
||||||
@@ -0,0 +1,180 @@
|
|||||||
|
"""bzzoiro stats 回填管线:已完赛比赛详细统计(/events/{id}/stats/)→ match_stats 表。
|
||||||
|
|
||||||
|
从 bzzoiro.py 拆出。上游限速(REQUEST_INTERVAL 秒/请求),大批量回填需分次触发;
|
||||||
|
只 add/flush 不 commit,事务由调用方 UnitOfWork 控制。
|
||||||
|
|
||||||
|
可替换协作者(_fetch_json_async / Bronze 写入助手 / REQUEST_INTERVAL)在运行期
|
||||||
|
经聚合门面 src.data.bzzoiro 解析 —— 与拆分前的单文件 monkeypatch 语义一致。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import logging
|
||||||
|
from collections.abc import Iterable
|
||||||
|
from datetime import datetime, timedelta, timezone
|
||||||
|
|
||||||
|
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
|
||||||
|
from src.data.config import BZZOIRO_LEAGUE_IDS
|
||||||
|
from src.db.models import MatchStats
|
||||||
|
from src.db.repositories import MatchRepository
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
# bzzoiro stats 字段 → MatchStats 字段映射(stats.home / stats.away 下)
|
||||||
|
_STATS_FIELD_MAP = {
|
||||||
|
"xg": ("home_xg", "away_xg"), # 回退 expected_goals
|
||||||
|
"ball_possession": ("home_possession", None), # 只取主队值,客队=100-home
|
||||||
|
"total_shots": ("home_shots", "away_shots"),
|
||||||
|
"shots_on_target": ("home_shots_on_target", "away_shots_on_target"),
|
||||||
|
"corner_kicks": ("home_corners", "away_corners"),
|
||||||
|
"yellow_cards": ("home_yellow_cards", "away_yellow_cards"),
|
||||||
|
"red_cards": ("home_red_cards", "away_red_cards"),
|
||||||
|
"big_chances": ("home_big_chances", "away_big_chances"),
|
||||||
|
"fouls": ("home_fouls", "away_fouls"),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _pick(d: dict, *keys):
|
||||||
|
"""按优先级取第一个非空字段值。"""
|
||||||
|
for k in keys:
|
||||||
|
v = d.get(k)
|
||||||
|
if v is not None:
|
||||||
|
return v
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _stats_from_payload(payload: dict) -> dict:
|
||||||
|
"""把 /events/{id}/stats/ 响应映射成 MatchStats 字段 dict。
|
||||||
|
|
||||||
|
响应结构: {"event_id": ..., "stats": {"home": {...}, "away": {...}}}
|
||||||
|
"""
|
||||||
|
stats = (payload or {}).get("stats") or {}
|
||||||
|
home = stats.get("home") or {}
|
||||||
|
away = stats.get("away") or {}
|
||||||
|
out: dict = {}
|
||||||
|
|
||||||
|
xg_h = _pick(home, "xg", "expected_goals")
|
||||||
|
xg_a = _pick(away, "xg", "expected_goals")
|
||||||
|
if xg_h is not None:
|
||||||
|
out["home_xg"] = _to_float_or_none(xg_h)
|
||||||
|
if xg_a is not None:
|
||||||
|
out["away_xg"] = _to_float_or_none(xg_a)
|
||||||
|
|
||||||
|
poss = home.get("ball_possession")
|
||||||
|
if poss is not None:
|
||||||
|
p = _to_float_or_none(poss)
|
||||||
|
if p is not None:
|
||||||
|
out["home_possession"] = p
|
||||||
|
|
||||||
|
for src, (h_fld, a_fld) in _STATS_FIELD_MAP.items():
|
||||||
|
if src in ("xg", "ball_possession"):
|
||||||
|
continue # 已处理
|
||||||
|
hv = home.get(src)
|
||||||
|
av = away.get(src)
|
||||||
|
if hv is not None and h_fld:
|
||||||
|
out[h_fld] = _to_int_or_none(hv)
|
||||||
|
if av is not None and a_fld:
|
||||||
|
out[a_fld] = _to_int_or_none(av)
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
async def ingest_bzzoiro_event_stats(
|
||||||
|
db,
|
||||||
|
*,
|
||||||
|
leagues: Iterable[str],
|
||||||
|
limit: int = 100,
|
||||||
|
only_missing: bool = True,
|
||||||
|
) -> dict:
|
||||||
|
"""回填已完赛比赛的详细统计(逐场调 /events/{id}/stats/)。
|
||||||
|
|
||||||
|
筛选条件: match_status=finished 且 source_event_id 非空。
|
||||||
|
only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新。
|
||||||
|
limit 控制单次最多处理的比赛数(上游限速 1.2s/请求,大批量需分次触发)。
|
||||||
|
"""
|
||||||
|
from src.data import bzzoiro as bz
|
||||||
|
|
||||||
|
result: dict = {"fetched": 0, "created": 0, "updated": 0, "skipped": 0, "errors": []}
|
||||||
|
|
||||||
|
league_ids = [BZZOIRO_LEAGUE_IDS[c] for c in leagues if c in BZZOIRO_LEAGUE_IDS]
|
||||||
|
if not league_ids:
|
||||||
|
result["errors"].append("无有效联赛代码")
|
||||||
|
return result
|
||||||
|
|
||||||
|
# D4: 候选比赛查询经 MatchRepository(含 stats 预加载,筛选/排序/limit 语义不变)
|
||||||
|
matches = await MatchRepository(db).find_finished_with_stats(
|
||||||
|
league_ids, limit=limit * 3 if only_missing else limit
|
||||||
|
)
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc)
|
||||||
|
processed = 0
|
||||||
|
for m in matches:
|
||||||
|
if processed >= limit:
|
||||||
|
break
|
||||||
|
if only_missing and m.stats is not None and m.stats.home_shots is not None:
|
||||||
|
result["skipped"] += 1
|
||||||
|
continue
|
||||||
|
processed += 1
|
||||||
|
try:
|
||||||
|
payload = await bz._fetch_json_async(f"/events/{m.source_event_id}/stats/")
|
||||||
|
except Exception as e:
|
||||||
|
logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e)
|
||||||
|
result["errors"].append(f"match {m.id}: {e}")
|
||||||
|
await bz._safe_write_ingest_failure(
|
||||||
|
db,
|
||||||
|
entity_type="match_stats",
|
||||||
|
source_record_id=str(m.source_event_id),
|
||||||
|
error=e,
|
||||||
|
raw_payload={"match_id": m.id},
|
||||||
|
)
|
||||||
|
await asyncio.sleep(bz.REQUEST_INTERVAL)
|
||||||
|
continue
|
||||||
|
|
||||||
|
result["fetched"] += 1
|
||||||
|
fields = _stats_from_payload(payload)
|
||||||
|
if not fields:
|
||||||
|
result["skipped"] += 1
|
||||||
|
await asyncio.sleep(bz.REQUEST_INTERVAL)
|
||||||
|
continue
|
||||||
|
|
||||||
|
if m.stats is None:
|
||||||
|
available_at = m.match_date + timedelta(hours=2) if m.match_date else now
|
||||||
|
m.stats = MatchStats(
|
||||||
|
match_id=m.id,
|
||||||
|
source="bzzoiro",
|
||||||
|
source_record_id=str(m.source_event_id),
|
||||||
|
retrieved_at=now,
|
||||||
|
available_at=available_at,
|
||||||
|
)
|
||||||
|
db.add(m.stats)
|
||||||
|
result["created"] += 1
|
||||||
|
else:
|
||||||
|
result["updated"] += 1
|
||||||
|
if m.stats.source is None:
|
||||||
|
m.stats.source = "bzzoiro"
|
||||||
|
m.stats.source_record_id = str(m.source_event_id)
|
||||||
|
if m.stats.retrieved_at is None:
|
||||||
|
m.stats.retrieved_at = now
|
||||||
|
if m.stats.available_at is None and m.match_date:
|
||||||
|
m.stats.available_at = m.match_date + timedelta(hours=2)
|
||||||
|
|
||||||
|
for fld, v in fields.items():
|
||||||
|
if hasattr(m.stats, fld):
|
||||||
|
setattr(m.stats, fld, v)
|
||||||
|
|
||||||
|
# 管线基础设施:写入 RawEvent + DataLineage
|
||||||
|
batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}"
|
||||||
|
try:
|
||||||
|
await bz._write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id)
|
||||||
|
await bz._write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id)
|
||||||
|
except Exception:
|
||||||
|
pass # 基础设施写入失败不影响主流程
|
||||||
|
|
||||||
|
await asyncio.sleep(bz.REQUEST_INTERVAL)
|
||||||
|
|
||||||
|
logger.info(
|
||||||
|
"bzzoiro stats 回填完成: 抓取 %d, 新建 %d, 更新 %d, 跳过 %d, 错误 %d",
|
||||||
|
result["fetched"], result["created"], result["updated"],
|
||||||
|
result["skipped"], len(result["errors"]),
|
||||||
|
)
|
||||||
|
return result
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
"""管线基础设施写入助手:RawEvent(Bronze 原始载荷)/ IngestFailure(死信)/ DataLineage(血缘)。
|
||||||
|
|
||||||
|
从 bzzoiro.py 拆出。约定(与拆分前一致):
|
||||||
|
- 只 add 不 commit —— 事务由调用方 UnitOfWork 控制,分批事务约定不变;
|
||||||
|
- 死信与 Bronze 写入同为 best-effort:失败只记 warning,绝不拖垮采集主流程。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
from src.db.models import DataLineage, IngestFailure, RawEvent
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_raw_event(db, source_system: str, source_record_id: str, raw_payload: dict, batch_id: str | None = None) -> None:
|
||||||
|
"""写入 Bronze 层原始事件(幂等:同 source_record_id 跳过)。"""
|
||||||
|
from sqlalchemy import select as _select
|
||||||
|
stmt = _select(RawEvent).where(
|
||||||
|
RawEvent.source_system == source_system,
|
||||||
|
RawEvent.source_record_id == source_record_id,
|
||||||
|
)
|
||||||
|
existing = (await db.execute(stmt)).scalar_one_or_none()
|
||||||
|
if existing is None:
|
||||||
|
db.add(RawEvent(
|
||||||
|
source_system=source_system,
|
||||||
|
source_record_id=source_record_id,
|
||||||
|
raw_payload=raw_payload,
|
||||||
|
ingest_batch_id=batch_id,
|
||||||
|
))
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_ingest_failure(db, source_system: str, entity_type: str, source_record_id: str | None, error_type: str, error_detail: str | None, raw_payload: dict | None = None) -> None:
|
||||||
|
"""写入采集失败死信。"""
|
||||||
|
db.add(IngestFailure(
|
||||||
|
source_system=source_system,
|
||||||
|
entity_type=entity_type,
|
||||||
|
source_record_id=source_record_id,
|
||||||
|
error_type=error_type,
|
||||||
|
error_detail=error_detail,
|
||||||
|
raw_payload=raw_payload,
|
||||||
|
))
|
||||||
|
|
||||||
|
|
||||||
|
async def _safe_write_ingest_failure(
|
||||||
|
db,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
source_record_id: str | None,
|
||||||
|
error: Exception,
|
||||||
|
raw_payload: dict | None = None,
|
||||||
|
) -> None:
|
||||||
|
"""抓取失败时尽力写入死信表(失败不影响主流程)。
|
||||||
|
|
||||||
|
死信是「可观测性」基础设施,与 RawEvent/Lineage 同级:写入失败只记
|
||||||
|
warning,绝不能让原始抓取错误之外的新异常打断采集循环。
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
await _write_ingest_failure(
|
||||||
|
db, "bzzoiro", entity_type, source_record_id,
|
||||||
|
"fetch_error", str(error), raw_payload,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
logger.warning(
|
||||||
|
"写入 ingest_failures 死信失败(entity=%s, record=%s): %s",
|
||||||
|
entity_type, source_record_id, error, exc_info=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
async def _write_lineage(db, source_system: str, source_record_id: str, target_table: str, target_id: int | None, transform_name: str, transform_detail: dict | None = None, batch_id: str | None = None) -> None:
|
||||||
|
"""写入 ETL 血缘追踪。"""
|
||||||
|
db.add(DataLineage(
|
||||||
|
source_system=source_system,
|
||||||
|
source_record_id=source_record_id,
|
||||||
|
target_table=target_table,
|
||||||
|
target_id=target_id,
|
||||||
|
transform_name=transform_name,
|
||||||
|
transform_detail=transform_detail,
|
||||||
|
batch_id=batch_id,
|
||||||
|
))
|
||||||
+19
-3
@@ -5,6 +5,10 @@ Repository 只负责查询,不负责事务提交。
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import logging
|
||||||
|
|
||||||
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from sqlalchemy import select
|
from sqlalchemy import select
|
||||||
from sqlalchemy.orm import selectinload
|
from sqlalchemy.orm import selectinload
|
||||||
@@ -108,10 +112,22 @@ class TeamRepository:
|
|||||||
return (await self._session.execute(stmt)).scalar_one_or_none()
|
return (await self._session.execute(stmt)).scalar_one_or_none()
|
||||||
|
|
||||||
async def get_or_create(self, name: str, *, name_zh: str | None = None) -> Team:
|
async def get_or_create(self, name: str, *, name_zh: str | None = None) -> Team:
|
||||||
"""按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。"""
|
"""按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。
|
||||||
team = await self.get_by_name(name)
|
|
||||||
|
归一化咽喉:所有入库 Team.name 必须经过 team_names.normalize,
|
||||||
|
此处统一收敛,避免各调用点散落归一化逻辑导致重复 Team。
|
||||||
|
创建新 Team 时 info 打出原始名与归一后的规范名,便于排查重名。
|
||||||
|
"""
|
||||||
|
from src.data.team_names import normalize as normalize_name
|
||||||
|
|
||||||
|
normalized = normalize_name(name) or name.strip()
|
||||||
|
team = await self.get_by_name(normalized)
|
||||||
if team is None:
|
if team is None:
|
||||||
team = Team(name=name, name_zh=name_zh)
|
logger.info(
|
||||||
|
"创建新 Team: %s -> %s",
|
||||||
|
name, normalized,
|
||||||
|
)
|
||||||
|
team = Team(name=normalized, name_zh=name_zh)
|
||||||
self._session.add(team)
|
self._session.add(team)
|
||||||
await self._session.flush()
|
await self._session.flush()
|
||||||
return team
|
return team
|
||||||
|
|||||||
+42
-11
@@ -12,7 +12,7 @@ from sqlalchemy import case, func, select
|
|||||||
|
|
||||||
from src.db.base import AsyncSession, AsyncSessionLocal
|
from src.db.base import AsyncSession, AsyncSessionLocal
|
||||||
from src.db.models import Match
|
from src.db.models import Match
|
||||||
from src.llm.predict import PredictResult
|
from src.llm.predict import PredictResult, _upsert_prediction
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
logger = logging.getLogger(__name__)
|
||||||
|
|
||||||
@@ -58,19 +58,23 @@ async def predict_baseline(
|
|||||||
|
|
||||||
返回 PredictResult(D2 统一结果类型):
|
返回 PredictResult(D2 统一结果类型):
|
||||||
provider=model="baseline", 不调用 LLM,latency_ms≈0。
|
provider=model="baseline", 不调用 LLM,latency_ms≈0。
|
||||||
prediction_id 为占位 0 —— baseline 不在服务层落库,
|
|
||||||
由路由层 _persist_baseline 落库后取得真实 id。
|
P3-2:baseline 落库下沉到服务层 —— 直接在服务层完成落库并回填真实
|
||||||
|
prediction_id,路由层不再需要特殊的 _persist_baseline,与 single/multi
|
||||||
|
路径统一(result.prediction_id 即可用)。对外 JSON 不变。
|
||||||
"""
|
"""
|
||||||
|
from src.db.unit_of_work import get_uow
|
||||||
|
|
||||||
async with AsyncSessionLocal() as db:
|
async with AsyncSessionLocal() as db:
|
||||||
match = await db.get(Match, match_id)
|
match = await db.get(Match, match_id)
|
||||||
if match is None:
|
if match is None:
|
||||||
raise ValueError(f"match {match_id} not found")
|
raise ValueError(f"match {match_id} not found")
|
||||||
|
|
||||||
before = None
|
before = None
|
||||||
if backtest and match.match_dt:
|
if backtest and match.match_date:
|
||||||
from datetime import timedelta
|
from datetime import timedelta
|
||||||
|
|
||||||
before = match.match_dt - timedelta(days=1)
|
before = match.match_date - timedelta(days=1)
|
||||||
elif cutoff_at is not None:
|
elif cutoff_at is not None:
|
||||||
before = cutoff_at
|
before = cutoff_at
|
||||||
|
|
||||||
@@ -93,8 +97,38 @@ async def predict_baseline(
|
|||||||
else:
|
else:
|
||||||
pred_1x2 = "X"
|
pred_1x2 = "X"
|
||||||
|
|
||||||
|
values = {
|
||||||
|
"prompt_version": "baseline_v1",
|
||||||
|
"prompt_tokens": 0,
|
||||||
|
"completion_tokens": 0,
|
||||||
|
"latency_ms": 0,
|
||||||
|
"pred_home_goals": float(pred_home),
|
||||||
|
"pred_away_goals": float(pred_away),
|
||||||
|
"pred_1x2": pred_1x2,
|
||||||
|
"subjective_confidence": 0.5,
|
||||||
|
"reasoning": (
|
||||||
|
f"基线估计(非投注建议): 主队主场场均进球 {home_avg:.2f} → 预测 {pred_home}; "
|
||||||
|
f"客队客场场均进球 {away_avg:.2f} → 预测 {pred_away}。"
|
||||||
|
),
|
||||||
|
"raw_response": {"home_avg": round(home_avg, 2), "away_avg": round(away_avg, 2)},
|
||||||
|
"status": "success",
|
||||||
|
}
|
||||||
|
|
||||||
|
# P3-2:服务层落库,回填真实 prediction_id(与 single/multi 统一)。
|
||||||
|
async with get_uow() as session:
|
||||||
|
pred = await _upsert_prediction(
|
||||||
|
session,
|
||||||
|
match_id=match_id,
|
||||||
|
provider_name="baseline",
|
||||||
|
model="baseline",
|
||||||
|
mode="baseline",
|
||||||
|
run_type="live",
|
||||||
|
values=values,
|
||||||
|
)
|
||||||
|
prediction_id = pred.id
|
||||||
|
|
||||||
return PredictResult(
|
return PredictResult(
|
||||||
prediction_id=0, # 占位:真实 id 由路由层 _persist_baseline 落库后返回
|
prediction_id=prediction_id,
|
||||||
provider="baseline",
|
provider="baseline",
|
||||||
model="baseline",
|
model="baseline",
|
||||||
prompt_version="baseline_v1",
|
prompt_version="baseline_v1",
|
||||||
@@ -105,14 +139,11 @@ async def predict_baseline(
|
|||||||
alt_pred_away_goals=None,
|
alt_pred_away_goals=None,
|
||||||
pred_1x2=pred_1x2,
|
pred_1x2=pred_1x2,
|
||||||
subjective_confidence=0.5,
|
subjective_confidence=0.5,
|
||||||
reasoning=(
|
reasoning=values["reasoning"],
|
||||||
f"基线估计(非投注建议): 主队主场场均进球 {home_avg:.2f} → 预测 {pred_home}; "
|
|
||||||
f"客队客场场均进球 {away_avg:.2f} → 预测 {pred_away}。"
|
|
||||||
),
|
|
||||||
context="", # baseline 不构建 LLM 上下文
|
context="", # baseline 不构建 LLM 上下文
|
||||||
status="success",
|
status="success",
|
||||||
latency_ms=0,
|
latency_ms=0,
|
||||||
prompt_tokens=0,
|
prompt_tokens=0,
|
||||||
completion_tokens=0,
|
completion_tokens=0,
|
||||||
raw={"home_avg": round(home_avg, 2), "away_avg": round(away_avg, 2)},
|
raw=values["raw_response"],
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -79,18 +79,18 @@ class TestWriteBufferStrategy:
|
|||||||
def test_bzzoirot_new_match_available_at_is_two_hours_after_kickoff(self):
|
def test_bzzoirot_new_match_available_at_is_two_hours_after_kickoff(self):
|
||||||
"""bzzoiro 新建比赛(stats 回填创建 MatchStats)时 available_at 应为开球 + 2 小时。"""
|
"""bzzoiro 新建比赛(stats 回填创建 MatchStats)时 available_at 应为开球 + 2 小时。"""
|
||||||
import inspect
|
import inspect
|
||||||
from src.data import bzzoiro
|
from src.data import bzzoiro_stats
|
||||||
|
|
||||||
source = inspect.getsource(bzzoiro)
|
source = inspect.getsource(bzzoiro_stats)
|
||||||
assert 'timedelta(hours=2)' in source, \
|
assert 'timedelta(hours=2)' in source, \
|
||||||
"bzzoiro 应使用 match_date + timedelta(hours=2) 作为 available_at"
|
"bzzoiro 应使用 match_date + timedelta(hours=2) 作为 available_at"
|
||||||
|
|
||||||
def test_bzzoirot_multiple_writes_use_two_hour_buffer(self):
|
def test_bzzoirot_multiple_writes_use_two_hour_buffer(self):
|
||||||
"""bzzoiro 多处写入(创建/更新)都应使用 2 小时缓冲。"""
|
"""bzzoiro 多处写入(创建/更新)都应使用 2 小时缓冲。"""
|
||||||
import inspect
|
import inspect
|
||||||
from src.data import bzzoiro
|
from src.data import bzzoiro_stats
|
||||||
|
|
||||||
source = inspect.getsource(bzzoiro)
|
source = inspect.getsource(bzzoiro_stats)
|
||||||
count = source.count('timedelta(hours=2)')
|
count = source.count('timedelta(hours=2)')
|
||||||
assert count >= 2, f"期望至少 2 处 timedelta(hours=2),实际 {count} 处"
|
assert count >= 2, f"期望至少 2 处 timedelta(hours=2),实际 {count} 处"
|
||||||
|
|
||||||
|
|||||||
+29
-2
@@ -5,6 +5,7 @@
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
@@ -12,6 +13,24 @@ import pytest
|
|||||||
from src.llm.baseline import _avg_goals, predict_baseline
|
from src.llm.baseline import _avg_goals, predict_baseline
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeUoW:
|
||||||
|
"""P3-2:baseline 在服务层落库,测试需 mock get_uow。"""
|
||||||
|
|
||||||
|
async def __aenter__(self):
|
||||||
|
return SimpleNamespace(
|
||||||
|
execute=lambda *a, **k: SimpleNamespace(scalar_one_or_none=lambda: None),
|
||||||
|
add=lambda *a, **k: None,
|
||||||
|
flush=lambda *a, **k: None,
|
||||||
|
)
|
||||||
|
|
||||||
|
async def __aexit__(self, *a):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
async def _fake_upsert(session, **kw):
|
||||||
|
return SimpleNamespace(id=1)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_avg_goals_no_data_returns_zero():
|
async def test_avg_goals_no_data_returns_zero():
|
||||||
"""无历史数据时场均进球为 0(不抛异常)。"""
|
"""无历史数据时场均进球为 0(不抛异常)。"""
|
||||||
@@ -68,7 +87,9 @@ async def test_predict_baseline_no_llm():
|
|||||||
match_status = "scheduled"
|
match_status = "scheduled"
|
||||||
|
|
||||||
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
||||||
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
|
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
|
||||||
|
patch("src.db.unit_of_work.get_uow", _FakeUoW), \
|
||||||
|
patch("src.llm.baseline._upsert_prediction", _fake_upsert):
|
||||||
class FakeSession:
|
class FakeSession:
|
||||||
async def get(self, cls, mid):
|
async def get(self, cls, mid):
|
||||||
return FakeMatch()
|
return FakeMatch()
|
||||||
@@ -93,6 +114,8 @@ async def test_predict_baseline_no_llm():
|
|||||||
assert result.pred_1x2 == "X"
|
assert result.pred_1x2 == "X"
|
||||||
assert result.subjective_confidence == 0.5
|
assert result.subjective_confidence == 0.5
|
||||||
assert "非投注建议" in result.reasoning
|
assert "非投注建议" in result.reasoning
|
||||||
|
# P3-2:服务层落库,回填真实 prediction_id
|
||||||
|
assert result.prediction_id == 1
|
||||||
# 确认未调用任何 LLM 相关模块
|
# 确认未调用任何 LLM 相关模块
|
||||||
assert "home_10" in captured and "away_20" in captured
|
assert "home_10" in captured and "away_20" in captured
|
||||||
|
|
||||||
@@ -112,7 +135,9 @@ async def test_predict_baseline_clamps_to_range():
|
|||||||
match_status = "scheduled"
|
match_status = "scheduled"
|
||||||
|
|
||||||
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
||||||
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
|
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
|
||||||
|
patch("src.db.unit_of_work.get_uow", _FakeUoW), \
|
||||||
|
patch("src.llm.baseline._upsert_prediction", _fake_upsert):
|
||||||
class FakeSession:
|
class FakeSession:
|
||||||
async def get(self, cls, mid):
|
async def get(self, cls, mid):
|
||||||
return FakeMatch()
|
return FakeMatch()
|
||||||
@@ -128,3 +153,5 @@ async def test_predict_baseline_clamps_to_range():
|
|||||||
assert result.pred_home_goals == 10.0 # clamped
|
assert result.pred_home_goals == 10.0 # clamped
|
||||||
assert result.pred_away_goals == 0.0 # clamped
|
assert result.pred_away_goals == 0.0 # clamped
|
||||||
assert result.pred_1x2 == "1" # 10:0 主胜
|
assert result.pred_1x2 == "1" # 10:0 主胜
|
||||||
|
# P3-2:服务层落库,回填真实 prediction_id
|
||||||
|
assert result.prediction_id == 1
|
||||||
|
|||||||
@@ -62,12 +62,35 @@ async def test_predict_baseline_returns_predict_result():
|
|||||||
async def __aexit__(self, *a):
|
async def __aexit__(self, *a):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
# P3-2:baseline 在服务层落库(get_uow + _upsert_prediction),需 mock 掉。
|
||||||
|
class FakeUoW:
|
||||||
|
async def __aenter__(self):
|
||||||
|
return _make_session()
|
||||||
|
|
||||||
|
async def __aexit__(self, *a):
|
||||||
|
return None
|
||||||
|
|
||||||
|
captured = {}
|
||||||
|
|
||||||
|
async def fake_upsert(session, **kw):
|
||||||
|
captured.update(kw)
|
||||||
|
return SimpleNamespace(id=77)
|
||||||
|
|
||||||
|
# baseline.py 内部 from-import get_uow / _upsert_prediction,需 patch 真实来源模块。
|
||||||
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
with patch("src.llm.baseline._avg_goals", fake_avg), \
|
||||||
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
|
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
|
||||||
|
patch("src.db.unit_of_work.get_uow", FakeUoW), \
|
||||||
|
patch("src.llm.baseline._upsert_prediction", fake_upsert):
|
||||||
SLC.return_value = FakeCM()
|
SLC.return_value = FakeCM()
|
||||||
|
|
||||||
result = await predict_baseline(1)
|
result = await predict_baseline(1)
|
||||||
|
|
||||||
|
# P3-2:验证服务层落库被调用且属性映射正确
|
||||||
|
assert captured["match_id"] == 1
|
||||||
|
assert captured["provider_name"] == "baseline"
|
||||||
|
assert captured["run_type"] == "live"
|
||||||
|
assert captured["values"]["pred_home_goals"] == 2.0
|
||||||
|
|
||||||
assert isinstance(result, PredictResult)
|
assert isinstance(result, PredictResult)
|
||||||
assert result.mode == "baseline"
|
assert result.mode == "baseline"
|
||||||
assert result.provider == "baseline"
|
assert result.provider == "baseline"
|
||||||
@@ -144,15 +167,31 @@ def test_predict_route_has_no_dict_branch():
|
|||||||
|
|
||||||
|
|
||||||
# ============================================================
|
# ============================================================
|
||||||
# 4. _persist_baseline 属性映射(baseline 落库语义不变)
|
# 4. P3-2:baseline 服务层落库属性映射(落库已从路由移到 baseline.py)
|
||||||
# ============================================================
|
# ============================================================
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeResult:
|
||||||
|
"""支持 .scalar_one_or_none() 的最小假结果集。"""
|
||||||
|
|
||||||
|
def __init__(self, items):
|
||||||
|
self._items = list(items)
|
||||||
|
|
||||||
|
def scalars(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return self._items
|
||||||
|
|
||||||
|
def scalar_one_or_none(self):
|
||||||
|
return self._items[0] if self._items else None
|
||||||
|
|
||||||
|
|
||||||
class _FakeUoW:
|
class _FakeUoW:
|
||||||
"""替代 get_uow 的最小上下文管理器。"""
|
"""替代 get_uow 的最小上下文管理器(session.execute 是 async 的)。"""
|
||||||
|
|
||||||
def __init__(self):
|
def __init__(self):
|
||||||
self.session = SimpleNamespace()
|
self.session = _make_session()
|
||||||
|
|
||||||
async def __aenter__(self):
|
async def __aenter__(self):
|
||||||
return self.session
|
return self.session
|
||||||
@@ -160,44 +199,67 @@ class _FakeUoW:
|
|||||||
async def __aexit__(self, *a):
|
async def __aexit__(self, *a):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
def __call__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
|
||||||
|
def _make_session(existing=None):
|
||||||
|
"""构造带 async execute / add / flush 的假 session。"""
|
||||||
|
sess = SimpleNamespace()
|
||||||
|
|
||||||
|
async def execute(*a, **k):
|
||||||
|
return _FakeResult(existing or [])
|
||||||
|
|
||||||
|
sess.execute = execute
|
||||||
|
sess.add = lambda *a, **k: None
|
||||||
|
|
||||||
|
async def flush(*a, **k):
|
||||||
|
return None
|
||||||
|
|
||||||
|
sess.flush = flush
|
||||||
|
return sess
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_persist_baseline_maps_attributes(monkeypatch):
|
async def test_baseline_service_persists_with_correct_attributes(monkeypatch):
|
||||||
|
"""P3-2:baseline 在服务层(predict_baseline)落库,属性映射与路由旧版一致。"""
|
||||||
captured = {}
|
captured = {}
|
||||||
|
|
||||||
async def fake_upsert(session, **kwargs):
|
async def fake_upsert(session, **kwargs):
|
||||||
captured.update(kwargs)
|
captured.update(kwargs)
|
||||||
return SimpleNamespace(id=77)
|
return SimpleNamespace(id=77)
|
||||||
|
|
||||||
|
class FakeMatch:
|
||||||
|
id = 1
|
||||||
|
home_team_id = 10
|
||||||
|
away_team_id = 20
|
||||||
|
league_id = 1
|
||||||
|
match_status = "scheduled"
|
||||||
|
|
||||||
|
class FakeSession:
|
||||||
|
async def get(self, cls, mid):
|
||||||
|
return FakeMatch()
|
||||||
|
|
||||||
|
class FakeSLC:
|
||||||
|
async def __aenter__(self):
|
||||||
|
return FakeSession()
|
||||||
|
|
||||||
|
async def __aexit__(self, *a):
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def fake_avg(db, *, team_id, side, league_id, before):
|
||||||
|
return 2.0 if side == "home" else 1.0
|
||||||
|
|
||||||
|
monkeypatch.setattr("src.llm.baseline._avg_goals", fake_avg)
|
||||||
|
monkeypatch.setattr("src.llm.baseline.AsyncSessionLocal", FakeSLC)
|
||||||
monkeypatch.setattr("src.db.unit_of_work.get_uow", lambda: _FakeUoW())
|
monkeypatch.setattr("src.db.unit_of_work.get_uow", lambda: _FakeUoW())
|
||||||
monkeypatch.setattr("src.llm.predict._upsert_prediction", fake_upsert)
|
# baseline.py 模块级 import _upsert_prediction(第 15 行),需 patch baseline 模块属性
|
||||||
|
monkeypatch.setattr("src.llm.baseline._upsert_prediction", fake_upsert)
|
||||||
|
|
||||||
from src.api.routes.predict import _persist_baseline
|
result = await predict_baseline(1)
|
||||||
|
|
||||||
baseline = PredictResult(
|
# 落库被调用且属性映射正确
|
||||||
prediction_id=0, # baseline 不在服务层落库,由 _persist_baseline 落库后取得真实 id
|
assert captured, f"predict_baseline 应调用 _upsert_prediction 落库,但 captured 为空(result.prediction_id={result.prediction_id!r})"
|
||||||
provider="baseline",
|
|
||||||
model="baseline",
|
|
||||||
prompt_version="baseline_v1",
|
|
||||||
mode="baseline",
|
|
||||||
pred_home_goals=2.0,
|
|
||||||
pred_away_goals=1.0,
|
|
||||||
alt_pred_home_goals=None,
|
|
||||||
alt_pred_away_goals=None,
|
|
||||||
pred_1x2="1",
|
|
||||||
subjective_confidence=0.5,
|
|
||||||
reasoning="r",
|
|
||||||
context="",
|
|
||||||
status="success",
|
|
||||||
latency_ms=0,
|
|
||||||
prompt_tokens=0,
|
|
||||||
completion_tokens=0,
|
|
||||||
raw={"home_avg": 2.1, "away_avg": 1.4},
|
|
||||||
)
|
|
||||||
|
|
||||||
pid = await _persist_baseline(1, baseline)
|
|
||||||
|
|
||||||
assert pid == 77
|
|
||||||
assert captured["match_id"] == 1
|
assert captured["match_id"] == 1
|
||||||
assert captured["provider_name"] == "baseline"
|
assert captured["provider_name"] == "baseline"
|
||||||
assert captured["model"] == "baseline"
|
assert captured["model"] == "baseline"
|
||||||
@@ -212,5 +274,28 @@ async def test_persist_baseline_maps_attributes(monkeypatch):
|
|||||||
assert v["prompt_tokens"] == 0
|
assert v["prompt_tokens"] == 0
|
||||||
assert v["completion_tokens"] == 0
|
assert v["completion_tokens"] == 0
|
||||||
assert v["latency_ms"] == 0
|
assert v["latency_ms"] == 0
|
||||||
assert v["raw_response"] == {"home_avg": 2.1, "away_avg": 1.4}
|
assert v["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
|
||||||
assert v["status"] == "success"
|
assert v["status"] == "success"
|
||||||
|
|
||||||
|
# 回填真实 prediction_id(服务层落库后取得)
|
||||||
|
assert result.prediction_id == 77
|
||||||
|
assert result.pred_1x2 == "1"
|
||||||
|
assert captured["match_id"] == 1
|
||||||
|
assert captured["provider_name"] == "baseline"
|
||||||
|
assert captured["model"] == "baseline"
|
||||||
|
assert captured["mode"] == "baseline"
|
||||||
|
assert captured["run_type"] == "live"
|
||||||
|
v = captured["values"]
|
||||||
|
assert v["prompt_version"] == "baseline_v1"
|
||||||
|
assert v["pred_home_goals"] == 2.0
|
||||||
|
assert v["pred_away_goals"] == 1.0
|
||||||
|
assert v["pred_1x2"] == "1"
|
||||||
|
assert v["subjective_confidence"] == 0.5
|
||||||
|
assert v["prompt_tokens"] == 0
|
||||||
|
assert v["completion_tokens"] == 0
|
||||||
|
assert v["latency_ms"] == 0
|
||||||
|
assert v["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
|
||||||
|
assert v["status"] == "success"
|
||||||
|
|
||||||
|
# 回填真实 prediction_id(服务层落库后取得)
|
||||||
|
assert result.prediction_id == 77
|
||||||
|
|||||||
@@ -0,0 +1,163 @@
|
|||||||
|
"""公开只读 API 回归(P1-2 / P1-3):context 与 leagues 匿名可访问。
|
||||||
|
|
||||||
|
背景:
|
||||||
|
- 公开站 MatchDetailSection 展开详情会请求 /matches/{id}/context,
|
||||||
|
此前该端点挂 require_admin,未登录 401 被 fetchMatchContext 的
|
||||||
|
catch 吞掉 → 近况/交锋静默为空;
|
||||||
|
- 公开站联赛筛选需要 /leagues,此前同样 require_admin,前端写死五大联赛。
|
||||||
|
|
||||||
|
守卫(双保险):
|
||||||
|
1. 功能层:最小 FastAPI app + dependency_overrides 注入 fake session,
|
||||||
|
匿名请求(无任何凭据)→ 200;不存在的 match → 404。
|
||||||
|
(不启动完整 app lifespan,遵循 test_api_critical.py 的既定约束)
|
||||||
|
2. 鉴权层:检查路由依赖声明,require_admin 不得出现在
|
||||||
|
/leagues 与 /matches/{match_id}/context。dev 环境 require_admin
|
||||||
|
未配置鉴权时 fail-open,功能层测不出「加回了 require_admin」的回归,
|
||||||
|
必须靠本层声明检查;并用 ingest 路由证明检查器本身有判别力
|
||||||
|
(变异保护:若有人给公开端点加回 require_admin,此处变红)。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
from src.api.deps import require_admin
|
||||||
|
from src.api.routes.ingest import router as ingest_router
|
||||||
|
from src.api.routes.matches import router as matches_router
|
||||||
|
from src.db.base import get_db_read
|
||||||
|
from src.db.models import League, Match
|
||||||
|
|
||||||
|
|
||||||
|
# ── fake DB(对齐 routes/matches.py 的实际查询面) ──────────────────
|
||||||
|
|
||||||
|
class _FakeResult:
|
||||||
|
def __init__(self, items):
|
||||||
|
self._items = list(items)
|
||||||
|
|
||||||
|
def scalars(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return list(self._items)
|
||||||
|
|
||||||
|
def scalar_one_or_none(self):
|
||||||
|
return self._items[0] if self._items else None
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeDB:
|
||||||
|
"""match_context 的查询次序:① match 主查询(scalar_one_or_none)
|
||||||
|
② home_recent ③ away_recent ④ h2h(均 scalars().all())。
|
||||||
|
League 查询按实体识别直接返回列表(对应 /leagues)。"""
|
||||||
|
|
||||||
|
def __init__(self, match=None, match_lists=(), leagues=()):
|
||||||
|
self._match = match
|
||||||
|
self._match_lists = list(match_lists)
|
||||||
|
self._leagues = list(leagues)
|
||||||
|
self._calls = 0
|
||||||
|
|
||||||
|
async def execute(self, stmt):
|
||||||
|
entity = stmt.column_descriptions[0]["entity"]
|
||||||
|
if entity is League:
|
||||||
|
return _FakeResult(self._leagues)
|
||||||
|
if self._calls == 0:
|
||||||
|
self._calls += 1
|
||||||
|
return _FakeResult([self._match] if self._match is not None else [])
|
||||||
|
idx = self._calls - 1
|
||||||
|
self._calls += 1
|
||||||
|
return _FakeResult(self._match_lists[idx] if idx < len(self._match_lists) else [])
|
||||||
|
|
||||||
|
|
||||||
|
def _client(fake_db: _FakeDB) -> TestClient:
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(matches_router)
|
||||||
|
app.dependency_overrides[get_db_read] = lambda: fake_db
|
||||||
|
# 不用 with:不触发 lifespan,无真实 DB 引擎连接
|
||||||
|
return TestClient(app)
|
||||||
|
|
||||||
|
|
||||||
|
def _league(lid: int, code: str) -> League:
|
||||||
|
return League(id=lid, code=code, name=f"League {code}", country=f"Country {code}")
|
||||||
|
|
||||||
|
|
||||||
|
def _match(mid: int) -> Match:
|
||||||
|
return Match(
|
||||||
|
id=mid,
|
||||||
|
match_date=datetime(2026, 9, 20, 15, 0, tzinfo=timezone.utc),
|
||||||
|
home_goals=2,
|
||||||
|
away_goals=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ── 功能层:匿名可访问(P1-2 / P1-3) ──────────────────────────────
|
||||||
|
|
||||||
|
def test_leagues_anonymous_200_and_shape():
|
||||||
|
"""/leagues 匿名 200;仅暴露 id/code/name/country 四字段。"""
|
||||||
|
db = _FakeDB(leagues=[_league(1, "E0"), _league(2, "SP1")])
|
||||||
|
r = _client(db).get("/api/v1/leagues")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert body == [
|
||||||
|
{"id": 1, "code": "E0", "name": "League E0", "country": "Country E0"},
|
||||||
|
{"id": 2, "code": "SP1", "name": "League SP1", "country": "Country SP1"},
|
||||||
|
]
|
||||||
|
# 不暴露敏感配置字段
|
||||||
|
assert all(set(item.keys()) == {"id", "code", "name", "country"} for item in body)
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_anonymous_200_empty_data():
|
||||||
|
"""/context 匿名 200;无数据时三个列表为空(前端空态),结构不变。"""
|
||||||
|
db = _FakeDB(match=_match(42), match_lists=[[], [], []])
|
||||||
|
r = _client(db).get("/api/v1/matches/42/context")
|
||||||
|
assert r.status_code == 200
|
||||||
|
assert r.json() == {"home_recent": [], "away_recent": [], "h2h": []}
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_anonymous_200_row_shape():
|
||||||
|
"""/context 行结构与既有前端契约一致(5 字段)。"""
|
||||||
|
db = _FakeDB(match=_match(42), match_lists=[[_match(1)], [], [_match(2), _match(3)]])
|
||||||
|
r = _client(db).get("/api/v1/matches/42/context")
|
||||||
|
assert r.status_code == 200
|
||||||
|
body = r.json()
|
||||||
|
assert len(body["home_recent"]) == 1
|
||||||
|
assert len(body["h2h"]) == 2
|
||||||
|
assert set(body["h2h"][0].keys()) == {
|
||||||
|
"match_date", "home_team", "away_team", "home_goals", "away_goals",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def test_context_not_found_404():
|
||||||
|
"""/context 匿名访问不存在的 match → 404(而非 401/503)。"""
|
||||||
|
db = _FakeDB(match=None)
|
||||||
|
r = _client(db).get("/api/v1/matches/999/context")
|
||||||
|
assert r.status_code == 404
|
||||||
|
|
||||||
|
|
||||||
|
# ── 鉴权层:路由依赖声明检查(防 require_admin 回潜) ────────────────
|
||||||
|
|
||||||
|
def _admin_paths(router) -> set[str]:
|
||||||
|
paths: set[str] = set()
|
||||||
|
for route in router.routes:
|
||||||
|
for dep in route.dependant.dependencies:
|
||||||
|
if dep.call is require_admin:
|
||||||
|
paths.add(route.path)
|
||||||
|
break
|
||||||
|
return paths
|
||||||
|
|
||||||
|
|
||||||
|
def test_public_routes_have_no_admin_dependency():
|
||||||
|
"""/leagues 与 /context 不得挂 require_admin;matches 路由全部公开只读。"""
|
||||||
|
admin_paths = _admin_paths(matches_router)
|
||||||
|
assert "/api/v1/leagues" not in admin_paths
|
||||||
|
assert "/api/v1/matches/{match_id}/context" not in admin_paths
|
||||||
|
assert admin_paths == set(), f"matches 路由应全部公开只读,仍有 {admin_paths}"
|
||||||
|
|
||||||
|
|
||||||
|
def test_guard_detector_has_discrimination_power():
|
||||||
|
"""变异保护:检查器必须能在 ingest 路由上发现 require_admin,
|
||||||
|
否则上一条「无 admin 依赖」断言恒真、毫无判别力。"""
|
||||||
|
assert "/api/v1/ingest/bzzoiro" in _admin_paths(ingest_router), (
|
||||||
|
"ingest 路由应仍存在 require_admin 保护;若本断言失败,"
|
||||||
|
"说明公开只读守卫的检查器已失效,请修复检查逻辑"
|
||||||
|
)
|
||||||
@@ -175,7 +175,7 @@ class TestBzzoiroLineage:
|
|||||||
return bad
|
return bad
|
||||||
|
|
||||||
def test_normalized_matches_carries_raw(self):
|
def test_normalized_matches_carries_raw(self):
|
||||||
src = _read("data/bzzoiro.py")
|
src = _read("data/bzzoiro_events.py")
|
||||||
# 规范化结果必须与原始 event 成对保存
|
# 规范化结果必须与原始 event 成对保存
|
||||||
assert "normalized_matches.append((nm, raw))" in src, (
|
assert "normalized_matches.append((nm, raw))" in src, (
|
||||||
"normalized_matches 未携带 (nm, raw) 元组 —— raw 变量泄漏会回归 (P0-3)"
|
"normalized_matches 未携带 (nm, raw) 元组 —— raw 变量泄漏会回归 (P0-3)"
|
||||||
@@ -193,7 +193,7 @@ class TestBzzoiroLineage:
|
|||||||
它不是 `raw.get(` 同一行,但同样正确。非法写法(回归)是直接
|
它不是 `raw.get(` 同一行,但同样正确。非法写法(回归)是直接
|
||||||
`existing_match.source_event_id = orphan_var`。
|
`existing_match.source_event_id = orphan_var`。
|
||||||
"""
|
"""
|
||||||
src = _read("data/bzzoiro.py")
|
src = _read("data/bzzoiro_events.py")
|
||||||
seg = self._consume_loop_body(src)
|
seg = self._consume_loop_body(src)
|
||||||
bad = self._bad_assignments(seg)
|
bad = self._bad_assignments(seg)
|
||||||
assert len(bad) == 0, (
|
assert len(bad) == 0, (
|
||||||
@@ -206,7 +206,7 @@ class TestBzzoiroLineage:
|
|||||||
下游 `_backfill_stats` 里合法地在 ORM 对象上访问 `m.source_event_id`
|
下游 `_backfill_stats` 里合法地在 ORM 对象上访问 `m.source_event_id`
|
||||||
(与配对 raw 无关)。若 seg 越界,test_no_orphan_raw_use 会误报。
|
(与配对 raw 无关)。若 seg 越界,test_no_orphan_raw_use 会误报。
|
||||||
"""
|
"""
|
||||||
src = _read("data/bzzoiro.py")
|
src = _read("data/bzzoiro_events.py")
|
||||||
seg = self._consume_loop_body(src)
|
seg = self._consume_loop_body(src)
|
||||||
assert "m.source_event_id" not in seg, (
|
assert "m.source_event_id" not in seg, (
|
||||||
"循环体截取越界,扫到了下游 stats 管线 —— 会误报 P0-3"
|
"循环体截取越界,扫到了下游 stats 管线 —— 会误报 P0-3"
|
||||||
|
|||||||
@@ -125,7 +125,7 @@ class _FakeDb:
|
|||||||
async def test_r2_standings_actually_upserts(monkeypatch):
|
async def test_r2_standings_actually_upserts(monkeypatch):
|
||||||
"""行为测试: 喂一份积分榜 payload,断言真的构造了 Standing 且计数 > 0。"""
|
"""行为测试: 喂一份积分榜 payload,断言真的构造了 Standing 且计数 > 0。"""
|
||||||
import src.data.bzzoiro as bz
|
import src.data.bzzoiro as bz
|
||||||
from src.db.models import League, Standing, Team
|
from src.db.models import DataLineage, League, RawEvent, Standing, Team
|
||||||
|
|
||||||
payload = {
|
payload = {
|
||||||
"season": {"start_date": "2025-08-01", "end_date": "2026-05-31"},
|
"season": {"start_date": "2025-08-01", "end_date": "2026-05-31"},
|
||||||
@@ -166,7 +166,10 @@ async def test_r2_standings_actually_upserts(monkeypatch):
|
|||||||
|
|
||||||
standings = [o for o in db.added if isinstance(o, Standing)]
|
standings = [o for o in db.added if isinstance(o, Standing)]
|
||||||
assert len(standings) == 2, "应真的构造 Standing 行"
|
assert len(standings) == 2, "应真的构造 Standing 行"
|
||||||
assert all(isinstance(o, (Standing, Team)) for o in db.added)
|
# standings 采集接线 Bronze 后(RawEvent + DataLineage),add 的对象类型白名单随之放宽
|
||||||
|
assert all(
|
||||||
|
isinstance(o, (Standing, Team, RawEvent, DataLineage)) for o in db.added
|
||||||
|
)
|
||||||
|
|
||||||
first = standings[0]
|
first = standings[0]
|
||||||
assert first.league_id == 42
|
assert first.league_id == 42
|
||||||
|
|||||||
@@ -0,0 +1,288 @@
|
|||||||
|
"""standings 成功路径 Bronze 层回归测试(RawEvent + DataLineage)。
|
||||||
|
|
||||||
|
背景: events/stats 管线成功后均已补写 Bronze 层,唯独 standings 采集
|
||||||
|
成功后既不留原始载荷,也不留血缘 —— 三条管线的溯源链条在积分榜一环
|
||||||
|
缺失。本测试守护(与 test_events_bronze.py 对称):
|
||||||
|
1. 联赛成功 upsert → RawEvent(幂等键=standings:{league}:{season})
|
||||||
|
+ Lineage(target_table="standings", transform_name="standings_ingest")
|
||||||
|
2. 更新已有快照(非插入)同样写 Bronze —— 积分榜是快照,刷新即采集
|
||||||
|
3. RawEvent 幂等: 同 source_record_id 已存在则跳过,血缘照写
|
||||||
|
4. 基础设施写入失败 → 只 warning,不拖垮采集主流程
|
||||||
|
5. 抓取失败路径继续走 _safe_write_ingest_failure,且不写 Bronze
|
||||||
|
|
||||||
|
范式: 假 db(按查询实体分发预置数据 + 记录 add,flush 分配自增 id)
|
||||||
|
+ monkeypatch 抓取函数,不依赖真实数据库。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
import src.data.bzzoiro as bz
|
||||||
|
from src.db.models import DataLineage, IngestFailure, League, RawEvent, Standing, Team
|
||||||
|
|
||||||
|
|
||||||
|
def _payload():
|
||||||
|
"""构造一份最小合法的 bzzoiro /leagues/{id}/standings/ 原始载荷。"""
|
||||||
|
return {
|
||||||
|
"season": {"start_date": "2025-08-01", "end_date": "2026-05-31"},
|
||||||
|
"standings": [
|
||||||
|
{
|
||||||
|
"position": 1, "team_name": "Arsenal FC",
|
||||||
|
"played": 10, "won": 8, "drawn": 1, "lost": 1,
|
||||||
|
"gf": 22, "ga": 8, "gd": 14, "pts": 25,
|
||||||
|
"zone": {"key": "champions_league", "label": "Champions League"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"position": 2, "team_name": "Chelsea FC",
|
||||||
|
"played": 10, "won": 6, "drawn": 2, "lost": 2,
|
||||||
|
"gf": 18, "ga": 12, "gd": 6, "pts": 20,
|
||||||
|
},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _patch_fetch(monkeypatch, payload):
|
||||||
|
async def _fetch(league_code, season=None):
|
||||||
|
return payload
|
||||||
|
|
||||||
|
monkeypatch.setattr(bz, "fetch_bzzoiro_standings", _fetch)
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeResult:
|
||||||
|
"""支持 .scalars().all() / .scalar_one_or_none() 的最小假结果集。"""
|
||||||
|
|
||||||
|
def __init__(self, items):
|
||||||
|
self._items = list(items)
|
||||||
|
|
||||||
|
def scalars(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
def __iter__(self):
|
||||||
|
return iter(self._items)
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return self._items
|
||||||
|
|
||||||
|
def scalar_one_or_none(self):
|
||||||
|
return self._items[0] if self._items else None
|
||||||
|
|
||||||
|
|
||||||
|
class _FakeDB:
|
||||||
|
"""按查询实体分发预置数据;记录 add();flush 为无 id 对象分配自增主键。"""
|
||||||
|
|
||||||
|
def __init__(self, leagues=(), teams=(), standings=(), raw_events=()):
|
||||||
|
self.added = []
|
||||||
|
self._by_entity = {
|
||||||
|
League: list(leagues),
|
||||||
|
Team: list(teams),
|
||||||
|
Standing: list(standings),
|
||||||
|
RawEvent: list(raw_events),
|
||||||
|
}
|
||||||
|
self._next_id = 0
|
||||||
|
|
||||||
|
def add(self, obj):
|
||||||
|
self.added.append(obj)
|
||||||
|
|
||||||
|
async def execute(self, stmt):
|
||||||
|
entities = set()
|
||||||
|
for d in (stmt.column_descriptions or []):
|
||||||
|
entities.add(d.get("entity") or d.get("type"))
|
||||||
|
for entity, items in self._by_entity.items():
|
||||||
|
if entity in entities:
|
||||||
|
return _FakeResult(self._filter(entity, items, stmt))
|
||||||
|
return _FakeResult([])
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _filter(entity, items, stmt):
|
||||||
|
"""RawEvent 查询按 source_record_id 过滤 —— 幂等测试需区分不同键。"""
|
||||||
|
if entity is RawEvent:
|
||||||
|
try:
|
||||||
|
params = stmt.compile().params
|
||||||
|
except Exception:
|
||||||
|
return items
|
||||||
|
rid = next((v for k, v in params.items() if "source_record_id" in k), None)
|
||||||
|
if rid is not None:
|
||||||
|
return [i for i in items if i.source_record_id == rid]
|
||||||
|
return items
|
||||||
|
|
||||||
|
async def flush(self):
|
||||||
|
for obj in self.added:
|
||||||
|
if getattr(obj, "id", None) is None:
|
||||||
|
self._next_id += 1
|
||||||
|
obj.id = self._next_id
|
||||||
|
|
||||||
|
|
||||||
|
def _preset_league():
|
||||||
|
lg = League(code="EPL", name="Premier League", country="England")
|
||||||
|
lg.id = 42
|
||||||
|
return lg
|
||||||
|
|
||||||
|
|
||||||
|
def _raw_events(db):
|
||||||
|
return [o for o in db.added if isinstance(o, RawEvent)]
|
||||||
|
|
||||||
|
|
||||||
|
def _lineages(db):
|
||||||
|
return [o for o in db.added if isinstance(o, DataLineage)]
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# 1. 成功 upsert → RawEvent + DataLineage
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
|
||||||
|
class TestStandingsBronzeOnUpsert:
|
||||||
|
async def test_upsert_writes_raw_event_and_lineage(self, monkeypatch):
|
||||||
|
_patch_fetch(monkeypatch, _payload())
|
||||||
|
db = _FakeDB(leagues=[_preset_league()])
|
||||||
|
|
||||||
|
result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
assert result["errors"] == []
|
||||||
|
assert result["total_upserted"] == 2
|
||||||
|
|
||||||
|
raws = _raw_events(db)
|
||||||
|
assert len(raws) == 1
|
||||||
|
raw = raws[0]
|
||||||
|
assert raw.source_system == "bzzoiro"
|
||||||
|
# 幂等键: 联赛 + 实际入库的赛季标签(由载荷日期推导,与 Standing.season 同口径)
|
||||||
|
assert raw.source_record_id == "standings:EPL:2025-2026"
|
||||||
|
assert raw.ingest_batch_id.startswith("bzzoiro-standings-EPL-")
|
||||||
|
# 整份原始载荷完整保留
|
||||||
|
assert raw.raw_payload["standings"][0]["team_name"] == "Arsenal FC"
|
||||||
|
|
||||||
|
lineages = _lineages(db)
|
||||||
|
assert len(lineages) == 1
|
||||||
|
lin = lineages[0]
|
||||||
|
assert lin.source_system == "bzzoiro"
|
||||||
|
assert lin.source_record_id == "standings:EPL:2025-2026"
|
||||||
|
assert lin.target_table == "standings"
|
||||||
|
assert lin.target_id == 42 # 联赛 id
|
||||||
|
assert lin.transform_name == "standings_ingest"
|
||||||
|
assert lin.transform_detail == {
|
||||||
|
"league": "EPL", "season": "2025-2026", "rows_upserted": 2,
|
||||||
|
}
|
||||||
|
# RawEvent 与 Lineage 同批次,便于按批追溯
|
||||||
|
assert lin.batch_id == raw.ingest_batch_id
|
||||||
|
|
||||||
|
async def test_updated_snapshot_also_writes_bronze(self, monkeypatch):
|
||||||
|
"""已有快照就地更新(非插入)同样是成功采集,必须留 Bronze 记录。"""
|
||||||
|
payload = _payload()
|
||||||
|
payload["standings"] = payload["standings"][:1] # 单队,便于命中同一行
|
||||||
|
_patch_fetch(monkeypatch, payload)
|
||||||
|
|
||||||
|
team = Team(name="Arsenal FC", name_zh="阿森纳")
|
||||||
|
team.id = 7
|
||||||
|
existing = Standing(league_id=42, season="2025-2026", team_id=7, position=9)
|
||||||
|
existing.points = 1
|
||||||
|
db = _FakeDB(leagues=[_preset_league()], teams=[team], standings=[existing])
|
||||||
|
|
||||||
|
result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
assert result["total_upserted"] == 1
|
||||||
|
assert result["leagues"]["EPL"]["teams_created"] == 0
|
||||||
|
# 快照刷新也要留痕: RawEvent(幂等) + 血缘
|
||||||
|
assert len(_raw_events(db)) == 1
|
||||||
|
lineages = _lineages(db)
|
||||||
|
assert len(lineages) == 1
|
||||||
|
assert lineages[0].transform_name == "standings_ingest"
|
||||||
|
assert lineages[0].transform_detail["rows_upserted"] == 1
|
||||||
|
|
||||||
|
async def test_empty_upsert_writes_no_bronze(self, monkeypatch):
|
||||||
|
"""载荷有行但全部队名为空 → 没有任何 upsert,不应产生 RawEvent/Lineage。"""
|
||||||
|
payload = {"standings": [{"position": 1, "team_name": ""}]}
|
||||||
|
_patch_fetch(monkeypatch, payload)
|
||||||
|
db = _FakeDB(leagues=[_preset_league()])
|
||||||
|
|
||||||
|
result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
assert result["total_upserted"] == 0
|
||||||
|
assert _raw_events(db) == []
|
||||||
|
assert _lineages(db) == []
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# 2. RawEvent 幂等: 同 source_record_id 跳过
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
|
||||||
|
class TestStandingsRawEventIdempotent:
|
||||||
|
async def test_existing_raw_event_is_skipped(self, monkeypatch):
|
||||||
|
existing = RawEvent(
|
||||||
|
source_system="bzzoiro",
|
||||||
|
source_record_id="standings:EPL:2025-2026",
|
||||||
|
raw_payload={"old": True},
|
||||||
|
)
|
||||||
|
_patch_fetch(monkeypatch, _payload())
|
||||||
|
db = _FakeDB(leagues=[_preset_league()], raw_events=[existing])
|
||||||
|
|
||||||
|
await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
new_raws = [r for r in _raw_events(db) if r is not existing]
|
||||||
|
assert new_raws == []
|
||||||
|
assert len(_lineages(db)) == 1 # 血缘仍然记录本次采集
|
||||||
|
|
||||||
|
async def test_different_season_writes_new_raw_event(self, monkeypatch):
|
||||||
|
"""幂等键含赛季: 同联赛不同赛季各留一条 RawEvent。"""
|
||||||
|
payload = _payload()
|
||||||
|
payload["season"] = {"start_date": "2024-08-01", "end_date": "2025-05-31"}
|
||||||
|
existing = RawEvent(
|
||||||
|
source_system="bzzoiro",
|
||||||
|
source_record_id="standings:EPL:2025-2026",
|
||||||
|
raw_payload={"old": True},
|
||||||
|
)
|
||||||
|
_patch_fetch(monkeypatch, payload)
|
||||||
|
db = _FakeDB(leagues=[_preset_league()], raw_events=[existing])
|
||||||
|
|
||||||
|
await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
new_raws = [r for r in _raw_events(db) if r is not existing]
|
||||||
|
assert len(new_raws) == 1
|
||||||
|
assert new_raws[0].source_record_id == "standings:EPL:2024-2025"
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# 3. 基础设施写入失败: 尽力而为,不拖垮主流程
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
|
||||||
|
class TestStandingsBronzeIsBestEffort:
|
||||||
|
async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch):
|
||||||
|
async def _boom(*args, **kwargs):
|
||||||
|
raise RuntimeError("infra down")
|
||||||
|
|
||||||
|
monkeypatch.setattr(bz, "_write_raw_event", _boom)
|
||||||
|
monkeypatch.setattr(bz, "_write_lineage", _boom)
|
||||||
|
_patch_fetch(monkeypatch, _payload())
|
||||||
|
db = _FakeDB(leagues=[_preset_league()])
|
||||||
|
|
||||||
|
# 不应抛异常:Bronze 写不进去只记 warning
|
||||||
|
result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
|
||||||
|
|
||||||
|
assert result["total_upserted"] == 2
|
||||||
|
assert [o for o in db.added if isinstance(o, Standing)]
|
||||||
|
|
||||||
|
|
||||||
|
# ============================================================
|
||||||
|
# 4. 抓取失败: 继续写死信,且不写 Bronze
|
||||||
|
# ============================================================
|
||||||
|
|
||||||
|
|
||||||
|
class TestStandingsFailurePathKeepsDeadLetter:
|
||||||
|
async def test_fetch_failure_writes_deadletter_and_no_bronze(self, monkeypatch):
|
||||||
|
async def _boom(league_code, season=None):
|
||||||
|
raise RuntimeError("upstream 500")
|
||||||
|
|
||||||
|
monkeypatch.setattr(bz, "fetch_bzzoiro_standings", _boom)
|
||||||
|
db = _FakeDB()
|
||||||
|
|
||||||
|
result = await bz.ingest_bzzoiro_standings(db, leagues=["SP1"], season="2025-2026")
|
||||||
|
|
||||||
|
assert result["errors"]
|
||||||
|
failures = [o for o in db.added if isinstance(o, IngestFailure)]
|
||||||
|
assert len(failures) == 1
|
||||||
|
assert failures[0].entity_type == "standings"
|
||||||
|
assert failures[0].error_type == "fetch_error"
|
||||||
|
# 失败路径绝不写 Bronze(没有任何成功 upsert)
|
||||||
|
assert _raw_events(db) == []
|
||||||
|
assert _lineages(db) == []
|
||||||
Reference in New Issue
Block a user