feat: 足球 LLM 预测服务初始提交
Profeto — 给 LLM 提供数据,让 LLM 预测足球比分。 核心模块: - FastAPI 后端 + PostgreSQL (SQLAlchemy async) - 多 Agent LLM 预测 (5 专家 + 终裁) - 数据采集 (bzzoiro / understat / injuries) - React 前端 (Vite + Tailwind) 包含: - 数据源抽象 (DataSource 协议 + 注册表) - Alembic 数据库迁移 - Prompt 模板 (单/多 Agent) - 核心路径单元测试
This commit is contained in:
shangfangjian
+175
@@ -0,0 +1,175 @@
|
||||
# 05 · 数据层与数据库
|
||||
|
||||
## 数据源
|
||||
|
||||
| 数据源 | 用途 | 必需 Key | 说明 |
|
||||
|---|---|---|---|
|
||||
| bzzoiro | 赛果/赛程(主源) | `BZZOIRO_KEY` | 五大联赛历史 + 实时 |
|
||||
| understat | xG 回填 | 无(公开) | 仅五大联赛,补 `match_stats.xg` |
|
||||
| api-football | 伤停 | `API_FOOTBALL_KEY` | 当前只采集计数,未接入 context |
|
||||
|
||||
### bzzoiro
|
||||
|
||||
- 端点:`/api/v2/events/`,按 `league_id` + 日期范围分页
|
||||
- 限速:`REQUEST_INTERVAL = 1.2s`,429 自动重试 3 次(轮换 key)
|
||||
- 联赛映射(`src/data/config.py`):
|
||||
```python
|
||||
BZZOIRO_LEAGUE_IDS = {"E0": 1, "SP1": 3, "D1": 5, "I1": 4, "F1": 6, "CL": 7, "EL": 8}
|
||||
```
|
||||
|
||||
### understat
|
||||
|
||||
- 端点:`/getLeagueData/{league}/{season}`,返回 JS 包裹的 JSON(需正则提取)
|
||||
- 只回填 xG(`match_stats.home_xg`/`away_xg`),**不创建新比赛**
|
||||
- 通过"天级日期 + 队名归一"匹配已有比赛
|
||||
|
||||
## 数据清洗契约
|
||||
|
||||
所有数据源统一清洗为 `NormalizedMatch`(`src/data/normalize.py`),字段:
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|---|---|---|
|
||||
| `league_type` | str | 联赛代码(小写,如 `E0`) |
|
||||
| `date` | datetime | UTC 时间 |
|
||||
| `home_team` / `away_team` | str | 归一化后的规范名 |
|
||||
| `match_status` | str | `finished`/`scheduled`/... |
|
||||
| `home_goals` / `away_goals` | int? | 全场比分 |
|
||||
| `home_ht_goals` / `away_ht_goals` | int? | 半场比分 |
|
||||
| `home_xg` / `away_xg` | float? | 期望进球 |
|
||||
| `home_shots` / `away_shots` | int? | 射门 |
|
||||
| `home_shots_on_target` / `away_shots_on_target` | int? | 射正 |
|
||||
| `home_corners` / `away_corners` | int? | 角球 |
|
||||
| `home_possession` | float? | 主队控球率 |
|
||||
| `home_yellow_cards` / `away_yellow_cards` | int? | 黄牌 |
|
||||
| `home_red_cards` / `away_red_cards` | int? | 红牌 |
|
||||
| `match_stage` | str? | 轮次(如"第 5 轮") |
|
||||
| `season_label` | str | 赛季标签(如 `2026-2027`) |
|
||||
|
||||
### 校验规则(`validate()`)
|
||||
|
||||
- `finished` 无比分 → 抛错(或降级为 `scheduled`)
|
||||
- 比分 0–30,xG 0–20,射门/射正/角球 0–100,红黄牌 0–20,控球率 0–100
|
||||
- 半场比分 ≤ 全场比分
|
||||
- 无比分小数(`_to_int` 严格: `"2.8"` → `None`,不截断)
|
||||
|
||||
### 队名归一化
|
||||
|
||||
`src/data/team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
|
||||
归一前先做 Unicode NFKD 去重音。
|
||||
|
||||
## 数据库 Schema
|
||||
|
||||
5 张表:
|
||||
|
||||
```sql
|
||||
-- 联赛
|
||||
CREATE TABLE leagues (
|
||||
id SERIAL PRIMARY KEY,
|
||||
code VARCHAR(20) UNIQUE NOT NULL, -- 'E0' / 'SP1'
|
||||
name VARCHAR(100) NOT NULL,
|
||||
country VARCHAR(50),
|
||||
created_at TIMESTAMPTZ
|
||||
);
|
||||
|
||||
-- 球队
|
||||
CREATE TABLE teams (
|
||||
id SERIAL PRIMARY KEY,
|
||||
name VARCHAR(120) UNIQUE NOT NULL, -- 规范名(归一后)
|
||||
name_zh VARCHAR(60), -- 中文名
|
||||
team_type VARCHAR(20) DEFAULT 'club',
|
||||
created_at TIMESTAMPTZ
|
||||
);
|
||||
|
||||
-- 比赛(核心)
|
||||
CREATE TABLE matches (
|
||||
id SERIAL PRIMARY KEY,
|
||||
league_id INT REFERENCES leagues(id),
|
||||
season VARCHAR(12), -- '2026-2027'
|
||||
home_team_id INT REFERENCES teams(id),
|
||||
away_team_id INT REFERENCES teams(id),
|
||||
match_date TIMESTAMPTZ NOT NULL,
|
||||
match_date_date DATE NOT NULL, -- 天级日期(去重键)
|
||||
match_status VARCHAR(20) DEFAULT 'scheduled',
|
||||
home_goals INT, away_goals INT,
|
||||
home_ht_goals INT, away_ht_goals INT,
|
||||
match_stage VARCHAR(100),
|
||||
created_at TIMESTAMPTZ, updated_at TIMESTAMPTZ
|
||||
);
|
||||
-- 唯一约束: 同联赛同对阵同天只存一场(天级去重)
|
||||
CREATE UNIQUE INDEX ix_matches_unique
|
||||
ON matches(league_id, home_team_id, away_team_id, match_date_date);
|
||||
|
||||
-- 比赛统计(xG、射门、控球等)
|
||||
CREATE TABLE match_stats (
|
||||
match_id INT PRIMARY KEY REFERENCES matches(id) ON DELETE CASCADE,
|
||||
home_xg FLOAT, away_xg FLOAT,
|
||||
home_shots INT, away_shots INT,
|
||||
home_shots_on_target INT, away_shots_on_target INT,
|
||||
home_corners INT, away_corners INT,
|
||||
home_possession FLOAT,
|
||||
home_yellow_cards INT, away_yellow_cards INT,
|
||||
home_red_cards INT, away_red_cards INT,
|
||||
updated_at TIMESTAMPTZ
|
||||
);
|
||||
|
||||
-- LLM 预测记录
|
||||
CREATE TABLE predictions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
match_id INT REFERENCES matches(id) ON DELETE CASCADE,
|
||||
provider VARCHAR(30) NOT NULL, -- 'openai' / 'anthropic'
|
||||
model VARCHAR(80) NOT NULL,
|
||||
prompt_version VARCHAR(20) NOT NULL DEFAULT 'v1',
|
||||
mode VARCHAR(20) NOT NULL DEFAULT 'single', -- 'single' / 'multi'
|
||||
prompt_tokens INT, completion_tokens INT,
|
||||
latency_ms INT,
|
||||
pred_home_goals FLOAT, pred_away_goals FLOAT,
|
||||
pred_1x2 VARCHAR(3), -- '1' / 'X' / '2'
|
||||
confidence FLOAT,
|
||||
reasoning TEXT,
|
||||
raw_response JSONB, -- LLM 完整原始响应
|
||||
agent_outputs JSONB, -- multi 模式: 5 份专家报告
|
||||
created_at TIMESTAMPTZ,
|
||||
actual_home_goals INT, actual_away_goals INT, -- 赛后回填
|
||||
settled BOOLEAN DEFAULT FALSE
|
||||
);
|
||||
```
|
||||
|
||||
### 关键设计点
|
||||
|
||||
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
|
||||
|
||||
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。
|
||||
|
||||
3. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测。
|
||||
|
||||
4. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比。
|
||||
|
||||
## 入库语义(幂等)
|
||||
|
||||
`ingest_bzzoiro` 的 upsert 逻辑:
|
||||
|
||||
- **不存在**: 插入新比赛 + 初始 stats
|
||||
- **已存在**: 只补空字段
|
||||
- 比分:只在原记录为 `None` 时覆盖
|
||||
- 状态:只允许单向升级(`scheduled` → `finished`),防止完赛行被覆盖成赛程
|
||||
- stats:只补空(`home_xg` 已有值时不覆盖)
|
||||
|
||||
`ingest_understat` 只回填 xG(也只补空),不创建比赛。
|
||||
|
||||
## 采集建议
|
||||
|
||||
```bash
|
||||
# 1. 首次采集: 5 大联赛近 2 赛季赛果
|
||||
curl -X POST /api/v1/ingest/bzzoiro \
|
||||
-d '{"leagues":["E0","SP1","D1","I1","F1"],"date_from":"2024-08-01","date_to":"2026-09-08"}'
|
||||
|
||||
# 2. 增量采集(每日 cron): 只拉最近 7 天
|
||||
curl -X POST /api/v1/ingest/bzzoiro \
|
||||
-d '{"leagues":["E0"],"date_from":"2026-09-01","date_to":"2026-09-08"}'
|
||||
|
||||
# 3. xG 回填(可选,提升 xg agent 质量)
|
||||
curl -X POST /api/v1/ingest/understat -d '{"league":"E0","season":2025}'
|
||||
curl -X POST /api/v1/ingest/understat -d '{"league":"E0","season":2026}'
|
||||
```
|
||||
|
||||
建议用外部 cron(如系统 crontab)定时触发,不引入 worker/redis。
|
||||
Reference in New Issue
Block a user