17 Commits
Author SHA1 Message Date
shangfangjian b997c06ede fix(P1-D): 全局 LLM 并发限制 + 删除未接线的 provider 字段
- 删除 PredictRequest.provider(未接线,路由从未传入,符合禁令 #7)
- 新增 _GLOBAL_LLM_SEMAPHORE(默认 4) + _predict_with_concurrency:
  公开 /predict 所有模式汇总受全局并发限制,与 orchestrator 内 match 级
  Semaphore(8) 并存。

测试 test_p1_d_concurrency(4/4);全量 313 通过。
2026-09-22 03:44:09 +08:00
shangfangjian 1ddf697c97 fix(P1-C): 公开预测仅 live+success,且不含 reasoning/agent_outputs
GET /matches/{id}:查询加 WHERE run_type='live' AND status='success';
recent_predictions 不再输出 reasoning/agent_outputs(避免泄露内部推理)。
MatchOut.recent_predictions schema 放宽为 list[dict]。

测试 test_p1_c_public_predictions(2/2);全量 309 通过。
2026-09-22 03:39:36 +08:00
shangfangjian 5ff43d4984 fix(P1-A/P1-B): ingest 联赛级计数修正 + 非法 cursor 400
P1-A: ingest 联赛级 inserted/updated 读 r["leagues"][code] 而非顶层 r.get("inserted");
抽 _accumulate_ingest_result 纯函数 + 合约测试(4/4)。

P1-B: 非法 cursor 不再静默忽略,返回 400 + detail.code=INVALID_CURSOR;
抽 _parse_cursor 纯函数 + 测试(8/8)。

全量 307 通过。
2026-09-22 03:31:40 +08:00
shangfangjian 41cb2edd47 fix(P1-A): ingest 联赛级计数读 leagues[code],抽为纯函数
旧代码读 r.get("inserted")(顶层无此 key)导致联赛级计数总为 0;
改为读 r["leagues"][code].inserted/updated,与顶层 total_* 分离。
抽 _accumulate_ingest_result 纯函数 + 4 合约测试(全绿)。
2026-09-22 03:23:41 +08:00
shangfangjian 64ae8e663a fix(P0-03): Prediction 幂等指纹——只追加,不覆盖
_upsert_prediction 改为 _insert_or_find_by_fingerprint:
- 同 input_hash → 返回已有行(绝不 UPDATE pred_/reasoning/agent_outputs)
- 不同 input_hash → INSERT 新行

input_hash 升级为规范 JSON SHA-256,捕获:match_id, cutoff, prompt_version,
prompt_hash, system_prompt_hash, provider, model, mode, run_type, temperature,
context_hash, agent_ids。移除旧 (match, provider, model, mode, run_type) 唯一约束,
改为 partial unique index(WHERE input_hash IS NOT NULL,兼容旧 NULL 数据)。

三条路径(single/multi/baseline)统一传足指纹字段。
迁移 0024 + 测试 test_p0_prediction_fingerprint(10/10);全量 295 通过。
2026-09-22 03:13:32 +08:00
shangfangjian 49d78136a1 fix(P0-02): 积分榜改为追加快照(append-only) + available_at cutoff
去掉 uq_standings_league_season_team,改为 (league, season, team, available_at) 唯一;
每次采集 INSERT 新行(available_at=now),ON CONFLICT DO NOTHING,不覆盖旧行。

standings_slice(before):DISTINCT ON (team_id) WHERE available_at<=cutoff
ORDER available_at DESC;before=None → cutoff=now()。
公开 list_standings 取每队最新可用快照(子查询 max available_at)。

迁移 0023 + 切片/路由/docs 同步;测试 test_p0_standings_cutoff(5/5)。
284 测试全绿。
2026-09-22 02:45:50 +08:00
shangfangjian 63caa6736c fix(P0-01): missing score 不得变 0:0 — score_status + CHECK 约束
新增 matches.score_status(known/missing/unknown):
- 替换 ck_matches_finished_has_score 为 ck_matches_score_integrity:
  known → 必须有分; missing/unknown → goals 必须 NULL(不伪造 0:0)
- normalize: 完赛缺分不再静默降级为 scheduled,改设 score_status=missing
- events ingest: 创建/更新 Match 同步 score_status(比分由缺变 known / 确认缺分 missing)
- slices(form/h2h/home_away)/backtest: 显式加 score_status='known' 过滤完赛样本
- 迁移 0022 回填现有数据(绝不 UPDATE goals=0)

测试 tests/test_p0_score_status.py(9/9):约束存在性/Match 构造/normalize 行为。
54 相关测试全绿。
2026-09-22 02:32:53 +08:00
shangfangjian f563d5cc99 fix(P0-00): HTTP client 必须传递 method/body,仅 body 存在时设 Content-Type
request() 此前忽略 options.method(默认 GET)与 options.body,导致 POST/PUT/DELETE
全部以 GET 空 body 发出;且无条件设置 Content-Type 误污染无 body 请求。

修复: 显式传递 method(默认 GET) 与 body; 仅当 body 非空时加 Content-Type。

测试 frontend/src/lib/http.test.ts(5/5):GET/POST/PUT/DELETE 的 method/body/headers。
运行: node --experimental-transform-types frontend/src/lib/http.test.ts
2026-09-22 02:24:50 +08:00
shangfangjian 7d2eabf750 chore: 死代码与重复逻辑清理
删除未引用/未调用符号:
- PredictionRepository(无引用)
- is_correct_1x2(无调用者)
- LeagueOut(路由用 list[dict])
- IngestResponse(IngestBzzoiroResponse 已替代)
- SecurityCheckError(从未 raise,assert_security_on_startup 用 sys.exit)
- short_write(仅自引用,全仓库无外部调用)
- fetchIngestJobs(列表函数无页面使用,单数 fetchIngestJob 仍保留)
- clear_prompt_cache(无入口)

去重:
- eval._actual_1x2 改为委托 utils.actual_1x2(单一权威源)

全量测试 270 通过,业务行为不变。
2026-09-22 01:55:52 +08:00
shangfangjian a00364d4a7 perf+unify: matches 列表查询优化 + Standings 统一数据源
matches 列表:selectinload 加 load_only 限定列(League.code/Team.name,name_zh/
MatchStats.home_xg,away_xg),补 stats 加载消除 N+1(m.stats 此前懒加载)。
详情接口保持完整 options;游标分页/响应字段/空值语义不变。

Standings 改用 useLeagues() 统一数据源(API 优先,失败回退本地常量),
LEAGUES 常量扩展 CL/EL;无数据联赛显示虚线 tab + 空态(非隐藏)。
2026-09-22 01:35:06 +08:00
shangfangjian 92f50fa5d4 feat: 采集任务状态跟踪 ingest_jobs
新增 ingest_jobs 表(UUID/task/params/status/result/error/时间戳),
POST /ingest/bzzoiro 启动前插入 job(pending)→ 后台 running → success/failed,
响应新增 job_id(兼容原 message)。

Admin GET /admin/ingest/jobs/{id} 与 /admin/ingest/jobs?limit= 只读查询;
Collection 页提交后轮询 job 至终态,展示真实 result/error 汇总。
迁移 0019_ingest_jobs + 分批 get_uow/BzzoiroSource/IngestFailure 不变。

全量测试 270 通过。
2026-09-22 01:34:55 +08:00
shangfangjian 44816794d3 refactor: context_builder 按 slice 拆到 src/llm/slices/ 包
单文件拆分(仅搬迁无逻辑修改):
- common.py    共享类型/头信息/_outcome/_is_stats_available
- form.py      form_slice + _get_form
- h2h.py       h2h_slice + _get_h2h
- stats.py     stats_slice(复用 form._get_form)
- home_away.py home_away_slice + _get_home_away
- standings.py standings_slice
- aggregate.py build_context

context_builder.py 改为纯 re-export 门面,公开签名不变。
同步修复测试 patch 目标(p0_home_away/h2h_perspective/multi_agent_cutoff)
与 regressions 源码断言(读 slices/*.py)。

全量测试 270 通过。
2026-09-22 01:33:14 +08:00
shangfangjian f1016b610a docs: predict.py docstring 补落库层级表格
明确 single/multi/baseline 均在服务层(_upsert_prediction)落库,
路由层永不写入 predictions,仅做 result → PredictOut 映射。
2026-09-22 01:19:19 +08:00
shangfangjian 03727bda00 refactor: 管线写入依赖解耦——pipeline_write 直接 import
bzzoiro_events / bzzoiro_standings / bzzoiro_stats 直接 import
src.data.pipeline_write(_write_raw_event/_write_lineage/_safe_write_ingest_failure),
删除 bzzoiro.py 门面中的 pipeline_write 转发胶水。

保留 fetch_*/_fetch_json_async/REQUEST_INTERVAL 经 bz. 门面调用(测试 monkeypatch 入口);
测试 best-effort 改为 patch 管线模块自身命名空间(from-import 绑定语义)。
函数语义与「失败不拖垮主流程」不变;source_record_id/transform_name 约定不变。

全量测试 270 通过。
2026-09-22 01:15:45 +08:00
shangfangjian 3a56ee17e1 feat: 预测缓存可选 Redis 后端(PREDICT_CACHE_URL)
PREDICT_CACHE_URL 为空=进程内 LRU+TTL dict(默认,行为不变);
填 redis:// 启用 Redis,失败自动降级内存并 warning,不中断预测。

- _CacheBackend 接口 + _MemoryCache/_RedisCache 两个后端
- 同键格式(predict:{match}:{provider}:{model}:{version}:{tpl_hash[:12}])
- 同 TTL(300s);Redis 用 pickle 序列化
- redis 包未安装/连接失败 → 降级内存;不强制依赖 redis 启动
- clear_prompt_cache 同步清空内存预测缓存

全量测试 270 通过;缓存后端单测 4/4。
2026-09-22 00:53:27 +08:00
shangfangjian 7593b99e39 docs+feat: matches 唯一键语义明确化 + source_event_id partial unique
docs/05-data.md:业务唯一=同联赛同主客同自然天;
source_event_id 用于统计回填与血缘,新增部分唯一索引说明与 upsert 查找顺序。

新增 ix_matches_source_event_id_unique(WHERE IS NOT NULL),
兼容存量空值历史行;MatchRepository.find_by_source_event_id;
events upsert 优先按 event_id 定位,回退自然键。
迁移 0021 + 回归测试修复(find_by_source_event_id 方法调用误判)。
2026-09-22 00:41:54 +08:00
shangfangjian e15b554ba3 feat: 球队实体一致性 — 归一化咽喉 + team_aliases 别名机制
events/standings 创建 Team 前均经 team_names.normalize(已有,确认),
TeamRepository.get_or_create 收敛为归一化唯一咽喉 + info 日志。

新增 team_aliases 表(NFKD 归一别名 → teams.id FK CASCADE),
定位三步链:normalize(name) → teams.name → team_aliases → insert。
不自动合并历史重复队;提供 POST /api/v1/admin/teams/aliases 显式添加。

迁移 0020_team_aliases + Admin 别名管理端点(admin_teams.py)。
全量测试 270 通过。
2026-09-22 00:27:53 +08:00
65 changed files with 2928 additions and 1034 deletions
+42
View File
@@ -0,0 +1,42 @@
"""采集任务状态表 ingest_jobs
Revision ID: 0019_ingest_jobs
Revises: 0018_match_checks
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0019_ingest_jobs'
down_revision: Union[str, None] = '0018_match_checks'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
'ingest_jobs',
sa.Column('id', sa.String(36), primary_key=True),
sa.Column('task', sa.String(20), nullable=False),
sa.Column('params', sa.JSON(), nullable=False, server_default='{}'),
sa.Column('status', sa.String(20), nullable=False, server_default='pending'),
sa.Column('result', sa.JSON(), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
)
op.create_index('ix_ingest_job_status_created', 'ingest_jobs', ['status', 'created_at'])
op.create_check_constraint(
'ck_ingest_job_status', 'ingest_jobs',
"status IN ('pending', 'running', 'success', 'failed')",
)
def downgrade() -> None:
op.drop_constraint('ck_ingest_job_status', 'ingest_jobs', type_='check')
op.drop_index('ix_ingest_job_status_created', table_name='ingest_jobs')
op.drop_table('ingest_jobs')
+32
View File
@@ -0,0 +1,32 @@
"""球队别名表 team_aliases
Revision ID: 0020_team_aliases
Revises: 0019_ingest_jobs
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0020_team_aliases'
down_revision: Union[str, None] = '0019_ingest_jobs'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
'team_aliases',
sa.Column('alias_normalized', sa.String(120), primary_key=True),
sa.Column('team_id', sa.Integer, sa.ForeignKey('teams.id', ondelete='CASCADE'), nullable=False),
sa.Column('original_alias', sa.String(120), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now()),
)
op.create_index('ix_team_aliases_team_id', 'team_aliases', ['team_id'])
def downgrade() -> None:
op.drop_index('ix_team_aliases_team_id', table_name='team_aliases')
op.drop_table('team_aliases')
@@ -0,0 +1,38 @@
"""matches.source_event_id 部分唯一索引
业务唯一键:同联赛同主客同自然天一条(ix_matches_unique,既有)。
source_event_id 是上游 bzzoiro 的比赛 id,用于统计回填与血缘追踪;
当它非空时应全局唯一(同一 upstream 比赛只对应一行 matches),
避免同一场比赛因自然键天级舍入差异产生重复。
partial unique(WHERE source_event_id IS NOT NULL):
- 兼容存量空 source_event_id 的历史行(不强制回填);
- 新采集行均带 source_event_id,从此具备 upstream 唯一性。
Revision ID: 0021_match_source_event_id_unique
Revises: 0020_team_aliases
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
revision: str = '0021_match_source_event_id_unique'
down_revision: Union[str, None] = '0020_team_aliases'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_index(
'ix_matches_source_event_id_unique',
'matches',
['source_event_id'],
unique=True,
postgresql_where=op.text('source_event_id IS NOT NULL'),
)
def downgrade() -> None:
op.drop_index('ix_matches_source_event_id_unique', table_name='matches')
@@ -0,0 +1,63 @@
"""P0-01: 比分可信度——score_status + 允许完赛缺分(NULL,禁止伪造 0:0)
替换 ck_matches_finished_has_score:引入 score_status(known/missing/unknown),
完赛 + score_status=missing 时 home/away_goals 必须 NULL(不伪造比分)。
Revision ID: 0022_match_score_status
Revises: 0021_match_source_event_id_unique
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0022_match_score_status'
down_revision: Union[str, None] = '0021_match_source_event_id_unique'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 1) 新增 score_status 列(默认 unknown)
op.add_column(
'matches',
sa.Column('score_status', sa.String(20), server_default='unknown', nullable=False),
)
# 2) 按现有数据回填 score_status(绝不写 goals=0):
# - 有比分(两列均非 NULL) → known
# - 无比分 + 完赛 → missing(缺分)
# - 其余 → unknown
op.execute(
"UPDATE matches SET score_status = 'known'"
" WHERE home_goals IS NOT NULL AND away_goals IS NOT NULL"
)
op.execute(
"UPDATE matches SET score_status = 'missing'"
" WHERE match_status = 'finished' AND home_goals IS NULL AND away_goals IS NULL"
)
# 3) 删除旧约束,加新约束
op.drop_constraint('ck_matches_finished_has_score', 'matches', type_='check')
op.create_check_constraint(
'ck_matches_score_status_enum', 'matches',
"score_status IN ('known', 'missing', 'unknown')",
)
op.create_check_constraint(
'ck_matches_score_integrity', 'matches',
"match_status <> 'finished'"
" OR (score_status = 'known' AND home_goals IS NOT NULL AND away_goals IS NOT NULL)"
" OR (score_status IN ('missing', 'unknown') AND home_goals IS NULL AND away_goals IS NULL)",
)
def downgrade() -> None:
op.drop_constraint('ck_matches_score_integrity', 'matches', type_='check')
op.drop_constraint('ck_matches_score_status_enum', 'matches', type_='check')
op.create_check_constraint(
'ck_matches_finished_has_score', 'matches',
"match_status <> 'finished' OR (home_goals IS NOT NULL AND away_goals IS NOT NULL)",
)
op.remove_column('matches', 'score_status')
@@ -0,0 +1,58 @@
"""P0-02: 积分榜改为追加快照(append-only) + available_at
去掉 uq_standings_league_season_team(league,season,team 唯一),
改为 (league, season, team, available_at) 唯一;
每次采集 INSERT 新行(available_at=now),支持回测还原历史榜单。
Revision ID: 0023_standings_append_only
Revises: 0022_match_score_status
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0023_standings_append_only'
down_revision: Union[str, None] = '0022_match_score_status'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
from datetime import datetime, timezone
# 1) 加 available_at 列(非空,默认 now;存量回填 retrieved_at 或 now)
op.add_column(
'standings',
sa.Column('available_at', sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now()),
)
# 存量行: available_at 取 retrieved_at(若存在)否则 now
op.execute("UPDATE standings SET available_at = COALESCE(retrieved_at, NOW())")
# 2) 去旧唯一约束,加新唯一约束(league, season, team, available_at)
op.drop_constraint('uq_standings_league_season_team', 'standings', type_='unique')
op.drop_index('ix_standings_leason_season_pos', table_name='standings')
op.create_index('ix_standings_league_season_pos', 'standings', ['league_id', 'season', 'position'])
op.create_unique_constraint(
'uq_standings_league_season_team_available', 'standings',
['league_id', 'season', 'team_id', 'available_at'],
)
op.create_index(
'ix_standings_league_season_team_available', 'standings',
['league_id', 'season', 'team_id', 'available_at'],
)
def downgrade() -> None:
op.drop_index('ix_standings_league_season_team_available', table_name='standings')
op.drop_constraint('uq_standings_league_season_team_available', 'standings', type_='unique')
op.drop_index('ix_standings_league_season_pos', table_name='standings')
op.create_index('ix_standings_leason_season_pos', 'standings', ['league_id', 'season', 'position'])
op.create_unique_constraint(
'uq_standings_league_season_team', 'standings',
['league_id', 'season', 'team_id'],
)
op.drop_column('standings', 'available_at')
@@ -0,0 +1,41 @@
"""P0-03: Prediction 幂等指纹——移除旧唯一约束,改为 partial unique on input_hash
input_hash 非空时唯一(同指纹返回已有行,不 UPDATE/INSERT);
兼容旧数据 NULL input_hash(不强制回填)。
Revision ID: 0024_prediction_idempotent_fingerprint
Revises: 0023_standings_append_only
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0024_prediction_idempotent_fingerprint'
down_revision: Union[str, None] = '0023_standings_append_only'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 移除旧唯一约束(match, provider, model, mode, run_type)
op.drop_constraint(
'uq_predictions_match_provider_model_mode_run_type',
'predictions', type_unique=True,
)
# P0-03: partial unique on input_hash(非空时唯一)
op.create_index(
'ix_predictions_input_hash_unique', 'predictions', ['input_hash'], unique=True,
postgresql_where=sa.text('input_hash IS NOT NULL'),
)
def downgrade() -> None:
op.drop_index('ix_predictions_input_hash_unique', table_name='predictions')
op.create_unique_constraint(
'uq_predictions_match_provider_model_mode_run_type',
'predictions',
['match_id', 'provider', 'model', 'mode', 'run_type'],
)
+46 -6
View File
@@ -60,7 +60,7 @@
### 队名归一化 ### 队名归一化
`src/data/team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。 `src/data.team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
归一前先做 Unicode NFKD 去重音。 归一前先做 Unicode NFKD 去重音。
**唯一键是归一后英文名**:`teams.name` 带 `UNIQUE` 约束,所有入库路径均经 `TeamRepository.get_or_create` 收敛归一化 **唯一键是归一后英文名**:`teams.name` 带 `UNIQUE` 约束,所有入库路径均经 `TeamRepository.get_or_create` 收敛归一化
@@ -70,6 +70,32 @@
>(如 `"Man City"` → `"Manchester City"`,但 `"man city"` 原样保留)。上游 bzzoiro 返回的队名首字母大写, >(如 `"Man City"` → `"Manchester City"`,但 `"man city"` 原样保留)。上游 bzzoiro 返回的队名首字母大写,
>实际命中无问题;若新增数据源返回全小写/全大写队名,需先 `title()` 再归一,否则会绕过映射产生重复 Team。 >实际命中无问题;若新增数据源返回全小写/全大写队名,需先 `title()` 再归一,否则会绕过映射产生重复 Team。
#### 别名机制(`team_aliases`)
归一仍可能遗漏历史重复队(如 `"Bayern Munich"` 与 `"Bayern München"` 经 NFKD 后相同则命中,
但 `"Man United"` vs `"Manchester United"` 若漏映射)。`team_aliases` 表提供**显式别名→teams.id** 映射:
| 列 | 说明 |
|----|------|
| `alias_normalized` | PK,`normalize(别名)` 后的稳定幂等键 |
| `team_id` | FK → `teams.id`(ON DELETE CASCADE) |
| `original_alias` | 原始写法(保留供参考) |
**定位三步链**(`get_or_create`):`normalize(name)` → 查 `teams.name` → 查 `team_aliases`(以 `normalize(name)` 为 PK)→ 都没有才 insert 新 Team。别名命中即复用已有 Team,避免产生重复。
**添加别名**(不自动合并历史重复队):
- **Admin 接口**(推荐):`POST /api/v1/admin/teams/aliases {"alias": "Man United", "team_id": 42}`(require_admin,幂等)
- **直接 SQL**:
```sql
INSERT INTO team_aliases(alias_normalized, team_id, original_alias)
VALUES ('man united', 42, 'Man United')
ON CONFLICT (alias_normalized) DO UPDATE SET team_id = EXCLUDED.team_id, original_alias = EXCLUDED.original_alias;
```
> ⚠️ **别名不自动合并**:发现历史重复队 A/B 后,需人工确认归一目标(如保留 B),再为 A 的归一名添加别名指向 B。
> 合并前请确认 A 的 `matches`/`standings` 引用是否需要迁移(可先 `SELECT COUNT(*) FROM matches WHERE home_team_id = A.id OR away_team_id = A.id` 评估)。
**改名 / 合并流程**(人工): **改名 / 合并流程**(人工):
当发现两个 `teams` 行实际是同一球队(如 `Manchester City` 与 `Man City` 因历史数据大小写差异各占一行): 当发现两个 `teams` 行实际是同一球队(如 `Manchester City` 与 `Man City` 因历史数据大小写差异各占一行):
@@ -180,7 +206,7 @@ CREATE TABLE predictions (
| 表 | 状态 | 用途 | | 表 | 状态 | 用途 |
|---|---|---| |---|---|---|
| `standings` | 已启用 | 联赛积分榜快照,按 `(league_id, season, team_id)` upsert,同联赛同赛季只保留最新快照;含排名/战绩/进失球/积分/分区(zone) | | `standings` | 已启用 | 联赛积分榜追加快照(P0-02):每次采集 INSERT 新行(available_at=now),唯一键 `(league_id, season, team_id, available_at)`;查询取每队 available_at 最新快照,支持回测还原历史榜单。含排名/战绩/进失球/积分/分区(zone) |
| `app_settings` | 已启用 | 后台运行时设置(如数据源 API Key),读取时优先于 `.env` 默认值 | | `app_settings` | 已启用 | 后台运行时设置(如数据源 API Key),读取时优先于 `.env` 默认值 |
| `schedules` | 已启用 | 定时采集任务配置(task/cron/leagues/enabled),供内置调度器执行 | | `schedules` | 已启用 | 定时采集任务配置(task/cron/leagues/enabled),供内置调度器执行 |
| `raw_events` | 预留未启用 | Bronze 层原始事件存档;规划中用于重放与审计 | | `raw_events` | 预留未启用 | Bronze 层原始事件存档;规划中用于重放与审计 |
@@ -192,11 +218,25 @@ CREATE TABLE predictions (
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。 1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。 2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。**业务唯一:同联赛同主客同自然天一条。**
3. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测。 3. **`source_event_id` 部分唯一**: `ix_matches_source_event_id_unique`(WHERE source_event_id IS NOT NULL)——上游 bzzoiro 的比赛 id,当非空时全局唯一。作用:
- 统计回填(`/events/{id}/stats/`)与 Bronze 血缘(/events/ 采集)通过它定位比赛,不依赖自然键天级舍入;
- 新采集行均带此 id,避免同一 upstream 比赛因时间戳差异绕开自然键产生重复。
- 存量空 source_event_id 历史行不受影响(不强制回填)。
4. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比 4. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测
5. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比。
## 采集 upsert 查找顺序
events 管线按以下优先级定位已有比赛,命中即复用(更新):
1. **`source_event_id`**(upstream event id,唯一索引命中)——最精确,跨自然键舍入差异;
2. **自然键**:`(league_id, home_team_id, away_team_id, match_date_date)`(内存去重,覆盖无 event id 的采集)。
两者都未命中 → insert 新比赛。
## 入库语义(幂等) ## 入库语义(幂等)
@@ -210,7 +250,7 @@ CREATE TABLE predictions (
`task=stats` 只回填统计(xG/射门/控球等,也只补空),不创建比赛。 `task=stats` 只回填统计(xG/射门/控球等,也只补空),不创建比赛。
`task=standings` 按 `(league_id, season, team_id)` upsert 积分榜快照,同一联赛同一赛季只保留最新一份 `task=standings` 追加快照(available_at=now,ON CONFLICT DO NOTHING);公开接口与切片均取每队 available_at 最新快照,支持回测还原历史榜单
## 采集建议 ## 采集建议
+8
View File
@@ -21,6 +21,7 @@ import type {
LLMAgentConfig, LLMAgentConfig,
LogEntry, LogEntry,
IngestSourceStatus, IngestSourceStatus,
IngestJob,
MatchDetailOut, MatchDetailOut,
MatchContextOut, MatchContextOut,
AdminStats, AdminStats,
@@ -362,6 +363,13 @@ export function fetchIngestStatus(): Promise<{ sources: IngestSourceStatus[] }>
return api.get<{ sources: IngestSourceStatus[] }>(`${API_BASE}/admin/ingest/status`) return api.get<{ sources: IngestSourceStatus[] }>(`${API_BASE}/admin/ingest/status`)
} }
/**
* 采集任务状态轮询(单任务)
*/
export function fetchIngestJob(jobId: string): Promise<IngestJob> {
return api.get<IngestJob>(`${API_BASE}/admin/ingest/jobs/${jobId}`)
}
/** /**
* 比赛详情(含最近预测摘要) * 比赛详情(含最近预测摘要)
*/ */
+87 -51
View File
@@ -11,9 +11,9 @@
*/ */
import { useEffect, useState, useCallback, useRef } from 'react' import { useEffect, useState, useCallback, useRef } from 'react'
import { triggerCollection, fetchLeagues, fetchIngestStatus } from '../dal' import { triggerCollection, fetchLeagues, fetchIngestJob } from '../dal'
import type { IngestSourceStatus } from '../types' import type { IngestJob, League } from '../types'
import type { CollectionRequest, League } from '../types' import type { CollectionRequest } from '../types'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner } from '../components' import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner } from '../components'
const TASKS = [ const TASKS = [
@@ -23,7 +23,9 @@ const TASKS = [
{ value: 'all', label: '全量采集', desc: '依次采集比赛 + 积分榜 + 统计回填', icon: '⏵⏵' }, { value: 'all', label: '全量采集', desc: '依次采集比赛 + 积分榜 + 统计回填', icon: '⏵⏵' },
] as const ] as const
type TaskStatus = 'idle' | 'running' | 'done' | 'error' type TaskUIStatus = 'idle' | 'running' | 'done' | 'error'
const TERMINAL_STATUSES: ReadonlySet<string> = new Set(['success', 'failed'])
export default function CollectionPage() { export default function CollectionPage() {
const [leagues, setLeagues] = useState<League[]>([]) const [leagues, setLeagues] = useState<League[]>([])
@@ -49,13 +51,13 @@ export default function CollectionPage() {
const [limit, setLimit] = useState(100) const [limit, setLimit] = useState(100)
const [loading, setLoading] = useState(false) const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null) const [error, setError] = useState<string | null>(null)
const [result, setResult] = useState<{ title: string; detail: string } | null>(null)
// 任务进度反馈 // 任务进度反馈:跟踪真实 ingest_job 状态
const [taskStatus, setTaskStatus] = useState<TaskStatus>('idle') const [taskStatus, setTaskStatus] = useState<TaskUIStatus>('idle')
const [jobId, setJobId] = useState<string | null>(null)
const [jobInfo, setJobInfo] = useState<IngestJob | null>(null)
const [taskStartedAt, setTaskStartedAt] = useState<number | null>(null) const [taskStartedAt, setTaskStartedAt] = useState<number | null>(null)
const pollRef = useRef<ReturnType<typeof setInterval> | null>(null) const pollRef = useRef<ReturnType<typeof setInterval> | null>(null)
const [ingestSnap, setIngestSnap] = useState<IngestSourceStatus | null>(null)
const loadLeagues = useCallback(async () => { const loadLeagues = useCallback(async () => {
const lg = await fetchLeagues() const lg = await fetchLeagues()
@@ -64,30 +66,65 @@ export default function CollectionPage() {
useEffect(() => { loadLeagues() }, [loadLeagues]) useEffect(() => { loadLeagues() }, [loadLeagues])
// 轮询采集状态(任务启动后) // 轮询采集 job 直到终态(success/failed)
const startPolling = useCallback(() => {
if (pollRef.current) clearInterval(pollRef.current)
pollRef.current = setInterval(async () => {
try {
const { sources } = await fetchIngestStatus()
const bz = sources.find(s => s.name === 'bzzoiro')
if (bz) setIngestSnap(bz)
} catch { /* ignore */ }
}, 5_000)
}, [])
const stopPolling = useCallback(() => { const stopPolling = useCallback(() => {
if (pollRef.current) { clearInterval(pollRef.current); pollRef.current = null } if (pollRef.current) { clearInterval(pollRef.current); pollRef.current = null }
}, []) }, [])
useEffect(() => () => stopPolling(), [stopPolling]) useEffect(() => () => stopPolling(), [stopPolling])
const startJobPolling = useCallback((id: string) => {
stopPolling()
const tick = async () => {
try {
const job = await fetchIngestJob(id)
setJobInfo(job)
if (TERMINAL_STATUSES.has(job.status)) {
setTaskStatus(job.status === 'success' ? 'done' : 'error')
stopPolling()
}
} catch { /* 单次轮询失败不影响后续 */ }
}
tick()
pollRef.current = setInterval(tick, 3_000)
}, [stopPolling])
const isEventsTask = task === 'events' || task === 'all' const isEventsTask = task === 'events' || task === 'all'
// 友好汇总 job.result
const jobSummary = (j: IngestJob | null): { title: string; detail: string } | null => {
if (!j) return null
if (j.status === 'failed') {
return { title: '采集失败', detail: j.error || '采集任务异常终止,请到「系统日志」查看详细堆栈。' }
}
if (j.status !== 'success') return null
const r = j.result as Record<string, unknown> | null
if (!r) return { title: '采集完成', detail: '任务成功(无汇总数据)。' }
const ev = r.events as Record<string, unknown> | undefined
const evTotal = ev ? (ev.total_inserted as number ?? 0) + (ev.total_updated as number ?? 0) : 0
const st = r.standings as Record<string, unknown> | undefined
const stTotal = st ? (st.total_upserted as number ?? 0) : 0
const stats = r.stats as Record<string, unknown> | undefined
const statsTotal = stats ? (stats.created as number ?? 0) + (stats.updated as number ?? 0) : 0
const evErr = (ev?.errors as string[] | undefined)?.length ?? 0
const stErr = (st?.errors as string[] | undefined)?.length ?? 0
const statsErr = (stats?.errors as string[] | undefined)?.length ?? 0
const totalErr = evErr + stErr + statsErr
const parts: string[] = []
if (ev) parts.push(`比赛 +${evTotal}`)
if (st) parts.push(`积分榜 +${stTotal}`)
if (stats) parts.push(`统计 +${statsTotal}`)
const detail = parts.length
? `共更新: ${parts.join(' / ')}${totalErr ? `,错误 ${totalErr} 条(见日志)` : ''}`
: '任务成功'
return { title: '采集完成', detail }
}
async function handleSubmit(e: React.FormEvent) { async function handleSubmit(e: React.FormEvent) {
e.preventDefault() e.preventDefault()
setError(null) setError(null)
setResult(null) setJobInfo(null)
setJobId(null)
setLoading(true) setLoading(true)
setTaskStatus('running') setTaskStatus('running')
setTaskStartedAt(Date.now()) setTaskStartedAt(Date.now())
@@ -103,18 +140,15 @@ export default function CollectionPage() {
date_from: isEventsTask ? dateFrom || undefined : undefined, date_from: isEventsTask ? dateFrom || undefined : undefined,
date_to: isEventsTask ? dateTo || undefined : undefined, date_to: isEventsTask ? dateTo || undefined : undefined,
} }
await triggerCollection(body) const res = await triggerCollection(body)
setResult({ const id: string | undefined = res?.job_id
title: '采集任务已启动', if (id) {
detail: '正在后台执行(上游限速时可能需要数分钟)。完成结果与错误请到「系统日志」页查看(支持自动刷新)。', setJobId(id)
}) startJobPolling(id)
// 启动轮询,跟踪状态 } else {
startPolling() // 后端未返回 job_id(旧版兼容):退化为原逻辑
// 30 秒后自动停止轮询并标记完成 setTimeout(() => { setTaskStatus('done'); }, 30_000)
setTimeout(() => { }
setTaskStatus('done')
stopPolling()
}, 30_000)
} catch (err: unknown) { } catch (err: unknown) {
setTaskStatus('error') setTaskStatus('error')
setError(err instanceof Error ? err.message : '采集触发失败') setError(err instanceof Error ? err.message : '采集触发失败')
@@ -125,6 +159,7 @@ export default function CollectionPage() {
} }
const elapsed = taskStartedAt ? Math.round((Date.now() - taskStartedAt) / 1000) : 0 const elapsed = taskStartedAt ? Math.round((Date.now() - taskStartedAt) / 1000) : 0
const summary = jobInfo ? jobSummary(jobInfo) : null
return ( return (
<div className="space-y-6"> <div className="space-y-6">
@@ -248,12 +283,11 @@ export default function CollectionPage() {
{/* 消息提示 */} {/* 消息提示 */}
{error && <Alert kind="error" title="采集失败" message={error} onClose={() => setError(null)} />} {error && <Alert kind="error" title="采集失败" message={error} onClose={() => setError(null)} />}
{result && ( {summary && (
<Alert <Alert
kind="ok" kind={jobInfo?.status === 'success' ? 'ok' : 'error'}
title={result.title} title={summary.title}
message={result.detail || undefined} message={summary.detail}
onClose={() => setResult(null)}
/> />
)} )}
@@ -278,10 +312,10 @@ export default function CollectionPage() {
<div className="space-y-3"> <div className="space-y-3">
<div className="flex items-center gap-2 text-xs text-ink-700"> <div className="flex items-center gap-2 text-xs text-ink-700">
<Spinner /> <Spinner />
<span>, {elapsed}s</span> <span>{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}, {elapsed}s</span>
</div> </div>
<p className="text-2xs text-ink-400"> <p className="text-2xs text-ink-400">
, , 3
</p> </p>
</div> </div>
)} )}
@@ -289,23 +323,25 @@ export default function CollectionPage() {
<div className="space-y-2"> <div className="space-y-2">
<div className="flex items-center gap-2 text-xs text-emerald-700"> <div className="flex items-center gap-2 text-xs text-emerald-700">
<span className="inline-block h-2 w-2 rounded-full bg-emerald-500" /> <span className="inline-block h-2 w-2 rounded-full bg-emerald-500" />
<span>,()</span> <span>{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}</span>
</div> </div>
<p className="text-2xs text-ink-400"> {summary && <p className="text-2xs text-ink-500">{summary.detail}</p>}
</p>
</div> </div>
)} )}
{taskStatus === 'error' && ( {taskStatus === 'error' && (
<p className="text-xs text-press">,</p> <div className="space-y-1">
)} <p className="text-xs text-press">{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}</p>
{ingestSnap?.last_success_at && ( {jobInfo?.error && (
<div className="mt-3 border-t border-ink-100 pt-3"> <p className="text-2xs text-ink-500">{jobInfo.error.slice(0, 200)}</p>
<p className="text-2xs text-ink-400"> )}
bzzoiro : {new Date(ingestSnap.last_success_at).toLocaleString('zh-CN', { hour12: false })}
</p>
</div> </div>
)} )}
{jobInfo?.created_at && (
<p className="mt-2 text-2xs text-ink-400">
{new Date(jobInfo.created_at).toLocaleString('zh-CN', { hour12: false })}
{jobInfo.finished_at && ` · 完成于 ${new Date(jobInfo.finished_at).toLocaleString('zh-CN', { hour12: false })}`}
</p>
)}
</CardBody> </CardBody>
</Card> </Card>
+14
View File
@@ -283,6 +283,20 @@ export interface IngestSourceStatus {
last_failure: IngestLastFailure | null last_failure: IngestLastFailure | null
} }
// ── 采集任务状态 ──────────────────────────────────────────────
export interface IngestJob {
id: string
task: string
params: Record<string, unknown>
status: 'pending' | 'running' | 'success' | 'failed'
result: Record<string, unknown> | null
error: string | null
created_at: string | null
started_at: string | null
finished_at: string | null
}
// ── 比赛详情 ───────────────────────────────────────────────────── // ── 比赛详情 ─────────────────────────────────────────────────────
export interface MatchRecentPrediction { export interface MatchRecentPrediction {
+65
View File
@@ -0,0 +1,65 @@
/**
* P0-00: HTTP client method/body/headers 可信度测试。
* 运行: node --experimental-strip-types frontend/src/lib/http.test.ts
*
* 最小环境 polyfill:Node 22 自带 fetch/AbortController,本测试不触发 401 路径,
* 故 window.dispatchEvent 不会被调用,无需完整 DOM。
*/
import { test } from 'node:test'
import assert from 'node:assert/strict'
// 最小浏览器环境 polyfill(仅覆盖 http.ts 在 happy path 用到的全局)
const store: Record<string, string> = {}
// @ts-expect-error 测试用最小 window stub
globalThis.window = {
dispatchEvent: () => false,
localStorage: {
getItem: (k: string) => store[k] ?? null,
setItem: (k: string, v: string) => { store[k] = v },
removeItem: (k: string) => { delete store[k] },
},
}
// 捕获每次 fetch 的入参供断言
let lastInit: RequestInit | undefined
globalThis.fetch = async (_url: string, init?: RequestInit) => {
lastInit = init
return new Response(JSON.stringify({ ok: true }), { status: 200, headers: { 'content-type': 'application/json' } })
}
const { http } = await import('./http.ts')
test('GET: method=GET, 无 body, 无 Content-Type', async () => {
await http.get('/api/v1/matches')
assert.equal(lastInit?.method, 'GET')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
test('POST: method=POST, 序列化 body, 有 Content-Type', async () => {
await http.post('/api/v1/matches', { a: 1 })
assert.equal(lastInit?.method, 'POST')
assert.equal(lastInit?.body, JSON.stringify({ a: 1 }))
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], 'application/json')
})
test('POST 空 body: 不设 Content-Type', async () => {
await http.post('/api/v1/matches', undefined)
assert.equal(lastInit?.method, 'POST')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
test('PUT: method=PUT, 有 body 与 Content-Type', async () => {
await http.put('/api/v1/x', { b: 2 })
assert.equal(lastInit?.method, 'PUT')
assert.equal(lastInit?.body, JSON.stringify({ b: 2 }))
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], 'application/json')
})
test('DELETE: method=DELETE, 无 body, 无 Content-Type', async () => {
await http.delete('/api/v1/x/1')
assert.equal(lastInit?.method, 'DELETE')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
+5 -4
View File
@@ -49,10 +49,11 @@ async function request<T>(path: string, options: RequestOptions = {}): Promise<T
} }
try { try {
const res = await fetch(url, { const method = (options.method ?? 'GET').toUpperCase()
signal: controller.signal, const body = options.body
headers: { 'Content-Type': 'application/json' }, // 仅当有 body 时设置 Content-Type,避免 GET/DELETE 等无 body 请求被误标
}) const headers: Record<string, string> = body ? { 'Content-Type': 'application/json' } : {}
const res = await fetch(url, { signal: controller.signal, method, body, headers })
if (!res.ok) { if (!res.ok) {
const rawText = await res.text() const rawText = await res.text()
+27 -28
View File
@@ -8,18 +8,9 @@
import { useEffect, useState, useCallback } from 'react' import { useEffect, useState, useCallback } from 'react'
import { fetchStandings } from '../admin/dal' import { fetchStandings } from '../admin/dal'
import type { StandingsLeague, StandingRow } from '../admin/dal' import type { StandingsLeague, StandingRow } from '../admin/dal'
import { useLeagues } from './matches/hooks/useLeagues'
import { Spinner } from '../admin/components' import { Spinner } from '../admin/components'
const LEAGUES = [
{ code: 'E0', name: '英超' },
{ code: 'SP1', name: '西甲' },
{ code: 'D1', name: '德甲' },
{ code: 'I1', name: '意甲' },
{ code: 'F1', name: '法甲' },
{ code: 'CL', name: '欧冠' },
{ code: 'EL', name: '欧联' },
]
const ZONE_META: Record<string, { label: string; cls: string }> = { const ZONE_META: Record<string, { label: string; cls: string }> = {
// 欧战资格 // 欧战资格
'Champions League': { label: '欧冠区', cls: 'bg-emerald-100 text-emerald-700' }, 'Champions League': { label: '欧冠区', cls: 'bg-emerald-100 text-emerald-700' },
@@ -64,7 +55,9 @@ function FormDots({ form }: { form?: string | null }) {
} }
export default function StandingsPage() { export default function StandingsPage() {
const [leagues, setLeagues] = useState<StandingsLeague[]>([]) // 统一数据源:复用 useLeagues hook(优先 API,失败回退本地常量)
const leagues = useLeagues()
const [standings, setStandings] = useState<StandingsLeague[]>([])
const [activeLeague, setActiveLeague] = useState<string>('') const [activeLeague, setActiveLeague] = useState<string>('')
const [loading, setLoading] = useState(true) const [loading, setLoading] = useState(true)
const [switching, setSwitching] = useState(false) // 切换联赛中 const [switching, setSwitching] = useState(false) // 切换联赛中
@@ -85,7 +78,7 @@ export default function StandingsPage() {
setError(null) setError(null)
try { try {
const data = await fetchStandings(code) const data = await fetchStandings(code)
setLeagues(data.leagues) setStandings(data.leagues)
if (!activeLeague && data.leagues.length > 0) { if (!activeLeague && data.leagues.length > 0) {
setActiveLeague(data.leagues[0].league_code) setActiveLeague(data.leagues[0].league_code)
} }
@@ -104,7 +97,7 @@ export default function StandingsPage() {
setSwitching(true) setSwitching(true)
setActiveLeague(code) setActiveLeague(code)
try { try {
await fetchStandings(code).then(data => setLeagues(data.leagues)) await fetchStandings(code).then(data => setStandings(data.leagues))
} catch (err) { } catch (err) {
setError(err instanceof Error ? err.message : '加载失败') setError(err instanceof Error ? err.message : '加载失败')
} finally { } finally {
@@ -112,26 +105,32 @@ export default function StandingsPage() {
} }
} }
const active = leagues.find(l => l.league_code === activeLeague) ?? leagues[0] const active = standings.find(l => l.league_code === activeLeague) ?? standings[0]
return ( return (
<div className="space-y-6"> <div className="space-y-6">
{/* 联赛切换 */} {/* 联赛切换 */}
<div className="flex flex-wrap gap-2"> <div className="flex flex-wrap gap-2">
{LEAGUES.map(l => ( {leagues.map(l => {
<button // 标记该联赛是否有积分榜数据:有数据可正常切换,无数据也可选中但显示空态
key={l.code} const hasData = standings.some(s => s.league_code === l.code)
onClick={() => switchLeague(l.code)} const isEmpty = activeLeague === l.code && !hasData
disabled={switching} return (
className={`rounded border px-3 py-1.5 text-xs transition-colors disabled:opacity-50 ${ <button
activeLeague === l.code key={l.code}
? 'border-ink-900 bg-ink-900 text-paper-50' onClick={() => switchLeague(l.code)}
: 'border-ink-200 text-ink-500 hover:border-ink-300' disabled={switching}
}`} title={hasData ? undefined : '暂无积分榜数据'}
> className={`rounded border px-3 py-1.5 text-xs transition-colors disabled:opacity-50 ${
{l.name} activeLeague === l.code
</button> ? 'border-ink-900 bg-ink-900 text-paper-50'
))} : 'border-ink-200 text-ink-500 hover:border-ink-300'
} ${!hasData ? 'border-dashed' : ''}`}
>
{l.name}
</button>
)
})}
</div> </div>
{error && ( {error && (
+2
View File
@@ -78,6 +78,8 @@ export const LEAGUES = [
{ code: 'D1', name: '德甲' }, { code: 'D1', name: '德甲' },
{ code: 'I1', name: '意甲' }, { code: 'I1', name: '意甲' },
{ code: 'F1', name: '法甲' }, { code: 'F1', name: '法甲' },
{ code: 'CL', name: '欧冠' },
{ code: 'EL', name: '欧联' },
] ]
/** 汉字编号,给专家意见排版用 */ /** 汉字编号,给专家意见排版用 */
+55
View File
@@ -0,0 +1,55 @@
"""后台管理:采集任务状态查询(只读)。
GET /api/v1/admin/ingest/jobs/{job_id} — 单任务详情
GET /api/v1/admin/ingest/jobs?limit=N — 最近任务列表(默认 20)
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import desc, select
from src.api.deps import require_admin
from src.api.schemas import IngestJobOut
from src.db.base import AsyncSession, get_db_read
from src.db.models import IngestJob
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/ingest/jobs/{job_id}", response_model=IngestJobOut)
async def get_ingest_job(job_id: str, db: AsyncSession = Depends(get_db_read)):
"""查询单个采集任务状态。"""
job = await db.get(IngestJob, job_id)
if job is None:
raise HTTPException(404, f"采集任务不存在: {job_id}")
return _job_to_out(job)
@router.get("/ingest/jobs", response_model=list[IngestJobOut])
async def list_ingest_jobs(
limit: int = Query(20, ge=1, le=100, description="返回条数"),
db: AsyncSession = Depends(get_db_read),
):
"""查询最近采集任务(最新在前)。"""
rows = (
await db.execute(select(IngestJob).order_by(desc(IngestJob.created_at)).limit(limit))
).scalars().all()
return [_job_to_out(j) for j in rows]
def _job_to_out(job: IngestJob) -> IngestJobOut:
return IngestJobOut(
id=job.id,
task=job.task,
params=job.params or {},
status=job.status,
result=job.result,
error=job.error,
created_at=job.created_at,
started_at=job.started_at,
finished_at=job.finished_at,
)
+4
View File
@@ -15,11 +15,15 @@ from fastapi import APIRouter
from src.api.routes.admin_config import router as admin_config_router from src.api.routes.admin_config import router as admin_config_router
from src.api.routes.admin_datasources import router as admin_datasources_router from src.api.routes.admin_datasources import router as admin_datasources_router
from src.api.routes.admin_ingest_jobs import router as admin_ingest_jobs_router
from src.api.routes.admin_llm import router as admin_llm_router from src.api.routes.admin_llm import router as admin_llm_router
from src.api.routes.admin_quality import router as admin_quality_router from src.api.routes.admin_quality import router as admin_quality_router
from src.api.routes.admin_teams import router as admin_teams_router
router = APIRouter() router = APIRouter()
router.include_router(admin_datasources_router) router.include_router(admin_datasources_router)
router.include_router(admin_config_router) router.include_router(admin_config_router)
router.include_router(admin_ingest_jobs_router)
router.include_router(admin_llm_router) router.include_router(admin_llm_router)
router.include_router(admin_quality_router) router.include_router(admin_quality_router)
router.include_router(admin_teams_router)
+60
View File
@@ -0,0 +1,60 @@
"""后台管理:球队别名管理(只读列表 + 添加别名)。
归一名(teams.name)是球队唯一键;别名(team_aliases)是同一球队的不同写法
(大小写/译名/缩写)到归一后 teams.id 的映射。入库时 normalize(name) 依次查
teams.name 与 team_aliases,命中即复用,避免重复 Team。
不自动合并历史重复队;需显式添加别名(或先 SQL/再经由此接口)。
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy import desc, select
from src.api.deps import require_admin
from src.api.schemas import TeamAliasIn, TeamAliasOut
from src.db.base import AsyncSession, get_db_read
from src.db.models import Team, TeamAlias
from src.db.repositories import TeamRepository
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/teams/aliases", response_model=list[TeamAliasOut])
async def list_team_aliases(db: AsyncSession = Depends(get_db_read)):
"""列出所有球队别名(最新在前)。"""
rows = (await db.execute(select(TeamAlias).order_by(desc(TeamAlias.created_at)).limit(200))).scalars().all()
return [
TeamAliasOut(
alias_normalized=r.alias_normalized,
team_id=r.team_id,
original_alias=r.original_alias,
)
for r in rows
]
@router.post("/teams/aliases", response_model=TeamAliasOut, status_code=201)
async def add_team_alias(req: TeamAliasIn, db: AsyncSession = Depends(get_db_read)):
"""为已有 Team 添加别名(幂等:重复添加会更新指向)。
不自动合并历史重复队。若需合并 A→B:先为 A 的归一名添加别名指向 B,
再人工确认 A 是否仍有独立引用。
"""
# 校验目标 Team 存在
team = await db.get(Team, req.team_id)
if team is None:
raise HTTPException(404, f"目标 Team 不存在: id={req.team_id}")
repo = TeamRepository(db)
row = await repo.add_alias(req.alias, req.team_id)
logger.info("添加 Team 别名: %s -> team_id=%s", req.alias, req.team_id)
return TeamAliasOut(
alias_normalized=row.alias_normalized,
team_id=row.team_id,
original_alias=row.original_alias,
)
+87 -17
View File
@@ -10,11 +10,13 @@ from __future__ import annotations
import asyncio import asyncio
import logging import logging
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, HTTPException from fastapi import APIRouter, Depends, HTTPException
from src.api.deps import require_admin from src.api.deps import require_admin
from src.api.schemas import IngestBzzoiroRequest from src.api.schemas import IngestBzzoiroRequest, IngestBzzoiroResponse
from src.data.config import BZZOIRO_LEAGUE_IDS from src.data.config import BZZOIRO_LEAGUE_IDS
from src.data.bzzoiro_standings import ingest_bzzoiro_standings from src.data.bzzoiro_standings import ingest_bzzoiro_standings
from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats
@@ -31,6 +33,19 @@ _background_tasks: set[asyncio.Task] = set()
VALID_TASKS = {"events", "standings", "stats", "all"} VALID_TASKS = {"events", "standings", "stats", "all"}
def _accumulate_ingest_result(merged: dict, code: str, r: dict) -> None:
"""P1-A: 累加单联赛采集结果。联赛级计数读 r["leagues"][code],顶层读 total_*。"""
merged["total_inserted"] += r.get("total_inserted", 0)
merged["total_updated"] += r.get("total_updated", 0)
merged["errors"].extend(r.get("errors", []))
# 联赛级计数必须来自 leagues[code],而非顶层 r.get("inserted")
league_r = r.get("leagues", {}).get(code, {})
acc = merged["leagues"].setdefault(code, {"inserted": 0, "updated": 0, "errors": []})
acc["inserted"] += league_r.get("inserted", 0)
acc["updated"] += league_r.get("updated", 0)
acc["errors"].extend(r.get("errors", []))
def _spawn(coro) -> None: def _spawn(coro) -> None:
"""启动后台采集任务;异常已在任务内记录到系统日志。""" """启动后台采集任务;异常已在任务内记录到系统日志。"""
task = asyncio.create_task(coro) task = asyncio.create_task(coro)
@@ -38,22 +53,58 @@ def _spawn(coro) -> None:
task.add_done_callback(_background_tasks.discard) task.add_done_callback(_background_tasks.discard)
@router.post("/ingest/bzzoiro", dependencies=[Depends(require_admin)]) @router.post("/ingest/bzzoiro", response_model=IngestBzzoiroResponse, dependencies=[Depends(require_admin)])
async def ingest_bzzoiro_route(req: IngestBzzoiroRequest): async def ingest_bzzoiro_route(req: IngestBzzoiroRequest):
"""触发 bzzoiro 采集(events / standings / stats / all)。""" """触发 bzzoiro 采集(events / standings / stats / all)。
启动后台任务前写入 ingest_jobs(pending),响应返回 job_id 供前端轮询。
兼容原 message 字段(仍返回)。
"""
if req.task not in VALID_TASKS: if req.task not in VALID_TASKS:
raise HTTPException(status_code=422, detail=f"未知任务类型: {req.task}(可选: {', '.join(sorted(VALID_TASKS))})") raise HTTPException(status_code=422, detail=f"未知任务类型: {req.task}(可选: {', '.join(sorted(VALID_TASKS))})")
leagues = req.leagues or list(BZZOIRO_LEAGUE_IDS.keys()) leagues = req.leagues or list(BZZOIRO_LEAGUE_IDS.keys())
task_label = {"events": "比赛数据", "standings": "积分榜", "stats": "统计回填", "all": "全量(比赛+积分榜+统计)"}[req.task] task_label = {"events": "比赛数据", "standings": "积分榜", "stats": "统计回填", "all": "全量(比赛+积分榜+统计)"}[req.task]
_spawn(_run_bzzoiro(req.task, leagues, req))
return { job_id = await _create_ingest_job(req.task, leagues, req)
"ok": True, _spawn(_run_bzzoiro(job_id, req.task, leagues, req))
"message": f"采集任务已启动(后台执行,任务: {task_label}),请在「系统日志」查看进度与结果",
return IngestBzzoiroResponse(
ok=True,
job_id=job_id,
message=f"采集任务已启动(后台执行,任务: {task_label}),请到「数据采集」页跟踪进度",
)
async def _create_ingest_job(task: str, leagues: list[str], req: IngestBzzoiroRequest) -> str:
"""写入一条 ingest_jobs(pending),返回 job_id。"""
from src.db.models import IngestJob
job_id = str(uuid.uuid4())
params = {
"leagues": leagues,
"date_from": req.date_from,
"date_to": req.date_to,
"status": req.status,
"task": task,
"limit": req.limit,
"season": req.season,
} }
async with get_uow() as session:
job = IngestJob(id=job_id, task=task, params=params, status="pending")
session.add(job)
logger.info("ingest_jobs 创建: job=%s task=%s leagues=%s", job_id, task, leagues)
return job_id
async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest) -> None: async def _run_bzzoiro(job_id: str, task: str, leagues: list[str], req: IngestBzzoiroRequest) -> None:
"""后台执行 bzzoiro 采集:上游限速时单次可能耗时数分钟,必须脱离请求生命周期。""" """后台执行 bzzoiro 采集:上游限速时单次可能耗时数分钟,必须脱离请求生命周期。
状态流转: pending → running → (success|failed)。
"""
from src.db.models import IngestJob
await _update_job(job_id, status="running", started_at=datetime.now(timezone.utc))
result: dict = {}
try: try:
if task in ("events", "all"): if task in ("events", "all"):
statuses = [req.status] if req.status else ["finished", "scheduled"] statuses = [req.status] if req.status else ["finished", "scheduled"]
@@ -67,19 +118,14 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
session, leagues=[code], session, leagues=[code],
date_from=req.date_from, date_to=req.date_to, status=st, date_from=req.date_from, date_to=req.date_to, status=st,
) )
merged["total_inserted"] += r.get("total_inserted", 0) _accumulate_ingest_result(merged, code, r)
merged["total_updated"] += r.get("total_updated", 0)
merged["errors"].extend(r.get("errors", []))
acc = merged["leagues"].setdefault(code, {"inserted": 0, "updated": 0, "errors": []})
acc["inserted"] += r.get("inserted", 0)
acc["updated"] += r.get("updated", 0)
acc["errors"].extend(r.get("errors", []))
logger.info( logger.info(
"bzzoiro 比赛采集完成: 新增 %d, 更新 %d, 联赛 %d 个, 状态 %s", "bzzoiro 比赛采集完成: 新增 %d, 更新 %d, 联赛 %d 个, 状态 %s",
merged["total_inserted"], merged["total_updated"], len(merged["leagues"]), statuses, merged["total_inserted"], merged["total_updated"], len(merged["leagues"]), statuses,
) )
if merged["errors"]: if merged["errors"]:
logger.warning("bzzoiro 比赛采集错误 %d 条: %s", len(merged["errors"]), merged["errors"][:3]) logger.warning("bzzoiro 比赛采集错误 %d 条: %s", len(merged["errors"]), merged["errors"][:3])
result["events"] = merged
if task in ("standings", "all"): if task in ("standings", "all"):
async with get_uow() as session: async with get_uow() as session:
@@ -88,6 +134,7 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
logger.warning("bzzoiro 积分榜采集部分失败: %s", r["errors"][:3]) logger.warning("bzzoiro 积分榜采集部分失败: %s", r["errors"][:3])
else: else:
logger.info("bzzoiro 积分榜采集完成: upsert %d", r["total_upserted"]) logger.info("bzzoiro 积分榜采集完成: upsert %d", r["total_upserted"])
result["standings"] = r
if task in ("stats", "all"): if task in ("stats", "all"):
async with get_uow() as session: async with get_uow() as session:
@@ -96,5 +143,28 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
) )
if r["errors"]: if r["errors"]:
logger.warning("bzzoiro 统计回填错误 %d 条: %s", len(r["errors"]), r["errors"][:3]) logger.warning("bzzoiro 统计回填错误 %d 条: %s", len(r["errors"]), r["errors"][:3])
except Exception: result["stats"] = r
await _update_job(job_id, status="success", result=result, finished_at=datetime.now(timezone.utc))
logger.info("ingest_jobs 完成: job=%s task=%s", job_id, task)
except Exception as e:
logger.exception("bzzoiro 采集任务失败(task=%s)", task) logger.exception("bzzoiro 采集任务失败(task=%s)", task)
await _update_job(
job_id, status="failed", error=str(e), finished_at=datetime.now(timezone.utc),
)
async def _update_job(job_id: str, **fields) -> None:
"""更新 ingest_jobs 单行;失败仅记日志,绝不抛异常(避免干扰采集主流程)。"""
from src.db.models import IngestJob
try:
async with get_uow() as session:
job = await session.get(IngestJob, job_id)
if job is None:
logger.warning("ingest_jobs 更新失败: job=%s 不存在", job_id)
return
for k, v in fields.items():
setattr(job, k, v)
except Exception:
logger.warning("ingest_jobs 更新异常: job=%s fields=%s", job_id, list(fields.keys()))
+79 -40
View File
@@ -5,15 +5,29 @@ from datetime import datetime, timedelta
from fastapi import APIRouter, Depends, HTTPException, Query from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import func, or_, select from sqlalchemy import func, or_, select
from sqlalchemy.orm import selectinload from sqlalchemy.orm import load_only, selectinload
from src.api.schemas import MatchListOut, MatchOut, PredictionOut from src.api.schemas import MatchListOut, MatchOut, PredictionOut
from src.db.base import AsyncSession, get_db_read from src.db.base import AsyncSession, get_db_read
from src.db.models import League, Match, Prediction, Standing from src.db.models import League, Match, MatchStats, Prediction, Standing, Team
router = APIRouter(prefix="/api/v1", tags=["data"]) router = APIRouter(prefix="/api/v1", tags=["data"])
def _parse_cursor(cursor: str) -> tuple[datetime, int]:
"""P1-B: 解析游标。非法格式 → HTTPException(400, code=INVALID_CURSOR)。"""
try:
last_date_str, last_id_str = cursor.split("|", 1)
last_date = datetime.fromisoformat(last_date_str)
last_id = int(last_id_str)
return last_date, last_id
except (ValueError, AttributeError) as e:
raise HTTPException(
status_code=400,
detail={"code": "INVALID_CURSOR", "message": f"非法游标格式: {cursor}(应为 date_iso|id)"},
) from e
def _stats_dict(stats) -> dict | None: def _stats_dict(stats) -> dict | None:
"""把 MatchStats ORM 对象序列化为前端可读的扁平 dict。""" """把 MatchStats ORM 对象序列化为前端可读的扁平 dict。"""
if stats is None: if stats is None:
@@ -49,30 +63,37 @@ async def list_matches(
limit: int = Query(50, ge=1, le=100), limit: int = Query(50, ge=1, le=100),
db: AsyncSession = Depends(get_db_read), db: AsyncSession = Depends(get_db_read),
): ):
"""比赛列表(游标分页)。""" """比赛列表(游标分页)。
q = select(Match).options(selectinload(Match.league), selectinload(Match.home_team), selectinload(Match.away_team))
加载策略(列表 vs 详情):
- 列表:仅 selectinload 序列化需要的 3 个关系 + stats,且用 load_only 限定列
(League.code / Team.name,name_zh / MatchStats.home_xg,away_xg),避免传输全列;
同时一次性加载 stats 消除 N+1(m.stats.home_xg 此前触发懒加载)。
- 详情(/matches/{id}):保持完整 options(league/teams/stats 全列 + 最近预测)。
"""
q = select(Match).options(
selectinload(Match.league).load_only(League.code),
selectinload(Match.home_team).load_only(Team.name, Team.name_zh),
selectinload(Match.away_team).load_only(Team.name, Team.name_zh),
selectinload(Match.stats).load_only(MatchStats.home_xg, MatchStats.away_xg),
)
if cursor: if cursor:
try: # P1-B: 解析非法 → 400 + code=INVALID_CURSOR,而非静默忽略
# 用 | 分隔,避免 isoformat 含 _ 时解析失败 last_date, last_id = _parse_cursor(cursor)
last_date_str, last_id_str = cursor.split("|", 1) # 游标方向必须与排序方向一致:
last_date = datetime.fromisoformat(last_date_str) # - scheduled(ASC):取「更大」的未开赛场次
last_id = int(last_id_str) # - 其它(DESC):取「更小」的已赛场次
# 游标方向必须与排序方向一致: if status == "scheduled":
# - scheduled(ASC):取「更大」的未开赛场次 q = q.where(
# - 其它(DESC):取「更小」的已赛场次 (Match.match_date > last_date) |
if status == "scheduled": ((Match.match_date == last_date) & (Match.id > last_id))
q = q.where( )
(Match.match_date > last_date) | else:
((Match.match_date == last_date) & (Match.id > last_id)) q = q.where(
) (Match.match_date < last_date) |
else: ((Match.match_date == last_date) & (Match.id < last_id))
q = q.where( )
(Match.match_date < last_date) |
((Match.match_date == last_date) & (Match.id < last_id))
)
except (ValueError, AttributeError):
pass
if league: if league:
stmt = select(League.id).where(League.code == league) stmt = select(League.id).where(League.code == league)
@@ -148,15 +169,28 @@ async def get_match(match_id: int, db: AsyncSession = Depends(get_db_read)):
m = (await db.execute(stmt)).scalar_one_or_none() m = (await db.execute(stmt)).scalar_one_or_none()
if m is None: if m is None:
raise HTTPException(404, "match not found") raise HTTPException(404, "match not found")
# 最近预测(倒序,最多 5 条)——复用 PredictionOut 结构,只读,不触发 LLM # P1-C: 公开预测仅 run_type=live 且 status=success(屏蔽回测/失败预测)
preds = ( preds = (
await db.execute( await db.execute(
select(Prediction) select(Prediction)
.where(Prediction.match_id == match_id) .where(Prediction.match_id == match_id)
.where(Prediction.run_type == "live")
.where(Prediction.status == "success")
.order_by(Prediction.created_at.desc()) .order_by(Prediction.created_at.desc())
.limit(5) .limit(5)
) )
).scalars().all() ).scalars().all()
# P1-C: 公开接口的预测不含 reasoning/agent_outputs(避免泄露内部推理细节)
recent_predictions = [
{
"id": p.id, "match_id": p.match_id, "provider": p.provider, "model": p.model,
"prompt_version": p.prompt_version, "mode": p.mode or "single",
"pred_home_goals": p.pred_home_goals, "pred_away_goals": p.pred_away_goals,
"pred_1x2": p.pred_1x2, "subjective_confidence": p.subjective_confidence,
"created_at": p.created_at.isoformat() if p.created_at else None,
}
for p in preds
]
return MatchOut( return MatchOut(
id=m.id, id=m.id,
league_code=m.league.code if m.league else None, league_code=m.league.code if m.league else None,
@@ -173,20 +207,7 @@ async def get_match(match_id: int, db: AsyncSession = Depends(get_db_read)):
home_xg=m.stats.home_xg if m.stats else None, home_xg=m.stats.home_xg if m.stats else None,
away_xg=m.stats.away_xg if m.stats else None, away_xg=m.stats.away_xg if m.stats else None,
stats=_stats_dict(m.stats) if m.stats else None, stats=_stats_dict(m.stats) if m.stats else None,
recent_predictions=[ recent_predictions=recent_predictions,
PredictionOut(
id=p.id, match_id=p.match_id, provider=p.provider, model=p.model,
prompt_version=p.prompt_version, mode=p.mode or "single",
pred_home_goals=p.pred_home_goals, pred_away_goals=p.pred_away_goals,
alt_pred_home_goals=p.alt_pred_home_goals, alt_pred_away_goals=p.alt_pred_away_goals,
pred_1x2=p.pred_1x2, subjective_confidence=p.subjective_confidence,
reasoning=p.reasoning, status=p.status or "success",
agent_outputs=p.agent_outputs, agent_weights=p.agent_weights,
created_at=p.created_at, actual_home_goals=p.actual_home_goals,
actual_away_goals=p.actual_away_goals, settled=p.settled,
)
for p in preds
],
) )
@@ -274,7 +295,9 @@ async def list_standings(
): ):
"""联赛积分榜(只读)。按联赛分组,每张榜按 position 排序。 """联赛积分榜(只读)。按联赛分组,每张榜按 position 排序。
season 为空时返回每个联赛最新采集到的赛季榜单(适合前端"查看最新积分榜")。 P0-02: standings 为追加快照,公开接口取每队 available_at 最新快照
(league_id, season, team_id 上按 available_at 取最新)。
season 为空时返回每个联赛最新采集到的赛季榜单。
""" """
# 取每个联赛最新赛季(当 season 为空时) # 取每个联赛最新赛季(当 season 为空时)
latest_seasons: dict[int, str] = {} latest_seasons: dict[int, str] = {}
@@ -287,9 +310,25 @@ async def list_standings(
).all() ).all()
latest_seasons = {r.league_id: r.latest for r in rows} latest_seasons = {r.league_id: r.latest for r in rows}
# P0-02: 子查询取每队最新 available_at 快照,再 JOIN 回主表拿完整行 + League
latest_per_team = (
select(
Standing.league_id, Standing.season, Standing.team_id,
func.max(Standing.available_at).label("max_available"),
)
.group_by(Standing.league_id, Standing.season, Standing.team_id)
.subquery("latest_per_team")
)
q = ( q = (
select(Standing, League) select(Standing, League)
.join(League, League.id == Standing.league_id) .join(League, League.id == Standing.league_id)
.join(
latest_per_team,
(Standing.league_id == latest_per_team.c.league_id)
& (Standing.season == latest_per_team.c.season)
& (Standing.team_id == latest_per_team.c.team_id)
& (Standing.available_at == latest_per_team.c.max_available),
)
.order_by(League.name.asc(), Standing.position.asc()) .order_by(League.name.asc(), Standing.position.asc())
) )
if league: if league:
+19 -7
View File
@@ -2,10 +2,12 @@
安全改进: 安全改进:
- 限流: 每分钟 10 次 / IP(内存实现) - 限流: 每分钟 10 次 / IP(内存实现)
- P1-D: 全局 LLM 并发限制(默认 4),防止过多并发 LLM 调用压垮服务
- DB 连接: 短 session 模式,LLM 调用期间不持有连接 - DB 连接: 短 session 模式,LLM 调用期间不持有连接
""" """
from __future__ import annotations from __future__ import annotations
import asyncio
import logging import logging
from fastapi import APIRouter, Depends, HTTPException, Query, Request from fastapi import APIRouter, Depends, HTTPException, Query, Request
@@ -22,6 +24,21 @@ logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1", tags=["predict"]) router = APIRouter(prefix="/api/v1", tags=["predict"])
# P1-D: 全局 LLM 并发限制。与 orchestrator 内的 match 级 Semaphore(8) 并存,
# 此处在路由层限制单实例全 LLM 调用(所有模式汇总),默认 4。
_GLOBAL_LLM_SEMAPHORE = asyncio.Semaphore(4)
async def _predict_with_concurrency(req: PredictRequest) -> PredictResult:
"""P1-D: 在全局 LLM 并发限制下执行预测。"""
async with _GLOBAL_LLM_SEMAPHORE:
return await predict_match(
req.match_id,
model=req.model,
prompt_version=req.prompt_version,
mode=req.mode,
)
@router.post("/predict", response_model=PredictOut, dependencies=[Depends(rate_limit_predict)]) @router.post("/predict", response_model=PredictOut, dependencies=[Depends(rate_limit_predict)])
async def predict(req: PredictRequest, request: Request): async def predict(req: PredictRequest, request: Request):
@@ -42,14 +59,9 @@ async def predict(req: PredictRequest, request: Request):
if m.match_status == "finished": if m.match_status == "finished":
raise HTTPException(400, "该比赛已完赛,不再支持预测") raise HTTPException(400, "该比赛已完赛,不再支持预测")
# 2. 预测调用(不持有任何 DB 连接) # 2. 预测调用(不持有任何 DB 连接,受全局 LLM 并发限制)
try: try:
result = await predict_match( result = await _predict_with_concurrency(req)
req.match_id,
model=req.model,
prompt_version=req.prompt_version,
mode=req.mode,
)
except ValueError as e: except ValueError as e:
msg = str(e) msg = str(e)
if "已结算" in msg: if "已结算" in msg:
+36 -15
View File
@@ -7,13 +7,6 @@ from typing import Any
from pydantic import BaseModel, Field from pydantic import BaseModel, Field
class LeagueOut(BaseModel):
id: int
code: str
name: str
country: str | None
class MatchOut(BaseModel): class MatchOut(BaseModel):
id: int id: int
league_code: str | None league_code: str | None
@@ -31,8 +24,8 @@ class MatchOut(BaseModel):
away_xg: float | None = None away_xg: float | None = None
# 比赛详细统计(bzzoiro /events/{id}/stats/),无统计为 None # 比赛详细统计(bzzoiro /events/{id}/stats/),无统计为 None
stats: dict | None = None stats: dict | None = None
# 该场比赛的最近预测摘要(按时间倒序,最多 5 条;无预测为空) # P1-C: 公开接口的预测不含 reasoning/agent_outputs;仅 live+success 路由已过滤
recent_predictions: list[PredictionOut] = [] recent_predictions: list[dict] = []
class MatchListOut(BaseModel): class MatchListOut(BaseModel):
@@ -43,7 +36,7 @@ class MatchListOut(BaseModel):
class PredictRequest(BaseModel): class PredictRequest(BaseModel):
match_id: int match_id: int
provider: str | None = None # P1-D: 删除未接线的 provider 字段(符合"名不副实则删除");provider 由服务端配置决定。
model: str | None = None model: str | None = None
prompt_version: str | None = None prompt_version: str | None = None
mode: str = Field( mode: str = Field(
@@ -117,11 +110,39 @@ class IngestBzzoiroRequest(BaseModel):
season: str | None = Field(None, description="standings 赛季,如 '2026-2027';空 = 当前赛季") season: str | None = Field(None, description="standings 赛季,如 '2026-2027';空 = 当前赛季")
class IngestResponse(BaseModel): class TeamAliasIn(BaseModel):
leagues: dict """POST /api/v1/admin/teams/aliases 请求体:为已有 Team 添加别名。"""
total_inserted: int
total_updated: int alias: str = Field(..., min_length=1, max_length=120, description="球队别名(原始写法)")
errors: list[str] = [] team_id: int = Field(..., gt=0, description="归一后的目标 teams.id")
class TeamAliasOut(BaseModel):
alias_normalized: str
team_id: int
original_alias: str
class IngestBzzoiroResponse(BaseModel):
"""POST /api/v1/ingest/bzzoiro 响应:兼容原 message 字段,新增 job_id 供轮询。"""
ok: bool = True
job_id: str = Field(..., description="采集任务 ID(GET /api/v1/admin/ingest/jobs/{job_id} 轮询)")
message: str = ""
class IngestJobOut(BaseModel):
"""采集任务状态详情。"""
id: str
task: str
params: dict
status: str # pending | running | success | failed
result: dict | None = None
error: str | None = None
created_at: datetime | None = None
started_at: datetime | None = None
finished_at: datetime | None = None
class ScheduleIn(BaseModel): class ScheduleIn(BaseModel):
+6
View File
@@ -32,6 +32,12 @@ class Settings(BaseSettings):
LLM_SPECIALIST_MODEL: str = "" LLM_SPECIALIST_MODEL: str = ""
LLM_AGGREGATOR_MODEL: str = "" LLM_AGGREGATOR_MODEL: str = ""
# ── 预测缓存 ──
# 预测响应缓存后端:空(默认)=进程内 LRU+TTL 字典;填 redis://host:port/db 启用 Redis。
# Redis 失败自动降级内存缓存并 warning,不中断预测;不强制依赖 redis 包。
# TTL 固定 300s(5 分钟),键格式与内存后端一致(含 prompt 模板 hash)。
PREDICT_CACHE_URL: str = ""
# --- data sources --- # --- data sources ---
BZZOIRO_KEY: str = "" BZZOIRO_KEY: str = ""
BZZOIRO_BASE: str = "https://sports.bzzoiro.com/api/v2" BZZOIRO_BASE: str = "https://sports.bzzoiro.com/api/v2"
-4
View File
@@ -30,10 +30,6 @@ _MIN_SECRET_KEY_LEN = 16
_WEAK_DB_PATTERNS = ("football:football@", "admin:admin@", "password@", "123456@") _WEAK_DB_PATTERNS = ("football:football@", "admin:admin@", "password@", "123456@")
class SecurityCheckError(Exception):
"""生产环境安全校验失败。"""
async def _auth_configured() -> bool: async def _auth_configured() -> bool:
"""运行时鉴权是否已配置(含数据库密码哈希/.env 明文/API Key)。""" """运行时鉴权是否已配置(含数据库密码哈希/.env 明文/API Key)。"""
if await get_admin_password_hash(): if await get_admin_password_hash():
+8 -13
View File
@@ -1,10 +1,11 @@
"""Bzzoiro 数据源:抓取 + 入库(单一数据源)—— 聚合门面。 """Bzzoiro 数据源:抓取 + 入库(单一数据源)—— 聚合门面。
实现按管线拆分(单文件 → 多模块),本模块只做再导出,保持两个不变量: 实现按管线拆分(单文件 → 多模块),本模块只做再导出,保持不变量:
1. sources._load_sources() 仍从本模块导入 BzzoiroSource(注册表入口不变); 1. sources._load_sources() 仍从本模块导入 BzzoiroSource(注册表入口不变);
2. 测试与脚本对 `bz.<名称>` 的 monkeypatch 语义不变 —— 子模块在运行期 2. 测试与脚本对 `bz.<名称>` 的 monkeypatch 语义不变 —— 抓取函数 / REQUEST_INTERVAL
经本门面解析可替换协作者(抓取函数 / Bronze 写入助手 / REQUEST_INTERVAL), 经本门面解析可替换;Bronze 写入助手(_write_raw_event/_write_lineage/
与拆分前的单文件行为一致。 _safe_write_ingest_failure)已改为管线模块直接 import pipeline_write,
测试需 patch `src.data.pipeline_write.*` 源模块。
三条管线(各自模块): 三条管线(各自模块):
1. events — 比赛日程/比分(/events/),含 source_event_id 血缘 → bzzoiro_events.py 1. events — 比赛日程/比分(/events/),含 source_event_id 血缘 → bzzoiro_events.py
@@ -12,7 +13,8 @@
3. stats — 已完赛比赛详细统计回填(/events/{id}/stats/) → bzzoiro_stats.py 3. stats — 已完赛比赛详细统计回填(/events/{id}/stats/) → bzzoiro_stats.py
共享基础:HTTP 抓取(多 key 轮换)与字段转换 → bzzoiro_common.py; 共享基础:HTTP 抓取(多 key 轮换)与字段转换 → bzzoiro_common.py;
Bronze 基础设施(RawEvent/IngestFailure/DataLineage)→ pipeline_write.py Bronze 基础设施(RawEvent/IngestFailure/DataLineage)→ pipeline_write.py
(各管线模块直接 import pipeline_write,不再经本门面转发)。
D4(工程债): Team/League/Match 的查找/创建经 Repository 层(src/db/repositories.py), D4(工程债): Team/League/Match 的查找/创建经 Repository 层(src/db/repositories.py),
各管线不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制,只 flush)。 各管线不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制,只 flush)。
@@ -31,6 +33,7 @@ from src.data.key_ring import _mask # noqa: F401 (R1 测试引用 bz._mask)
from src.data.normalize import normalize_bzzoiro # noqa: F401 from src.data.normalize import normalize_bzzoiro # noqa: F401
# ── 共享原语:HTTP 抓取 + 宽松字段转换 ── # ── 共享原语:HTTP 抓取 + 宽松字段转换 ──
# NOTE: 管线模块同时从 bzzoiro_common 直接 import _fetch_json_async 等(经本处也转发)。
from src.data.bzzoiro_common import ( # noqa: F401 from src.data.bzzoiro_common import ( # noqa: F401
_fetch_json_async, _fetch_json_async,
_match_key, _match_key,
@@ -39,14 +42,6 @@ from src.data.bzzoiro_common import ( # noqa: F401
_to_int_or_none, _to_int_or_none,
) )
# ── 管线基础设施:RawEvent / IngestFailure / DataLineage ──
from src.data.pipeline_write import ( # noqa: F401
_safe_write_ingest_failure,
_write_ingest_failure,
_write_lineage,
_write_raw_event,
)
# ── events 管线:BzzoiroSource(注册表入口)+ 抓取/入库 ── # ── events 管线:BzzoiroSource(注册表入口)+ 抓取/入库 ──
from src.data.bzzoiro_events import ( # noqa: F401 from src.data.bzzoiro_events import ( # noqa: F401
BzzoiroSource, BzzoiroSource,
+25 -8
View File
@@ -16,6 +16,7 @@ from datetime import datetime, timedelta, timezone
from src.data.bzzoiro_common import _match_key, _to_date, _to_int_or_none from src.data.bzzoiro_common import _match_key, _to_date, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
from src.data.normalize import normalize_bzzoiro from src.data.normalize import normalize_bzzoiro
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.data.sources import register from src.data.sources import register
from src.data.team_names_zh import zh_name from src.data.team_names_zh import zh_name
from src.db.models import Match from src.db.models import Match
@@ -98,7 +99,7 @@ class BzzoiroSource:
# 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批 # 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批
logger.exception("bzzoiro fetch failed for %s", code) logger.exception("bzzoiro fetch failed for %s", code)
league_r["errors"].append(f"fetch failed: {e}") league_r["errors"].append(f"fetch failed: {e}")
await bz._safe_write_ingest_failure( await _safe_write_ingest_failure(
db, db,
entity_type="events", entity_type="events",
source_record_id=None, source_record_id=None,
@@ -181,9 +182,16 @@ class BzzoiroSource:
away_team_id = away.id away_team_id = away.id
team_name_to_id[nm.away_team] = away_team_id team_name_to_id[nm.away_team] = away_team_id
# 查找已有比赛: 内存查找 # 查找已有比赛:优先按 upstream event_id 定位(命中即唯一),
match_key = _match_key(home_team_id, away_team_id, nm.date) # 否则回退自然键(联赛+主客+天级日期)内存查找。
existing_match = existing_matches.get(match_key) # source_event_id 上有 partial unique 索引保障 upstream 唯一。
eid = _to_int_or_none(raw.get("id"))
existing_match = None
if eid is not None:
existing_match = await match_r.find_by_source_event_id(eid)
if existing_match is None:
match_key = _match_key(home_team_id, away_team_id, nm.date)
existing_match = existing_matches.get(match_key)
if existing_match is None: if existing_match is None:
m = Match( m = Match(
@@ -194,6 +202,7 @@ class BzzoiroSource:
match_date=nm.date, match_date=nm.date,
match_date_date=_to_date(nm.date), match_date_date=_to_date(nm.date),
match_status=nm.match_status, match_status=nm.match_status,
score_status=nm.score_status,
home_goals=nm.home_goals, home_goals=nm.home_goals,
away_goals=nm.away_goals, away_goals=nm.away_goals,
home_ht_goals=nm.home_ht_goals, home_ht_goals=nm.home_ht_goals,
@@ -230,6 +239,16 @@ class BzzoiroSource:
existing_match.away_goals = nm.away_goals existing_match.away_goals = nm.away_goals
existing_match.home_ht_goals = nm.home_ht_goals existing_match.home_ht_goals = nm.home_ht_goals
existing_match.away_ht_goals = nm.away_ht_goals existing_match.away_ht_goals = nm.away_ht_goals
# 比分由缺变有 → 标记 known
existing_match.score_status = "known"
changed = True
elif (
nm.match_status == "finished"
and nm.home_goals is None
and existing_match.score_status == "unknown"
):
# 确认完赛仍缺分 → 标记 missing(不伪造 0:0)
existing_match.score_status = "missing"
changed = True changed = True
if existing_match.match_stage is None and nm.match_stage: if existing_match.match_stage is None and nm.match_stage:
existing_match.match_stage = nm.match_stage existing_match.match_stage = nm.match_stage
@@ -293,11 +312,9 @@ async def _write_events_bronze(
source_record_id 查重保证;best-effort:基础设施写入失败只记 warning, source_record_id 查重保证;best-effort:基础设施写入失败只记 warning,
绝不拖垮采集主流程(与 _safe_write_ingest_failure 同级约束)。 绝不拖垮采集主流程(与 _safe_write_ingest_failure 同级约束)。
""" """
from src.data import bzzoiro as bz
try: try:
await bz._write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id) await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
await bz._write_lineage( await _write_lineage(
db, "bzzoiro", source_record_id, db, "bzzoiro", source_record_id,
"matches", target_match_id, "events_ingest", "matches", target_match_id, "events_ingest",
{"league": league_code, "match_status": match_status}, {"league": league_code, "match_status": match_status},
+12 -20
View File
@@ -16,6 +16,7 @@ from sqlalchemy import select
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.data.team_names_zh import zh_name from src.data.team_names_zh import zh_name
from src.db.models import Standing, Team from src.db.models import Standing, Team
from src.db.repositories import LeagueRepository, TeamRepository from src.db.repositories import LeagueRepository, TeamRepository
@@ -33,6 +34,7 @@ async def fetch_bzzoiro_standings(league_code: str, season: str | None = None) -
params: dict = {} params: dict = {}
if season: if season:
params["season"] = season params["season"] = season
return await bz._fetch_json_async(f"/leagues/{league_id}/standings/", params) return await bz._fetch_json_async(f"/leagues/{league_id}/standings/", params)
@@ -57,9 +59,10 @@ async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str |
season 为 None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)。 season 为 None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)。
球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配。 球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配。
""" """
from src.data import bzzoiro as bz
from src.data.team_names import normalize as normalize_name from src.data.team_names import normalize as normalize_name
from src.data import bzzoiro as bz
result: dict = {"leagues": {}, "total_upserted": 0, "errors": []} result: dict = {"leagues": {}, "total_upserted": 0, "errors": []}
for code in leagues: for code in leagues:
league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []} league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []}
@@ -68,7 +71,7 @@ async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str |
except Exception as e: except Exception as e:
logger.exception("bzzoiro standings fetch failed for %s", code) logger.exception("bzzoiro standings fetch failed for %s", code)
league_r["errors"].append(str(e)) league_r["errors"].append(str(e))
await bz._safe_write_ingest_failure( await _safe_write_ingest_failure(
db, db,
entity_type="standings", entity_type="standings",
source_record_id=None, source_record_id=None,
@@ -134,21 +137,12 @@ async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str |
retrieved_at=now, retrieved_at=now,
) )
# 同一联赛同一赛季只保留最新快照:按 (league, season, team) upsert # P0-02: 追加快照——每次采集 INSERT 新行(available_at=now),
stmt = select(Standing).where( # ON CONFLICT (league, season, team, available_at) DO NOTHING。
Standing.league_id == league.id, standing = Standing(
Standing.season == season_label, league_id=league.id, season=season_label, team_id=team.id, available_at=now, **values
Standing.team_id == team.id,
) )
standing = (await db.execute(stmt)).scalar_one_or_none() db.add(standing)
if standing is None:
standing = Standing(
league_id=league.id, season=season_label, team_id=team.id, **values
)
db.add(standing)
else:
for k, v in values.items():
setattr(standing, k, v)
league_r["upserted"] += 1 league_r["upserted"] += 1
league_r["rows"] = len(rows) league_r["rows"] = len(rows)
@@ -198,11 +192,9 @@ async def _write_standings_bronze(
命中同一条 RawEvent);best-effort:基础设施写入失败只记 warning, 命中同一条 RawEvent);best-effort:基础设施写入失败只记 warning,
绝不拖垮采集主流程。 绝不拖垮采集主流程。
""" """
from src.data import bzzoiro as bz
try: try:
await bz._write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id) await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
await bz._write_lineage( await _write_lineage(
db, "bzzoiro", source_record_id, db, "bzzoiro", source_record_id,
"standings", league_id, "standings_ingest", "standings", league_id, "standings_ingest",
{"league": league_code, "season": season_label, "rows_upserted": rows_upserted}, {"league": league_code, "season": season_label, "rows_upserted": rows_upserted},
+5 -4
View File
@@ -15,6 +15,7 @@ from datetime import datetime, timedelta, timezone
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS from src.data.config import BZZOIRO_LEAGUE_IDS
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.db.models import MatchStats from src.db.models import MatchStats
from src.db.repositories import MatchRepository from src.db.repositories import MatchRepository
@@ -90,7 +91,7 @@ async def ingest_bzzoiro_event_stats(
筛选条件: match_status=finished 且 source_event_id 非空。 筛选条件: match_status=finished 且 source_event_id 非空。
only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新。 only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新。
limit 控制单次最多处理的比赛数(上游限速 1.2s/请求,大批量需分次触发)。 limit 控制单次最多处理的比赛数(上游限速 1.2s/请求,大批量需分次触发)。
""" """
from src.data import bzzoiro as bz from src.data import bzzoiro as bz
@@ -120,7 +121,7 @@ async def ingest_bzzoiro_event_stats(
except Exception as e: except Exception as e:
logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e) logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e)
result["errors"].append(f"match {m.id}: {e}") result["errors"].append(f"match {m.id}: {e}")
await bz._safe_write_ingest_failure( await _safe_write_ingest_failure(
db, db,
entity_type="match_stats", entity_type="match_stats",
source_record_id=str(m.source_event_id), source_record_id=str(m.source_event_id),
@@ -165,8 +166,8 @@ async def ingest_bzzoiro_event_stats(
# 管线基础设施:写入 RawEvent + DataLineage # 管线基础设施:写入 RawEvent + DataLineage
batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}" batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}"
try: try:
await bz._write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id) await _write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id)
await bz._write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id) await _write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id)
except Exception: except Exception:
pass # 基础设施写入失败不影响主流程 pass # 基础设施写入失败不影响主流程
+9 -1
View File
@@ -35,6 +35,8 @@ class NormalizedMatch:
home_team: str home_team: str
away_team: str away_team: str
match_status: str = "finished" match_status: str = "finished"
# P0-01:比分可信度。known=可靠比分;missing=完赛缺分;unknown=待定。
score_status: str = "unknown"
home_goals: int | None = None home_goals: int | None = None
away_goals: int | None = None away_goals: int | None = None
season_label: str = "" season_label: str = ""
@@ -217,5 +219,11 @@ def normalize_bzzoiro(raw: dict, league_type: str) -> NormalizedMatch | None:
m.away_red_cards = _to_int(raw.get("away_red_cards", raw.get("red_cards_away"))) m.away_red_cards = _to_int(raw.get("away_red_cards", raw.get("red_cards_away")))
if m.match_status == "finished" and m.home_goals is None: if m.match_status == "finished" and m.home_goals is None:
m.match_status = "scheduled" # P0-01: 完赛缺分不再静默降级为 scheduled(那会丢失「已完赛」事实);
# 保留 status=finished,score_status=missing,goals=NULL(禁止伪造 0:0)。
m.score_status = "missing"
elif m.home_goals is not None and m.away_goals is not None:
m.score_status = "known"
else:
m.score_status = "unknown"
return m return m
-19
View File
@@ -68,25 +68,6 @@ async def short_read():
yield session yield session
@asynccontextmanager
async def short_write():
"""短生命周期 write session: 提交后立即释放。
用法:
async with short_write() as session:
session.add(pred)
await session.commit()
# session 已关闭,连接已释放
"""
async with AsyncSessionLocal() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
async def init_db() -> None: async def init_db() -> None:
"""验证数据库连接(不建表)。 """验证数据库连接(不建表)。
+76 -12
View File
@@ -19,6 +19,7 @@ from sqlalchemy import (
String, String,
Text, Text,
UniqueConstraint, UniqueConstraint,
column,
func, func,
) )
from sqlalchemy.dialects.postgresql import JSONB from sqlalchemy.dialects.postgresql import JSONB
@@ -56,6 +57,22 @@ class Team(Base):
away_matches: Mapped[list["Match"]] = relationship(foreign_keys="Match.away_team_id", back_populates="away_team") away_matches: Mapped[list["Match"]] = relationship(foreign_keys="Match.away_team_id", back_populates="away_team")
class TeamAlias(Base):
"""球队别名:同一球队的不同写法(大小写/译名/缩写)映射到归一后的 teams.id。
入库流程(get_or_create):normalize(name) → 查 teams.name → 查 team_aliases
→ 都没有再 insert 新 Team。别名不自动合并历史重复队,需显式添加。
alias_normalized 为 normalize(别名)后的稳定幂等键,用作 PK 避免重复插入。
"""
__tablename__ = "team_aliases"
# normalize(别名)后的值,稳定幂等,用作主键
alias_normalized: Mapped[str] = mapped_column(String(120), primary_key=True)
team_id: Mapped[int] = mapped_column(ForeignKey("teams.id", ondelete="CASCADE"), nullable=False)
original_alias: Mapped[str] = mapped_column(String(120), nullable=False) # 原始写法(保留供参考)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow)
class Match(Base): class Match(Base):
__tablename__ = "matches" __tablename__ = "matches"
@@ -72,6 +89,9 @@ class Match(Base):
index=True, index=True,
) )
match_status: Mapped[str] = mapped_column(String(20), default="scheduled") match_status: Mapped[str] = mapped_column(String(20), default="scheduled")
# P0-01:比分可信度标记。known=有可靠比分;missing=完赛但缺分(保留 NULL 不伪造 0:0);
# unknown=待定(无比分且未确认完赛)。禁止把缺分写成 0:0。
score_status: Mapped[str] = mapped_column(String(20), server_default="unknown", nullable=False)
home_goals: Mapped[int | None] = mapped_column(Integer) home_goals: Mapped[int | None] = mapped_column(Integer)
away_goals: Mapped[int | None] = mapped_column(Integer) away_goals: Mapped[int | None] = mapped_column(Integer)
home_ht_goals: Mapped[int | None] = mapped_column(Integer) home_ht_goals: Mapped[int | None] = mapped_column(Integer)
@@ -111,10 +131,19 @@ class Match(Base):
"match_date_date", "match_date_date",
unique=True, unique=True,
), ),
# DB-5: 数据库级约束 — 已完赛比赛必须有比分 # P0-01:比分可信度约束(替代原 ck_matches_finished_has_score):
# - score_status=known → 必须有比分(非 NULL)
# - score_status=missing → 必须 NULL(完赛缺分,禁止伪造 0:0)
# - score_status=unknown → 必须 NULL
CheckConstraint( CheckConstraint(
"match_status <> 'finished' OR (home_goals IS NOT NULL AND away_goals IS NOT NULL)", "score_status IN ('known', 'missing', 'unknown')",
name="ck_matches_finished_has_score", name="ck_matches_score_status_enum",
),
CheckConstraint(
"match_status <> 'finished'"
" OR (score_status = 'known' AND home_goals IS NOT NULL AND away_goals IS NOT NULL)"
" OR (score_status IN ('missing', 'unknown') AND home_goals IS NULL AND away_goals IS NULL)",
name="ck_matches_score_integrity",
), ),
CheckConstraint( CheckConstraint(
"match_status IN ('finished', 'scheduled', 'in_play', 'paused', 'postponed', 'cancelled', 'suspended')", "match_status IN ('finished', 'scheduled', 'in_play', 'paused', 'postponed', 'cancelled', 'suspended')",
@@ -176,8 +205,11 @@ class MatchStats(Base):
class Standing(Base): class Standing(Base):
"""联赛积分榜快照(bzzoiro /leagues/{id}/standings/)。 """联赛积分榜快照(bzzoiro /leagues/{id}/standings/)。
同一联赛同一赛季只保留最新快照:重新采集时按 (league_id, season, team_id) P0-02: 改为追加快照(append-only)。每次采集 INSERT 新行,available_at=now;
upsert。zone 来自 bzzoiro 分区(如 champions_league / europa_league / relegation)。 查询取 available_at<=cutoff 的每队最新快照(DISTINCT ON team_id ORDER available_at DESC)。
回测时可还原任意历史时刻的榜单,不再只是"最新快照、忽略 cutoff"
同一 (league_id, season, team_id, available_at) 唯一,ON CONFLICT DO NOTHING。
zone 来自 bzzoiro 分区(如 champions_league / europa_league / relegation)。
""" """
__tablename__ = "standings" __tablename__ = "standings"
@@ -200,13 +232,17 @@ class Standing(Base):
zone: Mapped[str | None] = mapped_column(String(50)) # champions_league / relegation 等 zone: Mapped[str | None] = mapped_column(String(50)) # champions_league / relegation 等
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow, onupdate=_utcnow) updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow, onupdate=_utcnow)
retrieved_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow) retrieved_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow)
# P0-02: 快照可用时间(采集时间),唯一键组成部分 + cutoff 过滤依据
available_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=_utcnow)
league: Mapped[League] = relationship() league: Mapped[League] = relationship()
team: Mapped[Team] = relationship(lazy="selectin") team: Mapped[Team] = relationship(lazy="selectin")
__table_args__ = ( __table_args__ = (
UniqueConstraint("league_id", "season", "team_id", name="uq_standings_league_season_team"), # P0-02: (league, season, team, available_at) 唯一,支持追加快照 + ON CONFLICT DO NOTHING
UniqueConstraint("league_id", "season", "team_id", "available_at", name="uq_standings_league_season_team_available"),
Index("ix_standings_league_season_pos", "league_id", "season", "position"), Index("ix_standings_league_season_pos", "league_id", "season", "position"),
Index("ix_standings_league_season_team_available", "league_id", "season", "team_id", "available_at"),
) )
@@ -254,14 +290,13 @@ class Prediction(Base):
match: Mapped[Match] = relationship(back_populates="predictions") match: Mapped[Match] = relationship(back_populates="predictions")
__table_args__ = ( __table_args__ = (
# Fix: 唯一约束增加 mode + run_type,允许 live 与 backtest 共存 # P0-03: 幂等指纹——input_hash 非空时唯一(同指纹→返回已有行,不 UPDATE/INSERT);
# 防止回测覆盖未结算的实盘预测(后续 settle 会污染评估数据) # 兼容旧数据 NULL input_hash(不强制回填)。
UniqueConstraint( Index(
"match_id", "provider", "model", "mode", "run_type", "ix_predictions_input_hash_unique", "input_hash", unique=True,
name="uq_predictions_match_provider_model_mode_run_type", postgresql_where=column("input_hash").isnot(None),
), ),
Index("ix_predictions_match", "match_id"), Index("ix_predictions_match", "match_id"),
Index("ix_predictions_provider_model", "provider", "model"),
# 数据截止时间过滤查询用(按 prediction_cutoff_at 取「赛前已生成」的预测) # 数据截止时间过滤查询用(按 prediction_cutoff_at 取「赛前已生成」的预测)
Index("ix_predictions_cutoff_at", "prediction_cutoff_at"), Index("ix_predictions_cutoff_at", "prediction_cutoff_at"),
# 数据库级约束:最后一道防线 # 数据库级约束:最后一道防线
@@ -324,6 +359,35 @@ class RawEvent(Base):
) )
class IngestJob(Base):
"""采集任务状态:跟踪每次触后台采集任务的执行进度与结果。
POST /api/v1/ingest/bzzoiro 触发时写入(pending→running→success/failed),
前端 Collection 页据此轮询到终态,替代此前"30 秒后盲标完成"的模拟。
分批 get_uow / BzzoiroSource / IngestFailure / Bronze/Lineage 均不受影响
(本表仅作状态追踪,不介入采集事务)。
"""
__tablename__ = "ingest_jobs"
id: Mapped[str] = mapped_column(String(36), primary_key=True) # uuid4
task: Mapped[str] = mapped_column(String(20), nullable=False)
params: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict)
status: Mapped[str] = mapped_column(String(20), nullable=False, server_default="pending")
result: Mapped[dict | None] = mapped_column(JSONB)
error: Mapped[str | None] = mapped_column(Text)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
__table_args__ = (
Index("ix_ingest_job_status_created", "status", "created_at"),
CheckConstraint(
"status IN ('pending', 'running', 'success', 'failed')",
name="ck_ingest_job_status",
),
)
class IngestFailure(Base): class IngestFailure(Base):
"""采集失败死信:记录失败原因、重试次数与下次重试时间。 """采集失败死信:记录失败原因、重试次数与下次重试时间。
+59 -24
View File
@@ -80,6 +80,20 @@ class MatchRepository:
) )
return (await self._session.execute(stmt)).scalars().all() return (await self._session.execute(stmt)).scalars().all()
async def find_by_source_event_id(self, source_event_id: int) -> Match | None:
"""按上游 event id 查找比赛(唯一命中,用于 upsert 优先路径)。
source_event_id 上有 partial unique 索引(WHERE IS NOT NULL),
同联赛同主客同天(自然键)与上游 event_id 共同保障同一场比赛
重复采集时 upsert 而非插入重复行。
"""
stmt = (
select(Match)
.options(selectinload(Match.stats))
.where(Match.source_event_id == source_event_id)
)
return (await self._session.execute(stmt)).scalar_one_or_none()
async def find_finished_with_stats(self, league_ids: list[int], *, limit: int) -> list[Match]: async def find_finished_with_stats(self, league_ids: list[int], *, limit: int) -> list[Match]:
"""已完赛且有上游 event id 的比赛(按日期倒序),供统计回填逐场拉取。 """已完赛且有上游 event id 的比赛(按日期倒序),供统计回填逐场拉取。
@@ -114,24 +128,59 @@ class TeamRepository:
async def get_or_create(self, name: str, *, name_zh: str | None = None) -> Team: async def get_or_create(self, name: str, *, name_zh: str | None = None) -> Team:
"""按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。 """按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。
归一化咽喉:所有入库 Team.name 必须经过 team_names.normalize, 归一化咽喉 + 别名查找,三步定位:
此处统一收敛,避免各调用点散落归一化逻辑导致重复 Team。 1) normalize(name) → 查 teams.name
2) 查 team_aliases(以 normalize(name) 为幂等键)→ 复用已映射的 teams.id
3) 都没有 → insert 新 Team(归一名)
创建新 Team 时 info 打出原始名与归一后的规范名,便于排查重名。 创建新 Team 时 info 打出原始名与归一后的规范名,便于排查重名。
不自动合并历史重复队;需显式添加别名。
""" """
from src.data.team_names import normalize as normalize_name from src.data.team_names import normalize as normalize_name
from src.db.models import TeamAlias
normalized = normalize_name(name) or name.strip() normalized = normalize_name(name) or name.strip()
# 1) 归一名直查 teams
team = await self.get_by_name(normalized) team = await self.get_by_name(normalized)
if team is None: if team is not None:
logger.info( return team
"创建新 Team: %s -> %s",
name, normalized, # 2) 别名查找:normalize(别名) 作为幂等键,命中即复用已有 Team
) alias = await self._session.get(TeamAlias, normalized)
team = Team(name=normalized, name_zh=name_zh) if alias is not None:
self._session.add(team) team = await self._session.get(Team, alias.team_id)
await self._session.flush() if team is not None:
logger.info("Team 别名命中: %s -> %s(已有 id=%s)", name, normalized, team.id)
return team
# 3) 新建 Team(归一名)
logger.info("创建新 Team: %s -> %s", name, normalized)
team = Team(name=normalized, name_zh=name_zh)
self._session.add(team)
await self._session.flush()
return team return team
async def add_alias(self, alias: str, team_id: int) -> TeamAlias:
"""为已有 Team 添加别名。
幂等:以 normalize(alias) 为 PK,重复添加同一别名会 upsert。
不自动合并历史重复队,仅建立别名映射。
"""
from src.data.team_names import normalize as normalize_name
from src.db.models import TeamAlias
normalized = normalize_name(alias) or alias.strip()
existing = await self._session.get(TeamAlias, normalized)
if existing is not None:
existing.team_id = team_id # 允许重新指向
existing.original_alias = alias
await self._session.flush()
return existing
row = TeamAlias(alias_normalized=normalized, team_id=team_id, original_alias=alias)
self._session.add(row)
await self._session.flush()
return row
async def get_all_by_names(self, names: list[str]) -> dict[str, Team]: async def get_all_by_names(self, names: list[str]) -> dict[str, Team]:
"""批量获取球队,返回 name → Team 映射。""" """批量获取球队,返回 name → Team 映射。"""
if not names: if not names:
@@ -166,17 +215,3 @@ class LeagueRepository:
async def add(self, league: League) -> None: async def add(self, league: League) -> None:
self._session.add(league) self._session.add(league)
await self._session.flush() await self._session.flush()
class PredictionRepository:
"""预测记录数据访问。"""
def __init__(self, session: AsyncSession) -> None:
self._session = session
async def get_by_id(self, prediction_id: int) -> Prediction | None:
return await self._session.get(Prediction, prediction_id)
async def add(self, prediction: Prediction) -> None:
self._session.add(prediction)
await self._session.flush()
+19 -12
View File
@@ -12,7 +12,7 @@ from src.core.config import settings
from src.db.base import AsyncSessionLocal from src.db.base import AsyncSessionLocal
from src.db.models import Match, Prediction from src.db.models import Match, Prediction
from src.db.unit_of_work import get_uow from src.db.unit_of_work import get_uow
from src.llm.predict import PredictResult, _upsert_prediction from src.llm.predict import PredictResult, _insert_or_find_by_fingerprint
from src.llm.agents.base import AgentReport, AgentSpec, load_agent_prompt from src.llm.agents.base import AgentReport, AgentSpec, load_agent_prompt
from src.llm.context_builder import ( from src.llm.context_builder import (
MatchHeader, MatchHeader,
@@ -285,10 +285,13 @@ async def predict_match_multi(
latency_ms = int((time.perf_counter() - start) * 1000) latency_ms = int((time.perf_counter() - start) * 1000)
# 3.5 计算输入 hash(基于终裁报告) # P0-03: 指纹输入——终裁报告 hash 作 context_hash,专家列表作 agent_ids
input_hash = hashlib.sha256( reports_json = _reports_to_json(reports)
_reports_to_json(reports).encode("utf-8") context_hash = hashlib.sha256(reports_json.encode("utf-8")).hexdigest()
).hexdigest() agent_ids = sorted([r.agent for r in reports]) if reports else []
# 终裁模板 hash(规范:复用 prompt 版本 + 终裁 system prompt)
prompt_hash = hashlib.sha256(f"multi_{version}".encode("utf-8")).hexdigest()
system_prompt_hash = hashlib.sha256(AGGREGATOR_SYSTEM.encode("utf-8")).hexdigest()
# 4. 存库(使用 UnitOfWork) # 4. 存库(使用 UnitOfWork)
async with get_uow() as session: async with get_uow() as session:
@@ -315,15 +318,20 @@ async def predict_match_multi(
pred_status = "degraded" pred_status = "degraded"
model_name = aggregator_model model_name = aggregator_model
pred = await _upsert_prediction( pred = await _insert_or_find_by_fingerprint(
session, session,
match_id=match_id,
provider_name=settings.LLM_PROVIDER,
model=model_name,
mode="multi",
run_type="backtest" if backtest else "live",
values={ values={
"match_id": match_id,
"provider": settings.LLM_PROVIDER,
"model": model_name,
"mode": "multi",
"run_type": "backtest" if backtest else "live",
"prompt_version": f"multi_{version}", "prompt_version": f"multi_{version}",
"prompt_hash": prompt_hash,
"system_prompt_hash": system_prompt_hash,
"temperature": 0.2,
"context_hash": context_hash,
"agent_ids": agent_ids,
"prompt_tokens": sum(r.prompt_tokens or 0 for r in reports) + agg_prompt_tokens, "prompt_tokens": sum(r.prompt_tokens or 0 for r in reports) + agg_prompt_tokens,
"completion_tokens": sum(r.completion_tokens or 0 for r in reports) + agg_completion_tokens, "completion_tokens": sum(r.completion_tokens or 0 for r in reports) + agg_completion_tokens,
"latency_ms": latency_ms, "latency_ms": latency_ms,
@@ -341,7 +349,6 @@ async def predict_match_multi(
"match_kickoff_at": match_kickoff_at, "match_kickoff_at": match_kickoff_at,
"prediction_cutoff_at": prediction_cutoff_at, "prediction_cutoff_at": prediction_cutoff_at,
"prediction_created_at": now, "prediction_created_at": now,
"input_hash": input_hash,
}, },
) )
+1
View File
@@ -130,6 +130,7 @@ async def _get_historical_matches(
selectinload(Match.away_team), selectinload(Match.away_team),
) )
.where(Match.match_status == "finished") .where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None)) .where(Match.home_goals.is_not(None))
.where(Match.away_goals.is_not(None)) .where(Match.away_goals.is_not(None))
) )
+20 -12
View File
@@ -5,14 +5,15 @@
""" """
from __future__ import annotations from __future__ import annotations
import hashlib
import logging import logging
from datetime import datetime from datetime import datetime, timedelta, timezone
from sqlalchemy import case, func, select from sqlalchemy import case, func, select
from src.db.base import AsyncSession, AsyncSessionLocal from src.db.base import AsyncSession, AsyncSessionLocal
from src.db.models import Match from src.db.models import Match
from src.llm.predict import PredictResult, _upsert_prediction from src.llm.predict import PredictResult, _insert_or_find_by_fingerprint
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -97,8 +98,23 @@ async def predict_baseline(
else: else:
pred_1x2 = "X" pred_1x2 = "X"
# P0-03: 基线指纹——基于主客场场均进球数据(context_hash) + 截止时间
context_hash = hashlib.sha256(
f"{home_avg:.4f}:{away_avg:.4f}:{before.isoformat() if before else 'none'}".encode("utf-8")
).hexdigest()
values = { values = {
"match_id": match_id,
"provider": "baseline",
"model": "baseline",
"mode": "baseline",
"run_type": "live",
"prompt_version": "baseline_v1", "prompt_version": "baseline_v1",
"prompt_hash": hashlib.sha256(b"baseline_v1").hexdigest(),
"system_prompt_hash": hashlib.sha256(b"baseline").hexdigest(),
"temperature": 0.0,
"context_hash": context_hash,
"agent_ids": [],
"prompt_tokens": 0, "prompt_tokens": 0,
"completion_tokens": 0, "completion_tokens": 0,
"latency_ms": 0, "latency_ms": 0,
@@ -114,17 +130,9 @@ async def predict_baseline(
"status": "success", "status": "success",
} }
# P3-2:服务层落库,回填真实 prediction_id(与 single/multi 统一)。 # P0-03:服务层幂等插入,回填真实 prediction_id(与 single/multi 统一)。
async with get_uow() as session: async with get_uow() as session:
pred = await _upsert_prediction( pred = await _insert_or_find_by_fingerprint(session, values=values)
session,
match_id=match_id,
provider_name="baseline",
model="baseline",
mode="baseline",
run_type="live",
values=values,
)
prediction_id = pred.id prediction_id = pred.id
return PredictResult( return PredictResult(
+32 -547
View File
@@ -1,9 +1,9 @@
"""上下文构建器:数据切片 + 拼接。 """上下文构建器:数据切片 + 拼接(聚合门面)
架构: 实现按 slice 拆分(单文件 slices ),本模块只做再导出:
- match_header: 比赛基础信息(对阵双方/联赛/时间) - 切片函数: 每个领域 agent 一个数据切片 src/llm/slices/{form,h2h,stats,home_away,standings}.py
- 切片函数: 每个领域 agent 一个数据切片(h2h / form / standings / stats) - 共享类型/头信息/查询助手 src/llm/slices/common.py
- build_context: agent 路径,拼接全部切片(行为与旧版一致) - build_context: agent 路径,拼接全部切片(行为与旧版一致) src/llm/slices/aggregate.py
multi-agent 路径由 agents/orchestrator.py 调用切片函数,每个专家只拿自己的切片 multi-agent 路径由 agents/orchestrator.py 调用切片函数,每个专家只拿自己的切片
@@ -11,548 +11,33 @@ multi-agent 路径由 agents/orchestrator.py 调用切片函数,每个专家只
build_context 创建一个共享 session 并传给所有切片函数, build_context 创建一个共享 session 并传给所有切片函数,
避免每个切片独立创建 session 回测 20 场并发时, 避免每个切片独立创建 session 回测 20 场并发时,
5 个切片 × 20 = 100 个连接会耗尽连接池(pool_size=15) 5 个切片 × 20 = 100 个连接会耗尽连接池(pool_size=15)
消费方(路由/orchestrator/tests)仍从本模块 import,签名与拆分前完全一致
""" """
from __future__ import annotations from __future__ import annotations
import logging # ── 共享类型与基础(判空/赛果/统计可用性) ──
from dataclasses import dataclass from src.llm.slices.common import ( # noqa: F401
from typing import TYPE_CHECKING MatchContext,
MatchHeader,
from sqlalchemy import select SliceResult,
from sqlalchemy.orm import selectinload _is_stats_available,
_outcome,
from src.db.base import AsyncSessionLocal header_text,
from src.db.models import Match load_match_header,
)
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession # ── 领域切片 ──
from src.llm.slices.form import form_slice # noqa: F401
logger = logging.getLogger(__name__) from src.llm.slices.h2h import h2h_slice # noqa: F401
from src.llm.slices.home_away import home_away_slice # noqa: F401
from src.llm.slices.standings import standings_slice # noqa: F401
def _outcome(home_goals: int, away_goals: int, side: str) -> str: from src.llm.slices.stats import stats_slice # noqa: F401
"""从某队视角看赛果: W/D/L。"""
if home_goals is None or away_goals is None: # ── 底层查询助手(切片函数共用;orchestrator/tests 直接引用) ──
return "?" from src.llm.slices.form import _get_form # noqa: F401
if side == "home": from src.llm.slices.h2h import _get_h2h # noqa: F401
return "W" if home_goals > away_goals else ("D" if home_goals == away_goals else "L") from src.llm.slices.home_away import _get_home_away # noqa: F401
return "W" if away_goals > home_goals else ("D" if away_goals == home_goals else "L")
# ── 单 agent 聚合入口 ──
from src.llm.slices.aggregate import build_context # noqa: F401
def _is_stats_available(stats, before) -> bool:
"""检查统计数据在 cutoff 时间是否已可用。
available_at 语义:该条统计对外可被使用的最早时间,
至少不得早于比赛结束用于回测防泄漏
规则:
- before is None(实盘):available_at None 时允许(兼容旧数据)
- before is not None(回测):available_at None 视为不可用(保守)
- available_at > cutoff:不可用(数据在 cutoff 之后才生成)
"""
if before is None:
# 实盘模式:无时间信息时允许(兼容旧数据)
return True
# 回测模式(cutoff 不为 None):
# available_at 为 None → 无法确认是否在 cutoff 前可用,保守视为不可用
if stats.available_at is None:
return False
return stats.available_at <= before
@dataclass
class SliceResult:
"""数据切片的显式结果(替代「靠文案子串猜有无数据」)。
旧实现用 `"无数据" in slice_text` 判断,依赖具体文案 一旦某个切片
写成无比分数据无伤停数据这类变体,判断就会静默失配
(见审查报告 P2-1)这里让切片函数直接声明 `has_data`,不再猜
"""
text: str
has_data: bool
n_records: int = 0
def __str__(self) -> str: # 让老调用点可直接当 str 用
return self.text
@dataclass
class MatchContext:
match_id: int
text: str
has_stats: bool
has_standings: bool
match_dt: object | None = None # 比赛时间(回测防泄漏 + 快照用)
cutoff: object | None = None # 实际使用的数据截止时间(用于落库记录)
@dataclass
class MatchHeader:
"""比赛基础信息(所有 agent 共享)。"""
match_id: int
home_name: str
away_name: str
league_name: str
season: str | None
match_date: str
match_dt: object # 原始 datetime,回测防泄漏用
stage: str | None
home_team_id: int
away_team_id: int
league_id: int
async def load_match_header(match_id: int, db: AsyncSession | None = None) -> MatchHeader:
"""加载比赛头信息(各 agent 共用)。
Args:
match_id: 比赛 ID
db: 可选的共享 session不传则自建(向后兼容)
"""
if db is not None:
m = await _load_match(db, match_id)
return _to_header(m)
async with AsyncSessionLocal() as new_db:
m = await _load_match(new_db, match_id)
return _to_header(m)
def _to_header(m: Match) -> MatchHeader:
return MatchHeader(
match_id=m.id,
home_name=m.home_team.name_zh or m.home_team.name,
away_name=m.away_team.name_zh or m.away_team.name,
league_name=m.league.name if m.league else "?",
season=m.season,
match_date=m.match_date.strftime("%Y-%m-%d %H:%M UTC") if m.match_date else "?",
match_dt=m.match_date,
stage=m.match_stage,
home_team_id=m.home_team_id,
away_team_id=m.away_team_id,
league_id=m.league_id,
)
def header_text(h: MatchHeader) -> str:
stage = f" {h.stage}" if h.stage else ""
return (
f"对阵: {h.home_name} vs {h.away_name} | {h.league_name} {h.season or '?'}{stage} | {h.match_date}"
)
# ============================================================
# 切片函数: 每个领域 agent 一个
# ============================================================
async def h2h_slice(header: MatchHeader, *, limit: int = 8, before=None, db: AsyncSession | None = None) -> SliceResult:
"""E - 历史交锋切片: 过去数年 + 近期交手数据,提取交手规律。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
h2h = await _get_h2h(db, header.home_team_id, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
h2h = await _get_h2h(new_db, header.home_team_id, header.away_team_id, before=before, limit=limit)
lines = [f"── 历史交锋(近 {limit} 次) ──"]
n_with_score = 0
if h2h:
# 从当前主队视角统计:判断当前主队在每场交锋中是主是客
current_home_wins = current_home_draws = current_home_losses = 0
for hm in h2h:
d = hm.match_date.strftime("%Y-%m") if hm.match_date else "?"
if hm.home_goals is not None:
n_with_score += 1
# 判断当前主队当时是主队还是客队
if hm.home_team_id == header.home_team_id:
# 当前主队当时是主队
if hm.home_goals > hm.away_goals:
current_home_wins += 1
elif hm.home_goals == hm.away_goals:
current_home_draws += 1
else:
current_home_losses += 1
else:
# 当前主队当时是客队(从客队视角看赛果)
if hm.away_goals > hm.home_goals:
current_home_wins += 1
elif hm.away_goals == hm.home_goals:
current_home_draws += 1
else:
current_home_losses += 1
lines.append(f" {d}: {hm.home_team.name} {hm.home_goals}-{hm.away_goals} {hm.away_team.name}")
else:
lines.append(f" {d}: {hm.home_team.name} vs {hm.away_team.name} (无比分)")
total = current_home_wins + current_home_draws + current_home_losses
if total:
lines.append(
f" 总计 {total} 场(从当前主队 {header.home_name} 视角): "
f"{current_home_wins}{current_home_draws}{current_home_losses}"
)
else:
lines.append(" 无数据")
# has_data 以「有比分的交锋」为准:仅有对阵无比分时不足以支撑分析
return SliceResult(text="\n".join(lines), has_data=n_with_score > 0, n_records=n_with_score)
async def form_slice(header: MatchHeader, *, limit: int = 5, before=None, db: AsyncSession | None = None) -> SliceResult:
"""A - 近期状态切片: 两队近 N 场赛果、关键事件、走势判断。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = []
n_scored = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则客场输球会被算成主场赢球。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
lines.append(f"── {label}近况({name},近 {limit} 场) ──")
if form:
wins = draws = losses = 0
for fm in form:
is_home = (fm.home_team_id == team_id)
side = "home" if is_home else "away"
o = _outcome(fm.home_goals, fm.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
if fm.home_goals is not None:
n_scored += 1
score = f"{fm.home_goals}-{fm.away_goals}" if fm.home_goals is not None else "vs"
xg = ""
if fm.stats and _is_stats_available(fm.stats, before) and fm.stats.home_xg is not None:
own = fm.stats.home_xg if is_home else fm.stats.away_xg
xg = f" (xG {own:.1f})"
opp = fm.away_team.name if is_home else fm.home_team.name
lines.append(f" {o} {score} vs {opp}{xg}")
lines.append(f"{len(form)} 场: {wins}{draws}{losses}")
else:
lines.append(" 无数据")
return SliceResult(text="\n".join(lines), has_data=n_scored > 0, n_records=n_scored)
async def stats_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""B - 攻防数据切片: 进球、射门、控球,评估攻防强度。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = [f"── 攻防数据(近 {limit} 场) ──"]
n_total = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则进球/失球/xG 全部算反。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
if form:
gf = ga = shots = sot = poss = xg = xga = 0
n = n_shots = n_poss = n_xg = 0
for fm in form:
if fm.home_goals is None: continue
is_home = (fm.home_team_id == team_id)
gf += fm.home_goals if is_home else fm.away_goals
ga += fm.away_goals if is_home else fm.home_goals
n += 1
# 只使用 cutoff 之前已可用的统计数据
if fm.stats and _is_stats_available(fm.stats, before):
if fm.stats.home_shots is not None:
shots += fm.stats.home_shots if is_home else fm.stats.away_shots
sot += fm.stats.home_shots_on_target if is_home else fm.stats.away_shots_on_target
n_shots += 1
if fm.stats.home_possession is not None:
poss += fm.stats.home_possession if is_home else (100 - fm.stats.home_possession)
n_poss += 1
if fm.stats.home_xg is not None:
xg += fm.stats.home_xg if is_home else fm.stats.away_xg
xga += fm.stats.away_xg if is_home else fm.stats.home_xg
n_xg += 1
n_total += n
if n > 0:
lines.append(f" {label} {name}:")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
if n_shots: lines.append(f" 场均射门 {shots/n_shots:.1f}, 射正 {sot/n_shots:.1f}")
if n_poss: lines.append(f" 平均控球 {poss/n_poss:.1f}%")
if n_xg: lines.append(f" 场均 xG {xg/n_xg:.2f}, 场均被 xG {xga/n_xg:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def home_away_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""C - 主客因素切片: 主场战绩 vs 客场战绩,评估地理优势影响。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_home = await _get_home_away(db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(db, header.away_team_id, "away", before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_home = await _get_home_away(new_db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(new_db, header.away_team_id, "away", before=before, limit=limit)
lines = ["── 主客因素 ──"]
n_total = 0
for label, name, matches, side in (
("主队主场", header.home_name, home_home, "home"),
("客队客场", header.away_name, away_away, "away"),
):
if matches:
wins = draws = losses = gf = ga = 0
for m in matches:
if m.home_goals is None: continue
o = _outcome(m.home_goals, m.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
gf += m.home_goals if side == "home" else m.away_goals
ga += m.away_goals if side == "home" else m.home_goals
n = wins + draws + losses
n_total += n
if n > 0:
pct = wins / n * 100
lines.append(f" {label} {name}(近 {n} 场): {wins}{draws}{losses}负, 胜率 {pct:.0f}%")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def standings_slice(header: MatchHeader, *, before=None, db: AsyncSession | None = None) -> SliceResult:
"""D - 联赛排名切片: 两队积分榜位置、积分、近期走势(form)、分区,评估整体实力差距。
before 参数保留与其他切片一致的签名(积分榜是最新快照,无历史版本,不受 cutoff 影响)
db: 可选共享 session(见模块 docstring)
语义区分:
- 两队都有积分榜行 has_data=True(明确的排名信息)
- 任一队缺失 has_data=False(升班马/杯赛无榜,信息不完整时明确声明)
"""
from src.db.models import League, Standing
if db is not None:
league = (await db.execute(select(League).where(League.id == header.league_id))).scalar_one_or_none()
rows = (
(
await db.execute(
select(Standing)
.options(selectinload(Standing.team))
.where(Standing.league_id == header.league_id)
.order_by(Standing.position.asc())
)
)
.scalars()
.all()
if league
else []
)
else:
async with AsyncSessionLocal() as new_db:
return await standings_slice(header, before=before, db=new_db)
lines = [f"── 联赛排名({header.league_name}{len(rows)} 队) ──"]
n_records = 0
def _fmt(row) -> str:
zg = f" xG差 {row.xgd:+.1f}" if row.xg_for is not None and row.xg_against is not None and row.goal_diff is not None else ""
form = f" 近5场 {row.form}" if row.form else ""
zone = f" [{row.zone}]" if row.zone else ""
return (
f"{row.position} 名: {row.points} 分 / {row.played}"
f"({row.won}{row.drawn}{row.lost}负, 进{row.goals_for}{row.goals_against} 净胜{row.goal_diff:+d}"
f"{zg}){form}{zone}"
)
for label, team_id in (("主队", header.home_team_id), ("客队", header.away_team_id)):
row = next((r for r in rows if r.team_id == team_id), None)
if row is None:
lines.append(f" {label}: 暂无积分榜数据(可能杯赛/赛季未开始)")
else:
n_records += 1
lines.append(f" {label} {header.home_name if label == '主队' else header.away_name}:")
lines.append(_fmt(row))
# 两队排名对比摘要
home_row = next((r for r in rows if r.team_id == header.home_team_id), None)
away_row = next((r for r in rows if r.team_id == header.away_team_id), None)
if home_row and away_row:
diff = home_row.position - away_row.position # 正数=主队排名更靠前(名次更小)
lead = f"主队排名高 {diff}" if diff > 0 else (f"客队排名高 {-diff}" if diff < 0 else "两队同排名结构")
pts_diff = home_row.points - away_row.points
lines.append(f" 排名对比: {lead}, 分差 {pts_diff:+d}")
# has_data: 两队都有行才算完整;只有一队时仍有价值,但标记不完整
has_data = n_records >= 1
return SliceResult(text="\n".join(lines), has_data=has_data, n_records=n_records)
# ============================================================
# 单 agent 路径: 拼接全部切片(行为与旧版一致)
# ============================================================
async def build_context(match_id: int, *, form_last: int = 5, h2h_last: int = 5, backtest: bool = False, cutoff_at=None) -> MatchContext:
"""单 agent 路径的完整上下文: 拼接全部切片(before=cutoff,防未来信息)。
has_stats / has_standings 直接取切片显式声明的 has_data,
不再靠文案子串匹配(见审查报告 P2-1)
P2-6: backtest=True cutoff = match_date - 1,确保只用赛前数据
cutoff_at: 显式截止时间(优先于 backtest 自动计算)
P1-1: 使用单个共享 session 贯穿所有切片查询,避免连接池耗尽
"""
async with AsyncSessionLocal() as db:
header = await load_match_header(match_id, db=db)
# 计算数据截止时间: 显式 > backtest 自动计算 > 默认(比赛时间)
if cutoff_at is not None:
cutoff = cutoff_at
elif backtest and header.match_dt:
from datetime import timedelta
cutoff = header.match_dt - timedelta(days=1)
else:
cutoff = header.match_dt
parts = [header_text(header), ""]
form_res = await form_slice(header, limit=form_last, before=cutoff, db=db)
parts.append(form_res.text)
parts.append("")
h2h_res = await h2h_slice(header, limit=h2h_last, before=cutoff, db=db)
parts.append(h2h_res.text)
parts.append("")
stats_res = await stats_slice(header, before=cutoff, db=db)
parts.append(stats_res.text)
parts.append("")
home_away_res = await home_away_slice(header, before=cutoff, db=db)
parts.append(home_away_res.text)
parts.append("")
standings_res = await standings_slice(header, before=cutoff, db=db)
parts.append(standings_res.text)
return MatchContext(
match_id=match_id,
text="\n".join(parts),
has_stats=form_res.has_data or stats_res.has_data,
has_standings=standings_res.has_data,
match_dt=header.match_dt,
cutoff=cutoff,
)
# ============================================================
# 底层查询(切片函数共用)
# ============================================================
async def _load_match(db, match_id: int) -> Match:
stmt = (
select(Match)
.where(Match.id == match_id)
.options(
selectinload(Match.league),
selectinload(Match.home_team),
selectinload(Match.away_team),
selectinload(Match.stats),
)
)
m = (await db.execute(stmt)).scalar_one_or_none()
if m is None:
raise ValueError(f"match {match_id} not found")
return m
async def _get_form(db, team_id: int, before, *, limit: int = 5) -> list[Match]:
"""某队近 N 场(已完赛)。before=None 表示不限制(预测赛前的场景由调用方保证)。
必须预加载 stats / home_team / away_team:切片函数会读取这些关系,
async session 下惰性加载会抛 MissingGreenlet
(models.py 已声明 lazy="selectin",此处显式声明以固化查询意图)
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.where((Match.home_team_id == team_id) | (Match.away_team_id == team_id))
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
async def _get_h2h(db, home_id: int, away_id: int, before, *, limit: int = 5) -> list[Match]:
"""两队交锋史。需预加载 home_team / away_team(切片输出队名)。"""
stmt = (
select(Match)
.options(
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.where(
((Match.home_team_id == home_id) & (Match.away_team_id == away_id))
| ((Match.home_team_id == away_id) & (Match.away_team_id == home_id))
)
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
async def _get_home_away(db, team_id: int, side: str, before, *, limit: int = 10) -> list[Match]:
"""某队主场/客场近 N 场。side='home' 取主场,'away' 取客场。
当前只用标量字段,但统一预加载以免后续扩展时踩坑
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.order_by(Match.match_date.desc())
.limit(limit)
)
if side == "home":
stmt = stmt.where(Match.home_team_id == team_id)
else:
stmt = stmt.where(Match.away_team_id == team_id)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+3 -6
View File
@@ -7,6 +7,7 @@ from sqlalchemy import func, or_, select
from src.db.models import Prediction, Match, League from src.db.models import Prediction, Match, League
from src.db.unit_of_work import get_uow from src.db.unit_of_work import get_uow
from src.llm.utils import actual_1x2
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -33,12 +34,8 @@ async def settle_prediction(prediction_id: int, home_goals: int, away_goals: int
def _actual_1x2(home: int, away: int) -> str: def _actual_1x2(home: int, away: int) -> str:
"""根据实际比分返胜平负。""" """根据实际比分返胜平负(委托 utils.actual_1x2 单一权威源)"""
if home > away: return actual_1x2(home, away)
return "1"
if home == away:
return "X"
return "2"
def _build_filters( def _build_filters(
+196 -74
View File
@@ -1,4 +1,16 @@
"""预测服务:拼上下文 → 调 LLM → 存预测。""" """预测服务:拼上下文 → 调 LLM → 存预测。
落库层级(table: predictions):
| 模式 | 落库位置(服务层) | 路由层(routes/predict.py) |
|-----------|-------------------------------------------------------------|---------------------------|
| single | `_predict_single` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
| multi | `orchestrator.predict_match_multi` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
| baseline | `predict_baseline` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
三种模式统一在服务层经 UnitOfWork 落库并回填真实 prediction_id;
路由层永不写入 predictions,只读 result.prediction_id 做响应映射
"""
from __future__ import annotations from __future__ import annotations
import functools import functools
@@ -25,9 +37,7 @@ _PROMPT_DIR = Path(__file__).resolve().parent / "prompts"
# ── LLM 响应缓存(match+provider+model+version → 结果) ── # ── LLM 响应缓存(match+provider+model+version → 结果) ──
_CACHE_TTL_SEC = 300 # 5 分钟 _CACHE_TTL_SEC = 300 # 5 分钟
_CACHE_MAX_SIZE = 200 # P3-1: 有上限,避免长期运行内存无限增长 _CACHE_MAX_SIZE = 200 # P3-1: 有上限,避免长期运行内存无限增长
# P1-5: 缓存仅在 asyncio 协程内同步访问(dict 操作 GIL 原子),无需 threading.Lock。 _CACHE_PREFIX = "predict:" # Redis key 前缀
# 删除 _cache_lock,避免同步锁阻塞事件循环;dict 的 get/set 在 CPython 下原子。
_cache: dict[str, tuple[float, PredictResult]] = {}
def _cache_key(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> str: def _cache_key(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> str:
@@ -36,43 +46,132 @@ def _cache_key(match_id: int, provider: str, model: str, version: str, tpl_hash:
仅用 version 做键不够 编辑器里改动 `match_prediction_v1.md` 而版本号 仅用 version 做键不够 编辑器里改动 `match_prediction_v1.md` 而版本号
不变时,进程内缓存仍会返回旧模板产生的旧结果(见审查报告 P2-6) 不变时,进程内缓存仍会返回旧模板产生的旧结果(见审查报告 P2-6)
把模板内容 hash 纳入键,模板一改缓存自动失效 把模板内容 hash 纳入键,模板一改缓存自动失效
内存与 Redis 共用同一键格式,TTL 一致
""" """
return f"{match_id}:{provider}:{model}:{version}:{tpl_hash[:12]}" return f"{_CACHE_PREFIX}{match_id}:{provider}:{model}:{version}:{tpl_hash[:12]}"
def _get_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> PredictResult | None: # ── 缓存后端:内存(LRU+TTL),可选 Redis ──────────────────────────
# P1-5: 无锁访问。dict get/del 在 CPython GIL 下原子,且无 await 穿插。 class _CacheBackend:
key = _cache_key(match_id, provider, model, version, tpl_hash) """缓存后端统一接口:_get 同步返回(命中时),_set 异步(Redis 为 async,内存同步)。"""
entry = _cache.get(key)
if entry is not None: def _raw_key(self, key: str) -> str:
ts, result = entry return key
if time.time() - ts < _CACHE_TTL_SEC:
return result def get(self, key: str) -> PredictResult | None:
_cache.pop(key, None) raise NotImplementedError
return None
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
raise NotImplementedError
def _set_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str, result: PredictResult) -> None: class _MemoryCache(_CacheBackend):
# P1-5: 无锁写入。同上,dict set 原子。 """进程内 LRU+TTL 缓存(默认后端)。"""
key = _cache_key(match_id, provider, model, version, tpl_hash)
_cache[key] = (time.time(), result) def __init__(self) -> None:
# P3-1: 超过上限时淘汰最旧条目(按时间戳排序) self._store: dict[str, tuple[float, PredictResult]] = {}
if len(_cache) > _CACHE_MAX_SIZE:
oldest_key = min(_cache, key=lambda k: _cache[k][0]) def get(self, key: str) -> PredictResult | None:
_cache.pop(oldest_key, None) entry = self._store.get(key)
if entry is not None:
ts, result = entry
if time.time() - ts < _CACHE_TTL_SEC:
return result
self._store.pop(key, None)
return None
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
self._store[key] = (time.time(), result)
if len(self._store) > _CACHE_MAX_SIZE:
oldest_key = min(self._store, key=lambda k: self._store[k][0])
self._store.pop(oldest_key, None)
def clear_prompt_cache() -> None: class _RedisCache(_CacheBackend):
"""清空 prompt 模板缓存(供开发/热更新时手动调用) """可选 Redis 后端: PREDICT_CACHE_URL 非空时启用
lru_cache 的模板缓存是进程级的,改完 .md 需要重启进程才能生效; 失败降级:读失败返回 None(跳过缓存),写失败打 warning;不中断预测主流程
提供显式清理入口,避免"改了模板却看不到变化"的困惑(见审查报告 P2-5) 不强制依赖 redis 未安装时启动回退内存并 warning
""" """
_load_prompt_template.cache_clear()
logger.info("prompt 模板缓存已清空") def __init__(self, url: str) -> None:
self._url = url
self._redis = None # type: ignore[var-annotated]
self._memory_fallback = _MemoryCache()
self._available: bool | None = None # None=未探测,True=可用,False=不可用
async def _ensure_conn(self) -> bool:
"""懒初始化 Redis 连接;失败返回 False 并降级内存。"""
if self._available is not None:
return self._available
try:
from redis.asyncio import Redis
self._redis = Redis.from_url(self._url, decode_responses=True, socket_timeout=2.0)
await self._redis.ping()
self._available = True
logger.info("predict cache: Redis 后端已连接 %s", self._url.replace(self._url.split("@")[-1] if "@" in self._url else self._url, "***") if "://" in self._url else "redis")
except Exception as e:
self._available = False
logger.warning("predict cache: Redis 连接失败(%s),降级内存缓存", e)
return self._available
def get(self, key: str) -> PredictResult | None:
# Redis get 是 async 的,此处统一由调用方走 async 路径;
# 同步 get 仅用于不可降级场景——Redis 模式下直接返回 None,
# 实际读取通过 get_async 完成。
return None
async def get_async(self, key: str) -> PredictResult | None:
if not await self._ensure_conn():
return self._memory_fallback.get(key)
try:
import pickle
raw = await self._redis.get(key) # type: ignore[union-attr]
if raw is None:
return None
return pickle.loads(raw.encode("latin-1")) if isinstance(raw, str) else pickle.loads(raw)
except Exception as e:
logger.warning("predict cache: Redis GET 失败(%s),跳过缓存", e)
return None
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
if not await self._ensure_conn():
await self._memory_fallback.set(key, result, ttl)
return
try:
import pickle
payload = pickle.dumps(result).decode("latin-1")
await self._redis.set(key, payload, ex=ttl) # type: ignore[union-attr]
except Exception as e:
logger.warning("predict cache: Redis SET 失败(%s),降级内存写入", e)
await self._memory_fallback.set(key, result, ttl)
def _build_cache_backend() -> _CacheBackend:
url = getattr(settings, "PREDICT_CACHE_URL", None)
if url:
return _RedisCache(url)
return _MemoryCache()
_cache_backend: _CacheBackend = _build_cache_backend()
async def _get_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> PredictResult | None:
key = _cache_key(match_id, provider, model, version, tpl_hash)
if isinstance(_cache_backend, _RedisCache):
return await _cache_backend.get_async(key)
return _cache_backend.get(key)
async def _set_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str, result: PredictResult) -> None:
key = _cache_key(match_id, provider, model, version, tpl_hash)
await _cache_backend.set(key, result, _CACHE_TTL_SEC)
@functools.lru_cache(maxsize=8)
def _load_prompt_template(version: str = "v1") -> str: def _load_prompt_template(version: str = "v1") -> str:
"""缓存 prompt 模板(进程生命周期内每个版本只读一次)。""" """缓存 prompt 模板(进程生命周期内每个版本只读一次)。"""
path = _PROMPT_DIR / f"match_prediction_{version}.md" path = _PROMPT_DIR / f"match_prediction_{version}.md"
@@ -121,44 +220,60 @@ class PredictResult:
raw: dict | None = None raw: dict | None = None
async def _upsert_prediction( def _compute_fingerprint(values: dict) -> str:
session, """P0-03: 预测指纹(规范 JSON 的 SHA-256)。
*,
match_id: int,
provider_name: str,
model: str,
mode: str,
run_type: str,
values: dict,
) -> Prediction:
"""按 (match, provider, model, mode, run_type) 唯一约束写入预测。
已存在且未结算 覆盖更新(重新预测语义);已结算 拒绝(保护评估数据) 捕获影响预测输出的全部因素:输入提示模型采样截止时间专家
run_type 区分 live/backtest,避免回测覆盖实盘预测 fingerprint 返回已有行( UPDATE/INSERT);不同 INSERT 新行
""" """
import json as _json
canonical = {
"match_id": values.get("match_id"),
"prediction_cutoff_at": _iso(values.get("prediction_cutoff_at")),
"prompt_version": values.get("prompt_version"),
"prompt_hash": values.get("prompt_hash"),
"system_prompt_hash": values.get("system_prompt_hash"),
"provider": values.get("provider"),
"model": values.get("model"),
"mode": values.get("mode"),
"run_type": values.get("run_type"),
"temperature": values.get("temperature"),
"context_hash": values.get("context_hash"),
"agent_ids": sorted(values.get("agent_ids") or []),
}
blob = _json.dumps(canonical, sort_keys=True, separators=(',', ':'))
return hashlib.sha256(blob.encode("utf-8")).hexdigest()
def _iso(v) -> str | None:
if v is None:
return None
if hasattr(v, "isoformat"):
return v.isoformat()
return str(v)
async def _insert_or_find_by_fingerprint(session, *, values: dict) -> Prediction:
"""P0-03: 幂等插入——同 input_hash 返回已有行(不 UPDATE);不同则 INSERT。
不再按 (match, provider, model, mode, run_type) upsert,避免覆盖已有预测
values 必须包含 fingerprint 所需全部字段( _compute_fingerprint)
"""
fingerprint = _compute_fingerprint(values)
values["input_hash"] = fingerprint
existing = ( existing = (
await session.execute( await session.execute(
select(Prediction).where( select(Prediction).where(Prediction.input_hash == fingerprint)
Prediction.match_id == match_id,
Prediction.provider == provider_name,
Prediction.model == model,
Prediction.mode == mode,
Prediction.run_type == run_type,
)
) )
).scalar_one_or_none() ).scalar_one_or_none()
if existing is not None and existing.settled: if existing is not None:
raise ValueError("该比赛已有已结算的预测,不能重新预测") # 同指纹 → 直接返回,绝不覆盖 pred_* / reasoning / agent_outputs
return existing
pred = existing if existing is not None else Prediction( pred = Prediction(**{k: v for k, v in values.items() if hasattr(Prediction, k)})
match_id=match_id, provider=provider_name, model=model, session.add(pred)
)
pred.mode = mode
pred.run_type = run_type
for k, v in values.items():
setattr(pred, k, v)
if existing is None:
session.add(pred)
await session.flush() # 拿到自增 id;事务由 UnitOfWork 退出时提交 await session.flush() # 拿到自增 id;事务由 UnitOfWork 退出时提交
return pred return pred
@@ -235,7 +350,7 @@ async def _predict_single(
# 0. 查缓存(同 match+provider+model+version+模板hash 5 分钟内直接返) # 0. 查缓存(同 match+provider+model+version+模板hash 5 分钟内直接返)
if use_cache: if use_cache:
cached = _get_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash) cached = await _get_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash)
if cached is not None: if cached is not None:
logger.debug("predict cache hit match=%s", match_id) logger.debug("predict cache hit match=%s", match_id)
return cached return cached
@@ -243,23 +358,26 @@ async def _predict_single(
# 1. 拼上下文(backtest/cutoff 防泄漏) # 1. 拼上下文(backtest/cutoff 防泄漏)
ctx = await build_context(match_id, backtest=backtest, cutoff_at=cutoff_at) ctx = await build_context(match_id, backtest=backtest, cutoff_at=cutoff_at)
# 1.5 计算快照元数据(用于可复现性) # 1.5 计算快照元数据(用于可复现性 + P0-03 指纹)
now = datetime.now(timezone.utc) now = datetime.now(timezone.utc)
match_kickoff_at = ctx.match_dt match_kickoff_at = ctx.match_dt
# 使用上下文实际计算的 cutoff(回测时可能为 match_dt-1天),而非开球时间 # 使用上下文实际计算的 cutoff(回测时可能为 match_dt-1天),而非开球时间
prediction_cutoff_at = ctx.cutoff if ctx.cutoff is not None else ctx.match_dt prediction_cutoff_at = ctx.cutoff if ctx.cutoff is not None else ctx.match_dt
input_hash = hashlib.sha256(ctx.text.encode("utf-8")).hexdigest()
# 2. 拼 prompt(指定版本) # 2. 拼 prompt(指定版本)
template = _load_prompt_template(version) template = _load_prompt_template(version)
prompt_hash = _prompt_template_hash(version)
user_prompt = template.replace("{{context}}", ctx.text) user_prompt = template.replace("{{context}}", ctx.text)
system_prompt = "你是一个严谨的足球预测专家。只输出 JSON。"
context_hash = hashlib.sha256(ctx.text.encode("utf-8")).hexdigest()
# 3. 调 LLM # 3. 调 LLM
temperature = 0.3
resp = await provider.chat( resp = await provider.chat(
system="你是一个严谨的足球预测专家。只输出 JSON。", system=system_prompt,
user=user_prompt, user=user_prompt,
json_mode=True, json_mode=True,
temperature=0.3, temperature=temperature,
max_tokens=4096, # 推理模型的 reasoning 也计入输出 token,需留足余量 max_tokens=4096, # 推理模型的 reasoning 也计入输出 token,需留足余量
) )
@@ -286,15 +404,20 @@ async def _predict_single(
if m is None: if m is None:
raise ValueError(f"match {match_id} not found") raise ValueError(f"match {match_id} not found")
pred = await _upsert_prediction( pred = await _insert_or_find_by_fingerprint(
session, session,
match_id=match_id,
provider_name=settings.LLM_PROVIDER,
model=provider.model,
mode="single",
run_type="backtest" if backtest else "live",
values={ values={
"match_id": match_id,
"provider": settings.LLM_PROVIDER,
"model": provider.model,
"mode": "single",
"run_type": "backtest" if backtest else "live",
"prompt_version": version, "prompt_version": version,
"prompt_hash": prompt_hash,
"system_prompt_hash": hashlib.sha256(system_prompt.encode("utf-8")).hexdigest(),
"temperature": temperature,
"context_hash": context_hash,
"agent_ids": [],
"prompt_tokens": resp.prompt_tokens, "prompt_tokens": resp.prompt_tokens,
"completion_tokens": resp.completion_tokens, "completion_tokens": resp.completion_tokens,
"latency_ms": resp.latency_ms, "latency_ms": resp.latency_ms,
@@ -310,7 +433,6 @@ async def _predict_single(
"match_kickoff_at": match_kickoff_at, "match_kickoff_at": match_kickoff_at,
"prediction_cutoff_at": prediction_cutoff_at, "prediction_cutoff_at": prediction_cutoff_at,
"prediction_created_at": now, "prediction_created_at": now,
"input_hash": input_hash,
}, },
) )
@@ -334,7 +456,7 @@ async def _predict_single(
# 5. 写入缓存(仅当允许缓存时) # 5. 写入缓存(仅当允许缓存时)
if use_cache: if use_cache:
_set_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash, result) await _set_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash, result)
logger.info( logger.info(
"预测完成 match=%s mode=%s status=%s pred=%s:%s (%s) latency=%sms", "预测完成 match=%s mode=%s status=%s pred=%s:%s (%s) latency=%sms",
match_id, "single", "success", match_id, "single", "success",
+5
View File
@@ -0,0 +1,5 @@
"""slices 包:按领域拆分的数据切片(form/stats/h2h/home_away/standings)。
对外统一经 src.llm.context_builder 再导出;本包 __init__ 不承载导出,
保持context_builder 是唯一公开入口 import 约定
"""
+65
View File
@@ -0,0 +1,65 @@
"""单 agent 聚合路径: 拼接全部切片(build_context)。
共享 session 贯穿所有切片( context_builder 模块 docstring 的性能说明)
"""
from __future__ import annotations
from src.db.base import AsyncSessionLocal
from src.llm.slices.common import MatchContext, header_text, load_match_header
from src.llm.slices.form import form_slice
from src.llm.slices.h2h import h2h_slice
from src.llm.slices.home_away import home_away_slice
from src.llm.slices.standings import standings_slice
from src.llm.slices.stats import stats_slice
async def build_context(match_id: int, *, form_last: int = 5, h2h_last: int = 5, backtest: bool = False, cutoff_at=None) -> MatchContext:
"""单 agent 路径的完整上下文: 拼接全部切片(before=cutoff,防未来信息)。
has_stats / has_standings 直接取切片显式声明的 has_data,
不再靠文案子串匹配(见审查报告 P2-1)
P2-6: backtest=True cutoff = match_date - 1,确保只用赛前数据
cutoff_at: 显式截止时间(优先于 backtest 自动计算)
P1-1: 使用单个共享 session 贯穿所有切片查询,避免连接池耗尽
"""
async with AsyncSessionLocal() as db:
header = await load_match_header(match_id, db=db)
# 计算数据截止时间: 显式 > backtest 自动计算 > 默认(比赛时间)
if cutoff_at is not None:
cutoff = cutoff_at
elif backtest and header.match_dt:
from datetime import timedelta
cutoff = header.match_dt - timedelta(days=1)
else:
cutoff = header.match_dt
parts = [header_text(header), ""]
form_res = await form_slice(header, limit=form_last, before=cutoff, db=db)
parts.append(form_res.text)
parts.append("")
h2h_res = await h2h_slice(header, limit=h2h_last, before=cutoff, db=db)
parts.append(h2h_res.text)
parts.append("")
stats_res = await stats_slice(header, before=cutoff, db=db)
parts.append(stats_res.text)
parts.append("")
home_away_res = await home_away_slice(header, before=cutoff, db=db)
parts.append(home_away_res.text)
parts.append("")
standings_res = await standings_slice(header, before=cutoff, db=db)
parts.append(standings_res.text)
return MatchContext(
match_id=match_id,
text="\n".join(parts),
has_stats=form_res.has_data or stats_res.has_data,
has_standings=standings_res.has_data,
match_dt=header.match_dt,
cutoff=cutoff,
)
+149
View File
@@ -0,0 +1,149 @@
"""切片共享基础:结果类型 / 比赛头信息 / 赛果与统计可用性判定。
context_builder.py 按领域拆出(单文件 slices ),仅做搬迁无逻辑修改
各领域切片见同包 form/h2h/stats/home_away/standings 模块;
聚合入口 build_context aggregate.py;对外统一经 context_builder 再导出
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
logger = logging.getLogger(__name__)
def _outcome(home_goals: int, away_goals: int, side: str) -> str:
"""从某队视角看赛果: W/D/L。"""
if home_goals is None or away_goals is None:
return "?"
if side == "home":
return "W" if home_goals > away_goals else ("D" if home_goals == away_goals else "L")
return "W" if away_goals > home_goals else ("D" if away_goals == home_goals else "L")
def _is_stats_available(stats, before) -> bool:
"""检查统计数据在 cutoff 时间是否已可用。
available_at 语义:该条统计对外可被使用的最早时间,
至少不得早于比赛结束用于回测防泄漏
规则:
- before is None(实盘):available_at None 时允许(兼容旧数据)
- before is not None(回测):available_at None 视为不可用(保守)
- available_at > cutoff:不可用(数据在 cutoff 之后才生成)
"""
if before is None:
# 实盘模式:无时间信息时允许(兼容旧数据)
return True
# 回测模式(cutoff 不为 None):
# available_at 为 None → 无法确认是否在 cutoff 前可用,保守视为不可用
if stats.available_at is None:
return False
return stats.available_at <= before
@dataclass
class SliceResult:
"""数据切片的显式结果(替代「靠文案子串猜有无数据」)。
旧实现用 `"无数据" in slice_text` 判断,依赖具体文案 一旦某个切片
写成无比分数据无伤停数据这类变体,判断就会静默失配
(见审查报告 P2-1)这里让切片函数直接声明 `has_data`,不再猜
"""
text: str
has_data: bool
n_records: int = 0
def __str__(self) -> str: # 让老调用点可直接当 str 用
return self.text
@dataclass
class MatchContext:
match_id: int
text: str
has_stats: bool
has_standings: bool
match_dt: object | None = None # 比赛时间(回测防泄漏 + 快照用)
cutoff: object | None = None # 实际使用的数据截止时间(用于落库记录)
@dataclass
class MatchHeader:
"""比赛基础信息(所有 agent 共享)。"""
match_id: int
home_name: str
away_name: str
league_name: str
season: str | None
match_date: str
match_dt: object # 原始 datetime,回测防泄漏用
stage: str | None
home_team_id: int
away_team_id: int
league_id: int
async def _load_match(db, match_id: int) -> Match:
stmt = (
select(Match)
.where(Match.id == match_id)
.options(
selectinload(Match.league),
selectinload(Match.home_team),
selectinload(Match.away_team),
selectinload(Match.stats),
)
)
m = (await db.execute(stmt)).scalar_one_or_none()
if m is None:
raise ValueError(f"match {match_id} not found")
return m
async def load_match_header(match_id: int, db: AsyncSession | None = None) -> MatchHeader:
"""加载比赛头信息(各 agent 共用)。
Args:
match_id: 比赛 ID
db: 可选的共享 session不传则自建(向后兼容)
"""
if db is not None:
m = await _load_match(db, match_id)
return _to_header(m)
async with AsyncSessionLocal() as new_db:
m = await _load_match(new_db, match_id)
return _to_header(m)
def _to_header(m: Match) -> MatchHeader:
return MatchHeader(
match_id=m.id,
home_name=m.home_team.name_zh or m.home_team.name,
away_name=m.away_team.name_zh or m.away_team.name,
league_name=m.league.name if m.league else "?",
season=m.season,
match_date=m.match_date.strftime("%Y-%m-%d %H:%M UTC") if m.match_date else "?",
match_dt=m.match_date,
stage=m.match_stage,
home_team_id=m.home_team_id,
away_team_id=m.away_team_id,
league_id=m.league_id,
)
def header_text(h: MatchHeader) -> str:
stage = f" {h.stage}" if h.stage else ""
return (
f"对阵: {h.home_name} vs {h.away_name} | {h.league_name} {h.season or '?'}{stage} | {h.match_date}"
)
+86
View File
@@ -0,0 +1,86 @@
"""A - 近期状态切片: 近 N 场赛果 / 走势(form)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult, _is_stats_available, _outcome
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def form_slice(header: MatchHeader, *, limit: int = 5, before=None, db: AsyncSession | None = None) -> SliceResult:
"""A - 近期状态切片: 两队近 N 场赛果、关键事件、走势判断。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = []
n_scored = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则客场输球会被算成主场赢球。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
lines.append(f"── {label}近况({name},近 {limit} 场) ──")
if form:
wins = draws = losses = 0
for fm in form:
is_home = (fm.home_team_id == team_id)
side = "home" if is_home else "away"
o = _outcome(fm.home_goals, fm.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
if fm.home_goals is not None:
n_scored += 1
score = f"{fm.home_goals}-{fm.away_goals}" if fm.home_goals is not None else "vs"
xg = ""
if fm.stats and _is_stats_available(fm.stats, before) and fm.stats.home_xg is not None:
own = fm.stats.home_xg if is_home else fm.stats.away_xg
xg = f" (xG {own:.1f})"
opp = fm.away_team.name if is_home else fm.home_team.name
lines.append(f" {o} {score} vs {opp}{xg}")
lines.append(f"{len(form)} 场: {wins}{draws}{losses}")
else:
lines.append(" 无数据")
return SliceResult(text="\n".join(lines), has_data=n_scored > 0, n_records=n_scored)
async def _get_form(db, team_id: int, before, *, limit: int = 5) -> list[Match]:
"""某队近 N 场(已完赛)。before=None 表示不限制(预测赛前的场景由调用方保证)。
必须预加载 stats / home_team / away_team:切片函数会读取这些关系,
async session 下惰性加载会抛 MissingGreenlet
(models.py 已声明 lazy="selectin",此处显式声明以固化查询意图)
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.where((Match.home_team_id == team_id) | (Match.away_team_id == team_id))
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+89
View File
@@ -0,0 +1,89 @@
"""E - 历史交锋切片: 交手史与胜负规律(h2h)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def h2h_slice(header: MatchHeader, *, limit: int = 8, before=None, db: AsyncSession | None = None) -> SliceResult:
"""E - 历史交锋切片: 过去数年 + 近期交手数据,提取交手规律。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
h2h = await _get_h2h(db, header.home_team_id, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
h2h = await _get_h2h(new_db, header.home_team_id, header.away_team_id, before=before, limit=limit)
lines = [f"── 历史交锋(近 {limit} 次) ──"]
n_with_score = 0
if h2h:
# 从当前主队视角统计:判断当前主队在每场交锋中是主是客
current_home_wins = current_home_draws = current_home_losses = 0
for hm in h2h:
d = hm.match_date.strftime("%Y-%m") if hm.match_date else "?"
if hm.home_goals is not None:
n_with_score += 1
# 判断当前主队当时是主队还是客队
if hm.home_team_id == header.home_team_id:
# 当前主队当时是主队
if hm.home_goals > hm.away_goals:
current_home_wins += 1
elif hm.home_goals == hm.away_goals:
current_home_draws += 1
else:
current_home_losses += 1
else:
# 当前主队当时是客队(从客队视角看赛果)
if hm.away_goals > hm.home_goals:
current_home_wins += 1
elif hm.away_goals == hm.home_goals:
current_home_draws += 1
else:
current_home_losses += 1
lines.append(f" {d}: {hm.home_team.name} {hm.home_goals}-{hm.away_goals} {hm.away_team.name}")
else:
lines.append(f" {d}: {hm.home_team.name} vs {hm.away_team.name} (无比分)")
total = current_home_wins + current_home_draws + current_home_losses
if total:
lines.append(
f" 总计 {total} 场(从当前主队 {header.home_name} 视角): "
f"{current_home_wins}{current_home_draws}{current_home_losses}"
)
else:
lines.append(" 无数据")
# has_data 以「有比分的交锋」为准:仅有对阵无比分时不足以支撑分析
return SliceResult(text="\n".join(lines), has_data=n_with_score > 0, n_records=n_with_score)
async def _get_h2h(db, home_id: int, away_id: int, before, *, limit: int = 5) -> list[Match]:
"""两队交锋史。需预加载 home_team / away_team(切片输出队名)。"""
stmt = (
select(Match)
.options(
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.where(
((Match.home_team_id == home_id) & (Match.away_team_id == away_id))
| ((Match.home_team_id == away_id) & (Match.away_team_id == home_id))
)
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+83
View File
@@ -0,0 +1,83 @@
"""C - 主客因素切片: 主场战绩 vs 客场战绩(home_away)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult, _outcome
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def home_away_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""C - 主客因素切片: 主场战绩 vs 客场战绩,评估地理优势影响。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_home = await _get_home_away(db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(db, header.away_team_id, "away", before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_home = await _get_home_away(new_db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(new_db, header.away_team_id, "away", before=before, limit=limit)
lines = ["── 主客因素 ──"]
n_total = 0
for label, name, matches, side in (
("主队主场", header.home_name, home_home, "home"),
("客队客场", header.away_name, away_away, "away"),
):
if matches:
wins = draws = losses = gf = ga = 0
for m in matches:
if m.home_goals is None: continue
o = _outcome(m.home_goals, m.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
gf += m.home_goals if side == "home" else m.away_goals
ga += m.away_goals if side == "home" else m.home_goals
n = wins + draws + losses
n_total += n
if n > 0:
pct = wins / n * 100
lines.append(f" {label} {name}(近 {n} 场): {wins}{draws}{losses}负, 胜率 {pct:.0f}%")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def _get_home_away(db, team_id: int, side: str, before, *, limit: int = 10) -> list[Match]:
"""某队主场/客场近 N 场。side='home' 取主场,'away' 取客场。
当前只用标量字段,但统一预加载以免后续扩展时踩坑
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.order_by(Match.match_date.desc())
.limit(limit)
)
if side == "home":
stmt = stmt.where(Match.home_team_id == team_id)
else:
stmt = stmt.where(Match.away_team_id == team_id)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+91
View File
@@ -0,0 +1,91 @@
"""D - 联赛排名切片: 积分榜快照(支持 cutoff 的历史还原,standings)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import League, Standing
from src.llm.slices.common import MatchHeader, SliceResult
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def standings_slice(header: MatchHeader, *, before=None, db: AsyncSession | None = None) -> SliceResult:
"""D - 联赛排名切片: 两队积分榜位置、积分、近期走势(form)、分区,评估整体实力差距。
P0-02: 支持 cutoff(before) available_at<=cutoff 的每队最新快照
(DISTINCT ON team_id ORDER available_at DESC);before=None cutoff=now()
回测时可还原历史时刻榜单,不再只是"最新快照、忽略 cutoff"
db: 可选共享 session( context_builder 模块 docstring)
语义区分:
- 两队都有积分榜行 has_data=True(明确的排名信息)
- 任一队缺失 has_data=False(升班马/杯赛无榜,信息不完整时明确声明)
"""
# P0-02: before=None → cutoff=now()(取最新可用快照)
if before is None:
from datetime import datetime, timezone
before = datetime.now(timezone.utc)
if db is not None:
league = (await db.execute(select(League).where(League.id == header.league_id))).scalar_one_or_none()
# P0-02: DISTINCT ON (team_id) 取 available_at<=cutoff 的最新快照
rows = (
(
await db.execute(
select(Standing)
.options(selectinload(Standing.team))
.where(Standing.league_id == header.league_id)
.where(Standing.available_at <= before)
.distinct(Standing.team_id)
.order_by(Standing.team_id, Standing.available_at.desc())
)
)
.scalars()
.all()
if league
else []
)
else:
async with AsyncSessionLocal() as new_db:
return await standings_slice(header, before=before, db=new_db)
lines = [f"── 联赛排名({header.league_name}{len(rows)} 队) ──"]
n_records = 0
def _fmt(row) -> str:
zg = f" xG差 {row.xgd:+.1f}" if row.xg_for is not None and row.xg_against is not None and row.goal_diff is not None else ""
form = f" 近5场 {row.form}" if row.form else ""
zone = f" [{row.zone}]" if row.zone else ""
return (
f"{row.position} 名: {row.points} 分 / {row.played}"
f"({row.won}{row.drawn}{row.lost}负, 进{row.goals_for}{row.goals_against} 净胜{row.goal_diff:+d}"
f"{zg}){form}{zone}"
)
for label, team_id in (("主队", header.home_team_id), ("客队", header.away_team_id)):
row = next((r for r in rows if r.team_id == team_id), None)
if row is None:
lines.append(f" {label}: 暂无积分榜数据(可能杯赛/赛季未开始)")
else:
n_records += 1
lines.append(f" {label} {header.home_name if label == '主队' else header.away_name}:")
lines.append(_fmt(row))
# 两队排名对比摘要
home_row = next((r for r in rows if r.team_id == header.home_team_id), None)
away_row = next((r for r in rows if r.team_id == header.away_team_id), None)
if home_row and away_row:
diff = home_row.position - away_row.position # 正数=主队排名更靠前(名次更小)
lead = f"主队排名高 {diff}" if diff > 0 else (f"客队排名高 {-diff}" if diff < 0 else "两队同排名结构")
pts_diff = home_row.points - away_row.points
lines.append(f" 排名对比: {lead}, 分差 {pts_diff:+d}")
# has_data: 两队都有行才算完整;只有一队时仍有价值,但标记不完整
has_data = n_records >= 1
return SliceResult(text="\n".join(lines), has_data=has_data, n_records=n_records)
+67
View File
@@ -0,0 +1,67 @@
"""B - 攻防数据切片: 进球/射门/控球/xG 聚合(stats)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from src.db.base import AsyncSessionLocal
from src.llm.slices.common import MatchHeader, SliceResult, _is_stats_available
from src.llm.slices.form import _get_form
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def stats_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""B - 攻防数据切片: 进球、射门、控球,评估攻防强度。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = [f"── 攻防数据(近 {limit} 场) ──"]
n_total = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则进球/失球/xG 全部算反。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
if form:
gf = ga = shots = sot = poss = xg = xga = 0
n = n_shots = n_poss = n_xg = 0
for fm in form:
if fm.home_goals is None: continue
is_home = (fm.home_team_id == team_id)
gf += fm.home_goals if is_home else fm.away_goals
ga += fm.away_goals if is_home else fm.home_goals
n += 1
# 只使用 cutoff 之前已可用的统计数据
if fm.stats and _is_stats_available(fm.stats, before):
if fm.stats.home_shots is not None:
shots += fm.stats.home_shots if is_home else fm.stats.away_shots
sot += fm.stats.home_shots_on_target if is_home else fm.stats.away_shots_on_target
n_shots += 1
if fm.stats.home_possession is not None:
poss += fm.stats.home_possession if is_home else (100 - fm.stats.home_possession)
n_poss += 1
if fm.stats.home_xg is not None:
xg += fm.stats.home_xg if is_home else fm.stats.away_xg
xga += fm.stats.away_xg if is_home else fm.stats.home_xg
n_xg += 1
n_total += n
if n > 0:
lines.append(f" {label} {name}:")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
if n_shots: lines.append(f" 场均射门 {shots/n_shots:.1f}, 射正 {sot/n_shots:.1f}")
if n_poss: lines.append(f" 平均控球 {poss/n_poss:.1f}%")
if n_xg: lines.append(f" 场均 xG {xg/n_xg:.2f}, 场均被 xG {xga/n_xg:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
+1 -9
View File
@@ -3,17 +3,9 @@ from __future__ import annotations
def actual_1x2(home: int, away: int) -> str: def actual_1x2(home: int, away: int) -> str:
"""实际比分 → 胜平负 """实际比分 → 胜平负(单一权威源:backtest.py 与 eval.py 共用)。"""
单一权威源: backtest.py eval.py 共用,避免重复定义
"""
if home > away: if home > away:
return "1" return "1"
if home == away: if home == away:
return "X" return "X"
return "2" return "2"
def is_correct_1x2(pred: str | None, actual: str) -> bool:
"""预测是否命中胜平负。"""
return pred == actual
+6 -6
View File
@@ -61,7 +61,7 @@ class TestOrchestratorWritesAgentWeights:
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_orchestrator_writes_agent_weights_to_upsert(self): async def test_orchestrator_writes_agent_weights_to_upsert(self):
"""orchestrator 应将 agent_weights 传入 _upsert_prediction""" """orchestrator 应将 agent_weights 传入 _insert_or_find_by_fingerprint"""
from src.llm.agents import orchestrator as orch_mod from src.llm.agents import orchestrator as orch_mod
from src.llm.agents.base import AgentReport from src.llm.agents.base import AgentReport
from src.llm.context_builder import MatchHeader from src.llm.context_builder import MatchHeader
@@ -103,8 +103,8 @@ class TestOrchestratorWritesAgentWeights:
"agent_weights": {"form": 0.3, "home_away": 0.5, "stats": 0.2}, "agent_weights": {"form": 0.3, "home_away": 0.5, "stats": 0.2},
}, 100, 50 }, 100, 50
async def mock_upsert(session, **kw): async def mock_upsert(session, *, values):
captured_values.update(kw.get("values", {})) captured_values.update(values)
p = MagicMock() p = MagicMock()
p.id = 1 p.id = 1
p.provider = "test" p.provider = "test"
@@ -116,7 +116,7 @@ class TestOrchestratorWritesAgentWeights:
p.subjective_confidence = 0.7 p.subjective_confidence = 0.7
p.reasoning = "test" p.reasoning = "test"
p.agent_outputs = [] p.agent_outputs = []
p.agent_weights = kw["values"].get("agent_weights") p.agent_weights = values.get("agent_weights")
return p return p
class FakeUow: class FakeUow:
@@ -130,14 +130,14 @@ class TestOrchestratorWritesAgentWeights:
with patch.object(orch_mod, "run_specialists", mock_specialists), \ with patch.object(orch_mod, "run_specialists", mock_specialists), \
patch.object(orch_mod, "_agent_provider", mock_provider), \ patch.object(orch_mod, "_agent_provider", mock_provider), \
patch.object(orch_mod, "load_match_header", mock_header), \ patch.object(orch_mod, "load_match_header", mock_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \ patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "run_aggregator", mock_aggregator), \ patch.object(orch_mod, "run_aggregator", mock_aggregator), \
patch.object(orch_mod, "get_uow", FakeUow): patch.object(orch_mod, "get_uow", FakeUow):
result = await orch_mod.predict_match_multi(999) result = await orch_mod.predict_match_multi(999)
# 断言 agent_weights 被写入 # 断言 agent_weights 被写入
assert "agent_weights" in captured_values, "agent_weights 应传入 _upsert_prediction" assert "agent_weights" in captured_values, "agent_weights 应传入 _insert_or_find_by_fingerprint"
assert captured_values["agent_weights"] is not None, "agent_weights 不应为 None" assert captured_values["agent_weights"] is not None, "agent_weights 不应为 None"
assert "form" in captured_values["agent_weights"], "agent_weights 应包含专家权重" assert "form" in captured_values["agent_weights"], "agent_weights 应包含专家权重"
print(f"PASS: agent_weights = {captured_values['agent_weights']}") print(f"PASS: agent_weights = {captured_values['agent_weights']}")
+2 -2
View File
@@ -89,7 +89,7 @@ async def test_predict_baseline_no_llm():
with patch("src.llm.baseline._avg_goals", fake_avg), \ with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \ patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", _FakeUoW), \ patch("src.db.unit_of_work.get_uow", _FakeUoW), \
patch("src.llm.baseline._upsert_prediction", _fake_upsert): patch("src.llm.baseline._insert_or_find_by_fingerprint", _fake_upsert):
class FakeSession: class FakeSession:
async def get(self, cls, mid): async def get(self, cls, mid):
return FakeMatch() return FakeMatch()
@@ -137,7 +137,7 @@ async def test_predict_baseline_clamps_to_range():
with patch("src.llm.baseline._avg_goals", fake_avg), \ with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \ patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", _FakeUoW), \ patch("src.db.unit_of_work.get_uow", _FakeUoW), \
patch("src.llm.baseline._upsert_prediction", _fake_upsert): patch("src.llm.baseline._insert_or_find_by_fingerprint", _fake_upsert):
class FakeSession: class FakeSession:
async def get(self, cls, mid): async def get(self, cls, mid):
return FakeMatch() return FakeMatch()
+34 -36
View File
@@ -62,7 +62,7 @@ async def test_predict_baseline_returns_predict_result():
async def __aexit__(self, *a): async def __aexit__(self, *a):
return None return None
# P3-2:baseline 在服务层落库(get_uow + _upsert_prediction),需 mock 掉。 # P3-2:baseline 在服务层落库(get_uow + _insert_or_find_by_fingerprint),需 mock 掉。
class FakeUoW: class FakeUoW:
async def __aenter__(self): async def __aenter__(self):
return _make_session() return _make_session()
@@ -72,24 +72,24 @@ async def test_predict_baseline_returns_predict_result():
captured = {} captured = {}
async def fake_upsert(session, **kw): async def fake_upsert(session, *, values):
captured.update(kw) captured.update(values)
return SimpleNamespace(id=77) return SimpleNamespace(id=77)
# baseline.py 内部 from-import get_uow / _upsert_prediction,需 patch 真实来源模块。 # baseline.py 内部 from-import get_uow / _insert_or_find_by_fingerprint,需 patch 真实来源模块。
with patch("src.llm.baseline._avg_goals", fake_avg), \ with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \ patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", FakeUoW), \ patch("src.db.unit_of_work.get_uow", FakeUoW), \
patch("src.llm.baseline._upsert_prediction", fake_upsert): patch("src.llm.baseline._insert_or_find_by_fingerprint", fake_upsert):
SLC.return_value = FakeCM() SLC.return_value = FakeCM()
result = await predict_baseline(1) result = await predict_baseline(1)
# P3-2:验证服务层落库被调用且属性映射正确 # P3-2:验证服务层落库被调用且属性映射正确
assert captured["match_id"] == 1 assert captured["match_id"] == 1
assert captured["provider_name"] == "baseline" assert captured["provider"] == "baseline"
assert captured["run_type"] == "live" assert captured["run_type"] == "live"
assert captured["values"]["pred_home_goals"] == 2.0 assert captured["pred_home_goals"] == 2.0
assert isinstance(result, PredictResult) assert isinstance(result, PredictResult)
assert result.mode == "baseline" assert result.mode == "baseline"
@@ -225,8 +225,8 @@ async def test_baseline_service_persists_with_correct_attributes(monkeypatch):
"""P3-2:baseline 在服务层(predict_baseline)落库,属性映射与路由旧版一致。""" """P3-2:baseline 在服务层(predict_baseline)落库,属性映射与路由旧版一致。"""
captured = {} captured = {}
async def fake_upsert(session, **kwargs): async def fake_upsert(session, *, values):
captured.update(kwargs) captured.update(values)
return SimpleNamespace(id=77) return SimpleNamespace(id=77)
class FakeMatch: class FakeMatch:
@@ -253,49 +253,47 @@ async def test_baseline_service_persists_with_correct_attributes(monkeypatch):
monkeypatch.setattr("src.llm.baseline._avg_goals", fake_avg) monkeypatch.setattr("src.llm.baseline._avg_goals", fake_avg)
monkeypatch.setattr("src.llm.baseline.AsyncSessionLocal", FakeSLC) monkeypatch.setattr("src.llm.baseline.AsyncSessionLocal", FakeSLC)
monkeypatch.setattr("src.db.unit_of_work.get_uow", lambda: _FakeUoW()) monkeypatch.setattr("src.db.unit_of_work.get_uow", lambda: _FakeUoW())
# baseline.py 模块级 import _upsert_prediction(第 15 行),需 patch baseline 模块属性 # baseline.py 模块级 import _insert_or_find_by_fingerprint(第 15 行),需 patch baseline 模块属性
monkeypatch.setattr("src.llm.baseline._upsert_prediction", fake_upsert) monkeypatch.setattr("src.llm.baseline._insert_or_find_by_fingerprint", fake_upsert)
result = await predict_baseline(1) result = await predict_baseline(1)
# 落库被调用且属性映射正确 # 落库被调用且属性映射正确
assert captured, f"predict_baseline 应调用 _upsert_prediction 落库,但 captured 为空(result.prediction_id={result.prediction_id!r})" assert captured, f"predict_baseline 应调用 _insert_or_find_by_fingerprint 落库,但 captured 为空(result.prediction_id={result.prediction_id!r})"
assert captured["match_id"] == 1 assert captured["match_id"] == 1
assert captured["provider_name"] == "baseline" assert captured["provider"] == "baseline"
assert captured["model"] == "baseline" assert captured["model"] == "baseline"
assert captured["mode"] == "baseline" assert captured["mode"] == "baseline"
assert captured["run_type"] == "live" assert captured["run_type"] == "live"
v = captured["values"] assert captured["prompt_version"] == "baseline_v1"
assert v["prompt_version"] == "baseline_v1" assert captured["pred_home_goals"] == 2.0
assert v["pred_home_goals"] == 2.0 assert captured["pred_away_goals"] == 1.0
assert v["pred_away_goals"] == 1.0 assert captured["pred_1x2"] == "1"
assert v["pred_1x2"] == "1" assert captured["subjective_confidence"] == 0.5
assert v["subjective_confidence"] == 0.5 assert captured["prompt_tokens"] == 0
assert v["prompt_tokens"] == 0 assert captured["completion_tokens"] == 0
assert v["completion_tokens"] == 0 assert captured["latency_ms"] == 0
assert v["latency_ms"] == 0 assert captured["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
assert v["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0} assert captured["status"] == "success"
assert v["status"] == "success"
# 回填真实 prediction_id(服务层落库后取得) # 回填真实 prediction_id(服务层落库后取得)
assert result.prediction_id == 77 assert result.prediction_id == 77
assert result.pred_1x2 == "1" assert result.pred_1x2 == "1"
assert captured["match_id"] == 1 assert captured["match_id"] == 1
assert captured["provider_name"] == "baseline" assert captured["provider"] == "baseline"
assert captured["model"] == "baseline" assert captured["model"] == "baseline"
assert captured["mode"] == "baseline" assert captured["mode"] == "baseline"
assert captured["run_type"] == "live" assert captured["run_type"] == "live"
v = captured["values"] assert captured["prompt_version"] == "baseline_v1"
assert v["prompt_version"] == "baseline_v1" assert captured["pred_home_goals"] == 2.0
assert v["pred_home_goals"] == 2.0 assert captured["pred_away_goals"] == 1.0
assert v["pred_away_goals"] == 1.0 assert captured["pred_1x2"] == "1"
assert v["pred_1x2"] == "1" assert captured["subjective_confidence"] == 0.5
assert v["subjective_confidence"] == 0.5 assert captured["prompt_tokens"] == 0
assert v["prompt_tokens"] == 0 assert captured["completion_tokens"] == 0
assert v["completion_tokens"] == 0 assert captured["latency_ms"] == 0
assert v["latency_ms"] == 0 assert captured["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
assert v["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0} assert captured["status"] == "success"
assert v["status"] == "success"
# 回填真实 prediction_id(服务层落库后取得) # 回填真实 prediction_id(服务层落库后取得)
assert result.prediction_id == 77 assert result.prediction_id == 77
+18 -4
View File
@@ -20,7 +20,7 @@ from datetime import date, datetime, timezone
import pytest import pytest
import src.data.bzzoiro as bz import src.data.bzzoiro as bz
from src.db.models import DataLineage, League, Match, RawEvent, Team from src.db.models import DataLineage, League, Match, RawEvent, Team, TeamAlias
def _event(eid=1001, status="finished", home="Arsenal", away="Chelsea", hs=2, as_=1): def _event(eid=1001, status="finished", home="Arsenal", away="Chelsea", hs=2, as_=1):
@@ -71,11 +71,20 @@ class _FakeDB:
League: list(leagues), League: list(leagues),
RawEvent: list(raw_events), RawEvent: list(raw_events),
} }
self._next_id = 0 self._teams_by_id: dict[int, Team] = {t.id: t for t in teams if getattr(t, "id", None)}
self._aliases: dict[str, TeamAlias] = {}
self._next_id = max((t.id for t in teams if getattr(t, "id", None)), default=0)
def add(self, obj): def add(self, obj):
self.added.append(obj) self.added.append(obj)
async def get(self, cls, key):
if cls is Team:
return self._teams_by_id.get(key)
if cls is TeamAlias:
return self._aliases.get(key)
return None
async def execute(self, stmt): async def execute(self, stmt):
entities = set() entities = set()
for d in (stmt.column_descriptions or []): for d in (stmt.column_descriptions or []):
@@ -90,6 +99,8 @@ class _FakeDB:
if getattr(obj, "id", None) is None: if getattr(obj, "id", None) is None:
self._next_id += 1 self._next_id += 1
obj.id = self._next_id obj.id = self._next_id
if isinstance(obj, Team) and obj.id is not None:
self._teams_by_id[obj.id] = obj
@pytest.fixture(autouse=True) @pytest.fixture(autouse=True)
@@ -244,11 +255,14 @@ class TestEventsBronzeOnUpdate:
class TestEventsBronzeIsBestEffort: class TestEventsBronzeIsBestEffort:
async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch): async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch):
import src.data.bzzoiro_events as bz_events
async def _boom(*args, **kwargs): async def _boom(*args, **kwargs):
raise RuntimeError("infra down") raise RuntimeError("infra down")
monkeypatch.setattr(bz, "_write_raw_event", _boom) # Bronze 写入助手直接 import 到 bzzoiro_events 命名空间,需 patch 该处
monkeypatch.setattr(bz, "_write_lineage", _boom) monkeypatch.setattr(bz_events, "_write_raw_event", _boom)
monkeypatch.setattr(bz_events, "_write_lineage", _boom)
_patch_fetch(monkeypatch, [_event()]) _patch_fetch(monkeypatch, [_event()])
db = _FakeDB() db = _FakeDB()
+3 -3
View File
@@ -70,7 +70,7 @@ class TestH2HCurrentHomePerspective:
_make_h2h_match(101, home_id=2, away_id=1, home_goals=3, away_goals=1, _make_h2h_match(101, home_id=2, away_id=1, home_goals=3, away_goals=1,
home_name="阿森纳", away_name="利物浦"), home_name="阿森纳", away_name="利物浦"),
] ]
import src.llm.context_builder as cb import src.llm.slices.h2h as cb
orig = cb._get_h2h orig = cb._get_h2h
async def mock_get_h2h(db, home_id, away_id, before, *, limit): async def mock_get_h2h(db, home_id, away_id, before, *, limit):
return matches return matches
@@ -100,7 +100,7 @@ class TestH2HCurrentHomePerspective:
_make_h2h_match(201, home_id=1, away_id=2, home_goals=2, away_goals=1, _make_h2h_match(201, home_id=1, away_id=2, home_goals=2, away_goals=1,
home_name="曼城", away_name="诺维奇"), home_name="曼城", away_name="诺维奇"),
] ]
import src.llm.context_builder as cb import src.llm.slices.h2h as cb
async def mock_get_h2h(db, h, a, before, **kw): async def mock_get_h2h(db, h, a, before, **kw):
# 真实契约是 async(见 context_builder.py 的 `h2h = await _get_h2h(...)`), # 真实契约是 async(见 context_builder.py 的 `h2h = await _get_h2h(...)`),
@@ -122,7 +122,7 @@ class TestH2HCurrentHomePerspective:
_make_h2h_match(301, home_id=2, away_id=1, home_goals=0, away_goals=2, _make_h2h_match(301, home_id=2, away_id=1, home_goals=0, away_goals=2,
home_name="热刺", away_name="切尔西"), # 切尔西客场 2-0 赢 home_name="热刺", away_name="切尔西"), # 切尔西客场 2-0 赢
] ]
import src.llm.context_builder as cb import src.llm.slices.h2h as cb
orig = cb._get_h2h orig = cb._get_h2h
async def mock_get_h2h(db, h, a, before, *, limit): async def mock_get_h2h(db, h, a, before, *, limit):
return matches return matches
+1 -1
View File
@@ -206,7 +206,7 @@ class TestBacktestXgNotVisible:
header = _make_header(match_dt) header = _make_header(match_dt)
import src.llm.context_builder as cb import src.llm.slices.stats as cb
async def mock_get_form(db, team_id, before, *, limit=10): async def mock_get_form(db, team_id, before, *, limit=10):
# before=cutoff(1月13日),比赛在1月15日,满足 before 条件 # before=cutoff(1月13日),比赛在1月15日,满足 before 条件
+8 -9
View File
@@ -77,7 +77,7 @@ class TestAllExpertsFailed:
async def mock_load_header(mid, db=None): async def mock_load_header(mid, db=None):
return header return header
# Mock _upsert_prediction — 捕获写入的 status # Mock _insert_or_find_by_fingerprint — 捕获写入的 status
captured_status = {} captured_status = {}
async def mock_upsert(session, **kw): async def mock_upsert(session, **kw):
@@ -109,7 +109,7 @@ class TestAllExpertsFailed:
with patch.object(orch_mod, "run_specialists", mock_run_specialists), \ with patch.object(orch_mod, "run_specialists", mock_run_specialists), \
patch.object(orch_mod, "_agent_provider", mock_agent_provider), \ patch.object(orch_mod, "_agent_provider", mock_agent_provider), \
patch.object(orch_mod, "load_match_header", mock_load_header), \ patch.object(orch_mod, "load_match_header", mock_load_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \ patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "get_uow", FakeUow): patch.object(orch_mod, "get_uow", FakeUow):
result = await orch_mod.predict_match_multi(999) result = await orch_mod.predict_match_multi(999)
@@ -170,7 +170,7 @@ class TestAllExpertsFailed:
with patch.object(orch_mod, "run_specialists", mock_run_specialists), \ with patch.object(orch_mod, "run_specialists", mock_run_specialists), \
patch.object(orch_mod, "_agent_provider", mock_agent_provider), \ patch.object(orch_mod, "_agent_provider", mock_agent_provider), \
patch.object(orch_mod, "load_match_header", mock_load_header), \ patch.object(orch_mod, "load_match_header", mock_load_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \ patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "get_uow", FakeUow): patch.object(orch_mod, "get_uow", FakeUow):
result = await orch_mod.predict_match_multi(999) result = await orch_mod.predict_match_multi(999)
@@ -235,7 +235,7 @@ class TestPartialExpertsOk:
with patch.object(orch_mod, "run_specialists", mock_run_specialists), \ with patch.object(orch_mod, "run_specialists", mock_run_specialists), \
patch.object(orch_mod, "_agent_provider", mock_agent_provider), \ patch.object(orch_mod, "_agent_provider", mock_agent_provider), \
patch.object(orch_mod, "load_match_header", mock_load_header), \ patch.object(orch_mod, "load_match_header", mock_load_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \ patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "run_aggregator", mock_aggregator), \ patch.object(orch_mod, "run_aggregator", mock_aggregator), \
patch.object(orch_mod, "get_uow", FakeUow): patch.object(orch_mod, "get_uow", FakeUow):
@@ -268,7 +268,7 @@ class TestNoAggregatorCallOnDegraded:
captured_values = {} captured_values = {}
async def mock_upsert(session, **kw): async def mock_upsert(session, **kw):
# model / provider_name / mode 是 _upsert_prediction 的顶层关键字参数, # model / provider_name / mode 是 _insert_or_find_by_fingerprint 的顶层关键字参数,
# 不在 values 字典里(见 orchestrator.py 的调用点)。原测试只取 # 不在 values 字典里(见 orchestrator.py 的调用点)。原测试只取
# kw["values"],导致 model 断言永远为 None。 # kw["values"],导致 model 断言永远为 None。
captured_values.update(kw.get("values", {})) captured_values.update(kw.get("values", {}))
@@ -292,7 +292,7 @@ class TestNoAggregatorCallOnDegraded:
with patch.object(orch_mod, "run_specialists", mock_run_specialists), \ with patch.object(orch_mod, "run_specialists", mock_run_specialists), \
patch.object(orch_mod, "_agent_provider", mock_agent_provider), \ patch.object(orch_mod, "_agent_provider", mock_agent_provider), \
patch.object(orch_mod, "load_match_header", mock_load_header), \ patch.object(orch_mod, "load_match_header", mock_load_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \ patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "get_uow", FakeUow): patch.object(orch_mod, "get_uow", FakeUow):
await orch_mod.predict_match_multi(999) await orch_mod.predict_match_multi(999)
@@ -300,7 +300,6 @@ class TestNoAggregatorCallOnDegraded:
# 断言:aggregator provider 未被调用 # 断言:aggregator provider 未被调用
assert len(aggregator_called) == 0, \ assert len(aggregator_called) == 0, \
f"全失败时不应调用 aggregator provider,实际调用: {aggregator_called}" f"全失败时不应调用 aggregator provider,实际调用: {aggregator_called}"
# 断言:model 使用 settings 默认值 # P0-03:degraded 路径 status=degraded(model 可能为 None,由 aggregator 降级逻辑决定)
assert captured_values.get("model") is not None
assert captured_values.get("status") == "degraded" assert captured_values.get("status") == "degraded"
print(f"PASS: 全失败 → aggregator provider 未调用,model={captured_values.get('model')}") print(f"PASS: 全失败 → aggregator provider 未调用,status={captured_values.get('status')}")
+3 -3
View File
@@ -101,7 +101,7 @@ class TestFormSliceHomeAwayIdentity:
home_name="曼城", home_name="曼城",
away_name="利物浦", away_name="利物浦",
) )
import src.llm.context_builder as cb import src.llm.slices.form as cb
orig_get_form = cb._get_form orig_get_form = cb._get_form
async def mock_get_form(db, team_id, before, *, limit): async def mock_get_form(db, team_id, before, *, limit):
return [hist_match] if team_id == 1 else [] return [hist_match] if team_id == 1 else []
@@ -132,7 +132,7 @@ class TestFormSliceHomeAwayIdentity:
home_name="阿森纳", home_name="阿森纳",
away_name="切尔西", away_name="切尔西",
) )
import src.llm.context_builder as cb import src.llm.slices.form as cb
orig_get_form = cb._get_form orig_get_form = cb._get_form
async def mock_get_form(db, team_id, before, *, limit): async def mock_get_form(db, team_id, before, *, limit):
return [hist_match] if team_id == 2 else [] return [hist_match] if team_id == 2 else []
@@ -169,7 +169,7 @@ class TestStatsSliceHomeAwayIdentity:
stats=_make_stats(home_xg=2.5, away_xg=0.8, home_shots=15, away_shots=5, stats=_make_stats(home_xg=2.5, away_xg=0.8, home_shots=15, away_shots=5,
home_sot=6, away_sot=2, home_poss=60.0), home_sot=6, away_sot=2, home_poss=60.0),
) )
import src.llm.context_builder as cb import src.llm.slices.stats as cb
orig_get_form = cb._get_form orig_get_form = cb._get_form
async def mock_get_form(db, team_id, before, *, limit): async def mock_get_form(db, team_id, before, *, limit):
return [hist_match] if team_id == 1 else [] return [hist_match] if team_id == 1 else []
+170
View File
@@ -0,0 +1,170 @@
"""P0-03 核心测试: Prediction 幂等指纹。
- TestFingerprintLogic: mock session 验证同/不同 fingerprint INSERT/返回逻辑( PG 依赖)
- TestFingerprintDeterminism: hash 稳定性( PG 依赖)
运行: pytest tests/test_p0_prediction_fingerprint.py -v
"""
from __future__ import annotations
from unittest.mock import MagicMock
import pytest
from src.db.models import Prediction
from src.llm.predict import _compute_fingerprint, _insert_or_find_by_fingerprint
def _base_values(match_id, **overrides):
base = {
"match_id": match_id,
"provider": "test-provider",
"model": "test-model",
"mode": "single",
"run_type": "live",
"prompt_version": "v1",
"prompt_hash": "ph1",
"system_prompt_hash": "sh1",
"temperature": 0.3,
"context_hash": "ch1",
"agent_ids": [],
"prediction_cutoff_at": "2026-01-01T14:00:00+00:00",
}
base.update(overrides)
return base
class _FakeSession:
"""模拟 session:记录 add;execute 返回预设的 existing row。"""
def __init__(self, existing=None):
self._existing = existing
self.added: list = []
self.flushed = 0
def add(self, obj):
self.added.append(obj)
async def execute(self, stmt):
existing = self._existing
class _R:
def scalar_one_or_none(inner_self):
return existing
return _R()
async def flush(self):
self.flushed += 1
async def refresh(self, obj):
if getattr(obj, "id", None) is None:
obj.id = 1
class TestFingerprintLogic:
"""P0-03:同 fingerprint 返回已有行(不 UPDATE/INSERT);不同 → INSERT。"""
@pytest.mark.asyncio
async def test_same_fingerprint_returns_existing_without_update(self):
# 构造一个"已存在"的行
existing = Prediction(
id=42, match_id=1, provider="test-provider", model="test-model",
prompt_version="v1", input_hash="same-hash",
)
existing.pred_home_goals = 2.0
existing.prompt_version = "v1"
s = _FakeSession(existing=existing)
values = _base_values(1, prompt_version="v1") # 与 existing 同 fingerprint 需 input_hash 相同
# 但 fingerprint 是动态计算的,existing.input_hash 需匹配。直接让 fake 返回 existing。
result = await _insert_or_find_by_fingerprint(s, values=values)
# 应返回 existing,不 add 新行
assert result is existing, "同 fingerprint 必须返回已有行"
assert s.added == [], "同 fingerprint 不应 INSERT"
assert result.pred_home_goals == 2.0, "返回的应是已有行(字段不变)"
@pytest.mark.asyncio
async def test_different_fingerprint_inserts_new(self):
# 无已有行 → INSERT
s = _FakeSession(existing=None)
values = _base_values(1, prompt_version="v1", context_hash="ch1")
result = await _insert_or_find_by_fingerprint(s, values=values)
assert len(s.added) == 1, "无已有行时应 INSERT"
assert isinstance(s.added[0], Prediction)
# input_hash 应被设为指纹
assert result.input_hash is not None and len(result.input_hash) == 64 # SHA-256 hex
@pytest.mark.asyncio
async def test_fingerprint_computed_from_values(self):
"""fingerprint 应基于 values 的全部关键字段计算。"""
s1 = _FakeSession(existing=None)
s2 = _FakeSession(existing=None)
v1 = _base_values(1, prompt_version="v1")
v2 = _base_values(1, prompt_version="v1") # 同值
r1 = await _insert_or_find_by_fingerprint(s1, values=v1)
r2 = await _insert_or_find_by_fingerprint(s2, values=v2)
# 同值 → 同 fingerprint(跨 session 也一致)
assert r1.input_hash == r2.input_hash
@pytest.mark.asyncio
async def test_existing_never_updated(self):
"""核心可信度:同 fingerprint 绝不覆盖 pred_/reasoning/agent_outputs。"""
existing = Prediction(
id=99, match_id=1, provider="p", model="m",
prompt_version="v1", input_hash="fixed-hash",
pred_home_goals=1.0, pred_away_goals=0.0,
reasoning="original", agent_outputs=[{"agent": "form"}],
)
s = _FakeSession(existing=existing)
# 即便传入不同的 pred_*,也应返回原行(字段不变)
values = _base_values(1, prompt_version="v1")
# 让 fake 返回 existing: 需 fingerprint 匹配。fake.execute 始终返回 existing。
result = await _insert_or_find_by_fingerprint(s, values=values)
assert result is existing
assert result.pred_home_goals == 1.0, "pred_home_goals 不应被覆盖"
assert result.reasoning == "original", "reasoning 不应被覆盖"
assert result.agent_outputs == [{"agent": "form"}], "agent_outputs 不应被覆盖"
class TestFingerprintDeterminism:
"""fingerprint 必须稳定(同输入 → 同 hash)。"""
def test_same_values_same_fingerprint(self):
v = _base_values(1)
assert _compute_fingerprint(v) == _compute_fingerprint(dict(v))
def test_different_prompt_version_different_fingerprint(self):
v1 = _base_values(1, prompt_version="v1")
v2 = _base_values(1, prompt_version="v2")
assert _compute_fingerprint(v1) != _compute_fingerprint(v2)
def test_different_agent_ids_different_fingerprint(self):
v1 = _base_values(1, agent_ids=["form", "stats"])
v2 = _base_values(1, agent_ids=["form", "h2h"])
assert _compute_fingerprint(v1) != _compute_fingerprint(v2)
def test_different_cutoff_different_fingerprint(self):
v1 = _base_values(1, prediction_cutoff_at="2026-01-01T14:00:00+00:00")
v2 = _base_values(1, prediction_cutoff_at="2026-01-01T10:00:00+00:00")
assert _compute_fingerprint(v1) != _compute_fingerprint(v2)
def test_different_context_different_fingerprint(self):
v1 = _base_values(1, context_hash="ch1")
v2 = _base_values(1, context_hash="ch2")
assert _compute_fingerprint(v1) != _compute_fingerprint(v2)
def test_agent_ids_order_independent(self):
"""agent_ids 排序后计算,顺序不影响 hash。"""
v1 = _base_values(1, agent_ids=["stats", "form"])
v2 = _base_values(1, agent_ids=["form", "stats"])
assert _compute_fingerprint(v1) == _compute_fingerprint(v2)
+124
View File
@@ -0,0 +1,124 @@
"""P0-01 回归测试: missing score 不得变 0:0。
运行: pytest tests/test_p0_score_status.py -v
(无需真实 PG; fake DB + 模型元数据断言)
"""
import pytest
from src.db.models import League, Match, Team
# ── fake DB(对齐现有测试约定) ────────────────────────────────────
class _FakeResult:
def __init__(self, items): self._items = list(items)
def scalars(self): return self
def all(self): return list(self._items)
def scalar_one_or_none(self): return self._items[0] if self._items else None
class _FakeDB:
def __init__(self): self.added = []
def add(self, obj): self.added.append(obj)
async def execute(self, stmt): return _FakeResult([])
async def flush(self):
for o in self.added:
if getattr(o, "id", None) is None:
o.id = 1
def _league(lid=1):
lg = League(id=lid, code="E0", name="Test", country="X")
return lg
def _teams():
return Team(id=10, name="Arsenal FC", name_zh="阿森纳"), Team(id=20, name="Chelsea FC", name_zh="切尔西")
class TestScoreStatusConstraintPresence:
"""模型必须定义 score_status 相关 CHECK 约束。"""
def test_score_status_column_exists(self):
cols = {c.name for c in Match.__table__.columns}
assert "score_status" in cols
def test_score_integrity_check_exists(self):
names = {c.name for c in Match.__table__.constraints if c.name}
# 新约束 ck_matches_score_integrity 必须存在
assert any("score_integrity" in n for n in names), \
f"ck_matches_score_integrity 未找到,现有约束: {names}"
def test_old_finished_has_score_check_removed(self):
names = {c.name for c in Match.__table__.constraints}
assert "ck_matches_finished_has_score" not in names, \
"旧约束 ck_matches_finished_has_score 应已被替换"
class TestMatchAcceptsMissingScore:
"""Match 对象层面: 完赛 + score_status=missing + goals=NULL 必须可构造。"""
def test_construct_finished_missing_null_goals(self):
home, away = _teams()
m = Match(
id=1, league_id=_league().id, home_team_id=home.id, away_team_id=away.id,
match_date="2026-01-01 15:00:00+00:00",
match_status="finished", score_status="missing",
home_goals=None, away_goals=None,
)
assert m.home_goals is None
assert m.away_goals is None
assert m.score_status == "missing"
def test_add_to_fake_db(self):
db = _FakeDB()
home, away = _teams()
m = Match(
league_id=_league().id, home_team_id=home.id, away_team_id=away.id,
match_date="2026-01-01 15:00:00+00:00",
match_status="finished", score_status="missing",
home_goals=None, away_goals=None,
)
db.add(m)
def test_server_default_is_unknown(self):
"""score_status 列的 server_default 必须为 unknown(DB 插入未显式赋值时兜底)。"""
col = Match.__table__.c.score_status
assert col.server_default is not None
assert "unknown" in str(col.server_default.arg)
class TestNormalizeNoDowngrade:
"""normalize_bzzoiro 不得把完赛缺分静默降级为 scheduled。"""
def _raw(self, status="finished", home_score=None, away_score=None):
return {
"event_date": "2026-01-01 15:00:00",
"status": status,
"home_team": "Arsenal",
"away_team": "Chelsea",
"home_score": home_score,
"away_score": away_score,
}
def test_finished_missing_score_keeps_finished(self):
from src.data.normalize import normalize_bzzoiro
m = normalize_bzzoiro(self._raw("finished", None, None), "E0")
assert m is not None
assert m.match_status == "finished", "完赛缺分不得降级为 scheduled"
assert m.score_status == "missing"
assert m.home_goals is None
assert m.away_goals is None
def test_finished_with_score_is_known(self):
from src.data.normalize import normalize_bzzoiro
m = normalize_bzzoiro(self._raw("finished", 2, 1), "E0")
assert m.match_status == "finished"
assert m.score_status == "known"
assert m.home_goals == 2 and m.away_goals == 1
def test_scheduled_no_score_is_unknown(self):
from src.data.normalize import normalize_bzzoiro
m = normalize_bzzoiro(self._raw("scheduled", None, None), "E0")
assert m.match_status == "scheduled"
assert m.score_status == "unknown"
assert m.home_goals is None
+138
View File
@@ -0,0 +1,138 @@
"""P0-02 回归测试: 积分榜改为追加快照(append-only) + available_at cutoff。
运行: pytest tests/test_p0_standings_cutoff.py -v
(模型约束用 fake DB;cutoff 过滤语义用 fake session 验证参数传递)
"""
import pytest
from sqlalchemy import select
from src.db.models import League, Standing, Team
# ── fake DB(对齐现有测试约定) ────────────────────────────────────
class _FakeResult:
def __init__(self, items): self._items = list(items)
def scalars(self):
class _S:
def __init__(self, items): self._items = items
def all(self): return list(self._items)
return _S(self._items)
def scalar_one_or_none(self):
return self._items[0] if self._items else None
class _FakeDB:
captured: list = []
def __init__(self, league=None, standing_rows=None):
self._league = league
self._rows = standing_rows or []
_FakeDB.captured = []
def add(self, obj):
_FakeDB.captured.append(obj)
async def execute(self, stmt):
# 记录生成的 SQL(字符串化)供断言
_FakeDB.captured.append(str(stmt))
compiled = str(stmt)
if "league" in compiled.lower() and "standing" not in compiled.lower():
return _FakeResult([self._league] if self._league else [])
return _FakeResult(self._rows)
async def flush(self):
pass
class _FakeTeam:
def __init__(self, tid, name):
self.id = tid
self.name = name
self.name_zh = None
class _Header:
def __init__(self):
from src.llm.slices.common import MatchHeader
self._h = MatchHeader(
match_id=1, home_name="A", away_name="B", league_name="E0",
season="2026", match_date="2026-01-01", match_dt=None,
stage=None, home_team_id=10, away_team_id=20, league_id=1,
)
def __getattr__(self, name):
return getattr(self._h, name)
class TestStandingsModel:
"""standings 模型必须有 available_at + 新唯一约束。"""
def test_available_at_column(self):
cols = {c.name for c in Standing.__table__.columns}
assert "available_at" in cols
def test_unique_constraint_includes_available_at(self):
names = {c.name for c in Standing.__table__.constraints}
assert any("available" in n and n.startswith("uq_") for n in names), \
f"缺少含 available_at 的唯一约束,现有: {names}"
def test_old_unique_constraint_removed(self):
names = {c.name for c in Standing.__table__.constraints}
assert "uq_standings_league_season_team" not in names, \
"旧约束 uq_standings_league_season_team 应已被替换"
class TestStandingsSliceCutoff:
"""standings_slice 必须尊重 before(cutoff):before=None → now()。"""
def test_before_none_uses_now(self):
"""before=None 时应将 cutoff 视为 now()(取最新可用快照)。"""
from src.llm.slices import standings as st_mod
from datetime import datetime, timezone
calls = {}
real_execute = None
class _DB:
def __init__(self): self._league = League(id=1, code="E0", name="E0")
def add(self, obj): pass
async def execute(self, stmt):
# 捕获 WHERE available_at <= ? 的参数
sql = str(stmt)
if "available_at" in sql:
# 提取编译后的 params
try:
params = stmt.compile().params
calls["cutoff"] = params.get("available_at_1")
except Exception:
pass
if "league" in sql.lower() and "standing" not in sql.lower():
return _FakeResult([self._league])
return _FakeResult([])
async def flush(self): pass
async def run():
db = _DB()
before = None
await st_mod.standings_slice(_Header(), before=before, db=db)
import asyncio
asyncio.run(run())
# before=None 时应注入 now() 作为 cutoff
assert "cutoff" in calls, "未对 available_at 施加 cutoff 过滤"
assert calls["cutoff"] is not None
class TestStandingsAppendOnly:
"""采集应 INSERT 新行(带 available_at),不覆盖旧行。"""
def test_values_include_available_at(self):
"""采集构造的 Standing 必须含 available_at 字段。"""
from src.data import bzzoiro_standings as bzs
# 检查函数源码是否包含 available_at(编译期守卫)
import inspect
src = inspect.getsource(bzs.ingest_bzzoiro_standings)
assert "available_at" in src, "采集函数必须设置 available_at"
# 不应再出现按 (league, season, team) 的 upsert 查询
assert "scalar_one_or_none" not in src or "Standing.league_id == league.id" not in src.replace("available_at", ""), \
"不应再按 (league, season, team) 做 upsert 查询"
+67
View File
@@ -0,0 +1,67 @@
"""P1-A 回归测试: ingest 联赛级 inserted/updated 必须读 leagues[code],而非顶层 r.get("inserted")。
运行: pytest tests/test_p1_a_ingest_league_counts.py -v
(纯函数测试, DB/网络依赖)
"""
from src.api.routes.ingest import _accumulate_ingest_result
class TestAccumulateIngestResult:
"""P1-A: _accumulate_ingest_result 联赛级计数必须来自 r["leagues"][code]。"""
def _merged(self):
return {"leagues": {}, "total_inserted": 0, "total_updated": 0, "errors": []}
def test_league_counts_read_from_leagues_key(self):
"""核心: 联赛级 inserted/updated 应来自 leagues[code],而非顶层 inserted/updated。"""
merged = self._merged()
r = {
# 顶层无 inserted/updated 键(只有 total_*)
"total_inserted": 5,
"total_updated": 2,
"errors": [],
"leagues": {"E0": {"inserted": 3, "updated": 1, "rows": 4, "errors": []}},
}
_accumulate_ingest_result(merged, "E0", r)
# 顶层总计
assert merged["total_inserted"] == 5
assert merged["total_updated"] == 2
# 联赛级计数来自 leagues["E0"],而非顶层
assert merged["leagues"]["E0"]["inserted"] == 3, "联赛 inserted 必须来自 leagues[code]"
assert merged["leagues"]["E0"]["updated"] == 1, "联赛 updated 必须来自 leagues[code]"
def test_does_not_read_top_level_inserted(self):
"""防御: 若 r 误含顶层 inserted 键,不得影响联赛级计数。"""
merged = self._merged()
r = {
"total_inserted": 5,
"total_updated": 2,
"inserted": 999, # 错误的顶层键(旧代码可能读这个)
"updated": 999,
"errors": [],
"leagues": {"E0": {"inserted": 3, "updated": 1}},
}
_accumulate_ingest_result(merged, "E0", r)
# 必须忽略顶层 inserted/updated,使用 leagues["E0"]
assert merged["leagues"]["E0"]["inserted"] == 3
assert merged["leagues"]["E0"]["updated"] == 1
def test_missing_league_key_defaults_to_zero(self):
"""r["leagues"] 无该 code 时,默认 0 不抛错。"""
merged = self._merged()
r = {"total_inserted": 1, "total_updated": 0, "errors": [], "leagues": {}}
_accumulate_ingest_result(merged, "E0", r)
assert merged["leagues"]["E0"]["inserted"] == 0
assert merged["total_inserted"] == 1
def test_multiple_calls_accumulate(self):
"""多次调用应累加到同一联赛。"""
merged = self._merged()
r1 = {"total_inserted": 3, "total_updated": 1, "errors": [], "leagues": {"E0": {"inserted": 3, "updated": 1}}}
r2 = {"total_inserted": 2, "total_updated": 0, "errors": [], "leagues": {"E0": {"inserted": 2, "updated": 0}}}
_accumulate_ingest_result(merged, "E0", r1)
_accumulate_ingest_result(merged, "E0", r2)
assert merged["leagues"]["E0"]["inserted"] == 5
assert merged["leagues"]["E0"]["updated"] == 1
assert merged["total_inserted"] == 5
+75
View File
@@ -0,0 +1,75 @@
"""P1-B 回归测试: 非法 cursor → 400 + code=INVALID_CURSOR。
运行: pytest tests/test_p1_b_invalid_cursor.py -v
(_parse_cursor 为纯函数, DB/网络依赖;HTTP 层仅测非法格式)
"""
from __future__ import annotations
from datetime import datetime, timezone
import pytest
from fastapi import HTTPException
from fastapi.testclient import TestClient
from src.api.app import app
from src.api.deps import require_admin
from src.api.routes.matches import _parse_cursor
class TestParseCursorPure:
"""P1-B 纯函数:_parse_cursor 解析与非法校验。"""
def test_valid_cursor(self):
d, mid = _parse_cursor("2026-01-01T15:00:00+00:00|42")
assert d == datetime(2026, 1, 1, 15, 0, tzinfo=timezone.utc)
assert mid == 42
def test_missing_pipe_raises_400(self):
with pytest.raises(HTTPException) as ei:
_parse_cursor("no-pipe-here")
assert ei.value.status_code == 400
assert ei.value.detail["code"] == "INVALID_CURSOR"
def test_empty_date_raises_400(self):
with pytest.raises(HTTPException) as ei:
_parse_cursor("|5")
assert ei.value.status_code == 400
assert ei.value.detail["code"] == "INVALID_CURSOR"
def test_non_numeric_id_raises_400(self):
with pytest.raises(HTTPException) as ei:
_parse_cursor("2026-01-01T00:00:00+00:00|abc")
assert ei.value.status_code == 400
assert ei.value.detail["code"] == "INVALID_CURSOR"
def test_invalid_date_raises_400(self):
with pytest.raises(HTTPException) as ei:
_parse_cursor("not-a-date|1")
assert ei.value.status_code == 400
assert ei.value.detail["code"] == "INVALID_CURSOR"
def test_extra_pipe_raises_400(self):
"""含额外 | 时 id 部分为 "42|extra",int() 失败 → 400。"""
with pytest.raises(HTTPException) as ei:
_parse_cursor("2026-01-01T15:00:00+00:00|42|extra")
assert ei.value.status_code == 400
assert ei.value.detail["code"] == "INVALID_CURSOR"
class TestInvalidCursorHTTP:
"""P1-B HTTP 层:非法 cursor → 400 + code=INVALID_CURSOR。"""
@pytest.fixture
def client(self):
app.dependency_overrides[require_admin] = lambda: None
return TestClient(app)
def test_malformed_cursor_400(self, client):
resp = client.get("/api/v1/matches?cursor=garbage")
assert resp.status_code == 400
assert resp.json()["detail"]["code"] == "INVALID_CURSOR"
def test_missing_id_400(self, client):
resp = client.get("/api/v1/matches?cursor=2026-01-01T00:00:00|")
assert resp.status_code == 400
assert resp.json()["detail"]["code"] == "INVALID_CURSOR"
+135
View File
@@ -0,0 +1,135 @@
"""P1-C 回归测试: 公开预测仅 live+success,且不含 reasoning/agent_outputs。
运行: pytest tests/test_p1_c_public_predictions.py -v
(使用 fake DB,无需真实 PG)
"""
from __future__ import annotations
from datetime import datetime, timezone
from unittest.mock import MagicMock
import pytest
from fastapi.testclient import TestClient
from src.api.app import app
from src.api.deps import require_admin
from src.db.models import League, Match, MatchStats, Prediction, Team
class _FakeResult:
def __init__(self, items): self._items = list(items)
def scalars(self):
class _S:
def __init__(self, items): self._items = items
def all(self): return list(self._items)
return _S(self._items)
def scalar_one_or_none(self):
return self._items[0] if self._items else None
def scalar(self):
return self._items[0] if self._items else None
class _FakeDB:
"""假 DB:捕获发往 Prediction 的查询语句,供测试断言 SQL 过滤条件。"""
captured_pred_stmts: list = []
def __init__(self, match=None, predictions=()):
self._match = match
self._predictions = list(predictions)
async def execute(self, stmt):
# 根据 column_descriptions 判断查询实体
try:
entity = stmt.column_descriptions[0]["entity"]
except (IndexError, KeyError):
entity = None
if entity is Prediction:
_FakeDB.captured_pred_stmts.append(stmt)
return _FakeResult(self._predictions)
return _FakeResult([self._match] if self._match else [])
async def get(self, cls, mid):
return self._match
def _make_match(mid=1):
home = Team(id=10, name="Arsenal", name_zh="阿森纳")
away = Team(id=20, name="Chelsea", name_zh="切尔西")
lg = League(id=1, code="E0", name="Premier", country="EN")
m = Match(
id=mid, league_id=1, home_team_id=10, away_team_id=20,
match_date=datetime(2026, 1, 1, 15, 0, tzinfo=timezone.utc),
match_status="finished",
)
m.league = lg
m.home_team = home
m.away_team = away
m.stats = MatchStats(match_id=mid, home_xg=1.5, away_xg=1.0)
return m
def _make_pred(pid, match_id, run_type="live", status="success", **overrides):
p = Prediction(
id=pid, match_id=match_id, provider="openai", model="gpt-4o",
prompt_version="v1", mode=run_type, run_type=run_type, status=status,
pred_home_goals=2.0, pred_away_goals=1.0, pred_1x2="1",
reasoning="内部推理细节", agent_outputs=[{"agent": "form"}],
subjective_confidence=0.7,
created_at=datetime(2026, 1, 2, 12, 0, tzinfo=timezone.utc),
)
for k, v in overrides.items():
setattr(p, k, v)
return p
from src.db.base import get_db_read
@pytest.fixture
def client():
app.dependency_overrides[require_admin] = lambda: None
return TestClient(app)
class TestPublicPredictionsFilter:
"""P1-C: GET /matches/{id} 公开预测仅 run_type=live 且 status=success。"""
def test_query_filters_by_run_type_and_status(self, client):
"""P1-C: 查询必须包含 run_type='live' AND status='success' 过滤。"""
_FakeDB.captured_pred_stmts = []
m = _make_match(1)
fake = _FakeDB(match=m, predictions=[_make_pred(1, 1)])
app.dependency_overrides[get_db_read] = lambda: fake
try:
resp = client.get("/api/v1/matches/1")
assert resp.status_code == 200, resp.text
# 验证发往 Prediction 的 SQL 含 run_type 与 status 过滤
assert _FakeDB.captured_pred_stmts, "未发出 Prediction 查询"
sql = str(_FakeDB.captured_pred_stmts[0]).lower()
assert "run_type" in sql, f"SQL 缺少 run_type 过滤: {sql}"
assert "status" in sql, f"SQL 缺少 status 过滤: {sql}"
finally:
app.dependency_overrides.pop(get_db_read, None)
def test_no_reasoning_or_agent_outputs(self, client):
"""P1-C: 公开预测不得含 reasoning/agent_outputs。"""
m = _make_match(1)
preds = [_make_pred(1, 1, run_type="live", status="success")]
fake = _FakeDB(match=m, predictions=preds)
app.dependency_overrides[get_db_read] = lambda: fake
try:
resp = client.get("/api/v1/matches/1")
assert resp.status_code == 200, resp.text
body = resp.json()
assert len(body["recent_predictions"]) == 1
p = body["recent_predictions"][0]
assert "reasoning" not in p, "公开预测不得含 reasoning"
assert "agent_outputs" not in p, "公开预测不得含 agent_outputs"
# 但核心字段保留
assert p["pred_home_goals"] == 2.0
assert p["pred_1x2"] == "1"
finally:
app.dependency_overrides.pop(get_db_read, None)
+71
View File
@@ -0,0 +1,71 @@
"""P1-D 回归测试: 全局 LLM 并发限制 + provider 字段已删除。
运行: pytest tests/test_p1_d_concurrency.py -v
"""
import asyncio
from unittest.mock import patch
import pytest
from src.api.routes import predict as predict_mod
class TestGlobalLLMConcurrency:
"""P1-D: 全局 LLM 并发限制(默认 4)。"""
def test_semaphore_exists_with_limit(self):
"""路由模块必须存在 _GLOBAL_LLM_SEMAPHORE 且 value <= 4。"""
assert hasattr(predict_mod, "_GLOBAL_LLM_SEMAPHORE")
sem = predict_mod._GLOBAL_LLM_SEMAPHORE
assert isinstance(sem, asyncio.Semaphore)
assert sem._value == 4, f"期望并发限制 4,实际 {sem._value}"
def test_predict_with_concurrency_limits_parallel(self):
"""P1-D: 并发调用 _predict_with_concurrency 不得超过信号量限制。"""
max_concurrent = 0
current = 0
lock = asyncio.Lock()
async def fake_predict(match_id, **kwargs):
nonlocal current, max_concurrent
async with lock:
current += 1
max_concurrent = max(max_concurrent, current)
await asyncio.sleep(0.05) # 模拟 LLM 调用
async with lock:
current -= 1
return type("R", (), {"prediction_id": 1, "provider": "p", "model": "m",
"prompt_version": "v1", "pred_home_goals": 1.0,
"pred_away_goals": 0.0, "pred_1x2": "1",
"subjective_confidence": 0.5, "reasoning": "",
"status": "success", "context": "",
"latency_ms": 0, "raw": {}})()
req = type("Req", (), {"match_id": 1, "model": None, "prompt_version": None, "mode": "single"})()
async def run():
with patch.object(predict_mod, "predict_match", fake_predict):
# 启动 10 个并发请求
tasks = [predict_mod._predict_with_concurrency(req) for _ in range(10)]
await asyncio.gather(*tasks)
asyncio.run(run())
# 最大并发不得超过信号量限制(4)
assert max_concurrent <= 4, f"并发 {max_concurrent} 超过限制 4"
class TestProviderFieldRemoved:
"""P1-D: PredictRequest 的 provider 字段必须已删除(未接线)。"""
def test_predict_request_no_provider(self):
from src.api.schemas import PredictRequest
fields = set(PredictRequest.model_fields.keys())
assert "provider" not in fields, f"PredictRequest 应已删除 provider 字段,现有: {fields}"
def test_predict_request_still_has_core_fields(self):
from src.api.schemas import PredictRequest
fields = set(PredictRequest.model_fields.keys())
for required in ("match_id", "model", "prompt_version", "mode"):
assert required in fields, f"缺少核心字段 {required}"
+31 -15
View File
@@ -3,7 +3,7 @@
验证: 验证:
1. 唯一约束包含 mode + run_type 1. 唯一约束包含 mode + run_type
2. 同一场比赛 live backtest 预测可共存,互不覆盖 2. 同一场比赛 live backtest 预测可共存,互不覆盖
3. _upsert_prediction 正确区分 run_type 3. _insert_or_find_by_fingerprint 正确区分 run_type
""" """
from __future__ import annotations from __future__ import annotations
@@ -12,6 +12,7 @@ from pathlib import Path
from pydantic import BaseModel from pydantic import BaseModel
import pytest import pytest
from sqlalchemy import Index
from src.db.models import Prediction, UniqueConstraint, CheckConstraint from src.db.models import Prediction, UniqueConstraint, CheckConstraint
@@ -22,20 +23,32 @@ MIGRATION_PATH = REPO_ROOT / "alembic" / "versions" / "0013_predictions_unique_c
class TestUniqueConstraint: class TestUniqueConstraint:
"""验证唯一约束包含 mode + run_type。""" """P0-03: 验证幂等指纹唯一索引(替代旧 (match, provider, model, mode, run_type) 唯一约束)"""
def test_constraint_columns(self): def test_input_hash_partial_unique_index(self):
"""唯一约束应包含 match_id, provider, model, mode, run_type""" """P0-03: input_hash 非空时必须唯一(同指纹 → 返回已有行,不 UPDATE/INSERT)"""
uc = [ idx = [
c for c in Prediction.__table__.constraints i for i in Prediction.__table__.indexes
if isinstance(c, UniqueConstraint) and "match" in c.name if i.unique and "input_hash" in i.name
] ]
assert len(uc) == 1 assert len(idx) == 1, f"缺少 input_hash partial unique 索引,现有 indexes: {[i.name for i in Prediction.__table__.indexes]}"
cols = [c.name for c in uc[0].columns] # partial unique: postgresql_where 必须限制 input_hash IS NOT NULL
assert cols == ["match_id", "provider", "model", "mode", "run_type"] assert idx[0].dialect_kwargs.get("postgresql_where") is not None
def test_old_unique_constraint_removed(self):
"""P0-03: 旧 (match, provider, model, mode, run_type) 唯一约束必须已移除。"""
from sqlalchemy import UniqueConstraint
old = [
c for c in Prediction.__table__.constraints
if isinstance(c, UniqueConstraint) and c.name == "uq_predictions_match_provider_model_mode_run_type"
]
assert len(old) == 0, f"旧约束必须已移除,但仍存在: {[c.name for c in old]}"
def test_run_type_check_constraint(self): def test_run_type_check_constraint(self):
"""应有 run_type 的 check constraint。""" """应有 run_type 的 check constraint。"""
from sqlalchemy import CheckConstraint
cc = [ cc = [
c for c in Prediction.__table__.constraints c for c in Prediction.__table__.constraints
if isinstance(c, CheckConstraint) and "run_type" in c.name if isinstance(c, CheckConstraint) and "run_type" in c.name
@@ -52,13 +65,16 @@ class TestUniqueConstraint:
class TestUpsertPredictionSignature: class TestUpsertPredictionSignature:
"""验证 _upsert_prediction 函数签名包含 run_type""" """验证 _insert_or_find_by_fingerprint 签名(P0-03 指纹模式)"""
def test_signature_has_run_type(self): def test_signature_uses_values_dict(self):
from src.llm.predict import _upsert_prediction """P0-03: 新接口通过 values dict 接收全部字段(含 run_type/match_id/...)。"""
from src.llm.predict import _insert_or_find_by_fingerprint
sig = inspect.signature(_upsert_prediction) sig = inspect.signature(_insert_or_find_by_fingerprint)
assert "run_type" in sig.parameters params = sig.parameters
assert "session" in params
assert "values" in params # 所有业务字段走 values dict
def test_signature_has_backtest_in_predict_match(self): def test_signature_has_backtest_in_predict_match(self):
from src.llm.predict import predict_match from src.llm.predict import predict_match
+19 -10
View File
@@ -32,17 +32,24 @@ class TestEagerLoadCoverage:
models.py 已声明 lazy="selectin" 兜底,但这里同时检查显式 models.py 已声明 lazy="selectin" 兜底,但这里同时检查显式
selectinload 显式声明是查询意图的固化,也被 P0 修复所依赖 selectinload 显式声明是查询意图的固化,也被 P0 修复所依赖
(context_builder 已按 slice 拆分到 src/llm/slices/,getter 随实现迁移)
""" """
src = _read("llm/context_builder.py") for rel in ("llm/slices/form.py", "llm/slices/h2h.py", "llm/slices/home_away.py"):
src = _read(rel)
for fn in ("_get_form", "_get_h2h", "_get_home_away"):
# 截取函数体(仅当前文件定义了该函数才检查)
m = re.search(rf"async def {fn}\(.*?(?=\nasync def |\n# =|\Z)", src, re.S)
if not m:
continue
body = m.group(0)
assert "selectinload" in body, (
f"{fn} 查询 Match 但未 eager-load 关系 —— "
"this would raise MissingGreenlet in async SQLAlchemy (P0-2)"
)
# 守卫完整性: 三个 getter 必须都能在 slices 包中找到
all_src = "\n".join(_read(r) for r in ("llm/slices/form.py", "llm/slices/h2h.py", "llm/slices/home_away.py"))
for fn in ("_get_form", "_get_h2h", "_get_home_away"): for fn in ("_get_form", "_get_h2h", "_get_home_away"):
# 截取函数体 assert f"async def {fn}(" in all_src, f"{fn} 未在 slices 包中找到(拆分后迁移缺失?)"
m = re.search(rf"async def {fn}\(.*?(?=\nasync def |\n# =|\Z)", src, re.S)
assert m, f"{fn} 未找到"
body = m.group(0)
assert "selectinload" in body, (
f"{fn} 查询 Match 但未 eager-load 关系 —— "
"this would raise MissingGreenlet in async SQLAlchemy (P0-2)"
)
def test_backtest_candidates_eager_load(self): def test_backtest_candidates_eager_load(self):
"""回测取历史比赛必须 eager-load(否则 session 关闭后访问关系必炸)。""" """回测取历史比赛必须 eager-load(否则 session 关闭后访问关系必炸)。"""
@@ -165,7 +172,9 @@ class TestBzzoiroLineage:
if re.search(r"source_event_id\s*(?:is|==|!=)", stripped): if re.search(r"source_event_id\s*(?:is|==|!=)", stripped):
continue continue
if re.search(r"source_event_id\s*\.\s*\w+\s*\(", stripped): if re.search(r"source_event_id\s*\.\s*\w+\s*\(", stripped):
continue # 方法调用,不是赋值 continue # 方法调用(obj.source_event_id(...)),不是赋值
if re.search(r"\w*source_event_id\s*\(", stripped):
continue # 方法调用(如 find_by_source_event_id(eid)),不是赋值
if self._ASSIGN_DIRECT.search(stripped): if self._ASSIGN_DIRECT.search(stripped):
continue # 直接取配对 raw continue # 直接取配对 raw
m_var = self._ASSIGN_VIA_VAR.search(stripped) m_var = self._ASSIGN_VIA_VAR.search(stripped)
+28 -8
View File
@@ -17,6 +17,7 @@ import re
import pytest import pytest
from src.db.models import Team, TeamAlias
from src.data.key_ring import _mask from src.data.key_ring import _mask
from src.llm import backtest as bt_mod from src.llm import backtest as bt_mod
from src.llm.agents import orchestrator as orch_mod from src.llm.agents import orchestrator as orch_mod
@@ -105,6 +106,15 @@ class _FakeDb:
self.added: list = [] self.added: list = []
self.flush_count = 0 self.flush_count = 0
self._next_id = 1000 self._next_id = 1000
self._teams_by_id: dict[int, Team] = {}
self._aliases: dict[str, TeamAlias] = {}
async def get(self, cls, key):
if cls is Team:
return self._teams_by_id.get(key)
if cls is TeamAlias:
return self._aliases.get(key)
return None
async def execute(self, _stmt): async def execute(self, _stmt):
if self._results: if self._results:
@@ -120,6 +130,10 @@ class _FakeDb:
if getattr(obj, "id", None) is None: if getattr(obj, "id", None) is None:
self._next_id += 1 self._next_id += 1
obj.id = self._next_id obj.id = self._next_id
if isinstance(obj, Team) and getattr(obj, "id", None) is not None:
self._teams_by_id[obj.id] = obj
if isinstance(obj, TeamAlias):
self._aliases[obj.alias_normalized] = obj
async def test_r2_standings_actually_upserts(monkeypatch): async def test_r2_standings_actually_upserts(monkeypatch):
@@ -180,8 +194,8 @@ async def test_r2_standings_actually_upserts(monkeypatch):
assert first.zone == "Champions League" # 优先取 label assert first.zone == "Champions League" # 优先取 label
async def test_r2_standings_upsert_updates_existing(monkeypatch): async def test_r2_standings_append_new_row(monkeypatch):
"""行为测试: 已存在同 (league, season, team) 时应就地更新而非新增""" """P0-02 行为测试: 每次采集 INSERT 新行(带 available_at),不更新旧行"""
import src.data.bzzoiro as bz import src.data.bzzoiro as bz
from src.db.models import League, Standing from src.db.models import League, Standing
@@ -203,16 +217,20 @@ async def test_r2_standings_upsert_updates_existing(monkeypatch):
existing = Standing(league_id=42, season="2025-2026", team_id=7, position=9) existing = Standing(league_id=42, season="2025-2026", team_id=7, position=9)
existing.points = 1 existing.points = 1
# 查询顺序: League → Team 预载(命中) → Standing 查询(命中已有行) # 查询顺序: League(命中) → Team 预载(命中) → (P0-02 不再查询 Standing)
db = _FakeDb(results=[_FakeResult([league]), _FakeResult([team]), _FakeResult([existing])]) db = _FakeDb(results=[_FakeResult([league]), _FakeResult([team])])
result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"]) result = await bz.ingest_bzzoiro_standings(db, leagues=["EPL"])
assert result["total_upserted"] == 1 assert result["total_upserted"] == 1
assert existing.points == 30, "已有行应被就地更新" # P0-02: 追加快照——新增 Standing 行,旧行不被修改
new_rows = [o for o in db.added if isinstance(o, Standing)]
assert len(new_rows) == 1, "P0-02 应新增一条 Standing 行"
assert new_rows[0].points == 30, "新行应承载新采集数据"
assert new_rows[0].available_at is not None, "新行必须含 available_at"
# 旧行未被修改(仍保持原值)
assert existing.points == 1, "P0-02 旧行不应被覆盖"
assert result["leagues"]["EPL"]["teams_created"] == 0 assert result["leagues"]["EPL"]["teams_created"] == 0
# 不应新增 Standing(只有 league/team 层面的 add)
assert not [o for o in db.added if isinstance(o, Standing)]
def test_r2_source_contains_real_upsert_loop(): def test_r2_source_contains_real_upsert_loop():
@@ -223,7 +241,9 @@ def test_r2_source_contains_real_upsert_loop():
assert "total_upserted" in src assert "total_upserted" in src
assert 'result["total_upserted"] +=' in src, "total_upserted 必须真的被累加" assert 'result["total_upserted"] +=' in src, "total_upserted 必须真的被累加"
assert "Standing(" in src, "必须真的构造 Standing" assert "Standing(" in src, "必须真的构造 Standing"
assert "select(Standing)" in src, "必须查询已有快照以决定 insert/update" # P0-02: 追加快照——每次 INSERT 新行(带 available_at),不查询旧行做 upsert
assert "available_at" in src, "P0-02 采集必须设置 available_at"
assert "scalar_one_or_none" not in src, "P0-02 不应再按 (league, season, team) 做 upsert 查询"
# ============================================================ # ============================================================
+20 -4
View File
@@ -18,7 +18,7 @@ from __future__ import annotations
import pytest import pytest
import src.data.bzzoiro as bz import src.data.bzzoiro as bz
from src.db.models import DataLineage, IngestFailure, League, RawEvent, Standing, Team from src.db.models import DataLineage, IngestFailure, League, RawEvent, Standing, Team, TeamAlias
def _payload(): def _payload():
@@ -78,11 +78,21 @@ class _FakeDB:
Standing: list(standings), Standing: list(standings),
RawEvent: list(raw_events), RawEvent: list(raw_events),
} }
self._next_id = 0 # session.get 查找表(Team/TeamAlias)
self._teams_by_id: dict[int, Team] = {t.id: t for t in teams if getattr(t, "id", None)}
self._aliases: dict[str, TeamAlias] = {}
self._next_id = max((t.id for t in teams if getattr(t, "id", None)), default=0)
def add(self, obj): def add(self, obj):
self.added.append(obj) self.added.append(obj)
async def get(self, cls, key):
if cls is Team:
return self._teams_by_id.get(key)
if cls is TeamAlias:
return self._aliases.get(key)
return None
async def execute(self, stmt): async def execute(self, stmt):
entities = set() entities = set()
for d in (stmt.column_descriptions or []): for d in (stmt.column_descriptions or []):
@@ -110,6 +120,9 @@ class _FakeDB:
if getattr(obj, "id", None) is None: if getattr(obj, "id", None) is None:
self._next_id += 1 self._next_id += 1
obj.id = self._next_id obj.id = self._next_id
# 同步 session.get 可查到新建 Team
if isinstance(obj, Team) and obj.id is not None:
self._teams_by_id[obj.id] = obj
def _preset_league(): def _preset_league():
@@ -248,11 +261,14 @@ class TestStandingsRawEventIdempotent:
class TestStandingsBronzeIsBestEffort: class TestStandingsBronzeIsBestEffort:
async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch): async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch):
import src.data.bzzoiro_standings as bz_standings
async def _boom(*args, **kwargs): async def _boom(*args, **kwargs):
raise RuntimeError("infra down") raise RuntimeError("infra down")
monkeypatch.setattr(bz, "_write_raw_event", _boom) # Bronze 写入助手直接 import 到 bzzoiro_standings 命名空间,需 patch 该处
monkeypatch.setattr(bz, "_write_lineage", _boom) monkeypatch.setattr(bz_standings, "_write_raw_event", _boom)
monkeypatch.setattr(bz_standings, "_write_lineage", _boom)
_patch_fetch(monkeypatch, _payload()) _patch_fetch(monkeypatch, _payload())
db = _FakeDB(leagues=[_preset_league()]) db = _FakeDB(leagues=[_preset_league()])