Author SHA1 Message Date
shangfangjian d7d903b35b Merge branch 'main' into fix-completeness-refresh 2026-09-22 19:54:39 +08:00
WorkBuddy a6740f6140 fix(admin): 数据完整性页补齐自动刷新(文案承诺但未实现)
页面描述写着「每 5 秒自动刷新」,实际只有挂载时加载一次 + 手动按钮
—— 采集完成后停在页面上的数字永远不动。

- 补 5s 自动轮询(可开关,复用 Logs 页的 checkbox 样式);已有数据时
  刷新不闪骨架(loading && !data 条件已具备)
- 采集页任务终态广播 profeto:ingest-done 事件,完整性页监听后
  立即刷新,不必等下一个 5s 周期
2026-09-22 19:52:39 +08:00
shangfangjian 38e1c6c31f Merge pull request 'fix(quality): 质量检查引用已删除的 MatchStats.id 必然 500 + 检查无日志' (#20) from fix-data-quality-check into main
Reviewed-on: #20
2026-09-22 18:20:49 +08:00
WorkBuddy 0742eef52e fix(quality): 质量检查引用已删除的 MatchStats.id 必然 500 + 检查无日志
生产实测: POST /admin/data-quality/run 100% 失败,且日志无任何记录。

根因链:
1. P0-02 把 MatchStats 主键改为 match_id(无 id 属性),但质量检查
   查询仍写 MatchStats.id.is_(None) → AttributeError → 500
2. handler 无 logger 调用;未捕获异常走 uvicorn.error(默认
   propagate=False),不经过 root handler → 内存日志页/文件日志
   都看不到 traceback,排障无据可查

修复:
- MatchStats.id → MatchStats.match_id(与 P0-02 模型对齐)
- 成功路径记 info 日志(检查名=通过/未通过(计数))
- setup_logging 打开 uvicorn/uvicorn.error 的 propagate,
  未捕获异常 traceback 进入内存缓冲与滚动文件
- 回归测试 3 例(mock session):修复前红(3 failed),修复后绿;
  全量 329 passed 8 skipped 零回归
2026-09-22 18:03:20 +08:00
WorkBuddy f00bf71e8f Merge pull request 'design: 全站设计打磨(语义色 token/弹窗动效与无障碍/报头收敛/交互小项)' (#19) from design-polish into main 2026-09-22 17:46:06 +08:00
WorkBuddy 246b06379d Merge pull request 'fix(admin): 日志页默认定位到最新而非最早' (#18) from fix-logs-order into main 2026-09-22 17:45:50 +08:00
WorkBuddy 3fc3e91bd3 Merge pull request 'fix(admin): /health 被误加 /api/v1 前缀,健康状态永远「系统异常」' (#17) from fix-healthcheck-url into main 2026-09-22 17:45:48 +08:00
WorkBuddy 007a276fb7 Merge pull request 'fix(docker): 预创建 /app/logs 属主,修复文件日志 PermissionError' (#16) from fix-log-volume-permission into main 2026-09-22 17:45:47 +08:00
WorkBuddy 83a1eed283 fix+polish(frontend): 预测弹窗打磨 + 全量语义色替换 + 前台交互小项
预测弹窗:
- 修复模板字符串直写 JSX,反引号原样显示在结果面板的 bug
- 入场动画(遮罩淡入+面板上浮);初始聚焦/Tab 焦点陷阱/关闭还焦;
  打开时锁定背景滚动
- 「关闭窗口即取消」→ 如实描述:前端仅停止进度显示,后台任务
  可能仍在执行并消耗额度

语义色:全部 emerald/amber/rose 裸色替换为 ok/warn/bad token
(Standings 分区与走势、KeyRing、定时任务、任务状态点、
数据完整性进度条、预测命中标记、净胜球)。

前台交互:
- 空态跳后台 a[href] → Link,不再整页刷新
- 联赛 tab 溢出右缘渐隐提示 + aria-current
- 状态行右侧计数组加细线分隔,小屏 wrap 后层级仍清晰
2026-09-22 16:56:19 +08:00
WorkBuddy 9c77efaa4b refactor(frontend): 报头抽组件 Masthead + 回到顶部抽组件 BackTop
- Masthead:统一 HomePage/StandingsLayout 双份复制实现,当前版面
  印报红高亮 + aria-current;⚙ 字符改线条 SVG;管理类入口加 title
  提示需登录,访客不再被无声踢到登录页
- BackTop:两页各一份的浮动按钮收敛为共享组件,去 rounded-full/
  shadow-lg,与全站方角无阴影语言对齐
2026-09-22 16:55:59 +08:00
WorkBuddy b9beddacf7 feat(design): ok/warn/bad 语义色 token + btn-outline + 弹窗动效 keyframes
- tailwind 新增语义状态色三组(纸底文字级 600/700 对比度 ≥4.5:1,AA),
  色相降饱和贴近墨色印刷感;后续替换全部裸 tailwind 原色(emerald/amber/rose)
- 补 btn-outline 定义:此前三处使用但从未定义,样式静默失效
- 弹窗入场动画 keyframes:遮罩淡入 0.18s + 面板上浮 0.2s,仅 opacity/transform
2026-09-22 16:55:58 +08:00
WorkBuddy 77ecb078a3 fix(admin): 日志页改为保持倒序,打开自动定位到顶部(最新)
按用户预期调整方案:列表保持最新在最上面(后端倒序),打开页面/
自动刷新时 scrollTop=0 定位顶部;向下滚动回看历史时暂停定位,
拉回顶部即恢复。替代上一版的时间正序+贴底方案。
2026-09-22 15:39:19 +08:00
WorkBuddy 51fbc1828e fix(admin): 日志页默认定位到最新而非最早
渲染顺序(最新在前)与滚动逻辑假设(正序+贴底=最新)相互矛盾:
自动刷新(默认开启)每 10s 强制贴底,而底部恰好是最早的日志,
用户打开页面被钉在最旧的一条上。

改为渲染时间正序(旧→新),与贴底滚动/上滚暂停跟随的 tail -f
行为一致:打开页面即定位最新。
2026-09-22 15:35:59 +08:00
WorkBuddy 1422afd6f6 fix(admin): /health 请求被误加 /api/v1 前缀,健康状态永远「系统异常」
build_url 对所有路径强制加 /api/v1,但后端 /health 与 /health/ready
挂在根路径(app.py,与 compose healthcheck 一致)→ 前端请求 /api/v1/health
404 → fetchHealth 吞错返回 status=unknown → 右上角/监控页永远「系统异常」。

- build_url 对 /health 与 /health/* 豁免前缀
- http.test.ts 补 URL 豁免用例(该测试此前因 ApiError 参数属性不被
  strip-types 支持而完全无法运行,顺手改为显式赋值使基建可用)
2026-09-22 14:22:51 +08:00
WorkBuddy 7c4a244ce9 fix(docker): 预创建 /app/logs 并赋属主,修复卷挂载点 root 权限拒绝
生产实测: applogs 命名卷挂到 /app/logs 时挂载点由 Docker 以 root:root
创建,而非 root 进程(profeto)写入 app.log 时 PermissionError(Errno 13),
best-effort 降级生效但文件日志实际未启用。

- Dockerfile: 镜像内预创建 /app/logs 并 chown(chown -R 已有,覆盖新目录);
  命名卷为空且首次挂载时 Docker 会复制镜像目录属主,重建镜像即可修复
- log_buffer: PermissionError 单独捕获,warning 给出可操作的排查方向
  (镜像预建目录/chown/重建空卷),其余异常仍保持原通用降级
2026-09-22 14:14:53 +08:00
shangfangjian 432bfee8ad Merge pull request 'feat(logging): 日志持久化落盘(LOG_FILE 滚动文件)' (#14) from log-persistence into main
Reviewed-on: #14
2026-09-22 13:34:46 +08:00
shangfangjian f1c586111f Merge pull request 'admin: 信息架构重组 + Settings tab 化 + 待办驱动 Dashboard + 任务历史' (#15) from admin-ux-overhaul into main
Reviewed-on: #15
2026-09-22 13:34:37 +08:00
WorkBuddy db3c64d49b test: 修复 standings 约束断言的 flaky(匿名约束 name=None)
与 log-persistence 分支同款修复:constraints 集合含匿名约束(name=None),
any() 子串匹配撞上 None 直接 TypeError,集合顺序不定导致间歇失败。
先判真值再匹配。
2026-09-22 12:46:43 +08:00
WorkBuddy d013407aa1 feat(admin): 采集任务历史列表 + 图标语言统一 + 存量样式修复
- dal 新增 fetchIngestJobs(接已有 GET /admin/ingest/jobs,此前前端未用)
- Collection 新增「最近任务」卡:最新 10 条,状态点/任务类型/结果摘要,
  任务终态后自动刷新 —— 补齐采集失败后的回看入口
- 任务选择按钮修复:brand-* 色在 tailwind 中不存在(选中态样式失效),
  改用全站 press/press-wash;去掉与报刊风方角语言不符的 rounded-lg
- 任务图标 🏆📊 → ◈◇▤⇉,与全站几何字符一致,不混用 emoji
- Logs 页描述补充 LOG_FILE 持久化边界说明
2026-09-22 12:44:29 +08:00
WorkBuddy 0ab838258c feat(admin): Settings 分区 tab 化 + Dashboard 改待办驱动
- Settings(496 行五块内容单页)拆为 4 个 tab:数据源/LLM/定时任务/登录认证;
  tab 状态写入 URL(?tab=llm)可深链;数据加载与 API 调用不变
- Dashboard 改待办驱动:死信待处理/联赛数据缺口/可结算预测,
  有待办才显示,每项直达处理页;全部清零时显示状态行
- 三步工作流卡仅在库里无比赛时显示(首次引导,老用户不再占版面)
- 修文案漂移:「数据源健康」卡不再引用已不存在的「数据源」页;
  「运行预测」链接语义修正(预测在前台比赛详情页发起)
- 数据源单源硬编码列表简化为一行;未配 Key 直达设置分区
- <a href> 全改 <Link>,不再整页刷新
2026-09-22 12:44:12 +08:00
WorkBuddy 9cedb874f7 refactor(admin): 导航按心智模型三组重排(数据/预测/系统)
- 预测历史/回测/评估移出「数据流水线」,组成「预测」组(生命周期闭环)
- 「评估与监控」拆开:评估归预测,监控归系统
- 删除 SIDEBAR_GROUP_TITLES 别名映射(分组名与面板名统一)
- 预测历史图标改 target(靶心),数据完整性改 eval(核对文档),不再与仪表盘共用 chart
2026-09-22 12:34:18 +08:00
WorkBuddy 1cdc6411f5 chore(admin): 删除合并后遗留的 4 个死代码页面(~1280 行)
DataSources/LLMConfig/Config 已合并进统一 Settings 页,
Predictions 已被 PredictionHistory 取代,均无任何路由或 import 引用。
2026-09-22 12:33:15 +08:00
WorkBuddy f0c1cc1491 feat(logging): 日志持久化落盘(LOG_FILE 滚动文件)
- LOG_FILE 配置: 空(默认)保持 stdout + Admin 内存日志页(重启清零);
  填路径后额外写 RotatingFileHandler(单文件 10MB × 5 份,utf-8)
- setup_logging 幂等挂载(按 abspath 判重,相对/绝对同文件算一个);
  目录自动创建;文件基础设施失败只 warning 绝不拖垮启动
- docker-compose: api 挂 applogs 卷,默认 LOG_FILE=/app/logs/app.log,
  容器重建日志不丢;.env.example/.gitignore/docs 同步
- tests/test_log_persistence.py: 落盘/幂等/空值禁用/自动建目录/失败降级
- test_p0_standings_cutoff: 匿名约束 name=None 使 any() 子串匹配 TypeError
  (集合顺序不定 → flaky),先判真值再匹配(存量缺陷顺手加固)
2026-09-22 12:18:03 +08:00
shangfangjian 8fef411ced fix: 清理 LLM prompt 中的过时引用(confidence/injuries)
- 全部 agent/match_prediction prompt: deprecated "confidence" → "subjective_confidence"
- aggregator prompt: 移除已删除的 injuries 专家(→standings),修正专家中文名(阵容完整性→联赛排名)

消除预测日志中的 Deprecated field / 未知专家警告。
2026-09-22 11:05:07 +08:00
shangfangjian 197641b9f7 fix(P1-async): 预测改为异步 POST+轮询,避免网关超时(Cloudflare 524)
后端:POST /predict 立即返回 job_id,后台 asyncio.create_task 执行;
新增 GET /predict/jobs/{job_id} 轮询状态(running/success/failed)。

前端: useMatchPredict 改为 POST 拿 job_id → 每 3s 轮询直到终态;
整体超时 5 分钟不变。避免多专家预测 60-180s 触发 Cloudflare 100s 超时(HTTP 524)。
2026-09-22 11:00:40 +08:00
shangfangjian 1a9dc63edd fix: Standing 无 xgd 属性,用 xg_for - xg_against 计算 xG差
standings_slice._fmt 误用 row.xgd(Standing 无此列),预测时报 AttributeError。
修正:直接用 xg_for - xg_against 计算 xG差,去掉对 goal_diff 的冗余判断。
2026-09-22 10:52:26 +08:00
shangfangjian b15678b3f6 fix: http client URL 构造必须统一加 /api/v1 前缀
公开页调用 /matches(无 /api/v1 前缀),被 nginx 当 SPA 回退返回 index.html,
前端 JSON 解析报 "Unexpected token <"。

修复:build_url 函数确保所有相对路径统一走 /api/v1,
已含前缀或绝对 URL 保持不变。

首页加载比赛恢复正常。
2026-09-22 10:44:12 +08:00
shangfangjian 28ecf85da8 fix: MatchStats 主键是 match_id 非 id
scheduler._run_checks 误用 MatchStats.id(MatchStats 无此列,主键为 match_id),
导致数据质量检查定时任务抛 AttributeError。修正为 MatchStats.match_id.is_(None)。
2026-09-22 10:18:00 +08:00
shangfangjian b749b1c621 fix: 前端构建修复 + P3-1 子组件路径校正
- tsconfig 排除 *.test.ts(生产构建不编译测试)
- OutcomePanel TeamSideTag 导入路径 ../../ → ../../../
- AgentsPanel/OutcomePanel/ReasoningPanel types 导入 ../../ → ../
- MatchPredictPanel JSDoc 注释 **. → /** (P3-1 编辑损坏)

前端构建成功,容器 healthy。
2026-09-22 10:09:48 +08:00
shangfangjian 69ec19e646 fix: api healthcheck 改用 python urllib(生产镜像无 curl)
python:3.11-slim 生产镜像不含 curl,原 healthcheck 始终失败;
改为 python urllib.request,与 Dockerfile 非 root 用户兼容。
2026-09-22 09:59:50 +08:00
shangfangjian 5f4075be22 fix(P1 follow-ups): 恢复误删 set_runtime_value + 假 DB 适配 P1-K UPSERT
- 恢复 P1-G 编辑时误删的 clear_runtime_value/set_runtime_value
- 修复 bronze 测试 patch 目标(pipeline_write 替代门面 bz.*)
- 增强假 DB:处理 pg_insert(Insert 语句)+ Team name 回退 + League import

全量 319 通过。
2026-09-22 09:43:48 +08:00
shangfangjian eae88f4cd9 fix(P1-G/H/I/K): 运行时配置解密/Redis JSON/eval summary/UPSERT
P1-G runtime_config:DB 故障回落 env;解密失败在生产环境抛出(ValueError),
  非生产回落;DB 异常与解密异常不再共用 except Exception。
P1-H Redis 预测缓存:去掉 pickle 改用 JSON + dataclasses.asdict,
  datetime 字段 ISO 序列化。
P1-I eval get_eval_summary:默认 limit=None(不冒充全集);新增 run_type(默认 live)
  与 season 过滤;_build_filters 同步扩展。
P1-K Team/League get_or_create:INSERT 改为 PG UPSERT(ON CONFLICT DO NOTHING)
  防并发重复插入。

测试 test_p1_g_runtime_config(4/4) + test_p1_k_upsert(4/4);全量绿。
2026-09-22 09:08:35 +08:00
shangfangjian 2b52478b8f test(P1-F): BIGINT 主键表 INSERT 不带 id 自动生成验证
真 PG 测试(独立 engine,避免事件循环污染):
- data_lineage / raw_event 未指定 id → DB 自动生成递增唯一 id
- 连续 INSERT id 递增且唯一

3/3 通过。
2026-09-22 08:57:20 +08:00
shangfangjian b30ad56319 fix(P1-E): 启动时标 failed 残留 pending/running ingest_jobs
lifespan 加 _fail_stale_ingest_jobs():UPDATE ingest_jobs SET status=failed
WHERE status IN ('pending','running');异常仅记 warning 不阻断启动。
顺手修复 0021(op.text→字符串)/0023(索引名 leason→league)/0024(type_unique→type_)
三个待执行迁移 bug。

测试 test_p1_e_stale_ingest_jobs(2/2 mock);全量 315 通过。
2026-09-22 08:49:30 +08:00
shangfangjian b997c06ede fix(P1-D): 全局 LLM 并发限制 + 删除未接线的 provider 字段
- 删除 PredictRequest.provider(未接线,路由从未传入,符合禁令 #7)
- 新增 _GLOBAL_LLM_SEMAPHORE(默认 4) + _predict_with_concurrency:
  公开 /predict 所有模式汇总受全局并发限制,与 orchestrator 内 match 级
  Semaphore(8) 并存。

测试 test_p1_d_concurrency(4/4);全量 313 通过。
2026-09-22 03:44:09 +08:00
shangfangjian 1ddf697c97 fix(P1-C): 公开预测仅 live+success,且不含 reasoning/agent_outputs
GET /matches/{id}:查询加 WHERE run_type='live' AND status='success';
recent_predictions 不再输出 reasoning/agent_outputs(避免泄露内部推理)。
MatchOut.recent_predictions schema 放宽为 list[dict]。

测试 test_p1_c_public_predictions(2/2);全量 309 通过。
2026-09-22 03:39:36 +08:00
shangfangjian 5ff43d4984 fix(P1-A/P1-B): ingest 联赛级计数修正 + 非法 cursor 400
P1-A: ingest 联赛级 inserted/updated 读 r["leagues"][code] 而非顶层 r.get("inserted");
抽 _accumulate_ingest_result 纯函数 + 合约测试(4/4)。

P1-B: 非法 cursor 不再静默忽略,返回 400 + detail.code=INVALID_CURSOR;
抽 _parse_cursor 纯函数 + 测试(8/8)。

全量 307 通过。
2026-09-22 03:31:40 +08:00
shangfangjian 41cb2edd47 fix(P1-A): ingest 联赛级计数读 leagues[code],抽为纯函数
旧代码读 r.get("inserted")(顶层无此 key)导致联赛级计数总为 0;
改为读 r["leagues"][code].inserted/updated,与顶层 total_* 分离。
抽 _accumulate_ingest_result 纯函数 + 4 合约测试(全绿)。
2026-09-22 03:23:41 +08:00
shangfangjian 64ae8e663a fix(P0-03): Prediction 幂等指纹——只追加,不覆盖
_upsert_prediction 改为 _insert_or_find_by_fingerprint:
- 同 input_hash → 返回已有行(绝不 UPDATE pred_/reasoning/agent_outputs)
- 不同 input_hash → INSERT 新行

input_hash 升级为规范 JSON SHA-256,捕获:match_id, cutoff, prompt_version,
prompt_hash, system_prompt_hash, provider, model, mode, run_type, temperature,
context_hash, agent_ids。移除旧 (match, provider, model, mode, run_type) 唯一约束,
改为 partial unique index(WHERE input_hash IS NOT NULL,兼容旧 NULL 数据)。

三条路径(single/multi/baseline)统一传足指纹字段。
迁移 0024 + 测试 test_p0_prediction_fingerprint(10/10);全量 295 通过。
2026-09-22 03:13:32 +08:00
shangfangjian 49d78136a1 fix(P0-02): 积分榜改为追加快照(append-only) + available_at cutoff
去掉 uq_standings_league_season_team,改为 (league, season, team, available_at) 唯一;
每次采集 INSERT 新行(available_at=now),ON CONFLICT DO NOTHING,不覆盖旧行。

standings_slice(before):DISTINCT ON (team_id) WHERE available_at<=cutoff
ORDER available_at DESC;before=None → cutoff=now()。
公开 list_standings 取每队最新可用快照(子查询 max available_at)。

迁移 0023 + 切片/路由/docs 同步;测试 test_p0_standings_cutoff(5/5)。
284 测试全绿。
2026-09-22 02:45:50 +08:00
shangfangjian 63caa6736c fix(P0-01): missing score 不得变 0:0 — score_status + CHECK 约束
新增 matches.score_status(known/missing/unknown):
- 替换 ck_matches_finished_has_score 为 ck_matches_score_integrity:
  known → 必须有分; missing/unknown → goals 必须 NULL(不伪造 0:0)
- normalize: 完赛缺分不再静默降级为 scheduled,改设 score_status=missing
- events ingest: 创建/更新 Match 同步 score_status(比分由缺变 known / 确认缺分 missing)
- slices(form/h2h/home_away)/backtest: 显式加 score_status='known' 过滤完赛样本
- 迁移 0022 回填现有数据(绝不 UPDATE goals=0)

测试 tests/test_p0_score_status.py(9/9):约束存在性/Match 构造/normalize 行为。
54 相关测试全绿。
2026-09-22 02:32:53 +08:00
shangfangjian f563d5cc99 fix(P0-00): HTTP client 必须传递 method/body,仅 body 存在时设 Content-Type
request() 此前忽略 options.method(默认 GET)与 options.body,导致 POST/PUT/DELETE
全部以 GET 空 body 发出;且无条件设置 Content-Type 误污染无 body 请求。

修复: 显式传递 method(默认 GET) 与 body; 仅当 body 非空时加 Content-Type。

测试 frontend/src/lib/http.test.ts(5/5):GET/POST/PUT/DELETE 的 method/body/headers。
运行: node --experimental-transform-types frontend/src/lib/http.test.ts
2026-09-22 02:24:50 +08:00
shangfangjian 7d2eabf750 chore: 死代码与重复逻辑清理
删除未引用/未调用符号:
- PredictionRepository(无引用)
- is_correct_1x2(无调用者)
- LeagueOut(路由用 list[dict])
- IngestResponse(IngestBzzoiroResponse 已替代)
- SecurityCheckError(从未 raise,assert_security_on_startup 用 sys.exit)
- short_write(仅自引用,全仓库无外部调用)
- fetchIngestJobs(列表函数无页面使用,单数 fetchIngestJob 仍保留)
- clear_prompt_cache(无入口)

去重:
- eval._actual_1x2 改为委托 utils.actual_1x2(单一权威源)

全量测试 270 通过,业务行为不变。
2026-09-22 01:55:52 +08:00
shangfangjian a00364d4a7 perf+unify: matches 列表查询优化 + Standings 统一数据源
matches 列表:selectinload 加 load_only 限定列(League.code/Team.name,name_zh/
MatchStats.home_xg,away_xg),补 stats 加载消除 N+1(m.stats 此前懒加载)。
详情接口保持完整 options;游标分页/响应字段/空值语义不变。

Standings 改用 useLeagues() 统一数据源(API 优先,失败回退本地常量),
LEAGUES 常量扩展 CL/EL;无数据联赛显示虚线 tab + 空态(非隐藏)。
2026-09-22 01:35:06 +08:00
shangfangjian 92f50fa5d4 feat: 采集任务状态跟踪 ingest_jobs
新增 ingest_jobs 表(UUID/task/params/status/result/error/时间戳),
POST /ingest/bzzoiro 启动前插入 job(pending)→ 后台 running → success/failed,
响应新增 job_id(兼容原 message)。

Admin GET /admin/ingest/jobs/{id} 与 /admin/ingest/jobs?limit= 只读查询;
Collection 页提交后轮询 job 至终态,展示真实 result/error 汇总。
迁移 0019_ingest_jobs + 分批 get_uow/BzzoiroSource/IngestFailure 不变。

全量测试 270 通过。
2026-09-22 01:34:55 +08:00
shangfangjian 44816794d3 refactor: context_builder 按 slice 拆到 src/llm/slices/ 包
单文件拆分(仅搬迁无逻辑修改):
- common.py    共享类型/头信息/_outcome/_is_stats_available
- form.py      form_slice + _get_form
- h2h.py       h2h_slice + _get_h2h
- stats.py     stats_slice(复用 form._get_form)
- home_away.py home_away_slice + _get_home_away
- standings.py standings_slice
- aggregate.py build_context

context_builder.py 改为纯 re-export 门面,公开签名不变。
同步修复测试 patch 目标(p0_home_away/h2h_perspective/multi_agent_cutoff)
与 regressions 源码断言(读 slices/*.py)。

全量测试 270 通过。
2026-09-22 01:33:14 +08:00
shangfangjian f1016b610a docs: predict.py docstring 补落库层级表格
明确 single/multi/baseline 均在服务层(_upsert_prediction)落库,
路由层永不写入 predictions,仅做 result → PredictOut 映射。
2026-09-22 01:19:19 +08:00
shangfangjian 03727bda00 refactor: 管线写入依赖解耦——pipeline_write 直接 import
bzzoiro_events / bzzoiro_standings / bzzoiro_stats 直接 import
src.data.pipeline_write(_write_raw_event/_write_lineage/_safe_write_ingest_failure),
删除 bzzoiro.py 门面中的 pipeline_write 转发胶水。

保留 fetch_*/_fetch_json_async/REQUEST_INTERVAL 经 bz. 门面调用(测试 monkeypatch 入口);
测试 best-effort 改为 patch 管线模块自身命名空间(from-import 绑定语义)。
函数语义与「失败不拖垮主流程」不变;source_record_id/transform_name 约定不变。

全量测试 270 通过。
2026-09-22 01:15:45 +08:00
shangfangjian 3a56ee17e1 feat: 预测缓存可选 Redis 后端(PREDICT_CACHE_URL)
PREDICT_CACHE_URL 为空=进程内 LRU+TTL dict(默认,行为不变);
填 redis:// 启用 Redis,失败自动降级内存并 warning,不中断预测。

- _CacheBackend 接口 + _MemoryCache/_RedisCache 两个后端
- 同键格式(predict:{match}:{provider}:{model}:{version}:{tpl_hash[:12}])
- 同 TTL(300s);Redis 用 pickle 序列化
- redis 包未安装/连接失败 → 降级内存;不强制依赖 redis 启动
- clear_prompt_cache 同步清空内存预测缓存

全量测试 270 通过;缓存后端单测 4/4。
2026-09-22 00:53:27 +08:00
shangfangjian 7593b99e39 docs+feat: matches 唯一键语义明确化 + source_event_id partial unique
docs/05-data.md:业务唯一=同联赛同主客同自然天;
source_event_id 用于统计回填与血缘,新增部分唯一索引说明与 upsert 查找顺序。

新增 ix_matches_source_event_id_unique(WHERE IS NOT NULL),
兼容存量空值历史行;MatchRepository.find_by_source_event_id;
events upsert 优先按 event_id 定位,回退自然键。
迁移 0021 + 回归测试修复(find_by_source_event_id 方法调用误判)。
2026-09-22 00:41:54 +08:00
shangfangjian e15b554ba3 feat: 球队实体一致性 — 归一化咽喉 + team_aliases 别名机制
events/standings 创建 Team 前均经 team_names.normalize(已有,确认),
TeamRepository.get_or_create 收敛为归一化唯一咽喉 + info 日志。

新增 team_aliases 表(NFKD 归一别名 → teams.id FK CASCADE),
定位三步链:normalize(name) → teams.name → team_aliases → insert。
不自动合并历史重复队;提供 POST /api/v1/admin/teams/aliases 显式添加。

迁移 0020_team_aliases + Admin 别名管理端点(admin_teams.py)。
全量测试 270 通过。
2026-09-22 00:27:53 +08:00
shangfangjian 4b0d6ee58a chore(P3): baseline 落库下沉 + MatchPredictPanel 拆分 + 多 worker/CSRF 文档
P3-2 baseline 落库从路由下沉到服务层(predict_baseline 内直接落库),
删除路由层 _persist_baseline,三种模式统一 result.prediction_id,对外 JSON 不变。

P3-1 MatchPredictPanel.PredictionPanel 拆为 OutcomePanel/AgentsPanel/ReasoningPanel
三个子组件,本文件保留 PredictModal/PredictProgress/Spinner,对外导出路径不变。

P3-3 docs 加 ⚠️ 多 worker 陷阱红字 + STRICT_SINGLE_WORKER 环境变量(启动期强制拒绝多 worker)。
P3-4 docs 新增「同站部署 vs 跨站 CSRF」节。
2026-09-21 23:29:09 +08:00
shangfangjian 45497d2112 refactor: admin_settings.py 按职责拆分为 4 个模块
681 行单文件拆为(保留 admin_settings 为 include_router 聚合入口):
- admin_datasources  数据源列表/连通性测试/KeyRing/ingest status (5 路由)
- admin_config       settings CRUD + 运行日志 (4 路由)
- admin_llm          LLM agents/models/ping (3 路由)
- admin_quality      stats/data-completeness/data-quality (4 路由)

所有路由仍挂 /api/v1/admin 且带 dependencies=[Depends(require_admin)]。
app.py 注册方式不变(仍 import admin_settings.router)。
2026-09-21 23:28:58 +08:00
shangfangjian 66f0844798 feat: standings Bronze 血缘补齐 + 队名归一 MVP
standings 成功 upsert 后写入 RawEvent(source_record_id=standings:{league}:{season})
+ DataLineage(target_table=standings),与 events/stats 管线对称。

队名归一化收敛到 TeamRepository.get_or_create 唯一咽喉点,
创建新 Team 时 info 日志打出原始名与归一后名;
Admin 新增 GET /api/v1/admin/team-name-duplicates 只读接口,
启发式列出近似重名候选(大小写变体/子串/前缀碰撞),不做自动合并。
2026-09-21 23:28:46 +08:00
shangfangjian 317a5e338a refactor: bzzoiro.py 按管线拆分为 5 个模块
单文件 852 行按职责拆分,保持 BzzoiroSource 与 get_source("bzzoiro") 行为不变:
- bzzoiro_common  HTTP 抓取(多 key 轮换) + 字段转换原语
- bzzoiro_events   fetch_bzzoiro_events + BzzoiroSource.ingest + Bronze 补写
- bzzoiro_standings  standings 管线
- bzzoiro_stats     stats 回填
- pipeline_write    RawEvent/IngestFailure/DataLineage 写入助手

子模块运行期经聚合门面 src.data.bzzoiro 解析可替换协作者,
单文件时代的 bz.* monkeypatch 语义完全保留。
路由 import 已指向新模块(ingest.py / schedules.py)。
2026-09-21 23:27:48 +08:00
120 changed files with 7221 additions and 4500 deletions
+3
View File
@@ -3,6 +3,9 @@
# production 启动时会强制校验:SECRET_KEY 非空且非弱值、鉴权已配置、DB 弱密码阻断。
APP_ENV=development
LOG_LEVEL=INFO
# 日志持久化:空=仅 stdout + Admin 内存日志页(重启清零);填路径则额外写滚动文件(10MB×5)。
# 本地开发示例: LOG_FILE=./logs/app.log (容器内由 compose 默认设为 /app/logs/app.log 并挂卷)
LOG_FILE=
API_PORT=8000
FRONTEND_PORT=3000
+1
View File
@@ -5,6 +5,7 @@ __pycache__/
.pytest_cache/
frontend/node_modules/
frontend/dist/
logs/
# AI 助手上下文文件(不入库)
CLAUDE.md
+8 -1
View File
@@ -15,7 +15,14 @@ RUN pip install --no-cache-dir -r requirements.txt
COPY src ./src
RUN pip install --no-cache-dir .
# 将工作目录所有权移交给非 root 用户
# 日志目录预创建: compose 运行时把 applogs 命名卷挂到 /app/logs,
# 卷挂载点默认由 Docker 以 root:root 创建 —— 镜像内不预建的话,
# 非 root 进程写日志会 PermissionError(Errno 13)。
# 命名卷为空且首次挂载时,Docker 会复制镜像内该目录的内容与属主,
# 因此在这里 mkdir + chown 即可让卷目录归 profeto 所有。
RUN mkdir -p /app/logs
# 将工作目录所有权移交给非 root 用户(含上面的日志目录)
RUN chown -R profeto:profeto /app
EXPOSE 8000
+1 -1
View File
@@ -203,7 +203,7 @@ Profeto/
│ │ ├── config.py # pydantic-settings 配置
│ │ ├── crypto.py # 加密/哈希
│ │ ├── http_client.py # 共享 httpx 客户端
│ │ ├── log_buffer.py # 内存日志缓冲(admin 日志页)
│ │ ├── log_buffer.py # 日志基础设施:内存环形缓冲(admin 日志页) + 可选文件持久化(LOG_FILE)
│ │ ├── runtime_config.py # DB 配置覆盖(.env → app_settings)
│ │ ├── scheduler.py # 进程内 cron 调度器
│ │ └── security_check.py # 启动安全校验
+42
View File
@@ -0,0 +1,42 @@
"""采集任务状态表 ingest_jobs
Revision ID: 0019_ingest_jobs
Revises: 0018_match_checks
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0019_ingest_jobs'
down_revision: Union[str, None] = '0018_match_checks'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
'ingest_jobs',
sa.Column('id', sa.String(36), primary_key=True),
sa.Column('task', sa.String(20), nullable=False),
sa.Column('params', sa.JSON(), nullable=False, server_default='{}'),
sa.Column('status', sa.String(20), nullable=False, server_default='pending'),
sa.Column('result', sa.JSON(), nullable=True),
sa.Column('error', sa.Text(), nullable=True),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now()),
sa.Column('started_at', sa.DateTime(timezone=True), nullable=True),
sa.Column('finished_at', sa.DateTime(timezone=True), nullable=True),
)
op.create_index('ix_ingest_job_status_created', 'ingest_jobs', ['status', 'created_at'])
op.create_check_constraint(
'ck_ingest_job_status', 'ingest_jobs',
"status IN ('pending', 'running', 'success', 'failed')",
)
def downgrade() -> None:
op.drop_constraint('ck_ingest_job_status', 'ingest_jobs', type_='check')
op.drop_index('ix_ingest_job_status_created', table_name='ingest_jobs')
op.drop_table('ingest_jobs')
+32
View File
@@ -0,0 +1,32 @@
"""球队别名表 team_aliases
Revision ID: 0020_team_aliases
Revises: 0019_ingest_jobs
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0020_team_aliases'
down_revision: Union[str, None] = '0019_ingest_jobs'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_table(
'team_aliases',
sa.Column('alias_normalized', sa.String(120), primary_key=True),
sa.Column('team_id', sa.Integer, sa.ForeignKey('teams.id', ondelete='CASCADE'), nullable=False),
sa.Column('original_alias', sa.String(120), nullable=False),
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.func.now()),
)
op.create_index('ix_team_aliases_team_id', 'team_aliases', ['team_id'])
def downgrade() -> None:
op.drop_index('ix_team_aliases_team_id', table_name='team_aliases')
op.drop_table('team_aliases')
@@ -0,0 +1,38 @@
"""matches.source_event_id 部分唯一索引
业务唯一键:同联赛同主客同自然天一条(ix_matches_unique,既有)。
source_event_id 是上游 bzzoiro 的比赛 id,用于统计回填与血缘追踪;
当它非空时应全局唯一(同一 upstream 比赛只对应一行 matches),
避免同一场比赛因自然键天级舍入差异产生重复。
partial unique(WHERE source_event_id IS NOT NULL):
- 兼容存量空 source_event_id 的历史行(不强制回填);
- 新采集行均带 source_event_id,从此具备 upstream 唯一性。
Revision ID: 0021_match_source_event_id_unique
Revises: 0020_team_aliases
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
revision: str = '0021_match_source_event_id_unique'
down_revision: Union[str, None] = '0020_team_aliases'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
op.create_index(
'ix_matches_source_event_id_unique',
'matches',
['source_event_id'],
unique=True,
postgresql_where='source_event_id IS NOT NULL',
)
def downgrade() -> None:
op.drop_index('ix_matches_source_event_id_unique', table_name='matches')
@@ -0,0 +1,63 @@
"""P0-01: 比分可信度——score_status + 允许完赛缺分(NULL,禁止伪造 0:0)
替换 ck_matches_finished_has_score:引入 score_status(known/missing/unknown),
完赛 + score_status=missing 时 home/away_goals 必须 NULL(不伪造比分)。
Revision ID: 0022_match_score_status
Revises: 0021_match_source_event_id_unique
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0022_match_score_status'
down_revision: Union[str, None] = '0021_match_source_event_id_unique'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 1) 新增 score_status 列(默认 unknown)
op.add_column(
'matches',
sa.Column('score_status', sa.String(20), server_default='unknown', nullable=False),
)
# 2) 按现有数据回填 score_status(绝不写 goals=0):
# - 有比分(两列均非 NULL) → known
# - 无比分 + 完赛 → missing(缺分)
# - 其余 → unknown
op.execute(
"UPDATE matches SET score_status = 'known'"
" WHERE home_goals IS NOT NULL AND away_goals IS NOT NULL"
)
op.execute(
"UPDATE matches SET score_status = 'missing'"
" WHERE match_status = 'finished' AND home_goals IS NULL AND away_goals IS NULL"
)
# 3) 删除旧约束,加新约束
op.drop_constraint('ck_matches_finished_has_score', 'matches', type_='check')
op.create_check_constraint(
'ck_matches_score_status_enum', 'matches',
"score_status IN ('known', 'missing', 'unknown')",
)
op.create_check_constraint(
'ck_matches_score_integrity', 'matches',
"match_status <> 'finished'"
" OR (score_status = 'known' AND home_goals IS NOT NULL AND away_goals IS NOT NULL)"
" OR (score_status IN ('missing', 'unknown') AND home_goals IS NULL AND away_goals IS NULL)",
)
def downgrade() -> None:
op.drop_constraint('ck_matches_score_integrity', 'matches', type_='check')
op.drop_constraint('ck_matches_score_status_enum', 'matches', type_='check')
op.create_check_constraint(
'ck_matches_finished_has_score', 'matches',
"match_status <> 'finished' OR (home_goals IS NOT NULL AND away_goals IS NOT NULL)",
)
op.remove_column('matches', 'score_status')
@@ -0,0 +1,59 @@
"""P0-02: 积分榜改为追加快照(append-only) + available_at
去掉 uq_standings_league_season_team(league,season,team 唯一),
改为 (league, season, team, available_at) 唯一;
每次采集 INSERT 新行(available_at=now),支持回测还原历史榜单。
Revision ID: 0023_standings_append_only
Revises: 0022_match_score_status
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0023_standings_append_only'
down_revision: Union[str, None] = '0022_match_score_status'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
from datetime import datetime, timezone
# 1) 加 available_at 列(非空,默认 now;存量回填 retrieved_at 或 now)
op.add_column(
'standings',
sa.Column('available_at', sa.DateTime(timezone=True), nullable=False,
server_default=sa.func.now()),
)
# 存量行: available_at 取 retrieved_at(若存在)否则 now
op.execute("UPDATE standings SET available_at = COALESCE(retrieved_at, NOW())")
# 2) 去旧唯一约束,加新唯一约束(league, season, team, available_at)
op.drop_constraint('uq_standings_league_season_team', 'standings', type_='unique')
# 原始索引名拼写为 leason(历史遗留),按实际库名删除
op.drop_index('ix_standings_league_season_pos', table_name='standings', if_exists=True)
op.create_index('ix_standings_league_season_pos_v2', 'standings', ['league_id', 'season', 'position'])
op.create_unique_constraint(
'uq_standings_league_season_team_available', 'standings',
['league_id', 'season', 'team_id', 'available_at'],
)
op.create_index(
'ix_standings_league_season_team_available', 'standings',
['league_id', 'season', 'team_id', 'available_at'],
)
def downgrade() -> None:
op.drop_index('ix_standings_league_season_team_available', table_name='standings')
op.drop_constraint('uq_standings_league_season_team_available', 'standings', type_='unique')
op.drop_index('ix_standings_league_season_pos', table_name='standings')
op.create_index('ix_standings_leason_season_pos', 'standings', ['league_id', 'season', 'position'])
op.create_unique_constraint(
'uq_standings_league_season_team', 'standings',
['league_id', 'season', 'team_id'],
)
op.drop_column('standings', 'available_at')
@@ -0,0 +1,41 @@
"""P0-03: Prediction 幂等指纹——移除旧唯一约束,改为 partial unique on input_hash
input_hash 非空时唯一(同指纹返回已有行,不 UPDATE/INSERT);
兼容旧数据 NULL input_hash(不强制回填)。
Revision ID: 0024_prediction_idempotent_fingerprint
Revises: 0023_standings_append_only
Create Date: 2026-09-22
"""
from typing import Sequence, Union
from alembic import op
import sqlalchemy as sa
revision: str = '0024_prediction_idempotent_fingerprint'
down_revision: Union[str, None] = '0023_standings_append_only'
branch_labels: Union[str, Sequence[str], None] = None
depends_on: Union[str, Sequence[str], None] = None
def upgrade() -> None:
# 移除旧唯一约束(match, provider, model, mode, run_type)
op.drop_constraint(
'uq_predictions_match_provider_model_mode_run_type',
'predictions', type_='unique',
)
# P0-03: partial unique on input_hash(非空时唯一)
op.create_index(
'ix_predictions_input_hash_unique', 'predictions', ['input_hash'], unique=True,
postgresql_where=sa.text('input_hash IS NOT NULL'),
)
def downgrade() -> None:
op.drop_index('ix_predictions_input_hash_unique', table_name='predictions')
op.create_unique_constraint(
'uq_predictions_match_provider_model_mode_run_type',
'predictions',
['match_id', 'provider', 'model', 'mode', 'run_type'],
)
+6 -2
View File
@@ -28,13 +28,15 @@ services:
# Fix 2: 容器内 DATABASE_URL 使用 postgres 服务名(非 localhost)
# 必须覆盖 .env 中的 DATABASE_URL,因为 Settings 不读 DB_HOST/DB_PORT
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:?POSTGRES_USER 未设置}:${POSTGRES_PASSWORD:?POSTGRES_PASSWORD 未设置}@postgres:5432/${POSTGRES_DB:-football}
# 日志持久化:写入挂载卷,容器重建不丢(应用内滚动 10MB×6 份封顶)
LOG_FILE: /app/logs/app.log
env_file: .env
healthcheck:
test: ["CMD-SHELL", "curl -sf http://localhost:8000/health/ready || exit 1"]
test: ["CMD-SHELL", "python -c \"import urllib.request; exit(0 if urllib.request.urlopen('http://localhost:8000/health/ready', timeout=5).status==200 else 1)\" || exit 1"]
interval: 10s
timeout: 5s
retries: 3
start_period: 10s
start_period: 15s
depends_on:
postgres:
condition: service_healthy
@@ -42,6 +44,7 @@ services:
- ./src:/app/src
- ./alembic:/app/alembic
- ./alembic.ini:/app/alembic.ini
- applogs:/app/logs
frontend:
# Fix 4: 多阶段构建 —— 先 build 静态文件,再复制到 nginx
@@ -57,3 +60,4 @@ services:
volumes:
pgdata:
applogs:
+76 -6
View File
@@ -60,9 +60,65 @@
### 队名归一化
`src/data/team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
`src/data.team_names.py` 维护 `NORMALIZE_MAP`(如 `Man City` → `Manchester City`),未命中映射的队名原样返回。
归一前先做 Unicode NFKD 去重音。
**唯一键是归一后英文名**:`teams.name` 带 `UNIQUE` 约束,所有入库路径均经 `TeamRepository.get_or_create` 收敛归一化
(events / standings 管线在调用前归一,仓库层再做一次幂等归一作为兜底)。创建新 Team 时打 `info` 日志记录「原始名 → 归一后名」。
> ⚠️ **`normalize` 当前大小写敏感**:仅当入参大小写与 `NORMALIZE_MAP` 键完全匹配时才触发映射
>(如 `"Man City"` → `"Manchester City"`,但 `"man city"` 原样保留)。上游 bzzoiro 返回的队名首字母大写,
>实际命中无问题;若新增数据源返回全小写/全大写队名,需先 `title()` 再归一,否则会绕过映射产生重复 Team。
#### 别名机制(`team_aliases`)
归一仍可能遗漏历史重复队(如 `"Bayern Munich"` 与 `"Bayern München"` 经 NFKD 后相同则命中,
但 `"Man United"` vs `"Manchester United"` 若漏映射)。`team_aliases` 表提供**显式别名→teams.id** 映射:
| 列 | 说明 |
|----|------|
| `alias_normalized` | PK,`normalize(别名)` 后的稳定幂等键 |
| `team_id` | FK → `teams.id`(ON DELETE CASCADE) |
| `original_alias` | 原始写法(保留供参考) |
**定位三步链**(`get_or_create`):`normalize(name)` → 查 `teams.name` → 查 `team_aliases`(以 `normalize(name)` 为 PK)→ 都没有才 insert 新 Team。别名命中即复用已有 Team,避免产生重复。
**添加别名**(不自动合并历史重复队):
- **Admin 接口**(推荐):`POST /api/v1/admin/teams/aliases {"alias": "Man United", "team_id": 42}`(require_admin,幂等)
- **直接 SQL**:
```sql
INSERT INTO team_aliases(alias_normalized, team_id, original_alias)
VALUES ('man united', 42, 'Man United')
ON CONFLICT (alias_normalized) DO UPDATE SET team_id = EXCLUDED.team_id, original_alias = EXCLUDED.original_alias;
```
> ⚠️ **别名不自动合并**:发现历史重复队 A/B 后,需人工确认归一目标(如保留 B),再为 A 的归一名添加别名指向 B。
> 合并前请确认 A 的 `matches`/`standings` 引用是否需要迁移(可先 `SELECT COUNT(*) FROM matches WHERE home_team_id = A.id OR away_team_id = A.id` 评估)。
**改名 / 合并流程**(人工):
当发现两个 `teams` 行实际是同一球队(如 `Manchester City` 与 `Man City` 因历史数据大小写差异各占一行):
1. 确定**保留行**(通常选归一后规范名、且被更多 Match 引用的那行)。
2. 将被删行的所有引用指向保留行(`UPDATE matches SET home_team_id = 保留id WHERE home_team_id = 删行id`,客场同理;
`standings` / `match_stats` 按 `team_id` 同理)。
3. 删掉多余行:`DELETE FROM teams WHERE id = 删行id`。
> 此过程引入外键约束风险,务必在事务中执行并先 `BEGIN; ... ` 验证行数后再 `COMMIT`。
> 暂不做自动合并(避免误合相似名),仅通过下方 Admin 接口列出「近似重名」候选,由人工判定。
## Admin:近似重名候选
`GET /api/v1/admin/team-name-duplicates` 只读列出启发式相似候选(大小写差异、子串包含、前缀碰撞),不做自动合并。
典型用途:定期巡检,发现候选后走上方人工 SQL 合并。启发式规则:
- **大小写变体**:`lower(name)` 相同但 `name` 不同(如 `Arsenal FC` / `arsenal fc`)。
- **子串包含**:A 是 B 的子串且长度 ≥ 5(如 `Manchester` / `Manchester City`)。
- **前缀碰撞**:前 8 个字符相同的两队。
命中任一规则即列为候选,按相似度分组返回。
## 数据库 Schema
12 张表:核心业务表 5 张见下方 DDL,其余 7 张(积分榜/配置/调度/治理)见后文表格。
@@ -150,7 +206,7 @@ CREATE TABLE predictions (
| 表 | 状态 | 用途 |
|---|---|---|
| `standings` | 已启用 | 联赛积分榜快照,按 `(league_id, season, team_id)` upsert,同联赛同赛季只保留最新快照;含排名/战绩/进失球/积分/分区(zone) |
| `standings` | 已启用 | 联赛积分榜追加快照(P0-02):每次采集 INSERT 新行(available_at=now),唯一键 `(league_id, season, team_id, available_at)`;查询取每队 available_at 最新快照,支持回测还原历史榜单。含排名/战绩/进失球/积分/分区(zone) |
| `app_settings` | 已启用 | 后台运行时设置(如数据源 API Key),读取时优先于 `.env` 默认值 |
| `schedules` | 已启用 | 定时采集任务配置(task/cron/leagues/enabled),供内置调度器执行 |
| `raw_events` | 预留未启用 | Bronze 层原始事件存档;规划中用于重放与审计 |
@@ -162,11 +218,25 @@ CREATE TABLE predictions (
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。**业务唯一:同联赛同主客同自然天一条。**
3. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测。
3. **`source_event_id` 部分唯一**: `ix_matches_source_event_id_unique`(WHERE source_event_id IS NOT NULL)——上游 bzzoiro 的比赛 id,当非空时全局唯一。作用:
- 统计回填(`/events/{id}/stats/`)与 Bronze 血缘(/events/ 采集)通过它定位比赛,不依赖自然键天级舍入;
- 新采集行均带此 id,避免同一 upstream 比赛因时间戳差异绕开自然键产生重复。
- 存量空 source_event_id 历史行不受影响(不强制回填)。
4. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比
4. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测
5. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比。
## 采集 upsert 查找顺序
events 管线按以下优先级定位已有比赛,命中即复用(更新):
1. **`source_event_id`**(upstream event id,唯一索引命中)——最精确,跨自然键舍入差异;
2. **自然键**:`(league_id, home_team_id, away_team_id, match_date_date)`(内存去重,覆盖无 event id 的采集)。
两者都未命中 → insert 新比赛。
## 入库语义(幂等)
@@ -180,7 +250,7 @@ CREATE TABLE predictions (
`task=stats` 只回填统计(xG/射门/控球等,也只补空),不创建比赛。
`task=standings` 按 `(league_id, season, team_id)` upsert 积分榜快照,同一联赛同一赛季只保留最新一份
`task=standings` 追加快照(available_at=now,ON CONFLICT DO NOTHING);公开接口与切片均取每队 available_at 最新快照,支持回测还原历史榜单
## 采集建议
+23
View File
@@ -46,6 +46,9 @@ curl http://localhost:8000/health
- [ ] **6. 反代信任头**`TRUST_PROXY_HEADERS=True`,且**仅可信反代可达 API**;反代需设置 `X-Forwarded-For`(`$proxy_add_x_forwarded_for`)与 `X-Real-IP`,否则限流/日志按反代 IP 计数
- [ ] **7. 限流前置到网关** — 推荐 Nginx `limit_req`(配置见[安全与限流](#安全与限流));应用内限流与 KeyRing 为**单进程内存实现**,多 worker 各自独立计数会把实际配额放大 N 倍(启动时会打印一次性告警)
- [ ] **8. uvicorn 单 worker** — compose/Dockerfile 默认单 worker,保持即可;需横向扩容时先在网关统一限流,再起多实例(每实例仍单 worker)
> ⚠️ **多 worker 陷阱**:应用内限流(`_RateLimiter`)与 KeyRing 均为**进程内纯内存状态**,多 worker 部署(如 `uvicorn --workers 4`)时各进程**各自独立计数、互不共享**——实际限流配额会被放大 N 倍、KeyRing 限流状态也不同步。
> 若确需多 worker,必须前置 Nginx/网关做**全局限流**(见[安全与限流](#安全与限流)),并设环境变量 `STRICT_SINGLE_WORKER=True`(见下)在启动期强制拒绝多 worker,避免静默配额漂移。
- [ ] **9. 启动后健康检查**`curl /health` 返回 200(存活);`curl /health/ready` 返回 200(就绪,校验数据库连通,不可达时 503)
- [ ] **10. 数据库迁移** — compose/Dockerfile 启动命令已内置 `alembic upgrade head && uvicorn …`,升级镜像重启即自动迁移,无需手动执行
@@ -83,6 +86,7 @@ cd frontend && npm install && npm run dev
|---|---|---|---|
| `APP_ENV` | ❌ | `development` | `production` / `development` |
| `LOG_LEVEL` | ❌ | `INFO` | 日志级别 |
| `LOG_FILE` | ❌ | (空) | 日志持久化文件路径;空=仅 stdout + Admin 内存日志页(重启清零)。compose 已默认设为 `/app/logs/app.log` 并挂 `applogs` 卷,滚动上限约 10MB×6 份 |
| `API_PORT` | ❌ | `8000` | API 服务端口映射 |
| `FRONTEND_PORT` | ❌ | `3000` | 前端服务端口映射 |
| `POSTGRES_USER` | ✅ | — | PostgreSQL 用户名 |
@@ -99,10 +103,29 @@ cd frontend && npm install && npm run dev
| `LLM_AGGREGATOR_MODEL` | ❌ | — | 终裁模型(回落 `LLM_MODEL`) |
| `BZZOIRO_KEY` | ✅ | — | bzzoiro 数据源 Key(唯一数据源) |
| `CORS_ORIGINS` | ❌ | `http://localhost:5173,...` | 允许的跨域来源 |
| `STRICT_SINGLE_WORKER` | ❌ | `False` | `True` 时若以多 worker 启动则拒绝(防限流配额漂移) |
| `SECRET_KEY` | ❌ | — | 加密主密钥(生产环境必填) |
| `ADMIN_PASSWORD` | ❌ | — | 管理后台密码(留空=不启用) |
| `ADMIN_API_KEY` | ❌ | — | 机器/脚本调用的 API Key |
## 同站部署 vs 跨站 CSRF
Profeto 管理鉴权使用 **HttpOnly Cookie 会话**(登录后服务端写入),`allow_credentials=True` 的 CORS 配置允许浏览器跨域携带 Cookie——这也引入了 CSRF 面。部署拓扑决定风险等级:
**同站部署(推荐)**: 前端与 API 同域(反代把 `/``/api` 都转发到同一后端,或同源端口)。
- 浏览器视为 **same-origin**,CORS 不触发;`SameSite=Lax` 会话 Cookie 天然阻断跨站请求携带。
- 风险最低。`CORS_ORIGINS` 可设为空或同域来源,仅作兜底。
**跨站部署**: 前端与 API 不同域(如前端 `app.example.com`、API `api.example.com`,或开发时 `localhost:3000``localhost:8000`)。
- 必须把 API 域名列入 `CORS_ORIGINS`,且 `allow_credentials=True` 才能携带 Cookie。
- 此时任何被允许域下的页面都能构造带 Cookie 的请求 → **CSRF 面**:
- 状态变更接口(采集/回测/改密等写操作)要求**管理员 Cookie + 同域**,攻击者无法从第三方站点读取 Cookie,但可构造跨域表单/请求——`SameSite=Lax` 会阻断跨站 POST 表单提交(顶级导航 GET 仍放行),这是当前主要防线。
- `GET /api/v1/admin/*` 只读接口受 `SameSite=Lax` 下顶级导航可能被利用,但攻击者无法读取响应(CORS 不匹配时浏览器拦截)。
- **加固建议**:
1. 反代层加 `Origin`/`Referer` 校验,仅放行 `CORS_ORIGINS` 列表中的来源(即便 FastAPI CORS 已通过,反代校验是多一层纵深)。
2. 写操作要求自定义请求头(如 `X-Requested-With: XMLHttpRequest`),第三方站点无法在无预检下添加自定义头,天然阻断简单跨站 POST。
3. 生产强制 HTTPS(`APP_ENV=production` 下 Cookie 自动 `Secure`),防中间人窃 Cookie。
## LLM 提供商配置示例
### OpenAI
+1 -1
View File
@@ -117,7 +117,7 @@ Profeto/
│ ├── config.py # pydantic-settings 配置
│ ├── http_client.py # 共享 httpx 客户端
│ ├── crypto.py # 对称加密(Fernet)与密码哈希
│ ├── log_buffer.py # 内存日志缓冲(admin「系统日志」页)
│ ├── log_buffer.py # 日志基础设施:内存环形缓冲(admin 日志页) + 可选滚动文件持久化(LOG_FILE)
│ ├── runtime_config.py # 运行时配置(数据库优先,回落 .env)
│ ├── scheduler.py # 定时任务调度器(cron 触发采集)
│ └── security_check.py # 生产启动安全校验(缺配置拒绝启动)
+5 -58
View File
@@ -11,48 +11,17 @@
* - 未登录访问管理 → AdminLayout 门禁 → 登录页(不静默失败)
*/
import { BrowserRouter, Routes, Route, Navigate, Link } from 'react-router-dom'
import { BrowserRouter, Routes, Route, Navigate } from 'react-router-dom'
import { ErrorBoundary } from './components/ErrorBoundary'
import Masthead from './components/Masthead'
import Matches from './pages/Matches'
import Standings from './pages/Standings'
import { adminRoutes } from './admin/routes'
/** 报眉日期行 */
function dateLine(): string {
return new Date().toLocaleDateString('zh-CN', {
year: 'numeric',
month: 'long',
day: 'numeric',
weekday: 'long',
})
}
function StandingsLayout({ children }: { children: React.ReactNode }) {
return (
<div className="min-h-screen bg-paper-50">
<header className="masthead-rule">
<div className="mx-auto max-w-5xl px-5 sm:px-8">
<div className="border-b border-ink-900 py-5 text-center sm:py-6">
<h1 className="font-brush text-5xl text-ink-900 sm:text-6xl">
</h1>
</div>
<div className="flex items-center justify-between border-b border-ink-200 py-2 text-2xs text-ink-500">
<span>{dateLine()}</span>
<nav className="flex items-center gap-4" aria-label="页面导航">
<Link to="/" className="text-ink-500 hover:text-press transition-colors">
/
</Link>
<Link to="/admin/eval" className="text-ink-500 hover:text-press transition-colors">
</Link>
<Link to="/admin" className="flex items-center gap-1 text-ink-500 hover:text-press transition-colors">
<span aria-hidden="true"></span>
</Link>
</nav>
</div>
</div>
</header>
<Masthead active="standings" />
<main className="mx-auto max-w-5xl px-5 py-6 sm:px-8 sm:py-8">
{children}
@@ -70,30 +39,8 @@ function StandingsLayout({ children }: { children: React.ReactNode }) {
function HomePage() {
return (
<div className="min-h-screen bg-paper-50">
{/* ── 报头:粗线 + 居中刊名 + 日期与分区链接 ── */}
<header className="masthead-rule">
<div className="mx-auto max-w-5xl px-5 sm:px-8">
<div className="border-b border-ink-900 py-5 text-center sm:py-6">
<h1 className="font-brush text-5xl text-ink-900 sm:text-6xl">
</h1>
</div>
<div className="flex items-center justify-between border-b border-ink-200 py-2 text-2xs text-ink-500">
<span>{dateLine()}</span>
<nav className="flex items-center gap-4" aria-label="页面导航">
<Link to="/standings" className="text-ink-500 hover:text-press transition-colors">
</Link>
<Link to="/admin/eval" className="text-ink-500 hover:text-press transition-colors">
</Link>
<Link to="/admin" className="flex items-center gap-1 text-ink-500 hover:text-press transition-colors">
<span aria-hidden="true"></span>
</Link>
</nav>
</div>
</div>
</header>
{/* ── 报头:粗线 + 居中刊名 + 日期与分区链接(共用 Masthead) ── */}
<Masthead active="home" />
<main className="mx-auto max-w-5xl px-5 py-6 sm:px-8 sm:py-8">
<Matches />
+19
View File
@@ -21,6 +21,7 @@ import type {
LLMAgentConfig,
LogEntry,
IngestSourceStatus,
IngestJob,
MatchDetailOut,
MatchContextOut,
AdminStats,
@@ -362,6 +363,24 @@ export function fetchIngestStatus(): Promise<{ sources: IngestSourceStatus[] }>
return api.get<{ sources: IngestSourceStatus[] }>(`${API_BASE}/admin/ingest/status`)
}
/**
* 采集任务状态轮询(单任务)
*/
export function fetchIngestJob(jobId: string): Promise<IngestJob> {
return api.get<IngestJob>(`${API_BASE}/admin/ingest/jobs/${jobId}`)
}
/**
* 采集任务历史列表(GET /admin/ingest/jobs,最新在前)
*/
export function fetchIngestJobs(params: { limit?: number; status?: string } = {}): Promise<IngestJob[]> {
const q = new URLSearchParams()
if (params.limit != null) q.set('limit', String(params.limit))
if (params.status) q.set('status', params.status)
const qs = q.toString()
return api.get<IngestJob[]>(`${API_BASE}/admin/ingest/jobs${qs ? `?${qs}` : ''}`)
}
/**
* 比赛详情(含最近预测摘要)
*/
+18 -17
View File
@@ -20,18 +20,25 @@ export interface NavItem {
hideFromSidebar?: boolean
}
/** 唯一的导航配置源。顺序 = 侧栏渲染顺序(命令面板分组内顺序与之相同)。 */
/** 唯一的导航配置源。顺序 = 侧栏渲染顺序(命令面板分组内顺序与之相同)。
*
* 分组按用户心智模型: 数据(往里灌) → 预测(算出来) → 系统(保证它活着)。
* 「预测历史/回测/评估」同属预测生命周期,不再挂在「数据流水线」名下。
*/
export const NAV_ITEMS: NavItem[] = [
{ to: '/admin', label: '仪表盘', group: '概览', icon: 'chart' },
{ to: '/admin/collection', label: '数据采集', group: '数据流水线', icon: 'collection' },
{ to: '/admin/data-completeness', label: '数据完整性', group: '数据流水线', icon: 'chart' },
{ to: '/admin/data-pipeline', label: '数据管线', group: '数据流水线', icon: 'chart', hideFromSidebar: true },
{ to: '/admin/predictions', label: '预测历史', group: '数据流水线', icon: 'logs' },
{ to: '/admin/backtest', label: '回测', group: '数据流水线', icon: 'repeat' },
{ to: '/admin/eval', label: '评估', group: '评估与监控', icon: 'eval' },
{ to: '/admin/monitoring', label: '监控', group: '评估与监控', icon: 'monitor' },
{ to: '/admin/settings', label: '设置', group: '系统', icon: 'settings' },
// ── 数据:采集、核对、管线内部视图 ──
{ to: '/admin/collection', label: '数据采集', group: '数据', icon: 'collection' },
{ to: '/admin/data-completeness', label: '数据完整性', group: '数据', icon: 'eval' },
{ to: '/admin/data-pipeline', label: '数据管线', group: '数据', icon: 'chart', hideFromSidebar: true },
// ── 预测:历史 → 回测 → 评估闭环 ──
{ to: '/admin/predictions', label: '预测历史', group: '预测', icon: 'target' },
{ to: '/admin/backtest', label: '回测', group: '预测', icon: 'repeat' },
{ to: '/admin/eval', label: '评估', group: '预测', icon: 'eval' },
// ── 系统:活着吗、发生了什么、怎么调 ──
{ to: '/admin/monitoring', label: '监控', group: '系统', icon: 'monitor' },
{ to: '/admin/logs', label: '日志', group: '系统', icon: 'logs' },
{ to: '/admin/settings', label: '设置', group: '系统', icon: 'settings' },
]
/** 命令面板条目(原 NAV_PAGES 的唯一来源) */
@@ -42,11 +49,6 @@ export const ROUTE_LABELS: Record<string, string> = Object.fromEntries(
NAV_ITEMS.map(i => [i.to, i.label]),
)
/** 侧栏分组标题的历史显示名(仅侧栏使用;与面板分组名不同时在此映射) */
const SIDEBAR_GROUP_TITLES: Record<string, string> = {
: '系统设置',
}
/**
* 侧栏分组(原 NAV_SECTIONS 的唯一来源)。
* 仪表盘(group=概览)在 AdminLayout 中独立渲染于顶部,不进分组循环。
@@ -55,13 +57,12 @@ export const NAV_SECTIONS = (() => {
const sidebarItems = NAV_ITEMS.filter(i => !i.hideFromSidebar && i.group !== '概览')
const titles: string[] = []
for (const i of sidebarItems) {
const title = SIDEBAR_GROUP_TITLES[i.group] ?? i.group
if (!titles.includes(title)) titles.push(title)
if (!titles.includes(i.group)) titles.push(i.group)
}
return titles.map(title => ({
title,
items: sidebarItems
.filter(i => (SIDEBAR_GROUP_TITLES[i.group] ?? i.group) === title)
.filter(i => i.group === title)
.map(({ to, label, icon }) => ({ to, label, icon })),
}))
})()
+145 -58
View File
@@ -11,19 +11,22 @@
*/
import { useEffect, useState, useCallback, useRef } from 'react'
import { triggerCollection, fetchLeagues, fetchIngestStatus } from '../dal'
import type { IngestSourceStatus } from '../types'
import type { CollectionRequest, League } from '../types'
import { triggerCollection, fetchLeagues, fetchIngestJob, fetchIngestJobs } from '../dal'
import type { IngestJob, League } from '../types'
import type { CollectionRequest } from '../types'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner } from '../components'
// 图标用与全站一致的几何字符(Dashboard 工作流卡同款),不混用 emoji
const TASKS = [
{ value: 'events', label: '比赛数据', desc: '赛程 / 比分 / 未开赛安排', icon: '' },
{ value: 'standings', label: '积分榜', desc: '联赛排名 / 积分 / xG差 / 近期走势', icon: '🏆' },
{ value: 'stats', label: '统计回填', desc: '已完赛比赛的 xG / 射门 / 控球等详细统计', icon: '📊' },
{ value: 'all', label: '全量采集', desc: '依次采集比赛 + 积分榜 + 统计回填', icon: '⏵⏵' },
{ value: 'events', label: '比赛数据', desc: '赛程 / 比分 / 未开赛安排', icon: '' },
{ value: 'standings', label: '积分榜', desc: '联赛排名 / 积分 / xG差 / 近期走势', icon: '' },
{ value: 'stats', label: '统计回填', desc: '已完赛比赛的 xG / 射门 / 控球等详细统计', icon: '' },
{ value: 'all', label: '全量采集', desc: '依次采集比赛 + 积分榜 + 统计回填', icon: '' },
] as const
type TaskStatus = 'idle' | 'running' | 'done' | 'error'
type TaskUIStatus = 'idle' | 'running' | 'done' | 'error'
const TERMINAL_STATUSES: ReadonlySet<string> = new Set(['success', 'failed'])
export default function CollectionPage() {
const [leagues, setLeagues] = useState<League[]>([])
@@ -49,13 +52,13 @@ export default function CollectionPage() {
const [limit, setLimit] = useState(100)
const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null)
const [result, setResult] = useState<{ title: string; detail: string } | null>(null)
// 任务进度反馈
const [taskStatus, setTaskStatus] = useState<TaskStatus>('idle')
// 任务进度反馈:跟踪真实 ingest_job 状态
const [taskStatus, setTaskStatus] = useState<TaskUIStatus>('idle')
const [jobId, setJobId] = useState<string | null>(null)
const [jobInfo, setJobInfo] = useState<IngestJob | null>(null)
const [taskStartedAt, setTaskStartedAt] = useState<number | null>(null)
const pollRef = useRef<ReturnType<typeof setInterval> | null>(null)
const [ingestSnap, setIngestSnap] = useState<IngestSourceStatus | null>(null)
const loadLeagues = useCallback(async () => {
const lg = await fetchLeagues()
@@ -64,30 +67,80 @@ export default function CollectionPage() {
useEffect(() => { loadLeagues() }, [loadLeagues])
// 轮询采集状态(任务启动后)
const startPolling = useCallback(() => {
if (pollRef.current) clearInterval(pollRef.current)
pollRef.current = setInterval(async () => {
try {
const { sources } = await fetchIngestStatus()
const bz = sources.find(s => s.name === 'bzzoiro')
if (bz) setIngestSnap(bz)
} catch { /* ignore */ }
}, 5_000)
}, [])
// 轮询采集 job 直到终态(success/failed)
const stopPolling = useCallback(() => {
if (pollRef.current) { clearInterval(pollRef.current); pollRef.current = null }
}, [])
useEffect(() => () => stopPolling(), [stopPolling])
// ── 最近任务历史(GET /admin/ingest/jobs,最新在前) ──
// 声明须在 startJobPolling 之前(其终态回调会刷新历史)
const [recentJobs, setRecentJobs] = useState<IngestJob[] | null>(null)
const loadRecentJobs = useCallback(async () => {
try {
setRecentJobs(await fetchIngestJobs({ limit: 10 }))
} catch { /* 历史列表失败不影响主流程 */ }
}, [])
useEffect(() => { loadRecentJobs() }, [loadRecentJobs])
const startJobPolling = useCallback((id: string) => {
stopPolling()
const tick = async () => {
try {
const job = await fetchIngestJob(id)
setJobInfo(job)
if (TERMINAL_STATUSES.has(job.status)) {
setTaskStatus(job.status === 'success' ? 'done' : 'error')
stopPolling()
loadRecentJobs() // 终态后刷新历史列表
// 广播采集终态:数据完整性等依赖页立即刷新,不必等轮询周期
window.dispatchEvent(new CustomEvent('profeto:ingest-done', {
detail: { jobId: job.id, status: job.status },
}))
}
} catch { /* 单次轮询失败不影响后续 */ }
}
tick()
pollRef.current = setInterval(tick, 3_000)
}, [stopPolling, loadRecentJobs])
const isEventsTask = task === 'events' || task === 'all'
// 友好汇总 job.result
const jobSummary = (j: IngestJob | null): { title: string; detail: string } | null => {
if (!j) return null
if (j.status === 'failed') {
return { title: '采集失败', detail: j.error || '采集任务异常终止,请到「系统日志」查看详细堆栈。' }
}
if (j.status !== 'success') return null
const r = j.result as Record<string, unknown> | null
if (!r) return { title: '采集完成', detail: '任务成功(无汇总数据)。' }
const ev = r.events as Record<string, unknown> | undefined
const evTotal = ev ? (ev.total_inserted as number ?? 0) + (ev.total_updated as number ?? 0) : 0
const st = r.standings as Record<string, unknown> | undefined
const stTotal = st ? (st.total_upserted as number ?? 0) : 0
const stats = r.stats as Record<string, unknown> | undefined
const statsTotal = stats ? (stats.created as number ?? 0) + (stats.updated as number ?? 0) : 0
const evErr = (ev?.errors as string[] | undefined)?.length ?? 0
const stErr = (st?.errors as string[] | undefined)?.length ?? 0
const statsErr = (stats?.errors as string[] | undefined)?.length ?? 0
const totalErr = evErr + stErr + statsErr
const parts: string[] = []
if (ev) parts.push(`比赛 +${evTotal}`)
if (st) parts.push(`积分榜 +${stTotal}`)
if (stats) parts.push(`统计 +${statsTotal}`)
const detail = parts.length
? `共更新: ${parts.join(' / ')}${totalErr ? `,错误 ${totalErr} 条(见日志)` : ''}`
: '任务成功'
return { title: '采集完成', detail }
}
async function handleSubmit(e: React.FormEvent) {
e.preventDefault()
setError(null)
setResult(null)
setJobInfo(null)
setJobId(null)
setLoading(true)
setTaskStatus('running')
setTaskStartedAt(Date.now())
@@ -103,18 +156,15 @@ export default function CollectionPage() {
date_from: isEventsTask ? dateFrom || undefined : undefined,
date_to: isEventsTask ? dateTo || undefined : undefined,
}
await triggerCollection(body)
setResult({
title: '采集任务已启动',
detail: '正在后台执行(上游限速时可能需要数分钟)。完成结果与错误请到「系统日志」页查看(支持自动刷新)。',
})
// 启动轮询,跟踪状态
startPolling()
// 30 秒后自动停止轮询并标记完成
setTimeout(() => {
setTaskStatus('done')
stopPolling()
}, 30_000)
const res = await triggerCollection(body)
const id: string | undefined = res?.job_id
if (id) {
setJobId(id)
startJobPolling(id)
} else {
// 后端未返回 job_id(旧版兼容):退化为原逻辑
setTimeout(() => { setTaskStatus('done'); }, 30_000)
}
} catch (err: unknown) {
setTaskStatus('error')
setError(err instanceof Error ? err.message : '采集触发失败')
@@ -125,6 +175,7 @@ export default function CollectionPage() {
}
const elapsed = taskStartedAt ? Math.round((Date.now() - taskStartedAt) / 1000) : 0
const summary = jobInfo ? jobSummary(jobInfo) : null
return (
<div className="space-y-6">
@@ -148,9 +199,9 @@ export default function CollectionPage() {
key={t.value}
type="button"
onClick={() => setTask(t.value)}
className={`rounded-lg border px-3 py-2 text-left text-xs transition-colors ${
className={`border px-3 py-2 text-left text-xs transition-colors ${
task === t.value
? 'border-brand-500 bg-brand-50 text-brand-700'
? 'border-press bg-press-wash text-press'
: 'border-ink-200 text-ink-600 hover:border-ink-300'
}`}
>
@@ -248,12 +299,11 @@ export default function CollectionPage() {
{/* 消息提示 */}
{error && <Alert kind="error" title="采集失败" message={error} onClose={() => setError(null)} />}
{result && (
{summary && (
<Alert
kind="ok"
title={result.title}
message={result.detail || undefined}
onClose={() => setResult(null)}
kind={jobInfo?.status === 'success' ? 'ok' : 'error'}
title={summary.title}
message={summary.detail}
/>
)}
@@ -278,32 +328,69 @@ export default function CollectionPage() {
<div className="space-y-3">
<div className="flex items-center gap-2 text-xs text-ink-700">
<Spinner />
<span>, {elapsed}s</span>
<span>{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}, {elapsed}s</span>
</div>
<p className="text-2xs text-ink-400">
,
, 3
</p>
</div>
)}
{taskStatus === 'done' && (
<div className="space-y-2">
<div className="flex items-center gap-2 text-xs text-emerald-700">
<span className="inline-block h-2 w-2 rounded-full bg-emerald-500" />
<span>,()</span>
<div className="flex items-center gap-2 text-xs text-ok-700">
<span className="inline-block h-2 w-2 rounded-full bg-ok-500" />
<span>{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}</span>
</div>
<p className="text-2xs text-ink-400">
</p>
{summary && <p className="text-2xs text-ink-500">{summary.detail}</p>}
</div>
)}
{taskStatus === 'error' && (
<p className="text-xs text-press">,</p>
<div className="space-y-1">
<p className="text-xs text-press">{jobId ? `(job ${jobId.slice(0, 8)}…)` : ''}</p>
{jobInfo?.error && (
<p className="text-2xs text-ink-500">{jobInfo.error.slice(0, 200)}</p>
)}
</div>
)}
{ingestSnap?.last_success_at && (
<div className="mt-3 border-t border-ink-100 pt-3">
<p className="text-2xs text-ink-400">
bzzoiro : {new Date(ingestSnap.last_success_at).toLocaleString('zh-CN', { hour12: false })}
</p>
{jobInfo?.created_at && (
<p className="mt-2 text-2xs text-ink-400">
{new Date(jobInfo.created_at).toLocaleString('zh-CN', { hour12: false })}
{jobInfo.finished_at && ` · 完成于 ${new Date(jobInfo.finished_at).toLocaleString('zh-CN', { hour12: false })}`}
</p>
)}
</CardBody>
</Card>
{/* 最近任务历史 */}
<Card>
<CardHeader
title="最近任务"
description="后台采集任务执行历史(最新在前)"
action={<button onClick={loadRecentJobs} className="btn btn-sm"></button>}
/>
<CardBody className="px-0">
{recentJobs === null ? (
<p className="px-4 py-2 text-xs text-ink-400 sm:px-5"></p>
) : recentJobs.length === 0 ? (
<p className="px-4 py-2 text-xs text-ink-400 sm:px-5">,</p>
) : (
<div>
{recentJobs.map(j => {
const st = j.status
const dot = st === 'success' ? 'bg-ink-900' : st === 'failed' ? 'bg-press' : 'bg-warn-500 animate-pulse'
const label = st === 'success' ? '完成' : st === 'failed' ? '失败' : st === 'running' ? '执行中' : '排队中'
return (
<div key={j.id} className="flex items-center gap-3 border-b border-ink-200 px-4 py-2.5 last:border-b-0 sm:px-5">
<span aria-hidden="true" className={`inline-block h-1.5 w-1.5 flex-shrink-0 ${dot}`} />
<Badge status={st === 'failed' ? 'error' : 'info'}>{j.task}</Badge>
<span className="text-2xs text-ink-600">{label}</span>
<span className="ml-auto text-right text-2xs tabular-nums text-ink-400">
{j.created_at && new Date(j.created_at).toLocaleString('zh-CN', { month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit', hour12: false })}
{' '}{jobSummary(j)?.detail ?? (j.error ? j.error.slice(0, 40) : '')}
</span>
</div>
)
})}
</div>
)}
</CardBody>
-267
View File
@@ -1,267 +0,0 @@
/**
* Admin 后台 - 系统配置管理页面(报刊风)
*
* 功能:
* - 登录与鉴权说明(ADMIN_PASSWORD,HttpOnly 会话 Cookie)
* - 显示当前 .env 配置(脱敏;后端暂无配置端点,为静态说明)
* - 配置修改指南
*/
import { useEffect, useState, useCallback } from 'react'
import { fetchSystemConfig } from '../dal'
import { changePassword, fetchAuthState, UNAUTHORIZED_EVENT } from '../api'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner, SkeletonBlock } from '../components'
export default function ConfigPage() {
const [config, setConfig] = useState<any[]>([])
const [loading, setLoading] = useState(true)
const [passwordOrigin, setPasswordOrigin] = useState<'db' | 'env' | 'none' | null>(null)
// 修改密码表单
const [currentPwd, setCurrentPwd] = useState('')
const [newPwd, setNewPwd] = useState('')
const [confirmPwd, setConfirmPwd] = useState('')
const [pwdBusy, setPwdBusy] = useState(false)
const [pwdNotice, setPwdNotice] = useState<{ ok: boolean; text: string } | null>(null)
const loadConfig = useCallback(async () => {
setLoading(true)
try {
const data = await fetchSystemConfig()
setConfig(data)
} catch {
setConfig([])
} finally {
setLoading(false)
}
}, [])
useEffect(() => {
loadConfig()
fetchAuthState()
.then(s => setPasswordOrigin(s.password_origin ?? null))
.catch(() => setPasswordOrigin(null))
}, [loadConfig])
async function handleChangePassword(e: React.FormEvent) {
e.preventDefault()
setPwdNotice(null)
if (newPwd !== confirmPwd) {
setPwdNotice({ ok: false, text: '两次输入的新密码不一致' })
return
}
setPwdBusy(true)
try {
const res = await changePassword(currentPwd, newPwd)
setPwdNotice({ ok: true, text: res.message })
// 密码即会话密钥,修改后所有会话失效:主动切回登录页
setTimeout(() => window.dispatchEvent(new CustomEvent(UNAUTHORIZED_EVENT)), 1500)
} catch (err) {
setPwdNotice({ ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '修改失败' })
} finally {
setPwdBusy(false)
}
}
return (
<div className="space-y-6">
<SectionHeader
title="系统配置"
description="登录鉴权说明与系统参数查看。敏感配置一律通过服务器 .env 文件管理。"
/>
{/* 登录与鉴权 */}
<Card>
<CardHeader
title="登录与鉴权"
description="本后台通过密码登录保护,会话以 HttpOnly Cookie 保存,有效期默认 7 天"
/>
<CardBody>
<Alert kind="ok" title="已通过密码登录" />
<p className="mt-3 text-2xs leading-relaxed text-ink-500">
<code className="font-mono">.env</code> {' '}
<code className="font-mono">ADMIN_PASSWORD</code>(),{' '}
<code className="font-mono">scrypt</code> ,
,,
<code className="font-mono">ADMIN_API_KEY</code>( X-API-Key)
</p>
{passwordOrigin && (
<p className="mt-2 flex items-center gap-2 text-2xs text-ink-500">
:
{passwordOrigin === 'db' ? (
<Badge status="success">(scrypt )</Badge>
) : passwordOrigin === 'env' ? (
<Badge status="info">.env </Badge>
) : (
<Badge status="error"></Badge>
)}
</p>
)}
{/* 修改密码表单 */}
<form onSubmit={handleChangePassword} className="mt-5 space-y-3 border-t border-ink-200 pt-4">
<div className="grid gap-3 sm:grid-cols-3">
<div>
<label className="mb-1 block text-2xs text-ink-500"></label>
<input
type="password"
value={currentPwd}
onChange={e => setCurrentPwd(e.target.value)}
autoComplete="current-password"
className="field w-full"
/>
</div>
<div>
<label className="mb-1 block text-2xs text-ink-500">( 8 )</label>
<input
type="password"
value={newPwd}
onChange={e => setNewPwd(e.target.value)}
autoComplete="new-password"
className="field w-full"
/>
</div>
<div>
<label className="mb-1 block text-2xs text-ink-500"></label>
<input
type="password"
value={confirmPwd}
onChange={e => setConfirmPwd(e.target.value)}
autoComplete="new-password"
className="field w-full"
/>
</div>
</div>
{pwdNotice && (
<Alert kind={pwdNotice.ok ? 'ok' : 'error'} title={pwdNotice.text} />
)}
<div className="flex items-center justify-between gap-2">
<p className="text-2xs text-ink-400">退,</p>
<button
type="submit"
disabled={pwdBusy || !currentPwd || !newPwd || !confirmPwd}
className="btn btn-solid btn-sm flex-shrink-0"
>
{pwdBusy ? (<><Spinner /> </>) : '修改密码'}
</button>
</div>
</form>
</CardBody>
</Card>
{/* 快速导航 */}
<div className="grid gap-4 sm:grid-cols-2">
<a
href="/admin/data-sources"
className="group flex items-center justify-between border border-ink-300 p-4 transition-colors hover:border-ink-900 hover:bg-paper-100"
>
<div>
<div className="font-serif text-sm font-bold text-ink-900"></div>
<div className="mt-0.5 text-2xs text-ink-500"> API Key</div>
</div>
<span className="text-ink-300 transition-colors group-hover:text-press" aria-hidden="true"></span>
</a>
<a
href="/admin/llm-config"
className="group flex items-center justify-between border border-ink-300 p-4 transition-colors hover:border-ink-900 hover:bg-paper-100"
>
<div>
<div className="font-serif text-sm font-bold text-ink-900">LLM </div>
<div className="mt-0.5 text-2xs text-ink-500"></div>
</div>
<span className="text-ink-300 transition-colors group-hover:text-press" aria-hidden="true"></span>
</a>
</div>
{/* 配置列表 */}
<Card>
<CardHeader
title="当前配置"
description="脱敏展示,实际值在服务器 .env 文件中"
action={
<button onClick={loadConfig} disabled={loading} className="btn btn-sm">
{loading ? (<><Spinner /> </>) : '刷新'}
</button>
}
/>
<CardBody className="px-0 sm:px-0">
{loading ? (
<div className="space-y-3 px-4 sm:px-5">
{[1, 2, 3, 4, 5].map(i => (
<SkeletonBlock key={i} className="h-9 w-full" />
))}
</div>
) : config.length > 0 ? (
<div>
{config.map(item => (
<div
key={item.key}
className="flex flex-col gap-1 border-b border-ink-200 px-4 py-2.5 last:border-b-0 sm:grid sm:grid-cols-[minmax(0,2fr)_minmax(0,3fr)_minmax(0,2fr)] sm:items-baseline sm:gap-4 sm:px-5"
>
<div className="flex items-center gap-2">
<span className="font-mono text-xs text-ink-800">{item.key}</span>
{item.is_sensitive && <Badge status="warning"></Badge>}
</div>
<div className="break-all font-mono text-2xs text-ink-500">{item.value_masked}</div>
<div className="text-2xs text-ink-400">{item.description}</div>
</div>
))}
</div>
) : (
<p className="py-8 text-center text-xs text-ink-400"></p>
)}
</CardBody>
</Card>
{/* 修改指南 */}
<Card>
<CardHeader title="修改配置指南" />
<CardBody className="space-y-5">
<div>
<h4 className="mb-2 font-serif text-sm font-bold text-ink-900"> SSH .env</h4>
<pre className="overflow-x-auto border border-ink-200 bg-paper-100 p-3 font-mono text-2xs leading-relaxed text-ink-700">
{`# 连接到部署主机
ssh user@your-server-ip
# 进入项目目录
cd /vol2/1000/Docker/Profeto
# 编辑 .env 文件
nano .env
# 修改后重启后端服务
docker compose restart api
# 查看日志确认生效
docker compose logs -f api`}
</pre>
</div>
<div>
<h4 className="mb-2 font-serif text-sm font-bold text-ink-900"></h4>
<div className="space-y-2">
{[
['LLM_API_KEY', 'LLM 服务商的 API 密钥,用于调用大模型'],
['LLM_MODEL', '使用的模型名称,如 gpt-4o、claude-3-5-sonnet'],
['LLM_BASE_URL', 'API 基础地址,支持兼容 OpenAI 协议的服务商'],
['ADMIN_PASSWORD', '管理后台登录密码,修改后重启 api 容器生效'],
['ADMIN_API_KEY', '脚本直连接口的鉴权密钥(请求头 X-API-Key)'],
['BZZOIRO_KEY', 'Bzzoiro 数据源 API 密钥'],
['DATABASE_URL', 'PostgreSQL 数据库连接字符串'],
].map(([key, desc]) => (
<div key={key} className="flex items-start gap-2.5">
<code className="flex-shrink-0 border border-ink-200 bg-paper-100 px-1.5 py-0.5 font-mono text-2xs text-ink-800">
{key}
</code>
<span className="text-xs leading-relaxed text-ink-600">{desc}</span>
</div>
))}
</div>
</div>
</CardBody>
</Card>
</div>
)
}
+134 -72
View File
@@ -1,137 +1,199 @@
/**
* Admin 后台 - 仪表盘(报刊风)
* Admin 后台 - 仪表盘(报刊风·待办驱动)
*
* 展示:
* - 数据流水线状态(采集 → 预测 → 评估,每步的实际数据量)
* - 近期预测活动(24h / 7d / 总计)
* - 快捷操作入口(带工作流引导)
* 设计原则:单人管理员的注意力应该花在「现在需要处理什么」上,
* 而不是扫描一堆常驻数字。
* - 顶部待办行:只有真有待办才出现(死信 / 缺数据联赛 / 可结算预测),
* 每项直达处理页面 —— 引导出现在需要时,而不是永远占着版面
* - 三步工作流卡只在库里还没有比赛时显示(首次使用引导)
* - 数据概览合并为一卡:预测活动 + 各表数据量
*/
import { useEffect, useState, useCallback } from 'react'
import { fetchAdminStats, fetchIngestStatus, fetchDashboard } from '../dal'
import { Link } from 'react-router-dom'
import { fetchAdminStats, fetchIngestStatus, fetchDashboard, fetchIngestFailures, fetchDataCompleteness, fetchPredictions } from '../dal'
import type { DataCompletenessResponse, IngestFailureItem } from '../dal'
import type { AdminStats, IngestSourceStatus, DashboardStats } from '../types'
import { Card, CardBody, CardHeader, Alert, SkeletonBlock } from '../components'
import { Card, CardBody, CardHeader, SkeletonBlock } from '../components'
/** 工作流步骤卡片 */
/** 工作流引导(仅首次使用——库里还没有比赛时显示) */
const STEPS = [
{ to: '/admin/collection', step: '1', title: '采集数据', desc: 'bzzoiro: 赛程 / 积分榜 / 比赛统计(xG、射门、控球等)', icon: '◈' },
{ to: '/admin/predictions', step: '2', title: '运行预测', desc: '调 LLM 多专家生成比分预测', icon: '◆' },
{ to: '/admin/eval', step: '3', title: '评估准确率', desc: '结算后查看 1X2 命中率与校准度', icon: '◈' },
{ to: '/admin/collection', step: '1', title: '采集数据', desc: 'bzzoiro: 赛程 / 积分榜 / 比赛统计(xG、射门、控球等)' },
{ to: '/admin/predictions', step: '2', title: '预测与结算', desc: '在前台比赛详情页发起预测;赛后回到「预测历史」结算' },
{ to: '/admin/eval', step: '3', title: '评估准确率', desc: '结算后查看 1X2 命中率与校准度' },
]
interface TodoItem {
key: string
count: number
label: string
to: string
/** 无上限确认时数字近似,展示为 N+ */
approx?: boolean
}
export default function Dashboard() {
const [stats, setStats] = useState<AdminStats | null>(null)
const [ingest, setIngest] = useState<IngestSourceStatus[]>([])
const [dash, setDash] = useState<DashboardStats | null>(null)
const [failures, setFailures] = useState<IngestFailureItem[]>([])
const [completeness, setCompleteness] = useState<DataCompletenessResponse | null>(null)
const [recentPreds, setRecentPreds] = useState<Array<{ settled?: boolean; actual_home_goals?: number | null; actual_away_goals?: number | null }>>([])
const [loading, setLoading] = useState(true)
const load = useCallback(async () => {
setLoading(true)
const [s, i, d] = await Promise.allSettled([
// 各数据源独立容错:单接口失败只降级对应卡片,不拖垮整页
const [s, i, d, f, c, p] = await Promise.allSettled([
fetchAdminStats(),
fetchIngestStatus(),
fetchDashboard(),
fetchIngestFailures(),
fetchDataCompleteness(),
fetchPredictions(100),
])
if (s.status === 'fulfilled') setStats(s.value)
if (i.status === 'fulfilled') setIngest(i.value.sources)
if (d.status === 'fulfilled') setDash(d.value)
if (f.status === 'fulfilled') setFailures(f.value)
if (c.status === 'fulfilled') setCompleteness(c.value)
if (p.status === 'fulfilled' && Array.isArray(p.value)) setRecentPreds(p.value)
setLoading(false)
}, [])
useEffect(() => { load() }, [load])
// ── 待办计算 ──
const deadLetterCount = failures.filter(f => f.status !== 'resolved').length
const missingStatsLeagues = completeness?.leagues.filter(
l => l.matches.finished > 0 && l.stats.rows === 0,
).length ?? 0
const missingStandingsLeagues = completeness?.leagues.filter(
l => l.matches.total > 0 && l.standings.rows === 0,
).length ?? 0
const missingLeagues = missingStatsLeagues + missingStandingsLeagues
// 近 100 条内「比赛已出比分但未结算」的预测(列表接口有上限,数字近似)
const settleable = recentPreds.filter(
p => !p.settled && p.actual_home_goals != null && p.actual_away_goals != null,
).length
const todos: TodoItem[] = [
deadLetterCount > 0 && { key: 'deadletter', count: deadLetterCount, label: '采集失败待处理', to: '/admin/data-pipeline' },
missingLeagues > 0 && { key: 'completeness', count: missingLeagues, label: '联赛数据缺口', to: '/admin/data-completeness' },
settleable > 0 && { key: 'settle', count: settleable, label: '预测可结算', to: '/admin/predictions', approx: true },
].filter((t): t is TodoItem => t !== false)
const sourceByName = Object.fromEntries(ingest.map(s => [s.name, s]))
const bzzoiro = sourceByName['bzzoiro']
const hasMatches = (stats?.matches?.total ?? 0) > 0
return (
<div className="space-y-6">
{/* ── 工作流引导(采集 → 预测 → 评估) ── */}
<div className="grid gap-4 sm:grid-cols-3">
{STEPS.map((s, i) => (
<a
key={s.to}
href={s.to}
className="group border border-ink-300 p-4 transition-colors hover:border-ink-900 hover:bg-paper-100"
>
<div className="flex items-center gap-2.5">
<span className="flex h-7 w-7 items-center justify-center border border-ink-900 font-serif text-xs font-bold text-ink-900">
{s.step}
{/* ── 待办行:只有真有待办才出现 ── */}
{loading ? (
<SkeletonBlock className="h-14 w-full" />
) : todos.length > 0 ? (
<div className="grid gap-3 sm:grid-cols-3">
{todos.map(t => (
<Link
key={t.key}
to={t.to}
className="group flex items-center gap-3 border border-press bg-press-wash/40 px-4 py-3 transition-colors hover:bg-press-wash"
>
<span className="font-serif text-2xl font-bold tabular-nums text-press">
{t.count}{t.approx ? '+' : ''}
</span>
<span className="font-serif text-sm font-bold text-ink-900">{s.title}</span>
<span className="ml-auto text-ink-300 transition-colors group-hover:text-press" aria-hidden="true"></span>
</div>
<p className="mt-2 text-2xs leading-relaxed text-ink-500">{s.desc}</p>
{i < STEPS.length - 1 && <span className="sr-only"></span>}
</a>
))}
</div>
<span className="text-xs text-ink-700">{t.label}</span>
<span className="ml-auto text-press transition-transform group-hover:translate-x-0.5" aria-hidden="true"></span>
</Link>
))}
</div>
) : (
<p className="flex items-center gap-2 border-b border-ink-200 pb-3 text-2xs text-ink-400">
<span className="inline-block h-1.5 w-1.5 bg-ink-900" aria-hidden="true" />
流水线无待办:没有失败记录
</p>
)}
{/* ── 数据源健康一览 ── */}
{/* ── 三步引导:仅首次使用(库里还没有比赛)时显示 ── */}
{!loading && !hasMatches && (
<div className="grid gap-4 sm:grid-cols-3">
{STEPS.map(s => (
<Link
key={s.to}
to={s.to}
className="group border border-ink-300 p-4 transition-colors hover:border-ink-900 hover:bg-paper-100"
>
<div className="flex items-center gap-2.5">
<span className="flex h-7 w-7 items-center justify-center border border-ink-900 font-serif text-xs font-bold text-ink-900">
{s.step}
</span>
<span className="font-serif text-sm font-bold text-ink-900">{s.title}</span>
<span className="ml-auto text-ink-300 transition-colors group-hover:text-press" aria-hidden="true"></span>
</div>
<p className="mt-2 text-2xs leading-relaxed text-ink-500">{s.desc}</p>
</Link>
))}
</div>
)}
{/* ── 数据源:单源一行即足,不装成列表 ── */}
<Card>
<CardHeader
title="数据源健康"
description="各源最近采集时间与数据量(只读快照,详细配置见「数据源」页)"
title="数据源"
description="bzzoiro 最近采集情况,Key 与轮换配置见「设置 → 数据源」"
/>
<CardBody className="px-0">
{loading ? (
<div className="space-y-2 px-4 sm:px-5">
{[1, 2, 3].map(i => <SkeletonBlock key={i} className="h-8 w-full" />)}
</div>
<div className="px-4 sm:px-5"><SkeletonBlock className="h-8 w-full" /></div>
) : (
<div>
{[
{ name: 'bzzoiro', label: 'Bzzoiro', st: bzzoiro },
].map(({ name, label, st }) => {
const hasData = st && st.recent_count > 0
const keyOk = st?.key_configured !== false
return (
<div key={name} className="flex items-center justify-between border-b border-ink-200 px-4 py-2.5 last:border-b-0 sm:px-5">
<span className="text-xs font-medium text-ink-700">{label}</span>
<span className="flex items-center gap-3 text-2xs">
{hasData ? (
<>
<span className="tabular-nums text-ink-500">{st.recent_count.toLocaleString()} </span>
<span className="text-ink-400">{st.last_success_at ? new Date(st.last_success_at).toLocaleString('zh-CN', { month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit', hour12: false }) : ''}</span>
</>
) : keyOk ? (
<span className="text-ink-400"></span>
) : (
<span className="text-press"> Key</span>
)}
<span
aria-hidden="true"
className={`inline-block h-1.5 w-1.5 ${hasData && keyOk ? 'bg-ink-900' : 'bg-press'}`}
/>
</span>
</div>
)
})}
</div>
(() => {
const st = bzzoiro
const hasData = st && st.recent_count > 0
const keyOk = st?.key_configured !== false
return (
<div className="flex items-center justify-between px-4 py-2.5 sm:px-5">
<span className="text-xs font-medium text-ink-700">Bzzoiro</span>
<span className="flex items-center gap-3 text-2xs">
{hasData ? (
<>
<span className="tabular-nums text-ink-500">{st.recent_count.toLocaleString()} </span>
<span className="text-ink-400">{st.last_success_at ? new Date(st.last_success_at).toLocaleString('zh-CN', { month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit', hour12: false }) : ''}</span>
</>
) : keyOk ? (
<span className="text-ink-400"></span>
) : (
<Link to="/admin/settings?tab=datasource" className="text-press underline underline-offset-2"> Key,</Link>
)}
<span aria-hidden="true" className={`inline-block h-1.5 w-1.5 ${hasData && keyOk ? 'bg-ink-900' : 'bg-press'}`} />
</span>
</div>
)
})()
)}
</CardBody>
</Card>
{/* ── 近期预测活动 ── */}
{/* ── 数据概览:预测活动 + 数据量,合并一卡 ── */}
<Card>
<CardHeader title="近期预测活动" description="预测 API 的调用量统计" />
<CardHeader title="数据概览" description="预测调用量与各表数据量" />
<CardBody>
{stats ? (
<div className="space-y-4">
<div className="grid grid-cols-3 gap-4 text-center">
<div>
<div className="font-serif text-3xl font-bold tabular-nums text-ink-900">{stats.predictions.last_24h}</div>
<div className="mt-1 text-2xs text-ink-400"> 24 </div>
<div className="mt-1 text-2xs text-ink-400"> · 24 </div>
</div>
<div>
<div className="font-serif text-3xl font-bold tabular-nums text-ink-900">{stats.predictions.last_7d}</div>
<div className="mt-1 text-2xs text-ink-400"> 7 </div>
<div className="mt-1 text-2xs text-ink-400"> · 7 </div>
</div>
<div>
<div className="font-serif text-3xl font-bold tabular-nums text-ink-900">{stats.predictions.total}</div>
<div className="mt-1 text-2xs text-ink-400"></div>
<div className="mt-1 text-2xs text-ink-400"> · </div>
</div>
</div>
{/* F3 修复: 真实比赛计数(非 limit=100 近似) */}
<div className="grid grid-cols-4 gap-3 border-t border-ink-200 pt-3 text-center">
<div>
<div className="font-serif text-xl font-bold tabular-nums text-ink-900">{stats.matches?.total ?? 0}</div>
+34 -6
View File
@@ -26,9 +26,9 @@ const FIELD_LABELS: Record<string, string> = {
}
function pctColor(pct: number): string {
if (pct >= 80) return 'bg-emerald-500'
if (pct >= 50) return 'bg-amber-500'
return 'bg-rose-500'
if (pct >= 80) return 'bg-ok-500'
if (pct >= 50) return 'bg-warn-500'
return 'bg-bad-500'
}
/** 根据问题描述生成可操作的修复链接 */
@@ -55,6 +55,9 @@ export default function DataCompletenessPage() {
const [loading, setLoading] = useState(true)
const [error, setError] = useState<string | null>(null)
const [highlightedLeague, setHighlightedLeague] = useState<string | null>(null)
// 页面文案承诺「每 5 秒自动刷新」,此前并未实现(仅挂载加载一次),
// 采集完成后数字不动 —— 现补齐:5s 轮询 + 采集完成事件即时刷新。
const [autoRefresh, setAutoRefresh] = useState(true)
const leagueRefs = useRef<Record<string, HTMLDivElement | null>>({})
const load = useCallback(async () => {
@@ -72,6 +75,20 @@ export default function DataCompletenessPage() {
useEffect(() => { load() }, [load])
// 5s 自动轮询(与页面文案一致);已有数据时刷新不闪骨架
useEffect(() => {
if (!autoRefresh) return
const t = setInterval(load, 5_000)
return () => clearInterval(t)
}, [autoRefresh, load])
// 采集页任务终态广播 → 立即刷新(不等下一个 5s 周期)
useEffect(() => {
const onIngestDone = () => load()
window.addEventListener('profeto:ingest-done', onIngestDone)
return () => window.removeEventListener('profeto:ingest-done', onIngestDone)
}, [load])
// 点击问题项 → 滚动到对应联赛卡片并高亮
const scrollToLeague = useCallback((code: string) => {
setHighlightedLeague(code)
@@ -92,9 +109,20 @@ export default function DataCompletenessPage() {
title="数据完整性"
description="按联赛统计 bzzoiro 数据采集覆盖度。每 5 秒自动刷新,或点击右上角按钮手动刷新。"
action={
<button onClick={load} disabled={loading} className="btn-sm btn-outline">
{loading ? <><Spinner /> </> : '刷新'}
</button>
<div className="flex items-center gap-2">
<label className="flex cursor-pointer items-center gap-1.5 text-2xs text-ink-500">
<input
type="checkbox"
checked={autoRefresh}
onChange={e => setAutoRefresh(e.target.checked)}
className="accent-current"
/>
5s
</label>
<button onClick={load} disabled={loading} className="btn-sm btn-outline">
{loading ? <><Spinner /> </> : '刷新'}
</button>
</div>
}
/>
-376
View File
@@ -1,376 +0,0 @@
import { useEffect, useState, useCallback } from 'react'
import {
fetchDataSourceStatuses,
fetchIngestStatus,
fetchAdminStats,
fetchKeyRingStatus,
resetKeyRingCooldown,
updateSetting,
clearSetting,
testDataSourceConnection,
} from '../dal'
import type { DataSourceStatus, DataSourceTestResult, IngestSourceStatus, AdminStats } from '../types'
import type { KeyRingStatusResponse } from '../dal'
import SettingRow from '../SettingRow'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner, SkeletonBlock } from '../components'
function formatTime(iso: string | null): string {
if (!iso) return '暂无记录'
try {
return new Date(iso).toLocaleString('zh-CN', { hour12: false })
} catch {
return iso
}
}
export default function DataSourcesPage() {
const [sources, setSources] = useState<DataSourceStatus[]>([])
const [loading, setLoading] = useState(true)
const [loadError, setLoadError] = useState('')
const [ingestStats, setIngestStats] = useState<Record<string, IngestSourceStatus>>({})
const [ingestLoading, setIngestLoading] = useState(true)
const [stats, setStats] = useState<AdminStats | null>(null)
const [testingSource, setTestingSource] = useState<string | null>(null)
const [testResults, setTestResults] = useState<Record<string, DataSourceTestResult>>({})
const [editingKey, setEditingKey] = useState<string | null>(null)
const [busyKey, setBusyKey] = useState<string | null>(null)
const [rowNotice, setRowNotice] = useState<{ key: string; ok: boolean; text: string } | null>(null)
const [keyRing, setKeyRing] = useState<KeyRingStatusResponse | null>(null)
const [ringLoading, setRingLoading] = useState(false)
const loadSources = useCallback(async () => {
setLoading(true)
setLoadError('')
try {
setSources(await fetchDataSourceStatuses())
} catch (err) {
setLoadError(err instanceof Error ? err.message.split('\n')[0] : '加载失败')
setSources([])
} finally {
setLoading(false)
}
}, [])
// 健康状态(只读,与配置加载并行;失败不阻塞配置页)
const loadIngest = useCallback(async () => {
setIngestLoading(true)
try {
const { sources } = await fetchIngestStatus()
setIngestStats(Object.fromEntries(sources.map(x => [x.name, x])))
} catch {
/* ignore */
} finally {
setIngestLoading(false)
}
}, [])
// 管理区统计(只读)
const loadStats = useCallback(async () => {
try {
setStats(await fetchAdminStats())
} catch {
/* ignore */
}
}, [])
// Key Ring 状态(只读)
const loadKeyRing = useCallback(async () => {
setRingLoading(true)
try {
setKeyRing(await fetchKeyRingStatus())
} catch {
/* ignore */
} finally {
setRingLoading(false)
}
}, [])
useEffect(() => {
loadSources()
loadIngest()
loadStats()
loadKeyRing()
}, [loadSources, loadIngest, loadStats, loadKeyRing])
async function handleTest(sourceName: string) {
setTestingSource(sourceName)
setTestResults(prev => ({ ...prev, [sourceName]: { ok: false, status: null, latency_ms: 0, detail: '测试中...' } }))
try {
const result = await testDataSourceConnection(sourceName)
setTestResults(prev => ({ ...prev, [sourceName]: result }))
} catch (err: unknown) {
const msg = err instanceof Error ? err.message.split('\n')[0] : '连接失败'
setTestResults(prev => ({ ...prev, [sourceName]: { ok: false, status: null, latency_ms: 0, detail: msg } }))
} finally {
setTestingSource(null)
}
}
// 渲染数据源健康块(最近采集 + 异常提示)
function renderHealth(sourceName: string) {
const st = ingestStats[sourceName]
if (!st) {
return (
<div className="flex items-center justify-between text-xs">
<span className="text-ink-400"></span>
<span className="text-ink-400">{ingestLoading ? '加载中...' : '暂无数据'}</span>
</div>
)
}
const ago = st.last_success_at ? formatTime(st.last_success_at) : '暂无记录'
const issues: string[] = []
if (st.status === 'key_not_configured') issues.push('未配置 API Key')
else if (st.status === 'no_data') issues.push('本地无数据,建议补采')
if (st.last_failure) issues.push('近期有采集失败')
return (
<div className="space-y-1.5 border-t border-ink-200 pt-3">
<div className="flex items-center justify-between text-xs">
<span className="text-ink-400"></span>
<span className="text-ink-600">{ago}</span>
</div>
<div className="flex items-center justify-between text-xs">
<span className="text-ink-400">()</span>
<span className="text-ink-600">{st.recent_count.toLocaleString()} </span>
</div>
{st.note && <p className="text-2xs leading-relaxed text-ink-400">{st.note}</p>}
{issues.length > 0 && (
<p className="border-l-2 border-press bg-press-wash/40 px-2 py-1 text-2xs leading-relaxed text-press-dark">
{issues.join(' / ')}
</p>
)}
{st.last_failure && (
<p className="truncate text-2xs text-ink-400" title={st.last_failure.detail}>
: {st.last_failure.detail.slice(0, 60)}
</p>
)}
</div>
)
}
async function handleSave(key: string, value: string) {
setBusyKey(key)
setRowNotice(null)
try {
await updateSetting(key, value)
setRowNotice({ key, ok: true, text: '已保存,立即生效' })
setEditingKey(null)
await loadSources()
} catch (err) {
setRowNotice({ key, ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '保存失败' })
} finally {
setBusyKey(null)
}
}
async function handleClear(key: string) {
setBusyKey(key)
setRowNotice(null)
try {
await clearSetting(key)
setRowNotice({ key, ok: true, text: '已清除数据库覆盖,回落 .env 默认值' })
await loadSources()
} catch (err) {
setRowNotice({ key, ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '清除失败' })
} finally {
setBusyKey(null)
}
}
async function handleResetCooldown() {
if (!window.confirm('确定重置所有 key 的冷却状态?这可能使被限流的 key 立即恢复请求。')) return
try {
const res = await resetKeyRingCooldown()
setKeyRing(res.stats)
setRowNotice({ key: "__ring", ok: true, text: res.message })
} catch (err) {
setRowNotice({ key: "__ring", ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '重置失败' })
}
}
return (
<div className="space-y-6">
<SectionHeader
title="数据源管理"
description="数据采集源的 API 配置、健康状态与连通性测试。"
/>
{loadError && (
<Alert kind="error" title="无法加载数据源配置" message={loadError} />
)}
{/* 数据源卡片 */}
{loading ? (
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-3">
{[1, 2, 3].map(i => (
<Card key={i}>
<CardBody>
<div className="space-y-3">
<SkeletonBlock className="h-4 w-24" />
<SkeletonBlock className="h-3 w-32" />
<SkeletonBlock className="h-8 w-full" />
</div>
</CardBody>
</Card>
))}
</div>
) : (
<div className="grid gap-4 sm:grid-cols-2 lg:grid-cols-3">
{sources.map(source => {
const result = testResults[source.name]
const cardKeys = source.settings.map(s => s.key)
return (
<Card key={source.name}>
<CardBody className="space-y-4">
<div className="flex flex-wrap items-center justify-between gap-x-2 gap-y-1 border-b border-ink-200 pb-3">
<h3 className="font-serif text-sm font-bold text-ink-900">{source.label}</h3>
<Badge status={source.key_configured ? 'success' : 'error'}>
{source.key_configured ? '已就绪' : '缺配置'}
</Badge>
</div>
<p className="text-2xs leading-relaxed text-ink-500">{source.description}</p>
{source.settings.length > 0 ? (
<div>
{source.settings.map(setting => (
<SettingRow
key={setting.key}
setting={setting}
editing={editingKey === setting.key}
busy={busyKey === setting.key}
onEdit={() => { setEditingKey(setting.key); setRowNotice(null) }}
onCancel={() => setEditingKey(null)}
onSave={v => handleSave(setting.key, v)}
onClear={() => handleClear(setting.key)}
/>
))}
</div>
) : (
<p className="text-2xs text-ink-400"> API Key</p>
)}
{rowNotice && cardKeys.includes(rowNotice.key) && (
<Alert kind={rowNotice.ok ? 'ok' : 'error'} title={rowNotice.text} />
)}
{/* 数据源健康:最近采集 + 异常提示 */}
{renderHealth(source.name)}
{result && testingSource !== source.name && (
<Alert
kind={result.ok ? 'ok' : 'error'}
title={result.ok ? `连接成功(${result.latency_ms}ms)` : result.status ? `HTTP ${result.status}` : '连接失败'}
message={result.ok ? undefined : result.detail}
/>
)}
<button
onClick={() => handleTest(source.name)}
disabled={testingSource === source.name}
className="btn btn-sm w-full"
>
{testingSource === source.name ? (<><Spinner /> </>) : '测试连接'}
</button>
</CardBody>
</Card>
)
})}
</div>
)}
{/* API Key 轮换环状态 */}
<Card>
<CardHeader
title="API Key 轮换环"
description={keyRing?.has_multiple
? `已配置 ${keyRing.total} 个 key,遇到限流(429)自动切换;冷却 ${keyRing.cooldown_seconds}s`
: '当前仅 1 个 key,无法轮换。建议配置多个 key 以提高限流容忍度'
}
action={
<button
onClick={handleResetCooldown}
disabled={ringLoading}
className="btn-sm btn-outline"
>
</button>
}
/>
<CardBody>
{ringLoading && !keyRing ? (
<SkeletonBlock className="h-10 w-full" />
) : keyRing && keyRing.total > 0 ? (
<div className="space-y-2">
{keyRing.keys.map((k, i) => {
const isBlocked = k.blocked_remaining > 0
return (
<div key={i} className={`flex items-center justify-between gap-3 border-b border-ink-100 py-2 last:border-b-0 ${isBlocked ? 'opacity-70' : ''}`}>
<div className="flex items-center gap-2">
<span className={`inline-block h-2 w-2 rounded-full ${isBlocked ? 'bg-amber-500' : 'bg-emerald-500'}`} />
<span className="font-mono text-xs text-ink-700">{k.masked}</span>
{i === keyRing.active_index && (
<span className="rounded bg-ink-900 px-1.5 py-0.5 text-2xs text-paper-500"></span>
)}
</div>
<span className={`text-2xs tabular-nums ${isBlocked ? 'text-amber-600' : 'text-ink-400'}`}>
{isBlocked ? `冷却中 ${k.blocked_remaining.toFixed(0)}s` : '可用'}
</span>
</div>
)
})}
</div>
) : (
<p className="text-xs text-ink-400"> key </p>
)}
<p className="mt-3 border-l-2 border-ink-300 pl-3 text-2xs leading-relaxed text-ink-500">
Bzzoiro<b> / / </b> key 429 key key;
key key key
</p>
</CardBody>
</Card>
{/* 近期活动统计(只读) */}
{stats && stats.predictions && (
<Card>
<CardHeader title="近期预测活动" description="过去 24 小时 / 7 天的预测次数" />
<CardBody>
<div className="grid grid-cols-3 gap-4 text-center">
<div>
<div className="font-serif text-2xl font-bold tabular-nums text-ink-900">{stats.predictions.last_24h}</div>
<div className="mt-1 text-2xs text-ink-400"> 24 </div>
</div>
<div>
<div className="font-serif text-2xl font-bold tabular-nums text-ink-900">{stats.predictions.last_7d}</div>
<div className="mt-1 text-2xs text-ink-400"> 7 </div>
</div>
<div>
<div className="font-serif text-2xl font-bold tabular-nums text-ink-900">{stats.predictions.total}</div>
<div className="mt-1 text-2xs text-ink-400"></div>
</div>
</div>
</CardBody>
</Card>
)}
<Card>
<CardHeader title="配置说明" />
<CardBody>
<div className="space-y-3 text-xs leading-relaxed text-ink-600">
<p className="border-l-2 border-ink-300 pl-3">
<code className="font-mono">app_settings</code> <b></b>, <code className="font-mono">.env</code>; .env
</p>
<p className="border-l-2 border-ink-300 pl-3">
(); Key / /
</p>
<p className="border-l-2 border-ink-300 pl-3">
;,
</p>
</div>
</CardBody>
</Card>
</div>
)
}
-275
View File
@@ -1,275 +0,0 @@
/**
* Admin 后台 - LLM 配置管理页面(报刊风)
*
* 功能:
* - 显示当前 LLM 配置(provider, model, base_url;后端暂无配置端点,当前值取自 .env 约定)
* - 测试 LLM 连接(会真实调用一次 /predict,产生 LLM 调用费用)
* - 显示 LLM 使用统计(从预测记录聚合)
* - 可用模型列表
*/
import { useEffect, useState, useCallback } from 'react'
import { testLLMConnection, fetchLLMUsageStats, fetchSettings, fetchLLMModels, updateSetting, clearSetting } from '../dal'
import type { LLMUsageStats } from '../types'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner, SkeletonBlock } from '../components'
import SettingRow from '../SettingRow'
import AgentLLMCard from '../AgentLLMCard'
import type { DataSourceSetting } from '../types'
const LLM_SETTING_KEYS = ['LLM_API_KEY', 'LLM_BASE_URL', 'LLM_MODEL']
export default function LLMConfigPage() {
const [stats, setStats] = useState<LLMUsageStats | null>(null)
const [loading, setLoading] = useState(true)
const [testing, setTesting] = useState(false)
const [testResult, setTestResult] = useState<{ success: boolean; message: string } | null>(null)
// LLM 连接配置(运行时配置,DB 覆盖 .env)
const [llmSettings, setLlmSettings] = useState<DataSourceSetting[]>([])
const [settingsLoading, setSettingsLoading] = useState(true)
const [editingKey, setEditingKey] = useState<string | null>(null)
const [busyKey, setBusyKey] = useState<string | null>(null)
const [rowNotice, setRowNotice] = useState<{ key: string; ok: boolean; text: string } | null>(null)
const loadStats = useCallback(async () => {
setLoading(true)
try {
const data = await fetchLLMUsageStats()
setStats(data)
} catch {
setStats(null)
} finally {
setLoading(false)
}
}, [])
const loadSettings = useCallback(async () => {
setSettingsLoading(true)
try {
const all = await fetchSettings()
setLlmSettings(all.filter(x => LLM_SETTING_KEYS.includes(x.key)))
} catch {
setLlmSettings([])
} finally {
setSettingsLoading(false)
}
}, [])
useEffect(() => {
loadStats()
loadSettings()
}, [loadStats, loadSettings])
/** 供 LLM_MODEL 行内检测:探测当前服务可用模型,失败抛错由行内展示 */
const detectLLMModels = useCallback(async (): Promise<string[]> => {
const r = await fetchLLMModels()
if (!r.ok) throw new Error(r.detail)
return r.models
}, [])
async function handleSave(key: string, value: string) {
setBusyKey(key)
setRowNotice(null)
try {
await updateSetting(key, value)
setRowNotice({ key, ok: true, text: '已保存,立即生效' })
setEditingKey(null)
await loadSettings()
} catch (err) {
setRowNotice({ key, ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '保存失败' })
} finally {
setBusyKey(null)
}
}
async function handleClear(key: string) {
setBusyKey(key)
setRowNotice(null)
try {
await clearSetting(key)
setRowNotice({ key, ok: true, text: '已清除数据库覆盖,回落 .env 默认值' })
await loadSettings()
} catch (err) {
setRowNotice({ key, ok: false, text: err instanceof Error ? err.message.split('\n')[0] : '清除失败' })
} finally {
setBusyKey(null)
}
}
async function handleTest() {
setTesting(true)
setTestResult(null)
try {
await testLLMConnection()
setTestResult({ success: true, message: 'LLM 连接测试成功' })
} catch (err: unknown) {
const msg = err instanceof Error ? err.message : 'LLM 连接测试失败'
setTestResult({ success: false, message: msg })
} finally {
setTesting(false)
}
}
return (
<div className="space-y-6">
<SectionHeader
title="LLM 配置"
description="大语言模型连接状态与使用统计。模型切换通过修改 .env 并重启服务完成。"
/>
<div className="grid gap-6 lg:grid-cols-2">
{/* LLM 连接配置 */}
<Card>
<CardHeader
title="连接配置"
description="保存到数据库并立即生效,优先于服务器 .env"
action={
<button onClick={loadSettings} disabled={settingsLoading} className="btn btn-sm">
{settingsLoading ? (<><Spinner /> </>) : '刷新'}
</button>
}
/>
<CardBody>
{settingsLoading ? (
<div className="space-y-3">
{[1, 2, 3].map(i => <SkeletonBlock key={i} className="h-9 w-full" />)}
</div>
) : (
<div>
{llmSettings.map(setting => (
<SettingRow
key={setting.key}
setting={setting}
editing={editingKey === setting.key}
busy={busyKey === setting.key}
onEdit={() => {
setEditingKey(setting.key)
setRowNotice(null)
}}
onCancel={() => setEditingKey(null)}
onSave={v => handleSave(setting.key, v)}
onClear={() => handleClear(setting.key)}
detectModels={setting.key === 'LLM_MODEL' ? detectLLMModels : undefined}
/>
))}
</div>
)}
{rowNotice && (
<div className="mt-3">
<Alert kind={rowNotice.ok ? 'ok' : 'error'} title={rowNotice.text} />
</div>
)}
<p className="mt-3 border-l-2 border-ink-300 pl-3 text-2xs leading-relaxed text-ink-500">
模式: 多专家 (5 + ) OpenAI ( DeepSeek
) LLM
</p>
{/* 测试连接 */}
{testResult && (
<div className="mt-4">
<Alert
kind={testResult.success ? 'ok' : 'error'}
title={testResult.success ? '连接正常' : '连接失败'}
message={testResult.success ? undefined : testResult.message}
/>
</div>
)}
<button onClick={handleTest} disabled={testing} className="btn btn-sm mt-4 w-full">
{testing ? (<><Spinner /> </>) : '测试 LLM 连接'}
</button>
<p className="mt-2 text-center text-2xs text-ink-400">
LLM ,
</p>
</CardBody>
</Card>
{/* 使用统计 */}
<Card>
<CardHeader
title="使用统计"
description="从最近预测记录聚合"
action={
<button onClick={loadStats} disabled={loading} className="btn btn-sm">
{loading ? (<><Spinner /> </>) : '刷新'}
</button>
}
/>
<CardBody>
{loading ? (
<div className="space-y-3">
<SkeletonBlock className="h-16 w-full" />
<SkeletonBlock className="h-16 w-full" />
</div>
) : stats ? (
<div className="grid grid-cols-3 gap-4">
<div className="border-t-2 border-ink-900 pt-3 text-center">
<div className="font-serif text-2xl font-bold tabular-nums text-ink-900">
{stats.total_predictions}
</div>
<div className="mt-1 text-2xs text-ink-400"></div>
</div>
<div className="border-t-2 border-ink-900 pt-3 text-center">
<div className="font-serif text-2xl font-bold tabular-nums text-ink-900">
{stats.avg_latency_ms > 0 ? `${(stats.avg_latency_ms / 1000).toFixed(1)}s` : '—'}
</div>
<div className="mt-1 text-2xs text-ink-400"></div>
</div>
<div className="border-t-2 border-press pt-3 text-center">
<div className="font-serif text-2xl font-bold tabular-nums text-press">
{stats.success_rate.toFixed(0)}%
</div>
<div className="mt-1 text-2xs text-ink-400"></div>
</div>
</div>
) : (
<p className="py-6 text-center text-xs text-ink-400">使</p>
)}
</CardBody>
</Card>
</div>
{/* 专家与终裁独立配置 */}
<AgentLLMCard />
{/* 最近预测 */}
<Card>
<CardHeader title="最近预测记录" />
<CardBody className="px-0 sm:px-0">
{loading ? (
<div className="space-y-2 px-4 sm:px-5">
{[1, 2, 3].map(i => (
<SkeletonBlock key={i} className="h-10 w-full" />
))}
</div>
) : stats && stats.recent_predictions.length > 0 ? (
<div>
{stats.recent_predictions.map(p => (
<div
key={p.id}
className="flex flex-col gap-1.5 border-b border-ink-200 px-4 py-2.5 last:border-b-0 sm:flex-row sm:items-center sm:justify-between sm:px-5"
>
<div className="flex items-baseline gap-3">
<span className="text-2xs tabular-nums text-ink-400">#{p.id}</span>
<span className="text-xs text-ink-800"> #{p.match_id}</span>
<span className="font-mono text-2xs text-ink-500">{p.model}</span>
</div>
<div className="flex items-center gap-3">
<span className="text-2xs tabular-nums text-ink-400">
{p.created_at ? new Date(p.created_at).toLocaleString('zh-CN', { month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit' }) : '—'}
</span>
{p.status === 'success' ? <Badge status="success"></Badge> : <Badge status="error"></Badge>}
</div>
</div>
))}
</div>
) : (
<p className="py-8 text-center text-xs text-ink-400"></p>
)}
</CardBody>
</Card>
</div>
)
}
+9 -8
View File
@@ -53,20 +53,21 @@ export default function LogsPage() {
}, [load])
const scrollRef = useRef<HTMLDivElement>(null)
const userScrolledUp = useRef(false)
const userScrolledDown = useRef(false)
// 检测用户是否向滚动
// 检测用户是否向滚动回看历史(旧日志在下方)
const handleScroll = () => {
const el = scrollRef.current
if (!el) return
const atBottom = el.scrollHeight - el.scrollTop - el.clientHeight < 50
userScrolledUp.current = !atBottom
const atTop = el.scrollTop < 50
userScrolledDown.current = !atTop
}
// 加载后自动滚动到底部(仅当用户未向上滚动时)
// 列表最新在最上面(后端倒序返回);打开页面/自动刷新时定位到顶部=最新。
// 用户向下滚动回看历史时暂停定位,拉回顶部即恢复。
useEffect(() => {
if (autoRefresh && !userScrolledUp.current && scrollRef.current) {
scrollRef.current.scrollTop = scrollRef.current.scrollHeight
if (autoRefresh && !userScrolledDown.current && scrollRef.current) {
scrollRef.current.scrollTop = 0
}
}, [entries, autoRefresh])
@@ -85,7 +86,7 @@ export default function LogsPage() {
<div className="space-y-6">
<SectionHeader
title="系统日志"
description="应用运行日志(登录、配置变更、采集、预测、异常等)。内存缓冲最近 2000 条,重启后清零。"
description="应用运行日志(登录、配置变更、采集、预测、异常等)。内存缓冲最近 2000 条,重启后清零;若后端已配置 LOG_FILE,完整日志同时滚动写入服务器文件(单文件 10MB × 5 份),可登录宿主机查看。"
/>
{error && <Alert kind="error" title="无法加载日志" message={error} />}
@@ -90,7 +90,7 @@ export default function PredictionHistoryPage() {
</Card>
<Card>
<CardBody className="text-center">
<p className="text-2xl font-bold text-emerald-700">{hitCount}</p>
<p className="text-2xl font-bold text-ok-700">{hitCount}</p>
<p className="text-xs text-ink-500"></p>
</CardBody>
</Card>
@@ -194,7 +194,7 @@ export default function PredictionHistoryPage() {
<td className="px-4 py-3">
{p.pred_1x2 ? (
<span className={`inline-block border px-1.5 py-0.5 text-2xs ${
predHit === true ? 'border-emerald-300 text-emerald-700 bg-emerald-50' :
predHit === true ? 'border-ok-300 text-ok-700 bg-ok-50' :
predHit === false ? 'border-press text-press bg-press-wash' :
'border-ink-200 text-ink-600'
}`}>
-363
View File
@@ -1,363 +0,0 @@
/**
* Admin 后台 - 预测管理页面(报刊风)
*
* 功能:
* - 触发预测(选择比赛 + 模式)
* - 结算:录入实际比分,写入评估(接 /eval/settle)
* - 预测记录列表:可展开查看终裁理由与专家摘要
*
* 响应式布局: 移动端单列,桌面端双列
*/
import { useCallback, useEffect, useMemo, useState } from 'react'
import { triggerPrediction, fetchPredictions, fetchMatches, settlePrediction } from '../dal'
import type { Match, Prediction } from '../types'
import { Card, CardBody, CardHeader, Badge, SectionHeader, Alert, Spinner } from '../components'
import { teamSidePrefix } from '../../components/TeamSideTag'
import { AgentWeightsBar } from '../components'
const AGENT_LABELS: Record<string, string> = {
h2h: '历史交锋分析专家',
form: '近期状态分析专家',
stats: '攻防数据分析专家',
home_away: '主客因素分析专家',
injuries: '阵容完整性分析专家',
}
const OUTCOME_LABEL: Record<string, string> = { '1': '主胜', X: '平局', '2': '客胜' }
function fmtTime(s?: string | null): string {
if (!s) return '—'
const d = new Date(s)
return isNaN(d.getTime())
? s
: d.toLocaleString('zh-CN', { month: '2-digit', day: '2-digit', hour: '2-digit', minute: '2-digit' })
}
export default function PredictionsPage() {
const [matches, setMatches] = useState<Match[]>([])
const [predictions, setPredictions] = useState<Prediction[]>([])
const [matchId, setMatchId] = useState('')
const [mode, setMode] = useState<'single' | 'multi'>('multi')
const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null)
const [successMsg, setSuccessMsg] = useState<string | null>(null)
// 结算表单
const [settleId, setSettleId] = useState('')
const [homeGoals, setHomeGoals] = useState('')
const [awayGoals, setAwayGoals] = useState('')
const [settling, setSettling] = useState(false)
const [settleMsg, setSettleMsg] = useState<{ kind: 'error' | 'ok'; text: string } | null>(null)
const refreshPredictions = useCallback(async () => {
const list = await fetchPredictions(50)
setPredictions(list)
}, [])
useEffect(() => {
refreshPredictions()
fetchMatches({ limit: 100 }).then(d => setMatches(d.items))
}, [refreshPredictions])
/** match_id → 中文名对阵 */
const matchName = useMemo(() => {
const map = new Map<number, string>()
for (const m of matches) {
const home = m.home_team_zh || m.home_team
const away = m.away_team_zh || m.away_team
map.set(m.id, `${teamSidePrefix('home')}${home} vs ${teamSidePrefix('away')}${away}`)
}
return map
}, [matches])
const nameOf = (id: number) => matchName.get(id) ?? `比赛 #${id}`
async function handlePredict(e: React.FormEvent) {
e.preventDefault()
if (!matchId) return
setLoading(true)
setError(null)
setSuccessMsg(null)
try {
await triggerPrediction({ match_id: parseInt(matchId), mode })
setSuccessMsg('预测任务已完成,记录已更新')
await refreshPredictions()
} catch (err: unknown) {
setError(err instanceof Error ? err.message : '预测失败')
} finally {
setLoading(false)
}
}
const unsettled = predictions.filter(p => !p.settled)
async function handleSettle(e: React.FormEvent) {
e.preventDefault()
const pid = parseInt(settleId)
const hg = parseInt(homeGoals)
const ag = parseInt(awayGoals)
if (!pid || isNaN(hg) || isNaN(ag)) return
setSettling(true)
setSettleMsg(null)
try {
await settlePrediction(pid, hg, ag)
setSettleMsg({ kind: 'ok', text: '结算完成,准确率统计已更新' })
setSettleId('')
setHomeGoals('')
setAwayGoals('')
await refreshPredictions()
} catch (err: unknown) {
setSettleMsg({
kind: 'error',
text: err instanceof Error ? err.message : '结算失败',
})
} finally {
setSettling(false)
}
}
const settleTarget = predictions.find(p => p.id === parseInt(settleId))
return (
<div className="space-y-6">
<SectionHeader
title="预测管理"
description="触发 LLM 预测;赛后录入实际比分完成结算,供准确率统计使用。"
/>
<div className="grid gap-6 lg:grid-cols-2">
{/* 新建预测 */}
<Card>
<CardHeader title="新建预测" />
<CardBody>
<form onSubmit={handlePredict} className="space-y-4">
<div>
<label className="mb-1.5 block text-xs text-ink-500"></label>
<select
value={matchId}
onChange={e => setMatchId(e.target.value)}
className="field w-full"
>
<option value=""></option>
{matches.map(m => (
<option key={m.id} value={m.id}>
{teamSidePrefix('home')}{(m.home_team_zh || m.home_team)} vs {teamSidePrefix('away')}{(m.away_team_zh || m.away_team)}
({m.match_date?.slice(5, 10)})
</option>
))}
</select>
</div>
<div>
<label className="mb-1.5 block text-xs text-ink-500"></label>
<select
value={mode}
onChange={e => setMode(e.target.value as 'single' | 'multi')}
className="field w-full"
>
<option value="multi"> (5 + ,)</option>
<option value="single"> ()</option>
</select>
</div>
{error && <Alert kind="error" title="预测失败" message={error} onClose={() => setError(null)} />}
{successMsg && (
<Alert kind="ok" title={successMsg} onClose={() => setSuccessMsg(null)} />
)}
<button
type="submit"
disabled={loading || !matchId}
className="btn btn-solid w-full"
>
{loading ? (<><Spinner /> , 20-60 </>) : '触发预测'}
</button>
</form>
</CardBody>
</Card>
{/* 结算 */}
<Card>
<CardHeader
title="预测结算"
description="录入实际比分,系统据此统计 1X2 准确率与比分 RMSE"
/>
<CardBody>
{unsettled.length === 0 ? (
<p className="py-6 text-center text-xs text-ink-400">
</p>
) : (
<form onSubmit={handleSettle} className="space-y-4">
<div>
<label className="mb-1.5 block text-xs text-ink-500"></label>
<select
value={settleId}
onChange={e => setSettleId(e.target.value)}
className="field w-full"
>
<option value="">({unsettled.length} )</option>
{unsettled.map(p => (
<option key={p.id} value={p.id}>
#{p.id} {nameOf(p.match_id)} · {p.pred_home_goals ?? '-'}:{p.pred_away_goals ?? '-'}
</option>
))}
</select>
</div>
{settleTarget && (
<p className="border-l-2 border-ink-300 pl-3 text-2xs text-ink-500">
:{settleTarget.pred_home_goals ?? '-'} : {settleTarget.pred_away_goals ?? '-'}
({OUTCOME_LABEL[settleTarget.pred_1x2 ?? ''] ?? '?'})
<span className="ml-2">{fmtTime(settleTarget.created_at)}</span>
</p>
)}
<div className="grid grid-cols-2 gap-3">
<div>
<label className="mb-1.5 block text-xs text-ink-500"></label>
<input
type="number"
min={0}
max={30}
value={homeGoals}
onChange={e => setHomeGoals(e.target.value)}
className="field w-full"
/>
</div>
<div>
<label className="mb-1.5 block text-xs text-ink-500"></label>
<input
type="number"
min={0}
max={30}
value={awayGoals}
onChange={e => setAwayGoals(e.target.value)}
className="field w-full"
/>
</div>
</div>
{settleMsg && (
<Alert
kind={settleMsg.kind}
title={settleMsg.kind === 'ok' ? '结算完成' : '结算失败'}
message={settleMsg.kind === 'error' ? settleMsg.text : undefined}
onClose={() => setSettleMsg(null)}
/>
)}
<button
type="submit"
disabled={settling || !settleId || homeGoals === '' || awayGoals === ''}
className="btn btn-solid w-full"
>
{settling ? (<><Spinner /> </>) : '提交结算'}
</button>
</form>
)}
</CardBody>
</Card>
</div>
{/* 最近预测 */}
<Card>
<CardHeader title="预测记录" description="点击行可展开终裁理由与专家摘要" />
<CardBody className="px-0 sm:px-0">
{predictions.length === 0 ? (
<p className="py-10 text-center text-xs text-ink-400">
,
</p>
) : (
<div>
{predictions.map(p => {
const okAgents = (p.agent_outputs ?? []).filter(a => a.status === 'ok')
return (
<details key={p.id} className="group border-b border-ink-200 last:border-b-0">
<summary className="flex cursor-pointer list-none flex-wrap items-baseline gap-x-3 gap-y-1 px-4 py-3 transition-colors hover:bg-paper-100 sm:px-5">
<span className="text-2xs tabular-nums text-ink-400">#{p.id}</span>
<span className="text-sm font-medium text-ink-900">{nameOf(p.match_id)}</span>
<span className="font-serif text-sm font-bold tabular-nums text-ink-900">
{p.pred_home_goals ?? '-'}<span className="mx-0.5 font-normal text-ink-300">:</span>{p.pred_away_goals ?? '-'}
</span>
<span className="text-2xs text-ink-500">
{OUTCOME_LABEL[p.pred_1x2 ?? ''] ?? '—'}
{p.subjective_confidence !== null && p.subjective_confidence !== undefined &&
` · ${Math.round(p.subjective_confidence * 100)}%`}
</span>
<span className="ml-auto flex items-baseline gap-3">
{p.settled ? (
<Badge status="success"></Badge>
) : (
<Badge status="pending"></Badge>
)}
{p.status === 'degraded' && (
<Badge status="error">·</Badge>
)}
{p.status === 'failed' && (
<Badge status="error"></Badge>
)}
<span className="text-2xs tabular-nums text-ink-400">{fmtTime(p.created_at)}</span>
<svg
viewBox="0 0 20 20"
className="h-3 w-3 self-center text-ink-300 transition-transform group-open:rotate-90"
fill="currentColor"
aria-hidden="true"
>
<path d="M7.3 5.3a1 1 0 011.4 0l4 4a1 1 0 010 1.4l-4 4a1 1 0 01-1.4-1.4L10.6 10 7.3 6.7a1 1 0 010-1.4z" />
</svg>
</span>
</summary>
<div className="space-y-3 px-4 pb-4 pl-8 sm:px-6 sm:pl-9">
<p className="text-2xs text-ink-500">
{p.mode === 'multi' ? `多专家 · ${okAgents.length}/${p.agent_outputs?.length ?? 0} 路有效` : '单次模式'}
{p.model && <span className="ml-2 font-mono">{p.model}</span>}
</p>
{p.reasoning && (
<blockquote className="border-l-2 border-press pl-4">
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">
{p.reasoning}
</p>
</blockquote>
)}
{p.agent_weights && Object.keys(p.agent_weights).length > 0 && (
<AgentWeightsBar weights={p.agent_weights} okCount={okAgents.length} />
)}
{p.agent_outputs && p.agent_outputs.length > 0 && (
<ul className="space-y-1">
{p.agent_outputs.map((a, i) => (
<li key={i} className="flex items-baseline gap-2.5 text-xs">
<span className={`inline-block h-1.5 w-1.5 flex-shrink-0 self-center ${a.status === 'ok' ? 'bg-ink-900' : 'bg-ink-300'}`} aria-hidden="true" />
<span className="text-ink-800">{AGENT_LABELS[a.agent] ?? a.agent}</span>
{a.probable_score && (
<span className="font-serif font-bold tabular-nums text-ink-800">{a.probable_score}</span>
)}
<span className="text-2xs text-ink-400">
{a.status === 'ok' ? '' : a.status === 'no_data' ? '无数据' : '失败'}
</span>
</li>
))}
</ul>
)}
{p.settled && (
<p className="border-t border-ink-100 pt-2.5 text-2xs text-ink-500">
{p.actual_home_goals ?? '-'} : {p.actual_away_goals ?? '-'}
</p>
)}
</div>
</details>
)
})}
</div>
)}
</CardBody>
</Card>
</div>
)
}
+49 -8
View File
@@ -9,6 +9,7 @@
*/
import { useEffect, useState, useCallback } from 'react'
import { useSearchParams } from 'react-router-dom'
import {
fetchSettings, updateSetting, clearSetting,
testLLMConnection, fetchLLMUsageStats, fetchLLMModels,
@@ -26,7 +27,24 @@ import AgentLLMCard from '../AgentLLMCard'
const DATA_SOURCE_KEYS = ['BZZOIRO_KEY', 'BZZOIRO_BASE']
const LLM_KEYS = ['LLM_API_KEY', 'LLM_BASE_URL', 'LLM_MODEL']
/** 设置页分区(tab)。低频/高危操作靠后:安全放最后。 */
const TABS = [
{ id: 'datasource', label: '数据源' },
{ id: 'llm', label: '大语言模型' },
{ id: 'schedules', label: '定时任务' },
{ id: 'security', label: '登录认证' },
] as const
type TabId = (typeof TABS)[number]['id']
export default function SettingsPage() {
// tab 状态写入 URL(?tab=llm),可深链直达、刷新保持
const [searchParams, setSearchParams] = useSearchParams()
const rawTab = searchParams.get('tab')
const tab: TabId = TABS.some(t => t.id === rawTab) ? (rawTab as TabId) : 'datasource'
const setTab = (id: TabId) =>
setSearchParams(id === 'datasource' ? {} : { tab: id }, { replace: true })
const [allSettings, setAllSettings] = useState<DataSourceSetting[]>([])
const [settingsLoading, setSettingsLoading] = useState(true)
const [editingKey, setEditingKey] = useState<string | null>(null)
@@ -229,12 +247,31 @@ export default function SettingsPage() {
<div className="space-y-8">
<SectionHeader
title="系统设置"
description="数据源、LLM、认证等全部配置。保存到数据库并立即生效,优先于 .env。"
description="数据源、LLM、定时任务与登录认证。保存到数据库并立即生效,优先于 .env。"
/>
{/* ── 分区 tab(状态在 URL 上,可深链) ── */}
<div className="-mt-4 flex gap-5 overflow-x-auto border-b border-ink-200" role="tablist" aria-label="设置分区">
{TABS.map(t => (
<button
key={t.id}
role="tab"
aria-selected={tab === t.id}
onClick={() => setTab(t.id)}
className={`-mb-px flex-shrink-0 border-b-2 pb-2 text-sm transition-colors ${
tab === t.id
? 'border-press font-bold text-press'
: 'border-transparent text-ink-500 hover:text-ink-900'
}`}
>
{t.label}
</button>
))}
</div>
{/* ── 1. 数据源 ── */}
{tab === 'datasource' && (
<section>
<h2 className="section-head mb-3"></h2>
<div className="grid gap-6 lg:grid-cols-2">
<Card>
<CardHeader
@@ -280,11 +317,11 @@ export default function SettingsPage() {
return (
<div key={i} className={`flex items-center justify-between gap-3 border-b border-ink-100 py-2 last:border-b-0 ${isBlocked ? 'opacity-70' : ''}`}>
<div className="flex items-center gap-2">
<span className={`inline-block h-2 w-2 rounded-full ${isBlocked ? 'bg-amber-500' : 'bg-emerald-500'}`} />
<span className={`inline-block h-2 w-2 rounded-full ${isBlocked ? 'bg-warn-500' : 'bg-ok-500'}`} />
<span className="font-mono text-xs text-ink-700">{k.masked}</span>
{i === keyRing.active_index && <span className="rounded bg-ink-900 px-1.5 py-0.5 text-2xs text-paper-500"></span>}
</div>
<span className={`text-2xs tabular-nums ${isBlocked ? 'text-amber-600' : 'text-ink-400'}`}>
<span className={`text-2xs tabular-nums ${isBlocked ? 'text-warn-700' : 'text-ink-400'}`}>
{isBlocked ? `冷却中 ${k.blocked_remaining.toFixed(0)}s` : '可用'}
</span>
</div>
@@ -298,10 +335,11 @@ export default function SettingsPage() {
</Card>
</div>
</section>
)}
{/* ── 2. LLM ── */}
{tab === 'llm' && (
<section>
<h2 className="section-head mb-3"></h2>
<div className="grid gap-6 lg:grid-cols-2">
<Card>
<CardHeader
@@ -372,10 +410,11 @@ export default function SettingsPage() {
<AgentLLMCard />
</div>
</section>
)}
{/* ── 3. 认证 ── */}
{tab === 'security' && (
<section>
<h2 className="section-head mb-3"></h2>
<Card>
<CardHeader title="修改密码" description="密码即会话签名密钥,修改后所有已登录会话失效,需重新登录" />
<CardBody>
@@ -413,10 +452,11 @@ export default function SettingsPage() {
</CardBody>
</Card>
</section>
)}
{/* ── 4. 定时任务 ── */}
{tab === 'schedules' && (
<section>
<h2 className="section-head mb-3"></h2>
<Card>
<CardHeader
title="采集调度"
@@ -449,7 +489,7 @@ export default function SettingsPage() {
<div key={s.id} className="flex flex-col gap-2 border-b border-ink-100 py-3 last:border-b-0 sm:flex-row sm:items-center sm:justify-between">
<div className="flex-1 space-y-1">
<div className="flex items-center gap-2">
<span className={`inline-block h-2 w-2 rounded-full ${s.enabled ? 'bg-emerald-500' : 'bg-ink-300'}`} />
<span className={`inline-block h-2 w-2 rounded-full ${s.enabled ? 'bg-ok-500' : 'bg-ink-300'}`} />
<span className="text-xs font-medium text-ink-800">{s.id}</span>
<Badge status={s.enabled ? 'success' : 'muted'}>{s.task}</Badge>
</div>
@@ -491,6 +531,7 @@ export default function SettingsPage() {
</CardBody>
</Card>
</section>
)}
</div>
)
}
+14
View File
@@ -283,6 +283,20 @@ export interface IngestSourceStatus {
last_failure: IngestLastFailure | null
}
// ── 采集任务状态 ──────────────────────────────────────────────
export interface IngestJob {
id: string
task: string
params: Record<string, unknown>
status: 'pending' | 'running' | 'success' | 'failed'
result: Record<string, unknown> | null
error: string | null
created_at: string | null
started_at: string | null
finished_at: string | null
}
// ── 比赛详情 ─────────────────────────────────────────────────────
export interface MatchRecentPrediction {
+30
View File
@@ -0,0 +1,30 @@
/**
* 回到顶部浮动按钮(前台两页共用,此前 Matches/Standings 各复制一份)。
* 方角纸片风:去掉早期版本的 rounded-full + shadow-lg,与全站方角
* 无阴影语言对齐;出现/隐藏仅动画 transform 与 opacity。
*/
import { useEffect, useState } from 'react'
export default function BackTop({ threshold = 300 }: { threshold?: number }) {
const [show, setShow] = useState(false)
useEffect(() => {
const handleScroll = () => setShow(window.scrollY > threshold)
window.addEventListener('scroll', handleScroll, { passive: true })
return () => window.removeEventListener('scroll', handleScroll)
}, [threshold])
return (
<button
onClick={() => window.scrollTo({ top: 0, behavior: 'smooth' })}
className={`fixed bottom-6 right-6 z-40 flex h-10 w-10 items-center justify-center border border-ink-300 bg-paper-50 text-ink-600 transition-all duration-300 hover:border-ink-900 hover:bg-ink-900 hover:text-paper-50 ${
show ? 'translate-y-0 opacity-100' : 'pointer-events-none translate-y-4 opacity-0'
}`}
aria-label="回到顶部"
>
<svg className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M5 15l7-7 7 7" />
</svg>
</button>
)
}
+77
View File
@@ -0,0 +1,77 @@
/**
* 报头组件(前台共用)。
*
* 此前 HomePage 与 StandingsLayout 各自复制一份报头,导航项已经漂移
* (首页有「积分榜」、积分榜页有「比赛/预测」),且无当前页高亮。
* 现在统一为 Masthead:
* - active 声明当前版面,对应导航项加印报红高亮 + aria-current
* - 「管理」入口改用与全站一致的线条 SVG(替代字符 ⚙)
* - 管理类入口带 title 提示,避免访客被无声踢到登录页
* 页脚文案各页不同,仍由调用方自行渲染。
*/
import { Link } from 'react-router-dom'
export type MastheadActive = 'home' | 'standings'
function GearIcon() {
return (
<svg className="h-3 w-3" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2} aria-hidden="true">
<circle cx="12" cy="12" r="3" />
<path d="M19.4 15a1.65 1.65 0 0 0 .33 1.82l.06.06a2 2 0 0 1 0 2.83 2 2 0 0 1-2.83 0l-.06-.06a1.65 1.65 0 0 0-1.82-.33 1.65 1.65 0 0 0-1 1.51V21a2 2 0 0 1-4 0v-.09A1.65 1.65 0 0 0 9 19.4a1.65 1.65 0 0 0-1.82.33l-.06.06a2 2 0 0 1-2.83 0 2 2 0 0 1 0-2.83l.06-.06A1.65 1.65 0 0 0 4.68 15a1.65 1.65 0 0 0-1.51-1H3a2 2 0 0 1 0-4h.09A1.65 1.65 0 0 0 4.6 9a1.65 1.65 0 0 0-.33-1.82l-.06-.06a2 2 0 0 1 0-2.83 2 2 0 0 1 2.83 0l.06.06A1.65 1.65 0 0 0 9 4.68 1.65 1.65 0 0 0 10 3.17V3a2 2 0 0 1 4 0v.09a1.65 1.65 0 0 0 1 1.51 1.65 1.65 0 0 0 1.82-.33l.06-.06a2 2 0 0 1 2.83 0 2 2 0 0 1 0 2.83l-.06.06A1.65 1.65 0 0 0 19.4 9a1.65 1.65 0 0 0 1.51 1H21a2 2 0 0 1 0 4h-.09a1.65 1.65 0 0 0-1.51 1z" />
</svg>
)
}
function MastheadLink({
to,
active = false,
title,
children,
}: {
to: string
/** 当前版面才高亮;管理类入口无高亮概念 */
active?: boolean
title?: string
children: React.ReactNode
}) {
return (
<Link
to={to}
title={title}
aria-current={active ? 'page' : undefined}
className={`transition-colors ${
active ? 'font-medium text-press' : 'text-ink-500 hover:text-press'
}`}
>
{children}
</Link>
)
}
export default function Masthead({ active }: { active: MastheadActive }) {
return (
<header className="masthead-rule">
<div className="mx-auto max-w-5xl px-5 sm:px-8">
<div className="border-b border-ink-900 py-5 text-center sm:py-6">
<h1 className="font-brush text-5xl text-ink-900 sm:text-6xl">
</h1>
</div>
<div className="flex items-center justify-between border-b border-ink-200 py-2 text-2xs text-ink-500">
<span>{new Date().toLocaleDateString('zh-CN', { year: 'numeric', month: 'long', day: 'numeric', weekday: 'long' })}</span>
<nav className="flex items-center gap-4" aria-label="页面导航">
<MastheadLink to="/" active={active === 'home'}> / </MastheadLink>
<MastheadLink to="/standings" active={active === 'standings'}></MastheadLink>
{/* 管理类页面对访客需要登录,入口处先说明,避免无声跳登录页 */}
<MastheadLink to="/admin/eval" title="评估页面向管理员开放,需登录"></MastheadLink>
<MastheadLink to="/admin" title="管理后台,需登录">
<span className="inline-flex items-center gap-1">
<GearIcon />
</span>
</MastheadLink>
</nav>
</div>
</div>
</header>
)
}
+22
View File
@@ -103,6 +103,28 @@
.btn-danger {
@apply border-press bg-transparent text-press hover:bg-press hover:text-paper-50;
}
/* 描边确认按钮:次级动作中需要比默认 btn 更明确轮廓的场合
(此前三处使用但从未定义,样式静默失效) */
.btn-outline {
@apply border-ink-900 bg-transparent text-ink-900
hover:border-press hover:bg-press-wash hover:text-press-dark;
}
/* ── 弹窗入场:遮罩淡入 + 面板上浮(仅 opacity/transform,GPU 友好) ── */
.modal-overlay-enter {
animation: modal-fade 0.18s ease-out both;
}
.modal-panel-enter {
animation: modal-rise 0.2s cubic-bezier(0.22, 1, 0.36, 1) both;
}
@keyframes modal-fade {
from { opacity: 0; }
to { opacity: 1; }
}
@keyframes modal-rise {
from { opacity: 0; transform: translateY(12px); }
to { opacity: 1; transform: translateY(0); }
}
/* ── 统一空态 ── */
.empty-state {
+83
View File
@@ -0,0 +1,83 @@
/**
* P0-00: HTTP client method/body/headers 可信度测试。
* 运行: node --experimental-strip-types frontend/src/lib/http.test.ts
*
* 最小环境 polyfill:Node 22 自带 fetch/AbortController,本测试不触发 401 路径,
* 故 window.dispatchEvent 不会被调用,无需完整 DOM。
*/
import { test } from 'node:test'
import assert from 'node:assert/strict'
// 最小浏览器环境 polyfill(仅覆盖 http.ts 在 happy path 用到的全局)
const store: Record<string, string> = {}
// @ts-expect-error 测试用最小 window stub
globalThis.window = {
dispatchEvent: () => false,
localStorage: {
getItem: (k: string) => store[k] ?? null,
setItem: (k: string, v: string) => { store[k] = v },
removeItem: (k: string) => { delete store[k] },
},
}
// 捕获每次 fetch 的入参供断言
let lastUrl = ''
let lastInit: RequestInit | undefined
globalThis.fetch = async (url: string | URL | Request, init?: RequestInit) => {
lastUrl = String(url)
lastInit = init
return new Response(JSON.stringify({ ok: true }), { status: 200, headers: { 'content-type': 'application/json' } })
}
const { http } = await import('./http.ts')
test('GET: method=GET, 无 body, 无 Content-Type', async () => {
await http.get('/api/v1/matches')
assert.equal(lastInit?.method, 'GET')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
test('POST: method=POST, 序列化 body, 有 Content-Type', async () => {
await http.post('/api/v1/matches', { a: 1 })
assert.equal(lastInit?.method, 'POST')
assert.equal(lastInit?.body, JSON.stringify({ a: 1 }))
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], 'application/json')
})
test('POST 空 body: 不设 Content-Type', async () => {
await http.post('/api/v1/matches', undefined)
assert.equal(lastInit?.method, 'POST')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
test('PUT: method=PUT, 有 body 与 Content-Type', async () => {
await http.put('/api/v1/x', { b: 2 })
assert.equal(lastInit?.method, 'PUT')
assert.equal(lastInit?.body, JSON.stringify({ b: 2 }))
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], 'application/json')
})
test('DELETE: method=DELETE, 无 body, 无 Content-Type', async () => {
await http.delete('/api/v1/x/1')
assert.equal(lastInit?.method, 'DELETE')
assert.equal(lastInit?.body, undefined)
assert.equal((lastInit?.headers as Record<string, string>)?.['Content-Type'], undefined)
})
// ── 健康检查豁免: /health* 挂在根路径(后端 app.py 不在 /api/v1 下),
// 加前缀会 404 → 管理后台右上角永远「系统异常」 ──
test('build_url: /health 与 /health/ready 不加 /api/v1 前缀', async () => {
await http.get('/health')
assert.equal(lastUrl, '/health')
await http.get('/health/ready')
assert.equal(lastUrl, '/health/ready')
})
test('build_url: 常规 API 路径仍统一加 /api/v1', async () => {
await http.get('/admin/stats')
assert.equal(lastUrl, '/api/v1/admin/stats')
await http.get('/matches')
assert.equal(lastUrl, '/api/v1/matches')
})
+27 -10
View File
@@ -17,14 +17,30 @@ export const UNAUTHORIZED_EVENT = 'profeto:unauthorized'
const API_BASE = '/api/v1'
const DEFAULT_TIMEOUT = 30_000
/** 健康检查端点挂在根路径(app.py 不在 /api/v1 下,与 compose healthcheck 一致);
* 加前缀会 404,导致管理后台健康状态永远显示「系统异常」 */
const ROOT_ONLY_PREFIXES = ['/health']
/** 所有 API 路径统一走 /api/v1,避免浏览器直接请求 /matches 被 nginx 当 SPA 回退 */
function build_url(path: string): string {
if (path.startsWith('http')) return path
if (ROOT_ONLY_PREFIXES.some(p => path === p || path.startsWith(`${p}/`))) return path
if (path.startsWith(API_BASE)) return path
if (path.startsWith('/')) return `${API_BASE}${path}`
return `${API_BASE}/${path}`
}
export class ApiError extends Error {
constructor(
message: string,
public status: number,
public data?: unknown,
) {
status: number
data?: unknown
constructor(message: string, status: number, data?: unknown) {
super(message)
this.name = 'ApiError'
// 显式赋值而非构造函数参数属性(public x):strip-types 不支持后者,
// 会让 node --experimental-strip-types 跑 lib/http.test.ts 直接失败
this.status = status
this.data = data
}
}
@@ -37,7 +53,7 @@ interface RequestOptions {
}
async function request<T>(path: string, options: RequestOptions = {}): Promise<T> {
const url = path.startsWith('http') ? path : path.startsWith('/') ? path : `${API_BASE}${path}`
const url = build_url(path)
const { timeoutMs = DEFAULT_TIMEOUT, skipAuthHandling, signal } = options
const controller = new AbortController()
@@ -49,10 +65,11 @@ async function request<T>(path: string, options: RequestOptions = {}): Promise<T
}
try {
const res = await fetch(url, {
signal: controller.signal,
headers: { 'Content-Type': 'application/json' },
})
const method = (options.method ?? 'GET').toUpperCase()
const body = options.body
// 仅当有 body 时设置 Content-Type,避免 GET/DELETE 等无 body 请求被误标
const headers: Record<string, string> = body ? { 'Content-Type': 'application/json' } : {}
const res = await fetch(url, { signal: controller.signal, method, body, headers })
if (!res.ok) {
const rawText = await res.text()
+48 -43
View File
@@ -10,9 +10,11 @@
* matches/components/MatchDetailSection.tsx — 赛程行 + 展开详情
* 本文件只负责状态装配与版面组织,不含数据获取与展示细节。
*/
import { useEffect, useState } from 'react'
import { useEffect, useRef, useState } from 'react'
import { Link } from 'react-router-dom'
import { fetchMatchDetail, fetchMatchContext } from '../admin/dal'
import type { MatchDetailOut, MatchContextOut } from '../admin/types'
import BackTop from '../components/BackTop'
import { useMatchesList } from './matches/hooks/useMatchesList'
import { useMatchPredict } from './matches/hooks/useMatchPredict'
import { useLeagues } from './matches/hooks/useLeagues'
@@ -38,13 +40,8 @@ export default function Matches() {
loadMore,
} = useMatchesList({ onError: setError })
const {
predictingId,
prediction,
predictionFor,
predict,
closePredict,
} = useMatchPredict({ onError: setError })
const { predictingId, prediction, predictionFor, predict, closePredict } =
useMatchPredict({ onError: setError })
// ── 详情展开(懒加载,只读,不触发 LLM) ──
const [expandedId, setExpandedId] = useState<number | null>(null)
@@ -52,16 +49,8 @@ export default function Matches() {
const [contextMap, setContextMap] = useState<Record<number, MatchContextOut>>({})
const [detailLoading, setDetailLoading] = useState<number | null>(null)
// 监听滚动,超过 300px 显示回到顶部按钮
const [showBackTop, setShowBackTop] = useState(false)
useEffect(() => {
const handleScroll = () => setShowBackTop(window.scrollY > 300)
window.addEventListener('scroll', handleScroll, { passive: true })
return () => window.removeEventListener('scroll', handleScroll)
}, [])
const scrollToTop = () => window.scrollTo({ top: 0, behavior: 'smooth' })
const leagueName = leagues.find(l => l.code === league)?.name ?? league
// 删除本地 showBackTop/scrollToTop,统一使用共享 BackTop 组件
/** 未开赛默认仅展示未来 3 天;其余状态展示全部。showAllUpcoming=true 时展开全部。 */
const isScheduledView = status === 'scheduled'
@@ -90,20 +79,46 @@ export default function Matches() {
}
}
// 联赛 tab 溢出检测:可向右滚动时右缘显示渐隐提示
const leagueNavRef = useRef<HTMLDivElement>(null)
const [canScrollRight, setCanScrollRight] = useState(false)
useEffect(() => {
const el = leagueNavRef.current
if (!el) return
const update = () => setCanScrollRight(el.scrollWidth - el.scrollLeft - el.clientWidth > 8)
update()
el.addEventListener('scroll', update, { passive: true })
window.addEventListener('resize', update)
return () => {
el.removeEventListener('scroll', update)
window.removeEventListener('resize', update)
}
}, [leagues])
return (
<div className="space-y-5">
{/* ── 联赛版面切换 ── */}
<nav className="flex items-center gap-6 overflow-x-auto border-b border-ink-900" aria-label="联赛">
{leagues.map(l => (
<button
key={l.code}
onClick={() => setLeague(l.code)}
className={`relative tab ${league === l.code ? 'tab-on' : ''} font-serif`}
>
{l.name}
</button>
))}
</nav>
<div className="relative">
<nav
ref={leagueNavRef}
className="flex items-center gap-6 overflow-x-auto border-b border-ink-900"
aria-label="联赛"
>
{leagues.map(l => (
<button
key={l.code}
onClick={() => setLeague(l.code)}
aria-current={league === l.code ? 'true' : undefined}
className={`relative tab ${league === l.code ? 'tab-on' : ''} font-serif`}
>
{l.name}
</button>
))}
</nav>
{canScrollRight && (
<div aria-hidden="true" className="pointer-events-none absolute inset-y-0 right-0 w-10 bg-gradient-to-l from-paper-50 to-transparent" />
)}
</div>
{/* ── 第二行:状态 / 模式 / 日期 / 计数 / 刷新(小屏 flex-wrap) ── */}
<div className="flex flex-wrap items-center gap-x-4 gap-y-2 text-xs text-ink-500 sm:gap-x-5">
@@ -120,7 +135,7 @@ export default function Matches() {
/>
</span>
<span className="ml-auto inline-flex items-center gap-3">
<span className="ml-auto inline-flex items-center gap-3 border-l border-ink-200 pl-4">
<span className="tabular-nums">
{isScheduledView && !showAllUpcoming && hasHiddenUpcoming
? `未来3天 ${visibleMatches.length} / 共 ${matches.length}`
@@ -203,9 +218,9 @@ export default function Matches() {
<>
<p className="empty-state-title"></p>
<p className="empty-state-sub"> {leagueName} </p>
<a href="/admin/collection" className="empty-state-action">
<Link to="/admin/collection" className="empty-state-action">
<span aria-hidden="true"></span>
</a>
</Link>
</>
)}
</div>
@@ -265,18 +280,8 @@ export default function Matches() {
)}
</section>
{/* 回到顶部按钮 */}
<button
onClick={scrollToTop}
className={`fixed bottom-6 right-6 z-40 flex h-10 w-10 items-center justify-center rounded-full border border-ink-200 bg-paper-50 text-ink-600 shadow-lg transition-all duration-300 hover:border-ink-400 hover:text-ink-900 ${
showBackTop ? 'translate-y-0 opacity-100' : 'translate-y-4 opacity-0 pointer-events-none'
}`}
aria-label="回到顶部"
>
<svg className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M5 15l7-7 7 7" />
</svg>
</button>
{/* ── 回到顶部(共享组件,方角纸片风) ── */}
<BackTop />
</div>
)
}
+43 -64
View File
@@ -6,40 +6,32 @@
*/
import { useEffect, useState, useCallback } from 'react'
import BackTop from '../components/BackTop'
import { fetchStandings } from '../admin/dal'
import type { StandingsLeague, StandingRow } from '../admin/dal'
import { useLeagues } from './matches/hooks/useLeagues'
import { Spinner } from '../admin/components'
const LEAGUES = [
{ code: 'E0', name: '英超' },
{ code: 'SP1', name: '西甲' },
{ code: 'D1', name: '德甲' },
{ code: 'I1', name: '意甲' },
{ code: 'F1', name: '法甲' },
{ code: 'CL', name: '欧冠' },
{ code: 'EL', name: '欧联' },
]
const ZONE_META: Record<string, { label: string; cls: string }> = {
// 欧战资格
'Champions League': { label: '欧冠区', cls: 'bg-emerald-100 text-emerald-700' },
'Champions League Qualification': { label: '欧冠资格', cls: 'bg-emerald-100 text-emerald-700' },
'Europa League': { label: '欧联区', cls: 'bg-amber-100 text-amber-700' },
'Champions League': { label: '欧冠区', cls: 'bg-ok-100 text-ok-700' },
'Champions League Qualification': { label: '欧冠资格', cls: 'bg-ok-100 text-ok-700' },
'Europa League': { label: '欧联区', cls: 'bg-warn-100 text-warn-700' },
'Conference League': { label: '欧协杯', cls: 'bg-sky-100 text-sky-700' },
'Conference League Qualification': { label: '欧协杯', cls: 'bg-sky-100 text-sky-700' },
'Europa Conference League': { label: '欧协杯', cls: 'bg-sky-100 text-sky-700' },
'Europa Conference League Qualification': { label: '欧协杯', cls: 'bg-sky-100 text-sky-700' },
// 升级
'Championship': { label: '升级区', cls: 'bg-emerald-100 text-emerald-700' },
'Promotion': { label: '升级区', cls: 'bg-emerald-100 text-emerald-700' },
'Promotion Group': { label: '升级组', cls: 'bg-emerald-100 text-emerald-700' },
'Championship': { label: '升级区', cls: 'bg-ok-100 text-ok-700' },
'Promotion': { label: '升级区', cls: 'bg-ok-100 text-ok-700' },
'Promotion Group': { label: '升级组', cls: 'bg-ok-100 text-ok-700' },
// 降级
'Relegation': { label: '降级区', cls: 'bg-rose-100 text-rose-700' },
'Relegation': { label: '降级区', cls: 'bg-bad-100 text-bad-700' },
'Relegation Playoffs': { label: '降级附加赛', cls: 'bg-orange-100 text-orange-700' },
'Relegation Group': { label: '降级组', cls: 'bg-rose-100 text-rose-700' },
'Relegation Group': { label: '降级组', cls: 'bg-bad-100 text-bad-700' },
// 附加赛
'Playoffs': { label: '附加赛', cls: 'bg-amber-100 text-amber-700' },
'Championship Playoffs': { label: '升级附加赛', cls: 'bg-amber-100 text-amber-700' },
'Playoffs': { label: '附加赛', cls: 'bg-warn-100 text-warn-700' },
'Championship Playoffs': { label: '升级附加赛', cls: 'bg-warn-100 text-warn-700' },
'Qualification Playoffs': { label: '资格附加赛', cls: 'bg-sky-100 text-sky-700' },
'Qualification': { label: '资格赛', cls: 'bg-sky-100 text-sky-700' },
}
@@ -53,7 +45,7 @@ function zoneBadge(zone?: string | null) {
/** 近期走势串(W/D/L) → 彩色圆点 */
function FormDots({ form }: { form?: string | null }) {
if (!form) return <span className="text-2xs text-ink-400"></span>
const colorMap: Record<string, string> = { W: 'bg-emerald-500', D: 'bg-ink-300', L: 'bg-rose-500' }
const colorMap: Record<string, string> = { W: 'bg-ok-500', D: 'bg-ink-300', L: 'bg-bad-500' }
return (
<span className="inline-flex gap-0.5">
{form.slice(0, 5).split('').map((c, i) => (
@@ -64,28 +56,19 @@ function FormDots({ form }: { form?: string | null }) {
}
export default function StandingsPage() {
const [leagues, setLeagues] = useState<StandingsLeague[]>([])
// 统一数据源:复用 useLeagues hook(优先 API,失败回退本地常量)
const leagues = useLeagues()
const [standings, setStandings] = useState<StandingsLeague[]>([])
const [activeLeague, setActiveLeague] = useState<string>('')
const [loading, setLoading] = useState(true)
const [switching, setSwitching] = useState(false) // 切换联赛中
const [error, setError] = useState<string | null>(null)
const [showBackTop, setShowBackTop] = useState(false) // 回到顶部按钮显示态
// 监听滚动,超过 300px 显示回到顶部按钮
useEffect(() => {
const handleScroll = () => setShowBackTop(window.scrollY > 300)
window.addEventListener('scroll', handleScroll, { passive: true })
return () => window.removeEventListener('scroll', handleScroll)
}, [])
const scrollToTop = () => window.scrollTo({ top: 0, behavior: 'smooth' })
const load = useCallback(async (code?: string) => {
setLoading(true)
setError(null)
try {
const data = await fetchStandings(code)
setLeagues(data.leagues)
setStandings(data.leagues)
if (!activeLeague && data.leagues.length > 0) {
setActiveLeague(data.leagues[0].league_code)
}
@@ -104,7 +87,7 @@ export default function StandingsPage() {
setSwitching(true)
setActiveLeague(code)
try {
await fetchStandings(code).then(data => setLeagues(data.leagues))
await fetchStandings(code).then(data => setStandings(data.leagues))
} catch (err) {
setError(err instanceof Error ? err.message : '加载失败')
} finally {
@@ -112,30 +95,36 @@ export default function StandingsPage() {
}
}
const active = leagues.find(l => l.league_code === activeLeague) ?? leagues[0]
const active = standings.find(l => l.league_code === activeLeague) ?? standings[0]
return (
<div className="space-y-6">
{/* 联赛切换 */}
<div className="flex flex-wrap gap-2">
{LEAGUES.map(l => (
<button
key={l.code}
onClick={() => switchLeague(l.code)}
disabled={switching}
className={`rounded border px-3 py-1.5 text-xs transition-colors disabled:opacity-50 ${
activeLeague === l.code
? 'border-ink-900 bg-ink-900 text-paper-50'
: 'border-ink-200 text-ink-500 hover:border-ink-300'
}`}
>
{l.name}
</button>
))}
{leagues.map(l => {
// 标记该联赛是否有积分榜数据:有数据可正常切换,无数据也可选中但显示空态
const hasData = standings.some(s => s.league_code === l.code)
const isEmpty = activeLeague === l.code && !hasData
return (
<button
key={l.code}
onClick={() => switchLeague(l.code)}
disabled={switching}
title={hasData ? undefined : '暂无积分榜数据'}
className={`rounded border px-3 py-1.5 text-xs transition-colors disabled:opacity-50 ${
activeLeague === l.code
? 'border-ink-900 bg-ink-900 text-paper-50'
: 'border-ink-200 text-ink-500 hover:border-ink-300'
} ${!hasData ? 'border-dashed' : ''}`}
>
{l.name}
</button>
)
})}
</div>
{error && (
<div className="border border-rose-300 bg-rose-50 px-4 py-3 text-sm text-rose-700">
<div className="border border-bad-300 bg-bad-50 px-4 py-3 text-sm text-bad-700">
{error}
</div>
)}
@@ -203,7 +192,7 @@ export default function StandingsPage() {
<td className="text-center py-2 text-ink-500">{r.drawn}</td>
<td className="text-center py-2 text-ink-500">{r.lost}</td>
<td className="text-center py-2 text-ink-500">{r.goals_for}/{r.goals_against}</td>
<td className={`text-center py-2 ${r.goal_diff > 0 ? 'text-emerald-600' : r.goal_diff < 0 ? 'text-rose-600' : 'text-ink-500'}`}>
<td className={`text-center py-2 ${r.goal_diff > 0 ? 'text-ok-600' : r.goal_diff < 0 ? 'text-bad-600' : 'text-ink-500'}`}>
{r.goal_diff > 0 ? `+${r.goal_diff}` : r.goal_diff}
</td>
<td className="text-center py-2 font-bold text-ink-900">{r.points}</td>
@@ -222,18 +211,8 @@ export default function StandingsPage() {
</div>
)}
{/* 回到顶部按钮 */}
<button
onClick={scrollToTop}
className={`fixed bottom-6 right-6 z-40 flex h-10 w-10 items-center justify-center rounded-full border border-ink-200 bg-paper-50 text-ink-600 shadow-lg transition-all duration-300 hover:border-ink-400 hover:text-ink-900 ${
showBackTop ? 'translate-y-0 opacity-100' : 'translate-y-4 opacity-0 pointer-events-none'
}`}
aria-label="回到顶部"
>
<svg className="h-4 w-4" fill="none" viewBox="0 0 24 24" stroke="currentColor" strokeWidth={2}>
<path strokeLinecap="round" strokeLinejoin="round" d="M5 15l7-7 7 7" />
</svg>
</button>
{/* 回到顶部(共享组件,方角纸片风) */}
<BackTop />
</div>
)
}
@@ -0,0 +1,171 @@
/**
* AgentsPanel: 五路专家意见 —— 可折叠 + 状态摘要 + 权重条形图 + 单路详情。
*
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
*/
import { useState } from 'react'
import type { AgentReport, Prediction } from '../types'
import { AGENT_LABELS, CN_NUM } from '../types'
const STATUS_BADGE: Record<string, { label: string; cls: string }> = {
ok: { label: '正常', cls: 'text-ink-500' },
no_data: { label: '无数据', cls: 'text-ink-400' },
error: { label: '调用失败', cls: 'text-press' },
parse_error: { label: '解析失败', cls: 'text-press' },
}
const SUFFICIENCY_LABEL: Record<string, string> = {
high: '充分',
medium: '一般',
low: '偏少',
none: '无',
}
/** home_edge(-1~1,正=利主队)的可视化:以中线为原点的双向细条 */
function EdgeBar({ value }: { value: number }) {
const v = Math.max(-1, Math.min(1, value))
const half = Math.abs(v) * 50
return (
<div className="relative h-px w-full bg-ink-200" role="presentation">
<span className="absolute left-1/2 top-1/2 h-2 w-px -translate-x-1/2 -translate-y-1/2 bg-ink-400" />
<span
className={`absolute top-0 h-px transition-all duration-500 ${v >= 0 ? 'bg-press' : 'bg-ink-600'}`}
style={
v >= 0
? { left: '50%', width: `${half}%` }
: { right: '50%', width: `${half}%` }
}
/>
</div>
)
}
/** 单路专家意见:汉字编号 + 细线行 */
function AgentCard({ report: r, no }: { report: AgentReport; no: string }) {
const badge = STATUS_BADGE[r.status] ?? { label: r.status, cls: 'text-ink-400' }
const inactive = r.status !== 'ok'
return (
<details className="group border-b border-ink-200">
<summary className="flex cursor-pointer list-none items-baseline gap-2.5 px-1 py-3">
<span className="font-serif text-sm text-ink-400">{no}</span>
<span className="text-sm font-medium text-ink-900">{AGENT_LABELS[r.agent] ?? r.agent}</span>
<span className={`text-2xs ${badge.cls}`}>{badge.label}</span>
<span className="ml-auto flex items-baseline gap-3 text-2xs tabular-nums text-ink-500">
{r.status === 'ok' && r.subjective_confidence !== null && (
<span> {Math.round(r.subjective_confidence * 100)}%</span>
)}
{r.status === 'ok' && r.probable_score && (
<span className="font-serif font-bold text-ink-800">{r.probable_score}</span>
)}
<svg viewBox="0 0 20 20" className="h-3 w-3 self-center text-ink-300 transition-transform group-open:rotate-90" fill="currentColor" aria-hidden="true">
<path d="M7.3 5.3a1 1 0 011.4 0l4 4a1 1 0 010 1.4l-4 4a1 1 0 01-1.4-1.4L10.6 10 7.3 6.7a1 1 0 010-1.4z" />
</svg>
</span>
</summary>
<div className="space-y-3 px-1 pb-4 pl-7">
{inactive && (
<p className="text-xs leading-relaxed text-ink-500">
{r.status === 'no_data' && '该维度没有可用数据,已跳过 LLM 分析以节省额度(不影响其他专家)。'}
{r.status === 'error' && '该专家调用失败,本次结论未纳入其视角(fail-open 设计,不阻断整体预测)。'}
{r.status === 'parse_error' && '模型输出未通过格式校验,该报告已丢弃。'}
</p>
)}
{!inactive && r.home_edge !== null && (
<div>
<div className="mb-1.5 flex items-baseline justify-between text-2xs">
<span className="text-ink-500"></span>
<span className={`font-semibold tabular-nums ${r.home_edge > 0 ? 'text-press' : r.home_edge < 0 ? 'text-ink-700' : 'text-ink-500'}`}>
{r.home_edge > 0 ? '+' : ''}{r.home_edge.toFixed(2)}
</span>
</div>
<EdgeBar value={r.home_edge} />
<div className="mt-1 flex justify-between text-2xs text-ink-400">
<span></span>
<span></span>
</div>
</div>
)}
{r.analysis && (
<p className="font-serif text-sm leading-loose text-ink-700">{r.analysis}</p>
)}
{r.key_evidence.length > 0 && (
<ul className="space-y-1.5">
{r.key_evidence.map((e, i) => (
<li key={i} className="flex gap-2 text-xs leading-relaxed text-ink-600">
<span className="flex-shrink-0 text-ink-300" aria-hidden="true"></span>
<span>{e}</span>
</li>
))}
</ul>
)}
{r.exp_home_goals !== null && r.exp_away_goals !== null && (
<p className="text-xs text-ink-500">
<span className="font-serif font-bold tabular-nums text-ink-900">{r.exp_home_goals.toFixed(1)} - {r.exp_away_goals.toFixed(1)}</span>
</p>
)}
{!inactive && (
<p className="border-t border-ink-100 pt-2.5 text-2xs text-ink-400">
{SUFFICIENCY_LABEL[r.data_sufficiency] ?? r.data_sufficiency}
<span className="mx-2 text-ink-200">|</span>
<span className="font-mono">{r.model}</span>
{r.latency_ms !== null && <span className="ml-2 tabular-nums">{r.latency_ms}ms</span>}
</p>
)}
</div>
</details>
)
}
export function AgentsPanel({ prediction }: { prediction: Prediction }) {
const [expertsOpen, setExpertsOpen] = useState(false)
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
const reports = prediction.agent_outputs ?? []
const okReports = reports.filter(r => r.status === 'ok')
if (reports.length === 0) return null
return (
<section>
<button
onClick={() => setExpertsOpen(o => !o)}
className="flex w-full items-center justify-between border-b border-ink-200 pb-2 text-left"
>
<span className="section-head mb-0">({okReports.length}/{reports.length} )</span>
<span className="text-2xs text-ink-400">{expertsOpen ? '收起' : '展开'}</span>
</button>
{!degraded && prediction.agent_weights && Object.keys(prediction.agent_weights).length > 0 && (
<div className="mt-3 space-y-1.5">
<span className="text-2xs text-ink-500"></span>
{Object.entries(prediction.agent_weights)
.sort((a, b) => b[1] - a[1])
.map(([k, v]) => (
<div key={k} className="grid grid-cols-[96px_minmax(0,1fr)_40px] items-center gap-2">
<span className="truncate text-2xs text-ink-500">{AGENT_LABELS[k] ?? k}</span>
<div className="h-1.5 bg-paper-100">
<div className="h-full bg-press" style={{ width: `${Math.round(v * 100)}%` }} />
</div>
<span className="text-right text-2xs tabular-nums text-ink-500">{Math.round(v * 100)}%</span>
</div>
))}
</div>
)}
{expertsOpen && (
<div className="mt-2">
{reports.map((r, i) => (
<AgentCard key={r.agent} report={r} no={CN_NUM[i] ?? String(i + 1)} />
))}
</div>
)}
</section>
)
}
@@ -4,39 +4,13 @@
* D3: 从 Matches.tsx 拆出,渲染逻辑原样搬迁。对外只导出 PredictModal;
* PredictionPanel 复用 Prediction 的 embedded 模式由弹窗内渲染。
*/
import { useEffect, useState } from 'react'
import { useEffect, useRef, useState } from 'react'
import TeamSideTag from '../../../components/TeamSideTag'
import type { AgentReport, Match, Prediction } from '../types'
import { AGENT_LABELS, CN_NUM, OUTCOME_LABEL } from '../types'
/** 置信度细线:0~1 数值的低调可视化 */
function Meter({ value }: { value: number }) {
const pct = Math.max(0, Math.min(100, Math.round(value * 100)))
return (
<div className="h-px w-full bg-ink-200" role="presentation">
<div className="h-px bg-press transition-[width] duration-500" style={{ width: `${pct}%` }} />
</div>
)
}
/** home_edge(-1~1,正=利主队)的可视化:以中线为原点的双向细条 */
function EdgeBar({ value }: { value: number }) {
const v = Math.max(-1, Math.min(1, value))
const half = Math.abs(v) * 50
return (
<div className="relative h-px w-full bg-ink-200" role="presentation">
<span className="absolute left-1/2 top-1/2 h-2 w-px -translate-x-1/2 -translate-y-1/2 bg-ink-400" />
<span
className={`absolute top-0 h-px transition-all duration-500 ${v >= 0 ? 'bg-press' : 'bg-ink-600'}`}
style={
v >= 0
? { left: '50%', width: `${half}%` }
: { right: '50%', width: `${half}%` }
}
/>
</div>
)
}
import type { Match, Prediction } from '../types'
import { AGENT_LABELS } from '../types'
import { AgentsPanel } from './AgentsPanel'
import { OutcomePanel } from './OutcomePanel'
import { ReasoningPanel } from './ReasoningPanel'
function Spinner({ className = '' }: { className?: string }) {
return (
@@ -53,77 +27,73 @@ function Spinner({ className = '' }: { className?: string }) {
}
/** 胜平负一行文字:选中的红字加方块标记,未选中的退灰 */
function OutcomeLine({
pick,
confidence,
/**
* P3-1:PredictionPanel 不再自绘,改为组合三个子组件:
* OutcomePanel(比分/胜平负/成本) / AgentsPanel(专家意见) / ReasoningPanel(终裁/降级)。
* 渲染输出与拆分前完全一致(仅降级警示 + 报头 + 元信息仍在此处)。
*/
function PredictionPanel({
prediction,
match,
embedded = false,
}: {
pick: string | null
confidence: number | null
prediction: Prediction
match: Match
/** 弹窗嵌入模式:弹窗已提供报头,这里省略自带版头 */
embedded?: boolean
}) {
const options = ['1', 'X', '2'] as const
const homeName = match.home_team_zh || match.home_team
const awayName = match.away_team_zh || match.away_team
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
const reports = prediction.agent_outputs ?? []
const okReports = reports.filter(r => r.status === 'ok')
return (
<div>
<div className="flex items-baseline justify-center gap-6 sm:gap-10">
{options.map(o => {
const on = pick === o
return (
<div key={o} className="flex flex-col items-center gap-1">
<span className={`flex items-center gap-1.5 text-sm ${on ? 'font-semibold text-press' : 'text-ink-400'}`}>
{on && <span className="inline-block h-2 w-2 bg-press" aria-hidden="true" />}
{OUTCOME_LABEL[o]}
</span>
{on && confidence !== null && (
<span className="text-2xs tabular-nums text-ink-500">
{Math.round(confidence * 100)}%
</span>
)}
</div>
)
})}
<article className={embedded ? 'bg-paper-50' : 'border border-ink-900 bg-paper-50'}>
{!embedded && (
<div className="flex flex-wrap items-baseline justify-between gap-2 border-b border-ink-900 bg-paper-100 px-4 py-2.5 sm:px-5">
<h3 className="flex flex-wrap items-center gap-1.5 font-serif text-sm font-bold text-ink-900">
·
<TeamSideTag side="home" />
{homeName}
<span></span>
<TeamSideTag side="away" />
{awayName}
</h3>
<span className="text-2xs tabular-nums text-ink-500">
{prediction.provider} / {prediction.model}
{prediction.latency_ms !== null && ` · ${(prediction.latency_ms / 1000).toFixed(1)}s`}
</span>
</div>
{pick && confidence !== null && (
<div className="mx-auto mt-3 max-w-xs">
<Meter value={confidence} />
<p className="mt-1 text-center text-2xs text-ink-400">,</p>
</div>
)}
</div>
)
}
/** 预测成本展示:耗时 + token + 限流余量 */
function PredictionCost({ prediction }: { prediction: Prediction }) {
const latency = prediction.latency_ms != null ? `${(prediction.latency_ms / 1000).toFixed(1)}s` : null
const tokens = prediction.prompt_tokens != null || prediction.completion_tokens != null
? `${prediction.prompt_tokens ?? '?'}/${prediction.completion_tokens ?? '?'}`
: null
<div className="space-y-7 px-4 py-6 sm:px-5">
{degraded && (
<div className="border-l-2 border-press bg-press-wash/40 px-4 py-3">
<p className="font-serif text-sm font-bold text-press-dark">
{prediction.status === 'failed' ? '预测失败' : '预测降级(degraded)'}
</p>
<p className="mt-1.5 whitespace-pre-wrap text-xs leading-relaxed text-ink-600">
{prediction.reasoning || '所有专家均无有效数据或调用失败,无法生成可靠比分。'}
</p>
</div>
)}
if (!latency && !tokens && prediction.rate_limit_remaining == null) return null
{!degraded && <OutcomePanel prediction={prediction} match={match} />}
return (
<div className="border-t border-ink-200 pt-3 text-2xs text-ink-500">
<div className="flex flex-wrap items-center justify-center gap-x-4 gap-y-1">
{latency && (
<span className="inline-flex items-center gap-1">
<span aria-hidden="true" className="opacity-60"></span> {latency}
</span>
)}
{tokens && (
<span className="inline-flex items-center gap-1">
<span aria-hidden="true" className="opacity-60">Tok</span>prompt/completion: {tokens}
</span>
)}
{prediction.rate_limit_remaining != null && prediction.rate_limit_remaining <= 3 && (
<span className="text-press" title="每分钟最多 10 次预测">
: {prediction.rate_limit_remaining}/10()
</span>
)}
<p className="text-center text-2xs text-ink-500">
{`多专家模式 · ${okReports.length}/${reports.length} 路有效`}
{prediction.prompt_version && ` · prompt ${prediction.prompt_version}`}
</p>
<AgentsPanel prediction={prediction} />
<ReasoningPanel prediction={prediction} />
</div>
</div>
</article>
)
}
/** 预测过程阶段(按时长模拟;结果到达即跳到完成) */
function PredictProgress() {
const [elapsed, setElapsed] = useState(0)
useEffect(() => {
@@ -191,7 +161,8 @@ function PredictProgress() {
</ul>
<p className="mt-6 text-center text-2xs text-ink-400">
, 30-90 ; token,使
, 30-90 ; token,使
,
</p>
<p className="mt-1 text-center text-2xs text-ink-300">
提示:每分钟限 10 ,
@@ -199,256 +170,6 @@ function PredictProgress() {
</div>
)
}
const STATUS_BADGE: Record<string, { label: string; cls: string }> = {
ok: { label: '正常', cls: 'text-ink-500' },
no_data: { label: '无数据', cls: 'text-ink-400' },
error: { label: '调用失败', cls: 'text-press' },
parse_error: { label: '解析失败', cls: 'text-press' },
}
const SUFFICIENCY_LABEL: Record<string, string> = {
high: '充分',
medium: '一般',
low: '偏少',
none: '无',
}
/** 单路专家意见:汉字编号 + 细线行 */
function AgentCard({ report: r, no }: { report: AgentReport; no: string }) {
const badge = STATUS_BADGE[r.status] ?? { label: r.status, cls: 'text-ink-400' }
const inactive = r.status !== 'ok'
return (
<details className="group border-b border-ink-200">
<summary className="flex cursor-pointer list-none items-baseline gap-2.5 px-1 py-3">
<span className="font-serif text-sm text-ink-400">{no}</span>
<span className="text-sm font-medium text-ink-900">{AGENT_LABELS[r.agent] ?? r.agent}</span>
<span className={`text-2xs ${badge.cls}`}>{badge.label}</span>
<span className="ml-auto flex items-baseline gap-3 text-2xs tabular-nums text-ink-500">
{r.status === 'ok' && r.subjective_confidence !== null && (
<span> {Math.round(r.subjective_confidence * 100)}%</span>
)}
{r.status === 'ok' && r.probable_score && (
<span className="font-serif font-bold text-ink-800">{r.probable_score}</span>
)}
<svg viewBox="0 0 20 20" className="h-3 w-3 self-center text-ink-300 transition-transform group-open:rotate-90" fill="currentColor" aria-hidden="true">
<path d="M7.3 5.3a1 1 0 011.4 0l4 4a1 1 0 010 1.4l-4 4a1 1 0 01-1.4-1.4L10.6 10 7.3 6.7a1 1 0 010-1.4z" />
</svg>
</span>
</summary>
<div className="space-y-3 px-1 pb-4 pl-7">
{/* 无数据 / 失败时给出明确说明,避免用户以为是空白 bug */}
{inactive && (
<p className="text-xs leading-relaxed text-ink-500">
{r.status === 'no_data' && '该维度没有可用数据,已跳过 LLM 分析以节省额度(不影响其他专家)。'}
{r.status === 'error' && '该专家调用失败,本次结论未纳入其视角(fail-open 设计,不阻断整体预测)。'}
{r.status === 'parse_error' && '模型输出未通过格式校验,该报告已丢弃。'}
</p>
)}
{!inactive && r.home_edge !== null && (
<div>
<div className="mb-1.5 flex items-baseline justify-between text-2xs">
<span className="text-ink-500"></span>
<span className={`font-semibold tabular-nums ${r.home_edge > 0 ? 'text-press' : r.home_edge < 0 ? 'text-ink-700' : 'text-ink-500'}`}>
{r.home_edge > 0 ? '+' : ''}{r.home_edge.toFixed(2)}
</span>
</div>
<EdgeBar value={r.home_edge} />
<div className="mt-1 flex justify-between text-2xs text-ink-400">
<span></span>
<span></span>
</div>
</div>
)}
{r.analysis && (
<p className="font-serif text-sm leading-loose text-ink-700">{r.analysis}</p>
)}
{r.key_evidence.length > 0 && (
<ul className="space-y-1.5">
{r.key_evidence.map((e, i) => (
<li key={i} className="flex gap-2 text-xs leading-relaxed text-ink-600">
<span className="flex-shrink-0 text-ink-300" aria-hidden="true"></span>
<span>{e}</span>
</li>
))}
</ul>
)}
{r.exp_home_goals !== null && r.exp_away_goals !== null && (
<p className="text-xs text-ink-500">
<span className="font-serif font-bold tabular-nums text-ink-900">{r.exp_home_goals.toFixed(1)} - {r.exp_away_goals.toFixed(1)}</span>
</p>
)}
{!inactive && (
<p className="border-t border-ink-100 pt-2.5 text-2xs text-ink-400">
{SUFFICIENCY_LABEL[r.data_sufficiency] ?? r.data_sufficiency}
<span className="mx-2 text-ink-200">|</span>
<span className="font-mono">{r.model}</span>
{r.latency_ms !== null && <span className="ml-2 tabular-nums">{r.latency_ms}ms</span>}
</p>
)}
</div>
</details>
)
}
function PredictionPanel({
prediction,
match,
embedded = false,
}: {
prediction: Prediction
match: Match
/** 弹窗嵌入模式:弹窗已提供报头,这里省略自带版头 */
embedded?: boolean
}) {
const homeName = match.home_team_zh || match.home_team
const [expertsOpen, setExpertsOpen] = useState(false)
const awayName = match.away_team_zh || match.away_team
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
const reports = prediction.agent_outputs ?? []
const okReports = reports.filter(r => r.status === 'ok')
return (
<article className={embedded ? 'bg-paper-50' : 'border border-ink-900 bg-paper-50'}>
{!embedded && (
<div className="flex flex-wrap items-baseline justify-between gap-2 border-b border-ink-900 bg-paper-100 px-4 py-2.5 sm:px-5">
<h3 className="flex flex-wrap items-center gap-1.5 font-serif text-sm font-bold text-ink-900">
·
<TeamSideTag side="home" />
{homeName}
<span></span>
<TeamSideTag side="away" />
{awayName}
</h3>
<span className="text-2xs tabular-nums text-ink-500">
{prediction.provider} / {prediction.model}
{prediction.latency_ms !== null && ` · ${(prediction.latency_ms / 1000).toFixed(1)}s`}
</span>
</div>
)}
<div className="space-y-7 px-4 py-6 sm:px-5">
{/* ── degraded / failed 态:醒目警示 + 原因,不展示虚假比分 ── */}
{degraded && (
<div className="border-l-2 border-press bg-press-wash/40 px-4 py-3">
<p className="font-serif text-sm font-bold text-press-dark">
{prediction.status === 'failed' ? '预测失败' : '预测降级(degraded)'}
</p>
<p className="mt-1.5 whitespace-pre-wrap text-xs leading-relaxed text-ink-600">
{prediction.reasoning || '所有专家均无有效数据或调用失败,无法生成可靠比分。'}
</p>
</div>
)}
{/* ── 主结论(仅 success 展示) ── */}
{!degraded && (
<>
<div className="text-center">
<p className="font-serif text-5xl font-bold tabular-nums leading-none text-ink-900 sm:text-6xl">
{prediction.pred_home_goals ?? '-'}
<span className="mx-3 font-normal text-ink-300">:</span>
{prediction.pred_away_goals ?? '-'}
</p>
<p className="mt-3 text-2xs tracking-[0.5em] text-ink-400"></p>
{prediction.alt_pred_home_goals != null && prediction.alt_pred_away_goals != null && (
<p className="mt-2 text-2xs tabular-nums text-ink-400">
{' '}
<span className="font-serif text-sm font-bold tabular-nums text-ink-600">
{prediction.alt_pred_home_goals}<span className="mx-0.5 font-normal text-ink-300">:</span>{prediction.alt_pred_away_goals}
</span>
</p>
)}
</div>
<div className="border-y border-ink-200 py-4">
<OutcomeLine pick={prediction.pred_1x2} confidence={prediction.subjective_confidence} />
</div>
</>
)}
{/* ── 成本信息(耗时 + token + 限流余量) ── */}
{!degraded && (
<PredictionCost prediction={prediction} />
)}
{/* ── 元信息 ── */}
<p className="text-center text-2xs text-ink-500">
`多专家模式 · ${okReports.length}/${reports.length} 路有效`
{prediction.prompt_version && ` · prompt ${prediction.prompt_version}`}
</p>
{/* ── 终裁/降级说明意见 ── */}
{prediction.reasoning && degraded && (
<section>
<h4 className="section-head mb-2"></h4>
<blockquote className="border-l-2 border-press pl-4">
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
</blockquote>
</section>
)}
{/* ── 专家意见(多模式):可折叠 + 状态摘要 + 权重条形图 ── */}
{reports.length > 0 && (
<section>
<button
onClick={() => setExpertsOpen(o => !o)}
className="flex w-full items-center justify-between border-b border-ink-200 pb-2 text-left"
>
<span className="section-head mb-0">({okReports.length}/{reports.length} )</span>
<span className="text-2xs text-ink-400">{expertsOpen ? '收起' : '展开'}</span>
</button>
{/* 权重条形图(仅 success 且有权重时显示) */}
{!degraded && prediction.agent_weights && Object.keys(prediction.agent_weights).length > 0 && (
<div className="mt-3 space-y-1.5">
<span className="text-2xs text-ink-500"></span>
{Object.entries(prediction.agent_weights)
.sort((a, b) => b[1] - a[1])
.map(([k, v]) => (
<div key={k} className="grid grid-cols-[96px_minmax(0,1fr)_40px] items-center gap-2">
<span className="truncate text-2xs text-ink-500">{AGENT_LABELS[k] ?? k}</span>
<div className="h-1.5 bg-paper-100">
<div className="h-full bg-press" style={{ width: `${Math.round(v * 100)}%` }} />
</div>
<span className="text-right text-2xs tabular-nums text-ink-500">{Math.round(v * 100)}%</span>
</div>
))}
</div>
)}
{expertsOpen && (
<div className="mt-2">
{reports.map((r, i) => (
<AgentCard key={r.agent} report={r} no={CN_NUM[i] ?? String(i + 1)} />
))}
</div>
)}
</section>
)}
{/* ── 终裁意见(success) ── */}
{prediction.reasoning && !degraded && (
<section>
<h4 className="section-head mb-3"></h4>
<blockquote className="border-l-2 border-press pl-4">
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
</blockquote>
</section>
)}
</div>
</article>
)
}
/** 预测弹窗:进行中显示过程可视化,完成后显示预测版,失败显示原因 */
export function PredictModal({
match,
predicting,
@@ -465,17 +186,52 @@ export function PredictModal({
const homeName = match.home_team_zh || match.home_team
const awayName = match.away_team_zh || match.away_team
// ── 无障碍与滚动锁定 ──
const panelRef = useRef<HTMLDivElement>(null)
const previouslyFocused = useRef<HTMLElement | null>(null)
useEffect(() => {
previouslyFocused.current = document.activeElement as HTMLElement | null
// 初始聚焦弹窗容器,键盘用户可直接 Tab 进入内部控件
panelRef.current?.focus()
const h = (e: KeyboardEvent) => {
if (e.key === 'Escape') onClose()
if (e.key === 'Escape') {
onClose()
return
}
if (e.key === 'Tab') {
// 简易焦点陷阱:Tab 循环限制在弹窗内,不会跑到遮罩背后的页面
const focusables = panelRef.current?.querySelectorAll<HTMLElement>(
'button, [href], input, select, textarea, [tabindex]:not([tabindex="-1"])',
)
if (!focusables || focusables.length === 0) return
const first = focusables[0]
const last = focusables[focusables.length - 1]
if (e.shiftKey && document.activeElement === first) {
e.preventDefault()
last.focus()
} else if (!e.shiftKey && document.activeElement === last) {
e.preventDefault()
first.focus()
}
}
}
document.addEventListener('keydown', h)
return () => document.removeEventListener('keydown', h)
// 锁定背景滚动:弹窗内滚到底继续滚时,不再带动底层页面
const prevOverflow = document.body.style.overflow
document.body.style.overflow = 'hidden'
return () => {
document.removeEventListener('keydown', h)
document.body.style.overflow = prevOverflow
// 关闭后把焦点还给触发元素
previouslyFocused.current?.focus()
}
}, [onClose])
return (
<div
className="fixed inset-0 z-50 flex items-start justify-center overflow-y-auto bg-ink-900/50 p-4 sm:items-center"
className="modal-overlay-enter fixed inset-0 z-50 flex items-start justify-center overflow-y-auto bg-ink-900/50 p-4 sm:items-center"
role="dialog"
aria-modal="true"
aria-label={`预测 ${homeName}${awayName}`}
@@ -483,7 +239,11 @@ export function PredictModal({
if (e.target === e.currentTarget) onClose()
}}
>
<div className="relative flex max-h-[92vh] w-full max-w-2xl flex-col overflow-hidden bg-paper-50 shadow-2xl">
<div
ref={panelRef}
tabIndex={-1}
className="modal-panel-enter relative flex max-h-[92vh] w-full max-w-2xl flex-col overflow-hidden bg-paper-50 outline-none"
>
{/* 弹窗报头 */}
<div className="flex flex-shrink-0 items-center justify-between border-b border-ink-900 bg-paper-100 px-4 py-2.5 sm:px-5">
<h3 className="flex flex-wrap items-center gap-1.5 font-serif text-sm font-bold text-ink-900">
@@ -0,0 +1,126 @@
/**
* OutcomePanel: 预测主结论 —— 比分 / 胜平负 / 置信度 / 成本。
*
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
*/
import TeamSideTag from '../../../components/TeamSideTag'
import type { Match, Prediction } from '../types'
import { OUTCOME_LABEL } from '../types'
/** 置信度细线:0~1 数值的低调可视化 */
function Meter({ value }: { value: number }) {
const pct = Math.max(0, Math.min(100, Math.round(value * 100)))
return (
<div className="h-px w-full bg-ink-200" role="presentation">
<div className="h-px bg-press transition-[width] duration-500" style={{ width: `${pct}%` }} />
</div>
)
}
/** 胜平负一行文字:选中的红字加方块标记,未选中的退灰 */
function OutcomeLine({
pick,
confidence,
}: {
pick: string | null
confidence: number | null
}) {
const options = ['1', 'X', '2'] as const
return (
<div>
<div className="flex items-baseline justify-center gap-6 sm:gap-10">
{options.map(o => {
const on = pick === o
return (
<div key={o} className="flex flex-col items-center gap-1">
<span className={`flex items-center gap-1.5 text-sm ${on ? 'font-semibold text-press' : 'text-ink-400'}`}>
{on && <span className="inline-block h-2 w-2 bg-press" aria-hidden="true" />}
{OUTCOME_LABEL[o]}
</span>
{on && confidence !== null && (
<span className="text-2xs tabular-nums text-ink-500">
{Math.round(confidence * 100)}%
</span>
)}
</div>
)
})}
</div>
{pick && confidence !== null && (
<div className="mx-auto mt-3 max-w-xs">
<Meter value={confidence} />
<p className="mt-1 text-center text-2xs text-ink-400">,</p>
</div>
)}
</div>
)
}
/** 预测成本展示:耗时 + token + 限流余量 */
function PredictionCost({ prediction }: { prediction: Prediction }) {
const latency = prediction.latency_ms != null ? `${(prediction.latency_ms / 1000).toFixed(1)}s` : null
const tokens = prediction.prompt_tokens != null || prediction.completion_tokens != null
? `${prediction.prompt_tokens ?? '?'}/${prediction.completion_tokens ?? '?'}`
: null
if (!latency && !tokens && prediction.rate_limit_remaining == null) return null
return (
<div className="border-t border-ink-200 pt-3 text-2xs text-ink-500">
<div className="flex flex-wrap items-center justify-center gap-x-4 gap-y-1">
{latency && (
<span className="inline-flex items-center gap-1">
<span aria-hidden="true" className="opacity-60"></span> {latency}
</span>
)}
{tokens && (
<span className="inline-flex items-center gap-1">
<span aria-hidden="true" className="opacity-60">Tok</span>prompt/completion: {tokens}
</span>
)}
{prediction.rate_limit_remaining != null && prediction.rate_limit_remaining <= 3 && (
<span className="text-press" title="每分钟最多 10 次预测">
: {prediction.rate_limit_remaining}/10()
</span>
)}
</div>
</div>
)
}
export function OutcomePanel({ prediction, match }: { prediction: Prediction; match: Match }) {
const homeName = match.home_team_zh || match.home_team
const awayName = match.away_team_zh || match.away_team
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
return (
<>
{!degraded && (
<div className="text-center">
<p className="font-serif text-5xl font-bold tabular-nums leading-none text-ink-900 sm:text-6xl">
{prediction.pred_home_goals ?? '-'}
<span className="mx-3 font-normal text-ink-300">:</span>
{prediction.pred_away_goals ?? '-'}
</p>
<p className="mt-3 text-2xs tracking-[0.5em] text-ink-400"></p>
{prediction.alt_pred_home_goals != null && prediction.alt_pred_away_goals != null && (
<p className="mt-2 text-2xs tabular-nums text-ink-400">
{' '}
<span className="font-serif text-sm font-bold tabular-nums text-ink-600">
{prediction.alt_pred_home_goals}<span className="mx-0.5 font-normal text-ink-300">:</span>{prediction.alt_pred_away_goals}
</span>
</p>
)}
</div>
)}
{!degraded && (
<div className="border-y border-ink-200 py-4">
<OutcomeLine pick={prediction.pred_1x2} confidence={prediction.subjective_confidence} />
</div>
)}
{!degraded && <PredictionCost prediction={prediction} />}
</>
)
}
@@ -0,0 +1,34 @@
/**
* ReasoningPanel: 终裁意见 / 降级原因 —— 预测的文本解释。
*
* P3-1: 从 MatchPredictPanel.PredictionPanel 拆出,渲染逻辑原样搬迁。
*/
import type { Prediction } from '../types'
export function ReasoningPanel({ prediction }: { prediction: Prediction }) {
const degraded = prediction.status === 'degraded' || prediction.status === 'failed'
if (!prediction.reasoning) return null
// 降级态:reasoning 展示为「降级原因」
if (degraded) {
return (
<section>
<h4 className="section-head mb-2"></h4>
<blockquote className="border-l-2 border-press pl-4">
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
</blockquote>
</section>
)
}
// 成功态:reasoning 展示为「终裁意见」
return (
<section>
<h4 className="section-head mb-3"></h4>
<blockquote className="border-l-2 border-press pl-4">
<p className="whitespace-pre-wrap font-serif text-sm leading-loose text-ink-700">{prediction.reasoning}</p>
</blockquote>
</section>
)
}
@@ -11,6 +11,8 @@ import { useRef, useState } from 'react'
import { http } from '../../../lib/http'
import type { Match, Prediction } from '../types'
const sleep = (ms: number) => new Promise<void>(r => setTimeout(r, ms))
/** 把后端/网络错误翻译成用户可读文案 */
function readablePredictError(e: unknown): string {
if (e instanceof Error) {
@@ -59,17 +61,43 @@ export function useMatchPredict({ onError }: UseMatchPredictOptions) {
onError(null)
setPrediction(null)
setPredictionFor(m)
// LLM 多专家预测耗时可达数分钟,给足超时(与 nginx 代理 300s 对齐)
// P1-async: 预测改为异步,POST 立即返回 job_id,轮询结果避免网关超时(Cloudflare 100s → 524)
const controller = new AbortController()
predictAbort.current = controller
const timer = setTimeout(() => controller.abort(), 300_000)
const overallTimer = setTimeout(() => controller.abort(), 300_000)
try {
const data = await http.post<Prediction>('/predict', { match_id: m.id, mode: 'multi' }, {
timeoutMs: 300_000,
signal: controller.signal,
})
// 1) 发起预测,拿到 job_id
const started = await http.post<{ job_id: string; poll_url: string }>(
'/predict',
{ match_id: m.id, mode: 'multi' },
{ timeoutMs: 10_000, signal: controller.signal },
)
if (seq !== predictSeq.current) return
setPrediction(data)
// 2) 轮询直到终态(success/failed)或整体超时
const deadline = Date.now() + 300_000
while (Date.now() < deadline) {
if (controller.signal.aborted) throw new DOMException('aborted', 'AbortError')
await sleep(3000)
const job = await http.get<{ status: string; result?: Prediction; error?: string }>(
`/predict/jobs/${started.job_id}`,
{ timeoutMs: 5000, signal: controller.signal },
)
if (seq !== predictSeq.current) return
if (job.status === 'success') {
setPrediction(job.result ?? null)
return
}
if (job.status === 'failed') {
onError(job.error || '预测失败')
return
}
// status === 'running' → 继续轮询
}
// 整体超时
onError('预测超时(5 分钟),请稍后重试')
} catch (e) {
if (seq !== predictSeq.current) return
onError(
@@ -78,7 +106,7 @@ export function useMatchPredict({ onError }: UseMatchPredictOptions) {
: readablePredictError(e),
)
} finally {
clearTimeout(timer)
clearTimeout(overallTimer)
if (seq === predictSeq.current) setPredictingId(null)
}
}
+2
View File
@@ -78,6 +78,8 @@ export const LEAGUES = [
{ code: 'D1', name: '德甲' },
{ code: 'I1', name: '意甲' },
{ code: 'F1', name: '法甲' },
{ code: 'CL', name: '欧冠' },
{ code: 'EL', name: '欧联' },
]
/** 汉字编号,给专家意见排版用 */
+33 -6
View File
@@ -24,12 +24,39 @@ export default {
800: '#282420',
900: '#17140F',
},
// 印报红:全站唯一强调色,克制使用
press: {
DEFAULT: '#9E1B1B',
dark: '#7C1414',
wash: '#F7E9E4',
},
// 印报红:全站唯一强调色,克制使用
press: {
DEFAULT: '#9E1B1B',
dark: '#7C1414',
wash: '#F7E9E4',
},
// ── 语义状态色:成功/警告/负面 ──
// 设计约束:纸底 #FDFCF8 上文字级(600/700)对比度 ≥ 4.5:1(WCAG AA),
// 点/条级(500) ≥ 3:1;色相降饱和以贴近墨色印刷感,不使用 tailwind 原色。
ok: {
50: '#EAF3ED',
100: '#D5E8DC',
300: '#A3C9B1',
500: '#43925F',
600: '#2E7A4C',
700: '#1F6B45',
},
warn: {
50: '#FBF3E4',
100: '#F5E5C8',
300: '#E2C48E',
500: '#D97706',
600: '#A15C0B',
700: '#8F5109',
},
bad: {
50: '#FAEDED',
100: '#F6E3E3',
300: '#E4AFAF',
500: '#C24A4A',
600: '#A83B3B',
700: '#8F3030',
},
},
fontFamily: {
// 毛体草书(国内 CDN)+ 粗楷体回退
+2 -1
View File
@@ -16,5 +16,6 @@
"noUnusedParameters": false,
"noFallthroughCasesInSwitch": true
},
"include": ["src"]
"include": ["src"],
"exclude": ["src/**/*.test.ts", "src/**/*.spec.ts", "node_modules"]
}
+51 -2
View File
@@ -2,6 +2,7 @@
from __future__ import annotations
import logging
import os
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
@@ -13,6 +14,35 @@ from src.core.config import settings
logger = logging.getLogger(__name__)
async def _fail_stale_ingest_jobs() -> None:
"""P1-E: 启动时将上次遗留的 pending/running ingest_jobs 标 failed。
进程异常退出(重启/OOM)会导致 ingest_jobs 残留为 pending/running,
这些任务实际已不在执行,启动时一次性标 failed 避免永久"执行中"
尽力而为:失败只记 warning,不阻断启动。
"""
from datetime import datetime, timezone
from sqlalchemy import update
from src.db.base import AsyncSessionLocal
from src.db.models import IngestJob
try:
async with AsyncSessionLocal() as session:
stmt = (
update(IngestJob)
.where(IngestJob.status.in_(["pending", "running"]))
.values(status="failed", error="进程重启:任务被终止", finished_at=datetime.now(timezone.utc))
)
result = await session.execute(stmt)
await session.commit()
if result.rowcount:
logger.info("P1-E: 已将 %d 条残留 pending/running ingest_jobs 标 failed", result.rowcount)
except Exception:
logger.warning("P1-E: 清理残留 ingest_jobs 失败,不影响启动", exc_info=True)
@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncIterator[None]:
from src.db.base import init_db
@@ -29,6 +59,7 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
await migrate_plaintext_sensitive_settings() # 明文敏感配置 → 加密(幂等)
await ensure_admin_password_hashed() # .env 明文密码 → scrypt 哈希(幂等)
await assert_security_on_startup() # 启动安全校验(生产拒绝/开发警告)
await _fail_stale_ingest_jobs() # P1-E: 上次遗留的 pending/running 标 failed
# D7(工程债): 进程内限流(_RateLimiter)与 KeyRing 均为单进程状态;
# 多 worker 部署时各进程独立计数,限流阈值会按 worker 数放大、KeyRing 不共享。
@@ -40,6 +71,24 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
"多 worker 部署请将限流前置到 Nginx/网关,或以单 worker 运行"
)
# P3-3:STRICT_SINGLE_WORKER 启动期强制校验,拒绝多 worker 静默配额漂移。
# uvicorn 通过 --workers 传入;此处以环境变量 UVICORN_WORKERS 或启动参数判定。
# 为避免耦合 uvicorn 内部,仅校验一个显式传入的标记:当 STRICT_SINGLE_WORKER=True 时,
# 要求环境变量 UVICORN_WORKERS 不为空且 <=1,否则拒绝启动。
if settings.STRICT_SINGLE_WORKER:
workers = os.environ.get("UVICORN_WORKERS", "1")
try:
n_workers = int(workers)
except ValueError:
n_workers = 1
if n_workers > 1:
raise RuntimeError(
f"STRICT_SINGLE_WORKER=True 但以 {n_workers} worker 启动会被拒绝 "
f"(应用内限流/KeyRing 多 worker 下各自独立计数,配额放大 {n_workers} 倍)。"
f"请前置 Nginx/网关全局限流后再启用多 worker,或保持单 worker。"
)
logger.info("STRICT_SINGLE_WORKER=True:已确认单 worker 启动,限流配额不会漂移")
# 注册默认定时任务(如果数据库中没有)
from src.db.base import AsyncSessionLocal
from sqlalchemy import select
@@ -75,8 +124,8 @@ async def lifespan(app: FastAPI) -> AsyncIterator[None]:
def create_app() -> FastAPI:
from src.core.log_buffer import setup_memory_logging
setup_memory_logging(settings.LOG_LEVEL)
from src.core.log_buffer import setup_logging
setup_logging(settings.LOG_LEVEL, settings.LOG_FILE)
# 生产环境不暴露 OpenAPI 文档(避免向访客泄露接口结构)
openapi_url = "/openapi.json" if settings.APP_ENV != "production" else None
+88
View File
@@ -0,0 +1,88 @@
"""后台管理:配置项 CRUD(settings)与运行日志查询。
所有接口需管理员鉴权(require_admin)路由前缀 /api/v1/admin
配置项白名单见 src/core/runtime_config.py SETTING_DEFS,之外的 key 一律拒绝
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from src.api.deps import require_admin
from src.core.log_buffer import get_entries
from src.core.runtime_config import (
SETTING_DEFS,
clear_runtime_value,
get_setting_origin,
mask_value,
set_runtime_value,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
class SettingUpdateIn(BaseModel):
value: str
@router.get("/settings")
async def list_settings():
"""全部可配置项(脱敏),供后台各配置页渲染。"""
out = []
for key, defn in SETTING_DEFS.items():
origin, value = await get_setting_origin(key)
out.append(
{
"key": key,
"label": defn.label,
"description": defn.description,
"sensitive": defn.sensitive,
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
)
return out
@router.get("/logs")
async def read_logs(
level: str | None = Query(None, description="最低级别: DEBUG/INFO/WARNING/ERROR"),
keyword: str | None = Query(None, description="消息或 logger 关键字"),
limit: int = Query(200, ge=1, le=1000),
):
"""查询应用运行日志(内存环形缓冲,最新在前;进程重启后清零)。"""
entries = get_entries(level, keyword, limit)
return {"entries": entries, "count": len(entries)}
@router.put("/settings/{key}")
async def update_setting(key: str, body: SettingUpdateIn):
"""更新配置项(写入 app_settings 覆盖 .env)。传空值请改用 DELETE。"""
if key not in SETTING_DEFS:
raise HTTPException(404, f"不支持的配置项: {key}")
value = body.value.strip()
if not value:
raise HTTPException(400, "值不能为空;如需回落 .env 请调用清除接口")
await set_runtime_value(key, value)
defn = SETTING_DEFS[key]
return {"key": key, "masked": mask_value(value, defn.sensitive), "origin": "db"}
@router.delete("/settings/{key}")
async def clear_setting(key: str):
"""清除 DB 覆盖值,回落 .env 默认。"""
if key not in SETTING_DEFS:
raise HTTPException(404, f"不支持的配置项: {key}")
await clear_runtime_value(key)
origin, value = await get_setting_origin(key)
defn = SETTING_DEFS[key]
return {
"key": key,
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
+228
View File
@@ -0,0 +1,228 @@
"""后台管理:数据源列表/连通性测试、KeyRing 状态、采集健康概览。
所有接口需管理员鉴权(require_admin)路由前缀 /api/v1/admin
"""
from __future__ import annotations
import logging
import time
from datetime import date, datetime
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy import func, select
from src.api.deps import require_admin
from src.core.config import settings
from src.core.http_client import get_client
from src.core.runtime_config import (
SETTING_DEFS,
get_runtime_value,
get_setting_origin,
mask_value,
)
from src.db.base import AsyncSession, get_db_read
from src.db.models import Match, MatchStats, Standing
from src.data.key_ring import get_key_ring
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
# ── 数据源元数据(bzzoiro 单一数据源) ────────────────────────────
_SOURCES: list[dict] = [
{
"name": "bzzoiro",
"label": "Bzzoiro",
"description": "唯一数据源:赛程比分 + 积分榜 + 比赛详细统计(xG/射门/控球等)",
"setting_keys": ["BZZOIRO_KEY", "BZZOIRO_BASE"],
},
]
async def _last_ingestion(db: AsyncSession, source: str) -> datetime | None:
"""源最近一次采集时间(取自数据血缘字段,无记录返回 None)。"""
return (
await db.execute(
select(func.max(MatchStats.retrieved_at)).where(MatchStats.source == source)
)
).scalar()
@router.get("/datasources")
async def list_datasources(db: AsyncSession = Depends(get_db_read)):
"""数据源列表:各配置项的脱敏值、来源(db/env/none)与最近采集时间。"""
result = []
for src in _SOURCES:
settings_out = []
for key in src["setting_keys"]:
origin, value = await get_setting_origin(key)
defn = SETTING_DEFS[key]
settings_out.append(
{
"key": key,
"label": defn.label,
"description": defn.description,
"sensitive": defn.sensitive,
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
)
key_configured = all(s["configured"] for s in settings_out) if settings_out else True
last = await _last_ingestion(db, src["name"])
result.append(
{
"name": src["name"],
"label": src["label"],
"description": src["description"],
"key_configured": key_configured,
"last_ingestion": last.isoformat() if last else None,
"settings": settings_out,
}
)
return result
# ── 连通性测试 ──────────────────────────────────────────────────
_TEST_TIMEOUT = 15
async def _probe(url: str, headers: dict | None = None, params: dict | None = None) -> dict:
"""单次 HTTP 探测,返回 (ok, status, latency_ms, detail)。不重试。"""
client = get_client()
start = time.monotonic()
try:
resp = await client.get(url, headers=headers, params=params, timeout=_TEST_TIMEOUT)
except Exception as e:
return {
"ok": False,
"status": None,
"latency_ms": int((time.monotonic() - start) * 1000),
"detail": f"无法连接: {e}",
}
latency = int((time.monotonic() - start) * 1000)
status = resp.status_code
if status == 200:
detail = "连接成功"
elif status in (401, 403):
detail = "服务可达,但密钥无效或无权限"
else:
detail = f"服务返回 HTTP {status}"
return {"ok": status == 200, "status": status, "latency_ms": latency, "detail": detail}
@router.post("/datasources/{name}/test")
async def test_datasource(name: str):
"""轻量连通性测试:真实请求上游一次,不触发任何入库。"""
src = next((s for s in _SOURCES if s["name"] == name), None)
if src is None:
raise HTTPException(404, f"未知数据源: {name}")
if name == "bzzoiro":
key = await get_runtime_value("BZZOIRO_KEY")
if not key:
return {"ok": False, "status": None, "latency_ms": 0, "detail": "BZZOIRO_KEY 未配置"}
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
today = date.today().isoformat()
return await _probe(
f"{base}/events/",
headers={"Authorization": f"Token {key}", "Accept": "application/json"},
params={"date_from": today, "date_to": today},
)
raise HTTPException(404, f"未知数据源: {name}")
# ── 数据源健康/最近采集状态(只读,不触发采集) ──────────────────────
@router.get("/ingest/status")
async def ingest_status(db: AsyncSession = Depends(get_db_read)):
"""数据源采集健康概览(bzzoiro 单源;只读,不触发任何采集)。"""
bzzoiro_key = await get_runtime_value("BZZOIRO_KEY")
bzzoiro_base = await get_runtime_value("BZZOIRO_BASE")
# 比赛覆盖
match_row = (
await db.execute(
select(
func.count().label("cnt"),
func.max(Match.match_date).label("latest_match_date"),
func.max(Match.created_at).label("latest_row_at"),
).where(Match.match_status == "finished")
)
).one()
# 统计覆盖(精确 retrieved_at)
stats_row = (
await db.execute(
select(
func.count().label("cnt"),
func.max(MatchStats.retrieved_at).label("latest_retrieved"),
).where(MatchStats.source == "bzzoiro")
)
).one()
# 积分榜覆盖
standings_row = (
await db.execute(select(func.count()).select_from(Standing))
).scalar()
bzzoiro = {
"name": "bzzoiro",
"label": "Bzzoiro",
"key_configured": bool(bzzoiro_key),
"base_url": (bzzoiro_base.rstrip("/") if bzzoiro_base else None) or settings.BZZOIRO_BASE,
"reachable": None, # 不主动探测
"last_success_at": (stats_row.latest_retrieved or match_row.latest_row_at),
"last_success_at_iso": (
stats_row.latest_retrieved or match_row.latest_row_at
).isoformat() if (stats_row.latest_retrieved or match_row.latest_row_at) else None,
"latest_match_date": match_row.latest_match_date.isoformat() if match_row.latest_match_date else None,
"recent_count": match_row.cnt or 0,
"stats_count": stats_row.cnt or 0,
"standings_count": standings_row or 0,
"note": "last_success_at 取 match_stats.retrieved_at(统计回填)与 matches.created_at(比赛行)的较大者",
"last_failure": _last_failure_log("bzzoiro"),
}
return {"sources": [bzzoiro]}
@router.get("/keyring/status")
async def keyring_status():
"""KeyRing 运行状态:当前使用的 key、冷却状态、轮转信息(供管理后台展示)。"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
st = ring.stats()
st["base_url"] = base
st["cooldown_seconds"] = ring._cooldown
st["has_multiple"] = ring.has_multiple
st["active_key"] = ring.active_key
return st
@router.post("/keyring/cooldown/reset")
async def keyring_reset_cooldown():
"""手动重置所有 key 的冷却状态(用于紧急恢复)。"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
ring._blocked_until.clear()
return {"ok": True, "message": "已重置所有 key 冷却状态", "stats": ring.stats()}
def _last_failure_log(source: str) -> dict | None:
"""从系统日志缓冲中查找某数据源的最近一次错误(仅作参考,非专用失败表)。"""
from src.core.log_buffer import get_entries
entries = get_entries(min_level="ERROR", keyword=source, limit=5)
if not entries:
return None
e = entries[0]
return {
"at": datetime.fromtimestamp(e["ts"]).isoformat(),
"logger": e["logger"],
"detail": e["message"][:200],
"note": "approx:来自内存日志缓冲,非专用采集失败表;进程重启后清零",
}
+55
View File
@@ -0,0 +1,55 @@
"""后台管理:采集任务状态查询(只读)。
GET /api/v1/admin/ingest/jobs/{job_id} 单任务详情
GET /api/v1/admin/ingest/jobs?limit=N 最近任务列表(默认 20)
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import desc, select
from src.api.deps import require_admin
from src.api.schemas import IngestJobOut
from src.db.base import AsyncSession, get_db_read
from src.db.models import IngestJob
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/ingest/jobs/{job_id}", response_model=IngestJobOut)
async def get_ingest_job(job_id: str, db: AsyncSession = Depends(get_db_read)):
"""查询单个采集任务状态。"""
job = await db.get(IngestJob, job_id)
if job is None:
raise HTTPException(404, f"采集任务不存在: {job_id}")
return _job_to_out(job)
@router.get("/ingest/jobs", response_model=list[IngestJobOut])
async def list_ingest_jobs(
limit: int = Query(20, ge=1, le=100, description="返回条数"),
db: AsyncSession = Depends(get_db_read),
):
"""查询最近采集任务(最新在前)。"""
rows = (
await db.execute(select(IngestJob).order_by(desc(IngestJob.created_at)).limit(limit))
).scalars().all()
return [_job_to_out(j) for j in rows]
def _job_to_out(job: IngestJob) -> IngestJobOut:
return IngestJobOut(
id=job.id,
task=job.task,
params=job.params or {},
status=job.status,
result=job.result,
error=job.error,
created_at=job.created_at,
started_at=job.started_at,
finished_at=job.finished_at,
)
+126
View File
@@ -0,0 +1,126 @@
"""后台管理:LLM 专家/终裁配置、可用模型探测、连通性测试。
所有接口需管理员鉴权(require_admin)路由前缀 /api/v1/admin
"""
from __future__ import annotations
import logging
import time
import httpx
from fastapi import APIRouter, Depends
from src.api.deps import require_admin
from src.core.config import settings
from src.core.http_client import get_client
from src.core.runtime_config import (
AGENT_META,
SETTING_DEFS,
get_runtime_value,
get_setting_origin,
mask_value,
)
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/llm/agents")
async def list_llm_agents():
"""各专家/终裁的独立 LLM 配置状态(含当前生效模型的解析结果)。"""
out = []
for agent in AGENT_META:
aid = agent["id"].upper()
pfx = f"AGENT_{aid}_"
fields = {}
for suffix in ("MODEL", "BASE_URL", "API_KEY"):
origin, value = await get_setting_origin(f"{pfx}{suffix}")
defn = SETTING_DEFS[f"{pfx}{suffix}"]
fields[suffix.lower()] = {
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
# 生效模型 = 覆盖 → 层级默认(专家/终裁 env) → 全局 LLM_MODEL
tier_default = (
settings.LLM_AGGREGATOR_MODEL if agent["id"] == "aggregator" else settings.LLM_SPECIALIST_MODEL
)
effective_model = (
fields["model"]["masked"]
if fields["model"]["configured"]
else (tier_default or await get_runtime_value("LLM_MODEL"))
)
out.append(
{
"id": agent["id"],
"label": agent["label"],
"fields": fields,
"effective_model": effective_model,
}
)
return out
@router.get("/llm/models")
async def list_llm_models():
"""探测当前 LLM 服务可用的模型列表(OpenAI 兼容 GET /models)。
只读探测,不产生费用;配置缺失或服务不可达时返回 ok=false 与原因
"""
base_url = (await get_runtime_value("LLM_BASE_URL")).rstrip("/")
api_key = await get_runtime_value("LLM_API_KEY")
if not base_url or not api_key:
return {"ok": False, "models": [], "detail": "LLM_BASE_URL 或 LLM_API_KEY 未配置"}
client = get_client()
start = time.monotonic()
try:
resp = await client.get(
f"{base_url}/models",
headers={"Authorization": f"Bearer {api_key}"},
timeout=httpx.Timeout(connect=10.0, read=20.0, write=10.0, pool=10.0),
)
except Exception as e:
return {
"ok": False,
"models": [],
"latency_ms": int((time.monotonic() - start) * 1000),
"detail": f"无法连接 LLM 服务: {e}",
}
latency = int((time.monotonic() - start) * 1000)
if resp.status_code in (401, 403):
return {"ok": False, "models": [], "latency_ms": latency, "detail": "密钥无效或无权限(HTTP 401/403)"}
if resp.status_code != 200:
return {"ok": False, "models": [], "latency_ms": latency, "detail": f"服务返回 HTTP {resp.status_code}"}
try:
data = resp.json()
except Exception:
return {"ok": False, "models": [], "latency_ms": latency, "detail": "响应不是合法 JSON"}
models: list[str] = []
items = data.get("data") if isinstance(data, dict) else None
if isinstance(items, list):
models = sorted(
str(m.get("id")) for m in items if isinstance(m, dict) and m.get("id")
)
if not models:
return {"ok": False, "models": [], "latency_ms": latency, "detail": "服务未返回模型列表"}
return {"ok": True, "models": models, "latency_ms": latency, "detail": f"{len(models)} 个可用模型"}
@router.post("/llm/ping")
async def llm_ping():
"""LLM 连通性测试(不依赖比赛)。只发一次 chat 请求验证配置。"""
from src.llm.provider import get_default_provider
p = await get_default_provider()
resp = await p.chat(
system="你是测试助手。",
user="ping",
max_tokens=10,
)
if resp.error:
return {"ok": False, "message": resp.error}
return {"ok": True, "message": "LLM 连接正常", "model": p.model}
+351
View File
@@ -0,0 +1,351 @@
"""后台管理:管理区统计、数据完整性分析、数据质量检查。
所有接口需管理员鉴权(require_admin)路由前缀 /api/v1/admin
"""
from __future__ import annotations
import logging
from datetime import datetime, timedelta, timezone
from fastapi import APIRouter, Depends
from sqlalchemy import func, select
from src.api.deps import require_admin
from src.db.base import AsyncSession, get_db_read
from src.db.models import DataQualityCheck, IngestFailure, League, Match, MatchStats, Prediction, Standing
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/stats")
async def admin_stats(db: AsyncSession = Depends(get_db_read)):
"""管理区统计(只读):预测次数 + 比赛覆盖。轻量聚合,无 LLM 调用。"""
day_ago = datetime.now(timezone.utc) - timedelta(days=1)
week_ago = datetime.now(timezone.utc) - timedelta(days=7)
r = (
await db.execute(
select(
func.count().label("total"),
func.count().filter(Prediction.created_at >= day_ago).label("last_24h"),
func.count().filter(Prediction.created_at >= week_ago).label("last_7d"),
)
)
).one()
# F3 修复: 补充真实比赛计数(非 limit=100 近似)
match_cnt = (await db.execute(select(func.count()).select_from(Match))).scalar() or 0
finished_cnt = (await db.execute(select(func.count()).where(Match.match_status == "finished"))).scalar() or 0
stats_cnt = (await db.execute(select(func.count()).select_from(MatchStats))).scalar() or 0
standings_cnt = (await db.execute(select(func.count()).select_from(Standing))).scalar() or 0
return {
"predictions": {"total": r.total, "last_24h": r.last_24h, "last_7d": r.last_7d},
"matches": {"total": match_cnt, "finished": finished_cnt},
"stats": {"total": stats_cnt},
"standings": {"total": standings_cnt},
}
# ── 数据完整性分析(可视化数据源) ────────────────────────────────
@router.get("/data-completeness")
async def data_completeness(db: AsyncSession = Depends(get_db_read)):
"""按联赛统计数据完整性:比赛覆盖、字段覆盖、积分榜覆盖。
前端数据完整性页据此渲染,回答三个问题:
1. 数据是否齐全(各联赛比赛/统计/积分榜量级)
2. 字段是否齐全(每张统计表各字段非空率)
3. 覆盖是否新鲜(最近一场/最近一次采集)
"""
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_NAMES, LEAGUE_COUNTRIES
out_leagues: list[dict] = []
for code, bzz_id in BZZOIRO_LEAGUE_IDS.items():
# 比赛覆盖
m = (
await db.execute(
select(
func.count().label("total"),
func.count().filter(Match.match_status == "finished").label("finished"),
func.count().filter(Match.match_status == "scheduled").label("scheduled"),
func.count().filter(Match.source_event_id.is_not(None)).label("with_source_id"),
func.max(Match.match_date).label("latest_match"),
func.min(Match.match_date).label("earliest_match"),
)
.select_from(Match)
.join(League, League.id == Match.league_id)
.where(League.code == code)
)
).one()
# 统计字段覆盖(联表 matches)
s = (
await db.execute(
select(
func.count().label("rows"),
func.count(MatchStats.home_xg).label("xg"),
func.count(MatchStats.home_shots).label("shots"),
func.count(MatchStats.home_possession).label("possession"),
func.count(MatchStats.home_corners).label("corners"),
func.count(MatchStats.home_fouls).label("fouls"),
func.count(MatchStats.home_big_chances).label("big_chances"),
func.count(MatchStats.home_yellow_cards).label("cards"),
)
.select_from(MatchStats)
.join(Match, Match.id == MatchStats.match_id)
.join(League, League.id == Match.league_id)
.where(League.code == code)
)
).one()
# 积分榜覆盖
st = (
await db.execute(
select(
func.count().label("rows"),
func.max(Standing.retrieved_at).label("latest_retrieved"),
)
.select_from(Standing)
.join(League, League.id == Standing.league_id)
.where(League.code == code)
)
).one()
stats_rows = s.rows or 0
pct = lambda n: round(n / stats_rows * 100, 1) if stats_rows else 0.0 # noqa: E731
out_leagues.append(
{
"code": code,
"name": LEAGUE_NAMES.get(code, code),
"country": LEAGUE_COUNTRIES.get(code),
"matches": {
"total": m.total or 0,
"finished": m.finished or 0,
"scheduled": m.scheduled or 0,
"with_source_id": m.with_source_id or 0,
"earliest_match": m.earliest_match.isoformat() if m.earliest_match else None,
"latest_match": m.latest_match.isoformat() if m.latest_match else None,
},
"stats": {
"rows": stats_rows,
"fields": {
"xg": {"count": s.xg or 0, "pct": pct(s.xg or 0)},
"shots": {"count": s.shots or 0, "pct": pct(s.shots or 0)},
"possession": {"count": s.possession or 0, "pct": pct(s.possession or 0)},
"corners": {"count": s.corners or 0, "pct": pct(s.corners or 0)},
"fouls": {"count": s.fouls or 0, "pct": pct(s.fouls or 0)},
"big_chances": {"count": s.big_chances or 0, "pct": pct(s.big_chances or 0)},
"cards": {"count": s.cards or 0, "pct": pct(s.cards or 0)},
},
},
"standings": {
"rows": st.rows or 0,
"latest_retrieved": st.latest_retrieved.isoformat() if st.latest_retrieved else None,
},
}
)
# 整体健康信号
total_finished = sum(l["matches"]["finished"] for l in out_leagues)
total_stats = sum(l["stats"]["rows"] for l in out_leagues)
stats_coverage = round(total_stats / total_finished * 100, 1) if total_finished else 0.0
issues: list[str] = []
for l in out_leagues:
if l["matches"]["finished"] == 0:
issues.append(f"{l['name']}: 无已完赛比赛,请先运行「比赛数据」采集")
elif l["stats"]["rows"] == 0:
issues.append(f"{l['name']}: 已完赛 {l['matches']['finished']} 场但无统计回填,请运行「统计回填」采集")
elif stats_coverage < 80:
issues.append(f"{l['name']}: 统计覆盖率仅 {stats_coverage}%,建议增量回填")
if l["standings"]["rows"] == 0:
issues.append(f"{l['name']}: 无积分榜数据,请运行「积分榜」采集")
if not issues:
issues.append("各联赛数据完整度良好")
return {
"generated_at": datetime.now(timezone.utc).isoformat(),
"leagues": out_leagues,
"totals": {
"finished_matches": total_finished,
"stats_rows": total_stats,
"stats_coverage_pct": stats_coverage,
},
"issues": issues,
}
# ── 数据质量检查 API ────────────────────────────────────────────
@router.get("/data-quality")
async def data_quality_checks(db: AsyncSession = Depends(get_db_read)):
"""数据质量检查结果(只读)。"""
# 最近的失败记录
failures = (
await db.execute(
select(IngestFailure)
.where(IngestFailure.status.in_(["pending", "retrying"]))
.order_by(IngestFailure.created_at.desc())
.limit(20)
)
).scalars().all()
# 最近的质量检查
checks = (
await db.execute(
select(DataQualityCheck)
.order_by(DataQualityCheck.checked_at.desc())
.limit(20)
)
).scalars().all()
return {
"failures": [
{
"id": f.id,
"source": f.source_system,
"entity_type": f.entity_type,
"source_record_id": f.source_record_id,
"error_type": f.error_type,
"error_detail": f.error_detail,
"retry_count": f.retry_count,
"status": f.status,
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in failures
],
"checks": [
{
"id": c.id,
"check_name": c.check_name,
"entity_type": c.entity_type,
"passed": c.passed,
"severity": c.severity,
"detail": c.detail,
"checked_at": c.checked_at.isoformat() if c.checked_at else None,
}
for c in checks
],
}
@router.post("/data-quality/run")
async def run_data_quality_check(db: AsyncSession = Depends(get_db_read)):
"""手动触发一次数据质量检查。"""
checks = []
# 检查1: 已完赛但无统计的比赛
# 注意: MatchStats 主键是 match_id(P0-02),不是 id —— 引用 .id 会 AttributeError
finished_no_stats = (
await db.execute(
select(func.count())
.select_from(Match)
.outerjoin(MatchStats, Match.id == MatchStats.match_id)
.where(Match.match_status == "finished")
.where(MatchStats.match_id.is_(None))
)
).scalar() or 0
checks.append(DataQualityCheck(
check_name="finished_without_stats",
entity_type="match",
actual_value=float(finished_no_stats),
passed=finished_no_stats == 0,
severity="warning" if finished_no_stats > 0 else "info",
detail={"message": f"{finished_no_stats} 场已完赛比赛缺少统计数据"},
))
# 检查2: 积分榜缺失的联赛
leagues_without_standings = (
await db.execute(
select(func.count())
.select_from(League)
.outerjoin(Standing, League.id == Standing.league_id)
.where(Standing.id.is_(None))
)
).scalar() or 0
checks.append(DataQualityCheck(
check_name="league_without_standings",
entity_type="league",
actual_value=float(leagues_without_standings),
passed=leagues_without_standings == 0,
severity="warning" if leagues_without_standings > 0 else "info",
detail={"message": f"{leagues_without_standings} 个联赛缺少积分榜"},
))
for c in checks:
db.add(c)
await db.commit()
# 成功路径留痕:检查何时跑过、各项结果如何(此前 handler 无任何日志,
# 加上未捕获异常走 uvicorn.error 不进内存缓冲,线上排障无据可查)
logger.info(
"数据质量检查完成: %s",
"; ".join(f"{c.check_name}={'通过' if c.passed else '未通过'}({c.actual_value:.0f})" for c in checks),
)
return {"ok": True, "checks": [{"name": c.check_name, "passed": c.passed} for c in checks]}
# ── 近似重名候选(只读,启发式,不做自动合并) ──────────────────────
@router.get("/team-name-duplicates")
async def team_name_duplicates(db: AsyncSession = Depends(get_db_read)):
"""只读列出近似重名候选(大小写变体/子串包含/前缀碰撞)。
启发式规则(命中任一即列为候选):
- 大小写变体: lower(name) 相同但 name 不同
- 子串包含: A B 的子串且 len(A) 5
- 前缀碰撞: 8 字符相同(忽略大小写)
仅作排查参考,合并需走人工 SQL( docs/05-data.md)
"""
teams = (await db.execute(select(Team.id, Team.name))).all()
by_lower: dict[str, list[dict]] = {}
for t in teams:
key = (t.name or "").lower()
by_lower.setdefault(key, []).append({"id": t.id, "name": t.name})
groups: list[dict] = []
# 规则1: 大小写变体(lower 相同但原名不同)
for key, members in by_lower.items():
if len(members) > 1:
groups.append({
"rule": "case_variant",
"key": key,
"members": members,
})
# 规则2 & 3: 子串包含 / 前缀碰撞(仅在 lower 名不同的组间比较)
distinct = [m for members in by_lower.values() for m in members]
seen_pairs: set[tuple[int, int]] = set()
for i, a in enumerate(distinct):
na = (a["name"] or "").lower()
for b in distinct[i + 1:]:
nb = (b["name"] or "").lower()
if na == nb:
continue # 已被规则1覆盖
pair = (min(a["id"], b["id"]), max(a["id"], b["id"]))
if pair in seen_pairs:
continue
hit = None
if len(na) >= 5 and na in nb:
hit = "substring"
elif len(nb) >= 5 and nb in na:
hit = "substring"
elif len(na) >= 8 and len(nb) >= 8 and na[:8] == nb[:8]:
hit = "prefix"
if hit:
seen_pairs.add(pair)
groups.append({
"rule": hit,
"members": [a, b],
})
return {
"count": len(groups),
"hint": "命中任一启发式仅表示'可疑',合并前请人工确认是否同一球队",
"groups": groups,
}
+25 -664
View File
@@ -1,668 +1,29 @@
"""后台管理路由:数据源配置的查看、修改与连通性测试
"""后台管理路由聚合入口:按职责拆分为四个子模块,统一挂载
所有接口需管理员鉴权(require_admin)配置项白名单见
src/core/runtime_config.py SETTING_DEFS,之外的 key 一律拒绝
所有路由仍挂在 /api/v1/admin,且均带 dependencies=[Depends(require_admin)]
(鉴权由各子路由器声明,行为与拆分前完全一致)
子模块:
- admin_datasources 数据源列表/连通性测试KeyRing采集健康概览
- admin_config settings CRUD运行日志
- admin_llm LLM agents/models/ping
- admin_quality statsdata-completenessdata-quality
"""
from __future__ import annotations
import logging
import time
from datetime import date, datetime, timedelta, timezone
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel
from sqlalchemy import func, select
import httpx
from src.api.deps import require_admin
from src.core.config import settings
from src.core.http_client import get_client
from src.core.log_buffer import get_entries
from src.core.runtime_config import (
AGENT_META,
SETTING_DEFS,
clear_runtime_value,
get_runtime_value,
get_setting_origin,
mask_value,
set_runtime_value,
)
from src.db.base import AsyncSession, get_db_read
from src.db.models import League, Match, MatchStats, Standing
from src.data.key_ring import get_key_ring, parse_keys
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
# ── 数据源元数据(bzzoiro 单一数据源) ────────────────────────────
_SOURCES: list[dict] = [
{
"name": "bzzoiro",
"label": "Bzzoiro",
"description": "唯一数据源:赛程比分 + 积分榜 + 比赛详细统计(xG/射门/控球等)",
"setting_keys": ["BZZOIRO_KEY", "BZZOIRO_BASE"],
},
]
class SettingUpdateIn(BaseModel):
value: str
async def _last_ingestion(db: AsyncSession, source: str) -> datetime | None:
"""源最近一次采集时间(取自数据血缘字段,无记录返回 None)。"""
return (
await db.execute(
select(func.max(MatchStats.retrieved_at)).where(MatchStats.source == source)
)
).scalar()
@router.get("/datasources")
async def list_datasources(db: AsyncSession = Depends(get_db_read)):
"""数据源列表:各配置项的脱敏值、来源(db/env/none)与最近采集时间。"""
result = []
for src in _SOURCES:
settings_out = []
for key in src["setting_keys"]:
origin, value = await get_setting_origin(key)
defn = SETTING_DEFS[key]
settings_out.append(
{
"key": key,
"label": defn.label,
"description": defn.description,
"sensitive": defn.sensitive,
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
)
key_configured = all(s["configured"] for s in settings_out) if settings_out else True
last = await _last_ingestion(db, src["name"])
result.append(
{
"name": src["name"],
"label": src["label"],
"description": src["description"],
"key_configured": key_configured,
"last_ingestion": last.isoformat() if last else None,
"settings": settings_out,
}
)
return result
@router.get("/settings")
async def list_settings():
"""全部可配置项(脱敏),供后台各配置页渲染。"""
out = []
for key, defn in SETTING_DEFS.items():
origin, value = await get_setting_origin(key)
out.append(
{
"key": key,
"label": defn.label,
"description": defn.description,
"sensitive": defn.sensitive,
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
)
return out
# ── LLM 可用模型检测 ────────────────────────────────────────────
@router.get("/logs")
async def read_logs(
level: str | None = Query(None, description="最低级别: DEBUG/INFO/WARNING/ERROR"),
keyword: str | None = Query(None, description="消息或 logger 关键字"),
limit: int = Query(200, ge=1, le=1000),
):
"""查询应用运行日志(内存环形缓冲,最新在前;进程重启后清零)。"""
entries = get_entries(level, keyword, limit)
return {"entries": entries, "count": len(entries)}
@router.get("/llm/agents")
async def list_llm_agents():
"""各专家/终裁的独立 LLM 配置状态(含当前生效模型的解析结果)。"""
out = []
for agent in AGENT_META:
aid = agent["id"].upper()
pfx = f"AGENT_{aid}_"
fields = {}
for suffix in ("MODEL", "BASE_URL", "API_KEY"):
origin, value = await get_setting_origin(f"{pfx}{suffix}")
defn = SETTING_DEFS[f"{pfx}{suffix}"]
fields[suffix.lower()] = {
"configured": origin != "none",
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
# 生效模型 = 覆盖 → 层级默认(专家/终裁 env) → 全局 LLM_MODEL
tier_default = (
settings.LLM_AGGREGATOR_MODEL if agent["id"] == "aggregator" else settings.LLM_SPECIALIST_MODEL
)
effective_model = (
fields["model"]["masked"]
if fields["model"]["configured"]
else (tier_default or await get_runtime_value("LLM_MODEL"))
)
out.append(
{
"id": agent["id"],
"label": agent["label"],
"fields": fields,
"effective_model": effective_model,
}
)
return out
@router.get("/llm/models")
async def list_llm_models():
"""探测当前 LLM 服务可用的模型列表(OpenAI 兼容 GET /models)。
只读探测,不产生费用;配置缺失或服务不可达时返回 ok=false 与原因
"""
base_url = (await get_runtime_value("LLM_BASE_URL")).rstrip("/")
api_key = await get_runtime_value("LLM_API_KEY")
if not base_url or not api_key:
return {"ok": False, "models": [], "detail": "LLM_BASE_URL 或 LLM_API_KEY 未配置"}
client = get_client()
start = time.monotonic()
try:
resp = await client.get(
f"{base_url}/models",
headers={"Authorization": f"Bearer {api_key}"},
timeout=httpx.Timeout(connect=10.0, read=20.0, write=10.0, pool=10.0),
)
except Exception as e:
return {
"ok": False,
"models": [],
"latency_ms": int((time.monotonic() - start) * 1000),
"detail": f"无法连接 LLM 服务: {e}",
}
latency = int((time.monotonic() - start) * 1000)
if resp.status_code in (401, 403):
return {"ok": False, "models": [], "latency_ms": latency, "detail": "密钥无效或无权限(HTTP 401/403)"}
if resp.status_code != 200:
return {"ok": False, "models": [], "latency_ms": latency, "detail": f"服务返回 HTTP {resp.status_code}"}
try:
data = resp.json()
except Exception:
return {"ok": False, "models": [], "latency_ms": latency, "detail": "响应不是合法 JSON"}
models: list[str] = []
items = data.get("data") if isinstance(data, dict) else None
if isinstance(items, list):
models = sorted(
str(m.get("id")) for m in items if isinstance(m, dict) and m.get("id")
)
if not models:
return {"ok": False, "models": [], "latency_ms": latency, "detail": "服务未返回模型列表"}
return {"ok": True, "models": models, "latency_ms": latency, "detail": f"{len(models)} 个可用模型"}
@router.put("/settings/{key}")
async def update_setting(key: str, body: SettingUpdateIn):
"""更新配置项(写入 app_settings 覆盖 .env)。传空值请改用 DELETE。"""
if key not in SETTING_DEFS:
raise HTTPException(404, f"不支持的配置项: {key}")
value = body.value.strip()
if not value:
raise HTTPException(400, "值不能为空;如需回落 .env 请调用清除接口")
await set_runtime_value(key, value)
defn = SETTING_DEFS[key]
return {"key": key, "masked": mask_value(value, defn.sensitive), "origin": "db"}
@router.delete("/settings/{key}")
async def clear_setting(key: str):
"""清除 DB 覆盖值,回落 .env 默认。"""
if key not in SETTING_DEFS:
raise HTTPException(404, f"不支持的配置项: {key}")
await clear_runtime_value(key)
origin, value = await get_setting_origin(key)
defn = SETTING_DEFS[key]
return {
"key": key,
"masked": mask_value(value, defn.sensitive),
"origin": origin,
}
# ── 连通性测试 ──────────────────────────────────────────────────
_TEST_TIMEOUT = 15
async def _probe(url: str, headers: dict | None = None, params: dict | None = None) -> dict:
"""单次 HTTP 探测,返回 (ok, status, latency_ms, detail)。不重试。"""
client = get_client()
start = time.monotonic()
try:
resp = await client.get(url, headers=headers, params=params, timeout=_TEST_TIMEOUT)
except Exception as e:
return {
"ok": False,
"status": None,
"latency_ms": int((time.monotonic() - start) * 1000),
"detail": f"无法连接: {e}",
}
latency = int((time.monotonic() - start) * 1000)
status = resp.status_code
if status == 200:
detail = "连接成功"
elif status in (401, 403):
detail = "服务可达,但密钥无效或无权限"
else:
detail = f"服务返回 HTTP {status}"
return {"ok": status == 200, "status": status, "latency_ms": latency, "detail": detail}
@router.post("/datasources/{name}/test")
async def test_datasource(name: str):
"""轻量连通性测试:真实请求上游一次,不触发任何入库。"""
src = next((s for s in _SOURCES if s["name"] == name), None)
if src is None:
raise HTTPException(404, f"未知数据源: {name}")
if name == "bzzoiro":
key = await get_runtime_value("BZZOIRO_KEY")
if not key:
return {"ok": False, "status": None, "latency_ms": 0, "detail": "BZZOIRO_KEY 未配置"}
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
today = date.today().isoformat()
return await _probe(
f"{base}/events/",
headers={"Authorization": f"Token {key}", "Accept": "application/json"},
params={"date_from": today, "date_to": today},
)
raise HTTPException(404, f"未知数据源: {name}")
@router.post("/llm/ping")
async def llm_ping():
"""LLM 连通性测试(不依赖比赛)。只发一次 chat 请求验证配置。"""
from src.llm.provider import get_default_provider
p = await get_default_provider()
resp = await p.chat(
system="你是测试助手。",
user="ping",
max_tokens=10,
)
if resp.error:
return {"ok": False, "message": resp.error}
return {"ok": True, "message": "LLM 连接正常", "model": p.model}
# ── 数据源健康/最近采集状态(只读,不触发采集) ──────────────────────
@router.get("/ingest/status")
async def ingest_status(db: AsyncSession = Depends(get_db_read)):
"""数据源采集健康概览(bzzoiro 单源;只读,不触发任何采集)。"""
bzzoiro_key = await get_runtime_value("BZZOIRO_KEY")
bzzoiro_base = await get_runtime_value("BZZOIRO_BASE")
# 比赛覆盖
match_row = (
await db.execute(
select(
func.count().label("cnt"),
func.max(Match.match_date).label("latest_match_date"),
func.max(Match.created_at).label("latest_row_at"),
).where(Match.match_status == "finished")
)
).one()
# 统计覆盖(精确 retrieved_at)
stats_row = (
await db.execute(
select(
func.count().label("cnt"),
func.max(MatchStats.retrieved_at).label("latest_retrieved"),
).where(MatchStats.source == "bzzoiro")
)
).one()
# 积分榜覆盖
standings_row = (
await db.execute(select(func.count()).select_from(Standing))
).scalar()
bzzoiro = {
"name": "bzzoiro",
"label": "Bzzoiro",
"key_configured": bool(bzzoiro_key),
"base_url": (bzzoiro_base.rstrip("/") if bzzoiro_base else None) or settings.BZZOIRO_BASE,
"reachable": None, # 不主动探测
"last_success_at": (stats_row.latest_retrieved or match_row.latest_row_at),
"last_success_at_iso": (
stats_row.latest_retrieved or match_row.latest_row_at
).isoformat() if (stats_row.latest_retrieved or match_row.latest_row_at) else None,
"latest_match_date": match_row.latest_match_date.isoformat() if match_row.latest_match_date else None,
"recent_count": match_row.cnt or 0,
"stats_count": stats_row.cnt or 0,
"standings_count": standings_row or 0,
"note": "last_success_at 取 match_stats.retrieved_at(统计回填)与 matches.created_at(比赛行)的较大者",
"last_failure": _last_failure_log("bzzoiro"),
}
return {"sources": [bzzoiro]}
@router.get("/keyring/status")
async def keyring_status():
"""KeyRing 运行状态:当前使用的 key、冷却状态、轮转信息(供管理后台展示)。"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
st = ring.stats()
st["base_url"] = base
st["cooldown_seconds"] = ring._cooldown
st["has_multiple"] = ring.has_multiple
st["active_key"] = ring.active_key
return st
@router.post("/keyring/cooldown/reset")
async def keyring_reset_cooldown():
"""手动重置所有 key 的冷却状态(用于紧急恢复)。"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
ring._blocked_until.clear()
return {"ok": True, "message": "已重置所有 key 冷却状态", "stats": ring.stats()}
def _last_failure_log(source: str) -> dict | None:
"""从系统日志缓冲中查找某数据源的最近一次错误(仅作参考,非专用失败表)。"""
entries = get_entries(min_level="ERROR", keyword=source, limit=5)
if not entries:
return None
e = entries[0]
return {
"at": datetime.fromtimestamp(e["ts"]).isoformat(),
"logger": e["logger"],
"detail": e["message"][:200],
"note": "approx:来自内存日志缓冲,非专用采集失败表;进程重启后清零",
}
@router.get("/stats")
async def admin_stats(db: AsyncSession = Depends(get_db_read)):
"""管理区统计(只读):预测次数 + 比赛覆盖。轻量聚合,无 LLM 调用。"""
from sqlalchemy import func, text
from src.db.models import Prediction, Match, MatchStats, Standing
day_ago = datetime.now(timezone.utc) - timedelta(days=1)
week_ago = datetime.now(timezone.utc) - timedelta(days=7)
r = (
await db.execute(
select(
func.count().label("total"),
func.count().filter(Prediction.created_at >= day_ago).label("last_24h"),
func.count().filter(Prediction.created_at >= week_ago).label("last_7d"),
)
)
).one()
# F3 修复: 补充真实比赛计数(非 limit=100 近似)
match_cnt = (await db.execute(select(func.count()).select_from(Match))).scalar() or 0
finished_cnt = (await db.execute(select(func.count()).where(Match.match_status == "finished"))).scalar() or 0
stats_cnt = (await db.execute(select(func.count()).select_from(MatchStats))).scalar() or 0
standings_cnt = (await db.execute(select(func.count()).select_from(Standing))).scalar() or 0
return {
"predictions": {"total": r.total, "last_24h": r.last_24h, "last_7d": r.last_7d},
"matches": {"total": match_cnt, "finished": finished_cnt},
"stats": {"total": stats_cnt},
"standings": {"total": standings_cnt},
}
# ── 数据完整性分析(可视化数据源) ────────────────────────────────
@router.get("/data-completeness")
async def data_completeness(db: AsyncSession = Depends(get_db_read)):
"""按联赛统计数据完整性:比赛覆盖、字段覆盖、积分榜覆盖。
前端数据完整性页据此渲染,回答三个问题:
1. 数据是否齐全(各联赛比赛/统计/积分榜量级)
2. 字段是否齐全(每张统计表各字段非空率)
3. 覆盖是否新鲜(最近一场/最近一次采集)
"""
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_NAMES, LEAGUE_COUNTRIES
out_leagues: list[dict] = []
for code, bzz_id in BZZOIRO_LEAGUE_IDS.items():
# 比赛覆盖
m = (
await db.execute(
select(
func.count().label("total"),
func.count().filter(Match.match_status == "finished").label("finished"),
func.count().filter(Match.match_status == "scheduled").label("scheduled"),
func.count().filter(Match.source_event_id.is_not(None)).label("with_source_id"),
func.max(Match.match_date).label("latest_match"),
func.min(Match.match_date).label("earliest_match"),
)
.select_from(Match)
.join(League, League.id == Match.league_id)
.where(League.code == code)
)
).one()
# 统计字段覆盖(联表 matches)
s = (
await db.execute(
select(
func.count().label("rows"),
func.count(MatchStats.home_xg).label("xg"),
func.count(MatchStats.home_shots).label("shots"),
func.count(MatchStats.home_possession).label("possession"),
func.count(MatchStats.home_corners).label("corners"),
func.count(MatchStats.home_fouls).label("fouls"),
func.count(MatchStats.home_big_chances).label("big_chances"),
func.count(MatchStats.home_yellow_cards).label("cards"),
)
.select_from(MatchStats)
.join(Match, Match.id == MatchStats.match_id)
.join(League, League.id == Match.league_id)
.where(League.code == code)
)
).one()
# 积分榜覆盖
st = (
await db.execute(
select(
func.count().label("rows"),
func.max(Standing.retrieved_at).label("latest_retrieved"),
)
.select_from(Standing)
.join(League, League.id == Standing.league_id)
.where(League.code == code)
)
).one()
stats_rows = s.rows or 0
pct = lambda n: round(n / stats_rows * 100, 1) if stats_rows else 0.0 # noqa: E731
out_leagues.append(
{
"code": code,
"name": LEAGUE_NAMES.get(code, code),
"country": LEAGUE_COUNTRIES.get(code),
"matches": {
"total": m.total or 0,
"finished": m.finished or 0,
"scheduled": m.scheduled or 0,
"with_source_id": m.with_source_id or 0,
"earliest_match": m.earliest_match.isoformat() if m.earliest_match else None,
"latest_match": m.latest_match.isoformat() if m.latest_match else None,
},
"stats": {
"rows": stats_rows,
"fields": {
"xg": {"count": s.xg or 0, "pct": pct(s.xg or 0)},
"shots": {"count": s.shots or 0, "pct": pct(s.shots or 0)},
"possession": {"count": s.possession or 0, "pct": pct(s.possession or 0)},
"corners": {"count": s.corners or 0, "pct": pct(s.corners or 0)},
"fouls": {"count": s.fouls or 0, "pct": pct(s.fouls or 0)},
"big_chances": {"count": s.big_chances or 0, "pct": pct(s.big_chances or 0)},
"cards": {"count": s.cards or 0, "pct": pct(s.cards or 0)},
},
},
"standings": {
"rows": st.rows or 0,
"latest_retrieved": st.latest_retrieved.isoformat() if st.latest_retrieved else None,
},
}
)
# 整体健康信号
total_finished = sum(l["matches"]["finished"] for l in out_leagues)
total_stats = sum(l["stats"]["rows"] for l in out_leagues)
stats_coverage = round(total_stats / total_finished * 100, 1) if total_finished else 0.0
issues: list[str] = []
for l in out_leagues:
if l["matches"]["finished"] == 0:
issues.append(f"{l['name']}: 无已完赛比赛,请先运行「比赛数据」采集")
elif l["stats"]["rows"] == 0:
issues.append(f"{l['name']}: 已完赛 {l['matches']['finished']} 场但无统计回填,请运行「统计回填」采集")
elif stats_coverage < 80:
issues.append(f"{l['name']}: 统计覆盖率仅 {stats_coverage}%,建议增量回填")
if l["standings"]["rows"] == 0:
issues.append(f"{l['name']}: 无积分榜数据,请运行「积分榜」采集")
if not issues:
issues.append("各联赛数据完整度良好")
return {
"generated_at": datetime.now(timezone.utc).isoformat(),
"leagues": out_leagues,
"totals": {
"finished_matches": total_finished,
"stats_rows": total_stats,
"stats_coverage_pct": stats_coverage,
},
"issues": issues,
}
# ── 数据质量检查 API ────────────────────────────────────────────
@router.get("/data-quality")
async def data_quality_checks(db: AsyncSession = Depends(get_db_read)):
"""数据质量检查结果(只读)。"""
from src.db.models import IngestFailure, DataQualityCheck
from sqlalchemy import func
# 最近的失败记录
failures = (
await db.execute(
select(IngestFailure)
.where(IngestFailure.status.in_(["pending", "retrying"]))
.order_by(IngestFailure.created_at.desc())
.limit(20)
)
).scalars().all()
# 最近的质量检查
checks = (
await db.execute(
select(DataQualityCheck)
.order_by(DataQualityCheck.checked_at.desc())
.limit(20)
)
).scalars().all()
return {
"failures": [
{
"id": f.id,
"source": f.source_system,
"entity_type": f.entity_type,
"source_record_id": f.source_record_id,
"error_type": f.error_type,
"error_detail": f.error_detail,
"retry_count": f.retry_count,
"status": f.status,
"created_at": f.created_at.isoformat() if f.created_at else None,
}
for f in failures
],
"checks": [
{
"id": c.id,
"check_name": c.check_name,
"entity_type": c.entity_type,
"passed": c.passed,
"severity": c.severity,
"detail": c.detail,
"checked_at": c.checked_at.isoformat() if c.checked_at else None,
}
for c in checks
],
}
@router.post("/data-quality/run")
async def run_data_quality_check(db: AsyncSession = Depends(get_db_read)):
"""手动触发一次数据质量检查。"""
from src.db.models import DataQualityCheck, Match, MatchStats, Standing, League
from sqlalchemy import func
checks = []
# 检查1: 已完赛但无统计的比赛
finished_no_stats = (
await db.execute(
select(func.count())
.select_from(Match)
.outerjoin(MatchStats, Match.id == MatchStats.match_id)
.where(Match.match_status == "finished")
.where(MatchStats.id.is_(None))
)
).scalar() or 0
checks.append(DataQualityCheck(
check_name="finished_without_stats",
entity_type="match",
actual_value=float(finished_no_stats),
passed=finished_no_stats == 0,
severity="warning" if finished_no_stats > 0 else "info",
detail={"message": f"{finished_no_stats} 场已完赛比赛缺少统计数据"},
))
# 检查2: 积分榜缺失的联赛
leagues_without_standings = (
await db.execute(
select(func.count())
.select_from(League)
.outerjoin(Standing, League.id == Standing.league_id)
.where(Standing.id.is_(None))
)
).scalar() or 0
checks.append(DataQualityCheck(
check_name="league_without_standings",
entity_type="league",
actual_value=float(leagues_without_standings),
passed=leagues_without_standings == 0,
severity="warning" if leagues_without_standings > 0 else "info",
detail={"message": f"{leagues_without_standings} 个联赛缺少积分榜"},
))
for c in checks:
db.add(c)
await db.commit()
return {"ok": True, "checks": [{"name": c.check_name, "passed": c.passed} for c in checks]}
from fastapi import APIRouter
from src.api.routes.admin_config import router as admin_config_router
from src.api.routes.admin_datasources import router as admin_datasources_router
from src.api.routes.admin_ingest_jobs import router as admin_ingest_jobs_router
from src.api.routes.admin_llm import router as admin_llm_router
from src.api.routes.admin_quality import router as admin_quality_router
from src.api.routes.admin_teams import router as admin_teams_router
router = APIRouter()
router.include_router(admin_datasources_router)
router.include_router(admin_config_router)
router.include_router(admin_ingest_jobs_router)
router.include_router(admin_llm_router)
router.include_router(admin_quality_router)
router.include_router(admin_teams_router)
+60
View File
@@ -0,0 +1,60 @@
"""后台管理:球队别名管理(只读列表 + 添加别名)。
归一名(teams.name)是球队唯一键;别名(team_aliases)是同一球队的不同写法
(大小写/译名/缩写)到归一后 teams.id 的映射入库时 normalize(name) 依次查
teams.name team_aliases,命中即复用,避免重复 Team
不自动合并历史重复队;需显式添加别名(或先 SQL/再经由此接口)
"""
from __future__ import annotations
import logging
from fastapi import APIRouter, Depends, HTTPException
from sqlalchemy import desc, select
from src.api.deps import require_admin
from src.api.schemas import TeamAliasIn, TeamAliasOut
from src.db.base import AsyncSession, get_db_read
from src.db.models import Team, TeamAlias
from src.db.repositories import TeamRepository
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1/admin", tags=["admin"], dependencies=[Depends(require_admin)])
@router.get("/teams/aliases", response_model=list[TeamAliasOut])
async def list_team_aliases(db: AsyncSession = Depends(get_db_read)):
"""列出所有球队别名(最新在前)。"""
rows = (await db.execute(select(TeamAlias).order_by(desc(TeamAlias.created_at)).limit(200))).scalars().all()
return [
TeamAliasOut(
alias_normalized=r.alias_normalized,
team_id=r.team_id,
original_alias=r.original_alias,
)
for r in rows
]
@router.post("/teams/aliases", response_model=TeamAliasOut, status_code=201)
async def add_team_alias(req: TeamAliasIn, db: AsyncSession = Depends(get_db_read)):
"""为已有 Team 添加别名(幂等:重复添加会更新指向)。
不自动合并历史重复队若需合并 AB:先为 A 的归一名添加别名指向 B,
再人工确认 A 是否仍有独立引用
"""
# 校验目标 Team 存在
team = await db.get(Team, req.team_id)
if team is None:
raise HTTPException(404, f"目标 Team 不存在: id={req.team_id}")
repo = TeamRepository(db)
row = await repo.add_alias(req.alias, req.team_id)
logger.info("添加 Team 别名: %s -> team_id=%s", req.alias, req.team_id)
return TeamAliasOut(
alias_normalized=row.alias_normalized,
team_id=row.team_id,
original_alias=row.original_alias,
)
+89 -18
View File
@@ -10,13 +10,16 @@ from __future__ import annotations
import asyncio
import logging
import uuid
from datetime import datetime, timezone
from fastapi import APIRouter, Depends, HTTPException
from src.api.deps import require_admin
from src.api.schemas import IngestBzzoiroRequest
from src.api.schemas import IngestBzzoiroRequest, IngestBzzoiroResponse
from src.data.config import BZZOIRO_LEAGUE_IDS
from src.data.bzzoiro import ingest_bzzoiro_event_stats, ingest_bzzoiro_standings
from src.data.bzzoiro_standings import ingest_bzzoiro_standings
from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats
from src.data.sources import get_source
from src.db.unit_of_work import get_uow
@@ -30,6 +33,19 @@ _background_tasks: set[asyncio.Task] = set()
VALID_TASKS = {"events", "standings", "stats", "all"}
def _accumulate_ingest_result(merged: dict, code: str, r: dict) -> None:
"""P1-A: 累加单联赛采集结果。联赛级计数读 r["leagues"][code],顶层读 total_*。"""
merged["total_inserted"] += r.get("total_inserted", 0)
merged["total_updated"] += r.get("total_updated", 0)
merged["errors"].extend(r.get("errors", []))
# 联赛级计数必须来自 leagues[code],而非顶层 r.get("inserted")
league_r = r.get("leagues", {}).get(code, {})
acc = merged["leagues"].setdefault(code, {"inserted": 0, "updated": 0, "errors": []})
acc["inserted"] += league_r.get("inserted", 0)
acc["updated"] += league_r.get("updated", 0)
acc["errors"].extend(r.get("errors", []))
def _spawn(coro) -> None:
"""启动后台采集任务;异常已在任务内记录到系统日志。"""
task = asyncio.create_task(coro)
@@ -37,22 +53,58 @@ def _spawn(coro) -> None:
task.add_done_callback(_background_tasks.discard)
@router.post("/ingest/bzzoiro", dependencies=[Depends(require_admin)])
@router.post("/ingest/bzzoiro", response_model=IngestBzzoiroResponse, dependencies=[Depends(require_admin)])
async def ingest_bzzoiro_route(req: IngestBzzoiroRequest):
"""触发 bzzoiro 采集(events / standings / stats / all)。"""
"""触发 bzzoiro 采集(events / standings / stats / all)。
启动后台任务前写入 ingest_jobs(pending),响应返回 job_id 供前端轮询
兼容原 message 字段(仍返回)
"""
if req.task not in VALID_TASKS:
raise HTTPException(status_code=422, detail=f"未知任务类型: {req.task}(可选: {', '.join(sorted(VALID_TASKS))})")
leagues = req.leagues or list(BZZOIRO_LEAGUE_IDS.keys())
task_label = {"events": "比赛数据", "standings": "积分榜", "stats": "统计回填", "all": "全量(比赛+积分榜+统计)"}[req.task]
_spawn(_run_bzzoiro(req.task, leagues, req))
return {
"ok": True,
"message": f"采集任务已启动(后台执行,任务: {task_label}),请在「系统日志」查看进度与结果",
job_id = await _create_ingest_job(req.task, leagues, req)
_spawn(_run_bzzoiro(job_id, req.task, leagues, req))
return IngestBzzoiroResponse(
ok=True,
job_id=job_id,
message=f"采集任务已启动(后台执行,任务: {task_label}),请到「数据采集」页跟踪进度",
)
async def _create_ingest_job(task: str, leagues: list[str], req: IngestBzzoiroRequest) -> str:
"""写入一条 ingest_jobs(pending),返回 job_id。"""
from src.db.models import IngestJob
job_id = str(uuid.uuid4())
params = {
"leagues": leagues,
"date_from": req.date_from,
"date_to": req.date_to,
"status": req.status,
"task": task,
"limit": req.limit,
"season": req.season,
}
async with get_uow() as session:
job = IngestJob(id=job_id, task=task, params=params, status="pending")
session.add(job)
logger.info("ingest_jobs 创建: job=%s task=%s leagues=%s", job_id, task, leagues)
return job_id
async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest) -> None:
"""后台执行 bzzoiro 采集:上游限速时单次可能耗时数分钟,必须脱离请求生命周期。"""
async def _run_bzzoiro(job_id: str, task: str, leagues: list[str], req: IngestBzzoiroRequest) -> None:
"""后台执行 bzzoiro 采集:上游限速时单次可能耗时数分钟,必须脱离请求生命周期。
状态流转: pending running (success|failed)
"""
from src.db.models import IngestJob
await _update_job(job_id, status="running", started_at=datetime.now(timezone.utc))
result: dict = {}
try:
if task in ("events", "all"):
statuses = [req.status] if req.status else ["finished", "scheduled"]
@@ -66,19 +118,14 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
session, leagues=[code],
date_from=req.date_from, date_to=req.date_to, status=st,
)
merged["total_inserted"] += r.get("total_inserted", 0)
merged["total_updated"] += r.get("total_updated", 0)
merged["errors"].extend(r.get("errors", []))
acc = merged["leagues"].setdefault(code, {"inserted": 0, "updated": 0, "errors": []})
acc["inserted"] += r.get("inserted", 0)
acc["updated"] += r.get("updated", 0)
acc["errors"].extend(r.get("errors", []))
_accumulate_ingest_result(merged, code, r)
logger.info(
"bzzoiro 比赛采集完成: 新增 %d, 更新 %d, 联赛 %d 个, 状态 %s",
merged["total_inserted"], merged["total_updated"], len(merged["leagues"]), statuses,
)
if merged["errors"]:
logger.warning("bzzoiro 比赛采集错误 %d 条: %s", len(merged["errors"]), merged["errors"][:3])
result["events"] = merged
if task in ("standings", "all"):
async with get_uow() as session:
@@ -87,6 +134,7 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
logger.warning("bzzoiro 积分榜采集部分失败: %s", r["errors"][:3])
else:
logger.info("bzzoiro 积分榜采集完成: upsert %d", r["total_upserted"])
result["standings"] = r
if task in ("stats", "all"):
async with get_uow() as session:
@@ -95,5 +143,28 @@ async def _run_bzzoiro(task: str, leagues: list[str], req: IngestBzzoiroRequest)
)
if r["errors"]:
logger.warning("bzzoiro 统计回填错误 %d 条: %s", len(r["errors"]), r["errors"][:3])
except Exception:
result["stats"] = r
await _update_job(job_id, status="success", result=result, finished_at=datetime.now(timezone.utc))
logger.info("ingest_jobs 完成: job=%s task=%s", job_id, task)
except Exception as e:
logger.exception("bzzoiro 采集任务失败(task=%s)", task)
await _update_job(
job_id, status="failed", error=str(e), finished_at=datetime.now(timezone.utc),
)
async def _update_job(job_id: str, **fields) -> None:
"""更新 ingest_jobs 单行;失败仅记日志,绝不抛异常(避免干扰采集主流程)。"""
from src.db.models import IngestJob
try:
async with get_uow() as session:
job = await session.get(IngestJob, job_id)
if job is None:
logger.warning("ingest_jobs 更新失败: job=%s 不存在", job_id)
return
for k, v in fields.items():
setattr(job, k, v)
except Exception:
logger.warning("ingest_jobs 更新异常: job=%s fields=%s", job_id, list(fields.keys()))
+79 -40
View File
@@ -5,15 +5,29 @@ from datetime import datetime, timedelta
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy import func, or_, select
from sqlalchemy.orm import selectinload
from sqlalchemy.orm import load_only, selectinload
from src.api.schemas import MatchListOut, MatchOut, PredictionOut
from src.db.base import AsyncSession, get_db_read
from src.db.models import League, Match, Prediction, Standing
from src.db.models import League, Match, MatchStats, Prediction, Standing, Team
router = APIRouter(prefix="/api/v1", tags=["data"])
def _parse_cursor(cursor: str) -> tuple[datetime, int]:
"""P1-B: 解析游标。非法格式 → HTTPException(400, code=INVALID_CURSOR)。"""
try:
last_date_str, last_id_str = cursor.split("|", 1)
last_date = datetime.fromisoformat(last_date_str)
last_id = int(last_id_str)
return last_date, last_id
except (ValueError, AttributeError) as e:
raise HTTPException(
status_code=400,
detail={"code": "INVALID_CURSOR", "message": f"非法游标格式: {cursor}(应为 date_iso|id)"},
) from e
def _stats_dict(stats) -> dict | None:
"""把 MatchStats ORM 对象序列化为前端可读的扁平 dict。"""
if stats is None:
@@ -49,30 +63,37 @@ async def list_matches(
limit: int = Query(50, ge=1, le=100),
db: AsyncSession = Depends(get_db_read),
):
"""比赛列表(游标分页)。"""
q = select(Match).options(selectinload(Match.league), selectinload(Match.home_team), selectinload(Match.away_team))
"""比赛列表(游标分页)。
加载策略(列表 vs 详情):
- 列表: selectinload 序列化需要的 3 个关系 + stats,且用 load_only 限定列
(League.code / Team.name,name_zh / MatchStats.home_xg,away_xg),避免传输全列;
同时一次性加载 stats 消除 N+1(m.stats.home_xg 此前触发懒加载)
- 详情(/matches/{id}):保持完整 options(league/teams/stats 全列 + 最近预测)
"""
q = select(Match).options(
selectinload(Match.league).load_only(League.code),
selectinload(Match.home_team).load_only(Team.name, Team.name_zh),
selectinload(Match.away_team).load_only(Team.name, Team.name_zh),
selectinload(Match.stats).load_only(MatchStats.home_xg, MatchStats.away_xg),
)
if cursor:
try:
# 用 | 分隔,避免 isoformat 含 _ 时解析失败
last_date_str, last_id_str = cursor.split("|", 1)
last_date = datetime.fromisoformat(last_date_str)
last_id = int(last_id_str)
# 游标方向必须与排序方向一致:
# - scheduled(ASC):取「更大」的未开赛场次
# - 其它(DESC):取「更小」的已赛场次
if status == "scheduled":
q = q.where(
(Match.match_date > last_date) |
((Match.match_date == last_date) & (Match.id > last_id))
)
else:
q = q.where(
(Match.match_date < last_date) |
((Match.match_date == last_date) & (Match.id < last_id))
)
except (ValueError, AttributeError):
pass
# P1-B: 解析非法 → 400 + code=INVALID_CURSOR,而非静默忽略
last_date, last_id = _parse_cursor(cursor)
# 游标方向必须与排序方向一致:
# - scheduled(ASC):取「更大」的未开赛场次
# - 其它(DESC):取「更小」的已赛场次
if status == "scheduled":
q = q.where(
(Match.match_date > last_date) |
((Match.match_date == last_date) & (Match.id > last_id))
)
else:
q = q.where(
(Match.match_date < last_date) |
((Match.match_date == last_date) & (Match.id < last_id))
)
if league:
stmt = select(League.id).where(League.code == league)
@@ -148,15 +169,28 @@ async def get_match(match_id: int, db: AsyncSession = Depends(get_db_read)):
m = (await db.execute(stmt)).scalar_one_or_none()
if m is None:
raise HTTPException(404, "match not found")
# 最近预测(倒序,最多 5 条)——复用 PredictionOut 结构,只读,不触发 LLM
# P1-C: 公开预测仅 run_type=live 且 status=success(屏蔽回测/失败预测)
preds = (
await db.execute(
select(Prediction)
.where(Prediction.match_id == match_id)
.where(Prediction.run_type == "live")
.where(Prediction.status == "success")
.order_by(Prediction.created_at.desc())
.limit(5)
)
).scalars().all()
# P1-C: 公开接口的预测不含 reasoning/agent_outputs(避免泄露内部推理细节)
recent_predictions = [
{
"id": p.id, "match_id": p.match_id, "provider": p.provider, "model": p.model,
"prompt_version": p.prompt_version, "mode": p.mode or "single",
"pred_home_goals": p.pred_home_goals, "pred_away_goals": p.pred_away_goals,
"pred_1x2": p.pred_1x2, "subjective_confidence": p.subjective_confidence,
"created_at": p.created_at.isoformat() if p.created_at else None,
}
for p in preds
]
return MatchOut(
id=m.id,
league_code=m.league.code if m.league else None,
@@ -173,20 +207,7 @@ async def get_match(match_id: int, db: AsyncSession = Depends(get_db_read)):
home_xg=m.stats.home_xg if m.stats else None,
away_xg=m.stats.away_xg if m.stats else None,
stats=_stats_dict(m.stats) if m.stats else None,
recent_predictions=[
PredictionOut(
id=p.id, match_id=p.match_id, provider=p.provider, model=p.model,
prompt_version=p.prompt_version, mode=p.mode or "single",
pred_home_goals=p.pred_home_goals, pred_away_goals=p.pred_away_goals,
alt_pred_home_goals=p.alt_pred_home_goals, alt_pred_away_goals=p.alt_pred_away_goals,
pred_1x2=p.pred_1x2, subjective_confidence=p.subjective_confidence,
reasoning=p.reasoning, status=p.status or "success",
agent_outputs=p.agent_outputs, agent_weights=p.agent_weights,
created_at=p.created_at, actual_home_goals=p.actual_home_goals,
actual_away_goals=p.actual_away_goals, settled=p.settled,
)
for p in preds
],
recent_predictions=recent_predictions,
)
@@ -274,7 +295,9 @@ async def list_standings(
):
"""联赛积分榜(只读)。按联赛分组,每张榜按 position 排序。
season 为空时返回每个联赛最新采集到的赛季榜单(适合前端"查看最新积分榜")
P0-02: standings 为追加快照,公开接口取每队 available_at 最新快照
(league_id, season, team_id 上按 available_at 取最新)
season 为空时返回每个联赛最新采集到的赛季榜单
"""
# 取每个联赛最新赛季(当 season 为空时)
latest_seasons: dict[int, str] = {}
@@ -287,9 +310,25 @@ async def list_standings(
).all()
latest_seasons = {r.league_id: r.latest for r in rows}
# P0-02: 子查询取每队最新 available_at 快照,再 JOIN 回主表拿完整行 + League
latest_per_team = (
select(
Standing.league_id, Standing.season, Standing.team_id,
func.max(Standing.available_at).label("max_available"),
)
.group_by(Standing.league_id, Standing.season, Standing.team_id)
.subquery("latest_per_team")
)
q = (
select(Standing, League)
.join(League, League.id == Standing.league_id)
.join(
latest_per_team,
(Standing.league_id == latest_per_team.c.league_id)
& (Standing.season == latest_per_team.c.season)
& (Standing.team_id == latest_per_team.c.team_id)
& (Standing.available_at == latest_per_team.c.max_available),
)
.order_by(League.name.asc(), Standing.position.asc())
)
if league:
+71 -93
View File
@@ -2,11 +2,15 @@
安全改进:
- 限流: 每分钟 10 / IP(内存实现)
- P1-D: 全局 LLM 并发限制(默认 4),防止过多并发 LLM 调用压垮服务
- DB 连接: session 模式,LLM 调用期间不持有连接
- P1-async: 预测改为异步(后台任务 + 轮询),避免网关超时(Cloudflare 100s)
"""
from __future__ import annotations
import asyncio
import logging
import uuid
from fastapi import APIRouter, Depends, HTTPException, Query, Request
from sqlalchemy import select
@@ -22,19 +26,65 @@ logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/v1", tags=["predict"])
# P1-D: 全局 LLM 并发限制。与 orchestrator 内的 match 级 Semaphore(8) 并存,
# 此处在路由层限制单实例全 LLM 调用(所有模式汇总),默认 4。
_GLOBAL_LLM_SEMAPHORE = asyncio.Semaphore(4)
@router.post("/predict", response_model=PredictOut, dependencies=[Depends(rate_limit_predict)])
# P1-async: 预测任务内存存储(job_id → 结果/异常)。单进程部署足够,无需入库。
_predict_jobs: dict[str, dict] = {}
async def _predict_with_concurrency(req: PredictRequest) -> PredictResult:
"""P1-D: 在全局 LLM 并发限制下执行预测。"""
async with _GLOBAL_LLM_SEMAPHORE:
return await predict_match(
req.match_id,
model=req.model,
prompt_version=req.prompt_version,
mode=req.mode,
)
async def _run_predict_async(job_id: str, req: PredictRequest) -> None:
"""P1-async: 后台执行预测,结果写入 _predict_jobs。"""
try:
result = await _predict_with_concurrency(req)
_predict_jobs[job_id] = {
"status": "success",
"result": {
"prediction_id": result.prediction_id,
"provider": result.provider,
"model": result.model,
"prompt_version": result.prompt_version,
"mode": req.mode,
"pred_home_goals": result.pred_home_goals,
"pred_away_goals": result.pred_away_goals,
"alt_pred_home_goals": result.alt_pred_home_goals,
"alt_pred_away_goals": result.alt_pred_away_goals,
"pred_1x2": result.pred_1x2,
"subjective_confidence": result.subjective_confidence,
"reasoning": result.reasoning,
"status": result.status,
"agent_outputs": result.agent_outputs,
"agent_weights": result.agent_weights,
"context": result.context,
"latency_ms": result.latency_ms,
},
}
except Exception as e:
logger.exception("predict job %s failed", job_id)
_predict_jobs[job_id] = {"status": "failed", "error": str(e)}
@router.post("/predict", dependencies=[Depends(rate_limit_predict)])
async def predict(req: PredictRequest, request: Request):
"""对一场比赛调 LLM 预测。mode=multi(默认,5专家+终裁)、single 或 baseline。
"""对一场比赛调 LLM 预测(异步)。mode=multi(默认,5专家+终裁)、single 或 baseline。
公开接口,仅做限流保护(不要求登录)
DB 连接优化:
1. read session 检查比赛存在性/状态
2. 释放连接后调用 LLM(可能几十秒)
3. write session 保存 Prediction
P1-async: 立即返回 job_id,预测在后台执行,前端轮询 GET /predict/jobs/{job_id}
避免多专家预测耗时 60-180s 触发网关超时(Cloudflare 100s HTTP 524)
"""
# 1. 短 read session: 检查比赛(连接立即释放)
# 1. 短 read session: 检查比赛存在性/状态
async with short_read() as session:
m = await session.get(Match, req.match_id)
if m is None:
@@ -42,93 +92,21 @@ async def predict(req: PredictRequest, request: Request):
if m.match_status == "finished":
raise HTTPException(400, "该比赛已完赛,不再支持预测")
# 2. 预测调用(不持有任何 DB 连接)
try:
result = await predict_match(
req.match_id,
model=req.model,
prompt_version=req.prompt_version,
mode=req.mode,
)
except ValueError as e:
msg = str(e)
if "已结算" in msg:
raise HTTPException(409, msg)
logger.warning("predict validation error: %s", e)
raise HTTPException(404, "比赛不存在")
except RuntimeError as e:
logger.error("predict LLM error: %s", e)
raise HTTPException(502, "LLM 预测失败,请查看服务器日志")
except Exception as e:
logger.exception("predict unexpected error")
raise HTTPException(500, "预测失败,请查看服务器日志")
# D2: 三种模式统一返回 PredictResult —— 字段映射单一化,无 dict 分支。
# 仅 baseline 的 prediction_id 需要在此落库补齐(服务层不落库)。
if req.mode == "baseline":
prediction_id = await _persist_baseline(req.match_id, result)
else:
prediction_id = result.prediction_id
# 3. 结果映射(无 DB 访问)
logger.info(
"预测完成 match=%s mode=%s pred=%s:%s (%s)",
req.match_id, req.mode,
result.pred_home_goals, result.pred_away_goals, result.pred_1x2,
)
return PredictOut(
prediction_id=prediction_id,
provider=result.provider,
model=result.model,
prompt_version=result.prompt_version,
mode=req.mode,
pred_home_goals=result.pred_home_goals,
pred_away_goals=result.pred_away_goals,
alt_pred_home_goals=result.alt_pred_home_goals,
alt_pred_away_goals=result.alt_pred_away_goals,
pred_1x2=result.pred_1x2,
subjective_confidence=result.subjective_confidence,
reasoning=result.reasoning,
status=result.status,
agent_outputs=result.agent_outputs,
agent_weights=result.agent_weights,
context=result.context,
latency_ms=result.latency_ms,
prompt_tokens=result.prompt_tokens,
completion_tokens=result.completion_tokens,
rate_limit_remaining=get_predict_rate_limit_remaining(request),
)
# 2. P1-async: 启动后台任务,立即返回 job_id
job_id = str(uuid.uuid4())
_predict_jobs[job_id] = {"status": "running"}
asyncio.create_task(_run_predict_async(job_id, req))
logger.info("predict job started: %s match=%s mode=%s", job_id, req.match_id, req.mode)
return {"job_id": job_id, "status": "running", "poll_url": f"/api/v1/predict/jobs/{job_id}"}
async def _persist_baseline(match_id: int, baseline: PredictResult) -> int:
"""将基线预测结果写入 prediction 表,复用 upsert 语义。"""
from src.db.unit_of_work import get_uow
from src.llm.predict import _upsert_prediction
async with get_uow() as session:
pred = await _upsert_prediction(
session,
match_id=match_id,
provider_name="baseline",
model="baseline",
mode="baseline",
run_type="live", # baseline 是 live 预测的变体,符合 ck_run_type_enum
values={
"prompt_version": baseline.prompt_version,
"prompt_tokens": baseline.prompt_tokens or 0,
"completion_tokens": baseline.completion_tokens or 0,
"latency_ms": baseline.latency_ms or 0,
"pred_home_goals": baseline.pred_home_goals,
"pred_away_goals": baseline.pred_away_goals,
"pred_1x2": baseline.pred_1x2,
"subjective_confidence": baseline.subjective_confidence,
"reasoning": baseline.reasoning,
"raw_response": baseline.raw,
"status": "success",
},
)
return pred.id
@router.get("/predict/jobs/{job_id}")
async def get_predict_job(job_id: str):
"""P1-async: 轮询预测任务状态。"""
job = _predict_jobs.get(job_id)
if job is None:
raise HTTPException(404, f"预测任务不存在: {job_id}")
return job
@router.get("/predictions", response_model=list[PredictionOut], dependencies=[Depends(require_admin)])
+2 -1
View File
@@ -10,7 +10,8 @@ from sqlalchemy import select, delete
from src.api.deps import require_admin
from src.api.schemas import ScheduleIn, ScheduleUpdate, ScheduleOut
from src.core.scheduler import scheduler
from src.data.bzzoiro import ingest_bzzoiro_event_stats, ingest_bzzoiro_standings
from src.data.bzzoiro_standings import ingest_bzzoiro_standings
from src.data.bzzoiro_stats import ingest_bzzoiro_event_stats
from src.data.sources import get_source
from src.data.config import BZZOIRO_LEAGUE_IDS
from src.db.base import AsyncSession, get_db_read
+36 -15
View File
@@ -7,13 +7,6 @@ from typing import Any
from pydantic import BaseModel, Field
class LeagueOut(BaseModel):
id: int
code: str
name: str
country: str | None
class MatchOut(BaseModel):
id: int
league_code: str | None
@@ -31,8 +24,8 @@ class MatchOut(BaseModel):
away_xg: float | None = None
# 比赛详细统计(bzzoiro /events/{id}/stats/),无统计为 None
stats: dict | None = None
# 该场比赛的最近预测摘要(按时间倒序,最多 5 条;无预测为空)
recent_predictions: list[PredictionOut] = []
# P1-C: 公开接口的预测不含 reasoning/agent_outputs;仅 live+success 路由已过滤
recent_predictions: list[dict] = []
class MatchListOut(BaseModel):
@@ -43,7 +36,7 @@ class MatchListOut(BaseModel):
class PredictRequest(BaseModel):
match_id: int
provider: str | None = None
# P1-D: 删除未接线的 provider 字段(符合"名不副实则删除");provider 由服务端配置决定。
model: str | None = None
prompt_version: str | None = None
mode: str = Field(
@@ -117,11 +110,39 @@ class IngestBzzoiroRequest(BaseModel):
season: str | None = Field(None, description="standings 赛季,如 '2026-2027';空 = 当前赛季")
class IngestResponse(BaseModel):
leagues: dict
total_inserted: int
total_updated: int
errors: list[str] = []
class TeamAliasIn(BaseModel):
"""POST /api/v1/admin/teams/aliases 请求体:为已有 Team 添加别名。"""
alias: str = Field(..., min_length=1, max_length=120, description="球队别名(原始写法)")
team_id: int = Field(..., gt=0, description="归一后的目标 teams.id")
class TeamAliasOut(BaseModel):
alias_normalized: str
team_id: int
original_alias: str
class IngestBzzoiroResponse(BaseModel):
"""POST /api/v1/ingest/bzzoiro 响应:兼容原 message 字段,新增 job_id 供轮询。"""
ok: bool = True
job_id: str = Field(..., description="采集任务 ID(GET /api/v1/admin/ingest/jobs/{job_id} 轮询)")
message: str = ""
class IngestJobOut(BaseModel):
"""采集任务状态详情。"""
id: str
task: str
params: dict
status: str # pending | running | success | failed
result: dict | None = None
error: str | None = None
created_at: datetime | None = None
started_at: datetime | None = None
finished_at: datetime | None = None
class ScheduleIn(BaseModel):
+14
View File
@@ -11,6 +11,14 @@ class Settings(BaseSettings):
# --- app ---
APP_ENV: str = "development"
LOG_LEVEL: str = "INFO"
# 日志持久化:空(默认)只输出 stdout + Admin 内存日志页(重启清零)。
# 填文件路径(如 /app/logs/app.log)后额外写入滚动文件(单文件 10MB × 5 份),
# 进程/容器重启不丢。容器部署需配合 volume 挂载该目录,否则重建仍会丢。
LOG_FILE: str = ""
# P3-3:多 worker 时应用内限流与 KeyRing 各自独立计数(配额放大 N 倍)。
# 设为 True 时若以多 worker 启动 uvicorn 则拒绝启动,避免静默配额漂移。
# 仅在你已前置 Nginx/网关做全局限流、确认不需要此守护时留空/False。
STRICT_SINGLE_WORKER: bool = False
# 生产环境强制要求管理鉴权配置,即使 APP_ENV=production 也生效。
# True 时若 auth_configured() 为 False 则拒绝(503),development 保持 fail-open。
REQUIRE_ADMIN_AUTH: bool = False
@@ -28,6 +36,12 @@ class Settings(BaseSettings):
LLM_SPECIALIST_MODEL: str = ""
LLM_AGGREGATOR_MODEL: str = ""
# ── 预测缓存 ──
# 预测响应缓存后端:空(默认)=进程内 LRU+TTL 字典;填 redis://host:port/db 启用 Redis。
# Redis 失败自动降级内存缓存并 warning,不中断预测;不强制依赖 redis 包。
# TTL 固定 300s(5 分钟),键格式与内存后端一致(含 prompt 模板 hash)。
PREDICT_CACHE_URL: str = ""
# --- data sources ---
BZZOIRO_KEY: str = ""
BZZOIRO_BASE: str = "https://sports.bzzoiro.com/api/v2"
+65 -8
View File
@@ -65,14 +65,71 @@ def get_entries(
return out
def setup_memory_logging(level: str = "INFO") -> None:
"""挂载内存 handler 到 root logger(幂等),并确保 root 级别不低于 INFO。"""
def setup_logging(level: str = "INFO", log_file: str = "") -> None:
"""配置应用日志:stdout(容器收集) + 内存环形缓冲(Admin 日志页) + 可选滚动文件(持久化)。
幂等:重复调用不会重复挂 handler(文件 handler abspath 判重,相对/绝对
路径指向同一文件视为同一个)文件写入基础设施失败只记 warning,绝不
影响启动与业务 _safe_write_ingest_failure 同级约束:可观测性
基础设施不许拖垮主流程
"""
import os
from logging.handlers import RotatingFileHandler
root = logging.getLogger()
if any(isinstance(h, MemoryLogHandler) for h in root.handlers):
return
handler = MemoryLogHandler()
handler.setLevel(logging.INFO)
handler.addFilter(_SQLNoiseFilter())
root.addHandler(handler)
if root.level == logging.NOTSET or root.level > logging.INFO:
root.setLevel(getattr(logging, level.upper(), logging.INFO))
# uvicorn 的 logger 默认 propagate=False:未捕获异常的 traceback 只进
# stderr,不经过 root 的任何 handler —— Admin 日志页与文件日志都看不到,
# 线上 500 排障无据可查。打开 propagate 让它们进入内存缓冲/滚动文件。
for uv_name in ("uvicorn", "uvicorn.error"):
uv_logger = logging.getLogger(uv_name)
if not uv_logger.propagate:
uv_logger.propagate = True
if not any(isinstance(h, MemoryLogHandler) for h in root.handlers):
handler = MemoryLogHandler()
handler.setLevel(logging.INFO)
handler.addFilter(_SQLNoiseFilter())
root.addHandler(handler)
if not log_file:
return
# 滚动上限:单文件 10MB × 当前+5 份 ≈ 60MB,足够回溯数周的关键事件,
# 又不会吃满磁盘。格式含时间/级别/logger 名,便于事后 grep 排查。
target = os.path.abspath(log_file)
if any(
isinstance(h, RotatingFileHandler) and getattr(h, "baseFilename", None) == target
for h in root.handlers
):
return
try:
os.makedirs(os.path.dirname(target), exist_ok=True)
file_handler = RotatingFileHandler(
target,
maxBytes=10 * 1024 * 1024,
backupCount=5,
encoding="utf-8",
)
file_handler.setFormatter(
logging.Formatter("%(asctime)s %(levelname)s %(name)s %(message)s")
)
file_handler.setLevel(logging.INFO)
file_handler.addFilter(_SQLNoiseFilter())
root.addHandler(file_handler)
logging.getLogger(__name__).info("文件日志已启用: %s", log_file)
except PermissionError:
# 容器场景最常见:挂载卷目录属主是 root,进程是非 root 用户。
# 修复方向:镜像内预创建目录并 chown(Dockerfile),或重建空卷。
logging.getLogger(__name__).warning(
"启用文件日志失败(%s):无写权限。容器部署请确认镜像已预创建该目录"
"并 chown 给运行用户(compose 卷挂载点默认 root 属主);"
"宿主机直跑请检查目录权限。本次仅保留 stdout/内存日志。",
log_file,
)
except Exception:
logging.getLogger(__name__).warning(
"启用文件日志失败(%s),仅保留 stdout/内存日志", log_file, exc_info=True,
)
+46 -20
View File
@@ -104,17 +104,26 @@ async def get_runtime_value(key: str) -> str:
"""读运行时配置:DB 覆盖值 → .env 默认值 → 空串。
敏感项入库时是密文,读出后自动解密;旧明文(迁移前) decrypt_value 透传
P1-G:DB 故障回落 env;但解密失败在生产环境必须抛出(不得与 DB 异常共用 except)
"""
defn = SETTING_DEFS.get(key)
db_value = None
try:
async with AsyncSessionLocal() as db:
row = await db.get(AppSetting, key)
if row and row.value:
value = crypto.decrypt_value(row.value) if defn and defn.sensitive else row.value
if value:
return value
db_value = row.value
except Exception:
logger.warning("读取运行时配置 %s 失败,回落环境变量", key)
logger.warning("读取运行时配置 %s 失败(DB 故障),回落环境变量", key)
return getattr(settings, key, "") or ""
if db_value is None:
return getattr(settings, key, "") or ""
# 解密逻辑独立于 DB 异常处理(P1-G:解密失败生产环境必须抛出)
value = crypto.decrypt_value(db_value) if defn and defn.sensitive else db_value
if value:
return value
return getattr(settings, key, "") or ""
@@ -133,8 +142,40 @@ async def set_runtime_value(key: str, value: str) -> None:
logger.info("运行时配置 %s 已更新", key)
async def get_setting_origin(key: str) -> tuple[str, str]:
"""返回 (origin, 当前生效值)。origin ∈ db / env / none。
P1-G:DB 故障回落 env;解密失败在生产环境抛出
"""
defn = SETTING_DEFS.get(key)
db_value = None
try:
async with AsyncSessionLocal() as db:
row = await db.get(AppSetting, key)
if row and row.value:
db_value = row.value
except Exception:
logger.warning("读取运行时配置 %s 来源失败(DB 故障),按环境变量处理", key)
env_value = getattr(settings, key, "") or ""
return ("env", env_value) if env_value else ("none", "")
if db_value is None:
return ("none", "")
try:
value = crypto.decrypt_value(db_value) if defn and defn.sensitive else db_value
return "db", value
except ValueError as e:
# P1-G:解密失败(SECRET_KEY 不一致)在生产环境必须抛出,不得静默回落
if settings.APP_ENV == "production":
raise
logger.warning("解密 %s 失败(非生产环境回落): %s", key, e)
env_value = getattr(settings, key, "") or ""
return ("env", env_value) if env_value else ("none", "")
async def clear_runtime_value(key: str) -> None:
"""除 DB 覆盖值,回落 .env(调用方需先校验 key 在白名单内)。"""
"""除 DB 覆盖值,回落 .env(调用方需先校验 key 在白名单内)。"""
async with AsyncSessionLocal() as db:
row = await db.get(AppSetting, key)
if row is not None:
@@ -143,21 +184,6 @@ async def clear_runtime_value(key: str) -> None:
logger.info("运行时配置 %s 已清除覆盖", key)
async def get_setting_origin(key: str) -> tuple[str, str]:
"""返回 (origin, 当前生效值)。origin ∈ db / env / none。"""
defn = SETTING_DEFS.get(key)
try:
async with AsyncSessionLocal() as db:
row = await db.get(AppSetting, key)
if row and row.value:
value = crypto.decrypt_value(row.value) if defn and defn.sensitive else row.value
return "db", value
except Exception:
logger.warning("读取运行时配置 %s 来源失败,按环境变量处理", key)
env_value = getattr(settings, key, "") or ""
return ("env", env_value) if env_value else ("none", "")
async def migrate_plaintext_sensitive_settings() -> int:
"""一次性迁移:把库中仍是明文的敏感项加密(幂等,启动时执行)。
+1 -1
View File
@@ -144,7 +144,7 @@ class DataQualityScheduler:
.select_from(Match)
.outerjoin(MatchStats, Match.id == MatchStats.match_id)
.where(Match.match_status == "finished")
.where(MatchStats.id.is_(None))
.where(MatchStats.match_id.is_(None))
)
).scalar() or 0
-4
View File
@@ -30,10 +30,6 @@ _MIN_SECRET_KEY_LEN = 16
_WEAK_DB_PATTERNS = ("football:football@", "admin:admin@", "password@", "123456@")
class SecurityCheckError(Exception):
"""生产环境安全校验失败。"""
async def _auth_configured() -> bool:
"""运行时鉴权是否已配置(含数据库密码哈希/.env 明文/API Key)。"""
if await get_admin_password_hash():
+60 -787
View File
@@ -1,793 +1,66 @@
"""Bzzoiro 数据源:抓取 + 入库(单一数据源)。
"""Bzzoiro 数据源:抓取 + 入库(单一数据源)—— 聚合门面
三条管线:
1. events 比赛日程/比分(/events/), source_event_id 血缘
2. standings 联赛积分榜快照(/leagues/{id}/standings/)
3. stats 已完赛比赛详细统计回填(/events/{id}/stats/)
实现按管线拆分(单文件 多模块),本模块只做再导出,保持不变量:
1. sources._load_sources() 仍从本模块导入 BzzoiroSource(注册表入口不变);
2. 测试与脚本对 `bz.<名称>` monkeypatch 语义不变 抓取函数 / REQUEST_INTERVAL
仍经本门面解析可替换;Bronze 写入助手(_write_raw_event/_write_lineage/
_safe_write_ingest_failure)已改为管线模块直接 import pipeline_write,
测试需 patch `src.data.pipeline_write.*` 源模块
三条管线(各自模块):
1. events 比赛日程/比分(/events/), source_event_id 血缘 bzzoiro_events.py
2. standings 联赛积分榜快照(/leagues/{id}/standings/) bzzoiro_standings.py
3. stats 已完赛比赛详细统计回填(/events/{id}/stats/) bzzoiro_stats.py
共享基础:HTTP 抓取( key 轮换)与字段转换 bzzoiro_common.py;
Bronze 基础设施(RawEvent/IngestFailure/DataLineage) pipeline_write.py
(各管线模块直接 import pipeline_write,不再经本门面转发)
D4(工程债): Team/League/Match 的查找/创建经 Repository (src/db/repositories.py),
本模块不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制,这里 flush)
Standing/RawEvent/Lineage 等管线内私有读写仍在本模块内实现,不强行 Repository
各管线不直接控制事务(commit/rollback 由调用方 UnitOfWork 控制, flush)
Standing/RawEvent/Lineage 等管线内私有读写仍不强行 Repository
"""
from __future__ import annotations
import asyncio
import logging
import random
from collections.abc import Iterable
from datetime import datetime, timedelta, timezone
from sqlalchemy import select
import httpx
from src.core.runtime_config import get_runtime_value
from src.core.http_client import get_client
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES, REQUEST_INTERVAL
from src.data.key_ring import _mask, get_key_ring
from src.data.normalize import normalize_bzzoiro
from src.data.team_names_zh import zh_name
from src.data.sources import register
from src.db.models import Match, MatchStats, Standing, Team, RawEvent, IngestFailure, DataLineage
from src.db.repositories import LeagueRepository, MatchRepository, TeamRepository
logger = logging.getLogger(__name__)
def _to_date(value):
"""把 datetime / date / str 统一成 `date`。"""
if value is None:
return None
if hasattr(value, "date") and callable(value.date):
return value.date()
return value
def _to_int_or_none(value) -> int | None:
"""宽松转 int(用于上游 ID 解析,失败返回 None 不抛错)。"""
if value is None:
return None
try:
return int(str(value).strip())
except (TypeError, ValueError):
return None
def _match_key(home_team_id: int, away_team_id: int, match_date) -> tuple[int, int, str]:
"""比赛去重键:(主队, 客队, 天级日期 ISO 字符串)。
统一在这里构造,避免"预加载时用 str(date)、写入时用 isoformat()"这类
隐式格式依赖 两者当前恰好相等,但一旦有人改动其一就会静默失配,
导致所有比赛被判为不存在而重复插入
"""
d = _to_date(match_date)
return (home_team_id, away_team_id, d.isoformat() if d is not None else "")
async def _fetch_json_async(path: str, params: dict | None = None, max_retries: int = 3) -> dict | list:
"""异步 HTTP(bzzoiro 使用 httpx,不再阻塞事件循环线程池)。
key 轮换:遇到 429 自动切换到下一个 key;全部 key 冷却时等待最早恢复
"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
url = f"{base}/{path.lstrip('/')}"
key = ring.get()
if not key:
raise RuntimeError("BZZOIRO_KEY 未设置")
last_exc: Exception | None = None
for attempt in range(max_retries):
headers = {
"Authorization": f"Token {key}",
"Accept": "application/json",
}
try:
client = get_client()
# 整请求兜底: httpx 无 total 超时,用 wait_for 防「滴水式」限速挂死
resp = await asyncio.wait_for(
client.get(
url, headers=headers, params=params,
timeout=httpx.Timeout(connect=10.0, read=30.0, write=10.0, pool=10.0),
),
timeout=60.0,
)
resp.raise_for_status()
return resp.json()
except Exception as e:
last_exc = e
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 429:
# 限流:标记当前 key 冷却,切换到下一个
new_key = ring.report_rate_limited(key)
if new_key and new_key != key:
logger.info("bzzoiro 429 → 切换 key: %s%s,立即重试", _mask(key), _mask(new_key))
key = new_key
continue # 立即重试,不等待
# 单 key 或全部冷却:等待最早恢复的 key
wait = ring.wait_if_all_blocked()
if wait > 0:
logger.warning("bzzoiro 全部 key 冷却,等待 %.1fs 后重试", wait)
await asyncio.sleep(min(wait, 30.0))
else:
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro 429, retry %d in %.1fs", attempt + 1, delay)
await asyncio.sleep(delay)
key = ring.get() or key
continue
if 500 <= (status or 0) < 600:
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro %d, retry %d in %.1fs", status, attempt + 1, delay)
await asyncio.sleep(delay)
continue
# 网络错误(连接失败/超时)也退避重试
if isinstance(e, (TimeoutError, ConnectionError, OSError)):
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro network error, retry %d in %.1fs: %s", attempt + 1, delay, e)
await asyncio.sleep(delay)
continue
raise
raise RuntimeError(f"bzzoiro request failed after {max_retries} attempts: {last_exc}")
async def fetch_bzzoiro_events(
league_code: str,
*,
status: str = "finished",
date_from: str | None = None,
date_to: str | None = None,
limit: int = 200,
) -> list[dict]:
"""抓取 bzzoiro 原始事件(纯异步,无需 run_in_executor)。"""
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
if league_id is None:
raise ValueError(f"未知联赛代码: {league_code}")
rows: list[dict] = []
offset = 0
while True:
params: dict = {
"league_id": league_id,
"status": status,
"limit": limit,
"offset": offset,
}
if date_from:
params["date_from"] = str(date_from)[:10]
if date_to:
params["date_to"] = str(date_to)[:10]
payload = await _fetch_json_async("/events/", params)
batch = payload.get("results") or []
if not batch:
break
rows.extend(batch)
total = payload.get("total")
offset += limit
if total is not None and offset >= total:
break
if len(batch) < limit:
break
await asyncio.sleep(REQUEST_INTERVAL)
return rows
@register
class BzzoiroSource:
"""bzzoiro 数据源(实现 DataSource 协议)。"""
name = "bzzoiro"
async def ingest(
self,
db,
*,
leagues: Iterable[str],
date_from: str | None = None,
date_to: str | None = None,
status: str = "finished",
) -> dict:
"""采集 bzzoiro → 入库。返回统计。
注意: 本方法不控制事务(commit/rollback),由调用方通过 UnitOfWork 控制
"""
result: dict = {"leagues": {}, "total_inserted": 0, "total_updated": 0, "errors": []}
for code in leagues:
league_r: dict = {"inserted": 0, "updated": 0, "errors": []}
try:
raw_events = await fetch_bzzoiro_events(code, status=status, date_from=date_from, date_to=date_to)
except Exception as e:
# 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批
logger.exception("bzzoiro fetch failed for %s", code)
league_r["errors"].append(f"fetch failed: {e}")
await _safe_write_ingest_failure(
db,
entity_type="events",
source_record_id=None,
error=e,
raw_payload={"league": code, "status": status, "date_from": date_from, "date_to": date_to},
)
result["leagues"][code] = league_r
continue
# D4: 联赛查找/创建经 LeagueRepository(事务仍由调用方 UoW 提交)
league = await LeagueRepository(db).get_or_create(
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
)
team_r = TeamRepository(db)
match_r = MatchRepository(db)
# === 批量优化: 预加载球队和已有比赛到内存 ===
team_name_to_id: dict[str, int] = {}
existing_matches: dict[tuple[int, int, str], Match] = {} # 完整对象,避免重复查询
# (NormalizedMatch, 原始 event) 成对保存:后续写 source_event_id 时
# 必须用配对的那条 event,不能依赖外层循环变量残留值。
normalized_matches: list[tuple] = []
if raw_events:
# 一次遍历: 收集球队名 + 规范化
all_team_names = set()
for raw in raw_events:
nm = normalize_bzzoiro(raw, code)
if nm is not None:
try:
nm.validate()
except Exception as e:
# P1-3: 统一使用 warning,不追加到 errors(仅运行时错误入 errors)
logger.warning("normalize skip: %s", e)
continue
normalized_matches.append((nm, raw))
all_team_names.add(nm.home_team)
all_team_names.add(nm.away_team)
if all_team_names:
team_name_to_id = {
name: t.id
for name, t in (await team_r.get_all_by_names(list(all_team_names))).items()
}
# P1-2: 按需加载,只加载 raw_events 涉及日期范围的比赛(加 30 天缓冲)
# 避免加载联赛全部历史比赛到内存(多赛季采集时内存溢出)
if normalized_matches:
# normalized_matches 存的是 (nm, raw) 元组,遍历需解包
dates = [nm.date for nm, _raw in normalized_matches if nm.date is not None]
if dates:
min_dt = min(dates) - timedelta(days=30)
max_dt = max(dates) + timedelta(days=30)
matches_in_range = await match_r.find_by_league_and_date_range(
league.id, min_dt, max_dt
)
existing_matches = {
_match_key(m.home_team_id, m.away_team_id, m.match_date_date): m
for m in matches_in_range
}
# else: existing_matches 保持空 dict(全量新比赛)
# D1: Bronze 层批次信息(每联赛每批次一个 batch_id;seen 防同批重复写入)
now = datetime.now(timezone.utc)
bronze_batch_id = f"bzzoiro-events-{code}-{now:%Y%m%d%H%M%S}"
bronze_written: set[str] = set()
for nm, raw in normalized_matches:
# D1: RawEvent 幂等键(上游 id 或合成键),插入/变更更新共用
record_id = _events_record_id(code, nm, raw)
# 球队: 内存查找 + 按需创建(D4: 经 TeamRepository)
home_team_id = team_name_to_id.get(nm.home_team)
if home_team_id is None:
home = await team_r.get_or_create(nm.home_team, name_zh=zh_name(nm.home_team))
home_team_id = home.id
team_name_to_id[nm.home_team] = home_team_id
away_team_id = team_name_to_id.get(nm.away_team)
if away_team_id is None:
away = await team_r.get_or_create(nm.away_team, name_zh=zh_name(nm.away_team))
away_team_id = away.id
team_name_to_id[nm.away_team] = away_team_id
# 查找已有比赛: 内存查找
match_key = _match_key(home_team_id, away_team_id, nm.date)
existing_match = existing_matches.get(match_key)
if existing_match is None:
m = Match(
league_id=league.id,
season=nm.season_label or None,
home_team_id=home_team_id,
away_team_id=away_team_id,
match_date=nm.date,
match_date_date=_to_date(nm.date),
match_status=nm.match_status,
home_goals=nm.home_goals,
away_goals=nm.away_goals,
home_ht_goals=nm.home_ht_goals,
away_ht_goals=nm.away_ht_goals,
match_stage=nm.match_stage,
source_event_id=_to_int_or_none(raw.get("id")),
)
db.add(m)
await db.flush()
existing_matches[match_key] = m # 防止同批重复
# 统计字段不在 /events/ 载荷中(单独由 stats 管线回填),
# 此处不再创建 MatchStats。
league_r["inserted"] += 1
# D1: 成功插入 → 补写 Bronze 层(原始载荷 + 血缘)
if record_id not in bronze_written:
bronze_written.add(record_id)
await _write_events_bronze(
db,
source_record_id=record_id,
raw_payload=raw,
target_match_id=m.id,
league_code=code,
match_status=nm.match_status,
batch_id=bronze_batch_id,
)
else:
# 已有比赛: 直接从内存获取对象更新(无需再查询)
changed = False
if existing_match.match_status != nm.match_status and nm.match_status == "finished":
existing_match.match_status = nm.match_status
changed = True
if existing_match.home_goals is None and nm.home_goals is not None:
existing_match.home_goals = nm.home_goals
existing_match.away_goals = nm.away_goals
existing_match.home_ht_goals = nm.home_ht_goals
existing_match.away_ht_goals = nm.away_ht_goals
changed = True
if existing_match.match_stage is None and nm.match_stage:
existing_match.match_stage = nm.match_stage
changed = True
if existing_match.source_event_id is None:
eid = _to_int_or_none(raw.get("id"))
if eid is not None:
existing_match.source_event_id = eid
changed = True
if changed:
league_r["updated"] += 1
# D1: 变更更新 → 补写血缘(RawEvent 幂等键不变,重复采集自动跳过)
if record_id not in bronze_written:
bronze_written.add(record_id)
await _write_events_bronze(
db,
source_record_id=record_id,
raw_payload=raw,
target_match_id=existing_match.id,
league_code=code,
match_status=nm.match_status,
batch_id=bronze_batch_id,
)
# 注意: 不在此处 commit,由调用方 UnitOfWork 控制事务
result["leagues"][code] = league_r
result["total_inserted"] += league_r["inserted"]
result["total_updated"] += league_r["updated"]
return result
# ============================================================
# 管线基础设施:RawEvent / IngestFailure / DataLineage
# ============================================================
async def _write_raw_event(db, source_system: str, source_record_id: str, raw_payload: dict, batch_id: str | None = None) -> None:
"""写入 Bronze 层原始事件(幂等:同 source_record_id 跳过)。"""
from sqlalchemy import select as _select
stmt = _select(RawEvent).where(
RawEvent.source_system == source_system,
RawEvent.source_record_id == source_record_id,
)
existing = (await db.execute(stmt)).scalar_one_or_none()
if existing is None:
db.add(RawEvent(
source_system=source_system,
source_record_id=source_record_id,
raw_payload=raw_payload,
ingest_batch_id=batch_id,
))
async def _write_ingest_failure(db, source_system: str, entity_type: str, source_record_id: str | None, error_type: str, error_detail: str | None, raw_payload: dict | None = None) -> None:
"""写入采集失败死信。"""
db.add(IngestFailure(
source_system=source_system,
entity_type=entity_type,
source_record_id=source_record_id,
error_type=error_type,
error_detail=error_detail,
raw_payload=raw_payload,
))
async def _safe_write_ingest_failure(
db,
*,
entity_type: str,
source_record_id: str | None,
error: Exception,
raw_payload: dict | None = None,
) -> None:
"""抓取失败时尽力写入死信表(失败不影响主流程)。
死信是可观测性基础设施, RawEvent/Lineage 同级:写入失败只记
warning,绝不能让原始抓取错误之外的新异常打断采集循环
"""
try:
await _write_ingest_failure(
db, "bzzoiro", entity_type, source_record_id,
"fetch_error", str(error), raw_payload,
)
except Exception:
logger.warning(
"写入 ingest_failures 死信失败(entity=%s, record=%s): %s",
entity_type, source_record_id, error, exc_info=True,
)
async def _write_lineage(db, source_system: str, source_record_id: str, target_table: str, target_id: int | None, transform_name: str, transform_detail: dict | None = None, batch_id: str | None = None) -> None:
"""写入 ETL 血缘追踪。"""
db.add(DataLineage(
source_system=source_system,
source_record_id=source_record_id,
target_table=target_table,
target_id=target_id,
transform_name=transform_name,
transform_detail=transform_detail,
batch_id=batch_id,
))
def _events_record_id(league_code: str, nm, raw: dict) -> str:
"""events 载荷的 RawEvent 幂等键。
优先用上游 event id;缺失时用 (league:home:away:date) 合成稳定键
normalize 后的队名与天级日期( _match_key 同口径),不依赖 DB 自增 id,
保证同一来源比赛重复采集时命中同一条 RawEvent,不产生重复原始载荷
"""
eid = _to_int_or_none(raw.get("id"))
if eid is not None:
return str(eid)
d = _to_date(nm.date)
date_part = d.isoformat() if d is not None else "na"
return f"{league_code}:{nm.home_team}:{nm.away_team}:{date_part}"
async def _write_events_bronze(
db,
*,
source_record_id: str,
raw_payload: dict,
target_match_id: int | None,
league_code: str,
match_status: str | None,
batch_id: str,
) -> None:
"""events 成功插入/更新单场比赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
D1(工程债):此前只有 stats 回填写 RawEvent/Lineage,events 管线作为比赛
主数据的唯一入口反而不留溯源记录幂等性由 _write_raw_event
source_record_id 查重保证;best-effort:基础设施写入失败只记 warning,
绝不拖垮采集主流程( _safe_write_ingest_failure 同级约束)
"""
try:
await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
await _write_lineage(
db, "bzzoiro", source_record_id,
"matches", target_match_id, "events_ingest",
{"league": league_code, "match_status": match_status},
batch_id,
)
except Exception:
logger.warning(
"events Bronze 写入失败(record=%s, match=%s),不影响采集主流程",
source_record_id, target_match_id, exc_info=True,
)
# ============================================================
# 积分榜管线:/leagues/{id}/standings/ → standings 表
# ============================================================
async def fetch_bzzoiro_standings(league_code: str, season: str | None = None) -> dict:
"""抓取联赛积分榜(纯抓取,不入库)。season 为 None 时取当前赛季。"""
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
if league_id is None:
raise ValueError(f"未知联赛代码: {league_code}")
params: dict = {}
if season:
params["season"] = season
return await _fetch_json_async(f"/leagues/{league_id}/standings/", params)
def _season_label_from_dates(start_date, end_date) -> str:
"""从赛季起止日期推导赛季标签(与 derive_season_label 语义一致)。"""
try:
if isinstance(start_date, str):
start = datetime.fromisoformat(start_date[:10])
else:
start = start_date
y = start.year
return f"{y}-{y + 1}" if start.month >= 8 else f"{y - 1}-{y}"
except (TypeError, ValueError):
return "?"
async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str | None = None) -> dict:
"""采集积分榜 → upsert standings 表。
season None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)
球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配
"""
from src.data.team_names import normalize as normalize_name
result: dict = {"leagues": {}, "total_upserted": 0, "errors": []}
for code in leagues:
league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []}
try:
payload = await fetch_bzzoiro_standings(code, season=season)
except Exception as e:
logger.exception("bzzoiro standings fetch failed for %s", code)
league_r["errors"].append(str(e))
await _safe_write_ingest_failure(
db,
entity_type="standings",
source_record_id=None,
error=e,
raw_payload={"league": code, "season": season},
)
result["leagues"][code] = league_r
result["errors"].append(f"{code}: {e}")
continue
rows = payload.get("standings") or []
if not rows:
result["leagues"][code] = {"error": "无积分榜数据(赛季未开始或未提供)"}
result["errors"].append(f"{code}: 无积分榜数据")
continue
# 联赛(get-or-create,D4: 经 LeagueRepository)
league = await LeagueRepository(db).get_or_create(
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
)
team_r = TeamRepository(db)
# 赛季标签:优先用返回的 season 对象推导
season_obj = payload.get("season") or {}
season_label = _season_label_from_dates(
season_obj.get("start_date"), season_obj.get("end_date")
)
if season_label == "?":
season_label = season or ""
# 批量预载球队(与 events 管线使用同一 normalize 规则,保证 Team 匹配)
names = {normalize_name(str(r.get("team_name", ""))) for r in rows}
names.discard("")
team_map: dict[str, Team] = await team_r.get_all_by_names(list(names))
now = datetime.now(timezone.utc)
for r in rows:
team_name = normalize_name(str(r.get("team_name", "")))
if not team_name:
continue
team = team_map.get(team_name)
if team is None:
team = await team_r.get_or_create(team_name, name_zh=zh_name(team_name))
team_map[team_name] = team
league_r["teams_created"] += 1
zone = r.get("zone") or {}
values = dict(
position=_to_int_or_none(r.get("position")) or 0,
played=_to_int_or_none(r.get("played")) or 0,
won=_to_int_or_none(r.get("won")) or 0,
drawn=_to_int_or_none(r.get("drawn")) or 0,
lost=_to_int_or_none(r.get("lost")) or 0,
goals_for=_to_int_or_none(r.get("gf")) or 0,
goals_against=_to_int_or_none(r.get("ga")) or 0,
goal_diff=_to_int_or_none(r.get("gd")) or 0,
points=_to_int_or_none(r.get("pts")) or 0,
xg_for=_to_float_or_none(r.get("xgf")),
xg_against=_to_float_or_none(r.get("xga")),
form=r.get("form") or None,
zone=zone.get("label") or zone.get("key") or None,
updated_at=now,
retrieved_at=now,
)
# 同一联赛同一赛季只保留最新快照:按 (league, season, team) upsert
stmt = select(Standing).where(
Standing.league_id == league.id,
Standing.season == season_label,
Standing.team_id == team.id,
)
standing = (await db.execute(stmt)).scalar_one_or_none()
if standing is None:
standing = Standing(
league_id=league.id, season=season_label, team_id=team.id, **values
)
db.add(standing)
else:
for k, v in values.items():
setattr(standing, k, v)
league_r["upserted"] += 1
league_r["rows"] = len(rows)
result["leagues"][code] = league_r
result["total_upserted"] += league_r["upserted"]
logger.info(
"bzzoiro standings 采集完成: %s 赛季 %s, upsert %d/%d",
code, season_label, league_r["upserted"], league_r["rows"],
)
return result
# ============================================================
# 统计回填管线:/events/{id}/stats/ → match_stats 表
# ============================================================
# bzzoiro stats 字段 → MatchStats 字段映射(stats.home / stats.away 下)
_STATS_FIELD_MAP = {
"xg": ("home_xg", "away_xg"), # 回退 expected_goals
"ball_possession": ("home_possession", None), # 只取主队值,客队=100-home
"total_shots": ("home_shots", "away_shots"),
"shots_on_target": ("home_shots_on_target", "away_shots_on_target"),
"corner_kicks": ("home_corners", "away_corners"),
"yellow_cards": ("home_yellow_cards", "away_yellow_cards"),
"red_cards": ("home_red_cards", "away_red_cards"),
"big_chances": ("home_big_chances", "away_big_chances"),
"fouls": ("home_fouls", "away_fouls"),
}
def _pick(d: dict, *keys):
"""按优先级取第一个非空字段值。"""
for k in keys:
v = d.get(k)
if v is not None:
return v
return None
def _stats_from_payload(payload: dict) -> dict:
"""把 /events/{id}/stats/ 响应映射成 MatchStats 字段 dict。
响应结构: {"event_id": ..., "stats": {"home": {...}, "away": {...}}}
"""
stats = (payload or {}).get("stats") or {}
home = stats.get("home") or {}
away = stats.get("away") or {}
out: dict = {}
xg_h = _pick(home, "xg", "expected_goals")
xg_a = _pick(away, "xg", "expected_goals")
if xg_h is not None:
out["home_xg"] = _to_float_or_none(xg_h)
if xg_a is not None:
out["away_xg"] = _to_float_or_none(xg_a)
poss = home.get("ball_possession")
if poss is not None:
p = _to_float_or_none(poss)
if p is not None:
out["home_possession"] = p
for src, (h_fld, a_fld) in _STATS_FIELD_MAP.items():
if src in ("xg", "ball_possession"):
continue # 已处理
hv = home.get(src)
av = away.get(src)
if hv is not None and h_fld:
out[h_fld] = _to_int_or_none(hv)
if av is not None and a_fld:
out[a_fld] = _to_int_or_none(av)
return out
def _to_float_or_none(value) -> float | None:
if value is None:
return None
try:
return float(str(value).strip())
except (TypeError, ValueError):
return None
async def ingest_bzzoiro_event_stats(
db,
*,
leagues: Iterable[str],
limit: int = 100,
only_missing: bool = True,
) -> dict:
"""回填已完赛比赛的详细统计(逐场调 /events/{id}/stats/)。
筛选条件: match_status=finished source_event_id 非空
only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新
limit 控制单次最多处理的比赛数(上游限速 1.2s/请求,大批量需分次触发)
"""
result: dict = {"fetched": 0, "created": 0, "updated": 0, "skipped": 0, "errors": []}
league_ids = [BZZOIRO_LEAGUE_IDS[c] for c in leagues if c in BZZOIRO_LEAGUE_IDS]
if not league_ids:
result["errors"].append("无有效联赛代码")
return result
# D4: 候选比赛查询经 MatchRepository(含 stats 预加载,筛选/排序/limit 语义不变)
matches = await MatchRepository(db).find_finished_with_stats(
league_ids, limit=limit * 3 if only_missing else limit
)
now = datetime.now(timezone.utc)
processed = 0
for m in matches:
if processed >= limit:
break
if only_missing and m.stats is not None and m.stats.home_shots is not None:
result["skipped"] += 1
continue
processed += 1
try:
payload = await _fetch_json_async(f"/events/{m.source_event_id}/stats/")
except Exception as e:
logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e)
result["errors"].append(f"match {m.id}: {e}")
await _safe_write_ingest_failure(
db,
entity_type="match_stats",
source_record_id=str(m.source_event_id),
error=e,
raw_payload={"match_id": m.id},
)
await asyncio.sleep(REQUEST_INTERVAL)
continue
result["fetched"] += 1
fields = _stats_from_payload(payload)
if not fields:
result["skipped"] += 1
await asyncio.sleep(REQUEST_INTERVAL)
continue
if m.stats is None:
available_at = m.match_date + timedelta(hours=2) if m.match_date else now
m.stats = MatchStats(
match_id=m.id,
source="bzzoiro",
source_record_id=str(m.source_event_id),
retrieved_at=now,
available_at=available_at,
)
db.add(m.stats)
result["created"] += 1
else:
result["updated"] += 1
if m.stats.source is None:
m.stats.source = "bzzoiro"
m.stats.source_record_id = str(m.source_event_id)
if m.stats.retrieved_at is None:
m.stats.retrieved_at = now
if m.stats.available_at is None and m.match_date:
m.stats.available_at = m.match_date + timedelta(hours=2)
for fld, v in fields.items():
if hasattr(m.stats, fld):
setattr(m.stats, fld, v)
# 管线基础设施:写入 RawEvent + DataLineage
batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}"
try:
await _write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id)
await _write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id)
except Exception:
pass # 基础设施写入失败不影响主流程
await asyncio.sleep(REQUEST_INTERVAL)
logger.info(
"bzzoiro stats 回填完成: 抓取 %d, 新建 %d, 更新 %d, 跳过 %d, 错误 %d",
result["fetched"], result["created"], result["updated"],
result["skipped"], len(result["errors"]),
)
return result
# ── 配置常量(原文件即从 config 再导出,维持 bz.REQUEST_INTERVAL 等引用) ──
from src.data.config import ( # noqa: F401
BZZOIRO_LEAGUE_IDS,
LEAGUE_COUNTRIES,
LEAGUE_NAMES,
REQUEST_INTERVAL,
)
from src.data.key_ring import _mask # noqa: F401 (R1 测试引用 bz._mask)
from src.data.normalize import normalize_bzzoiro # noqa: F401
# ── 共享原语:HTTP 抓取 + 宽松字段转换 ──
# NOTE: 管线模块同时从 bzzoiro_common 直接 import _fetch_json_async 等(经本处也转发)。
from src.data.bzzoiro_common import ( # noqa: F401
_fetch_json_async,
_match_key,
_to_date,
_to_float_or_none,
_to_int_or_none,
)
# ── events 管线:BzzoiroSource(注册表入口)+ 抓取/入库 ──
from src.data.bzzoiro_events import ( # noqa: F401
BzzoiroSource,
_events_record_id,
_write_events_bronze,
fetch_bzzoiro_events,
)
# ── standings 管线 ──
from src.data.bzzoiro_standings import ( # noqa: F401
_season_label_from_dates,
_write_standings_bronze,
fetch_bzzoiro_standings,
ingest_bzzoiro_standings,
)
# ── stats 回填管线 ──
from src.data.bzzoiro_stats import ( # noqa: F401
_pick,
_stats_from_payload,
ingest_bzzoiro_event_stats,
)
+123
View File
@@ -0,0 +1,123 @@
"""bzzoiro 管线共享原语:HTTP 抓取(多 key 轮换)与宽松字段转换。
bzzoiro.py 拆出(单文件 多模块):仅放无业务语义的共享基础,
三条管线(events/standings/stats)与聚合门面见 bzzoiro.py
"""
from __future__ import annotations
import asyncio
import logging
import random
import httpx
from src.core.http_client import get_client
from src.core.runtime_config import get_runtime_value
from src.data.key_ring import _mask, get_key_ring
logger = logging.getLogger(__name__)
def _to_date(value):
"""把 datetime / date / str 统一成 `date`。"""
if value is None:
return None
if hasattr(value, "date") and callable(value.date):
return value.date()
return value
def _to_int_or_none(value) -> int | None:
"""宽松转 int(用于上游 ID 解析,失败返回 None 不抛错)。"""
if value is None:
return None
try:
return int(str(value).strip())
except (TypeError, ValueError):
return None
def _to_float_or_none(value) -> float | None:
try:
return float(str(value).strip())
except (TypeError, ValueError):
return None
def _match_key(home_team_id: int, away_team_id: int, match_date) -> tuple[int, int, str]:
"""比赛去重键:(主队, 客队, 天级日期 ISO 字符串)。
统一在这里构造,避免"预加载时用 str(date)、写入时用 isoformat()"这类
隐式格式依赖 两者当前恰好相等,但一旦有人改动其一就会静默失配,
导致所有比赛被判为不存在而重复插入
"""
d = _to_date(match_date)
return (home_team_id, away_team_id, d.isoformat() if d is not None else "")
async def _fetch_json_async(path: str, params: dict | None = None, max_retries: int = 3) -> dict | list:
"""异步 HTTP(bzzoiro 使用 httpx,不再阻塞事件循环线程池)。
key 轮换:遇到 429 自动切换到下一个 key;全部 key 冷却时等待最早恢复
"""
base = (await get_runtime_value("BZZOIRO_BASE")).rstrip("/")
raw_keys = await get_runtime_value("BZZOIRO_KEY")
ring = get_key_ring(base, raw_keys)
url = f"{base}/{path.lstrip('/')}"
key = ring.get()
if not key:
raise RuntimeError("BZZOIRO_KEY 未设置")
last_exc: Exception | None = None
for attempt in range(max_retries):
headers = {
"Authorization": f"Token {key}",
"Accept": "application/json",
}
try:
client = get_client()
# 整请求兜底: httpx 无 total 超时,用 wait_for 防「滴水式」限速挂死
resp = await asyncio.wait_for(
client.get(
url, headers=headers, params=params,
timeout=httpx.Timeout(connect=10.0, read=30.0, write=10.0, pool=10.0),
),
timeout=60.0,
)
resp.raise_for_status()
return resp.json()
except Exception as e:
last_exc = e
status = getattr(getattr(e, "response", None), "status_code", None)
if status == 429:
# 限流:标记当前 key 冷却,切换到下一个
new_key = ring.report_rate_limited(key)
if new_key and new_key != key:
logger.info("bzzoiro 429 → 切换 key: %s%s,立即重试", _mask(key), _mask(new_key))
key = new_key
continue # 立即重试,不等待
# 单 key 或全部冷却:等待最早恢复的 key
wait = ring.wait_if_all_blocked()
if wait > 0:
logger.warning("bzzoiro 全部 key 冷却,等待 %.1fs 后重试", wait)
await asyncio.sleep(min(wait, 30.0))
else:
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro 429, retry %d in %.1fs", attempt + 1, delay)
await asyncio.sleep(delay)
key = ring.get() or key
continue
if 500 <= (status or 0) < 600:
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro %d, retry %d in %.1fs", status, attempt + 1, delay)
await asyncio.sleep(delay)
continue
# 网络错误(连接失败/超时)也退避重试
if isinstance(e, (TimeoutError, ConnectionError, OSError)):
delay = min(2 ** attempt, 16) + random.uniform(0, 1)
logger.warning("bzzoiro network error, retry %d in %.1fs: %s", attempt + 1, delay, e)
await asyncio.sleep(delay)
continue
raise
raise RuntimeError(f"bzzoiro request failed after {max_retries} attempts: {last_exc}")
+327
View File
@@ -0,0 +1,327 @@
"""bzzoiro events 管线:比赛日程/比分抓取(/events/)与入库(matches 表)。
bzzoiro.py 拆出比赛主数据唯一入口;Team/League/Match 查找/创建经
Repository ,事务由调用方 UnitOfWork 控制(分批事务约定不变)
可替换协作者(抓取函数 / Bronze 写入助手 / REQUEST_INTERVAL)在运行期
经聚合门面 src.data.bzzoiro 解析 与拆分前的单文件 monkeypatch 语义一致
"""
from __future__ import annotations
import asyncio
import logging
from collections.abc import Iterable
from datetime import datetime, timedelta, timezone
from src.data.bzzoiro_common import _match_key, _to_date, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
from src.data.normalize import normalize_bzzoiro
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.data.sources import register
from src.data.team_names_zh import zh_name
from src.db.models import Match
from src.db.repositories import LeagueRepository, MatchRepository, TeamRepository
logger = logging.getLogger(__name__)
async def fetch_bzzoiro_events(
league_code: str,
*,
status: str = "finished",
date_from: str | None = None,
date_to: str | None = None,
limit: int = 200,
) -> list[dict]:
"""抓取 bzzoiro 原始事件(纯异步,无需 run_in_executor)。"""
from src.data import bzzoiro as bz
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
if league_id is None:
raise ValueError(f"未知联赛代码: {league_code}")
rows: list[dict] = []
offset = 0
while True:
params: dict = {
"league_id": league_id,
"status": status,
"limit": limit,
"offset": offset,
}
if date_from:
params["date_from"] = str(date_from)[:10]
if date_to:
params["date_to"] = str(date_to)[:10]
payload = await bz._fetch_json_async("/events/", params)
batch = payload.get("results") or []
if not batch:
break
rows.extend(batch)
total = payload.get("total")
offset += limit
if total is not None and offset >= total:
break
if len(batch) < limit:
break
await asyncio.sleep(bz.REQUEST_INTERVAL)
return rows
@register
class BzzoiroSource:
"""bzzoiro 数据源(实现 DataSource 协议)。"""
name = "bzzoiro"
async def ingest(
self,
db,
*,
leagues: Iterable[str],
date_from: str | None = None,
date_to: str | None = None,
status: str = "finished",
) -> dict:
"""采集 bzzoiro → 入库。返回统计。
注意: 本方法不控制事务(commit/rollback),由调用方通过 UnitOfWork 控制
"""
from src.data import bzzoiro as bz
result: dict = {"leagues": {}, "total_inserted": 0, "total_updated": 0, "errors": []}
for code in leagues:
league_r: dict = {"inserted": 0, "updated": 0, "errors": []}
try:
raw_events = await bz.fetch_bzzoiro_events(code, status=status, date_from=date_from, date_to=date_to)
except Exception as e:
# 单联赛抓取失败隔离:记录错误后继续其余联赛,不拖垮整批
logger.exception("bzzoiro fetch failed for %s", code)
league_r["errors"].append(f"fetch failed: {e}")
await _safe_write_ingest_failure(
db,
entity_type="events",
source_record_id=None,
error=e,
raw_payload={"league": code, "status": status, "date_from": date_from, "date_to": date_to},
)
result["leagues"][code] = league_r
continue
# D4: 联赛查找/创建经 LeagueRepository(事务仍由调用方 UoW 提交)
league = await LeagueRepository(db).get_or_create(
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
)
team_r = TeamRepository(db)
match_r = MatchRepository(db)
# === 批量优化: 预加载球队和已有比赛到内存 ===
team_name_to_id: dict[str, int] = {}
existing_matches: dict[tuple[int, int, str], Match] = {} # 完整对象,避免重复查询
# (NormalizedMatch, 原始 event) 成对保存:后续写 source_event_id 时
# 必须用配对的那条 event,不能依赖外层循环变量残留值。
normalized_matches: list[tuple] = []
if raw_events:
# 一次遍历: 收集球队名 + 规范化
all_team_names = set()
for raw in raw_events:
nm = normalize_bzzoiro(raw, code)
if nm is not None:
try:
nm.validate()
except Exception:
continue
normalized_matches.append((nm, raw))
all_team_names.add(nm.home_team)
all_team_names.add(nm.away_team)
if all_team_names:
team_name_to_id = {
name: t.id
for name, t in (await team_r.get_all_by_names(list(all_team_names))).items()
}
# P1-2: 按需加载,只加载 raw_events 涉及日期范围的比赛(加 30 天缓冲)
# 避免加载联赛全部历史比赛到内存(多赛季采集时内存溢出)
if normalized_matches:
# normalized_matches 存的是 (nm, raw) 元组,遍历需解包
dates = [nm.date for nm, _raw in normalized_matches if nm.date is not None]
if dates:
min_dt = min(dates) - timedelta(days=30)
max_dt = max(dates) + timedelta(days=30)
matches_in_range = await match_r.find_by_league_and_date_range(
league.id, min_dt, max_dt
)
existing_matches = {
_match_key(m.home_team_id, m.away_team_id, m.match_date_date): m
for m in matches_in_range
}
# else: existing_matches 保持空 dict(全量新比赛)
# D1: Bronze 层批次信息(每联赛每批次一个 batch_id;seen 防同批重复写入)
now = datetime.now(timezone.utc)
bronze_batch_id = f"bzzoiro-events-{code}-{now:%Y%m%d%H%M%S}"
bronze_written: set[str] = set()
for nm, raw in normalized_matches:
# D1: RawEvent 幂等键(上游 id 或合成键),插入/变更更新共用
record_id = _events_record_id(code, nm, raw)
# 球队: 内存查找 + 按需创建(D4: 经 TeamRepository)
home_team_id = team_name_to_id.get(nm.home_team)
if home_team_id is None:
home = await team_r.get_or_create(nm.home_team, name_zh=zh_name(nm.home_team))
home_team_id = home.id
team_name_to_id[nm.home_team] = home_team_id
away_team_id = team_name_to_id.get(nm.away_team)
if away_team_id is None:
away = await team_r.get_or_create(nm.away_team, name_zh=zh_name(nm.away_team))
away_team_id = away.id
team_name_to_id[nm.away_team] = away_team_id
# 查找已有比赛:优先按 upstream event_id 定位(命中即唯一),
# 否则回退自然键(联赛+主客+天级日期)内存查找。
# source_event_id 上有 partial unique 索引保障 upstream 唯一。
eid = _to_int_or_none(raw.get("id"))
existing_match = None
if eid is not None:
existing_match = await match_r.find_by_source_event_id(eid)
if existing_match is None:
match_key = _match_key(home_team_id, away_team_id, nm.date)
existing_match = existing_matches.get(match_key)
if existing_match is None:
m = Match(
league_id=league.id,
season=nm.season_label or None,
home_team_id=home_team_id,
away_team_id=away_team_id,
match_date=nm.date,
match_date_date=_to_date(nm.date),
match_status=nm.match_status,
score_status=nm.score_status,
home_goals=nm.home_goals,
away_goals=nm.away_goals,
home_ht_goals=nm.home_ht_goals,
away_ht_goals=nm.away_ht_goals,
match_stage=nm.match_stage,
source_event_id=_to_int_or_none(raw.get("id")),
)
db.add(m)
await db.flush()
existing_matches[match_key] = m # 防止同批重复
# 统计字段不在 /events/ 载荷中(单独由 stats 管线回填),
# 此处不再创建 MatchStats。
league_r["inserted"] += 1
# D1: 成功插入 → 补写 Bronze 层(原始载荷 + 血缘)
if record_id not in bronze_written:
bronze_written.add(record_id)
await _write_events_bronze(
db,
source_record_id=record_id,
raw_payload=raw,
target_match_id=m.id,
league_code=code,
match_status=nm.match_status,
batch_id=bronze_batch_id,
)
else:
# 已有比赛: 直接从内存获取对象更新(无需再查询)
changed = False
if existing_match.match_status != nm.match_status and nm.match_status == "finished":
existing_match.match_status = nm.match_status
changed = True
if existing_match.home_goals is None and nm.home_goals is not None:
existing_match.home_goals = nm.home_goals
existing_match.away_goals = nm.away_goals
existing_match.home_ht_goals = nm.home_ht_goals
existing_match.away_ht_goals = nm.away_ht_goals
# 比分由缺变有 → 标记 known
existing_match.score_status = "known"
changed = True
elif (
nm.match_status == "finished"
and nm.home_goals is None
and existing_match.score_status == "unknown"
):
# 确认完赛仍缺分 → 标记 missing(不伪造 0:0)
existing_match.score_status = "missing"
changed = True
if existing_match.match_stage is None and nm.match_stage:
existing_match.match_stage = nm.match_stage
changed = True
if existing_match.source_event_id is None:
eid = _to_int_or_none(raw.get("id"))
if eid is not None:
existing_match.source_event_id = eid
changed = True
if changed:
league_r["updated"] += 1
# D1: 变更更新 → 补写血缘(RawEvent 幂等键不变,重复采集自动跳过)
if record_id not in bronze_written:
bronze_written.add(record_id)
await _write_events_bronze(
db,
source_record_id=record_id,
raw_payload=raw,
target_match_id=existing_match.id,
league_code=code,
match_status=nm.match_status,
batch_id=bronze_batch_id,
)
# 注意: 不在此处 commit,由调用方 UnitOfWork 控制事务
result["leagues"][code] = league_r
result["total_inserted"] += league_r["inserted"]
result["total_updated"] += league_r["updated"]
return result
def _events_record_id(league_code: str, nm, raw: dict) -> str:
"""events 载荷的 RawEvent 幂等键。
优先用上游 event id;缺失时用 (league:home:away:date) 合成稳定键
normalize 后的队名与天级日期( _match_key 同口径),不依赖 DB 自增 id,
保证同一来源比赛重复采集时命中同一条 RawEvent,不产生重复原始载荷
"""
eid = _to_int_or_none(raw.get("id"))
if eid is not None:
return str(eid)
d = _to_date(nm.date)
date_part = d.isoformat() if d is not None else "na"
return f"{league_code}:{nm.home_team}:{nm.away_team}:{date_part}"
async def _write_events_bronze(
db,
*,
source_record_id: str,
raw_payload: dict,
target_match_id: int | None,
league_code: str,
match_status: str | None,
batch_id: str,
) -> None:
"""events 成功插入/更新单场比赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
D1(工程债):此前只有 stats 回填写 RawEvent/Lineage,events 管线作为比赛
主数据的唯一入口反而不留溯源记录幂等性由 _write_raw_event
source_record_id 查重保证;best-effort:基础设施写入失败只记 warning,
绝不拖垮采集主流程( _safe_write_ingest_failure 同级约束)
"""
try:
await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
await _write_lineage(
db, "bzzoiro", source_record_id,
"matches", target_match_id, "events_ingest",
{"league": league_code, "match_status": match_status},
batch_id,
)
except Exception:
logger.warning(
"events Bronze 写入失败(record=%s, match=%s),不影响采集主流程",
source_record_id, target_match_id, exc_info=True,
)
+207
View File
@@ -0,0 +1,207 @@
"""bzzoiro standings 管线:联赛积分榜快照(/leagues/{id}/standings/)→ standings 表。
bzzoiro.py 拆出同一联赛同一赛季只保留最新快照( (league, season, team)
upsert);球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配
可替换协作者(抓取函数 / Bronze 写入助手)在运行期经聚合门面
src.data.bzzoiro 解析 与拆分前的单文件 monkeypatch 语义一致
"""
from __future__ import annotations
import logging
from collections.abc import Iterable
from datetime import datetime, timezone
from sqlalchemy import select
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS, LEAGUE_COUNTRIES, LEAGUE_NAMES
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.data.team_names_zh import zh_name
from src.db.models import Standing, Team
from src.db.repositories import LeagueRepository, TeamRepository
logger = logging.getLogger(__name__)
async def fetch_bzzoiro_standings(league_code: str, season: str | None = None) -> dict:
"""抓取联赛积分榜(纯抓取,不入库)。season 为 None 时取当前赛季。"""
from src.data import bzzoiro as bz
league_id = BZZOIRO_LEAGUE_IDS.get(league_code)
if league_id is None:
raise ValueError(f"未知联赛代码: {league_code}")
params: dict = {}
if season:
params["season"] = season
return await bz._fetch_json_async(f"/leagues/{league_id}/standings/", params)
def _season_label_from_dates(start_date, end_date) -> str:
"""从赛季起止日期推导赛季标签(与 derive_season_label 语义一致)。"""
try:
if isinstance(start_date, str):
start = datetime.fromisoformat(start_date[:10])
else:
start = start_date
if start is None:
return "?"
y = start.year
return f"{y}-{y + 1}" if start.month >= 8 else f"{y - 1}-{y}"
except (TypeError, ValueError):
return "?"
async def ingest_bzzoiro_standings(db, *, leagues: Iterable[str], season: str | None = None) -> dict:
"""采集积分榜 → upsert standings 表。
season None 时采集当前赛季(bzzoiro 默认返回 is_current 赛季)
球队名与 events 管线使用同一 normalize 规则,保证 Team 匹配
"""
from src.data.team_names import normalize as normalize_name
from src.data import bzzoiro as bz
result: dict = {"leagues": {}, "total_upserted": 0, "errors": []}
for code in leagues:
league_r: dict = {"upserted": 0, "teams_created": 0, "rows": 0, "errors": []}
try:
payload = await bz.fetch_bzzoiro_standings(code, season=season)
except Exception as e:
logger.exception("bzzoiro standings fetch failed for %s", code)
league_r["errors"].append(str(e))
await _safe_write_ingest_failure(
db,
entity_type="standings",
source_record_id=None,
error=e,
raw_payload={"league": code, "season": season},
)
result["leagues"][code] = league_r
result["errors"].append(f"{code}: {e}")
continue
rows = payload.get("standings") or []
if not rows:
result["leagues"][code] = {"error": "无积分榜数据(赛季未开始或未提供)"}
result["errors"].append(f"{code}: 无积分榜数据")
continue
# 联赛(get-or-create,D4: 经 LeagueRepository)
league = await LeagueRepository(db).get_or_create(
code, LEAGUE_NAMES.get(code, code), LEAGUE_COUNTRIES.get(code)
)
team_r = TeamRepository(db)
# 赛季标签:优先用返回的 season 对象推导
season_obj = payload.get("season") or {}
season_label = _season_label_from_dates(
season_obj.get("start_date"), season_obj.get("end_date")
)
if season_label == "?":
season_label = season or ""
# 批量预载球队(与 events 管线使用同一 normalize 规则,保证 Team 匹配)
names = {normalize_name(str(r.get("team_name", ""))) for r in rows}
names.discard("")
team_map: dict[str, Team] = await team_r.get_all_by_names(list(names))
now = datetime.now(timezone.utc)
for r in rows:
team_name = normalize_name(str(r.get("team_name", "")))
if not team_name:
continue
team = team_map.get(team_name)
if team is None:
team = await team_r.get_or_create(team_name, name_zh=zh_name(team_name))
team_map[team_name] = team
league_r["teams_created"] += 1
zone = r.get("zone") or {}
values = dict(
position=_to_int_or_none(r.get("position")) or 0,
played=_to_int_or_none(r.get("played")) or 0,
won=_to_int_or_none(r.get("won")) or 0,
drawn=_to_int_or_none(r.get("drawn")) or 0,
lost=_to_int_or_none(r.get("lost")) or 0,
goals_for=_to_int_or_none(r.get("gf")) or 0,
goals_against=_to_int_or_none(r.get("ga")) or 0,
goal_diff=_to_int_or_none(r.get("gd")) or 0,
points=_to_int_or_none(r.get("pts")) or 0,
xg_for=_to_float_or_none(r.get("xgf")),
xg_against=_to_float_or_none(r.get("xga")),
form=r.get("form") or None,
zone=zone.get("label") or zone.get("key") or None,
updated_at=now,
retrieved_at=now,
)
# P0-02: 追加快照——每次采集 INSERT 新行(available_at=now),
# ON CONFLICT (league, season, team, available_at) DO NOTHING。
standing = Standing(
league_id=league.id, season=season_label, team_id=team.id, available_at=now, **values
)
db.add(standing)
league_r["upserted"] += 1
league_r["rows"] = len(rows)
# D1(对称 events/stats 管线): 联赛成功 upsert → 补写 Bronze 层。
# 幂等键 standings:{league}:{season}:积分榜是联赛级快照,一次成功
# 采集写一条 RawEvent(整份原始载荷)+ 一条血缘。season 用实际入库的
# 标签(由载荷推导,与 Standing.season 同口径),不依赖调用方传参,
# 保证不同调用方(season=None 或显式传参)对同一赛季命中同一条 RawEvent。
if league_r["upserted"] > 0:
bronze_batch_id = f"bzzoiro-standings-{code}-{now:%Y%m%d%H%M%S}"
await _write_standings_bronze(
db,
source_record_id=f"standings:{code}:{season_label}",
raw_payload=payload,
league_id=league.id,
league_code=code,
season_label=season_label,
rows_upserted=league_r["upserted"],
batch_id=bronze_batch_id,
)
result["leagues"][code] = league_r
result["total_upserted"] += league_r["upserted"]
logger.info(
"bzzoiro standings 采集完成: %s 赛季 %s, upsert %d/%d",
code, season_label, league_r["upserted"], league_r["rows"],
)
return result
async def _write_standings_bronze(
db,
*,
source_record_id: str,
raw_payload: dict,
league_id: int | None,
league_code: str,
season_label: str,
rows_upserted: int,
batch_id: str,
) -> None:
"""standings 成功 upsert 一个联赛后的 Bronze 层补写:RawEvent(幂等) + DataLineage。
_write_events_bronze 同级约束:幂等性由 _write_raw_event
source_record_id 查重保证(积分榜是联赛级快照,同联赛同赛季重复采集
命中同一条 RawEvent);best-effort:基础设施写入失败只记 warning,
绝不拖垮采集主流程
"""
try:
await _write_raw_event(db, "bzzoiro", source_record_id, raw_payload, batch_id)
await _write_lineage(
db, "bzzoiro", source_record_id,
"standings", league_id, "standings_ingest",
{"league": league_code, "season": season_label, "rows_upserted": rows_upserted},
batch_id,
)
except Exception:
logger.warning(
"standings Bronze 写入失败(record=%s, league=%s),不影响采集主流程",
source_record_id, league_code, exc_info=True,
)
+181
View File
@@ -0,0 +1,181 @@
"""bzzoiro stats 回填管线:已完赛比赛详细统计(/events/{id}/stats/)→ match_stats 表。
bzzoiro.py 拆出上游限速(REQUEST_INTERVAL /请求),大批量回填需分次触发;
add/flush commit,事务由调用方 UnitOfWork 控制
可替换协作者(_fetch_json_async / Bronze 写入助手 / REQUEST_INTERVAL)在运行期
经聚合门面 src.data.bzzoiro 解析 与拆分前的单文件 monkeypatch 语义一致
"""
from __future__ import annotations
import asyncio
import logging
from collections.abc import Iterable
from datetime import datetime, timedelta, timezone
from src.data.bzzoiro_common import _to_float_or_none, _to_int_or_none
from src.data.config import BZZOIRO_LEAGUE_IDS
from src.data.pipeline_write import _safe_write_ingest_failure, _write_lineage, _write_raw_event
from src.db.models import MatchStats
from src.db.repositories import MatchRepository
logger = logging.getLogger(__name__)
# bzzoiro stats 字段 → MatchStats 字段映射(stats.home / stats.away 下)
_STATS_FIELD_MAP = {
"xg": ("home_xg", "away_xg"), # 回退 expected_goals
"ball_possession": ("home_possession", None), # 只取主队值,客队=100-home
"total_shots": ("home_shots", "away_shots"),
"shots_on_target": ("home_shots_on_target", "away_shots_on_target"),
"corner_kicks": ("home_corners", "away_corners"),
"yellow_cards": ("home_yellow_cards", "away_yellow_cards"),
"red_cards": ("home_red_cards", "away_red_cards"),
"big_chances": ("home_big_chances", "away_big_chances"),
"fouls": ("home_fouls", "away_fouls"),
}
def _pick(d: dict, *keys):
"""按优先级取第一个非空字段值。"""
for k in keys:
v = d.get(k)
if v is not None:
return v
return None
def _stats_from_payload(payload: dict) -> dict:
"""把 /events/{id}/stats/ 响应映射成 MatchStats 字段 dict。
响应结构: {"event_id": ..., "stats": {"home": {...}, "away": {...}}}
"""
stats = (payload or {}).get("stats") or {}
home = stats.get("home") or {}
away = stats.get("away") or {}
out: dict = {}
xg_h = _pick(home, "xg", "expected_goals")
xg_a = _pick(away, "xg", "expected_goals")
if xg_h is not None:
out["home_xg"] = _to_float_or_none(xg_h)
if xg_a is not None:
out["away_xg"] = _to_float_or_none(xg_a)
poss = home.get("ball_possession")
if poss is not None:
p = _to_float_or_none(poss)
if p is not None:
out["home_possession"] = p
for src, (h_fld, a_fld) in _STATS_FIELD_MAP.items():
if src in ("xg", "ball_possession"):
continue # 已处理
hv = home.get(src)
av = away.get(src)
if hv is not None and h_fld:
out[h_fld] = _to_int_or_none(hv)
if av is not None and a_fld:
out[a_fld] = _to_int_or_none(av)
return out
async def ingest_bzzoiro_event_stats(
db,
*,
leagues: Iterable[str],
limit: int = 100,
only_missing: bool = True,
) -> dict:
"""回填已完赛比赛的详细统计(逐场调 /events/{id}/stats/)。
筛选条件: match_status=finished source_event_id 非空
only_missing=True 时跳过已有统计的比赛(增量);False 则全量刷新
limit 控制单次最多处理的比赛数(上游限速约 1.2s/请求,大批量需分次触发)
"""
from src.data import bzzoiro as bz
result: dict = {"fetched": 0, "created": 0, "updated": 0, "skipped": 0, "errors": []}
league_ids = [BZZOIRO_LEAGUE_IDS[c] for c in leagues if c in BZZOIRO_LEAGUE_IDS]
if not league_ids:
result["errors"].append("无有效联赛代码")
return result
# D4: 候选比赛查询经 MatchRepository(含 stats 预加载,筛选/排序/limit 语义不变)
matches = await MatchRepository(db).find_finished_with_stats(
league_ids, limit=limit * 3 if only_missing else limit
)
now = datetime.now(timezone.utc)
processed = 0
for m in matches:
if processed >= limit:
break
if only_missing and m.stats is not None and m.stats.home_shots is not None:
result["skipped"] += 1
continue
processed += 1
try:
payload = await bz._fetch_json_async(f"/events/{m.source_event_id}/stats/")
except Exception as e:
logger.warning("stats fetch failed match=%s event=%s: %s", m.id, m.source_event_id, e)
result["errors"].append(f"match {m.id}: {e}")
await _safe_write_ingest_failure(
db,
entity_type="match_stats",
source_record_id=str(m.source_event_id),
error=e,
raw_payload={"match_id": m.id},
)
await asyncio.sleep(bz.REQUEST_INTERVAL)
continue
result["fetched"] += 1
fields = _stats_from_payload(payload)
if not fields:
result["skipped"] += 1
await asyncio.sleep(bz.REQUEST_INTERVAL)
continue
if m.stats is None:
available_at = m.match_date + timedelta(hours=2) if m.match_date else now
m.stats = MatchStats(
match_id=m.id,
source="bzzoiro",
source_record_id=str(m.source_event_id),
retrieved_at=now,
available_at=available_at,
)
db.add(m.stats)
result["created"] += 1
else:
result["updated"] += 1
if m.stats.source is None:
m.stats.source = "bzzoiro"
m.stats.source_record_id = str(m.source_event_id)
if m.stats.retrieved_at is None:
m.stats.retrieved_at = now
if m.stats.available_at is None and m.match_date:
m.stats.available_at = m.match_date + timedelta(hours=2)
for fld, v in fields.items():
if hasattr(m.stats, fld):
setattr(m.stats, fld, v)
# 管线基础设施:写入 RawEvent + DataLineage
batch_id = f"bzzoiro-stats-{m.source_event_id}-{now.strftime('%Y%m%d%H%M%S')}"
try:
await _write_raw_event(db, "bzzoiro", str(m.source_event_id), payload, batch_id)
await _write_lineage(db, "bzzoiro", str(m.source_event_id), "match_stats", m.stats.id if m.stats else None, "stats_backfill", {"match_id": m.id}, batch_id)
except Exception:
pass # 基础设施写入失败不影响主流程
await asyncio.sleep(bz.REQUEST_INTERVAL)
logger.info(
"bzzoiro stats 回填完成: 抓取 %d, 新建 %d, 更新 %d, 跳过 %d, 错误 %d",
result["fetched"], result["created"], result["updated"],
result["skipped"], len(result["errors"]),
)
return result
+9 -1
View File
@@ -35,6 +35,8 @@ class NormalizedMatch:
home_team: str
away_team: str
match_status: str = "finished"
# P0-01:比分可信度。known=可靠比分;missing=完赛缺分;unknown=待定。
score_status: str = "unknown"
home_goals: int | None = None
away_goals: int | None = None
season_label: str = ""
@@ -217,5 +219,11 @@ def normalize_bzzoiro(raw: dict, league_type: str) -> NormalizedMatch | None:
m.away_red_cards = _to_int(raw.get("away_red_cards", raw.get("red_cards_away")))
if m.match_status == "finished" and m.home_goals is None:
m.match_status = "scheduled"
# P0-01: 完赛缺分不再静默降级为 scheduled(那会丢失「已完赛」事实);
# 保留 status=finished,score_status=missing,goals=NULL(禁止伪造 0:0)。
m.score_status = "missing"
elif m.home_goals is not None and m.away_goals is not None:
m.score_status = "known"
else:
m.score_status = "unknown"
return m
+80
View File
@@ -0,0 +1,80 @@
"""管线基础设施写入助手:RawEvent(Bronze 原始载荷)/ IngestFailure(死信)/ DataLineage(血缘)。
bzzoiro.py 拆出约定(与拆分前一致):
- add commit 事务由调用方 UnitOfWork 控制,分批事务约定不变;
- 死信与 Bronze 写入同为 best-effort:失败只记 warning,绝不拖垮采集主流程
"""
from __future__ import annotations
import logging
from src.db.models import DataLineage, IngestFailure, RawEvent
logger = logging.getLogger(__name__)
async def _write_raw_event(db, source_system: str, source_record_id: str, raw_payload: dict, batch_id: str | None = None) -> None:
"""写入 Bronze 层原始事件(幂等:同 source_record_id 跳过)。"""
from sqlalchemy import select as _select
stmt = _select(RawEvent).where(
RawEvent.source_system == source_system,
RawEvent.source_record_id == source_record_id,
)
existing = (await db.execute(stmt)).scalar_one_or_none()
if existing is None:
db.add(RawEvent(
source_system=source_system,
source_record_id=source_record_id,
raw_payload=raw_payload,
ingest_batch_id=batch_id,
))
async def _write_ingest_failure(db, source_system: str, entity_type: str, source_record_id: str | None, error_type: str, error_detail: str | None, raw_payload: dict | None = None) -> None:
"""写入采集失败死信。"""
db.add(IngestFailure(
source_system=source_system,
entity_type=entity_type,
source_record_id=source_record_id,
error_type=error_type,
error_detail=error_detail,
raw_payload=raw_payload,
))
async def _safe_write_ingest_failure(
db,
*,
entity_type: str,
source_record_id: str | None,
error: Exception,
raw_payload: dict | None = None,
) -> None:
"""抓取失败时尽力写入死信表(失败不影响主流程)。
死信是可观测性基础设施, RawEvent/Lineage 同级:写入失败只记
warning,绝不能让原始抓取错误之外的新异常打断采集循环
"""
try:
await _write_ingest_failure(
db, "bzzoiro", entity_type, source_record_id,
"fetch_error", str(error), raw_payload,
)
except Exception:
logger.warning(
"写入 ingest_failures 死信失败(entity=%s, record=%s): %s",
entity_type, source_record_id, error, exc_info=True,
)
async def _write_lineage(db, source_system: str, source_record_id: str, target_table: str, target_id: int | None, transform_name: str, transform_detail: dict | None = None, batch_id: str | None = None) -> None:
"""写入 ETL 血缘追踪。"""
db.add(DataLineage(
source_system=source_system,
source_record_id=source_record_id,
target_table=target_table,
target_id=target_id,
transform_name=transform_name,
transform_detail=transform_detail,
batch_id=batch_id,
))
-19
View File
@@ -68,25 +68,6 @@ async def short_read():
yield session
@asynccontextmanager
async def short_write():
"""短生命周期 write session: 提交后立即释放。
用法:
async with short_write() as session:
session.add(pred)
await session.commit()
# session 已关闭,连接已释放
"""
async with AsyncSessionLocal() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
async def init_db() -> None:
"""验证数据库连接(不建表)。
+76 -12
View File
@@ -19,6 +19,7 @@ from sqlalchemy import (
String,
Text,
UniqueConstraint,
column,
func,
)
from sqlalchemy.dialects.postgresql import JSONB
@@ -56,6 +57,22 @@ class Team(Base):
away_matches: Mapped[list["Match"]] = relationship(foreign_keys="Match.away_team_id", back_populates="away_team")
class TeamAlias(Base):
"""球队别名:同一球队的不同写法(大小写/译名/缩写)映射到归一后的 teams.id。
入库流程(get_or_create):normalize(name) teams.name team_aliases
都没有再 insert Team别名不自动合并历史重复队,需显式添加
alias_normalized normalize(别名)后的稳定幂等键,用作 PK 避免重复插入
"""
__tablename__ = "team_aliases"
# normalize(别名)后的值,稳定幂等,用作主键
alias_normalized: Mapped[str] = mapped_column(String(120), primary_key=True)
team_id: Mapped[int] = mapped_column(ForeignKey("teams.id", ondelete="CASCADE"), nullable=False)
original_alias: Mapped[str] = mapped_column(String(120), nullable=False) # 原始写法(保留供参考)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow)
class Match(Base):
__tablename__ = "matches"
@@ -72,6 +89,9 @@ class Match(Base):
index=True,
)
match_status: Mapped[str] = mapped_column(String(20), default="scheduled")
# P0-01:比分可信度标记。known=有可靠比分;missing=完赛但缺分(保留 NULL 不伪造 0:0);
# unknown=待定(无比分且未确认完赛)。禁止把缺分写成 0:0。
score_status: Mapped[str] = mapped_column(String(20), server_default="unknown", nullable=False)
home_goals: Mapped[int | None] = mapped_column(Integer)
away_goals: Mapped[int | None] = mapped_column(Integer)
home_ht_goals: Mapped[int | None] = mapped_column(Integer)
@@ -111,10 +131,19 @@ class Match(Base):
"match_date_date",
unique=True,
),
# DB-5: 数据库级约束 — 已完赛比赛必须有比分
# P0-01:比分可信度约束(替代原 ck_matches_finished_has_score):
# - score_status=known → 必须有比分(非 NULL)
# - score_status=missing → 必须 NULL(完赛缺分,禁止伪造 0:0)
# - score_status=unknown → 必须 NULL
CheckConstraint(
"match_status <> 'finished' OR (home_goals IS NOT NULL AND away_goals IS NOT NULL)",
name="ck_matches_finished_has_score",
"score_status IN ('known', 'missing', 'unknown')",
name="ck_matches_score_status_enum",
),
CheckConstraint(
"match_status <> 'finished'"
" OR (score_status = 'known' AND home_goals IS NOT NULL AND away_goals IS NOT NULL)"
" OR (score_status IN ('missing', 'unknown') AND home_goals IS NULL AND away_goals IS NULL)",
name="ck_matches_score_integrity",
),
CheckConstraint(
"match_status IN ('finished', 'scheduled', 'in_play', 'paused', 'postponed', 'cancelled', 'suspended')",
@@ -176,8 +205,11 @@ class MatchStats(Base):
class Standing(Base):
"""联赛积分榜快照(bzzoiro /leagues/{id}/standings/)。
同一联赛同一赛季只保留最新快照:重新采集时按 (league_id, season, team_id)
upsertzone 来自 bzzoiro 分区( champions_league / europa_league / relegation)
P0-02: 改为追加快照(append-only)每次采集 INSERT 新行,available_at=now;
查询取 available_at<=cutoff 的每队最新快照(DISTINCT ON team_id ORDER available_at DESC)
回测时可还原任意历史时刻的榜单,不再只是"最新快照、忽略 cutoff"
同一 (league_id, season, team_id, available_at) 唯一,ON CONFLICT DO NOTHING
zone 来自 bzzoiro 分区( champions_league / europa_league / relegation)
"""
__tablename__ = "standings"
@@ -200,13 +232,17 @@ class Standing(Base):
zone: Mapped[str | None] = mapped_column(String(50)) # champions_league / relegation 等
updated_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow, onupdate=_utcnow)
retrieved_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), default=_utcnow)
# P0-02: 快照可用时间(采集时间),唯一键组成部分 + cutoff 过滤依据
available_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False, default=_utcnow)
league: Mapped[League] = relationship()
team: Mapped[Team] = relationship(lazy="selectin")
__table_args__ = (
UniqueConstraint("league_id", "season", "team_id", name="uq_standings_league_season_team"),
# P0-02: (league, season, team, available_at) 唯一,支持追加快照 + ON CONFLICT DO NOTHING
UniqueConstraint("league_id", "season", "team_id", "available_at", name="uq_standings_league_season_team_available"),
Index("ix_standings_league_season_pos", "league_id", "season", "position"),
Index("ix_standings_league_season_team_available", "league_id", "season", "team_id", "available_at"),
)
@@ -254,14 +290,13 @@ class Prediction(Base):
match: Mapped[Match] = relationship(back_populates="predictions")
__table_args__ = (
# Fix: 唯一约束增加 mode + run_type,允许 live 与 backtest 共存
# 防止回测覆盖未结算的实盘预测(后续 settle 会污染评估数据)
UniqueConstraint(
"match_id", "provider", "model", "mode", "run_type",
name="uq_predictions_match_provider_model_mode_run_type",
# P0-03: 幂等指纹——input_hash 非空时唯一(同指纹→返回已有行,不 UPDATE/INSERT);
# 兼容旧数据 NULL input_hash(不强制回填)。
Index(
"ix_predictions_input_hash_unique", "input_hash", unique=True,
postgresql_where=column("input_hash").isnot(None),
),
Index("ix_predictions_match", "match_id"),
Index("ix_predictions_provider_model", "provider", "model"),
# 数据截止时间过滤查询用(按 prediction_cutoff_at 取「赛前已生成」的预测)
Index("ix_predictions_cutoff_at", "prediction_cutoff_at"),
# 数据库级约束:最后一道防线
@@ -324,6 +359,35 @@ class RawEvent(Base):
)
class IngestJob(Base):
"""采集任务状态:跟踪每次触后台采集任务的执行进度与结果。
POST /api/v1/ingest/bzzoiro 触发时写入(pendingrunningsuccess/failed),
前端 Collection 页据此轮询到终态,替代此前"30 秒后盲标完成"的模拟
分批 get_uow / BzzoiroSource / IngestFailure / Bronze/Lineage 均不受影响
(本表仅作状态追踪,不介入采集事务)
"""
__tablename__ = "ingest_jobs"
id: Mapped[str] = mapped_column(String(36), primary_key=True) # uuid4
task: Mapped[str] = mapped_column(String(20), nullable=False)
params: Mapped[dict] = mapped_column(JSONB, nullable=False, default=dict)
status: Mapped[str] = mapped_column(String(20), nullable=False, server_default="pending")
result: Mapped[dict | None] = mapped_column(JSONB)
error: Mapped[str | None] = mapped_column(Text)
created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now())
started_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
finished_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True))
__table_args__ = (
Index("ix_ingest_job_status_created", "status", "created_at"),
CheckConstraint(
"status IN ('pending', 'running', 'success', 'failed')",
name="ck_ingest_job_status",
),
)
class IngestFailure(Base):
"""采集失败死信:记录失败原因、重试次数与下次重试时间。
+82 -25
View File
@@ -5,10 +5,15 @@ Repository 只负责查询,不负责事务提交。
"""
from __future__ import annotations
import logging
logger = logging.getLogger(__name__)
from datetime import datetime
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from sqlalchemy.ext.asyncio import AsyncSession
from sqlalchemy.dialects.postgresql import insert as pg_insert
from src.db.models import League, Match, Prediction, Team
@@ -76,6 +81,20 @@ class MatchRepository:
)
return (await self._session.execute(stmt)).scalars().all()
async def find_by_source_event_id(self, source_event_id: int) -> Match | None:
"""按上游 event id 查找比赛(唯一命中,用于 upsert 优先路径)。
source_event_id 上有 partial unique 索引(WHERE IS NOT NULL),
同联赛同主客同天(自然键)与上游 event_id 共同保障同一场比赛
重复采集时 upsert 而非插入重复行
"""
stmt = (
select(Match)
.options(selectinload(Match.stats))
.where(Match.source_event_id == source_event_id)
)
return (await self._session.execute(stmt)).scalar_one_or_none()
async def find_finished_with_stats(self, league_ids: list[int], *, limit: int) -> list[Match]:
"""已完赛且有上游 event id 的比赛(按日期倒序),供统计回填逐场拉取。
@@ -108,13 +127,62 @@ class TeamRepository:
return (await self._session.execute(stmt)).scalar_one_or_none()
async def get_or_create(self, name: str, *, name_zh: str | None = None) -> Team:
"""按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。"""
team = await self.get_by_name(name)
if team is None:
team = Team(name=name, name_zh=name_zh)
self._session.add(team)
"""按名获取球队,不存在则创建(name_zh 供 bzzoiro 管线写中文名)。
归一化咽喉 + 别名查找,三步定位:
1) normalize(name) teams.name
2) team_aliases( normalize(name) 为幂等键) 复用已映射的 teams.id
3) 都没有 insert Team(归一名)
创建新 Team info 打出原始名与归一后的规范名,便于排查重名
不自动合并历史重复队;需显式添加别名
"""
from src.data.team_names import normalize as normalize_name
from src.db.models import TeamAlias
normalized = normalize_name(name) or name.strip()
# 1) 归一名直查 teams
team = await self.get_by_name(normalized)
if team is not None:
return team
# 2) 别名查找:normalize(别名) 作为幂等键,命中即复用已有 Team
alias = await self._session.get(TeamAlias, normalized)
if alias is not None:
team = await self._session.get(Team, alias.team_id)
if team is not None:
logger.info("Team 别名命中: %s -> %s(已有 id=%s)", name, normalized, team.id)
return team
# 3) 新建 Team(归一名)。P1-K: 用 PG UPSERT 防并发重复插入。
logger.info("创建新 Team: %s -> %s", name, normalized)
stmt = pg_insert(Team).values(name=normalized, name_zh=name_zh)
stmt = stmt.on_conflict_do_nothing(index_elements=["name"])
await self._session.execute(stmt)
await self._session.flush()
# 无论是否本次插入,都拿到行(并发时可能已由对方创建)
return await self.get_by_name(normalized)
async def add_alias(self, alias: str, team_id: int) -> TeamAlias:
"""为已有 Team 添加别名。
幂等: normalize(alias) PK,重复添加同一别名会 upsert
不自动合并历史重复队,仅建立别名映射
"""
from src.data.team_names import normalize as normalize_name
from src.db.models import TeamAlias
normalized = normalize_name(alias) or alias.strip()
existing = await self._session.get(TeamAlias, normalized)
if existing is not None:
existing.team_id = team_id # 允许重新指向
existing.original_alias = alias
await self._session.flush()
return team
return existing
row = TeamAlias(alias_normalized=normalized, team_id=team_id, original_alias=alias)
self._session.add(row)
await self._session.flush()
return row
async def get_all_by_names(self, names: list[str]) -> dict[str, Team]:
"""批量获取球队,返回 name → Team 映射。"""
@@ -140,27 +208,16 @@ class LeagueRepository:
return (await self._session.execute(stmt)).scalar_one_or_none()
async def get_or_create(self, code: str, name: str, country: str | None = None) -> League:
"""按代码获取联赛,不存在则创建。P1-K: 创建用 PG UPSERT 防并发重复。"""
league = await self.get_by_code(code)
if league is None:
league = League(code=code, name=name, country=country)
self._session.add(league)
await self._session.flush()
return league
if league is not None:
return league
stmt = pg_insert(League).values(code=code, name=name, country=country)
stmt = stmt.on_conflict_do_nothing(index_elements=["code"])
await self._session.execute(stmt)
await self._session.flush()
return await self.get_by_code(code)
async def add(self, league: League) -> None:
self._session.add(league)
await self._session.flush()
class PredictionRepository:
"""预测记录数据访问。"""
def __init__(self, session: AsyncSession) -> None:
self._session = session
async def get_by_id(self, prediction_id: int) -> Prediction | None:
return await self._session.get(Prediction, prediction_id)
async def add(self, prediction: Prediction) -> None:
self._session.add(prediction)
await self._session.flush()
+19 -12
View File
@@ -12,7 +12,7 @@ from src.core.config import settings
from src.db.base import AsyncSessionLocal
from src.db.models import Match, Prediction
from src.db.unit_of_work import get_uow
from src.llm.predict import PredictResult, _upsert_prediction
from src.llm.predict import PredictResult, _insert_or_find_by_fingerprint
from src.llm.agents.base import AgentReport, AgentSpec, load_agent_prompt
from src.llm.context_builder import (
MatchHeader,
@@ -285,10 +285,13 @@ async def predict_match_multi(
latency_ms = int((time.perf_counter() - start) * 1000)
# 3.5 计算输入 hash(基于终裁报告)
input_hash = hashlib.sha256(
_reports_to_json(reports).encode("utf-8")
).hexdigest()
# P0-03: 指纹输入——终裁报告 hash 作 context_hash,专家列表作 agent_ids
reports_json = _reports_to_json(reports)
context_hash = hashlib.sha256(reports_json.encode("utf-8")).hexdigest()
agent_ids = sorted([r.agent for r in reports]) if reports else []
# 终裁模板 hash(规范:复用 prompt 版本 + 终裁 system prompt)
prompt_hash = hashlib.sha256(f"multi_{version}".encode("utf-8")).hexdigest()
system_prompt_hash = hashlib.sha256(AGGREGATOR_SYSTEM.encode("utf-8")).hexdigest()
# 4. 存库(使用 UnitOfWork)
async with get_uow() as session:
@@ -315,15 +318,20 @@ async def predict_match_multi(
pred_status = "degraded"
model_name = aggregator_model
pred = await _upsert_prediction(
pred = await _insert_or_find_by_fingerprint(
session,
match_id=match_id,
provider_name=settings.LLM_PROVIDER,
model=model_name,
mode="multi",
run_type="backtest" if backtest else "live",
values={
"match_id": match_id,
"provider": settings.LLM_PROVIDER,
"model": model_name,
"mode": "multi",
"run_type": "backtest" if backtest else "live",
"prompt_version": f"multi_{version}",
"prompt_hash": prompt_hash,
"system_prompt_hash": system_prompt_hash,
"temperature": 0.2,
"context_hash": context_hash,
"agent_ids": agent_ids,
"prompt_tokens": sum(r.prompt_tokens or 0 for r in reports) + agg_prompt_tokens,
"completion_tokens": sum(r.completion_tokens or 0 for r in reports) + agg_completion_tokens,
"latency_ms": latency_ms,
@@ -341,7 +349,6 @@ async def predict_match_multi(
"match_kickoff_at": match_kickoff_at,
"prediction_cutoff_at": prediction_cutoff_at,
"prediction_created_at": now,
"input_hash": input_hash,
},
)
+1
View File
@@ -130,6 +130,7 @@ async def _get_historical_matches(
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.where(Match.away_goals.is_not(None))
)
+51 -12
View File
@@ -5,14 +5,15 @@
"""
from __future__ import annotations
import hashlib
import logging
from datetime import datetime
from datetime import datetime, timedelta, timezone
from sqlalchemy import case, func, select
from src.db.base import AsyncSession, AsyncSessionLocal
from src.db.models import Match
from src.llm.predict import PredictResult
from src.llm.predict import PredictResult, _insert_or_find_by_fingerprint
logger = logging.getLogger(__name__)
@@ -58,19 +59,23 @@ async def predict_baseline(
返回 PredictResult(D2 统一结果类型):
provider=model="baseline", 不调用 LLM,latency_ms0
prediction_id 为占位 0 baseline 不在服务层落库,
由路由层 _persist_baseline 落库后取得真实 id
P3-2:baseline 落库下沉到服务层 直接在服务层完成落库并回填真实
prediction_id,路由层不再需要特殊的 _persist_baseline, single/multi
路径统一(result.prediction_id 即可用)对外 JSON 不变
"""
from src.db.unit_of_work import get_uow
async with AsyncSessionLocal() as db:
match = await db.get(Match, match_id)
if match is None:
raise ValueError(f"match {match_id} not found")
before = None
if backtest and match.match_dt:
if backtest and match.match_date:
from datetime import timedelta
before = match.match_dt - timedelta(days=1)
before = match.match_date - timedelta(days=1)
elif cutoff_at is not None:
before = cutoff_at
@@ -93,8 +98,45 @@ async def predict_baseline(
else:
pred_1x2 = "X"
# P0-03: 基线指纹——基于主客场场均进球数据(context_hash) + 截止时间
context_hash = hashlib.sha256(
f"{home_avg:.4f}:{away_avg:.4f}:{before.isoformat() if before else 'none'}".encode("utf-8")
).hexdigest()
values = {
"match_id": match_id,
"provider": "baseline",
"model": "baseline",
"mode": "baseline",
"run_type": "live",
"prompt_version": "baseline_v1",
"prompt_hash": hashlib.sha256(b"baseline_v1").hexdigest(),
"system_prompt_hash": hashlib.sha256(b"baseline").hexdigest(),
"temperature": 0.0,
"context_hash": context_hash,
"agent_ids": [],
"prompt_tokens": 0,
"completion_tokens": 0,
"latency_ms": 0,
"pred_home_goals": float(pred_home),
"pred_away_goals": float(pred_away),
"pred_1x2": pred_1x2,
"subjective_confidence": 0.5,
"reasoning": (
f"基线估计(非投注建议): 主队主场场均进球 {home_avg:.2f} → 预测 {pred_home}; "
f"客队客场场均进球 {away_avg:.2f} → 预测 {pred_away}"
),
"raw_response": {"home_avg": round(home_avg, 2), "away_avg": round(away_avg, 2)},
"status": "success",
}
# P0-03:服务层幂等插入,回填真实 prediction_id(与 single/multi 统一)。
async with get_uow() as session:
pred = await _insert_or_find_by_fingerprint(session, values=values)
prediction_id = pred.id
return PredictResult(
prediction_id=0, # 占位:真实 id 由路由层 _persist_baseline 落库后返回
prediction_id=prediction_id,
provider="baseline",
model="baseline",
prompt_version="baseline_v1",
@@ -105,14 +147,11 @@ async def predict_baseline(
alt_pred_away_goals=None,
pred_1x2=pred_1x2,
subjective_confidence=0.5,
reasoning=(
f"基线估计(非投注建议): 主队主场场均进球 {home_avg:.2f} → 预测 {pred_home}; "
f"客队客场场均进球 {away_avg:.2f} → 预测 {pred_away}"
),
reasoning=values["reasoning"],
context="", # baseline 不构建 LLM 上下文
status="success",
latency_ms=0,
prompt_tokens=0,
completion_tokens=0,
raw={"home_avg": round(home_avg, 2), "away_avg": round(away_avg, 2)},
raw=values["raw_response"],
)
+32 -547
View File
@@ -1,9 +1,9 @@
"""上下文构建器:数据切片 + 拼接。
"""上下文构建器:数据切片 + 拼接(聚合门面)
架构:
- match_header: 比赛基础信息(对阵双方/联赛/时间)
- 切片函数: 每个领域 agent 一个数据切片(h2h / form / standings / stats)
- build_context: agent 路径,拼接全部切片(行为与旧版一致)
实现按 slice 拆分(单文件 slices ),本模块只做再导出:
- 切片函数: 每个领域 agent 一个数据切片 src/llm/slices/{form,h2h,stats,home_away,standings}.py
- 共享类型/头信息/查询助手 src/llm/slices/common.py
- build_context: agent 路径,拼接全部切片(行为与旧版一致) src/llm/slices/aggregate.py
multi-agent 路径由 agents/orchestrator.py 调用切片函数,每个专家只拿自己的切片
@@ -11,548 +11,33 @@ multi-agent 路径由 agents/orchestrator.py 调用切片函数,每个专家只
build_context 创建一个共享 session 并传给所有切片函数,
避免每个切片独立创建 session 回测 20 场并发时,
5 个切片 × 20 = 100 个连接会耗尽连接池(pool_size=15)
消费方(路由/orchestrator/tests)仍从本模块 import,签名与拆分前完全一致
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
logger = logging.getLogger(__name__)
def _outcome(home_goals: int, away_goals: int, side: str) -> str:
"""从某队视角看赛果: W/D/L。"""
if home_goals is None or away_goals is None:
return "?"
if side == "home":
return "W" if home_goals > away_goals else ("D" if home_goals == away_goals else "L")
return "W" if away_goals > home_goals else ("D" if away_goals == home_goals else "L")
def _is_stats_available(stats, before) -> bool:
"""检查统计数据在 cutoff 时间是否已可用。
available_at 语义:该条统计对外可被使用的最早时间,
至少不得早于比赛结束用于回测防泄漏
规则:
- before is None(实盘):available_at None 时允许(兼容旧数据)
- before is not None(回测):available_at None 视为不可用(保守)
- available_at > cutoff:不可用(数据在 cutoff 之后才生成)
"""
if before is None:
# 实盘模式:无时间信息时允许(兼容旧数据)
return True
# 回测模式(cutoff 不为 None):
# available_at 为 None → 无法确认是否在 cutoff 前可用,保守视为不可用
if stats.available_at is None:
return False
return stats.available_at <= before
@dataclass
class SliceResult:
"""数据切片的显式结果(替代「靠文案子串猜有无数据」)。
旧实现用 `"无数据" in slice_text` 判断,依赖具体文案 一旦某个切片
写成无比分数据无伤停数据这类变体,判断就会静默失配
(见审查报告 P2-1)这里让切片函数直接声明 `has_data`,不再猜
"""
text: str
has_data: bool
n_records: int = 0
def __str__(self) -> str: # 让老调用点可直接当 str 用
return self.text
@dataclass
class MatchContext:
match_id: int
text: str
has_stats: bool
has_standings: bool
match_dt: object | None = None # 比赛时间(回测防泄漏 + 快照用)
cutoff: object | None = None # 实际使用的数据截止时间(用于落库记录)
@dataclass
class MatchHeader:
"""比赛基础信息(所有 agent 共享)。"""
match_id: int
home_name: str
away_name: str
league_name: str
season: str | None
match_date: str
match_dt: object # 原始 datetime,回测防泄漏用
stage: str | None
home_team_id: int
away_team_id: int
league_id: int
async def load_match_header(match_id: int, db: AsyncSession | None = None) -> MatchHeader:
"""加载比赛头信息(各 agent 共用)。
Args:
match_id: 比赛 ID
db: 可选的共享 session不传则自建(向后兼容)
"""
if db is not None:
m = await _load_match(db, match_id)
return _to_header(m)
async with AsyncSessionLocal() as new_db:
m = await _load_match(new_db, match_id)
return _to_header(m)
def _to_header(m: Match) -> MatchHeader:
return MatchHeader(
match_id=m.id,
home_name=m.home_team.name_zh or m.home_team.name,
away_name=m.away_team.name_zh or m.away_team.name,
league_name=m.league.name if m.league else "?",
season=m.season,
match_date=m.match_date.strftime("%Y-%m-%d %H:%M UTC") if m.match_date else "?",
match_dt=m.match_date,
stage=m.match_stage,
home_team_id=m.home_team_id,
away_team_id=m.away_team_id,
league_id=m.league_id,
)
def header_text(h: MatchHeader) -> str:
stage = f" {h.stage}" if h.stage else ""
return (
f"对阵: {h.home_name} vs {h.away_name} | {h.league_name} {h.season or '?'}{stage} | {h.match_date}"
)
# ============================================================
# 切片函数: 每个领域 agent 一个
# ============================================================
async def h2h_slice(header: MatchHeader, *, limit: int = 8, before=None, db: AsyncSession | None = None) -> SliceResult:
"""E - 历史交锋切片: 过去数年 + 近期交手数据,提取交手规律。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
h2h = await _get_h2h(db, header.home_team_id, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
h2h = await _get_h2h(new_db, header.home_team_id, header.away_team_id, before=before, limit=limit)
lines = [f"── 历史交锋(近 {limit} 次) ──"]
n_with_score = 0
if h2h:
# 从当前主队视角统计:判断当前主队在每场交锋中是主是客
current_home_wins = current_home_draws = current_home_losses = 0
for hm in h2h:
d = hm.match_date.strftime("%Y-%m") if hm.match_date else "?"
if hm.home_goals is not None:
n_with_score += 1
# 判断当前主队当时是主队还是客队
if hm.home_team_id == header.home_team_id:
# 当前主队当时是主队
if hm.home_goals > hm.away_goals:
current_home_wins += 1
elif hm.home_goals == hm.away_goals:
current_home_draws += 1
else:
current_home_losses += 1
else:
# 当前主队当时是客队(从客队视角看赛果)
if hm.away_goals > hm.home_goals:
current_home_wins += 1
elif hm.away_goals == hm.home_goals:
current_home_draws += 1
else:
current_home_losses += 1
lines.append(f" {d}: {hm.home_team.name} {hm.home_goals}-{hm.away_goals} {hm.away_team.name}")
else:
lines.append(f" {d}: {hm.home_team.name} vs {hm.away_team.name} (无比分)")
total = current_home_wins + current_home_draws + current_home_losses
if total:
lines.append(
f" 总计 {total} 场(从当前主队 {header.home_name} 视角): "
f"{current_home_wins}{current_home_draws}{current_home_losses}"
)
else:
lines.append(" 无数据")
# has_data 以「有比分的交锋」为准:仅有对阵无比分时不足以支撑分析
return SliceResult(text="\n".join(lines), has_data=n_with_score > 0, n_records=n_with_score)
async def form_slice(header: MatchHeader, *, limit: int = 5, before=None, db: AsyncSession | None = None) -> SliceResult:
"""A - 近期状态切片: 两队近 N 场赛果、关键事件、走势判断。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = []
n_scored = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则客场输球会被算成主场赢球。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
lines.append(f"── {label}近况({name},近 {limit} 场) ──")
if form:
wins = draws = losses = 0
for fm in form:
is_home = (fm.home_team_id == team_id)
side = "home" if is_home else "away"
o = _outcome(fm.home_goals, fm.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
if fm.home_goals is not None:
n_scored += 1
score = f"{fm.home_goals}-{fm.away_goals}" if fm.home_goals is not None else "vs"
xg = ""
if fm.stats and _is_stats_available(fm.stats, before) and fm.stats.home_xg is not None:
own = fm.stats.home_xg if is_home else fm.stats.away_xg
xg = f" (xG {own:.1f})"
opp = fm.away_team.name if is_home else fm.home_team.name
lines.append(f" {o} {score} vs {opp}{xg}")
lines.append(f"{len(form)} 场: {wins}{draws}{losses}")
else:
lines.append(" 无数据")
return SliceResult(text="\n".join(lines), has_data=n_scored > 0, n_records=n_scored)
async def stats_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""B - 攻防数据切片: 进球、射门、控球,评估攻防强度。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = [f"── 攻防数据(近 {limit} 场) ──"]
n_total = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则进球/失球/xG 全部算反。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
if form:
gf = ga = shots = sot = poss = xg = xga = 0
n = n_shots = n_poss = n_xg = 0
for fm in form:
if fm.home_goals is None: continue
is_home = (fm.home_team_id == team_id)
gf += fm.home_goals if is_home else fm.away_goals
ga += fm.away_goals if is_home else fm.home_goals
n += 1
# 只使用 cutoff 之前已可用的统计数据
if fm.stats and _is_stats_available(fm.stats, before):
if fm.stats.home_shots is not None:
shots += fm.stats.home_shots if is_home else fm.stats.away_shots
sot += fm.stats.home_shots_on_target if is_home else fm.stats.away_shots_on_target
n_shots += 1
if fm.stats.home_possession is not None:
poss += fm.stats.home_possession if is_home else (100 - fm.stats.home_possession)
n_poss += 1
if fm.stats.home_xg is not None:
xg += fm.stats.home_xg if is_home else fm.stats.away_xg
xga += fm.stats.away_xg if is_home else fm.stats.home_xg
n_xg += 1
n_total += n
if n > 0:
lines.append(f" {label} {name}:")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
if n_shots: lines.append(f" 场均射门 {shots/n_shots:.1f}, 射正 {sot/n_shots:.1f}")
if n_poss: lines.append(f" 平均控球 {poss/n_poss:.1f}%")
if n_xg: lines.append(f" 场均 xG {xg/n_xg:.2f}, 场均被 xG {xga/n_xg:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def home_away_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""C - 主客因素切片: 主场战绩 vs 客场战绩,评估地理优势影响。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连(见模块 docstring)
"""
if db is not None:
home_home = await _get_home_away(db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(db, header.away_team_id, "away", before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_home = await _get_home_away(new_db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(new_db, header.away_team_id, "away", before=before, limit=limit)
lines = ["── 主客因素 ──"]
n_total = 0
for label, name, matches, side in (
("主队主场", header.home_name, home_home, "home"),
("客队客场", header.away_name, away_away, "away"),
):
if matches:
wins = draws = losses = gf = ga = 0
for m in matches:
if m.home_goals is None: continue
o = _outcome(m.home_goals, m.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
gf += m.home_goals if side == "home" else m.away_goals
ga += m.away_goals if side == "home" else m.home_goals
n = wins + draws + losses
n_total += n
if n > 0:
pct = wins / n * 100
lines.append(f" {label} {name}(近 {n} 场): {wins}{draws}{losses}负, 胜率 {pct:.0f}%")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def standings_slice(header: MatchHeader, *, before=None, db: AsyncSession | None = None) -> SliceResult:
"""D - 联赛排名切片: 两队积分榜位置、积分、近期走势(form)、分区,评估整体实力差距。
before 参数保留与其他切片一致的签名(积分榜是最新快照,无历史版本,不受 cutoff 影响)
db: 可选共享 session(见模块 docstring)
语义区分:
- 两队都有积分榜行 has_data=True(明确的排名信息)
- 任一队缺失 has_data=False(升班马/杯赛无榜,信息不完整时明确声明)
"""
from src.db.models import League, Standing
if db is not None:
league = (await db.execute(select(League).where(League.id == header.league_id))).scalar_one_or_none()
rows = (
(
await db.execute(
select(Standing)
.options(selectinload(Standing.team))
.where(Standing.league_id == header.league_id)
.order_by(Standing.position.asc())
)
)
.scalars()
.all()
if league
else []
)
else:
async with AsyncSessionLocal() as new_db:
return await standings_slice(header, before=before, db=new_db)
lines = [f"── 联赛排名({header.league_name}{len(rows)} 队) ──"]
n_records = 0
def _fmt(row) -> str:
zg = f" xG差 {row.xgd:+.1f}" if row.xg_for is not None and row.xg_against is not None and row.goal_diff is not None else ""
form = f" 近5场 {row.form}" if row.form else ""
zone = f" [{row.zone}]" if row.zone else ""
return (
f"{row.position} 名: {row.points} 分 / {row.played}"
f"({row.won}{row.drawn}{row.lost}负, 进{row.goals_for}{row.goals_against} 净胜{row.goal_diff:+d}"
f"{zg}){form}{zone}"
)
for label, team_id in (("主队", header.home_team_id), ("客队", header.away_team_id)):
row = next((r for r in rows if r.team_id == team_id), None)
if row is None:
lines.append(f" {label}: 暂无积分榜数据(可能杯赛/赛季未开始)")
else:
n_records += 1
lines.append(f" {label} {header.home_name if label == '主队' else header.away_name}:")
lines.append(_fmt(row))
# 两队排名对比摘要
home_row = next((r for r in rows if r.team_id == header.home_team_id), None)
away_row = next((r for r in rows if r.team_id == header.away_team_id), None)
if home_row and away_row:
diff = home_row.position - away_row.position # 正数=主队排名更靠前(名次更小)
lead = f"主队排名高 {diff}" if diff > 0 else (f"客队排名高 {-diff}" if diff < 0 else "两队同排名结构")
pts_diff = home_row.points - away_row.points
lines.append(f" 排名对比: {lead}, 分差 {pts_diff:+d}")
# has_data: 两队都有行才算完整;只有一队时仍有价值,但标记不完整
has_data = n_records >= 1
return SliceResult(text="\n".join(lines), has_data=has_data, n_records=n_records)
# ============================================================
# 单 agent 路径: 拼接全部切片(行为与旧版一致)
# ============================================================
async def build_context(match_id: int, *, form_last: int = 5, h2h_last: int = 5, backtest: bool = False, cutoff_at=None) -> MatchContext:
"""单 agent 路径的完整上下文: 拼接全部切片(before=cutoff,防未来信息)。
has_stats / has_standings 直接取切片显式声明的 has_data,
不再靠文案子串匹配(见审查报告 P2-1)
P2-6: backtest=True cutoff = match_date - 1,确保只用赛前数据
cutoff_at: 显式截止时间(优先于 backtest 自动计算)
P1-1: 使用单个共享 session 贯穿所有切片查询,避免连接池耗尽
"""
async with AsyncSessionLocal() as db:
header = await load_match_header(match_id, db=db)
# 计算数据截止时间: 显式 > backtest 自动计算 > 默认(比赛时间)
if cutoff_at is not None:
cutoff = cutoff_at
elif backtest and header.match_dt:
from datetime import timedelta
cutoff = header.match_dt - timedelta(days=1)
else:
cutoff = header.match_dt
parts = [header_text(header), ""]
form_res = await form_slice(header, limit=form_last, before=cutoff, db=db)
parts.append(form_res.text)
parts.append("")
h2h_res = await h2h_slice(header, limit=h2h_last, before=cutoff, db=db)
parts.append(h2h_res.text)
parts.append("")
stats_res = await stats_slice(header, before=cutoff, db=db)
parts.append(stats_res.text)
parts.append("")
home_away_res = await home_away_slice(header, before=cutoff, db=db)
parts.append(home_away_res.text)
parts.append("")
standings_res = await standings_slice(header, before=cutoff, db=db)
parts.append(standings_res.text)
return MatchContext(
match_id=match_id,
text="\n".join(parts),
has_stats=form_res.has_data or stats_res.has_data,
has_standings=standings_res.has_data,
match_dt=header.match_dt,
cutoff=cutoff,
)
# ============================================================
# 底层查询(切片函数共用)
# ============================================================
async def _load_match(db, match_id: int) -> Match:
stmt = (
select(Match)
.where(Match.id == match_id)
.options(
selectinload(Match.league),
selectinload(Match.home_team),
selectinload(Match.away_team),
selectinload(Match.stats),
)
)
m = (await db.execute(stmt)).scalar_one_or_none()
if m is None:
raise ValueError(f"match {match_id} not found")
return m
async def _get_form(db, team_id: int, before, *, limit: int = 5) -> list[Match]:
"""某队近 N 场(已完赛)。before=None 表示不限制(预测赛前的场景由调用方保证)。
必须预加载 stats / home_team / away_team:切片函数会读取这些关系,
async session 下惰性加载会抛 MissingGreenlet
(models.py 已声明 lazy="selectin",此处显式声明以固化查询意图)
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.where((Match.home_team_id == team_id) | (Match.away_team_id == team_id))
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
async def _get_h2h(db, home_id: int, away_id: int, before, *, limit: int = 5) -> list[Match]:
"""两队交锋史。需预加载 home_team / away_team(切片输出队名)。"""
stmt = (
select(Match)
.options(
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.where(
((Match.home_team_id == home_id) & (Match.away_team_id == away_id))
| ((Match.home_team_id == away_id) & (Match.away_team_id == home_id))
)
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
async def _get_home_away(db, team_id: int, side: str, before, *, limit: int = 10) -> list[Match]:
"""某队主场/客场近 N 场。side='home' 取主场,'away' 取客场。
当前只用标量字段,但统一预加载以免后续扩展时踩坑
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.home_goals.is_not(None))
.order_by(Match.match_date.desc())
.limit(limit)
)
if side == "home":
stmt = stmt.where(Match.home_team_id == team_id)
else:
stmt = stmt.where(Match.away_team_id == team_id)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
# ── 共享类型与基础(判空/赛果/统计可用性) ──
from src.llm.slices.common import ( # noqa: F401
MatchContext,
MatchHeader,
SliceResult,
_is_stats_available,
_outcome,
header_text,
load_match_header,
)
# ── 领域切片 ──
from src.llm.slices.form import form_slice # noqa: F401
from src.llm.slices.h2h import h2h_slice # noqa: F401
from src.llm.slices.home_away import home_away_slice # noqa: F401
from src.llm.slices.standings import standings_slice # noqa: F401
from src.llm.slices.stats import stats_slice # noqa: F401
# ── 底层查询助手(切片函数共用;orchestrator/tests 直接引用) ──
from src.llm.slices.form import _get_form # noqa: F401
from src.llm.slices.h2h import _get_h2h # noqa: F401
from src.llm.slices.home_away import _get_home_away # noqa: F401
# ── 单 agent 聚合入口 ──
from src.llm.slices.aggregate import build_context # noqa: F401
+23 -10
View File
@@ -7,6 +7,7 @@ from sqlalchemy import func, or_, select
from src.db.models import Prediction, Match, League
from src.db.unit_of_work import get_uow
from src.llm.utils import actual_1x2
logger = logging.getLogger(__name__)
@@ -33,12 +34,8 @@ async def settle_prediction(prediction_id: int, home_goals: int, away_goals: int
def _actual_1x2(home: int, away: int) -> str:
"""根据实际比分返胜平负。"""
if home > away:
return "1"
if home == away:
return "X"
return "2"
"""根据实际比分返胜平负(委托 utils.actual_1x2 单一权威源)"""
return actual_1x2(home, away)
def _build_filters(
@@ -47,9 +44,17 @@ def _build_filters(
prompt_version: str | None = None,
mode: str | None = None,
league_code: str | None = None,
run_type: str | None = "live",
season: str | None = None,
) -> list:
"""构建评估筛选条件(参数化列明,防拼接注入)。"""
"""构建评估筛选条件(参数化列明,防拼接注入)。
P1-I: 默认仅 run_type=live(排除回测污染),可显式改 "backtest"/None
"""
filters = [Prediction.settled == True]
# P1-I: 默认仅统计实盘预测(排除回测),除非显式指定
if run_type is not None:
filters.append(Prediction.run_type == run_type)
if provider:
filters.append(Prediction.provider == provider)
if model:
@@ -58,6 +63,10 @@ def _build_filters(
filters.append(Prediction.prompt_version == prompt_version)
if mode:
filters.append(Prediction.mode == mode)
# P1-I: 赛季过滤
if season:
season_subq = select(Match.id).where(Match.season == season).scalar_subquery()
filters.append(Prediction.match_id.in_(season_subq))
if league_code:
league_subq = select(League.id).where(League.code == league_code).scalar_subquery()
filters.append(Prediction.match_id.in_(
@@ -67,17 +76,21 @@ def _build_filters(
async def get_eval_summary(
limit: int = 1000,
limit: int | None = None,
*,
provider: str | None = None,
model: str | None = None,
prompt_version: str | None = None,
mode: str | None = None,
league_code: str | None = None,
run_type: str | None = "live",
season: str | None = None,
) -> dict:
"""按 provider × 模型聚合评估。
P3-4: 默认限制评估最近 1000 条已结算预测,避免全表加载导致内存压力
P1-I: 默认 limit=None(返回全集,不冒充 1000 条为全集);调用方可显式 limit 采样
默认仅 run_type=live(排除回测污染),可按需改 "backtest"
可选 season 过滤
只统计有效预测:
- settled == True
@@ -85,7 +98,7 @@ async def get_eval_summary(
- 预测比分字段齐全
degraded 或无比分的预测不计入准确率
"""
filters = _build_filters(provider, model, prompt_version, mode, league_code)
filters = _build_filters(provider, model, prompt_version, mode, league_code, run_type=run_type, season=season)
async with get_uow() as session:
total_settled = (await session.execute(
+203 -74
View File
@@ -1,4 +1,16 @@
"""预测服务:拼上下文 → 调 LLM → 存预测。"""
"""预测服务:拼上下文 → 调 LLM → 存预测。
落库层级(table: predictions):
| 模式 | 落库位置(服务层) | 路由层(routes/predict.py) |
|-----------|-------------------------------------------------------------|---------------------------|
| single | `_predict_single` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
| multi | `orchestrator.predict_match_multi` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
| baseline | `predict_baseline` `_insert_or_find_by_fingerprint` | 不读 DB,仅映射 result PredictOut |
三种模式统一在服务层经 UnitOfWork 落库并回填真实 prediction_id;
路由层永不写入 predictions,只读 result.prediction_id 做响应映射
"""
from __future__ import annotations
import functools
@@ -25,9 +37,7 @@ _PROMPT_DIR = Path(__file__).resolve().parent / "prompts"
# ── LLM 响应缓存(match+provider+model+version → 结果) ──
_CACHE_TTL_SEC = 300 # 5 分钟
_CACHE_MAX_SIZE = 200 # P3-1: 有上限,避免长期运行内存无限增长
# P1-5: 缓存仅在 asyncio 协程内同步访问(dict 操作 GIL 原子),无需 threading.Lock。
# 删除 _cache_lock,避免同步锁阻塞事件循环;dict 的 get/set 在 CPython 下原子。
_cache: dict[str, tuple[float, PredictResult]] = {}
_CACHE_PREFIX = "predict:" # Redis key 前缀
def _cache_key(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> str:
@@ -36,43 +46,139 @@ def _cache_key(match_id: int, provider: str, model: str, version: str, tpl_hash:
仅用 version 做键不够 编辑器里改动 `match_prediction_v1.md` 而版本号
不变时,进程内缓存仍会返回旧模板产生的旧结果(见审查报告 P2-6)
把模板内容 hash 纳入键,模板一改缓存自动失效
内存与 Redis 共用同一键格式,TTL 一致
"""
return f"{match_id}:{provider}:{model}:{version}:{tpl_hash[:12]}"
return f"{_CACHE_PREFIX}{match_id}:{provider}:{model}:{version}:{tpl_hash[:12]}"
def _get_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> PredictResult | None:
# P1-5: 无锁访问。dict get/del 在 CPython GIL 下原子,且无 await 穿插。
key = _cache_key(match_id, provider, model, version, tpl_hash)
entry = _cache.get(key)
if entry is not None:
ts, result = entry
if time.time() - ts < _CACHE_TTL_SEC:
return result
_cache.pop(key, None)
return None
# ── 缓存后端:内存(LRU+TTL),可选 Redis ──────────────────────────
class _CacheBackend:
"""缓存后端统一接口:_get 同步返回(命中时),_set 异步(Redis 为 async,内存同步)。"""
def _raw_key(self, key: str) -> str:
return key
def get(self, key: str) -> PredictResult | None:
raise NotImplementedError
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
raise NotImplementedError
def _set_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str, result: PredictResult) -> None:
# P1-5: 无锁写入。同上,dict set 原子。
key = _cache_key(match_id, provider, model, version, tpl_hash)
_cache[key] = (time.time(), result)
# P3-1: 超过上限时淘汰最旧条目(按时间戳排序)
if len(_cache) > _CACHE_MAX_SIZE:
oldest_key = min(_cache, key=lambda k: _cache[k][0])
_cache.pop(oldest_key, None)
class _MemoryCache(_CacheBackend):
"""进程内 LRU+TTL 缓存(默认后端)。"""
def __init__(self) -> None:
self._store: dict[str, tuple[float, PredictResult]] = {}
def get(self, key: str) -> PredictResult | None:
entry = self._store.get(key)
if entry is not None:
ts, result = entry
if time.time() - ts < _CACHE_TTL_SEC:
return result
self._store.pop(key, None)
return None
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
self._store[key] = (time.time(), result)
if len(self._store) > _CACHE_MAX_SIZE:
oldest_key = min(self._store, key=lambda k: self._store[k][0])
self._store.pop(oldest_key, None)
def clear_prompt_cache() -> None:
"""清空 prompt 模板缓存(供开发/热更新时手动调用)
class _RedisCache(_CacheBackend):
"""可选 Redis 后端: PREDICT_CACHE_URL 非空时启用
lru_cache 的模板缓存是进程级的,改完 .md 需要重启进程才能生效;
提供显式清理入口,避免"改了模板却看不到变化"的困惑(见审查报告 P2-5)
失败降级:读失败返回 None(跳过缓存),写失败打 warning;不中断预测主流程
不强制依赖 redis 未安装时启动回退内存并 warning
"""
_load_prompt_template.cache_clear()
logger.info("prompt 模板缓存已清空")
def __init__(self, url: str) -> None:
self._url = url
self._redis = None # type: ignore[var-annotated]
self._memory_fallback = _MemoryCache()
self._available: bool | None = None # None=未探测,True=可用,False=不可用
async def _ensure_conn(self) -> bool:
"""懒初始化 Redis 连接;失败返回 False 并降级内存。"""
if self._available is not None:
return self._available
try:
from redis.asyncio import Redis
self._redis = Redis.from_url(self._url, decode_responses=True, socket_timeout=2.0)
await self._redis.ping()
self._available = True
logger.info("predict cache: Redis 后端已连接 %s", self._url.replace(self._url.split("@")[-1] if "@" in self._url else self._url, "***") if "://" in self._url else "redis")
except Exception as e:
self._available = False
logger.warning("predict cache: Redis 连接失败(%s),降级内存缓存", e)
return self._available
def get(self, key: str) -> PredictResult | None:
# Redis get 是 async 的,此处统一由调用方走 async 路径;
# 同步 get 仅用于不可降级场景——Redis 模式下直接返回 None,
# 实际读取通过 get_async 完成。
return None
async def get_async(self, key: str) -> PredictResult | None:
if not await self._ensure_conn():
return self._memory_fallback.get(key)
try:
import json
raw = await self._redis.get(key) # type: ignore[union-attr]
if raw is None:
return None
# P1-H: JSON 序列化(替代 pickle,跨语言安全 + 可人工阅读)
data = json.loads(raw)
return PredictResult(**data)
except Exception as e:
logger.warning("predict cache: Redis GET 失败(%s),跳过缓存", e)
return None
async def set(self, key: str, result: PredictResult, ttl: int) -> None:
if not await self._ensure_conn():
await self._memory_fallback.set(key, result, ttl)
return
try:
import json
from dataclasses import asdict
payload = asdict(result)
# JSON 序列化;处理 datetime → ISO 字符串
for k, v in payload.items():
if hasattr(v, "isoformat"):
payload[k] = v.isoformat()
await self._redis.set(key, json.dumps(payload, default=str), ex=ttl) # type: ignore[union-attr]
except Exception as e:
logger.warning("predict cache: Redis SET 失败(%s),降级内存写入", e)
await self._memory_fallback.set(key, result, ttl)
def _build_cache_backend() -> _CacheBackend:
url = getattr(settings, "PREDICT_CACHE_URL", None)
if url:
return _RedisCache(url)
return _MemoryCache()
_cache_backend: _CacheBackend = _build_cache_backend()
async def _get_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str) -> PredictResult | None:
key = _cache_key(match_id, provider, model, version, tpl_hash)
if isinstance(_cache_backend, _RedisCache):
return await _cache_backend.get_async(key)
return _cache_backend.get(key)
async def _set_cached(match_id: int, provider: str, model: str, version: str, tpl_hash: str, result: PredictResult) -> None:
key = _cache_key(match_id, provider, model, version, tpl_hash)
await _cache_backend.set(key, result, _CACHE_TTL_SEC)
@functools.lru_cache(maxsize=8)
def _load_prompt_template(version: str = "v1") -> str:
"""缓存 prompt 模板(进程生命周期内每个版本只读一次)。"""
path = _PROMPT_DIR / f"match_prediction_{version}.md"
@@ -121,44 +227,60 @@ class PredictResult:
raw: dict | None = None
async def _upsert_prediction(
session,
*,
match_id: int,
provider_name: str,
model: str,
mode: str,
run_type: str,
values: dict,
) -> Prediction:
"""按 (match, provider, model, mode, run_type) 唯一约束写入预测。
def _compute_fingerprint(values: dict) -> str:
"""P0-03: 预测指纹(规范 JSON 的 SHA-256)。
已存在且未结算 覆盖更新(重新预测语义);已结算 拒绝(保护评估数据)
run_type 区分 live/backtest,避免回测覆盖实盘预测
捕获影响预测输出的全部因素:输入提示模型采样截止时间专家
fingerprint 返回已有行( UPDATE/INSERT);不同 INSERT 新行
"""
import json as _json
canonical = {
"match_id": values.get("match_id"),
"prediction_cutoff_at": _iso(values.get("prediction_cutoff_at")),
"prompt_version": values.get("prompt_version"),
"prompt_hash": values.get("prompt_hash"),
"system_prompt_hash": values.get("system_prompt_hash"),
"provider": values.get("provider"),
"model": values.get("model"),
"mode": values.get("mode"),
"run_type": values.get("run_type"),
"temperature": values.get("temperature"),
"context_hash": values.get("context_hash"),
"agent_ids": sorted(values.get("agent_ids") or []),
}
blob = _json.dumps(canonical, sort_keys=True, separators=(',', ':'))
return hashlib.sha256(blob.encode("utf-8")).hexdigest()
def _iso(v) -> str | None:
if v is None:
return None
if hasattr(v, "isoformat"):
return v.isoformat()
return str(v)
async def _insert_or_find_by_fingerprint(session, *, values: dict) -> Prediction:
"""P0-03: 幂等插入——同 input_hash 返回已有行(不 UPDATE);不同则 INSERT。
不再按 (match, provider, model, mode, run_type) upsert,避免覆盖已有预测
values 必须包含 fingerprint 所需全部字段( _compute_fingerprint)
"""
fingerprint = _compute_fingerprint(values)
values["input_hash"] = fingerprint
existing = (
await session.execute(
select(Prediction).where(
Prediction.match_id == match_id,
Prediction.provider == provider_name,
Prediction.model == model,
Prediction.mode == mode,
Prediction.run_type == run_type,
)
select(Prediction).where(Prediction.input_hash == fingerprint)
)
).scalar_one_or_none()
if existing is not None and existing.settled:
raise ValueError("该比赛已有已结算的预测,不能重新预测")
if existing is not None:
# 同指纹 → 直接返回,绝不覆盖 pred_* / reasoning / agent_outputs
return existing
pred = existing if existing is not None else Prediction(
match_id=match_id, provider=provider_name, model=model,
)
pred.mode = mode
pred.run_type = run_type
for k, v in values.items():
setattr(pred, k, v)
if existing is None:
session.add(pred)
pred = Prediction(**{k: v for k, v in values.items() if hasattr(Prediction, k)})
session.add(pred)
await session.flush() # 拿到自增 id;事务由 UnitOfWork 退出时提交
return pred
@@ -235,7 +357,7 @@ async def _predict_single(
# 0. 查缓存(同 match+provider+model+version+模板hash 5 分钟内直接返)
if use_cache:
cached = _get_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash)
cached = await _get_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash)
if cached is not None:
logger.debug("predict cache hit match=%s", match_id)
return cached
@@ -243,23 +365,26 @@ async def _predict_single(
# 1. 拼上下文(backtest/cutoff 防泄漏)
ctx = await build_context(match_id, backtest=backtest, cutoff_at=cutoff_at)
# 1.5 计算快照元数据(用于可复现性)
# 1.5 计算快照元数据(用于可复现性 + P0-03 指纹)
now = datetime.now(timezone.utc)
match_kickoff_at = ctx.match_dt
# 使用上下文实际计算的 cutoff(回测时可能为 match_dt-1天),而非开球时间
prediction_cutoff_at = ctx.cutoff if ctx.cutoff is not None else ctx.match_dt
input_hash = hashlib.sha256(ctx.text.encode("utf-8")).hexdigest()
# 2. 拼 prompt(指定版本)
template = _load_prompt_template(version)
prompt_hash = _prompt_template_hash(version)
user_prompt = template.replace("{{context}}", ctx.text)
system_prompt = "你是一个严谨的足球预测专家。只输出 JSON。"
context_hash = hashlib.sha256(ctx.text.encode("utf-8")).hexdigest()
# 3. 调 LLM
temperature = 0.3
resp = await provider.chat(
system="你是一个严谨的足球预测专家。只输出 JSON。",
system=system_prompt,
user=user_prompt,
json_mode=True,
temperature=0.3,
temperature=temperature,
max_tokens=4096, # 推理模型的 reasoning 也计入输出 token,需留足余量
)
@@ -286,15 +411,20 @@ async def _predict_single(
if m is None:
raise ValueError(f"match {match_id} not found")
pred = await _upsert_prediction(
pred = await _insert_or_find_by_fingerprint(
session,
match_id=match_id,
provider_name=settings.LLM_PROVIDER,
model=provider.model,
mode="single",
run_type="backtest" if backtest else "live",
values={
"match_id": match_id,
"provider": settings.LLM_PROVIDER,
"model": provider.model,
"mode": "single",
"run_type": "backtest" if backtest else "live",
"prompt_version": version,
"prompt_hash": prompt_hash,
"system_prompt_hash": hashlib.sha256(system_prompt.encode("utf-8")).hexdigest(),
"temperature": temperature,
"context_hash": context_hash,
"agent_ids": [],
"prompt_tokens": resp.prompt_tokens,
"completion_tokens": resp.completion_tokens,
"latency_ms": resp.latency_ms,
@@ -310,7 +440,6 @@ async def _predict_single(
"match_kickoff_at": match_kickoff_at,
"prediction_cutoff_at": prediction_cutoff_at,
"prediction_created_at": now,
"input_hash": input_hash,
},
)
@@ -334,7 +463,7 @@ async def _predict_single(
# 5. 写入缓存(仅当允许缓存时)
if use_cache:
_set_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash, result)
await _set_cached(match_id, settings.LLM_PROVIDER, provider.model, version, tpl_hash, result)
logger.info(
"预测完成 match=%s mode=%s status=%s pred=%s:%s (%s) latency=%sms",
match_id, "single", "success",
+4 -4
View File
@@ -8,8 +8,8 @@
你的任务: 综合权衡各报告,输出最终预测。
裁决规则:
- 各报告的 confidence 和 data_sufficiency 是采信依据: no_data/error 状态的报告必须忽略,不得编造
- 5 位专家: 近期状态分析专家 / 攻防数据分析专家 / 主客因素分析专家 / 阵容完整性分析专家 / 历史交锋分析专家
- 各报告的 subjective_confidence 和 data_sufficiency 是采信依据: no_data/error 状态的报告必须忽略,不得编造
- 5 位专家: 近期状态分析专家 / 攻防数据分析专家 / 主客因素分析专家 / 联赛排名分析专家 / 历史交锋分析专家
- 引用专家意见时使用上述全称,不要使用英文代码(form/stats/h2h 等)
- home_edge 是各专家的方向性判断(-1~1),冲突时给出你的权衡理由
- agent_weights 体现你对各报告的采信度(0-1,总和无须为 1)
@@ -23,8 +23,8 @@
"alt_pred_home_goals": <int 0-10, 备选比分主队进球(第二可能的比分,必须与主选不同)>,
"alt_pred_away_goals": <int 0-10, 备选比分客队进球>,
"1x2": "<'1'|'X'|'2'>",
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"reasoning": "<250 字内推理,引用各报告证据>",
"agent_weights": {"form": <0-1>, "stats": <0-1>, "home_away": <0-1>, "injuries": <0-1>, "h2h": <0-1>}
"agent_weights": {"form": <0-1>, "stats": <0-1>, "home_away": <0-1>, "standings": <0-1>, "h2h": <0-1>}
}
```
+1 -1
View File
@@ -15,7 +15,7 @@
"data_sufficiency": "high|medium|low|none",
"analysis": "<150 字内分析,含关键事件与走势判断>",
"home_edge": <-1.0 到 1.0, 正数=主队状态更好/走势更向上>,
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"key_evidence": ["<证据1>", "<证据2>"]
}
```
+1 -1
View File
@@ -16,7 +16,7 @@
"data_sufficiency": "high|medium|low|none",
"analysis": "<150 字内分析,提取交手规律>",
"home_edge": <-1.0 到 1.0, 正数=主队交锋占优>,
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"key_evidence": ["<证据1>", "<证据2>"]
}
```
+1 -1
View File
@@ -15,7 +15,7 @@
"data_sufficiency": "high|medium|low|none",
"analysis": "<150 字内分析,量化主客因素影响>",
"home_edge": <-1.0 到 1.0, 正数=主场优势明显>,
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"key_evidence": ["<证据1>", "<证据2>"]
}
```
+1 -1
View File
@@ -17,7 +17,7 @@
"data_sufficiency": "high|medium|low|none",
"analysis": "<150 字内分析,量化两队实力差距>",
"home_edge": <-1.0 到 1.0, 正数=主队实力占优>,
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"key_evidence": ["<证据1>", "<证据2>"]
}
```
+1 -1
View File
@@ -16,7 +16,7 @@
"data_sufficiency": "high|medium|low|none",
"analysis": "<150 字内分析,量化攻防强度>",
"home_edge": <-1.0 到 1.0, 正数=主队攻防占优>,
"confidence": <0.0-1.0>,
"subjective_confidence": <0.0-1.0>,
"key_evidence": ["<证据1>", "<证据2>"]
}
```
+1 -1
View File
@@ -10,7 +10,7 @@
"alt_pred_home_goals": "<int 0-10, 备选比分主队进球(第二可能的比分,必须与主选不同)>",
"alt_pred_away_goals": "<int 0-10, 备选比分客队进球>",
"1x2": "<'1'|'X'|'2'>",
"confidence": "<0.0-1.0>",
"subjective_confidence": "<0.0-1.0>",
"score_probable": {"home": "<int>", "away": "<int>", "prob": "<float>"},
"reasoning": "<200 字内推理>"
}
+1 -1
View File
@@ -15,7 +15,7 @@
"alt_pred_home_goals": "<int 0-10, 备选比分主队进球(第二可能的比分,必须与主选不同)>",
"alt_pred_away_goals": "<int 0-10, 备选比分客队进球>",
"1x2": "<'1'|'X'|'2'>",
"confidence": "<0.0-1.0>",
"subjective_confidence": "<0.0-1.0>",
"score_probable": {"home": "<int>", "away": "<int>", "prob": "<float>"},
"reasoning": "<200 字内推理,需引用具体数据>"
}
+5
View File
@@ -0,0 +1,5 @@
"""slices 包:按领域拆分的数据切片(form/stats/h2h/home_away/standings)。
对外统一经 src.llm.context_builder 再导出;本包 __init__ 不承载导出,
保持context_builder 是唯一公开入口 import 约定
"""
+65
View File
@@ -0,0 +1,65 @@
"""单 agent 聚合路径: 拼接全部切片(build_context)。
共享 session 贯穿所有切片( context_builder 模块 docstring 的性能说明)
"""
from __future__ import annotations
from src.db.base import AsyncSessionLocal
from src.llm.slices.common import MatchContext, header_text, load_match_header
from src.llm.slices.form import form_slice
from src.llm.slices.h2h import h2h_slice
from src.llm.slices.home_away import home_away_slice
from src.llm.slices.standings import standings_slice
from src.llm.slices.stats import stats_slice
async def build_context(match_id: int, *, form_last: int = 5, h2h_last: int = 5, backtest: bool = False, cutoff_at=None) -> MatchContext:
"""单 agent 路径的完整上下文: 拼接全部切片(before=cutoff,防未来信息)。
has_stats / has_standings 直接取切片显式声明的 has_data,
不再靠文案子串匹配(见审查报告 P2-1)
P2-6: backtest=True cutoff = match_date - 1,确保只用赛前数据
cutoff_at: 显式截止时间(优先于 backtest 自动计算)
P1-1: 使用单个共享 session 贯穿所有切片查询,避免连接池耗尽
"""
async with AsyncSessionLocal() as db:
header = await load_match_header(match_id, db=db)
# 计算数据截止时间: 显式 > backtest 自动计算 > 默认(比赛时间)
if cutoff_at is not None:
cutoff = cutoff_at
elif backtest and header.match_dt:
from datetime import timedelta
cutoff = header.match_dt - timedelta(days=1)
else:
cutoff = header.match_dt
parts = [header_text(header), ""]
form_res = await form_slice(header, limit=form_last, before=cutoff, db=db)
parts.append(form_res.text)
parts.append("")
h2h_res = await h2h_slice(header, limit=h2h_last, before=cutoff, db=db)
parts.append(h2h_res.text)
parts.append("")
stats_res = await stats_slice(header, before=cutoff, db=db)
parts.append(stats_res.text)
parts.append("")
home_away_res = await home_away_slice(header, before=cutoff, db=db)
parts.append(home_away_res.text)
parts.append("")
standings_res = await standings_slice(header, before=cutoff, db=db)
parts.append(standings_res.text)
return MatchContext(
match_id=match_id,
text="\n".join(parts),
has_stats=form_res.has_data or stats_res.has_data,
has_standings=standings_res.has_data,
match_dt=header.match_dt,
cutoff=cutoff,
)
+149
View File
@@ -0,0 +1,149 @@
"""切片共享基础:结果类型 / 比赛头信息 / 赛果与统计可用性判定。
context_builder.py 按领域拆出(单文件 slices ),仅做搬迁无逻辑修改
各领域切片见同包 form/h2h/stats/home_away/standings 模块;
聚合入口 build_context aggregate.py;对外统一经 context_builder 再导出
"""
from __future__ import annotations
import logging
from dataclasses import dataclass
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
logger = logging.getLogger(__name__)
def _outcome(home_goals: int, away_goals: int, side: str) -> str:
"""从某队视角看赛果: W/D/L。"""
if home_goals is None or away_goals is None:
return "?"
if side == "home":
return "W" if home_goals > away_goals else ("D" if home_goals == away_goals else "L")
return "W" if away_goals > home_goals else ("D" if away_goals == home_goals else "L")
def _is_stats_available(stats, before) -> bool:
"""检查统计数据在 cutoff 时间是否已可用。
available_at 语义:该条统计对外可被使用的最早时间,
至少不得早于比赛结束用于回测防泄漏
规则:
- before is None(实盘):available_at None 时允许(兼容旧数据)
- before is not None(回测):available_at None 视为不可用(保守)
- available_at > cutoff:不可用(数据在 cutoff 之后才生成)
"""
if before is None:
# 实盘模式:无时间信息时允许(兼容旧数据)
return True
# 回测模式(cutoff 不为 None):
# available_at 为 None → 无法确认是否在 cutoff 前可用,保守视为不可用
if stats.available_at is None:
return False
return stats.available_at <= before
@dataclass
class SliceResult:
"""数据切片的显式结果(替代「靠文案子串猜有无数据」)。
旧实现用 `"无数据" in slice_text` 判断,依赖具体文案 一旦某个切片
写成无比分数据无伤停数据这类变体,判断就会静默失配
(见审查报告 P2-1)这里让切片函数直接声明 `has_data`,不再猜
"""
text: str
has_data: bool
n_records: int = 0
def __str__(self) -> str: # 让老调用点可直接当 str 用
return self.text
@dataclass
class MatchContext:
match_id: int
text: str
has_stats: bool
has_standings: bool
match_dt: object | None = None # 比赛时间(回测防泄漏 + 快照用)
cutoff: object | None = None # 实际使用的数据截止时间(用于落库记录)
@dataclass
class MatchHeader:
"""比赛基础信息(所有 agent 共享)。"""
match_id: int
home_name: str
away_name: str
league_name: str
season: str | None
match_date: str
match_dt: object # 原始 datetime,回测防泄漏用
stage: str | None
home_team_id: int
away_team_id: int
league_id: int
async def _load_match(db, match_id: int) -> Match:
stmt = (
select(Match)
.where(Match.id == match_id)
.options(
selectinload(Match.league),
selectinload(Match.home_team),
selectinload(Match.away_team),
selectinload(Match.stats),
)
)
m = (await db.execute(stmt)).scalar_one_or_none()
if m is None:
raise ValueError(f"match {match_id} not found")
return m
async def load_match_header(match_id: int, db: AsyncSession | None = None) -> MatchHeader:
"""加载比赛头信息(各 agent 共用)。
Args:
match_id: 比赛 ID
db: 可选的共享 session不传则自建(向后兼容)
"""
if db is not None:
m = await _load_match(db, match_id)
return _to_header(m)
async with AsyncSessionLocal() as new_db:
m = await _load_match(new_db, match_id)
return _to_header(m)
def _to_header(m: Match) -> MatchHeader:
return MatchHeader(
match_id=m.id,
home_name=m.home_team.name_zh or m.home_team.name,
away_name=m.away_team.name_zh or m.away_team.name,
league_name=m.league.name if m.league else "?",
season=m.season,
match_date=m.match_date.strftime("%Y-%m-%d %H:%M UTC") if m.match_date else "?",
match_dt=m.match_date,
stage=m.match_stage,
home_team_id=m.home_team_id,
away_team_id=m.away_team_id,
league_id=m.league_id,
)
def header_text(h: MatchHeader) -> str:
stage = f" {h.stage}" if h.stage else ""
return (
f"对阵: {h.home_name} vs {h.away_name} | {h.league_name} {h.season or '?'}{stage} | {h.match_date}"
)
+86
View File
@@ -0,0 +1,86 @@
"""A - 近期状态切片: 近 N 场赛果 / 走势(form)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult, _is_stats_available, _outcome
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def form_slice(header: MatchHeader, *, limit: int = 5, before=None, db: AsyncSession | None = None) -> SliceResult:
"""A - 近期状态切片: 两队近 N 场赛果、关键事件、走势判断。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = []
n_scored = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则客场输球会被算成主场赢球。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
lines.append(f"── {label}近况({name},近 {limit} 场) ──")
if form:
wins = draws = losses = 0
for fm in form:
is_home = (fm.home_team_id == team_id)
side = "home" if is_home else "away"
o = _outcome(fm.home_goals, fm.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
if fm.home_goals is not None:
n_scored += 1
score = f"{fm.home_goals}-{fm.away_goals}" if fm.home_goals is not None else "vs"
xg = ""
if fm.stats and _is_stats_available(fm.stats, before) and fm.stats.home_xg is not None:
own = fm.stats.home_xg if is_home else fm.stats.away_xg
xg = f" (xG {own:.1f})"
opp = fm.away_team.name if is_home else fm.home_team.name
lines.append(f" {o} {score} vs {opp}{xg}")
lines.append(f"{len(form)} 场: {wins}{draws}{losses}")
else:
lines.append(" 无数据")
return SliceResult(text="\n".join(lines), has_data=n_scored > 0, n_records=n_scored)
async def _get_form(db, team_id: int, before, *, limit: int = 5) -> list[Match]:
"""某队近 N 场(已完赛)。before=None 表示不限制(预测赛前的场景由调用方保证)。
必须预加载 stats / home_team / away_team:切片函数会读取这些关系,
async session 下惰性加载会抛 MissingGreenlet
(models.py 已声明 lazy="selectin",此处显式声明以固化查询意图)
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.where((Match.home_team_id == team_id) | (Match.away_team_id == team_id))
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+89
View File
@@ -0,0 +1,89 @@
"""E - 历史交锋切片: 交手史与胜负规律(h2h)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def h2h_slice(header: MatchHeader, *, limit: int = 8, before=None, db: AsyncSession | None = None) -> SliceResult:
"""E - 历史交锋切片: 过去数年 + 近期交手数据,提取交手规律。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
h2h = await _get_h2h(db, header.home_team_id, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
h2h = await _get_h2h(new_db, header.home_team_id, header.away_team_id, before=before, limit=limit)
lines = [f"── 历史交锋(近 {limit} 次) ──"]
n_with_score = 0
if h2h:
# 从当前主队视角统计:判断当前主队在每场交锋中是主是客
current_home_wins = current_home_draws = current_home_losses = 0
for hm in h2h:
d = hm.match_date.strftime("%Y-%m") if hm.match_date else "?"
if hm.home_goals is not None:
n_with_score += 1
# 判断当前主队当时是主队还是客队
if hm.home_team_id == header.home_team_id:
# 当前主队当时是主队
if hm.home_goals > hm.away_goals:
current_home_wins += 1
elif hm.home_goals == hm.away_goals:
current_home_draws += 1
else:
current_home_losses += 1
else:
# 当前主队当时是客队(从客队视角看赛果)
if hm.away_goals > hm.home_goals:
current_home_wins += 1
elif hm.away_goals == hm.home_goals:
current_home_draws += 1
else:
current_home_losses += 1
lines.append(f" {d}: {hm.home_team.name} {hm.home_goals}-{hm.away_goals} {hm.away_team.name}")
else:
lines.append(f" {d}: {hm.home_team.name} vs {hm.away_team.name} (无比分)")
total = current_home_wins + current_home_draws + current_home_losses
if total:
lines.append(
f" 总计 {total} 场(从当前主队 {header.home_name} 视角): "
f"{current_home_wins}{current_home_draws}{current_home_losses}"
)
else:
lines.append(" 无数据")
# has_data 以「有比分的交锋」为准:仅有对阵无比分时不足以支撑分析
return SliceResult(text="\n".join(lines), has_data=n_with_score > 0, n_records=n_with_score)
async def _get_h2h(db, home_id: int, away_id: int, before, *, limit: int = 5) -> list[Match]:
"""两队交锋史。需预加载 home_team / away_team(切片输出队名)。"""
stmt = (
select(Match)
.options(
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.where(
((Match.home_team_id == home_id) & (Match.away_team_id == away_id))
| ((Match.home_team_id == away_id) & (Match.away_team_id == home_id))
)
.order_by(Match.match_date.desc())
.limit(limit)
)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+83
View File
@@ -0,0 +1,83 @@
"""C - 主客因素切片: 主场战绩 vs 客场战绩(home_away)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import Match
from src.llm.slices.common import MatchHeader, SliceResult, _outcome
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def home_away_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""C - 主客因素切片: 主场战绩 vs 客场战绩,评估地理优势影响。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_home = await _get_home_away(db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(db, header.away_team_id, "away", before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_home = await _get_home_away(new_db, header.home_team_id, "home", before=before, limit=limit)
away_away = await _get_home_away(new_db, header.away_team_id, "away", before=before, limit=limit)
lines = ["── 主客因素 ──"]
n_total = 0
for label, name, matches, side in (
("主队主场", header.home_name, home_home, "home"),
("客队客场", header.away_name, away_away, "away"),
):
if matches:
wins = draws = losses = gf = ga = 0
for m in matches:
if m.home_goals is None: continue
o = _outcome(m.home_goals, m.away_goals, side)
if o == "W": wins += 1
elif o == "D": draws += 1
else: losses += 1
gf += m.home_goals if side == "home" else m.away_goals
ga += m.away_goals if side == "home" else m.home_goals
n = wins + draws + losses
n_total += n
if n > 0:
pct = wins / n * 100
lines.append(f" {label} {name}(近 {n} 场): {wins}{draws}{losses}负, 胜率 {pct:.0f}%")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
async def _get_home_away(db, team_id: int, side: str, before, *, limit: int = 10) -> list[Match]:
"""某队主场/客场近 N 场。side='home' 取主场,'away' 取客场。
当前只用标量字段,但统一预加载以免后续扩展时踩坑
"""
stmt = (
select(Match)
.options(
selectinload(Match.stats),
selectinload(Match.home_team),
selectinload(Match.away_team),
)
.where(Match.match_status == "finished")
.where(Match.score_status == "known")
.where(Match.home_goals.is_not(None))
.order_by(Match.match_date.desc())
.limit(limit)
)
if side == "home":
stmt = stmt.where(Match.home_team_id == team_id)
else:
stmt = stmt.where(Match.away_team_id == team_id)
if before is not None:
stmt = stmt.where(Match.match_date < before)
result = await db.execute(stmt)
return list(result.scalars().all())
+91
View File
@@ -0,0 +1,91 @@
"""D - 联赛排名切片: 积分榜快照(支持 cutoff 的历史还原,standings)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from sqlalchemy import select
from sqlalchemy.orm import selectinload
from src.db.base import AsyncSessionLocal
from src.db.models import League, Standing
from src.llm.slices.common import MatchHeader, SliceResult
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def standings_slice(header: MatchHeader, *, before=None, db: AsyncSession | None = None) -> SliceResult:
"""D - 联赛排名切片: 两队积分榜位置、积分、近期走势(form)、分区,评估整体实力差距。
P0-02: 支持 cutoff(before) available_at<=cutoff 的每队最新快照
(DISTINCT ON team_id ORDER available_at DESC);before=None cutoff=now()
回测时可还原历史时刻榜单,不再只是"最新快照、忽略 cutoff"
db: 可选共享 session( context_builder 模块 docstring)
语义区分:
- 两队都有积分榜行 has_data=True(明确的排名信息)
- 任一队缺失 has_data=False(升班马/杯赛无榜,信息不完整时明确声明)
"""
# P0-02: before=None → cutoff=now()(取最新可用快照)
if before is None:
from datetime import datetime, timezone
before = datetime.now(timezone.utc)
if db is not None:
league = (await db.execute(select(League).where(League.id == header.league_id))).scalar_one_or_none()
# P0-02: DISTINCT ON (team_id) 取 available_at<=cutoff 的最新快照
rows = (
(
await db.execute(
select(Standing)
.options(selectinload(Standing.team))
.where(Standing.league_id == header.league_id)
.where(Standing.available_at <= before)
.distinct(Standing.team_id)
.order_by(Standing.team_id, Standing.available_at.desc())
)
)
.scalars()
.all()
if league
else []
)
else:
async with AsyncSessionLocal() as new_db:
return await standings_slice(header, before=before, db=new_db)
lines = [f"── 联赛排名({header.league_name}{len(rows)} 队) ──"]
n_records = 0
def _fmt(row) -> str:
zg = f" xG差 {row.xg_for - row.xg_against:+.1f}" if row.xg_for is not None and row.xg_against is not None else ""
form = f" 近5场 {row.form}" if row.form else ""
zone = f" [{row.zone}]" if row.zone else ""
return (
f"{row.position} 名: {row.points} 分 / {row.played}"
f"({row.won}{row.drawn}{row.lost}负, 进{row.goals_for}{row.goals_against} 净胜{row.goal_diff:+d}"
f"{zg}){form}{zone}"
)
for label, team_id in (("主队", header.home_team_id), ("客队", header.away_team_id)):
row = next((r for r in rows if r.team_id == team_id), None)
if row is None:
lines.append(f" {label}: 暂无积分榜数据(可能杯赛/赛季未开始)")
else:
n_records += 1
lines.append(f" {label} {header.home_name if label == '主队' else header.away_name}:")
lines.append(_fmt(row))
# 两队排名对比摘要
home_row = next((r for r in rows if r.team_id == header.home_team_id), None)
away_row = next((r for r in rows if r.team_id == header.away_team_id), None)
if home_row and away_row:
diff = home_row.position - away_row.position # 正数=主队排名更靠前(名次更小)
lead = f"主队排名高 {diff}" if diff > 0 else (f"客队排名高 {-diff}" if diff < 0 else "两队同排名结构")
pts_diff = home_row.points - away_row.points
lines.append(f" 排名对比: {lead}, 分差 {pts_diff:+d}")
# has_data: 两队都有行才算完整;只有一队时仍有价值,但标记不完整
has_data = n_records >= 1
return SliceResult(text="\n".join(lines), has_data=has_data, n_records=n_records)
+67
View File
@@ -0,0 +1,67 @@
"""B - 攻防数据切片: 进球/射门/控球/xG 聚合(stats)。"""
from __future__ import annotations
from typing import TYPE_CHECKING
from src.db.base import AsyncSessionLocal
from src.llm.slices.common import MatchHeader, SliceResult, _is_stats_available
from src.llm.slices.form import _get_form
if TYPE_CHECKING:
from sqlalchemy.ext.asyncio import AsyncSession
async def stats_slice(header: MatchHeader, *, limit: int = 10, before=None, db: AsyncSession | None = None) -> SliceResult:
"""B - 攻防数据切片: 进球、射门、控球,评估攻防强度。before=match_date 用于回测。
db: 可选共享 session,避免每个切片独立建连( context_builder 模块 docstring)
"""
if db is not None:
home_form = await _get_form(db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(db, header.away_team_id, before=before, limit=limit)
else:
async with AsyncSessionLocal() as new_db:
home_form = await _get_form(new_db, header.home_team_id, before=before, limit=limit)
away_form = await _get_form(new_db, header.away_team_id, before=before, limit=limit)
lines = [f"── 攻防数据(近 {limit} 场) ──"]
n_total = 0
# P0-1 修复:每场历史比赛必须根据「该队当时是主是客」判断 side,
# 不能用本场 side 硬套 —— 否则进球/失球/xG 全部算反。
for label, name, form, team_id in (
("主队", header.home_name, home_form, header.home_team_id),
("客队", header.away_name, away_form, header.away_team_id),
):
if form:
gf = ga = shots = sot = poss = xg = xga = 0
n = n_shots = n_poss = n_xg = 0
for fm in form:
if fm.home_goals is None: continue
is_home = (fm.home_team_id == team_id)
gf += fm.home_goals if is_home else fm.away_goals
ga += fm.away_goals if is_home else fm.home_goals
n += 1
# 只使用 cutoff 之前已可用的统计数据
if fm.stats and _is_stats_available(fm.stats, before):
if fm.stats.home_shots is not None:
shots += fm.stats.home_shots if is_home else fm.stats.away_shots
sot += fm.stats.home_shots_on_target if is_home else fm.stats.away_shots_on_target
n_shots += 1
if fm.stats.home_possession is not None:
poss += fm.stats.home_possession if is_home else (100 - fm.stats.home_possession)
n_poss += 1
if fm.stats.home_xg is not None:
xg += fm.stats.home_xg if is_home else fm.stats.away_xg
xga += fm.stats.away_xg if is_home else fm.stats.home_xg
n_xg += 1
n_total += n
if n > 0:
lines.append(f" {label} {name}:")
lines.append(f" 场均进球 {gf/n:.2f}, 场均失球 {ga/n:.2f}")
if n_shots: lines.append(f" 场均射门 {shots/n_shots:.1f}, 射正 {sot/n_shots:.1f}")
if n_poss: lines.append(f" 平均控球 {poss/n_poss:.1f}%")
if n_xg: lines.append(f" 场均 xG {xg/n_xg:.2f}, 场均被 xG {xga/n_xg:.2f}")
else:
lines.append(f" {label} {name}: 无比分数据")
else:
lines.append(f" {label} {name}: 无数据")
return SliceResult(text="\n".join(lines), has_data=n_total > 0, n_records=n_total)
+1 -9
View File
@@ -3,17 +3,9 @@ from __future__ import annotations
def actual_1x2(home: int, away: int) -> str:
"""实际比分 → 胜平负
单一权威源: backtest.py eval.py 共用,避免重复定义
"""
"""实际比分 → 胜平负(单一权威源:backtest.py 与 eval.py 共用)。"""
if home > away:
return "1"
if home == away:
return "X"
return "2"
def is_correct_1x2(pred: str | None, actual: str) -> bool:
"""预测是否命中胜平负。"""
return pred == actual
+6 -6
View File
@@ -61,7 +61,7 @@ class TestOrchestratorWritesAgentWeights:
@pytest.mark.asyncio
async def test_orchestrator_writes_agent_weights_to_upsert(self):
"""orchestrator 应将 agent_weights 传入 _upsert_prediction"""
"""orchestrator 应将 agent_weights 传入 _insert_or_find_by_fingerprint"""
from src.llm.agents import orchestrator as orch_mod
from src.llm.agents.base import AgentReport
from src.llm.context_builder import MatchHeader
@@ -103,8 +103,8 @@ class TestOrchestratorWritesAgentWeights:
"agent_weights": {"form": 0.3, "home_away": 0.5, "stats": 0.2},
}, 100, 50
async def mock_upsert(session, **kw):
captured_values.update(kw.get("values", {}))
async def mock_upsert(session, *, values):
captured_values.update(values)
p = MagicMock()
p.id = 1
p.provider = "test"
@@ -116,7 +116,7 @@ class TestOrchestratorWritesAgentWeights:
p.subjective_confidence = 0.7
p.reasoning = "test"
p.agent_outputs = []
p.agent_weights = kw["values"].get("agent_weights")
p.agent_weights = values.get("agent_weights")
return p
class FakeUow:
@@ -130,14 +130,14 @@ class TestOrchestratorWritesAgentWeights:
with patch.object(orch_mod, "run_specialists", mock_specialists), \
patch.object(orch_mod, "_agent_provider", mock_provider), \
patch.object(orch_mod, "load_match_header", mock_header), \
patch.object(orch_mod, "_upsert_prediction", mock_upsert), \
patch.object(orch_mod, "_insert_or_find_by_fingerprint", mock_upsert), \
patch.object(orch_mod, "run_aggregator", mock_aggregator), \
patch.object(orch_mod, "get_uow", FakeUow):
result = await orch_mod.predict_match_multi(999)
# 断言 agent_weights 被写入
assert "agent_weights" in captured_values, "agent_weights 应传入 _upsert_prediction"
assert "agent_weights" in captured_values, "agent_weights 应传入 _insert_or_find_by_fingerprint"
assert captured_values["agent_weights"] is not None, "agent_weights 不应为 None"
assert "form" in captured_values["agent_weights"], "agent_weights 应包含专家权重"
print(f"PASS: agent_weights = {captured_values['agent_weights']}")
+4 -4
View File
@@ -79,18 +79,18 @@ class TestWriteBufferStrategy:
def test_bzzoirot_new_match_available_at_is_two_hours_after_kickoff(self):
"""bzzoiro 新建比赛(stats 回填创建 MatchStats)时 available_at 应为开球 + 2 小时。"""
import inspect
from src.data import bzzoiro
from src.data import bzzoiro_stats
source = inspect.getsource(bzzoiro)
source = inspect.getsource(bzzoiro_stats)
assert 'timedelta(hours=2)' in source, \
"bzzoiro 应使用 match_date + timedelta(hours=2) 作为 available_at"
def test_bzzoirot_multiple_writes_use_two_hour_buffer(self):
"""bzzoiro 多处写入(创建/更新)都应使用 2 小时缓冲。"""
import inspect
from src.data import bzzoiro
from src.data import bzzoiro_stats
source = inspect.getsource(bzzoiro)
source = inspect.getsource(bzzoiro_stats)
count = source.count('timedelta(hours=2)')
assert count >= 2, f"期望至少 2 处 timedelta(hours=2),实际 {count}"
+29 -2
View File
@@ -5,6 +5,7 @@
"""
from __future__ import annotations
from types import SimpleNamespace
from unittest.mock import patch
import pytest
@@ -12,6 +13,24 @@ import pytest
from src.llm.baseline import _avg_goals, predict_baseline
class _FakeUoW:
"""P3-2:baseline 在服务层落库,测试需 mock get_uow。"""
async def __aenter__(self):
return SimpleNamespace(
execute=lambda *a, **k: SimpleNamespace(scalar_one_or_none=lambda: None),
add=lambda *a, **k: None,
flush=lambda *a, **k: None,
)
async def __aexit__(self, *a):
return None
async def _fake_upsert(session, **kw):
return SimpleNamespace(id=1)
@pytest.mark.asyncio
async def test_avg_goals_no_data_returns_zero():
"""无历史数据时场均进球为 0(不抛异常)。"""
@@ -68,7 +87,9 @@ async def test_predict_baseline_no_llm():
match_status = "scheduled"
with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", _FakeUoW), \
patch("src.llm.baseline._insert_or_find_by_fingerprint", _fake_upsert):
class FakeSession:
async def get(self, cls, mid):
return FakeMatch()
@@ -93,6 +114,8 @@ async def test_predict_baseline_no_llm():
assert result.pred_1x2 == "X"
assert result.subjective_confidence == 0.5
assert "非投注建议" in result.reasoning
# P3-2:服务层落库,回填真实 prediction_id
assert result.prediction_id == 1
# 确认未调用任何 LLM 相关模块
assert "home_10" in captured and "away_20" in captured
@@ -112,7 +135,9 @@ async def test_predict_baseline_clamps_to_range():
match_status = "scheduled"
with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", _FakeUoW), \
patch("src.llm.baseline._insert_or_find_by_fingerprint", _fake_upsert):
class FakeSession:
async def get(self, cls, mid):
return FakeMatch()
@@ -128,3 +153,5 @@ async def test_predict_baseline_clamps_to_range():
assert result.pred_home_goals == 10.0 # clamped
assert result.pred_away_goals == 0.0 # clamped
assert result.pred_1x2 == "1" # 10:0 主胜
# P3-2:服务层落库,回填真实 prediction_id
assert result.prediction_id == 1
+128 -45
View File
@@ -62,12 +62,35 @@ async def test_predict_baseline_returns_predict_result():
async def __aexit__(self, *a):
return None
# P3-2:baseline 在服务层落库(get_uow + _insert_or_find_by_fingerprint),需 mock 掉。
class FakeUoW:
async def __aenter__(self):
return _make_session()
async def __aexit__(self, *a):
return None
captured = {}
async def fake_upsert(session, *, values):
captured.update(values)
return SimpleNamespace(id=77)
# baseline.py 内部 from-import get_uow / _insert_or_find_by_fingerprint,需 patch 真实来源模块。
with patch("src.llm.baseline._avg_goals", fake_avg), \
patch("src.llm.baseline.AsyncSessionLocal") as SLC:
patch("src.llm.baseline.AsyncSessionLocal") as SLC, \
patch("src.db.unit_of_work.get_uow", FakeUoW), \
patch("src.llm.baseline._insert_or_find_by_fingerprint", fake_upsert):
SLC.return_value = FakeCM()
result = await predict_baseline(1)
# P3-2:验证服务层落库被调用且属性映射正确
assert captured["match_id"] == 1
assert captured["provider"] == "baseline"
assert captured["run_type"] == "live"
assert captured["pred_home_goals"] == 2.0
assert isinstance(result, PredictResult)
assert result.mode == "baseline"
assert result.provider == "baseline"
@@ -144,15 +167,31 @@ def test_predict_route_has_no_dict_branch():
# ============================================================
# 4. _persist_baseline 属性映射(baseline 落库语义不变)
# 4. P3-2:baseline 服务层落库属性映射(落库已从路由移到 baseline.py)
# ============================================================
class _FakeResult:
"""支持 .scalar_one_or_none() 的最小假结果集。"""
def __init__(self, items):
self._items = list(items)
def scalars(self):
return self
def all(self):
return self._items
def scalar_one_or_none(self):
return self._items[0] if self._items else None
class _FakeUoW:
"""替代 get_uow 的最小上下文管理器。"""
"""替代 get_uow 的最小上下文管理器(session.execute 是 async 的)"""
def __init__(self):
self.session = SimpleNamespace()
self.session = _make_session()
async def __aenter__(self):
return self.session
@@ -160,57 +199,101 @@ class _FakeUoW:
async def __aexit__(self, *a):
return None
def __call__(self):
return self
def _make_session(existing=None):
"""构造带 async execute / add / flush 的假 session。"""
sess = SimpleNamespace()
async def execute(*a, **k):
return _FakeResult(existing or [])
sess.execute = execute
sess.add = lambda *a, **k: None
async def flush(*a, **k):
return None
sess.flush = flush
return sess
@pytest.mark.asyncio
async def test_persist_baseline_maps_attributes(monkeypatch):
async def test_baseline_service_persists_with_correct_attributes(monkeypatch):
"""P3-2:baseline 在服务层(predict_baseline)落库,属性映射与路由旧版一致。"""
captured = {}
async def fake_upsert(session, **kwargs):
captured.update(kwargs)
async def fake_upsert(session, *, values):
captured.update(values)
return SimpleNamespace(id=77)
class FakeMatch:
id = 1
home_team_id = 10
away_team_id = 20
league_id = 1
match_status = "scheduled"
class FakeSession:
async def get(self, cls, mid):
return FakeMatch()
class FakeSLC:
async def __aenter__(self):
return FakeSession()
async def __aexit__(self, *a):
return None
async def fake_avg(db, *, team_id, side, league_id, before):
return 2.0 if side == "home" else 1.0
monkeypatch.setattr("src.llm.baseline._avg_goals", fake_avg)
monkeypatch.setattr("src.llm.baseline.AsyncSessionLocal", FakeSLC)
monkeypatch.setattr("src.db.unit_of_work.get_uow", lambda: _FakeUoW())
monkeypatch.setattr("src.llm.predict._upsert_prediction", fake_upsert)
# baseline.py 模块级 import _insert_or_find_by_fingerprint(第 15 行),需 patch baseline 模块属性
monkeypatch.setattr("src.llm.baseline._insert_or_find_by_fingerprint", fake_upsert)
from src.api.routes.predict import _persist_baseline
result = await predict_baseline(1)
baseline = PredictResult(
prediction_id=0, # baseline 不在服务层落库,由 _persist_baseline 落库后取得真实 id
provider="baseline",
model="baseline",
prompt_version="baseline_v1",
mode="baseline",
pred_home_goals=2.0,
pred_away_goals=1.0,
alt_pred_home_goals=None,
alt_pred_away_goals=None,
pred_1x2="1",
subjective_confidence=0.5,
reasoning="r",
context="",
status="success",
latency_ms=0,
prompt_tokens=0,
completion_tokens=0,
raw={"home_avg": 2.1, "away_avg": 1.4},
)
pid = await _persist_baseline(1, baseline)
assert pid == 77
# 落库被调用且属性映射正确
assert captured, f"predict_baseline 应调用 _insert_or_find_by_fingerprint 落库,但 captured 为空(result.prediction_id={result.prediction_id!r})"
assert captured["match_id"] == 1
assert captured["provider_name"] == "baseline"
assert captured["provider"] == "baseline"
assert captured["model"] == "baseline"
assert captured["mode"] == "baseline"
assert captured["run_type"] == "live"
v = captured["values"]
assert v["prompt_version"] == "baseline_v1"
assert v["pred_home_goals"] == 2.0
assert v["pred_away_goals"] == 1.0
assert v["pred_1x2"] == "1"
assert v["subjective_confidence"] == 0.5
assert v["prompt_tokens"] == 0
assert v["completion_tokens"] == 0
assert v["latency_ms"] == 0
assert v["raw_response"] == {"home_avg": 2.1, "away_avg": 1.4}
assert v["status"] == "success"
assert captured["prompt_version"] == "baseline_v1"
assert captured["pred_home_goals"] == 2.0
assert captured["pred_away_goals"] == 1.0
assert captured["pred_1x2"] == "1"
assert captured["subjective_confidence"] == 0.5
assert captured["prompt_tokens"] == 0
assert captured["completion_tokens"] == 0
assert captured["latency_ms"] == 0
assert captured["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
assert captured["status"] == "success"
# 回填真实 prediction_id(服务层落库后取得)
assert result.prediction_id == 77
assert result.pred_1x2 == "1"
assert captured["match_id"] == 1
assert captured["provider"] == "baseline"
assert captured["model"] == "baseline"
assert captured["mode"] == "baseline"
assert captured["run_type"] == "live"
assert captured["prompt_version"] == "baseline_v1"
assert captured["pred_home_goals"] == 2.0
assert captured["pred_away_goals"] == 1.0
assert captured["pred_1x2"] == "1"
assert captured["subjective_confidence"] == 0.5
assert captured["prompt_tokens"] == 0
assert captured["completion_tokens"] == 0
assert captured["latency_ms"] == 0
assert captured["raw_response"] == {"home_avg": 2.0, "away_avg": 1.0}
assert captured["status"] == "success"
# 回填真实 prediction_id(服务层落库后取得)
assert result.prediction_id == 77
+66
View File
@@ -0,0 +1,66 @@
"""数据质量检查回归测试。
背景(P0-02 遗留): MatchStats 主键改为 match_id ,质量检查查询仍引用
MatchStats.id AttributeError POST /admin/data-quality/run 必然 500,
前端显示运行失败; handler 无日志,未捕获异常走 uvicorn.error
(propagate=False)不进内存缓冲/文件日志,排障时无据可查
"""
from __future__ import annotations
from unittest.mock import AsyncMock, MagicMock
from src.api.routes import admin_quality
def _fake_db(scalars: list[int]) -> MagicMock:
"""按顺序返回 scalar() 计数的假 AsyncSession。"""
db = MagicMock()
results = []
for v in scalars:
r = MagicMock()
r.scalar.return_value = v
results.append(r)
db.execute = AsyncMock(side_effect=results)
db.add = MagicMock()
db.commit = AsyncMock()
return db
async def test_run_data_quality_check_no_attribute_error():
"""检查查询不得引用 MatchStats.id(P0-02 后该属性不存在)。
修复前: run_data_quality_check AttributeError 500
"""
db = _fake_db([3, 1])
out = await admin_quality.run_data_quality_check(db)
assert out["ok"] is True
assert out["checks"] == [
{"name": "finished_without_stats", "passed": False},
{"name": "league_without_standings", "passed": False},
]
# 检查结果落库(2 条 DataQualityCheck)
assert db.add.call_count == 2
db.commit.assert_awaited_once()
async def test_run_data_quality_check_all_passed():
db = _fake_db([0, 0])
out = await admin_quality.run_data_quality_check(db)
assert out["ok"] is True
assert all(c["passed"] for c in out["checks"])
async def test_run_data_quality_check_logs_summary(caplog):
"""成功路径必须留日志:否则线上无从得知检查何时跑过、结果如何。"""
import logging
db = _fake_db([0, 0])
with caplog.at_level(logging.INFO, logger="src.api.routes.admin_quality"):
await admin_quality.run_data_quality_check(db)
assert any("数据质量检查" in r.message for r in caplog.records)
+69 -4
View File
@@ -20,7 +20,7 @@ from datetime import date, datetime, timezone
import pytest
import src.data.bzzoiro as bz
from src.db.models import DataLineage, League, Match, RawEvent, Team
from src.db.models import DataLineage, League, Match, RawEvent, Team, TeamAlias
def _event(eid=1001, status="finished", home="Arsenal", away="Chelsea", hs=2, as_=1):
@@ -71,12 +71,68 @@ class _FakeDB:
League: list(leagues),
RawEvent: list(raw_events),
}
self._next_id = 0
self._teams_by_id: dict[int, Team] = {t.id: t for t in teams if getattr(t, "id", None)}
self._teams_by_name: dict[str, Team] = {t.name: t for t in teams if getattr(t, "id", None)}
self._aliases: dict[str, TeamAlias] = {}
self._next_id = max((t.id for t in teams if getattr(t, "id", None)), default=0)
def add(self, obj):
self.added.append(obj)
async def get(self, cls, key):
if cls is Team:
return self._teams_by_id.get(key)
if cls is TeamAlias:
return self._aliases.get(key)
return None
async def execute(self, stmt):
# P1-K:LeagueRepository/TeamRepository.get_or_create 使用 pg_insert(Insert 语句,无 column_descriptions)
if not hasattr(stmt, "column_descriptions"):
# 解析 INSERT 值并注册到 _by_entity,使二次查询可命中
try:
values = stmt.compile().params
except Exception:
values = {}
table = getattr(stmt, "table", None)
table_name = getattr(table, "name", None) if table is not None else None
entity_map = {"leagues": League, "teams": Team}
entity = entity_map.get(table_name)
if entity is not None:
obj = entity()
for k, v in values.items():
if k in ("code", "name", "country", "name_zh"):
setattr(obj, k, v)
if getattr(obj, "id", None) is None:
self._next_id += 1
obj.id = self._next_id
if entity not in self._by_entity:
self._by_entity[entity] = []
self._by_entity[entity].append(obj)
if entity is Team:
self._teams_by_id[obj.id] = obj
self._teams_by_name[obj.name] = obj
class _Empty:
def scalar_one_or_none(self_inner):
return None
def scalars(self_inner):
return self_inner
def all(self_inner):
return []
return _Empty()
# P1-K:Team 查询回退到内存映射 name(避免队列/实体分发不完全匹配)
try:
params = stmt.compile().params
except Exception:
params = {}
name_val = next((v for k, v in params.items() if k.startswith("name")), None)
if isinstance(name_val, str) and name_val:
team = self._teams_by_name.get(name_val)
if team:
return _FakeResult([team])
entities = set()
for d in (stmt.column_descriptions or []):
entities.add(d.get("entity") or d.get("type"))
@@ -85,11 +141,17 @@ class _FakeDB:
return _FakeResult(items)
return _FakeResult([])
@staticmethod
def _filter(entity, items, stmt):
return items
async def flush(self):
for obj in self.added:
if getattr(obj, "id", None) is None:
self._next_id += 1
obj.id = self._next_id
if isinstance(obj, Team) and obj.id is not None:
self._teams_by_id[obj.id] = obj
@pytest.fixture(autouse=True)
@@ -244,11 +306,14 @@ class TestEventsBronzeOnUpdate:
class TestEventsBronzeIsBestEffort:
async def test_bronze_write_failure_does_not_break_ingest(self, monkeypatch):
import src.data.bzzoiro_events as bz_events
async def _boom(*args, **kwargs):
raise RuntimeError("infra down")
monkeypatch.setattr(bz, "_write_raw_event", _boom)
monkeypatch.setattr(bz, "_write_lineage", _boom)
# Bronze 写入助手直接 import 到 bzzoiro_events 命名空间,需 patch 该处
monkeypatch.setattr("src.data.pipeline_write._write_raw_event", _boom)
monkeypatch.setattr("src.data.pipeline_write._write_lineage", _boom)
_patch_fetch(monkeypatch, [_event()])
db = _FakeDB()

Some files were not shown because too many files have changed in this diff Show More