docs+feat: matches 唯一键语义明确化 + source_event_id partial unique
docs/05-data.md:业务唯一=同联赛同主客同自然天; source_event_id 用于统计回填与血缘,新增部分唯一索引说明与 upsert 查找顺序。 新增 ix_matches_source_event_id_unique(WHERE IS NOT NULL), 兼容存量空值历史行;MatchRepository.find_by_source_event_id; events upsert 优先按 event_id 定位,回退自然键。 迁移 0021 + 回归测试修复(find_by_source_event_id 方法调用误判)。
This commit is contained in:
@@ -0,0 +1,38 @@
|
||||
"""matches.source_event_id 部分唯一索引
|
||||
|
||||
业务唯一键:同联赛同主客同自然天一条(ix_matches_unique,既有)。
|
||||
source_event_id 是上游 bzzoiro 的比赛 id,用于统计回填与血缘追踪;
|
||||
当它非空时应全局唯一(同一 upstream 比赛只对应一行 matches),
|
||||
避免同一场比赛因自然键天级舍入差异产生重复。
|
||||
|
||||
partial unique(WHERE source_event_id IS NOT NULL):
|
||||
- 兼容存量空 source_event_id 的历史行(不强制回填);
|
||||
- 新采集行均带 source_event_id,从此具备 upstream 唯一性。
|
||||
|
||||
Revision ID: 0021_match_source_event_id_unique
|
||||
Revises: 0020_team_aliases
|
||||
Create Date: 2026-09-22
|
||||
"""
|
||||
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
|
||||
revision: str = '0021_match_source_event_id_unique'
|
||||
down_revision: Union[str, None] = '0020_team_aliases'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_index(
|
||||
'ix_matches_source_event_id_unique',
|
||||
'matches',
|
||||
['source_event_id'],
|
||||
unique=True,
|
||||
postgresql_where=op.text('source_event_id IS NOT NULL'),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index('ix_matches_source_event_id_unique', table_name='matches')
|
||||
+17
-3
@@ -218,11 +218,25 @@ CREATE TABLE predictions (
|
||||
|
||||
1. **`match_date_date`(天级日期)**: 用于天级去重。bzzoiro 返回的时间带时分秒,精确匹配不可靠,故拆出 `DATE` 列做唯一键。
|
||||
|
||||
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。
|
||||
2. **`ix_matches_unique`**: `(league_id, home_team_id, away_team_id, match_date_date)` 唯一,保证同一场比赛重复采集时 upsert 而非插入重复行。**业务唯一:同联赛同主客同自然天一条。**
|
||||
|
||||
3. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测。
|
||||
3. **`source_event_id` 部分唯一**: `ix_matches_source_event_id_unique`(WHERE source_event_id IS NOT NULL)——上游 bzzoiro 的比赛 id,当非空时全局唯一。作用:
|
||||
- 统计回填(`/events/{id}/stats/`)与 Bronze 血缘(/events/ 采集)通过它定位比赛,不依赖自然键天级舍入;
|
||||
- 新采集行均带此 id,避免同一 upstream 比赛因时间戳差异绕开自然键产生重复。
|
||||
- 存量空 source_event_id 历史行不受影响(不强制回填)。
|
||||
|
||||
4. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比。
|
||||
4. **`predictions` 级联删除**: `ON DELETE CASCADE`,删比赛时自动清其预测。
|
||||
|
||||
5. **`mode` + `prompt_version`**: `single` 模式存 `v1`/`v2`,`multi` 模式存 `multi_v1`/`multi_v2`,eval summary 按这两列天然分组对比。
|
||||
|
||||
## 采集 upsert 查找顺序
|
||||
|
||||
events 管线按以下优先级定位已有比赛,命中即复用(更新):
|
||||
|
||||
1. **`source_event_id`**(upstream event id,唯一索引命中)——最精确,跨自然键舍入差异;
|
||||
2. **自然键**:`(league_id, home_team_id, away_team_id, match_date_date)`(内存去重,覆盖无 event id 的采集)。
|
||||
|
||||
两者都未命中 → insert 新比赛。
|
||||
|
||||
## 入库语义(幂等)
|
||||
|
||||
|
||||
@@ -181,9 +181,16 @@ class BzzoiroSource:
|
||||
away_team_id = away.id
|
||||
team_name_to_id[nm.away_team] = away_team_id
|
||||
|
||||
# 查找已有比赛: 内存查找
|
||||
match_key = _match_key(home_team_id, away_team_id, nm.date)
|
||||
existing_match = existing_matches.get(match_key)
|
||||
# 查找已有比赛:优先按 upstream event_id 定位(命中即唯一),
|
||||
# 否则回退自然键(联赛+主客+天级日期)内存查找。
|
||||
# source_event_id 上有 partial unique 索引保障 upstream 唯一。
|
||||
eid = _to_int_or_none(raw.get("id"))
|
||||
existing_match = None
|
||||
if eid is not None:
|
||||
existing_match = await match_r.find_by_source_event_id(eid)
|
||||
if existing_match is None:
|
||||
match_key = _match_key(home_team_id, away_team_id, nm.date)
|
||||
existing_match = existing_matches.get(match_key)
|
||||
|
||||
if existing_match is None:
|
||||
m = Match(
|
||||
|
||||
@@ -80,6 +80,20 @@ class MatchRepository:
|
||||
)
|
||||
return (await self._session.execute(stmt)).scalars().all()
|
||||
|
||||
async def find_by_source_event_id(self, source_event_id: int) -> Match | None:
|
||||
"""按上游 event id 查找比赛(唯一命中,用于 upsert 优先路径)。
|
||||
|
||||
source_event_id 上有 partial unique 索引(WHERE IS NOT NULL),
|
||||
同联赛同主客同天(自然键)与上游 event_id 共同保障同一场比赛
|
||||
重复采集时 upsert 而非插入重复行。
|
||||
"""
|
||||
stmt = (
|
||||
select(Match)
|
||||
.options(selectinload(Match.stats))
|
||||
.where(Match.source_event_id == source_event_id)
|
||||
)
|
||||
return (await self._session.execute(stmt)).scalar_one_or_none()
|
||||
|
||||
async def find_finished_with_stats(self, league_ids: list[int], *, limit: int) -> list[Match]:
|
||||
"""已完赛且有上游 event id 的比赛(按日期倒序),供统计回填逐场拉取。
|
||||
|
||||
|
||||
@@ -165,7 +165,9 @@ class TestBzzoiroLineage:
|
||||
if re.search(r"source_event_id\s*(?:is|==|!=)", stripped):
|
||||
continue
|
||||
if re.search(r"source_event_id\s*\.\s*\w+\s*\(", stripped):
|
||||
continue # 方法调用,不是赋值
|
||||
continue # 方法调用(obj.source_event_id(...)),不是赋值
|
||||
if re.search(r"\w*source_event_id\s*\(", stripped):
|
||||
continue # 方法调用(如 find_by_source_event_id(eid)),不是赋值
|
||||
if self._ASSIGN_DIRECT.search(stripped):
|
||||
continue # 直接取配对 raw
|
||||
m_var = self._ASSIGN_VIA_VAR.search(stripped)
|
||||
|
||||
Reference in New Issue
Block a user