列表接口按行发查询是从阶段 3 一路带过来的写法:`Promise.all(rows.map(...))` 看着是并发的,但每行都往库里打一次,行数一多就是几百上千次往返。同样的模式 也散在几个后台批处理和写入路径里。全部改成「先收集 id,一条 inArray/group by 查回来建 Map」——`routes/problem.ts` 的 getProblemTags 早就是这么写的,这次 只是把剩下的地方对齐。 用户可见的列表: - `GET /problem-sets` 每行 5 条(题目数/我的进度/奖章/已获奖章/创建者), limit 上限 250 就是 1250 次往返。改成固定 5 条,与行数无关。 - `GET /contests` 每场比赛一条 creator 查询。后台的比赛列表本来就是 join 出来的,只有这条公开列表漏了。 - `GET /admin/problems`、`GET /admin/contests/:id/problems` 每题一条标签查询。 - `GET /admin/problem-sets` 每行 3 条;`.../badges` 每个奖章一条 count。 后台批处理: - `refreshContestJoinedForAll` 原来是每个用户 1 条 count 加一个独立事务里的 insert/select for update/update。改成一条 group by 出全部用户的场次,再分批 upsert,`metrics || excluded.metrics` 是 jsonb 浅合并,只覆盖 contest_joined 一个键,其余指标原样保留 —— 合并在一条语句里完成,for update 那把锁不再需要。 - `rescanAchievement` 补发循环、`unlockAchievements`:命中的一次插完, onConflictDoNothing 的 returning 就是真新解锁的那批,unlockCount 改成一次 +N。 - `resyncProgress` 逐行 UPDATE 改成一条,completed 用 least() 夹住。 写入路径: - 标签解析抽出 normalizeTagNames + findTagsByName(一条 lower(name) IN), 新建题、改题、批量打标签三条路共用。 - 克隆比赛:题面一条 INSERT、标签一条 SELECT 加一条 INSERT。新旧题的对应 关系靠 _id 认,不依赖 returning 的行序。 - `POST /admin/website` 8 个键一条多行 upsert。 - 题单奖章判定一次插完。 `/ai/duration`:原来每个时间桶两条查询、桶之间还串行,一年 12 个桶 24 次往返。 改成先算桶、再一条查询把整段区间拉回来在内存里分桶。时间戳用 `extract(epoch) * 1000` 取毫秒回来比,别指望 Date.parse 认 pg 那个 `2026-08-12 00:00:00+00` 格式。**相邻桶首尾相接、两端闭区间**(落在边界上的 提交两个桶都算)这条旧语义是照搬的,不要顺手改成半开区间。 验证:本机起 dev 栈,造了覆盖各分支的种子数据(创建者重复的题单、零题目/零 奖章的题单、completed > total 的脏进度、除不尽的百分比、大小写混写的已有标签、 带/不带标签的比赛题、正好落在分桶边界上的提交),旧代码跑一遍、新代码跑一遍: - 51 个接口响应逐字节一致 - 12 张表的快照逐行一致(唯一差别是 progress_detail 里的 submit_time 墙钟值) - 打开 log_statement=all 数过条数,例如 `GET /admin/problems` 20 道题 24 → 5,`GET /problem-sets` 28 → 8,`/ai/duration` years:1 28 → 5, 202 个用户的成就补发 1828 → 614 一处可观察的行为变化:补发现在整批共用一个 unlockTime,原来是每人一个 new Date()。rescanAchievement 上方的注释本来就写着补发会给几百人盖同一个 时间戳、前端据此只显示「已获得」不显示日期,所以这个方向是对的。 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
f00c941ede
commit
a75c70c82d
9 files changed
+392
-247
No files matched your search
@@ -55,11 +55,14 @@ function progressSummary(progress: typeof schema.problemsetProgress.$inferSelect
|
||||
}
|
||||
}
|
||||
|
||||
async function problemSetCreator(id: number) {
|
||||
const [row] = await db.select({ id: schema.user.id, username: schema.user.username, realName: schema.userProfile.realName })
|
||||
async function problemSetCreators(ids: number[]) {
|
||||
const map = new Map<number, ReturnType<typeof sampleUser>>()
|
||||
if (ids.length === 0) return map
|
||||
const rows = await db.select({ id: schema.user.id, username: schema.user.username, realName: schema.userProfile.realName })
|
||||
.from(schema.user).leftJoin(schema.userProfile, eq(schema.userProfile.userId, schema.user.id))
|
||||
.where(eq(schema.user.id, id)).limit(1)
|
||||
return sampleUser(row ?? { id, username: "" }, row?.realName)
|
||||
.where(inArray(schema.user.id, ids))
|
||||
for (const row of rows) map.set(row.id, sampleUser(row, row.realName))
|
||||
return map
|
||||
}
|
||||
|
||||
function badgeData(badge: typeof schema.problemsetBadge.$inferSelect, earned?: boolean) {
|
||||
@@ -75,35 +78,58 @@ function badgeData(badge: typeof schema.problemsetBadge.$inferSelect, earned?: b
|
||||
})
|
||||
}
|
||||
|
||||
async function serializeProblemSet(
|
||||
row: ProblemSetRow,
|
||||
/**
|
||||
* 一次把整页题单的附属数据全查回来,再在内存里按 problemsetId 分组。
|
||||
*
|
||||
* 以前是每行 5 条查询(题目数 / 我的进度 / 奖章 / 已获奖章 / 创建者),
|
||||
* limit 最大 250 就是 1250 次往返。这里固定 5 条,与行数无关。
|
||||
*/
|
||||
async function serializeProblemSets(
|
||||
rows: ProblemSetRow[],
|
||||
userId?: number,
|
||||
includeBadges = false,
|
||||
) {
|
||||
const [[problemCount], [progress], badges, earnedRows] = await Promise.all([
|
||||
db.select({ value: count() }).from(schema.problemsetProblem).where(eq(schema.problemsetProblem.problemsetId, row.id)),
|
||||
userId ? db.select().from(schema.problemsetProgress).where(and(eq(schema.problemsetProgress.problemsetId, row.id), eq(schema.problemsetProgress.userId, userId))).limit(1) : Promise.resolve([]),
|
||||
includeBadges ? db.select().from(schema.problemsetBadge).where(eq(schema.problemsetBadge.problemsetId, row.id)) : Promise.resolve([]),
|
||||
if (rows.length === 0) return []
|
||||
const ids = rows.map((row) => row.id)
|
||||
const [problemCounts, progresses, badges, earnedRows, creators] = await Promise.all([
|
||||
db.select({ problemsetId: schema.problemsetProblem.problemsetId, value: count() })
|
||||
.from(schema.problemsetProblem).where(inArray(schema.problemsetProblem.problemsetId, ids))
|
||||
.groupBy(schema.problemsetProblem.problemsetId),
|
||||
userId ? db.select().from(schema.problemsetProgress)
|
||||
.where(and(inArray(schema.problemsetProgress.problemsetId, ids), eq(schema.problemsetProgress.userId, userId)))
|
||||
: Promise.resolve([] as (typeof schema.problemsetProgress.$inferSelect)[]),
|
||||
includeBadges ? db.select().from(schema.problemsetBadge)
|
||||
.where(inArray(schema.problemsetBadge.problemsetId, ids)).orderBy(asc(schema.problemsetBadge.id))
|
||||
: Promise.resolve([] as (typeof schema.problemsetBadge.$inferSelect)[]),
|
||||
includeBadges && userId ? db.select({ id: schema.userBadge.badgeId }).from(schema.userBadge)
|
||||
.innerJoin(schema.problemsetBadge, eq(schema.userBadge.badgeId, schema.problemsetBadge.id))
|
||||
.where(and(eq(schema.userBadge.userId, userId), eq(schema.problemsetBadge.problemsetId, row.id))) : Promise.resolve([]),
|
||||
.where(and(eq(schema.userBadge.userId, userId), inArray(schema.problemsetBadge.problemsetId, ids)))
|
||||
: Promise.resolve([] as { id: number }[]),
|
||||
problemSetCreators([...new Set(rows.map((row) => row.createdById))]),
|
||||
])
|
||||
const countBySet = new Map(problemCounts.map((item) => [item.problemsetId, item.value]))
|
||||
const progressBySet = new Map(progresses.map((item) => [item.problemsetId, item]))
|
||||
const badgesBySet = new Map<number, (typeof schema.problemsetBadge.$inferSelect)[]>()
|
||||
for (const badge of badges) badgesBySet.set(badge.problemsetId, [...(badgesBySet.get(badge.problemsetId) ?? []), badge])
|
||||
const earned = new Set(earnedRows.map((item) => item.id))
|
||||
return problemSetSchema.parse({
|
||||
id: row.id,
|
||||
title: row.title,
|
||||
description: row.description,
|
||||
createdBy: await problemSetCreator(row.createdById),
|
||||
createTime: row.createTime,
|
||||
lastUpdateTime: row.lastUpdateTime,
|
||||
difficulty: row.difficulty,
|
||||
status: row.status,
|
||||
endTime: row.endTime,
|
||||
visible: row.visible,
|
||||
problemsCount: problemCount?.value ?? 0,
|
||||
completedCount: progress?.completedProblemsCount ?? 0,
|
||||
userProgress: progressSummary(progress),
|
||||
badges: includeBadges ? badges.map((badge) => badgeData(badge, earned.has(badge.id))) : undefined,
|
||||
return rows.map((row) => {
|
||||
const progress = progressBySet.get(row.id)
|
||||
return problemSetSchema.parse({
|
||||
id: row.id,
|
||||
title: row.title,
|
||||
description: row.description,
|
||||
createdBy: creators.get(row.createdById) ?? sampleUser({ id: row.createdById, username: "" }, null),
|
||||
createTime: row.createTime,
|
||||
lastUpdateTime: row.lastUpdateTime,
|
||||
difficulty: row.difficulty,
|
||||
status: row.status,
|
||||
endTime: row.endTime,
|
||||
visible: row.visible,
|
||||
problemsCount: countBySet.get(row.id) ?? 0,
|
||||
completedCount: progress?.completedProblemsCount ?? 0,
|
||||
userProgress: progressSummary(progress),
|
||||
badges: includeBadges ? (badgesBySet.get(row.id) ?? []).map((badge) => badgeData(badge, earned.has(badge.id))) : undefined,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -123,7 +149,7 @@ problemsetRoutes.get("/problem-sets", optionalAuth, async (c) => {
|
||||
db.select().from(schema.problemset).where(where).orderBy(desc(schema.problemset.createTime)).limit(limit).offset(offset),
|
||||
])
|
||||
return success(c, problemSetListSchema.parse({
|
||||
results: await Promise.all(rows.map((row) => serializeProblemSet(row, c.get("user")?.id, true))),
|
||||
results: await serializeProblemSets(rows, c.get("user")?.id, true),
|
||||
total: totalRows[0]?.value ?? 0,
|
||||
}))
|
||||
})
|
||||
@@ -133,7 +159,8 @@ problemsetRoutes.get("/problem-sets/:id", optionalAuth, async (c) => {
|
||||
const [row] = await db.select().from(schema.problemset)
|
||||
.where(and(eq(schema.problemset.id, id), eq(schema.problemset.visible, true), ne(schema.problemset.status, "draft"))).limit(1)
|
||||
if (!row) return failure(c, 404, "problem-set-not-found", "题单不存在")
|
||||
return success(c, await serializeProblemSet(row, c.get("user")?.id))
|
||||
const [data] = await serializeProblemSets([row], c.get("user")?.id)
|
||||
return success(c, data)
|
||||
})
|
||||
|
||||
problemsetRoutes.get("/problem-sets/:id/problems", optionalAuth, async (c) => {
|
||||
@@ -282,22 +309,21 @@ problemsetRoutes.put("/problem-set-progress", requireAuth, async (c) => {
|
||||
})
|
||||
}
|
||||
const badges = await tx.select().from(schema.problemsetBadge).where(eq(schema.problemsetBadge.problemsetId, problemSet.id))
|
||||
const earned: typeof schema.problemsetBadge.$inferSelect[] = []
|
||||
for (const badge of badges) {
|
||||
const hit = badge.conditionType === "all_problems"
|
||||
? updated.totalProblemsCount > 0 && updated.completedProblemsCount === updated.totalProblemsCount
|
||||
: badge.conditionType === "problem_count"
|
||||
? updated.completedProblemsCount >= badge.conditionValue
|
||||
: badge.conditionType === "score" && updated.totalScore >= badge.conditionValue
|
||||
if (!hit) continue
|
||||
const inserted = await tx.insert(schema.userBadge).values({
|
||||
userId: user.id,
|
||||
badgeId: badge.id,
|
||||
earnedTime: new Date().toISOString(),
|
||||
}).onConflictDoNothing({ target: [schema.userBadge.badgeId, schema.userBadge.userId] }).returning({ id: schema.userBadge.id })
|
||||
if (inserted.length) earned.push(badge)
|
||||
}
|
||||
return { earned }
|
||||
const hits = badges.filter((badge) => badge.conditionType === "all_problems"
|
||||
? updated.totalProblemsCount > 0 && updated.completedProblemsCount === updated.totalProblemsCount
|
||||
: badge.conditionType === "problem_count"
|
||||
? updated.completedProblemsCount >= badge.conditionValue
|
||||
: badge.conditionType === "score" && updated.totalScore >= badge.conditionValue)
|
||||
if (hits.length === 0) return { earned: [] as (typeof schema.problemsetBadge.$inferSelect)[] }
|
||||
// 达标的奖章一次插完,冲突忽略后 returning 回来的就是这次真拿到的
|
||||
const inserted = await tx.insert(schema.userBadge).values(hits.map((badge) => ({
|
||||
userId: user.id,
|
||||
badgeId: badge.id,
|
||||
earnedTime: new Date().toISOString(),
|
||||
}))).onConflictDoNothing({ target: [schema.userBadge.badgeId, schema.userBadge.userId] })
|
||||
.returning({ badgeId: schema.userBadge.badgeId })
|
||||
const insertedIds = new Set(inserted.map((row) => row.badgeId))
|
||||
return { earned: hits.filter((badge) => insertedIds.has(badge.id)) }
|
||||
})
|
||||
if ("error" in result && result.error) {
|
||||
const error = result.error
|
||||
|
||||
Reference in new issue
Block a user