perf(流程图): 列表裁掉整行 select、count 去掉 join,统计面板的数值全下推 SQL
Some checks failed
Deploy / deploy (push) Has been cancelled
Some checks failed
Deploy / deploy (push) Has been cancelled
流程图提交量涨上去之后,先扛不住的是教师统计面板:它一条不带 limit 的 select
把整个时间窗的行拉进内存再用 JS 算,词云那个 3000 条上限是在 JS 里截的,行早就
全回来了。列表那边则是 `select({ flowchart: 整行, problem: 整行 })`,把
mermaid_code、flowchart_data、三个 AI 文本列和整张题目表一起拉回来,响应一个
都用不到(快照实测流程图行均 4.9KB、题目行均 2.2KB,10 行一页白拉 ~70KB,
limit=250 时 1.7MB)。
- 列表改成白名单列 flowchartListColumns,对齐 submission 那边的
submissionListColumns(那边同样刻意不取 code / info)。
- 题号 / 用户名筛选先解析成 flowchart_submission 自己的列,count 因此一个 join
都不用挂,回得到最小索引上的 index-only scan;筛条件落在驱动表上,规划器也走
得上 flowchart_user_time_idx / flowchart_problem_time_idx。
- 统计面板拆成五条各自和行数脱钩的查询:数值聚合、等级分布、各项平均分
(jsonb_each + group by)、词云原料(order by ... limit 3000)、谁没做。
每项满分改从词云那批行里顺手取,省掉一次 21 万行的排序。
- matchedUsers() 从 submission.ts 挪进 helpers.ts,两条统计共用。
拿生产快照(2134 条)复制一份、另插三行脏数据(标量 jsonb、数组 jsonb、分数和
满分写成字符串),新旧两版各跑 24 个请求组合:23 个逐字节一致;剩下 1 个只是三行
create_time 完全相同的记录先后不同 —— 既有的不确定性(ORDER BY create_time 不是
全序),留给后面的 keyset 分页一并解决。
把表灌到 5.3 万 / 21.3 万行实测(HTTP 端到端,5 次取最好,旧 → 新):
列表 limit=10 35 → 4 ms | 56 → 8 ms
列表 limit=250 36 → 5 ms | 54 → 8 ms
列表 offset=5万 108 → 39 ms | 57 → 45 ms
列表 按班级 33 → 4 ms | 53 → 8 ms
统计 全部时段 288 → 187 ms | 1169 → 704 ms
统计 一个班 143 → 43 ms | 388 → 129 ms
统计 一道题 94 → 194 ms | 339 → 277 ms
「统计 一道题」在 5 万行量级是退步的:那个筛选命中全表 23% 的行,PG 侧的 jsonb
算子比「原样输出让 Bun 去 parse」更费 CPU,要到 21 万行才反超。它跟的是筛出来的
行数、不跟总量走,200ms 的教师面板可以接受,没有再调。
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -10,7 +10,7 @@ import {
|
|||||||
type FlowchartStatistics,
|
type FlowchartStatistics,
|
||||||
type FlowchartSubmission,
|
type FlowchartSubmission,
|
||||||
} from "@oj2/contract"
|
} from "@oj2/contract"
|
||||||
import { and, asc, count, desc, eq, ilike, isNull, sql } from "drizzle-orm"
|
import { and, asc, count, desc, eq, inArray, isNull, sql, type SQL } from "drizzle-orm"
|
||||||
import { Hono } from "hono"
|
import { Hono } from "hono"
|
||||||
|
|
||||||
import { requireAuth, requireTeacher, type AppEnv } from "../auth/middleware"
|
import { requireAuth, requireTeacher, type AppEnv } from "../auth/middleware"
|
||||||
@@ -24,6 +24,7 @@ import { buildWordFrequencies } from "../services/word-frequency"
|
|||||||
import { todayStart } from "../time"
|
import { todayStart } from "../time"
|
||||||
import {
|
import {
|
||||||
isAdminRole,
|
isAdminRole,
|
||||||
|
matchedUsers,
|
||||||
objectValue,
|
objectValue,
|
||||||
queryInteger,
|
queryInteger,
|
||||||
rounded,
|
rounded,
|
||||||
@@ -109,11 +110,77 @@ flowchartRoutes.post("/flowcharts", requireAuth, async (c) => {
|
|||||||
return success(c, { submissionId: id, status: "pending" } satisfies CreateFlowchartResponse, 201)
|
return success(c, { submissionId: id, status: "pending" } satisfies CreateFlowchartResponse, 201)
|
||||||
})
|
})
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 题号 / 用户名筛选一律先解析成 `flowchart_submission` 自己的列,不靠 join 之后比
|
||||||
|
* `problem._id` / `user.username`。同一套做法见 submission.ts 的
|
||||||
|
* problemFilter / usernameFilter,这里是两个好处:
|
||||||
|
*
|
||||||
|
* - 列表的 count 因此**一个 join 都不用挂**。挂了就回不到最小索引上的 index-only
|
||||||
|
* scan,而这张表每行带 3KB 的 flowchart_data + 1.2KB 的 mermaid_code,堆页密度低,
|
||||||
|
* 回表比 submission 那边贵。
|
||||||
|
* - 筛条件落在驱动表上,规划器能走 flowchart_user_time_idx / flowchart_problem_time_idx,
|
||||||
|
* 不必顺着时间索引倒扫再逐行 join 过滤。
|
||||||
|
*
|
||||||
|
* 查无此题 / 此人时留**恒假**条件 —— 少推一个 filter 就成了「不筛」,
|
||||||
|
* 「查无此班」会变成「全站」。
|
||||||
|
*/
|
||||||
|
async function flowchartProblemFilter(displayId: string) {
|
||||||
|
const problems = await db
|
||||||
|
.select({ id: schema.problem.id })
|
||||||
|
.from(schema.problem)
|
||||||
|
.where(and(
|
||||||
|
sql`lower(${schema.problem.displayId}) = lower(${displayId})`,
|
||||||
|
// 流程图题都是公开题(快照里那 12 道 contest_id 全为空),
|
||||||
|
// 比赛题的 _id 撞号是常态,不该被筛进来
|
||||||
|
isNull(schema.problem.contestId),
|
||||||
|
))
|
||||||
|
return problems.length
|
||||||
|
? inArray(schema.flowchartSubmission.problemId, problems.map((row) => row.id))
|
||||||
|
: sql`false`
|
||||||
|
}
|
||||||
|
|
||||||
|
async function flowchartUserFilter(username: string) {
|
||||||
|
const ids = (await matchedUsers(username)).map((row) => row.id)
|
||||||
|
return ids.length ? inArray(schema.flowchartSubmission.userId, ids) : sql`false`
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 列表只取这几列。原来是 `select({ flowchart: 整行, problem: 整行 })`,把
|
||||||
|
* mermaid_code、flowchart_data、ai_feedback、ai_suggestions、ai_criteria_details
|
||||||
|
* 和**整张题目表**(description / 标准答案 / 标准流程图…)一起拉回来,而响应一个
|
||||||
|
* 都用不到:生产快照实测流程图行均 4.9KB(p90 6.9KB)、题目行均 2.2KB,默认 10 行
|
||||||
|
* 一页白拉 ~70KB,limit=250 时 1.7MB。
|
||||||
|
*
|
||||||
|
* 对齐 submission.ts 的 submissionListColumns —— 那边同样是手写白名单,
|
||||||
|
* 刻意不取 code / info。
|
||||||
|
*/
|
||||||
|
const flowchartListColumns = {
|
||||||
|
flowchart: {
|
||||||
|
id: schema.flowchartSubmission.id,
|
||||||
|
// showLink 判定要,序列化本身用不到
|
||||||
|
userId: schema.flowchartSubmission.userId,
|
||||||
|
status: schema.flowchartSubmission.status,
|
||||||
|
createTime: schema.flowchartSubmission.createTime,
|
||||||
|
aiScore: schema.flowchartSubmission.aiScore,
|
||||||
|
aiGrade: schema.flowchartSubmission.aiGrade,
|
||||||
|
aiProvider: schema.flowchartSubmission.aiProvider,
|
||||||
|
aiModel: schema.flowchartSubmission.aiModel,
|
||||||
|
processingTime: schema.flowchartSubmission.processingTime,
|
||||||
|
evaluationTime: schema.flowchartSubmission.evaluationTime,
|
||||||
|
},
|
||||||
|
username: schema.user.username,
|
||||||
|
problem: {
|
||||||
|
displayId: schema.problem.displayId,
|
||||||
|
title: schema.problem.title,
|
||||||
|
// 同上,canView 要
|
||||||
|
createdById: schema.problem.createdById,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
flowchartRoutes.get("/flowcharts", requireAuth, async (c) => {
|
flowchartRoutes.get("/flowcharts", requireAuth, async (c) => {
|
||||||
const user = c.get("user")!
|
const user = c.get("user")!
|
||||||
const limit = queryInteger(c.req.query("limit"), 10, { min: 1, max: 250 })
|
const limit = queryInteger(c.req.query("limit"), 10, { min: 1, max: 250 })
|
||||||
const offset = queryInteger(c.req.query("offset"), 0, { min: 0 })
|
const offset = queryInteger(c.req.query("offset"), 0, { min: 0 })
|
||||||
const filters = []
|
|
||||||
const displayId = c.req.query("problemId")?.trim()
|
const displayId = c.req.query("problemId")?.trim()
|
||||||
const username = c.req.query("username")?.trim()
|
const username = c.req.query("username")?.trim()
|
||||||
const grade = c.req.query("grade")
|
const grade = c.req.query("grade")
|
||||||
@@ -123,17 +190,25 @@ flowchartRoutes.get("/flowcharts", requireAuth, async (c) => {
|
|||||||
if (!(await getBooleanOption("submission_list_show_all", true)) && !isAdminRole(user)) {
|
if (!(await getBooleanOption("submission_list_show_all", true)) && !isAdminRole(user)) {
|
||||||
return success(c, { results: [], total: 0 } satisfies FlowchartList)
|
return success(c, { results: [], total: 0 } satisfies FlowchartList)
|
||||||
}
|
}
|
||||||
if (displayId) filters.push(sql`lower(${schema.problem.displayId}) = lower(${displayId})`)
|
// 「只看自己」盖过用户名;普通学生不填用户名时也只看自己
|
||||||
if (c.req.query("myself") === "1" || (!username && user.adminType === "Regular User")) filters.push(eq(schema.flowchartSubmission.userId, user.id))
|
const onlyMyself = c.req.query("myself") === "1" || (!username && user.adminType === "Regular User")
|
||||||
else if (username) filters.push(ilike(schema.user.username, `%${username}%`))
|
const filters: Array<SQL | undefined> = []
|
||||||
|
filters.push(...await Promise.all([
|
||||||
|
displayId ? flowchartProblemFilter(displayId) : undefined,
|
||||||
|
!onlyMyself && username ? flowchartUserFilter(username) : undefined,
|
||||||
|
]))
|
||||||
|
if (onlyMyself) filters.push(eq(schema.flowchartSubmission.userId, user.id))
|
||||||
if (c.req.query("today") === "1") filters.push(sql`${schema.flowchartSubmission.createTime} >= ${todayStart()}`)
|
if (c.req.query("today") === "1") filters.push(sql`${schema.flowchartSubmission.createTime} >= ${todayStart()}`)
|
||||||
if (["S", "A", "B", "C"].includes(grade ?? "")) filters.push(eq(schema.flowchartSubmission.aiGrade, grade!))
|
if (["S", "A", "B", "C"].includes(grade ?? "")) filters.push(eq(schema.flowchartSubmission.aiGrade, grade!))
|
||||||
const where = filters.length ? and(...filters) : undefined
|
const where = and(...filters)
|
||||||
const [totalRows, rows] = await Promise.all([
|
const [totalRows, rows] = await Promise.all([
|
||||||
db.select({ value: count() }).from(schema.flowchartSubmission).innerJoin(schema.user, eq(schema.flowchartSubmission.userId, schema.user.id)).innerJoin(schema.problem, eq(schema.flowchartSubmission.problemId, schema.problem.id)).where(where),
|
// 筛条件已经全落在 flowchart_submission 自己的列上,count 不挂任何 join
|
||||||
db.select({ flowchart: schema.flowchartSubmission, username: schema.user.username, problem: schema.problem })
|
db.select({ value: count() }).from(schema.flowchartSubmission).where(where),
|
||||||
.from(schema.flowchartSubmission).innerJoin(schema.user, eq(schema.flowchartSubmission.userId, schema.user.id))
|
db.select(flowchartListColumns)
|
||||||
.innerJoin(schema.problem, eq(schema.flowchartSubmission.problemId, schema.problem.id)).where(where)
|
.from(schema.flowchartSubmission)
|
||||||
|
.innerJoin(schema.user, eq(schema.flowchartSubmission.userId, schema.user.id))
|
||||||
|
.innerJoin(schema.problem, eq(schema.flowchartSubmission.problemId, schema.problem.id))
|
||||||
|
.where(where)
|
||||||
.orderBy(desc(schema.flowchartSubmission.createTime)).limit(limit).offset(offset),
|
.orderBy(desc(schema.flowchartSubmission.createTime)).limit(limit).offset(offset),
|
||||||
])
|
])
|
||||||
return success(c, {
|
return success(c, {
|
||||||
@@ -159,15 +234,22 @@ flowchartRoutes.get("/flowcharts", requireAuth, async (c) => {
|
|||||||
const FLOWCHART_COMPLETED = 2
|
const FLOWCHART_COMPLETED = 2
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 词云的分词条数上限。
|
* 词云取的提交条数上限,同时也是分词的文本条数上限。
|
||||||
*
|
*
|
||||||
* 数值统计(总数、均分、等级分布、各项平均分、完成人数)仍然按整个时间窗**精确**
|
* 数值统计(总数、均分、等级分布、各项平均分、完成人数)按整个时间窗**精确**计算,
|
||||||
* 计算 —— 那只是已取回行上的算术,不额外花钱。真正会随数据量线性变重的是分词:
|
* 但那几项现在全是 SQL 聚合,代价不随窗口里的行数走。**不能采样** —— 采了之后老师
|
||||||
* 每条 feedback / suggestions / comment 都要走一遍 jieba,而前端的「全部时段」
|
* 看到的完成率和均分就是错的,而且从界面上看不出来。
|
||||||
* 是不带 start 的,攒一学年就得把所有评语重新 cut 一遍。
|
|
||||||
*
|
*
|
||||||
* 词云是辅助性的,看的是高频问题,取最近这些条足够;数值不能采样 —— 采了之后
|
* 会随数据量线性变重的只剩词云:每条 feedback / suggestions / comment 都要走一遍
|
||||||
* 老师看到的完成率和均分就是错的,而且从界面上看不出来。
|
* jieba,而前端的「全部时段」是不带 start 的(FlowchartStatisticsPanel.vue 那个
|
||||||
|
* `duration === "all"`),攒一学年就得把所有评语重新 cut 一遍。词云是辅助性的,
|
||||||
|
* 看的是高频问题,取最近这些条足够。
|
||||||
|
*
|
||||||
|
* 这里**同时**卡了两道:SQL 侧 `order by create_time desc limit N` 只取最近 N 条提交,
|
||||||
|
* JS 侧 pushText 再卡 N 条文本。生产快照实测一条提交出 5.97 段文本(几项 comment +
|
||||||
|
* feedback + suggestions,最少的一条也有 1 段),所以先到的一直是文本那道闸——3000 段
|
||||||
|
* 在 500 条出头就满了,行数那道只是兜底:真遇到一批评语全空的提交,词云少看几条,
|
||||||
|
* 可以接受。原来只有 JS 那道,行早就整批拉回内存了。
|
||||||
*/
|
*/
|
||||||
const WORDCLOUD_TEXT_LIMIT = 3000
|
const WORDCLOUD_TEXT_LIMIT = 3000
|
||||||
|
|
||||||
@@ -176,7 +258,7 @@ flowchartRoutes.get("/flowcharts/statistics", requireTeacher, async (c) => {
|
|||||||
if (!end) return failure(c, 400, "invalid-request", "end is required")
|
if (!end) return failure(c, 400, "invalid-request", "end is required")
|
||||||
const start = c.req.query("start")?.trim()
|
const start = c.req.query("start")?.trim()
|
||||||
|
|
||||||
const filters = [
|
const filters: Array<SQL | undefined> = [
|
||||||
eq(schema.flowchartSubmission.status, FLOWCHART_COMPLETED),
|
eq(schema.flowchartSubmission.status, FLOWCHART_COMPLETED),
|
||||||
sql`${schema.flowchartSubmission.createTime} <= ${end}`,
|
sql`${schema.flowchartSubmission.createTime} <= ${end}`,
|
||||||
]
|
]
|
||||||
@@ -198,85 +280,148 @@ flowchartRoutes.get("/flowcharts/statistics", requireTeacher, async (c) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const username = c.req.query("username")?.trim()
|
const username = c.req.query("username")?.trim()
|
||||||
if (username) filters.push(ilike(schema.user.username, `%${username}%`))
|
// 用户名先解析成账号,再拿 user_id 去筛 —— 理由同代码提交的统计接口
|
||||||
|
// (submission.ts 的 GET /submissions/statistics),顺带让下面这几条一个 join 都不用挂
|
||||||
// 只有指定了用户名才谈得上「班级人数」,不指定时分母无意义
|
const matched = username ? await matchedUsers(username) : []
|
||||||
|
if (username) {
|
||||||
|
const ids = matched.map((row) => row.id)
|
||||||
|
// 一个账号都没匹配上时得留个恒假条件,否则「查无此班」变成「全站统计」
|
||||||
|
filters.push(ids.length ? inArray(schema.flowchartSubmission.userId, ids) : sql`false`)
|
||||||
|
}
|
||||||
|
const where = and(...filters)
|
||||||
|
// 花名册:只有指定了用户名才谈得上「班级人数」,不指定时分母无意义。
|
||||||
|
// 未禁用的普通用户才进分母,教师和管理员不算
|
||||||
const roster = username
|
const roster = username
|
||||||
? await db
|
? matched.filter((row) => !row.isDisabled && row.adminType === "Regular User")
|
||||||
.select({ username: schema.user.username, className: schema.user.className })
|
|
||||||
.from(schema.user)
|
|
||||||
.where(and(
|
|
||||||
ilike(schema.user.username, `%${username}%`),
|
|
||||||
eq(schema.user.isDisabled, false),
|
|
||||||
eq(schema.user.adminType, "Regular User"),
|
|
||||||
))
|
|
||||||
: []
|
: []
|
||||||
|
|
||||||
const rows = await db
|
/**
|
||||||
.select({
|
* 五条查询,每条的代价都和窗口里的行数脱钩(词云那条卡了 limit)。
|
||||||
username: schema.user.username,
|
*
|
||||||
score: schema.flowchartSubmission.aiScore,
|
* 原来是**一条**不带 limit 的 `select(username, score, grade, criteria, feedback,
|
||||||
grade: schema.flowchartSubmission.aiGrade,
|
* suggestions) order by create_time desc`,把整个时间窗的行拉进内存再用 JS 算 ——
|
||||||
criteria: schema.flowchartSubmission.aiCriteriaDetails,
|
* 词云的 3000 条上限是在 JS 里截的,行早就全回来了。备份实测每行的 AI 文本约 366B
|
||||||
feedback: schema.flowchartSubmission.aiFeedback,
|
* (criteria 255 + suggestions 64 + feedback 47),现在 2134 条无感,5 万条就是一次
|
||||||
suggestions: schema.flowchartSubmission.aiSuggestions,
|
* 点击 18MB,而老师是开着面板反复切时段、切班的。
|
||||||
})
|
*/
|
||||||
.from(schema.flowchartSubmission)
|
const [[totals], gradeRows, criteriaRows, textRows, submittedRows] = await Promise.all([
|
||||||
.innerJoin(schema.user, eq(schema.flowchartSubmission.userId, schema.user.id))
|
db
|
||||||
.where(and(...filters))
|
.select({
|
||||||
// 按时间倒序,好让词云取到的那部分是最近的
|
total: count(),
|
||||||
.orderBy(desc(schema.flowchartSubmission.createTime))
|
/**
|
||||||
|
* 均分拆成 sum / count 两项,不直接用 `avg()`:分母是**有分数的条数**而不是
|
||||||
|
* 总条数(对齐 Django 的 Avg(),它跳过 NULL),拆开之后这个口径在代码里是
|
||||||
|
* 写明的,也省掉 avg() 在空集上回 NULL 还要兜底。
|
||||||
|
*/
|
||||||
|
scoreSum: sql<number>`coalesce(sum(${schema.flowchartSubmission.aiScore}), 0)`.mapWith(Number),
|
||||||
|
scoreCount: sql<number>`count(${schema.flowchartSubmission.aiScore})::int`.mapWith(Number),
|
||||||
|
// 完成人数。user_id 和 username 一一对应,按哪个 distinct 都一样,
|
||||||
|
// 按 user_id 就不必 join user
|
||||||
|
completedCount: sql<number>`count(distinct ${schema.flowchartSubmission.userId})::int`.mapWith(Number),
|
||||||
|
})
|
||||||
|
.from(schema.flowchartSubmission)
|
||||||
|
.where(where),
|
||||||
|
db
|
||||||
|
.select({ grade: schema.flowchartSubmission.aiGrade, n: count() })
|
||||||
|
.from(schema.flowchartSubmission)
|
||||||
|
.where(where)
|
||||||
|
.groupBy(schema.flowchartSubmission.aiGrade),
|
||||||
|
/**
|
||||||
|
* 各项**平均分**。`ai_criteria_details` 是 `{ 项名: { score, max, comment } }`,
|
||||||
|
* 用 jsonb_each 展开之后按项名分组。分数不是数字的项整项跳过,和原来 JS 那句
|
||||||
|
* `typeof detail.score !== "number"` 的 continue 一致。
|
||||||
|
*
|
||||||
|
* **那道 `jsonb_typeof(...) = 'object'` 的闸不能省,而且要写在 jsonb_each 的参数里。**
|
||||||
|
* 不能省:撞上标量(历史脏数据)jsonb_each 直接抛错,整个面板 500 ——
|
||||||
|
* 拿 `'5'::jsonb` 和 `'[1,2]'::jsonb` 各插一行验过。
|
||||||
|
*
|
||||||
|
* 写在哪儿则纯是规划器的脸色:挪进 where 当基表过滤条件时,53350 行的探针上
|
||||||
|
* 实测 180ms → 360ms,因为计划从「并行 Partial HashAggregate」换成了「串行
|
||||||
|
* GroupAggregate + 21 万行外部归并排序、落盘 26MB」。两种写法都正确,选快的那个。
|
||||||
|
*
|
||||||
|
* 每项的**满分**不在这里取,见下面 criteriaMax 的注释:在这条 SQL 里按
|
||||||
|
* create_time 取「最新那条」要给 21 万行(4 项 × 5 万条)排序,同一个探针上
|
||||||
|
* 实测 254ms → 842ms,而满分本来就是几个常数。
|
||||||
|
*/
|
||||||
|
db.execute<{ key: string; avg: number }>(sql`
|
||||||
|
select e.key as key, avg((e.value->>'score')::double precision) as avg
|
||||||
|
from ${schema.flowchartSubmission}
|
||||||
|
cross join lateral jsonb_each(
|
||||||
|
case when jsonb_typeof(${schema.flowchartSubmission.aiCriteriaDetails}) = 'object'
|
||||||
|
then ${schema.flowchartSubmission.aiCriteriaDetails}
|
||||||
|
else '{}'::jsonb end
|
||||||
|
) e
|
||||||
|
where ${where} and jsonb_typeof(e.value->'score') = 'number'
|
||||||
|
group by e.key
|
||||||
|
`),
|
||||||
|
// 词云的原料。只有这条要读大列,所以只有它按时间倒序取最近的 N 条
|
||||||
|
db
|
||||||
|
.select({
|
||||||
|
criteria: schema.flowchartSubmission.aiCriteriaDetails,
|
||||||
|
feedback: schema.flowchartSubmission.aiFeedback,
|
||||||
|
suggestions: schema.flowchartSubmission.aiSuggestions,
|
||||||
|
})
|
||||||
|
.from(schema.flowchartSubmission)
|
||||||
|
.where(where)
|
||||||
|
.orderBy(desc(schema.flowchartSubmission.createTime))
|
||||||
|
.limit(WORDCLOUD_TEXT_LIMIT),
|
||||||
|
// 「谁没做」只在有花名册时算得出来,行数也就一个班
|
||||||
|
roster.length
|
||||||
|
? db
|
||||||
|
.selectDistinct({ userId: schema.flowchartSubmission.userId })
|
||||||
|
.from(schema.flowchartSubmission)
|
||||||
|
.where(where)
|
||||||
|
: [],
|
||||||
|
])
|
||||||
|
|
||||||
const empty = {
|
if (!totals || totals.total === 0) {
|
||||||
totalCount: 0,
|
return success(c, {
|
||||||
avgScore: 0,
|
totalCount: 0,
|
||||||
gradeDistribution: {},
|
avgScore: 0,
|
||||||
criteriaAverages: {},
|
gradeDistribution: {},
|
||||||
personCount: roster.length,
|
criteriaAverages: {},
|
||||||
completedCount: 0,
|
personCount: roster.length,
|
||||||
wordFrequencies: [],
|
completedCount: 0,
|
||||||
// 一条提交都没有时,花名册上的人**全都**是「没做」—— 原来这里写死空数组,
|
wordFrequencies: [],
|
||||||
// 于是一节课刚开始、最该点名的时候,教师面板反而一个名字都不给
|
// 一条提交都没有时,花名册上的人**全都**是「没做」—— 原来这里写死空数组,
|
||||||
dataUnaccepted: roster.map((row) => ({
|
// 于是一节课刚开始、最该点名的时候,教师面板反而一个名字都不给
|
||||||
username: row.username,
|
dataUnaccepted: roster.map((row) => ({
|
||||||
realName: stripClassPrefix(row.username, row.className),
|
username: row.username,
|
||||||
})),
|
realName: stripClassPrefix(row.username, row.className),
|
||||||
|
})),
|
||||||
|
} satisfies FlowchartStatistics)
|
||||||
}
|
}
|
||||||
if (rows.length === 0) return success(c, empty satisfies FlowchartStatistics)
|
|
||||||
|
|
||||||
const gradeDistribution: Record<string, number> = {}
|
const gradeDistribution: Record<string, number> = {}
|
||||||
const criteriaTotals = new Map<string, { sum: number; count: number; max: number }>()
|
for (const row of gradeRows) {
|
||||||
|
// 旧后端用 values_list("ai_grade") 分组,null 也会成为一个桶;这里保持同样的口径。
|
||||||
|
// null 和空串会分成两组,合并到同一个桶里
|
||||||
|
const grade = row.grade ?? ""
|
||||||
|
gradeDistribution[grade] = (gradeDistribution[grade] ?? 0) + row.n
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 词云原料和每项满分都从同一批行里取 —— 这批行本来就要读(见下面的 textRows),
|
||||||
|
* 白嫖一遍,不额外查库。
|
||||||
|
*
|
||||||
|
* 满分的口径是「按 create_time 倒序,某项**第一次**出现时写的那个 max,不是数字就
|
||||||
|
* 退回 100」,和原来逐行遍历时那句 `if (bucket) ... else set(max)` 完全一致,只是
|
||||||
|
* 遍历范围从整个时间窗收成最近 WORDCLOUD_TEXT_LIMIT 条。满分是评分标准里的常数
|
||||||
|
* (完整性 30、逻辑正确性 40…),几万条里换一次都算多;真出现一项**只**在更早的
|
||||||
|
* 行里有过,它的平均分照常出(那是 SQL 全窗口算的),满分退回 100。
|
||||||
|
*/
|
||||||
|
const criteriaMax = new Map<string, number>()
|
||||||
const texts: string[] = []
|
const texts: string[] = []
|
||||||
const pushText = (value: string) => {
|
const pushText = (value: string) => {
|
||||||
if (texts.length < WORDCLOUD_TEXT_LIMIT) texts.push(value)
|
if (texts.length < WORDCLOUD_TEXT_LIMIT) texts.push(value)
|
||||||
}
|
}
|
||||||
const submitted = new Set<string>()
|
for (const row of textRows) {
|
||||||
let scoreSum = 0
|
|
||||||
let scoreCount = 0
|
|
||||||
|
|
||||||
for (const row of rows) {
|
|
||||||
submitted.add(row.username)
|
|
||||||
// 旧后端用 values_list("ai_grade") 分组,null 也会成为一个桶;这里保持同样的口径
|
|
||||||
const grade = row.grade ?? ""
|
|
||||||
gradeDistribution[grade] = (gradeDistribution[grade] ?? 0) + 1
|
|
||||||
if (row.score !== null) {
|
|
||||||
scoreSum += row.score
|
|
||||||
scoreCount += 1
|
|
||||||
}
|
|
||||||
for (const [key, value] of Object.entries(objectValue(row.criteria))) {
|
for (const [key, value] of Object.entries(objectValue(row.criteria))) {
|
||||||
const detail = objectValue(value)
|
const detail = objectValue(value)
|
||||||
|
// 和上面那条聚合同一道闸:分数不是数字的项当没配过,满分和评语也都不收
|
||||||
if (typeof detail.score !== "number") continue
|
if (typeof detail.score !== "number") continue
|
||||||
const bucket = criteriaTotals.get(key)
|
if (!criteriaMax.has(key)) {
|
||||||
if (bucket) {
|
criteriaMax.set(key, typeof detail.max === "number" ? detail.max : 100)
|
||||||
bucket.sum += detail.score
|
|
||||||
bucket.count += 1
|
|
||||||
} else {
|
|
||||||
// max 取第一次见到的那条,与旧后端 `if key not in criteria_max` 一致
|
|
||||||
criteriaTotals.set(key, {
|
|
||||||
sum: detail.score,
|
|
||||||
count: 1,
|
|
||||||
max: typeof detail.max === "number" ? detail.max : 100,
|
|
||||||
})
|
|
||||||
}
|
}
|
||||||
if (typeof detail.comment === "string" && detail.comment) pushText(detail.comment)
|
if (typeof detail.comment === "string" && detail.comment) pushText(detail.comment)
|
||||||
}
|
}
|
||||||
@@ -285,21 +430,21 @@ flowchartRoutes.get("/flowcharts/statistics", requireTeacher, async (c) => {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const criteriaAverages: Record<string, { avg: number; max: number }> = {}
|
const criteriaAverages: Record<string, { avg: number; max: number }> = {}
|
||||||
for (const [key, bucket] of criteriaTotals) {
|
for (const row of criteriaRows) {
|
||||||
criteriaAverages[key] = { avg: rounded(bucket.sum / bucket.count, 1), max: bucket.max }
|
criteriaAverages[row.key] = { avg: rounded(row.avg, 1), max: criteriaMax.get(row.key) ?? 100 }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const submitted = new Set(submittedRows.map((row) => row.userId))
|
||||||
return success(c, {
|
return success(c, {
|
||||||
totalCount: rows.length,
|
totalCount: totals.total,
|
||||||
// 分母是有分数的条数,不是总条数 —— 对齐 Django 的 Avg(),它跳过 NULL
|
avgScore: totals.scoreCount ? rounded(totals.scoreSum / totals.scoreCount, 1) : 0,
|
||||||
avgScore: scoreCount ? rounded(scoreSum / scoreCount, 1) : 0,
|
|
||||||
gradeDistribution,
|
gradeDistribution,
|
||||||
criteriaAverages,
|
criteriaAverages,
|
||||||
personCount: roster.length,
|
personCount: roster.length,
|
||||||
completedCount: submitted.size,
|
completedCount: totals.completedCount,
|
||||||
wordFrequencies: await buildWordFrequencies(texts),
|
wordFrequencies: await buildWordFrequencies(texts),
|
||||||
dataUnaccepted: roster
|
dataUnaccepted: roster
|
||||||
.filter((row) => !submitted.has(row.username))
|
.filter((row) => !submitted.has(row.id))
|
||||||
.map((row) => ({
|
.map((row) => ({
|
||||||
username: row.username,
|
username: row.username,
|
||||||
realName: stripClassPrefix(row.username, row.className),
|
realName: stripClassPrefix(row.username, row.className),
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
import { ADMIN_ROLES, TEACHER_ROLES, type SampleUser } from "@oj2/contract"
|
import { ADMIN_ROLES, TEACHER_ROLES, type SampleUser } from "@oj2/contract"
|
||||||
|
|
||||||
import { and, count, eq, notInArray } from "drizzle-orm"
|
import { and, count, eq, ilike, notInArray } from "drizzle-orm"
|
||||||
|
|
||||||
import type { AuthUser } from "../auth/session"
|
import type { AuthUser } from "../auth/session"
|
||||||
import { db, schema } from "../db"
|
import { db, schema } from "../db"
|
||||||
@@ -128,3 +128,33 @@ export async function countFailedSubmissions(userId: number, problemId: number)
|
|||||||
)
|
)
|
||||||
return failed?.value ?? 0
|
return failed?.value ?? 0
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 用户名模糊匹配到的账号。统计的两件事都从它出发:**筛哪些提交**(拿 id),
|
||||||
|
* 以及**花名册**(班级人数、谁没做,见调用处的过滤)。
|
||||||
|
*
|
||||||
|
* 这里必须查 `user` 表而不是 `submission.username` —— 后者是提交那一刻冻结的
|
||||||
|
* 快照,学生改名之后旧提交还挂着旧名字,`ilike submission.username` 匹配不上。
|
||||||
|
*
|
||||||
|
* 生产快照实测(2026-09-08):24 级数媒两个班改成编号制用户名之后,85 人的
|
||||||
|
* 提交挂在旧名下。查 `ks249` 旧口径 0 条 / 新口径 7 条 —— 整个班 48 人全掉进
|
||||||
|
* 「一条没交」;查 `ks248` 20 条 / 54 条,13 个人的成绩查不出来。
|
||||||
|
*
|
||||||
|
* 返回**全部**匹配到的账号,禁用的和教师也在内 —— 「谁交过」不该受这两个条件
|
||||||
|
* 影响。花名册那一份在调用处再筛(未禁用 + 普通用户),教师和管理员不进分母。
|
||||||
|
*
|
||||||
|
* 代码提交和流程图两条统计都走这里。流程图那张表连冻结用户名都没有(只有
|
||||||
|
* `user_id`),更是只能从这儿拿 id。
|
||||||
|
*/
|
||||||
|
export async function matchedUsers(username: string) {
|
||||||
|
return db
|
||||||
|
.select({
|
||||||
|
id: schema.user.id,
|
||||||
|
username: schema.user.username,
|
||||||
|
className: schema.user.className,
|
||||||
|
isDisabled: schema.user.isDisabled,
|
||||||
|
adminType: schema.user.adminType,
|
||||||
|
})
|
||||||
|
.from(schema.user)
|
||||||
|
.where(ilike(schema.user.username, `%${username}%`))
|
||||||
|
}
|
||||||
|
|||||||
@@ -37,7 +37,14 @@ import { CodeFormatError, formatCode } from "../services/format-code"
|
|||||||
import { getBooleanOption } from "../services/options"
|
import { getBooleanOption } from "../services/options"
|
||||||
import { consumeToken } from "../services/throttling"
|
import { consumeToken } from "../services/throttling"
|
||||||
import { todayStart } from "../time"
|
import { todayStart } from "../time"
|
||||||
import { asFilterValue, isAdminRole, queryInteger, rounded, stripClassPrefix } from "./helpers"
|
import {
|
||||||
|
asFilterValue,
|
||||||
|
isAdminRole,
|
||||||
|
matchedUsers,
|
||||||
|
queryInteger,
|
||||||
|
rounded,
|
||||||
|
stripClassPrefix,
|
||||||
|
} from "./helpers"
|
||||||
|
|
||||||
export const submissionRoutes = new Hono<ContestEnv>()
|
export const submissionRoutes = new Hono<ContestEnv>()
|
||||||
|
|
||||||
@@ -323,33 +330,6 @@ async function astOnlyByUser(where: SQL | undefined, userIds: number[]) {
|
|||||||
return byUser
|
return byUser
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
|
||||||
* 用户名模糊匹配到的账号。统计的两件事都从它出发:**筛哪些提交**(拿 id),
|
|
||||||
* 以及**花名册**(班级人数、谁没做,见下面的过滤)。
|
|
||||||
*
|
|
||||||
* 这里必须查 `user` 表而不是 `submission.username` —— 后者是提交那一刻冻结的
|
|
||||||
* 快照,学生改名之后旧提交还挂着旧名字,`ilike submission.username` 匹配不上。
|
|
||||||
*
|
|
||||||
* 生产快照实测(2026-09-08):24 级数媒两个班改成编号制用户名之后,85 人的
|
|
||||||
* 提交挂在旧名下。查 `ks249` 旧口径 0 条 / 新口径 7 条 —— 整个班 48 人全掉进
|
|
||||||
* 「一条没交」;查 `ks248` 20 条 / 54 条,13 个人的成绩查不出来。
|
|
||||||
*
|
|
||||||
* 返回**全部**匹配到的账号,禁用的和教师也在内 —— 「谁交过」不该受这两个条件
|
|
||||||
* 影响。花名册那一份在调用处再筛(未禁用 + 普通用户),教师和管理员不进分母。
|
|
||||||
*/
|
|
||||||
async function matchedUsers(username: string) {
|
|
||||||
return db
|
|
||||||
.select({
|
|
||||||
id: schema.user.id,
|
|
||||||
username: schema.user.username,
|
|
||||||
className: schema.user.className,
|
|
||||||
isDisabled: schema.user.isDisabled,
|
|
||||||
adminType: schema.user.adminType,
|
|
||||||
})
|
|
||||||
.from(schema.user)
|
|
||||||
.where(ilike(schema.user.username, `%${username}%`))
|
|
||||||
}
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* 两条提交列表的用户名筛选。**两边都要匹配**:
|
* 两条提交列表的用户名筛选。**两边都要匹配**:
|
||||||
*
|
*
|
||||||
|
|||||||
Reference in New Issue
Block a user