feat: 提示词分类按“指挥 AI 做网页”的场景重写标准,并带上题目和上一轮

- 以前只看单句、例子是学前端知识,把题目要求原样贴过去也判成 L6(首轮近一半 L6)。
  现在把题目要求和同一对话里上一轮的提问一起发给模型,只看学生自己想的部分:
  照搬题目功能清单算 L3,重复上一轮不高于上一轮
- L1–L6 说明跟着改成新的标准
- 生产库副本抽 60 条:新标准两次结果一致 57 条;L3 占 75%,L6 从 25 条降到 4 条

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
xuyueandClaude Opus 5.5 committed 2026-10-27 20:35:33 +08:00
1 parent 30630e8125
commit f41fda3c19
3 files changed
+61 -17

No files matched your search

+1 -1
View File
@@ -17,7 +17,7 @@ The backend was rewritten from Django in 2026-10 to cut memory use (~25 MB vs. 1
- `src/auth.ts` — Django-compatible password hashing; JWT login, no cookies: `POST /account/login` returns a 15-min HS256 `access_token` (sent as `Authorization: Bearer`, signed with `JWT_SECRET`, random per process if unset) and a 14-day opaque `refresh_token` (only its sha256 is stored, in `auth_refresh_token`; `POST /account/refresh` trades it for a new access token, logout / password change / deactivation delete it). An invalid or expired Bearer token is a 401 even on public endpoints so the client knows to refresh. `requireUser/requireAdmin/requireSuper` (401 / 403) - `src/auth.ts` — Django-compatible password hashing; JWT login, no cookies: `POST /account/login` returns a 15-min HS256 `access_token` (sent as `Authorization: Bearer`, signed with `JWT_SECRET`, random per process if unset) and a 14-day opaque `refresh_token` (only its sha256 is stored, in `auth_refresh_token`; `POST /account/refresh` trades it for a new access token, logout / password change / deactivation delete it). An invalid or expired Bearer token is a 401 even on public endpoints so the client knows to refresh. `requireUser/requireAdmin/requireSuper` (401 / 403)
- `src/routes/` — `account`, `tasks` (tutorial / challenge / creation), `files` (task assets + image upload), `submission`, `showcase` (mounted under `/submission/showcase`), `review` (mounted under `/submission/review`: teacher class × task wall — no `classname` means all classes —, students' peer works with `scope=class|all`), `prompt` - `src/routes/` — `account`, `tasks` (tutorial / challenge / creation), `files` (task assets + image upload), `submission`, `showcase` (mounted under `/submission/showcase`), `review` (mounted under `/submission/review`: teacher class × task wall — no `classname` means all classes —, students' peer works with `scope=class|all`), `prompt`
- Who can read a submission (`canView` in `routes/submission.ts`): staff, the owner, anyone if it's in the showcase, or any student who has submitted the same task (any class). Students' `GET /submission/` only lists their own. Peer works and the random-rating item are anonymous until the viewer rates them (`PUT /:id/score` returns the author). `flag` now only carries `present` (teacher's in-class presentation queue); the old four colors are no longer shown - Who can read a submission (`canView` in `routes/submission.ts`): staff, the owner, anyone if it's in the showcase, or any student who has submitted the same task (any class). Students' `GET /submission/` only lists their own. Peer works and the random-rating item are anonymous until the viewer rates them (`PUT /:id/score` returns the author). `flag` now only carries `present` (teacher's in-class presentation queue); the old four colors are no longer shown
- `src/services/` — `score.ts` (rating weights super×0.5, admin×0.3, normal×0.2, then Bayesian smoothing with C=3 and zones), `gradebook.ts`, `llm.ts` (OpenAI-compatible streaming via fetch; DeepSeek default, ARK for doubao models; chat context is the last 20 messages with older code blocks elided), `classifier.ts` (Bloom-level L1–L6 prompt tagging, runs in background), `cascade.ts` (Django's CASCADE deletes done by hand, plus the shared delete rules: `hasRatingsOrAwards`, `deleteRound`), `pyround.ts` (`pyRound`/`pySum` reproduce Python's `round()` and `sum()` so scores match the old system bit-for-bit) - `src/services/` — `score.ts` (rating weights super×0.5, admin×0.3, normal×0.2, then Bayesian smoothing with C=3 and zones), `gradebook.ts`, `llm.ts` (OpenAI-compatible streaming via fetch; DeepSeek default, ARK for doubao models; chat context is the last 20 messages with older code blocks elided), `classifier.ts` (Bloom-level L1–L6 prompt tagging, runs in background; the model sees the task brief and the previous prompt, temperature 0, prompts under 4 chars get no level), `cascade.ts` (Django's CASCADE deletes done by hand, plus the shared delete rules: `hasRatingsOrAwards`, `deleteRound`), `pyround.ts` (`pyRound`/`pySum` reproduce Python's `round()` and `sum()` so scores match the old system bit-for-bit)
- `src/ws.ts` — `/ws/prompt/<task_id>/?token=<access_token>` (code generation chat; the AI context and the page's chat include `conversation`, `edit` and `manual` messages) and `/ws/guidance/<task_id>/?token=...` (prompt coaching); one message per connection at a time (a message sent while the AI is still replying gets an error, payloads capped at 64 KB); on disconnect the pending user message is deleted and the LLM request aborted - `src/ws.ts` — `/ws/prompt/<task_id>/?token=<access_token>` (code generation chat; the AI context and the page's chat include `conversation`, `edit` and `manual` messages) and `/ws/guidance/<task_id>/?token=...` (prompt coaching); one message per connection at a time (a message sent while the AI is still replying gets an error, payloads capped at 64 KB); on disconnect the pending user message is deleted and the LLM request aborted
- `scripts/` — one-off CLIs: `create-superuser.ts`, `classify-prompts.ts`, `recalculate-scores.ts` - `scripts/` — one-off CLIs: `create-superuser.ts`, `classify-prompts.ts`, `recalculate-scores.ts`
- Roles: `super`, `admin`, `normal`. Task model is multi-table: `task_task` + one of `task_tutorial` / `task_challenge` / `task_creation` (`task_ptr_id`) - Roles: `super`, `admin`, `normal`. Task model is multi-table: `task_task` + one of `task_tutorial` / `task_challenge` / `task_creation` (`task_ptr_id`)
+6 -6
View File
@@ -27,12 +27,12 @@ export function sourceLabel(source: string | null | undefined) {
// 提问的 Bloom 等级(services/classifier.ts 后台打的标签) // 提问的 Bloom 等级(services/classifier.ts 后台打的标签)
export const PROMPT_LEVELS: Record<number, { name: string; desc: string }> = { export const PROMPT_LEVELS: Record<number, { name: string; desc: string }> = {
1: { name: "记忆", desc: "复述或识别知识点" }, 1: { name: "记忆", desc: "只说要做什么,没有自己的要求" },
2: { name: "理解", desc: "用自己的话解释概念" }, 2: { name: "理解", desc: "说出看到的现象或疑问" },
3: { name: "应用", desc: "把知识用到新情境" }, 3: { name: "应用", desc: "提出具体要求(颜色、布局、功能),照搬题目要求也算" },
4: { name: "分析", desc: "拆解结构、找出规律" }, 4: { name: "分析", desc: "说清在什么操作下出了什么问题,或推测原因" },
5: { name: "评价", desc: "按标准作出判断" }, 5: { name: "评价", desc: "按标准判断好坏并说明理由,或比较不同做法" },
6: { name: "创造", desc: "整合信息做出新东西" }, 6: { name: "创造", desc: "题目之外自己的设计:主题、玩法、交互" },
} }
export function levelLabel(level: number | null | undefined) { export function levelLabel(level: number | null | undefined) {
+54 -10
View File
@@ -1,19 +1,27 @@
import { all, get, run } from "../db" import { all, get, run } from "../db"
import { complete, llmConfigured } from "./llm" import { complete, llmConfigured } from "./llm"
const CLASSIFY_SYSTEM_PROMPT = `你是一个教育评估专家。根据布鲁姆认知分类学,分析以下学生在前端学习中发送给AI助手的一条提示词,判断该提示词所体现的认知层级。 // 学生在这里是“指挥 AI 做网页”,看的是这句提问里学生自己想了多少,不是要 AI 做的东西有多大:
// 把题目要求原样贴过去,功能列得再多也只是照着做(L3)
const CLASSIFY_SYSTEM_PROMPT = `你是一个教育评估专家。学生在网页制作课上让 AI 生成或修改网页,下面给出题目要求、学生上一轮的提问和这一轮的提问。请按布鲁姆认知分类学,判断“这一轮提问”体现的学生自己的思考层级。
层级定义: 层级定义:
- L1 记忆:能背诵HTML标签语法(例:"帮我写一个按钮") - L1 记忆:只说要做什么,没有自己的要求,或者原样复述题目名称(例:"帮我做一个象棋游戏"、"继续")
- L2 理解:能解释flex布局原理(例:"为什么这里不居中?") - L2 理解:用自己的话说出看到的现象或疑问,但没有具体要求(例:"为什么点了没反应"、"还是没有显示")
- L3 应用:能独立搭建页面结构(例:"用flex做导航栏,间距16px") - L3 应用:给出具体、可执行的要求,如颜色、布局、要加的功能;照搬或改写题目要求里的功能清单也属于这一级(例:"方块大一点,旁边加一个橡皮擦和清空按钮")
- L4 分析:能定位跨浏览器兼容性bug(例:"Safari中margin失效,原因?") - L4 分析:说清楚在什么操作下出了什么问题,或推测原因、拆解问题出在哪部分(例:"拖动绘制时松开鼠标还在画,是不是没监听 mouseup")
- L5 评价:能对比并选择方案(例:"对比Grid与Flex方案优劣") - L5 评价:按某个标准判断现有结果好不好并说明理由,或比较不同做法再做选择(例:"现在按钮太多,小学生看不懂,改成只留三个主要按钮")
- L6 创造:能设计并实现原创交互作品(例:"设计夜间/日间切换效果") - L6 创造:在题目要求之外提出自己的设计,如原创的主题、玩法、交互或整体风格,并说明它们怎么配合(例:"改成星空主题:转盘是星座,抽中稀有奖励时流星划过并解锁对应星座的图鉴")
判断要点:
- 只看学生自己想出来的部分。和题目要求重合的内容不算创造,要求写得长不等于层级高。
- 结合上一轮看:重复上一轮的提问,层级不高于上一轮。
只返回一个数字(1-6),不要解释。` 只返回一个数字(1-6),不要解释。`
const MIN_PROMPT_CHARS = 4 const MIN_PROMPT_CHARS = 4
const BRIEF_CHARS = 800
const PROMPT_CHARS = 1500
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)) const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms))
@@ -40,9 +48,45 @@ async function callLlm(content: string): Promise<number | null> {
} }
} }
// 题目要求去掉图片、压成一段,太长的截掉
function briefOf(content: string) {
const text = content
.replace(/!\[[^\]]*\]\([^)]*\)/g, "")
.replace(/\s+/g, " ")
.trim()
return text.length > BRIEF_CHARS ? text.slice(0, BRIEF_CHARS) + "……" : text
}
function clip(text: string) {
return text.length > PROMPT_CHARS ? text.slice(0, PROMPT_CHARS) + "……" : text
}
// 发给模型的:题目要求、同一对话里上一轮的提问(引导对话只和引导对话比)、这一轮的提问
function classifyInput(msg: { id: number; content: string; source: string; conversation_id: string; created: string }) {
const task = get<{ title: string; content: string }>(
`SELECT t.title, t.content FROM prompt_conversation c JOIN task_task t ON t.id = c.task_id WHERE c.id = $conv`,
{ conv: msg.conversation_id },
)
const prev = get<{ content: string }>(
`SELECT content FROM prompt_message
WHERE conversation_id = $conv AND role = 'user' AND source != 'edit'
AND (source = 'guidance') = $guidance
AND (created < $created OR (created = $created AND id < $id))
ORDER BY created DESC, id DESC LIMIT 1`,
{ conv: msg.conversation_id, guidance: msg.source === "guidance", created: msg.created, id: msg.id },
)
return [
`题目:${task?.title ?? "(未知)"}`,
`题目要求:${task ? briefOf(task.content) : "(无)"}`,
`上一轮提问:${prev ? clip(prev.content) : "(这是第一轮)"}`,
`这一轮提问:${clip(msg.content)}`,
].join("\n\n")
}
export async function classifyMessage(messageId: number): Promise<number | null> { export async function classifyMessage(messageId: number): Promise<number | null> {
const msg = get<{ content: string }>( const msg = get<{ id: number; content: string; source: string; conversation_id: string; created: string }>(
`SELECT content FROM prompt_message WHERE id = $id AND role = 'user' AND source != 'edit'`, `SELECT id, content, source, conversation_id, created FROM prompt_message
WHERE id = $id AND role = 'user' AND source != 'edit'`,
{ id: messageId }, { id: messageId },
) )
if (!msg) return null if (!msg) return null
@@ -51,7 +95,7 @@ export async function classifyMessage(messageId: number): Promise<number | null>
run(`UPDATE prompt_message SET prompt_level = NULL WHERE id = $id`, { id: messageId }) run(`UPDATE prompt_message SET prompt_level = NULL WHERE id = $id`, { id: messageId })
return null return null
} }
const level = await callLlm(msg.content) const level = await callLlm(classifyInput(msg))
if (level !== null) { if (level !== null) {
run(`UPDATE prompt_message SET prompt_level = $level WHERE id = $id`, { run(`UPDATE prompt_message SET prompt_level = $level WHERE id = $id`, {
level, level,