feat: 提示词分类按“指挥 AI 做网页”的场景重写标准,并带上题目和上一轮
- 以前只看单句、例子是学前端知识,把题目要求原样贴过去也判成 L6(首轮近一半 L6)。 现在把题目要求和同一对话里上一轮的提问一起发给模型,只看学生自己想的部分: 照搬题目功能清单算 L3,重复上一轮不高于上一轮 - L1–L6 说明跟着改成新的标准 - 生产库副本抽 60 条:新标准两次结果一致 57 条;L3 占 75%,L6 从 25 条降到 4 条 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
30630e8125
commit
f41fda3c19
3 files changed
+61
-17
No files matched your search
@@ -17,7 +17,7 @@ The backend was rewritten from Django in 2026-10 to cut memory use (~25 MB vs. 1
|
||||
- `src/auth.ts` — Django-compatible password hashing; JWT login, no cookies: `POST /account/login` returns a 15-min HS256 `access_token` (sent as `Authorization: Bearer`, signed with `JWT_SECRET`, random per process if unset) and a 14-day opaque `refresh_token` (only its sha256 is stored, in `auth_refresh_token`; `POST /account/refresh` trades it for a new access token, logout / password change / deactivation delete it). An invalid or expired Bearer token is a 401 even on public endpoints so the client knows to refresh. `requireUser/requireAdmin/requireSuper` (401 / 403)
|
||||
- `src/routes/` — `account`, `tasks` (tutorial / challenge / creation), `files` (task assets + image upload), `submission`, `showcase` (mounted under `/submission/showcase`), `review` (mounted under `/submission/review`: teacher class × task wall — no `classname` means all classes —, students' peer works with `scope=class|all`), `prompt`
|
||||
- Who can read a submission (`canView` in `routes/submission.ts`): staff, the owner, anyone if it's in the showcase, or any student who has submitted the same task (any class). Students' `GET /submission/` only lists their own. Peer works and the random-rating item are anonymous until the viewer rates them (`PUT /:id/score` returns the author). `flag` now only carries `present` (teacher's in-class presentation queue); the old four colors are no longer shown
|
||||
- `src/services/` — `score.ts` (rating weights super×0.5, admin×0.3, normal×0.2, then Bayesian smoothing with C=3 and zones), `gradebook.ts`, `llm.ts` (OpenAI-compatible streaming via fetch; DeepSeek default, ARK for doubao models; chat context is the last 20 messages with older code blocks elided), `classifier.ts` (Bloom-level L1–L6 prompt tagging, runs in background), `cascade.ts` (Django's CASCADE deletes done by hand, plus the shared delete rules: `hasRatingsOrAwards`, `deleteRound`), `pyround.ts` (`pyRound`/`pySum` reproduce Python's `round()` and `sum()` so scores match the old system bit-for-bit)
|
||||
- `src/services/` — `score.ts` (rating weights super×0.5, admin×0.3, normal×0.2, then Bayesian smoothing with C=3 and zones), `gradebook.ts`, `llm.ts` (OpenAI-compatible streaming via fetch; DeepSeek default, ARK for doubao models; chat context is the last 20 messages with older code blocks elided), `classifier.ts` (Bloom-level L1–L6 prompt tagging, runs in background; the model sees the task brief and the previous prompt, temperature 0, prompts under 4 chars get no level), `cascade.ts` (Django's CASCADE deletes done by hand, plus the shared delete rules: `hasRatingsOrAwards`, `deleteRound`), `pyround.ts` (`pyRound`/`pySum` reproduce Python's `round()` and `sum()` so scores match the old system bit-for-bit)
|
||||
- `src/ws.ts` — `/ws/prompt/<task_id>/?token=<access_token>` (code generation chat; the AI context and the page's chat include `conversation`, `edit` and `manual` messages) and `/ws/guidance/<task_id>/?token=...` (prompt coaching); one message per connection at a time (a message sent while the AI is still replying gets an error, payloads capped at 64 KB); on disconnect the pending user message is deleted and the LLM request aborted
|
||||
- `scripts/` — one-off CLIs: `create-superuser.ts`, `classify-prompts.ts`, `recalculate-scores.ts`
|
||||
- Roles: `super`, `admin`, `normal`. Task model is multi-table: `task_task` + one of `task_tutorial` / `task_challenge` / `task_creation` (`task_ptr_id`)
|
||||
|
||||
@@ -27,12 +27,12 @@ export function sourceLabel(source: string | null | undefined) {
|
||||
|
||||
// 提问的 Bloom 等级(services/classifier.ts 后台打的标签)
|
||||
export const PROMPT_LEVELS: Record<number, { name: string; desc: string }> = {
|
||||
1: { name: "记忆", desc: "复述或识别知识点" },
|
||||
2: { name: "理解", desc: "用自己的话解释概念" },
|
||||
3: { name: "应用", desc: "把知识用到新情境" },
|
||||
4: { name: "分析", desc: "拆解结构、找出规律" },
|
||||
5: { name: "评价", desc: "按标准作出判断" },
|
||||
6: { name: "创造", desc: "整合信息做出新东西" },
|
||||
1: { name: "记忆", desc: "只说要做什么,没有自己的要求" },
|
||||
2: { name: "理解", desc: "说出看到的现象或疑问" },
|
||||
3: { name: "应用", desc: "提出具体要求(颜色、布局、功能),照搬题目要求也算" },
|
||||
4: { name: "分析", desc: "说清在什么操作下出了什么问题,或推测原因" },
|
||||
5: { name: "评价", desc: "按标准判断好坏并说明理由,或比较不同做法" },
|
||||
6: { name: "创造", desc: "题目之外自己的设计:主题、玩法、交互" },
|
||||
}
|
||||
|
||||
export function levelLabel(level: number | null | undefined) {
|
||||
|
||||
@@ -1,19 +1,27 @@
|
||||
import { all, get, run } from "../db"
|
||||
import { complete, llmConfigured } from "./llm"
|
||||
|
||||
const CLASSIFY_SYSTEM_PROMPT = `你是一个教育评估专家。根据布鲁姆认知分类学,分析以下学生在前端学习中发送给AI助手的一条提示词,判断该提示词所体现的认知层级。
|
||||
// 学生在这里是“指挥 AI 做网页”,看的是这句提问里学生自己想了多少,不是要 AI 做的东西有多大:
|
||||
// 把题目要求原样贴过去,功能列得再多也只是照着做(L3)
|
||||
const CLASSIFY_SYSTEM_PROMPT = `你是一个教育评估专家。学生在网页制作课上让 AI 生成或修改网页,下面给出题目要求、学生上一轮的提问和这一轮的提问。请按布鲁姆认知分类学,判断“这一轮提问”体现的学生自己的思考层级。
|
||||
|
||||
层级定义:
|
||||
- L1 记忆:能背诵HTML标签语法(例:"帮我写一个按钮")
|
||||
- L2 理解:能解释flex布局原理(例:"为什么这里不居中?")
|
||||
- L3 应用:能独立搭建页面结构(例:"用flex做导航栏,间距16px")
|
||||
- L4 分析:能定位跨浏览器兼容性bug(例:"Safari中margin失效,原因?")
|
||||
- L5 评价:能对比并选择方案(例:"对比Grid与Flex方案优劣")
|
||||
- L6 创造:能设计并实现原创交互作品(例:"设计夜间/日间切换效果")
|
||||
- L1 记忆:只说要做什么,没有自己的要求,或者原样复述题目名称(例:"帮我做一个象棋游戏"、"继续")
|
||||
- L2 理解:用自己的话说出看到的现象或疑问,但没有具体要求(例:"为什么点了没反应"、"还是没有显示")
|
||||
- L3 应用:给出具体、可执行的要求,如颜色、布局、要加的功能;照搬或改写题目要求里的功能清单也属于这一级(例:"方块大一点,旁边加一个橡皮擦和清空按钮")
|
||||
- L4 分析:说清楚在什么操作下出了什么问题,或推测原因、拆解问题出在哪部分(例:"拖动绘制时松开鼠标还在画,是不是没监听 mouseup")
|
||||
- L5 评价:按某个标准判断现有结果好不好并说明理由,或比较不同做法再做选择(例:"现在按钮太多,小学生看不懂,改成只留三个主要按钮")
|
||||
- L6 创造:在题目要求之外提出自己的设计,如原创的主题、玩法、交互或整体风格,并说明它们怎么配合(例:"改成星空主题:转盘是星座,抽中稀有奖励时流星划过并解锁对应星座的图鉴")
|
||||
|
||||
判断要点:
|
||||
- 只看学生自己想出来的部分。和题目要求重合的内容不算创造,要求写得长不等于层级高。
|
||||
- 结合上一轮看:重复上一轮的提问,层级不高于上一轮。
|
||||
|
||||
只返回一个数字(1-6),不要解释。`
|
||||
|
||||
const MIN_PROMPT_CHARS = 4
|
||||
const BRIEF_CHARS = 800
|
||||
const PROMPT_CHARS = 1500
|
||||
|
||||
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms))
|
||||
|
||||
@@ -40,9 +48,45 @@ async function callLlm(content: string): Promise<number | null> {
|
||||
}
|
||||
}
|
||||
|
||||
// 题目要求去掉图片、压成一段,太长的截掉
|
||||
function briefOf(content: string) {
|
||||
const text = content
|
||||
.replace(/!\[[^\]]*\]\([^)]*\)/g, "")
|
||||
.replace(/\s+/g, " ")
|
||||
.trim()
|
||||
return text.length > BRIEF_CHARS ? text.slice(0, BRIEF_CHARS) + "……" : text
|
||||
}
|
||||
|
||||
function clip(text: string) {
|
||||
return text.length > PROMPT_CHARS ? text.slice(0, PROMPT_CHARS) + "……" : text
|
||||
}
|
||||
|
||||
// 发给模型的:题目要求、同一对话里上一轮的提问(引导对话只和引导对话比)、这一轮的提问
|
||||
function classifyInput(msg: { id: number; content: string; source: string; conversation_id: string; created: string }) {
|
||||
const task = get<{ title: string; content: string }>(
|
||||
`SELECT t.title, t.content FROM prompt_conversation c JOIN task_task t ON t.id = c.task_id WHERE c.id = $conv`,
|
||||
{ conv: msg.conversation_id },
|
||||
)
|
||||
const prev = get<{ content: string }>(
|
||||
`SELECT content FROM prompt_message
|
||||
WHERE conversation_id = $conv AND role = 'user' AND source != 'edit'
|
||||
AND (source = 'guidance') = $guidance
|
||||
AND (created < $created OR (created = $created AND id < $id))
|
||||
ORDER BY created DESC, id DESC LIMIT 1`,
|
||||
{ conv: msg.conversation_id, guidance: msg.source === "guidance", created: msg.created, id: msg.id },
|
||||
)
|
||||
return [
|
||||
`题目:${task?.title ?? "(未知)"}`,
|
||||
`题目要求:${task ? briefOf(task.content) : "(无)"}`,
|
||||
`上一轮提问:${prev ? clip(prev.content) : "(这是第一轮)"}`,
|
||||
`这一轮提问:${clip(msg.content)}`,
|
||||
].join("\n\n")
|
||||
}
|
||||
|
||||
export async function classifyMessage(messageId: number): Promise<number | null> {
|
||||
const msg = get<{ content: string }>(
|
||||
`SELECT content FROM prompt_message WHERE id = $id AND role = 'user' AND source != 'edit'`,
|
||||
const msg = get<{ id: number; content: string; source: string; conversation_id: string; created: string }>(
|
||||
`SELECT id, content, source, conversation_id, created FROM prompt_message
|
||||
WHERE id = $id AND role = 'user' AND source != 'edit'`,
|
||||
{ id: messageId },
|
||||
)
|
||||
if (!msg) return null
|
||||
@@ -51,7 +95,7 @@ export async function classifyMessage(messageId: number): Promise<number | null>
|
||||
run(`UPDATE prompt_message SET prompt_level = NULL WHERE id = $id`, { id: messageId })
|
||||
return null
|
||||
}
|
||||
const level = await callLlm(msg.content)
|
||||
const level = await callLlm(classifyInput(msg))
|
||||
if (level !== null) {
|
||||
run(`UPDATE prompt_message SET prompt_level = $level WHERE id = $id`, {
|
||||
level,
|
||||
|
||||
Reference in new issue
Block a user