复合评分

把一个判断拆分为原子分数,再用你控制的权重进行组合——复合指标由 Dcision 计算,并可在策略中使用。基于“简历筛选”模板构建。

把一个复杂的判断拆分为相互独立的维度,分别为每个维度打分,再用你控制的权重将它们组合起来——权重写在决策中,而不是写在提示词里。一个宽泛的问题(“这是一位好的候选人吗?”)会把几个判断藏在一个回答背后;原子分数把它们暴露出来,让你可以检查、调整和组合。

好处:成本、可靠性和速度。TypeSafe 的指南:复合评分。

为什么有效

  • 每个维度都是一个简短、字面化的问题——正是 Jev 最擅长回答的那种——而且它们全部在一次调用中返回。
  • 计算留在模型之外。Dcision 精确地计算加权组合;Jev 擅长判断,却不擅长算术。
  • 你能看到分数是如何构成的。每一项都在响应中,所以当排名出乎意料时,你知道该改哪个维度——或哪个权重。
  • 同一组回答,多种排名。基于同样四个问题的两个复合指标不会产生额外成本:没有新问题,也没有新的引擎调用。

何时使用

基于多个标准进行排序或资格评估:线索评分、简历匹配度、供应商风险、内容质量、待办事项的优先级。每当你想在一条指令中写下“综合考虑 A、B 和 C”时,就分别询问 A、B 和 C,再把它们组合起来。

如何对应到 Dcision

在 TypeSafe 的指南中在 Dcision 中
每个维度一个 score 问题每个维度一个 score(或 probability、或 choice)问题
py = answers["python_depth"].score / 4自动完成:score 项按 (weighted level − 1) / (levels − 1) 归一化到 0–1
在你的代码中写 ic_score = 0.40 * py + 0.10 * lead + …一个使用这些权重的复合指标:Σ(weight × value) / Σ|weight|
在代码中排序和过滤在响应中读取 composites,并在规则中使用复合指标来设定 action

构建

在应用中

打开模板,选择简历筛选(复合评分)。将决策命名为 Resume Screening,使其 slug 为 resume-screening,然后点击创建决策。

State 有一个必填的字符串字段 resume。

四个 score 问题,每个五个等级,每个维度一个:

问题等级,从低到高
python_depthnone、basic、working、strong、expert
team_leadershipnone、informal、tech lead、manager、manager of managers
system_designnone、basic、working、strong、expert
generalistnarrow、some、broad、very broad、full stack and ops

在复合指标卡片(编辑器标签页)中,点击两次添加复合指标:

  • senior_ic——Senior IC fit: depth and design first:0.4 × python_depth + 0.1 × team_leadership + 0.4 × system_design + 0.1 × generalist。
  • eng_manager——Engineering Manager fit: leadership first:0.15 × python_depth + 0.4 × team_leadership + 0.2 × system_design + 0.25 × generalist。

在策略中,复合指标出现在字段列表的复合指标下:

  1. IF senior_ic ≥ 0.7——THEN continue。
  2. IF eng_manager ≥ 0.7——THEN continue。
  3. IF senior_ic < 0.35,+ AND eng_manager < 0.35——THEN block。
  4. IF senior_ic < 0.7——THEN escalate:中间区段交给招聘人员。

在 Playground 中测试几份简历——复合指标卡片会显示这两个值——然后部署 v1。

决策 schema

{
  "stateSchema": {
    "kind": "object",
    "fields": [{ "key": "resume", "type": "string", "required": true, "description": "Resume text" }]
  },
  "questions": [
    { "key": "python_depth", "type": "score", "instructions": "How deep is the candidate's Python experience?", "scale": ["none", "basic", "working", "strong", "expert"] },
    { "key": "team_leadership", "type": "score", "instructions": "How much team leadership has the candidate shown?", "scale": ["none", "informal", "tech lead", "manager", "manager of managers"] },
    { "key": "system_design", "type": "score", "instructions": "How strong is the candidate's system design experience?", "scale": ["none", "basic", "working", "strong", "expert"] },
    { "key": "generalist", "type": "score", "instructions": "How broad is the candidate's experience across the stack?", "scale": ["narrow", "some", "broad", "very broad", "full stack and ops"] }
  ],
  "composites": [
    {
      "key": "senior_ic",
      "description": "Senior IC fit: depth and design first",
      "terms": [
        { "question": "python_depth", "weight": 0.4 },
        { "question": "team_leadership", "weight": 0.1 },
        { "question": "system_design", "weight": 0.4 },
        { "question": "generalist", "weight": 0.1 }
      ]
    },
    {
      "key": "eng_manager",
      "description": "Engineering Manager fit: leadership first",
      "terms": [
        { "question": "python_depth", "weight": 0.15 },
        { "question": "team_leadership", "weight": 0.4 },
        { "question": "system_design", "weight": 0.2 },
        { "question": "generalist", "weight": 0.25 }
      ]
    }
  ],
  "policies": [
    { "field": "senior_ic", "on": "output", "operator": "gte", "value": 0.7, "action": "continue" },
    { "field": "eng_manager", "on": "output", "operator": "gte", "value": 0.7, "action": "continue" },
    {
      "field": "senior_ic",
      "on": "output",
      "operator": "lt",
      "value": 0.35,
      "and": [{ "field": "eng_manager", "on": "output", "operator": "lt", "value": 0.35 }],
      "action": "block"
    },
    { "field": "senior_ic", "on": "output", "operator": "lt", "value": 0.7, "action": "escalate" }
  ]
}

调用它

curl -X POST https://api.dcision.io/v1/decisions/resume-screening \
  -H "Authorization: Bearer $DCISION_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: candidate-3307" \
  -d '{
    "state": {
      "resume": "8 years building Python services (Django, FastAPI). Designed the event pipeline for 20M users at a fintech. Tech lead of 4 engineers for 2 years. Some Terraform and on-call experience."
    }
  }'
200 OK
{
  "decision_id": "dec_Hn6Vx2Kq8Zt4Wm1Lp7Rc",
  "execution_id": "exec_Cs9Lw3Tq7Xm1Vn5Kd8Pz",
  "schema": "resume-screening",
  "version": 1,
  "result": {
    "python_depth": "strong",
    "team_leadership": "tech lead",
    "system_design": "strong",
    "generalist": "broad"
  },
  "confidence": {
    "python_depth": 0.625,
    "team_leadership": 0.8167,
    "system_design": 0.75,
    "generalist": 0.7083
  },
  "scores": {
    "python_depth": 4.35,
    "team_leadership": 3.02,
    "system_design": 4.1,
    "generalist": 2.85
  },
  "composites": { "senior_ic": 0.7418, "eng_manager": 0.5982 },
  "action": "continue",
  "action_reason": { "type": "policy", "rule": 0 },
  "metrics": {
    "latency_ms": 391,
    "engine": "jev",
    "model": "jev-1.13.0",
    "estimated_cost_usd": 0.0000189,
    "input_tokens": 450,
    "output_tokens": 47
  }
}

senior_ic 是如何由 scores 中的加权等级构成的——五个等级,所以每一项都是 (level − 1) / 4:

python_depth     (4.35 − 1) / 4 = 0.8375   × 0.4
team_leadership  (3.02 − 1) / 4 = 0.505    × 0.1
system_design    (4.10 − 1) / 4 = 0.775    × 0.4
generalist       (2.85 − 1) / 4 = 0.4625   × 0.1
senior_ic = 0.335 + 0.0505 + 0.31 + 0.04625 = 0.74175 → 0.7418   (the weights add up to 1)

senior_ic 为 0.74,高于 0.7:规则 0 匹配,候选人作为 Senior IC 进入下一步。注意,result 保存的是每个问题最可能的等级,而复合指标使用的是加权等级——python_depth 是 strong,但由于 expert 占 40%,它的加权等级是 4.35。

据此行动

按你正在招聘的岗位对应的复合指标为候选人排序,并让行动决定下一步:

const screened = await Promise.all(resumes.map((resume) => decide("resume-screening", { resume: resume.text })));

const shortlist = screened
  .map((decision, index) => ({ candidate: resumes[index], decision }))
  .filter(({ decision }) => decision.action !== "block")
  .sort((a, b) => b.decision.composites.senior_ic - a.decision.composites.senior_ic)
  .slice(0, 10);

这里的 decide 是对 POST /v1/decisions/{slug} 的一个小封装——可用于生产环境的版本参见幂等重试。

调优

  • 改权重,而不是改指令。如果排名靠前的候选人不符合你的预期,调整权重并部署新版本;问题——以及它们的校准——保持不变。
  • 描述每一个等级。简短的标签也能用,但为每个等级提供评分细则对引擎来说更清晰。结构化等级在 API 中保留简短的标签:{ "label": "strong", "rubric": "primary language across several projects" }。
  • 以问题的形式添加维度。新增一个标准就是多一个问题、多一项——仍然只需一次调用。
  • 对危险信号使用负权重,例如在复合指标中加入 − 2 × job_hopping。
  • 设阈值,不要插值。加权等级 4.35 的意思是“介于 strong 和 expert 之间,更接近 strong”——适合排序和阈值,而不适合还原出确切的年数。
  • 让人参与其中。筛选会影响到人:中间区段会被升级,即使是 continue 也应该进入人工面试,而不是自动决定。

相关:复合指标 · 线索资格评估

本页内容