Freeze native evaluation conditions for result attribution

docs/en/reference/judge-api.md

@@ -12,8 +12,9 @@ The judge server exposes a REST API for registering runs, submitting archives, p
 |--------|------------|-----------|
 | `sforge run` (host) | admin secret | `register`, auto-eval `submit`, full `history`, `usage` |
 | Agent in the work container | session token | `submit`, `result`, agent `history`, `llm/...`, game endpoints |
+| Feedback reader | feedback relay ID | read-only registration `context` and that registration's agent `result` |
 
-The admin secret (`SFORGE_ADMIN_SECRET`) never enters a container. The work container gets only its session token (`SFORGE_TOKEN`) and the judge URL (`SFORGE_JUDGE_URL`).
+The admin secret (`SFORGE_ADMIN_SECRET`) never enters a container. The work container gets its session token (`SFORGE_TOKEN`), the judge URL (`SFORGE_JUDGE_URL`) and a read-only feedback URL (`SFORGE_FEEDBACK_URL`). Treat the feedback URL as a credential for reading feedback: it cannot submit, read auto-evaluations or access another run/task. Results with a recorded registration binding are further restricted to that registration.
 
 ## Session Registration
 
@@ -35,11 +36,23 @@ Optional fields: `judge_cpu_limit`, `judge_mem_limit`, `max_agent_submissions`,
 **Response:**
 
 ```json
-{"token": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"}
+{
+  "token": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4",
+  "feedback_relay_id": "<opaque-read-capability>",
+  "evaluation_context": {
+    "schema_version": "sforge_run_evaluation_context_v1",
+    "task_id": "ad_placement_optimization",
+    "run_id": "run-001",
+    "registration_id": "<server-assigned-id>",
+    "evaluation_conditions": null
+  }
+}
 ```
 
 The token identifies the task and run and counts agent rounds (`agent-1`, `agent-2`, ...) and auto-eval rounds (`auto-1`, ...) separately.
 
+When evaluation conditions are known, registering the same run/task with identical conditions and registration options returns the original token, relay and counters. Changed conditions or options return `409`; use a fresh `run_id`. A retry cannot reset the submission budget or switch the run's model gateway. Registrations with unknown conditions retain the previous registration behavior and carry no verified attribution claim.
+
 ## Submit Archive
 
 ```http
@@ -57,6 +70,7 @@ kind: agent
 | `archive` | yes | `.tar.gz` of the workspace; filtered by the task's submission paths |
 | `kind` | no | `agent` (default) or `auto`; `auto` also needs `admin_secret` |
 | `admin_secret` | for `auto` | Admin secret |
+| `include_receipt` | no | `true` adds the accepted archive receipt; default `false` preserves the existing response |
 
 **Response:**
 
@@ -109,6 +123,63 @@ The report is the grader's `result.json` plus timing. The 0–100 score is not i
 | `completed` | Graded; `report` is available (it may still be `valid: false`) |
 | `error` | Grading failed outright; see `error` |
 
+## Evaluation Conditions and Read-Only Feedback
+
+Read the original registration context once before accepting measurements:
+
+```http
+GET /api/v1/feedback/{feedback_relay_id}/context
+```
+
+It returns the same `sforge_run_evaluation_context_v1` object as registration. `evaluation_conditions`, when available, is an object with exactly these fields:
+
+| Field | Meaning |
+|-------|---------|
+| `schema_version` | `sforge_evaluation_conditions_v1` |
+| `task_sha256` | SHA-256 of the `task.toml` bytes parsed by the task loader |
+| `benchmark_sha256` | SHA-256 of the `benchmark.toml` bytes parsed by the benchmark loader |
+| `task_spec_sha256` | SHA-256 of the effective logical task, including instructions, judge command, score direction/rescaling and declared Blob path/digest inventory; no host paths |
+| `native_source_sha256` | SHA-256 inventory of the service's installed Python framework source, observed once at boot |
+| `judge_image_id` | Actual local Docker content ID (`sha256:...`), resolved at registration and used for grading |
+| `execution_config_sha256` | SHA-256 of effective judge CPU/memory/environment/network, timeout and platform |
+| `score_direction` | `maximize` or `minimize` from that frozen task |
+| `selection` | `valid_then_score`: existing valid/completed filtering, raw-score direction and first-tie selection |
+
+The server freezes the loaded task, effective config and backend for accepted work. Changing files or retagging an image after registration does not change its context or the image used by subsequent submissions. Blob downloads and SHA checks remain owned by the normal image-build path; the Docker ID identifies the evaluator and data actually inside the image. The digest is not a claim of deterministic or hermetic grading and does not attest mutable external services/data. No evaluator contents or raw environment values are exposed by this contract.
+
+Poll one result with both its original feedback and provenance:
+
+```http
+GET /api/v1/feedback/{feedback_relay_id}/result/{submission_id}?include_receipt=true
+```
+
+This is the ordinary result response plus:
+
+```json
+{
+  "receipt": {
+    "schema_version": "sforge_submission_receipt_v1",
+    "task_id": "ad_placement_optimization",
+    "round_id": "agent-1",
+    "archive_sha256": "<sha256-of-accepted-upload-bytes>",
+    "archive_bytes": 1234,
+    "evaluation_context": {
+      "schema_version": "sforge_run_evaluation_context_v1",
+      "task_id": "ad_placement_optimization",
+      "run_id": "run-001",
+      "registration_id": "<original-server-assigned-id>",
+      "evaluation_conditions": null
+    }
+  }
+}
+```
+
+The context above shows the unknown-conditions case; known conditions contain the object described in the table. The archive digest covers the accepted compressed upload before submission-path filtering. The same optional receipt is available on `GET /api/v1/result/{submission_id}?include_receipt=true` and `POST /api/v1/submit` with the form field `include_receipt=true`. Omitting the flag preserves the original feedback body, including the raw-score-only report.
+
+A reader can compare this result's context to its installation snapshot with one GET per result. A missing context, unknown conditions or a mismatch must stay unknown for measurement attribution; keep the native feedback usable. Context/result reads never rescan source, inspect the current image, build an image or download a Blob. Unsupported backends, game sessions, unprepared images and unobserved/manual task specifications return `evaluation_conditions: null`. Historical results missing provenance return a null context/digest instead of borrowing current conditions. This interface adds no checkpoint/restore mechanism.
+
+The relay returns `404` for foreign runs/tasks/recorded registrations, auto-evaluations and unknown IDs. Legacy results without a registration binding remain visible only within their matching run/task and carry no recovered attribution. The relay has no submit, history, model-gateway or admin route. Existing session-token APIs and generated submit-client behavior remain available; retained-upload and asynchronous-client options are separate client features.
+
 ## Run History
 
 ```http

docs/zh/reference/judge-api.md

@@ -12,8 +12,9 @@ Judge server 提供 REST API，用于登记运行、提交 archive、轮询结
 |--------|------|------|
 | `sforge run`（宿主机） | admin secret | `register`、auto-eval 的 `submit`、完整 `history`、`usage` |
 | work 容器里的 Agent | session token | `submit`、`result`、Agent 视角的 `history`、`llm/...`、游戏端点 |
+| 反馈读取方 | feedback relay ID | 只读登记 `context` 与该登记的 Agent `result` |
 
-Admin secret（`SFORGE_ADMIN_SECRET`）从不进入容器。Work 容器只拿到自己的 session token（`SFORGE_TOKEN`）和 judge URL（`SFORGE_JUDGE_URL`）。
+Admin secret（`SFORGE_ADMIN_SECRET`）从不进入容器。Work 容器拿到自己的 session token（`SFORGE_TOKEN`）、judge URL（`SFORGE_JUDGE_URL`）和只读反馈 URL（`SFORGE_FEEDBACK_URL`）。反馈 URL 是读取反馈的凭据，不能用于提交、读取 auto-eval 或访问其他 run/task。已记录登记绑定的结果进一步限定在该登记内。
 
 ## 登记会话
 
@@ -35,11 +36,23 @@ Content-Type: application/json
 **响应：**
 
 ```json
-{"token": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4"}
+{
+  "token": "a1b2c3d4e5f6a1b2c3d4e5f6a1b2c3d4",
+  "feedback_relay_id": "<opaque-read-capability>",
+  "evaluation_context": {
+    "schema_version": "sforge_run_evaluation_context_v1",
+    "task_id": "ad_placement_optimization",
+    "run_id": "run-001",
+    "registration_id": "<server-assigned-id>",
+    "evaluation_conditions": null
+  }
+}
 ```
 
 Token 标识任务和运行，并分别计数 Agent 轮次（`agent-1`、`agent-2`、...）和 auto-eval 轮次（`auto-1`、...）。
 
+评测条件已知时，同一 run/task 在条件和登记选项完全相同时返回原 token、relay 与计数器；条件或选项变化返回 `409`，需要新的 `run_id`。重试不能重置提交预算或切换运行的模型网关。条件未知的登记保留原有登记行为，不声明归因已验证。
+
 ## 提交 Archive
 
 ```http
@@ -57,6 +70,7 @@ kind: agent
 | `archive` | 是 | 工作区的 `.tar.gz`，按该题的提交路径过滤 |
 | `kind` | 否 | `agent`（默认）或 `auto`；`auto` 还需要 `admin_secret` |
 | `admin_secret` | `auto` 时 | Admin secret |
+| `include_receipt` | 否 | `true` 附带已接受归档的 receipt；默认 `false` 保留原响应 |
 
 **响应：**
 
@@ -109,6 +123,63 @@ GET /api/v1/result/{submission_id}
 | `completed` | 评分完成，`report` 可用（仍可能是 `valid: false`） |
 | `error` | 评分直接失败，见 `error` |
 
+## 评测条件与只读反馈
+
+接受测量前，先读取一次原登记上下文：
+
+```http
+GET /api/v1/feedback/{feedback_relay_id}/context
+```
+
+响应与登记返回的 `sforge_run_evaluation_context_v1` 对象相同。条件可用时，`evaluation_conditions` 对象恰好包含以下字段：
+
+| 字段 | 含义 |
+|------|------|
+| `schema_version` | `sforge_evaluation_conditions_v1` |
+| `task_sha256` | Task loader 实际解析的 `task.toml` 字节的 SHA-256 |
+| `benchmark_sha256` | Benchmark loader 实际解析的 `benchmark.toml` 字节的 SHA-256 |
+| `task_spec_sha256` | 有效逻辑任务的 SHA-256，包括指令、judge 命令、分数方向/缩放及 Blob 路径和摘要清单；不含宿主机路径 |
+| `native_source_sha256` | 服务启动时一次观察的已安装 Python 框架源码清单的 SHA-256 |
+| `judge_image_id` | 登记时解析、评分时实际使用的本地 Docker 内容 ID（`sha256:...`） |
+| `execution_config_sha256` | 有效 judge CPU/内存/环境/网络、超时及平台的 SHA-256 |
+| `score_direction` | 冻结任务的 `maximize` 或 `minimize` |
+| `selection` | `valid_then_score`：已有的有效且已完成筛选、原始分数方向比较与并列取首条规则 |
+
+服务为已接受的工作冻结已加载任务、有效配置及 backend。登记后修改文件或镜像标签，不会改变原上下文或后续提交实际使用的镜像。Blob 下载与 SHA 校验仍归原生镜像构建路径负责；Docker ID 标识镜像中实际的评测器与数据。摘要不代表评分确定性或封闭性，也不证明外部可变服务/数据未变。协议不暴露评测器内容或原始环境变量值。
+
+一次请求读取结果原文与来源：
+
+```http
+GET /api/v1/feedback/{feedback_relay_id}/result/{submission_id}?include_receipt=true
+```
+
+响应在原结果之上增加：
+
+```json
+{
+  "receipt": {
+    "schema_version": "sforge_submission_receipt_v1",
+    "task_id": "ad_placement_optimization",
+    "round_id": "agent-1",
+    "archive_sha256": "<sha256-of-accepted-upload-bytes>",
+    "archive_bytes": 1234,
+    "evaluation_context": {
+      "schema_version": "sforge_run_evaluation_context_v1",
+      "task_id": "ad_placement_optimization",
+      "run_id": "run-001",
+      "registration_id": "<original-server-assigned-id>",
+      "evaluation_conditions": null
+    }
+  }
+}
+```
+
+上例展示条件未知的情况；条件已知时包含表中描述的对象。归档摘要覆盖按提交路径过滤之前收到的压缩上传字节。`GET /api/v1/result/{submission_id}?include_receipt=true` 与带 `include_receipt=true` 表单字段的 `POST /api/v1/submit` 也提供同一可选 receipt。省略开关时保留原反馈正文，报告仍只含原始分数。
+
+读取方每个结果只需一次 GET，即可将其上下文与安装时快照比较。上下文缺失、条件未知或不匹配时，测量归因必须保留未知，原生反馈仍可使用。读取 context/result 不会重扫源码、查询当前镜像、构建镜像或下载 Blob。不支持的 backend、游戏会话、未准备镜像及未经 loader 观察的手工 TaskSpec 返回 `evaluation_conditions: null`。旧结果缺少来源时返回 null 上下文/摘要，不借用当前条件。本接口不增加 checkpoint/restore 机制。
+
+对其他 run/task/已记录登记、auto-eval 和未知 ID，relay 返回 `404`。没有登记绑定的旧结果仍只在匹配的 run/task 内可见，归因信息不会被恢复。Relay 没有提交、历史、模型网关或 admin 路由。已有 session-token API 和生成的提交客户端保持可用；保留上传归档与异步客户端选项属于独立的客户端功能。
+
 ## 运行历史
 
 ```http

pyproject.toml

@@ -22,6 +22,7 @@ dependencies = [
     "tqdm",
     "rich",
     "fastapi",
+    "pydantic>=2.0",
     "python-multipart",
     "uvicorn[standard]",
     "jinja2",

sforge/harness/evaluation_conditions.py

@@ -0,0 +1,129 @@
+# Copyright (c) 2026 ByteDance Ltd. and/or its affiliates
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Service-observed evaluation inputs, distinct from a caller's version labels."""
+
+from __future__ import annotations
+
+import hashlib
+import json
+import re
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Literal
+
+from pydantic import BaseModel, ConfigDict, Field
+
+from sforge.harness.backend import ContainerBackend
+from sforge.harness.config import SForgeConfig
+from sforge.harness.run_evaluation import judge_container_options
+from sforge.harness.task_spec import TaskSpec, task_spec_sha256
+
+
+class EvaluationConditions(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    schema_version: Literal["sforge_evaluation_conditions_v1"] = "sforge_evaluation_conditions_v1"
+    task_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    benchmark_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    task_spec_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    native_source_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    judge_image_id: str = Field(pattern=r"^sha256:[0-9a-f]{64}$")
+    execution_config_sha256: str = Field(pattern=r"^[0-9a-f]{64}$")
+    score_direction: Literal["maximize", "minimize"]
+    selection: str
+
+
+class RunEvaluationContext(BaseModel):
+    model_config = ConfigDict(frozen=True, extra="forbid")
+
+    schema_version: Literal["sforge_run_evaluation_context_v1"] = "sforge_run_evaluation_context_v1"
+    task_id: str
+    run_id: str
+    registration_id: str
+    evaluation_conditions: EvaluationConditions | None
+
+
+@dataclass(frozen=True)
+class RegisteredEvaluator:
+    """Keep accepted work on its own registration's executable inputs."""
+
+    task: TaskSpec
+    config: SForgeConfig
+    backend: ContainerBackend
+    context: RunEvaluationContext
+
+
+def native_source_sha256(source_root: Path | None = None) -> str | None:
+    """Observe the installed Python source once at service boot, not per read."""
+    if source_root is None:
+        import sforge
+
+        source_root = Path(sforge.__file__).parent
+    digest = hashlib.sha256()
+    try:
+        paths = sorted(source_root.rglob("*.py"))
+        if not paths:
+            return None
+        for path in paths:
+            digest.update(path.relative_to(source_root).as_posix().encode() + b"\0")
+            digest.update(hashlib.sha256(path.read_bytes()).digest())
+    except OSError:
+        return None
+    return digest.hexdigest()
+
+
+def capture_conditions(
+    task: TaskSpec, backend: ContainerBackend, source_sha256: str | None, config: SForgeConfig,
+) -> EvaluationConditions | None:
+    """Unknown image/data identity preserves feedback without claiming attribution.
+
+    The Docker content ID identifies the evaluator and data in that image. It
+    does not attest mutable external services or deterministic/hermetic grading.
+    Reading context never builds an image or downloads a task blob.
+    """
+    revision = getattr(task, "_source_revision", None)
+    if revision is None or source_sha256 is None or task.game_mode or backend.backend_name != "docker":
+        return None
+    try:
+        effective = task_spec_sha256(task)
+    except (TypeError, ValueError):
+        return None
+    if effective != revision.task_spec_sha256 or task.judge.direction not in ("maximize", "minimize"):
+        return None
+    try:
+        image_id = backend.client.images.get(task.judge_image_key).id
+    except Exception:
+        return None
+    if not isinstance(image_id, str) or not re.fullmatch(r"sha256:[0-9a-f]{64}", image_id):
+        return None
+    execution = {**judge_container_options(task, config),
+                 "timeout": task.judge.timeout, "platform": task.platform}
+    try:
+        raw = json.dumps(execution, sort_keys=True, separators=(",", ":"),
+                         ensure_ascii=False, allow_nan=False).encode()
+    except (TypeError, ValueError):
+        return None
+    return EvaluationConditions(
+        task_sha256=revision.task_sha256,
+        benchmark_sha256=revision.benchmark_sha256,
+        task_spec_sha256=effective,
+        native_source_sha256=source_sha256,
+        judge_image_id=image_id,
+        execution_config_sha256=hashlib.sha256(raw).hexdigest(),
+        score_direction=task.judge.direction,
+        # Modern grading.select_best filters valid completed entries, compares
+        # their raw score in this direction and retains the first tie.
+        selection="valid_then_score",
+    )