From d92c7a2973831d5da5214befbc888e8587387eb4 Mon Sep 17 00:00:00 2001 From: yinziqi Date: Sat, 3 Oct 2026 17:21:11 +0800 Subject: [PATCH 01/22] docs: add MatCreator agent and run guidance --- AGENTS.md | 107 ++++++++++++++++++++++++++++++++++++++++++++++++++++++ run.md | 3 ++ 2 files changed, 110 insertions(+) create mode 100644 AGENTS.md create mode 100644 run.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..6e14194 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,107 @@ +# AGENTS.md + +This file provides guidance to Qoder (qoder.com) when working with code in this repository. + +## Project Overview + +MatCreator is an agentic AI platform for computational materials science, built as an agent harness around the Google ADK runtime, a markdown-based skill library, a workspace, and a persistent knowledge graph. It offers CLI, API, and Web interfaces. + +## Commands + +### Environment setup + +```bash +pip install uv +uv venv .venv --python 3.12 # Python 3.12+ required (.python-version pins 3.12) +source .venv/bin/activate +uv pip install -e . # includes git deps (mat-bench, mat-know-base) — needs network +``` + +Frontend (only needed for Web UI development): + +```bash +cd web/vite-frontend +npm ci # Node.js >= 24.14.1 required (.nvmrc pins 24.14.1) +``` + +### Tests + +Backend (pytest; no pytest config file — run against `tests/` directly): + +```bash +python -m pytest tests/ -v # full suite +python -m pytest tests/test_ports_config.py -v # single file +python -m pytest tests/test_ports_config.py -k "name_pattern" -v # single test +``` + +Note: CI (`.github/workflows/test.yml`) runs only a subset of test files plus the frontend check. + +Frontend: + +```bash +cd web/vite-frontend +npm run check # runs node --test + vite build (this is what CI enforces) +npm test # node --test only +npm run build # vite build only +``` + +### Running + +```bash +bash script/start_matcreator.sh # Web UI: starts ADK API (:8000) + FastAPI (:8001) + Vite (:5173) +matcreator chat --workspace . # CLI, Flash mode (direct interaction) +matcreator chat --workspace . --plan # CLI, Plan mode (thinking agent -> execution graph -> execution agent) +``` + +The start script writes logs to `~/.matcreator/logs/{api-server,web-main,vite}.log` and resolves hosts/ports with precedence: environment variable > `~/.matcreator/config.yaml` > built-in default (logic in `src/matcreator/ports.py`). + +### Configuration + +Persistent settings live in `~/.matcreator/config.yaml` (relocatable via `MATCREATOR_HOME` or `MATCREATOR_CONFIG_PATH`): + +```bash +matcreator config set llm.model=openai/qwen3-plus +matcreator config set llm.api_key=your-api-key +matcreator config set llm.base_url=https://api.example.com/v1 +matcreator config show +``` + +Precedence rule: in local mode pre-existing environment variables win over `config.yaml`; in server mode (`MATCREATOR_MODE=server`) `config.yaml` wins. Full reference including multi-model `llm.executor_cards`: `docs/configuration.md`. + +## Architecture + +Request flow (see `docs/architecture_components.md` for details): + +``` +Vite frontend (web/vite-frontend) -> FastAPI middleware (web/main.py) -> ADK API server -> MatCreator agent (src/matcreator) +``` + +### Layers + +- **Frontend** (`web/vite-frontend`): Vite app (Svelte + React mix, Ketcher, vis-network). Owns UI state, SSE consumption, and browser-reload reconnect to managed runs. Entry: `src/main.js`. +- **FastAPI middleware** (`web/main.py`): the application backend. Two roles: (1) ADK protocol proxy (`/run_sse`, `/apps/*`, `/list-apps`); (2) MatCreator-specific APIs (sessions, workspace files, structures, skills, settings, cancellation). In server mode it also acts as a Docker worker control plane (one worker container per user). Managed runs are process-local — execution does not survive a middleware restart. +- **Agent runtime** (`src/matcreator`): hosted on Google ADK. Entry `src/matcreator/agent.py`; CLI/API-server startup in `src/matcreator/scripts/start_agent.py`. + +### Agent internals + +- **Orchestrator** (`src/matcreator/agents/orchestrator/agent.py`): every turn starts with planning; if the planning agent approves an execution graph, control moves to the execution agent, which can return to planning within the same ADK invocation. Flash mode bypasses this loop. +- **Thinking/execution agents** (`src/matcreator/agents/thinking_agent/`, `src/matcreator/agents/execution_agent/`): graph planning vs. step executors running tools/skills/remote jobs. +- **Skills** (`src/matcreator/skills/`): modular markdown capabilities. Each skill is a directory with a `SKILL.md` carrying YAML frontmatter (`name`, `description`, `metadata.tools`, `metadata.dependent_skills`, `metadata.tags`). Skills declare dependencies on each other (e.g. `eos` depends on `ase`, `deepmd`, `bohrium`, ...). +- **Knowledge** (`src/matcreator/knowledge/`): self-improving Know-Do Graph (extraction, review, migration, query) that accumulates experience across executions. Extraction/review frequency is configurable (`MATCREATOR_MEMORIZATION_FREQUENCY`, `MATCREATOR_REVIEW_FREQUENCY`; `0` disables). +- **Control plane** (`src/matcreator/control_plane/`): durable remote-job store/service/monitor (E2B sandboxes), benchmark/evaluation runtime, worker supervisor. +- **Workspace** (`src/matcreator/workspace.py`): sessions and artifacts live under the `--workspace` directory; ADK session state under `~/.matcreator/.adk/session.db`. + +### Source layout conventions + +- Python packages live under `src/matcreator` (`[tool.setuptools.packages.find] where = ["src"]`) — the import root is `src/`, not the repo root. `script/start_matcreator.sh` sets `PYTHONPATH` to `src/` so the checkout, not site-packages, is executed. +- Skill files (`.md`, `.py`, `.yaml` under `skills/`) are shipped via `[tool.setuptools.package-data]` — new skill assets in those formats are packaged automatically. +- Port/host resolution for all services is centralized in `src/matcreator/ports.py`. + +## Testing conventions + +- `tests/conftest.py` maintains `ALL_CONFIG_ENV_VARS`, the canonical list of configuration environment variables. When adding a new config env var to `src/matcreator/ports.py`, add it there so config-cleanup tests clear it automatically. +- Frontend tests live in `web/vite-frontend/test/` and use the Node built-in test runner (`node --test`). + +## Docker + +Multi-stage build (`Dockerfile`): `worker` (API only, :8000), `control-plane` (API + built frontend, :8001), `local` (full dev image incl. Node, :8000/:8001/:5173). Port configuration via `.env` (see `.env.example`; copy to `.env` before `docker compose up`). Deployment docs: `docs/deployment/`. diff --git a/run.md b/run.md new file mode 100644 index 0000000..c48e86e --- /dev/null +++ b/run.md @@ -0,0 +1,3 @@ +启动命令 bash script/start_matcreator.sh，ctrl-C一键停止。 + +然后 http://localhost:5173 处前端口，日志在~/.matcreator/logs From c59aa54b54bcf9a9aef6b5cf7ece9d008b7e143f Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 12:47:32 +0800 Subject: [PATCH 02/22] feat(aidb): preflight silicon tasks before calculation preparation (#2) --- docs/aidb_skill_hook.md | 56 +++++ src/matcreator/agents/aidb_skill_hook.py | 230 ++++++++++++++++++ .../agents/execution_agent/step_executor.py | 3 + src/matcreator/agents/thinking_agent/agent.py | 3 + src/matcreator/skill.py | 4 +- tests/test_aidb_skill_hook.py | 223 +++++++++++++++++ 6 files changed, 518 insertions(+), 1 deletion(-) create mode 100644 docs/aidb_skill_hook.md create mode 100644 src/matcreator/agents/aidb_skill_hook.py create mode 100644 tests/test_aidb_skill_hook.py diff --git a/docs/aidb_skill_hook.md b/docs/aidb_skill_hook.md new file mode 100644 index 0000000..10e4adc --- /dev/null +++ b/docs/aidb_skill_hook.md @@ -0,0 +1,56 @@ +# aidb 计算前预检（issue #2，开发期 A） + +主规格为 [Ai-ready_Database #1](https://github.com/Zikkying/Ai-ready_Database/issues/1)， +本次只实现 [T1 / #2](https://github.com/Zikkying/Ai-ready_Database/issues/2)。 + +主 thinking agent 和内部 step executor 共用工具回调。加载 `vasp-pymatgen`、 +硅任务的 `atomic-structure`，以及受支持任务直接调用准备或远程工具时， +先建立当前任务上下文，再通过 ai-ready-db Skill 的公开 adapter 执行本地预检。 +技能名称去除首尾空白并忽略大小写。纯概念请求与训练技能不触发。 + +信息来源为当前自然请求、内部步骤的 action / prior_context / 父任务 goal， +以及请求中明确引用的工作区结构文件。后续“继续 / 改用 / 重算”请求可以补充 +前一用户请求，但当前方法优先。未设置 goal 不会跳过提取。 +材料身份无法确定时先澄清；已确定 Si 时可先发现候选。PBE、bulk、diamond +缺失时只是检索默认值，反馈中的 known / retrieval_defaults / unknown 分别保存。 +准备计算前仍需明确结构模型。非首期晶相、体系或方法会暂停并说明支持边界。 +这是固定硅样例的有限识别，不是通用自然语言或结构等价判定器。 + +## 配置与公开接口 + +优先使用已安装 `ai-ready-db` Skill 的 `scripts/aidb_loop_adapter.py`。 +也可使用 `AI_READY_DB_ROOT` 指定仓库的标准 Skill 包；禁用 Skill 时不绕过。 +client/server 的根目录、Python、配置、socket 等配置沿用 aidb 自身契约。 +例如本地开发设置 `AI_READY_DB_ROOT` 与 `AI_READY_DB_PYTHON`，隔离测试额外 +设置 `AI_READY_DB_CONFIG` 和 `AI_READY_DB_BRIDGE_SOCK`。 + +不导入 aidb 内部模块、不连接数据库表。adapter 调用公开 preflight，单材料 +min_records=1，无相关材料扩展。每个 CLI 调用等待最多 20 秒，整个 adapter +最多 50 秒；取消或超时终止本次 adapter 客户端，数据库桥接服务仍按自身生命周期运行。 + +## 反馈与阻断 + +Skill 正文正常加载；after_tool_callback 将 `aidb_preflight` 加入实际工具响应， +模型能够读取并向用户反馈。反馈同时写入 session_log 和运行日志。 +成功响应读取完整 `preflight-report.json`，保留完整候选 ID、报告、审计位置。 +`found` 是候选命中，不能当成已经核验可复用；本票不实现候选复用/归档。 +`not_found` 是成功查询空候选；`failed` 是真实配置、连接、超时或协议失败。 +任务上下文与原始 adapter 交换记录保存在工作区 `.aidb/requests/`，完整查询 +报告和 audit 保存在 adapter 的 `.aidb/runs/` 下。失败诊断也保留位置。 + +开发期 A 中，失败或材料澄清状态阻止新结构/输入准备及 tracked remote-job +提交/上传/执行。只准备的请求即使查询成功也不授权提交。状态/结果读取工具 +仍可使用。同任务同次 invocation 的重复入口共用回执；新轮次重新查库， +条件变化不会复用旧回执。此约束覆盖登记的工具入口，不承诺约束任意自由 +shell/Python 入口。后续 T5 才切换最终 B，不能把本票称为完整闭环验收。 + +## 验证 + +```bash +PYTHONPATH=src .venv/bin/python -m pytest tests/test_aidb_skill_hook.py -q +``` + +测试使用确定性 ADK runner、实际两种 agent 接线、受控准备/提交工具边界， +和相邻 Ai-ready_Database 的真实公开桥接及隔离本地库。命中样本仅用于测试， +不是本任务提交的真实计算。没有相邻 aidb checkout/runtime 时集成用例跳过。 +不提交 Bohrium 作业、不归档未完成计算、不自动上传云端数据库。 diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py new file mode 100644 index 0000000..d00dced --- /dev/null +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -0,0 +1,230 @@ +"""Development-stage material preflight through the public aidb Skill adapter. + +Only candidate discovery is implemented here. A successful query is not proof +of scientific compatibility, and never authorizes a new remote calculation. +""" +from __future__ import annotations + +import asyncio +import hashlib +import json +import logging +import os +from pathlib import Path +import re +import sys +import uuid + +from .session_log import append_session_log_entry + +logger = logging.getLogger(__name__) +AIDB_PREFLIGHT_SKILLS = frozenset({'vasp-pymatgen', 'atomic-structure'}) +_PREPARATION_TOOLS = frozenset({'run_python', 'run_bash', 'run_skill_script', + 'submit_bohr_batchjob', 'submit_bohr_sandbox', 'upload_remote_job_input', + 'start_remote_job_command', 'run_remote_job_command', 'run_sub_agent'}) +_REMOTE_TOOLS = frozenset({'submit_bohr_batchjob', 'submit_bohr_sandbox', + 'upload_remote_job_input', 'start_remote_job_command', 'run_remote_job_command'}) + + +def _request(tool_context): + """Current user content wins over a previous goal or persisted receipt.""" + invocation = tool_context._invocation_context + content = invocation.user_content + text = '\n'.join(p.text for p in (content.parts or []) if p.text) + try: + step = json.loads(text) + except (ValueError, TypeError): + step = None + if isinstance(step, dict) and 'action' in step: + text = '\n'.join(str(step.get(k) or '') for k in ('action', 'prior_context')) + # Step requests inherit the parent's goal; it is current task context, + # rather than a prior user turn, in this isolated executor invocation. + text += '\n' + str(tool_context.state.get('goal') or '') + elif re.search(r'继续|这个|该结构|改用|重算|重新计算|same|\bit\b|recompute', text, re.I): + for event in reversed(invocation.session.events): + prior = event.content + if prior and prior.role == 'user' and event.invocation_id != tool_context.invocation_id: + text += '\n' + '\n'.join(p.text for p in (prior.parts or []) if p.text) + break + return text + + +def _task_context(text, workspace): + lower = text.lower() + calculation = bool(re.search(r'弛豫|优化|总能|计算|准备|生成|relax|total energy|calculat|prepare|generate', lower)) + conceptual = bool(re.search(r'解释|介绍|什么是|概念|explain|what is|概念说明', lower)) + if not calculation or (conceptual and not re.search(r'请.*(?:准备|生成|执行)|please.*(?:prepare|run|calculate)', lower)): + return None + known = {} + if re.search(r'单质硅|硅晶|硅体|硅单体|金刚石.*硅|\bsilicon\b|\bsi\b|硅', lower) and not re.search(r'氧化硅|二氧化硅|碳化硅|氮化硅|silica|sic\b|sio2\b', lower): + known['formula'] = 'Si' + if re.search(r'金刚石|diamond', lower): + known['structure_model'] = 'diamond' + if re.search(r'beta[- ]tin|β[- ]?锡|非金刚石|六方|hexagonal', lower): + known['structure_model'] = 'outside_scope' + if re.search(r'体相|晶体|bulk|crystal', lower): + known['domain'] = 'bulk' + if re.search(r'孤立|原子|表面|缺陷|isolated|surface|defect', lower): + known['domain'] = 'outside_scope' + method = re.search(r'\b(pbe\+u|pbesol|hse06|scan|lda|pbe)\b', lower) + if method: + known['functional'] = method.group(1).upper() + # Read explicitly supplied existing structures; never create one at lookup. + for match in re.finditer(r'[\w./-]+\.(?:cif|xyz|extxyz)|\bPOSCAR\b', text, re.I): + path = (workspace / match.group()).resolve() + if path.is_relative_to(workspace) and path.is_file(): + from ase.io import read + atoms = read(path) + known['structure_path'] = str(path) + known['formula'] = 'Si' if set(atoms.get_chemical_symbols()) == {'Si'} else atoms.get_chemical_formula() + known['domain'] = 'bulk' if all(atoms.pbc) else 'outside_scope' + # A file does not establish the crystal model by composition alone. + defaults = {key: value for key, value in { + 'functional': 'PBE', 'domain': 'bulk', 'structure_model': 'diamond', + }.items() if key not in known} + targets = [] + if re.search(r'弛豫|优化|relax', lower): + targets.append('relaxation') + if re.search(r'总能|能量|total energy|energy', lower): + targets.append('total_energy') + return {'known': known, 'retrieval_defaults': defaults, + 'unknown': [k for k in ('formula', 'structure_model', 'domain', 'functional') if k not in known], + 'targets': targets, 'recalculate': bool(re.search(r'重算|重新计算|recalculat|recompute', lower)), + 'prepare_only': bool(re.search(r'不提交|do not submit|only.*input|仅生成', lower)) or ( + bool(re.search(r'准备|prepare', lower)) and not re.search(r'提交|submit|执行计算|run calculation', lower)), + 'workspace': str(workspace)} + + +def _adapter_path(): + from matcreator.skill import skill_bundle_info, is_skill_disabled + if is_skill_disabled('ai-ready-db'): + raise RuntimeError('ai-ready-db Skill is disabled') + bundle = skill_bundle_info('ai-ready-db') + if bundle and bundle.get('skill_dir'): + path = Path(bundle['skill_dir']) / 'scripts/aidb_loop_adapter.py' + if path.is_file(): + return path + # The canonical Skill package supplied by the configured public bridge. + root = os.environ.get('AI_READY_DB_ROOT') + if root: + path = Path(root).expanduser() / 'skills/ai-ready-db/scripts/aidb_loop_adapter.py' + if path.is_file(): + return path + raise RuntimeError('ai-ready-db adapter unavailable; install the Skill or configure AI_READY_DB_ROOT') + + +async def _query(task, task_id): + workspace = Path(task['workspace']) + directory = workspace / '.aidb' / 'requests' / task_id / uuid.uuid4().hex + directory.mkdir(parents=True, exist_ok=True) + task_path = directory / 'task.json' + diagnostic_path = directory / 'adapter-result.json' + task_path.write_text(json.dumps({'task_id': task_id, 'purpose': task['targets'], + 'target': {'formula': task['known']['formula'], 'min_records': 1, + 'functional': task['known'].get('functional', 'PBE'), 'domain': 'bulk', + 'allow_dft_u': False}, 'related_materials': []}), encoding='utf-8') + diagnostic = {'task_path': str(task_path), 'query_attempted': False} + try: + process = await asyncio.create_subprocess_exec(sys.executable, str(_adapter_path()), + '--workspace', str(workspace), 'preflight', '--task-json', str(task_path), + '--command-timeout', '20', stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE) + diagnostic['query_attempted'] = True + try: + stdout, stderr = await asyncio.wait_for(process.communicate(), timeout=50) + except BaseException: + if process.returncode is None: + process.kill() + await process.communicate() + raise + diagnostic.update(returncode=process.returncode, stdout=stdout.decode(errors='replace'), + stderr=stderr.decode(errors='replace')) + receipt = json.loads(stdout.decode()) + if not isinstance(receipt, dict): + raise ValueError('adapter receipt must be an object') + if process.returncode or receipt.get('ok') is not True: + return {**receipt, 'status': 'failed', 'diagnostic_path': str(diagnostic_path)} + report_path = Path(receipt['report_path']) + report = json.loads(report_path.read_text(encoding='utf-8')) + query = report['queries'][0] + if query['formula'] != task['known']['formula'] or query['status'] not in {'found', 'not_found'}: + raise ValueError('invalid target query report') + ids = query['record_ids'] + if not isinstance(ids, list) or bool(ids) != (query['status'] == 'found'): + raise ValueError('inconsistent candidate report') + return {'status': query['status'], 'candidate_ids': ids, 'report_path': str(report_path), + 'audit_path': receipt['audit_path'], 'report': report, 'diagnostic_path': str(diagnostic_path)} + except Exception as exc: + diagnostic['error'] = str(exc) or type(exc).__name__ + return {'status': 'failed', 'detail': diagnostic['error'], + 'diagnostic_path': str(diagnostic_path)} + finally: + diagnostic_path.write_text(json.dumps(diagnostic, ensure_ascii=False), encoding='utf-8') + + +async def before_aidb_skill_load(tool, args, tool_context): + name = getattr(tool, 'name', '') + skill_name = str(args.get('skill_name') or '').strip().lower() + feedback_key = f'temp:aidb_skill_feedback_{tool_context.function_call_id}' + if name == 'load_skill': + tool_context.state[feedback_key] = None + if skill_name not in AIDB_PREFLIGHT_SKILLS: + return None + elif name not in _PREPARATION_TOOLS: + return None + try: + workspace = Path(tool_context.state.get('workspace_dir') or tool_context.state.get('workdir') or '.').resolve() + text = _request(tool_context) + if name != 'load_skill' or skill_name == 'atomic-structure': + if not re.search(r'vasp|硅|\bsilicon\b|\bSi\b', text, re.I): + return None + task = _task_context(text, workspace) + if task is None: + return None + fingerprint = hashlib.sha256(json.dumps(task, sort_keys=True).encode()).hexdigest() + key = f'{tool_context.invocation_id}:{fingerprint}' + saved = tool_context.state.get('aidb_preflight') or {} + if saved.get('key') == key: + feedback = saved['feedback'] + else: + known = task['known'] + if not known.get('formula'): + feedback = {'status': 'needs_clarification', 'message': '请明确材料身份；尚未查询，新增计算暂停。'} + elif known['formula'] != 'Si' or known.get('domain') == 'outside_scope' or known.get('structure_model') == 'outside_scope' or known.get('functional', 'PBE') != 'PBE': + feedback = {'status': 'needs_clarification', 'message': '首期仅支持单质硅金刚石体相与普通 PBE；请明确当前目标，新增计算暂停。'} + else: + feedback = await _query(task, fingerprint[:24]) + feedback['task_context'] = task + feedback['query_executed'] = feedback['status'] in {'found', 'not_found'} + if feedback['status'] == 'found': + feedback['message'] = '本地查库成功，有候选记录；候选尚未核验结构、结果和成功状态，不能直接认定可复用。' + elif feedback['status'] == 'not_found': + feedback['message'] = '本地查库成功，候选为空；仅继续原授权范围内的准备。' + elif feedback['status'] == 'failed': + feedback['message'] = '本地查库失败，开发期 A 暂停新增计算；这不是未命中。请检查诊断产物。' + tool_context.state['aidb_preflight'] = {'key': key, 'feedback': feedback} + logger.info('[aidb preflight] %s', feedback) + append_session_log_entry(tool_context, {'kind': 'aidb_hook_feedback', **feedback}) + except Exception as exc: + logger.warning('aidb preflight failed', exc_info=True) + feedback = {'status': 'failed', 'query_executed': False, 'detail': str(exc), + 'message': '本地预检失败，开发期 A 暂停新增计算；不代表无数据。'} + tool_context.state['aidb_preflight'] = {'feedback': feedback} + if name == 'load_skill': + tool_context.state[feedback_key] = feedback + return None + if feedback['status'] not in {'found', 'not_found'}: + return {'status': 'blocked', 'aidb_preflight': feedback} + if 'structure_model' in task['unknown']: + return {'status': 'blocked', 'message': '已查候选；计算准备前请明确结构模型，检索默认值不是计算授权。', 'aidb_preflight': feedback} + if name in _REMOTE_TOOLS and task['prepare_only']: + return {'status': 'blocked', 'message': '当前请求仅授权准备，未授权提交计算。', 'aidb_preflight': feedback} + return None + + +def after_aidb_skill_load(tool, args, tool_context, tool_response): + """Put lookup evidence in the actual tool response seen by the agent/user.""" + if getattr(tool, 'name', '') == 'load_skill' and str(args.get('skill_name') or '').strip().lower() in AIDB_PREFLIGHT_SKILLS: + feedback = tool_context.state.get(f'temp:aidb_skill_feedback_{tool_context.function_call_id}') + if feedback and isinstance(tool_response, dict): + return {**tool_response, 'aidb_preflight': feedback} + return None diff --git a/src/matcreator/agents/execution_agent/step_executor.py b/src/matcreator/agents/execution_agent/step_executor.py index 9a2da7c..b81895e 100644 --- a/src/matcreator/agents/execution_agent/step_executor.py +++ b/src/matcreator/agents/execution_agent/step_executor.py @@ -10,6 +10,7 @@ from google.adk.tools.tool_context import ToolContext from pydantic import BaseModel, Field, ValidationError, field_validator, model_validator +from ..aidb_skill_hook import before_aidb_skill_load, after_aidb_skill_load from ...llm_cards import LLMCard from ...skill import ALL_SKILLS_TOOLSET from ...knowledge.query import get_related_skills, query_knowledge_graph, read_knowledge_node @@ -297,6 +298,8 @@ def build_step_executor_agent(llm_card: LLMCard) -> LlmAgent: ), instruction=_STEP_EXECUTOR_INSTRUCTION, input_schema=StepExecutorInput, + before_tool_callback=before_aidb_skill_load, + after_tool_callback=after_aidb_skill_load, tools=[ FunctionTool(run_sub_agent), FunctionTool(submit_step_result), diff --git a/src/matcreator/agents/thinking_agent/agent.py b/src/matcreator/agents/thinking_agent/agent.py index a1adf72..7ebdaa4 100644 --- a/src/matcreator/agents/thinking_agent/agent.py +++ b/src/matcreator/agents/thinking_agent/agent.py @@ -12,6 +12,7 @@ from google.adk.tools.tool_context import ToolContext from google.adk.agents.callback_context import CallbackContext +from ..aidb_skill_hook import before_aidb_skill_load, after_aidb_skill_load from ...constants import LLM_API_KEY, LLM_BASE_URL, LLM_MODEL from ...adk_compat import install_lenient_tool_argument_parsing from .planning import validate_plan, validate_graph @@ -505,4 +506,6 @@ def before_agent_callback(callback_context: CallbackContext) -> None: show_structure, ], before_agent_callback=before_agent_callback, + before_tool_callback=before_aidb_skill_load, + after_tool_callback=after_aidb_skill_load, ) diff --git a/src/matcreator/skill.py b/src/matcreator/skill.py index 89eed7d..650a84c 100644 --- a/src/matcreator/skill.py +++ b/src/matcreator/skill.py @@ -443,7 +443,9 @@ def __init__(self, skills: list): ] def _get_skill(self, skill_name: str): - skill = super()._get_skill(skill_name) + normalized = (skill_name or '').strip().lower() + canonical = next((s.name for s in ALL_SKILLS if s.name.lower() == normalized), skill_name) + skill = super()._get_skill(canonical) if skill is not None and is_skill_disabled(skill.name): return None return skill diff --git a/tests/test_aidb_skill_hook.py b/tests/test_aidb_skill_hook.py new file mode 100644 index 0000000..78f4614 --- /dev/null +++ b/tests/test_aidb_skill_hook.py @@ -0,0 +1,223 @@ +from __future__ import annotations + +import asyncio +import json +import os +from pathlib import Path + +import pytest +from google.adk.agents import LlmAgent +from google.adk.models.base_llm import BaseLlm +from google.adk.models.llm_response import LlmResponse +from google.adk.runners import Runner +from google.adk.sessions import InMemorySessionService +from google.genai import types +from pydantic import PrivateAttr + +from matcreator.agents import aidb_skill_hook as hook + + +@pytest.fixture +def local_aidb(tmp_path, monkeypatch): + root = Path(__file__).resolve().parents[2] / 'Ai-ready_Database' + if not (root / '.venv/bin/python').exists(): + pytest.skip('adjacent aidb checkout and runtime required for public bridge integration') + monkeypatch.setenv('AI_READY_DB_ROOT', str(root)) + monkeypatch.setenv('AI_READY_DB_PYTHON', str(root / '.venv/bin/python')) + monkeypatch.setenv('AI_READY_DB_BRIDGE_SOCK', str(tmp_path / 'bridge.sock')) + config = tmp_path / 'config.json' + config.write_text(json.dumps({'ase_sqlite': {'path': str(tmp_path / 'local.db'), + 'json_sidecar_path': str(tmp_path / 'sidecar.json')}, 'work_dir': str(tmp_path), + 'xc_functional': 'PBE', 'dft_u': False, 'system_type': 'bulk'})) + monkeypatch.setenv('AI_READY_DB_CONFIG', str(config)) + monkeypatch.setenv('AI_READY_DB_BRIDGE_IDLE_TIMEOUT', '1') + yield root + # Ask the isolated read-only bridge to exit; never touch the user's server. + import signal + pid_file = Path(str(tmp_path / 'bridge.sock') + '.pid') + if pid_file.exists(): + try: + os.kill(int(pid_file.read_text()), signal.SIGTERM) + except (OSError, ValueError): + pass + + +def run_task(tmp_path, request, calls, state=None, entry=None, followup=None): + class Model(BaseLlm): + _index: int = PrivateAttr(default=0) + async def generate_content_async(self, llm_request, stream=False): + if self._index < len(calls): + name, args = calls[self._index] + part = types.Part(function_call=types.FunctionCall(name=name, args=args)) + else: + if followup and self._index == len(calls) + 1: + name, args = calls[(self._index - len(calls) - 1) % len(calls)] + part = types.Part(function_call=types.FunctionCall(name=name, args=args)) + else: + part = types.Part(text='finished') + self._index += 1 + yield LlmResponse(content=types.Content(role='model', parts=[part])) + + prepared = [] + def load_skill(skill_name: str) -> dict: + return {'instructions': 'VASP instructions', 'skill': skill_name} + def run_python(code: str) -> dict: + prepared.append(code) + return {'status': 'prepared'} + def submit_bohr_batchjob(name: str) -> dict: + prepared.append('remote submission') + return {'status': 'submitted'} + + async def run(): + service = InMemorySessionService() + session = await service.create_session(app_name='preflight_test', user_id='test', + state={'workspace_dir': str(tmp_path), **(state or {})}) + agent = LlmAgent(name='task_agent', model=Model(model='test'), + tools=[load_skill, run_python, submit_bohr_batchjob], + before_tool_callback=hook.before_aidb_skill_load, + after_tool_callback=hook.after_aidb_skill_load) + if entry == 'thinking': + from matcreator.agents.thinking_agent.agent import thinking_agent + agent = thinking_agent.model_copy(update={'name': 'thinking_test', 'model': Model(model='test'), + 'tools': [load_skill, run_python, submit_bohr_batchjob], 'instruction': '', + 'before_agent_callback': None, 'sub_agents': []}) + elif entry == 'step': + from matcreator.agents.execution_agent.step_executor import build_step_executor_agent + from matcreator.llm_cards import LLMCard + agent = build_step_executor_agent(LLMCard(name='test', model='openai/test')).model_copy( + update={'model': Model(model='test'), 'tools': [load_skill, run_python, submit_bohr_batchjob], + 'instruction': '', 'input_schema': None, 'sub_agents': []}) + runner = Runner(agent=agent, app_name='preflight_test', session_service=service) + responses = [] + async for event in runner.run_async(user_id='test', session_id=session.id, + new_message=types.Content(role='user', parts=[types.Part(text=request)])): + for part in (event.content.parts if event.content else []): + if part.function_response: + responses.append(part.function_response.response) + if followup: + async for event in runner.run_async(user_id='test', session_id=session.id, + new_message=types.Content(role='user', parts=[types.Part(text=followup)])): + for part in (event.content.parts if event.content else []): + if part.function_response: + responses.append(part.function_response.response) + return responses + return asyncio.run(run()), prepared + + +def test_natural_silicon_request_queries_before_preparation_without_goal(local_aidb, tmp_path): + responses, prepared = run_task(tmp_path, '请准备金刚石结构单质硅体相的 VASP 弛豫与总能输入，普通 PBE，不提交计算', + [('load_skill', {'skill_name': ' VASP-PYMATGEN '}), ('run_python', {'code': 'prepare'})]) + feedback = responses[0]['aidb_preflight'] + assert feedback['status'] == 'not_found' + assert Path(feedback['report_path']).is_file() + assert json.loads(Path(feedback['report_path']).read_text())['queries'][0]['record_ids'] == [] + assert prepared == ['prepare'] + +@pytest.mark.parametrize('prompt', ['请解释硅的 VASP 弛豫概念', 'What is VASP relaxation of silicon?']) +def test_concept_only_does_not_query(local_aidb, tmp_path, prompt): + responses, _ = run_task(tmp_path, prompt, [('load_skill', {'skill_name': 'vasp-pymatgen'})]) + assert 'aidb_preflight' not in responses[0] + assert not (tmp_path / '.aidb').exists() + + +def test_missing_identity_pauses_preparation_without_query(local_aidb, tmp_path): + responses, prepared = run_task(tmp_path, '请用 VASP 准备弛豫和总能输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('run_python', {'code': 'prepare'})]) + assert responses[0]['aidb_preflight']['status'] == 'needs_clarification' + assert prepared == [] + assert not (tmp_path / '.aidb').exists() + + +def test_failure_is_not_empty_and_blocks_preparation_and_submission(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, prepared = run_task(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能', + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('run_python', {'code': 'prepare'}), + ('submit_bohr_batchjob', {'name': 'Si'})]) + assert responses[0]['aidb_preflight']['status'] == 'failed' + assert 'config does not exist' in responses[0]['aidb_preflight']['detail'] + assert prepared == [] + + +def test_repeated_entry_reuses_one_current_report_and_preserves_unknowns(local_aidb, tmp_path): + responses, _ = run_task(tmp_path, '准备硅的 VASP 弛豫与总能输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('load_skill', {'skill_name': 'atomic-structure'})]) + assert responses[0]['aidb_preflight']['report_path'] == responses[1]['aidb_preflight']['report_path'] + assert len(list((tmp_path / '.aidb/runs').glob('*/*/audit.json'))) == 1 + context = responses[0]['aidb_preflight']['task_context'] + assert context['known'] == {'formula': 'Si'} + assert context['retrieval_defaults']['functional'] == 'PBE' + assert 'structure_model' in context['unknown'] + + +def test_prepare_only_never_submits_even_after_success(local_aidb, tmp_path): + responses, prepared = run_task(tmp_path, '只准备金刚石硅晶体 PBE 弛豫输入，不提交计算', + [('submit_bohr_batchjob', {'name': 'Si'})]) + assert responses[0]['status'] == 'blocked' + assert prepared == [] + + +def test_internal_executor_uses_inherited_context_without_requiring_json_from_user(local_aidb, tmp_path): + step = json.dumps({'action': 'Prepare VASP inputs for relaxation and total energy', + 'prior_context': 'Diamond bulk silicon, ordinary PBE', 'workspace_dir': str(tmp_path)}) + responses, prepared = run_task(tmp_path, step, + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('run_python', {'code': 'prepare'})]) + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert prepared == ['prepare'] + + +def test_non_diamond_phase_is_visible_and_does_not_silently_prepare(local_aidb, tmp_path): + responses, prepared = run_task(tmp_path, '计算 beta-tin 硅体相 PBE 总能', + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('run_python', {'code': 'prepare'})]) + assert responses[0]['aidb_preflight']['status'] == 'needs_clarification' + assert prepared == [] + +@pytest.mark.parametrize('entry', ['thinking', 'step']) +def test_production_agents_query_at_the_registered_tool_boundary(local_aidb, tmp_path, entry): + responses, _ = run_task(tmp_path, 'Prepare diamond bulk silicon PBE VASP relaxation and total energy inputs', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], entry=entry) + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert Path(responses[0]['aidb_preflight']['audit_path']).is_file() + + +def test_new_turn_queries_again_and_method_switch_invalidates_old_receipt(local_aidb, tmp_path): + responses, _ = run_task(tmp_path, '准备金刚石硅体相 PBE VASP 弛豫输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], followup='改用 HSE06 重算硅晶体弛豫') + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert responses[1]['aidb_preflight']['status'] == 'needs_clarification' + assert responses[1]['aidb_preflight']['task_context']['known']['functional'] == 'HSE06' + + +def test_new_independent_request_queries_instead_of_using_old_logs(local_aidb, tmp_path): + responses, _ = run_task(tmp_path, '准备金刚石硅体相 PBE 弛豫输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], followup='准备金刚石硅体相 PBE 弛豫输入') + assert responses[0]['aidb_preflight']['report_path'] != responses[1]['aidb_preflight']['report_path'] + + +def test_concept_followup_does_not_receive_previous_receipt(local_aidb, tmp_path): + responses, _ = run_task(tmp_path, '准备金刚石硅体相 PBE 弛豫输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], followup='解释硅弛豫概念') + assert 'aidb_preflight' not in responses[1] + + +def test_found_returns_all_candidate_ids_and_report_without_claiming_reuse(local_aidb, tmp_path): + import subprocess + from ase.build import bulk + from pymatgen.io.ase import AseAtomsAdaptor + record = tmp_path / 'sample.json' + record.write_text(json.dumps({'source_id': 'silicon-fixture', 'source_dataset': 'test-only', + 'formula_reduced': 'Si', 'formula_pretty': 'Si', 'composition': {'Si': 2}, + 'functional': 'PBE', 'energy': -10.0, + 'structure': AseAtomsAdaptor.get_structure(bulk('Si', 'diamond', a=5.43)).as_dict()})) + result = subprocess.run([os.environ['AI_READY_DB_PYTHON'], + str(local_aidb / 'skills/ai-ready-db/scripts/aidb_client.py'), 'archive-local', + '--record-json', str(record), '--namespace', 'bulk_base_pbe', + '--origin', 'local_calculation', '--sync-status', 'local_only', '--config', os.environ['AI_READY_DB_CONFIG']], + capture_output=True, text=True, timeout=30) + assert result.returncode == 0, result.stderr + responses, _ = run_task(tmp_path, '准备金刚石硅体相普通 PBE 弛豫和总能输入', + [('load_skill', {'skill_name': 'vasp-pymatgen'})]) + feedback = responses[0]['aidb_preflight'] + assert feedback['status'] == 'found' + assert feedback['candidate_ids'] + assert feedback['report']['queries'][0]['record_ids'] == feedback['candidate_ids'] + assert '尚未核验' in feedback['message'] From 071cfab6202b704704a27376d7f136dec8e5b884 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 13:06:21 +0800 Subject: [PATCH 03/22] fix(aidb): preserve task authorization and close preflight bypasses --- docs/aidb_preflight_test_results.json | 108 +++++++++++++ docs/aidb_preflight_verification_20261007.md | 64 ++++++++ docs/aidb_skill_hook.md | 6 +- src/matcreator/agents/aidb_skill_hook.py | 153 +++++++++++++------ src/matcreator/skill.py | 5 +- tests/test_aidb_skill_hook.py | 102 ++++++++++++- tests/test_skill_tools.py | 7 +- 7 files changed, 392 insertions(+), 53 deletions(-) create mode 100644 docs/aidb_preflight_test_results.json create mode 100644 docs/aidb_preflight_verification_20261007.md diff --git a/docs/aidb_preflight_test_results.json b/docs/aidb_preflight_test_results.json new file mode 100644 index 0000000..a129fa9 --- /dev/null +++ b/docs/aidb_preflight_test_results.json @@ -0,0 +1,108 @@ +{ + "date": "2026-10-07", + "stage": "T1 development A", + "baseline_commit": "d92c7a2973831d5da5214befbc888e8587387eb4", + "baseline": { + "summary": "39 failed, 710 passed, 578 warnings in 41.51s", + "failures": [ + "tests/test_benchmark_config.py::test_benchmark_client_registers_and_persists_missing_development_token", + "tests/test_disabled_skills.py::test_set_disabled_skill_names_updates_graph_nodes", + "tests/test_evaluation_routes.py::test_toggle_skill_graph_node_updates_and_reports_graph_state", + "tests/test_kdg_extractor.py::test_extractor_advances_cursor_for_empty_valid_response", + "tests/test_kdg_extractor.py::test_extractor_migrates_legacy_extracted_session", + "tests/test_kdg_extractor.py::test_extractor_skips_unchanged_session_and_processes_only_delta", + "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_routes_through_protected_reviewer_policy", + "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_scopes_memory_review_to_current_session", + "tests/test_kdg_progressive_retrieval.py::test_graph_disabled_skills_are_hidden_from_discovery_and_reads", + "tests/test_skill_graph_seed.py::test_seed_removes_stale_managed_skill_and_uses_virtual_dependency", + "tests/test_skill_graph_seed.py::test_seed_reuses_disabled_skill_node_and_merges_legacy_duplicate", + "tests/test_step_executor_runner.py::test_step_executor_retries_only_malformed_streamed_tool_arguments", + "tests/test_structure_modeling.py::test_crystal_is_created_through_matcraft_bulk_builder", + "tests/test_structure_modeling.py::test_matcraft_perturbation_is_seeded_and_changes_positions", + "tests/test_structure_modeling.py::test_molecule_can_be_created_or_appended", + "tests/test_structure_modeling.py::test_smiles_molecule_can_be_appended_at_requested_center", + "tests/test_structure_modeling.py::test_supercell_supports_general_integer_matrix", + "tests/test_web_concurrent_sessions.py::test_active_session_transitions_recompute_composer_state", + "tests/test_web_concurrent_sessions.py::test_all_chat_disclosures_share_the_reading_position_controller", + "tests/test_web_concurrent_sessions.py::test_bottom_attachment_and_node_toggle_use_one_scroll_policy", + "tests/test_web_concurrent_sessions.py::test_chat_messages_use_one_model_scheduler_and_stable_regions", + "tests/test_web_concurrent_sessions.py::test_completed_request_handoffs_live_turn_before_durable_refresh", + "tests/test_web_concurrent_sessions.py::test_frontend_tracks_requests_per_session", + "tests/test_web_concurrent_sessions.py::test_image_and_all_timeline_updates_preserve_node_disclosures", + "tests/test_web_concurrent_sessions.py::test_new_session_ids_are_not_limited_to_one_second_resolution", + "tests/test_web_concurrent_sessions.py::test_plain_text_blocks_keep_history_order_and_stable_identity", + "tests/test_web_concurrent_sessions.py::test_remote_job_polling_is_scoped_to_the_active_session", + "tests/test_web_concurrent_sessions.py::test_remote_jobs_are_collapsed_and_keep_lifecycle_status_visible", + "tests/test_web_concurrent_sessions.py::test_reselecting_the_active_session_does_not_rebuild_the_view", + "tests/test_web_concurrent_sessions.py::test_roadmap_fits_after_the_popup_has_a_stable_layout", + "tests/test_web_concurrent_sessions.py::test_running_session_switch_discovers_and_reconnects_managed_run", + "tests/test_web_concurrent_sessions.py::test_session_creation_does_not_start_knowledge_review", + "tests/test_web_concurrent_sessions.py::test_session_switch_loads_page_scoped_graph_after_the_transcript_page", + "tests/test_web_concurrent_sessions.py::test_session_switch_renders_cached_snapshot_immediately", + "tests/test_web_concurrent_sessions.py::test_sse_request_uses_captured_session_context", + "tests/test_web_concurrent_sessions.py::test_startup_keeps_session_api_ready_while_maintenance_runs_in_background", + "tests/test_web_concurrent_sessions.py::test_startup_restores_only_an_accessible_session_owner_tuple", + "tests/test_web_concurrent_sessions.py::test_step_cancellation_identifies_the_active_session_owner", + "tests/test_web_concurrent_sessions.py::test_stop_and_plan_refreshes_preserve_open_node_dialogs" + ] + }, + "final": { + "summary": "39 failed, 743 passed, 585 warnings in 61.57s (0:01:01)", + "failures": [ + "tests/test_benchmark_config.py::test_benchmark_client_registers_and_persists_missing_development_token", + "tests/test_disabled_skills.py::test_set_disabled_skill_names_updates_graph_nodes", + "tests/test_evaluation_routes.py::test_toggle_skill_graph_node_updates_and_reports_graph_state", + "tests/test_kdg_extractor.py::test_extractor_advances_cursor_for_empty_valid_response", + "tests/test_kdg_extractor.py::test_extractor_migrates_legacy_extracted_session", + "tests/test_kdg_extractor.py::test_extractor_skips_unchanged_session_and_processes_only_delta", + "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_routes_through_protected_reviewer_policy", + "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_scopes_memory_review_to_current_session", + "tests/test_kdg_progressive_retrieval.py::test_graph_disabled_skills_are_hidden_from_discovery_and_reads", + "tests/test_skill_graph_seed.py::test_seed_removes_stale_managed_skill_and_uses_virtual_dependency", + "tests/test_skill_graph_seed.py::test_seed_reuses_disabled_skill_node_and_merges_legacy_duplicate", + "tests/test_step_executor_runner.py::test_step_executor_retries_only_malformed_streamed_tool_arguments", + "tests/test_structure_modeling.py::test_crystal_is_created_through_matcraft_bulk_builder", + "tests/test_structure_modeling.py::test_matcraft_perturbation_is_seeded_and_changes_positions", + "tests/test_structure_modeling.py::test_molecule_can_be_created_or_appended", + "tests/test_structure_modeling.py::test_smiles_molecule_can_be_appended_at_requested_center", + "tests/test_structure_modeling.py::test_supercell_supports_general_integer_matrix", + "tests/test_web_concurrent_sessions.py::test_active_session_transitions_recompute_composer_state", + "tests/test_web_concurrent_sessions.py::test_all_chat_disclosures_share_the_reading_position_controller", + "tests/test_web_concurrent_sessions.py::test_bottom_attachment_and_node_toggle_use_one_scroll_policy", + "tests/test_web_concurrent_sessions.py::test_chat_messages_use_one_model_scheduler_and_stable_regions", + "tests/test_web_concurrent_sessions.py::test_completed_request_handoffs_live_turn_before_durable_refresh", + "tests/test_web_concurrent_sessions.py::test_frontend_tracks_requests_per_session", + "tests/test_web_concurrent_sessions.py::test_image_and_all_timeline_updates_preserve_node_disclosures", + "tests/test_web_concurrent_sessions.py::test_new_session_ids_are_not_limited_to_one_second_resolution", + "tests/test_web_concurrent_sessions.py::test_plain_text_blocks_keep_history_order_and_stable_identity", + "tests/test_web_concurrent_sessions.py::test_remote_job_polling_is_scoped_to_the_active_session", + "tests/test_web_concurrent_sessions.py::test_remote_jobs_are_collapsed_and_keep_lifecycle_status_visible", + "tests/test_web_concurrent_sessions.py::test_reselecting_the_active_session_does_not_rebuild_the_view", + "tests/test_web_concurrent_sessions.py::test_roadmap_fits_after_the_popup_has_a_stable_layout", + "tests/test_web_concurrent_sessions.py::test_running_session_switch_discovers_and_reconnects_managed_run", + "tests/test_web_concurrent_sessions.py::test_session_creation_does_not_start_knowledge_review", + "tests/test_web_concurrent_sessions.py::test_session_switch_loads_page_scoped_graph_after_the_transcript_page", + "tests/test_web_concurrent_sessions.py::test_session_switch_renders_cached_snapshot_immediately", + "tests/test_web_concurrent_sessions.py::test_sse_request_uses_captured_session_context", + "tests/test_web_concurrent_sessions.py::test_startup_keeps_session_api_ready_while_maintenance_runs_in_background", + "tests/test_web_concurrent_sessions.py::test_startup_restores_only_an_accessible_session_owner_tuple", + "tests/test_web_concurrent_sessions.py::test_step_cancellation_identifies_the_active_session_owner", + "tests/test_web_concurrent_sessions.py::test_stop_and_plan_refreshes_preserve_open_node_dialogs" + ] + }, + "new_failures": [], + "resolved_baseline_failures": [], + "collection_errors_reproduced_at_baseline": [ + "tests/test_execution_grouping.py", + "tests/test_matcreator_phase_and_skills.py" + ], + "focused_tests": { + "passed": 40, + "failed": 0 + }, + "aidb_bridge_tests": { + "passed": 14, + "failed": 0 + }, + "real_calculation_submitted": false +} diff --git a/docs/aidb_preflight_verification_20261007.md b/docs/aidb_preflight_verification_20261007.md new file mode 100644 index 0000000..4a0dd91 --- /dev/null +++ b/docs/aidb_preflight_verification_20261007.md @@ -0,0 +1,64 @@ +# issue #2 开发期 A：实现与验证记录 + +日期：2026-10-07。主规格：[Ai-ready_Database #1](https://github.com/Zikkying/Ai-ready_Database/issues/1)； +任务：[T1 / #2](https://github.com/Zikkying/Ai-ready_Database/issues/2)。实现仓库为 MatCreator。 + +已读取两个仓库的 AGENTS、issue tracker / domain 指引、aidb 术语表、公开 adapter +契约和现有需求记录。沿用已有 hook 接线与公开 Skill adapter；相邻数据库仓库 +没有业务代码改动。原工作区存在的其他改动保留。 + +## 已实现行为 + +- 当前自然请求、内部 action / prior_context / 父任务 goal、明确引用的现有结构 + 提供任务信息；goal 未设置也会提取或澄清。已知、检索默认、未知分开反馈。 +- 主 thinking agent 与 step executor 在受支持的新结构/输入准备及远程工具前 + 实际预检。Skill 名规范化与两种加载器一致；完整报告、候选 ID、审计与诊断 + 随 Skill 或直接准备工具的响应返回。 +- 命中、成功空候选、失败明确区分；开发期 A 失败暂停新增准备/计算。 + 同次调用重复与并发入口共用回执；新请求重新查库，当前材料/方法优先。 +- 纯概念请求不查库；材料不明先澄清。“只准备/只生成”不授权提交；内部步骤 + 不能扩大父任务范围；“继续/重算”保留原授权，明确新提交指令才改变范围。 +- 本票仅做候选发现。候选尚未核验可复用；不归档未完成计算，不新增数据库连接。 + +使用说明见 [aidb_skill_hook.md](aidb_skill_hook.md)。这是首期硅任务的有限上下文 +识别，不是通用自然语言解析或任意自由 shell/Python 入口的约束保证。 + +## 测试 + +| 检查 | 结果 | +| --- | --- | +| MatCreator hook + 真实 Skill 加载器针对性回归 | 40 passed | +| 相邻 aidb adapter/client/server 契约回归 | 14 passed | +| Python compileall 与本次 diff whitespace 检查 | 通过 | +| 完整 `pytest tests/` | 两个既有旧导入错误导致收集失败 | +| 排除两个无法收集文件后的完整运行 | 743 passed / 39 failed；失败集合与基线完全相同，见 [基线对比](aidb_preflight_test_results.json) | + +完整测试的两个收集错误为 `test_execution_grouping.py` 和 +`test_matcreator_phase_and_skills.py` 使用过时的 `agents.MatCreator` 导入。 +已在起始提交 `d92c7a2973831d5da5214befbc888e8587387eb4` 的独立临时导出中 +复现同样错误。对可收集的完整测试也运行了原始基线，对比失败用例集合。 +原始基线为 710 passed / 39 failed；失败涵盖既有 benchmark、知识图谱、旧 +retry_config 断言、结构建模和前端文本断言。未为本票修改这些无关行为。 + +未安装或配置 mypy/pyright，本次执行 Python 编译检查与公开流程测试，未宣称 +静态类型检查已通过。没有前端业务改动，未运行前端构建。 +测试的查询使用真实隔离本地库与公开 CLI/client/server；准备和远程提交工具 +为受控测试边界，命中记录为隔离测试样本。没有提交真实 Bohrium 作业，没有 +真实科学结果归档验收，也没有云端数据库上传。 + +## Standards + +按 code-review 技能，由独立规范审查代理检查起始提交后的变更及后续修正。 +最初发现材料澄清绕过、提取失败缺少持久日志、继续/重算识别与授权继承问题。 +均增加公开工具边界测试并修正；统一继续识别规则以防词表漂移。 +最终复审：无剩余可操作规范问题。 + +## Spec + +由独立规格审查代理对照 issue #1 和 #2 检查。 +最初发现只准备授权扩大、旧硅目标污染新材料、直接准备工具缺少查询反馈， +以及继续请求绕过。均已修正，相关公开流程回归通过。 +最终复审：无剩余 T1 开发期 A 规格问题。 + +最终结果为 Standards 0 / Spec 0 个剩余可操作发现。 +完整计算、自动归档/重查、独立会话复用与最终 B 切换仍属于后续 T2–T5。 diff --git a/docs/aidb_skill_hook.md b/docs/aidb_skill_hook.md index 10e4adc..bfd6504 100644 --- a/docs/aidb_skill_hook.md +++ b/docs/aidb_skill_hook.md @@ -13,7 +13,7 @@ 前一用户请求，但当前方法优先。未设置 goal 不会跳过提取。 材料身份无法确定时先澄清；已确定 Si 时可先发现候选。PBE、bulk、diamond 缺失时只是检索默认值，反馈中的 known / retrieval_defaults / unknown 分别保存。 -准备计算前仍需明确结构模型。非首期晶相、体系或方法会暂停并说明支持边界。 +准备计算前仍需明确结构模型或提供已有结构。非首期晶相、体系或方法会暂停并说明支持边界。 这是固定硅样例的有限识别，不是通用自然语言或结构等价判定器。 ## 配置与公开接口 @@ -31,7 +31,7 @@ min_records=1，无相关材料扩展。每个 CLI 调用等待最多 20 秒， ## 反馈与阻断 Skill 正文正常加载；after_tool_callback 将 `aidb_preflight` 加入实际工具响应， -模型能够读取并向用户反馈。反馈同时写入 session_log 和运行日志。 +模型能够读取并向用户反馈；直接调用准备工具时其响应也携带同样证据。反馈同时写入 session_log 和运行日志。 成功响应读取完整 `preflight-report.json`，保留完整候选 ID、报告、审计位置。 `found` 是候选命中，不能当成已经核验可复用；本票不实现候选复用/归档。 `not_found` 是成功查询空候选；`failed` 是真实配置、连接、超时或协议失败。 @@ -41,7 +41,7 @@ Skill 正文正常加载；after_tool_callback 将 `aidb_preflight` 加入实际 开发期 A 中，失败或材料澄清状态阻止新结构/输入准备及 tracked remote-job 提交/上传/执行。只准备的请求即使查询成功也不授权提交。状态/结果读取工具 仍可使用。同任务同次 invocation 的重复入口共用回执；新轮次重新查库， -条件变化不会复用旧回执。此约束覆盖登记的工具入口，不承诺约束任意自由 +条件变化不会复用旧回执。并发入口通过同调用锁共用查询；自然“继续/重算”沿用任务条件与原授权，新一轮仍重新查库，明确提交授权才能扩大原准备范围。此约束覆盖登记的工具入口，不承诺约束任意自由 shell/Python 入口。后续 T5 才切换最终 B，不能把本票称为完整闭环验收。 ## 验证 diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index d00dced..520160e 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -14,10 +14,14 @@ import re import sys import uuid +import weakref from .session_log import append_session_log_entry logger = logging.getLogger(__name__) +# Locks live only while callbacks are running; no Futures enter ADK state. +_preflight_locks = weakref.WeakValueDictionary() +_CONTINUATION_RE = re.compile(r'继续|照旧|这个|该结构|改用|换成|切换材料|重算|重新计算|same|continue|recalculat|recompute|\bit\b', re.I) AIDB_PREFLIGHT_SKILLS = frozenset({'vasp-pymatgen', 'atomic-structure'}) _PREPARATION_TOOLS = frozenset({'run_python', 'run_bash', 'run_skill_script', 'submit_bohr_batchjob', 'submit_bohr_sandbox', 'upload_remote_job_input', @@ -35,63 +39,97 @@ def _request(tool_context): step = json.loads(text) except (ValueError, TypeError): step = None - if isinstance(step, dict) and 'action' in step: - text = '\n'.join(str(step.get(k) or '') for k in ('action', 'prior_context')) + tool_context.state['temp:aidb_internal_step'] = isinstance(step, dict) and 'action' in step + if tool_context.state['temp:aidb_internal_step']: + text = str(step['action']) + '\n[AIDB_PRIOR_CONTEXT]\n' + str(step.get('prior_context') or '') # Step requests inherit the parent's goal; it is current task context, # rather than a prior user turn, in this isolated executor invocation. text += '\n' + str(tool_context.state.get('goal') or '') - elif re.search(r'继续|这个|该结构|改用|重算|重新计算|same|\bit\b|recompute', text, re.I): + elif _CONTINUATION_RE.search(text): for event in reversed(invocation.session.events): prior = event.content if prior and prior.role == 'user' and event.invocation_id != tool_context.invocation_id: - text += '\n' + '\n'.join(p.text for p in (prior.parts or []) if p.text) + text += '\n[AIDB_PRIOR_CONTEXT]\n' + '\n'.join(p.text for p in (prior.parts or []) if p.text) break + if not tool_context.state['temp:aidb_internal_step']: + tool_context.state['aidb_user_request'] = text.partition('\n[AIDB_PRIOR_CONTEXT]\n')[0] return text -def _task_context(text, workspace): +def _known_context(text, workspace): lower = text.lower() - calculation = bool(re.search(r'弛豫|优化|总能|计算|准备|生成|relax|total energy|calculat|prepare|generate', lower)) - conceptual = bool(re.search(r'解释|介绍|什么是|概念|explain|what is|概念说明', lower)) - if not calculation or (conceptual and not re.search(r'请.*(?:准备|生成|执行)|please.*(?:prepare|run|calculate)', lower)): - return None known = {} - if re.search(r'单质硅|硅晶|硅体|硅单体|金刚石.*硅|\bsilicon\b|\bsi\b|硅', lower) and not re.search(r'氧化硅|二氧化硅|碳化硅|氮化硅|silica|sic\b|sio2\b', lower): + if re.search(r'单质硅|硅晶|硅体|硅单体|金刚石.*硅|\bsilicon\b|(? Date: Wed, 7 Oct 2026 14:23:38 +0800 Subject: [PATCH 04/22] feat: verify tracked silicon VASP outputs for T2 Implement scientific collection for Zikkying/Ai-ready_Database#3 under #1. Keep existing MatCreator calculation workflow and development A; real compute acceptance remains pending. --- docs/silicon_vasp_t2.md | 62 ++++++ .../agents/execution_agent/step_executor.py | 2 + src/matcreator/agents/thinking_agent/agent.py | 2 + src/matcreator/skills/vasp-pymatgen/SKILL.md | 22 +++ src/matcreator/tools/silicon_vasp.py | 162 ++++++++++++++++ tests/test_aidb_skill_hook.py | 8 +- tests/test_silicon_vasp_workflow.py | 178 ++++++++++++++++++ 7 files changed, 432 insertions(+), 4 deletions(-) create mode 100644 docs/silicon_vasp_t2.md create mode 100644 src/matcreator/tools/silicon_vasp.py create mode 100644 tests/test_silicon_vasp_workflow.py diff --git a/docs/silicon_vasp_t2.md b/docs/silicon_vasp_t2.md new file mode 100644 index 0000000..f01b004 --- /dev/null +++ b/docs/silicon_vasp_t2.md @@ -0,0 +1,62 @@ +# Silicon VASP T2: tracked collection and scientific completion + +Scope: [Ai-ready_Database #3](https://github.com/Zikkying/Ai-ready_Database/issues/3), +with [#1](https://github.com/Zikkying/Ai-ready_Database/issues/1) as the main spec. +T1 (#2) is closed. Development A remains in force: a failed aidb lookup pauses +new preparation and submission. T2 does not implement local archive, reuse or B. + +MatCreator owns the calculation. The existing atomic-structure / vasp-pymatgen +skills choose and generate the structure and pymatgen input sets. Bohrium +discovery, tracked submission, monitoring, reconnect and deduplicated collection +continue to use the existing remote-job tools. No new submission platform or +VASP parameter framework is introduced. + +For fixed diamond bulk Si / ordinary PBE, the VASP skill directs both agents to +`collect_silicon_vasp_result`. The tool is registered on the thinking agent and +the isolated step executor. It checks session ownership, respects user controls, +and collects only a platform-successful Batch Job. Running, failed, unknown and +unsupported jobs return a needs-replanning receipt without submitting anything. +Already-collected jobs reuse their durable artifact paths after reconnect. + +The parser requires complete XML, original and retained INCAR/POSCAR, CONTCAR, +OUTCAR, OSZICAR and original POTCAR provenance. It checks electronic convergence, +ionic convergence and the OUTCAR accuracy/termination evidence for relaxation, +separate static-step settings, PBE pseudopotential/method evidence, diamond Si +symmetry, finite energy, and agreement between XML and structure files. These +are fixed-example checks, not a general scientific compatibility scorer. + +Static total energy additionally requires the existing relaxation job ID in the +same session. The source output is parsed again; cached success JSON alone does +not establish success. The static input must match that relaxed structure and +use the same pseudopotentials. Changed, incomplete or unverified sources are +rejected. The two steps retain their separate identities and energies. + +`silicon-result.json` retains success/invalid status, tracked/provider identities, +actual structure, energy in eV, INCAR and effective parameters, pseudopotential +identity/hash, available KPOINTS, submission image/machine/settings, raw artifact +paths/checksums and relaxation source. Original outputs are preserved on failure. +`archived` is always false. T3 must only consume verified successful handoffs; +T2 never claims a record was saved or a local loop was completed. + +## Validation and real-run evidence + +```bash +PYTHONPATH=src .venv/bin/python -m pytest tests/test_silicon_vasp_workflow.py tests/test_aidb_skill_hook.py tests/test_remote_job_tools.py -q +``` + +The new tests use a controlled provider and explicitly synthetic VASP XML, +parsed by the real pymatgen parser. Agent-runner cases use a real isolated aidb +public bridge/database for query ordering and user-visible receipts. These +tests prove orchestration and scientific failure handling, not real computation. + +On 2026-10-07 a real natural request was dispatched to MatCreator with session +`silicon-t2-20261007`, using the existing user configuration and local aidb. +The real preflight succeeded with no Si/PBE/bulk candidates (no records deleted). +Private run evidence is retained under `.workspace/silicon-t2/`, including +`events.jsonl`, `trajectories/`, `.aidb/runs/` reports and audits. Real Bohrium +completion is still pending; no successful job ID or real T3 handoff is claimed. + +The initial noninteractive launcher omitted the NVM bohr path; the installed +CLI is `/home/shik-mechrevo-wsl/.nvm/versions/node/v24.14.1/bin/bohr` (2.6.100). +Future launches should prepend both the repository `.venv/bin` and this Node +directory to PATH, so `run_python` and bohr use the intended environment. diff --git a/src/matcreator/agents/execution_agent/step_executor.py b/src/matcreator/agents/execution_agent/step_executor.py index b81895e..2daf4c2 100644 --- a/src/matcreator/agents/execution_agent/step_executor.py +++ b/src/matcreator/agents/execution_agent/step_executor.py @@ -17,6 +17,7 @@ from ...tools.remoteagent_tool import load_remote_a2a_agents from ...tools.util_tools import show_artifact, show_plot, show_structure from ...tools.workspace_tools import get_user_skills_root, run_bash, run_python +from ...tools.silicon_vasp import collect_silicon_vasp_result from .remote_job_tools import ( attach_bohr_batchjob, collect_remote_job_outputs, @@ -320,6 +321,7 @@ def build_step_executor_agent(llm_card: LLMCard) -> LlmAgent: FunctionTool(upload_remote_job_input), FunctionTool(download_remote_job_output), FunctionTool(collect_remote_job_outputs), + FunctionTool(collect_silicon_vasp_result), FunctionTool(pause_remote_job), FunctionTool(terminate_remote_job), ALL_SKILLS_TOOLSET, diff --git a/src/matcreator/agents/thinking_agent/agent.py b/src/matcreator/agents/thinking_agent/agent.py index 7ebdaa4..1a3271d 100644 --- a/src/matcreator/agents/thinking_agent/agent.py +++ b/src/matcreator/agents/thinking_agent/agent.py @@ -47,6 +47,7 @@ ) from .history_tools import read_session_log from ..execution_agent.remote_job_tools import list_remote_jobs +from ...tools.silicon_vasp import collect_silicon_vasp_result logger = logging.getLogger(__name__) @@ -500,6 +501,7 @@ def before_agent_callback(callback_context: CallbackContext) -> None: FunctionTool(run_flash_step), FunctionTool(read_session_log), FunctionTool(list_remote_jobs), + FunctionTool(collect_silicon_vasp_result), FunctionTool(load_skill), show_artifact, show_plot, diff --git a/src/matcreator/skills/vasp-pymatgen/SKILL.md b/src/matcreator/skills/vasp-pymatgen/SKILL.md index eb88562..9c122c2 100644 --- a/src/matcreator/skills/vasp-pymatgen/SKILL.md +++ b/src/matcreator/skills/vasp-pymatgen/SKILL.md @@ -15,6 +15,7 @@ metadata: - run_python - run_bash - load_skill_resource + - collect_silicon_vasp_result dependent_skills: - bohrium - remote-job @@ -108,6 +109,27 @@ its own defaults — the agent never writes a raw INCAR. Run exactly **one property step at a time**. Do not chain relaxation + SCF in a single step. +### Fixed silicon local-loop task (development A) + +For diamond bulk Si / ordinary PBE relaxation and total energy, inspect the +actual `aidb_preflight` tool response before creating a structure or inputs. +Failed lookup pauses new computation. Missing identity/model must be clarified; +retrieval defaults are not calculation authorization. Do not archive in T2. + +Use the existing ASE diamond structure recipe and `MPRelaxSet`, then tracked +Batch Job submission. On completion call `collect_silicon_vasp_result(job_id, +destination_path, calculation_type="relaxation")`. Proceed to `MPStaticSet` +only after `status="success"`, using that exact relaxed CONTCAR. Submit the +static step separately. Collect with `calculation_type="static"` and the +verified `relaxation_job_id`. The tool rechecks the source outputs and their +relationship, and writes `silicon-result.json` for T3 with structure, energy +(eV), actual conditions, both job identities and checksums. Retain INCAR, +POSCAR, KPOINTS, POTCAR in the local input directory and declare the standard +outputs in the Batch Job reference. Neither a platform `succeeded` nor a +partial XML is scientific completion. On `invalid` / `needs_replanning`, +report the evidence and preserve the job and outputs; never auto-resubmit. +On reconnect use the same job_id and existing collected artifacts. + For **MLFF energy/force labeling**, use `prepare_label` as a standalone step — it replaces the relaxation → SCF chain with a single static calculation optimized for dataset generation (no charge density, no relaxation, KSPACING-based k-points). --- diff --git a/src/matcreator/tools/silicon_vasp.py b/src/matcreator/tools/silicon_vasp.py new file mode 100644 index 0000000..9060a0b --- /dev/null +++ b/src/matcreator/tools/silicon_vasp.py @@ -0,0 +1,162 @@ +"""Fixed silicon recipe and scientific handoff for tracked Bohrium jobs. + +Submission, monitoring and collection remain owned by remote-job. This module +does not archive results or upload to a cloud database. +""" +from __future__ import annotations + +import hashlib +import json +import math +from pathlib import Path +from typing import Literal + +from monty.json import MontyEncoder + +from google.adk.tools.tool_context import ToolContext + + +def _digest(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def _output_directory(artifacts: list, workspace: Path) -> Path: + paths = [Path(a['destination']).resolve() for a in artifacts if a.get('destination')] + if not paths or any(not p.is_relative_to(workspace) for p in paths): + raise ValueError('Collected outputs are unavailable in this workspace.') + candidates = {p if p.is_dir() else p.parent for p in paths} + matches = [p for p in candidates if (p / 'vasprun.xml').is_file()] + if len(matches) == 1: + return matches[0] + if len(candidates) == 1: + return candidates.pop() + raise ValueError('Cannot identify a unique VASP output directory.') + + +def _validate(directory: Path, calculation_type: str, input_directory: Path) -> dict: + from pymatgen.io.vasp.inputs import Incar, Poscar + from pymatgen.io.vasp.outputs import Vasprun + from pymatgen.symmetry.analyzer import SpacegroupAnalyzer + import numpy as np + + required = ['vasprun.xml', 'OUTCAR', 'OSZICAR', 'CONTCAR', 'INCAR', 'POSCAR'] + missing = [name for name in required if not (directory / name).is_file()] + if missing: + raise ValueError(f'Missing VASP outputs: {", ".join(missing)}; scientific completion unknown.') + for name in ('INCAR', 'POSCAR', 'POTCAR'): + if not (input_directory / name).is_file(): + raise ValueError(f'Original {name} unavailable; input provenance unknown.') + if name != 'POTCAR' and _digest(directory / name) != _digest(input_directory / name): + raise ValueError(f'Retained {name} differs from original job input.') + run = Vasprun(directory / 'vasprun.xml', parse_dos=False, parse_eigen=False, + parse_projected_eigen=False, parse_potcar_file=False, exception_on_bad_xml=True) + incar = Incar.from_file(directory / 'INCAR') + if not run.ionic_steps or any(not step.get('electronic_steps') for step in run.ionic_steps): + raise ValueError('Electronic/ionic step evidence missing.') + if not run.converged_electronic or any(len(step['electronic_steps']) >= int(run.parameters['NELM']) for step in run.ionic_steps): + raise ValueError('Electronic convergence not reached.') + outcar = (directory / 'OUTCAR').read_text(errors='replace') + if 'General timing and accounting informations for this job' not in outcar: + raise ValueError('VASP termination evidence missing from OUTCAR.') + nsw = int(run.parameters.get('NSW', 0)) + ibrion = int(run.parameters.get('IBRION', -1)) + if calculation_type == 'relaxation': + if nsw <= 1 or ibrion not in {1, 2} or not run.converged_ionic or 'reached required accuracy' not in outcar: + raise ValueError('Ionic relaxation convergence not established.') + elif nsw != 0 or ibrion != -1: + raise ValueError('Expected a separate static total-energy step (NSW=0, IBRION=-1).') + if run.parameters.get('LDAU', False) or run.parameters.get('LHFCALC', False) or run.incar.get('ML_LMLFF', False) or run.parameters.get('METAGGA', '') not in {'', 'None', 'NONE'}: + raise ValueError('Only ordinary PBE is supported; corrections/hybrid/meta-GGA detected.') + if str(run.parameters.get('GGA', 'PE')).upper() != 'PE' or not run.potcar_symbols or any(not s.startswith('PAW_PBE Si ') for s in run.potcar_symbols): + raise ValueError('Ordinary PBE Si pseudopotential evidence missing or incompatible.') + for structure in (run.initial_structure, run.final_structure): + if not structure.is_ordered or set(str(s) for s in structure.species) != {'Si'} or SpacegroupAnalyzer(structure, symprec=0.01).get_space_group_number() != 227: + raise ValueError('Output is outside fixed diamond bulk silicon scope.') + initial = Poscar.from_file(directory / 'POSCAR').structure + final = Poscar.from_file(directory / 'CONTCAR').structure + for actual, expected in ((initial, run.initial_structure), (final, run.final_structure)): + if actual.species != expected.species or not np.allclose(actual.lattice.matrix, expected.lattice.matrix, atol=1e-5) or not np.allclose(actual.frac_coords, expected.frac_coords, atol=1e-5): + raise ValueError('POSCAR/CONTCAR disagrees with vasprun.xml structure.') + if calculation_type == 'static' and (not np.allclose(initial.lattice.matrix, final.lattice.matrix, atol=1e-5) or not np.allclose(initial.frac_coords, final.frac_coords, atol=1e-5)): + raise ValueError('Static calculation changed its input structure.') + energy = float(run.final_energy) + if not math.isfinite(energy): + raise ValueError('Final total energy unavailable or non-finite.') + return {'structure': run.final_structure.as_dict(), 'initial_structure': run.initial_structure.as_dict(), + 'total_energy': {'value': energy, 'unit': 'eV'}, + 'conditions': {'functional': 'PBE', 'domain': 'bulk', 'structure_model': 'diamond', + 'software': 'VASP', 'version': run.vasp_version, 'incar': dict(incar), + 'parameters': run.parameters, 'potcar_symbols': run.potcar_symbols, + 'potcar_sha256': _digest(input_directory / 'POTCAR'), + 'kpoints': (directory / 'KPOINTS').read_text() if (directory / 'KPOINTS').is_file() else None}, + 'completion': {'electronic_converged': True, + 'ionic_converged': True if calculation_type == 'relaxation' else None, + 'ionic_steps': len(run.ionic_steps), 'outcar_termination': True}, + 'artifacts': {p.name: {'path': str(p), 'sha256': _digest(p)} for p in directory.iterdir() if p.is_file() and p.name != 'silicon-result.json'}} + + +def collect_silicon_vasp_result(job_id: str, destination_path: str, + calculation_type: Literal['relaxation', 'static'], tool_context: ToolContext, relaxation_job_id: str = '') -> dict: + """Collect an existing tracked Si Batch Job and verify scientific completion. + + Never submits or replaces a job. A platform success alone is not a + scientific success. Replays use the original durable collection paths. + """ + from matcreator.agents.execution_agent.remote_job_tools import ( + get_remote_job_status, collect_remote_job_outputs, _service) + if calculation_type not in {'relaxation', 'static'}: + return {'status': 'invalid', 'job_id': job_id, 'message': 'Expected relaxation or static.'} + job = get_remote_job_status(job_id, tool_context) + if job.get('status') == 'error': + return job + if job['provider'] != 'bohr_batchjob' or job.get('user_control') or job['status'] not in {'succeeded', 'collected'}: + return {'status': 'needs_replanning', 'job_id': job_id, 'platform_status': job['status'], + 'message': f"Existing job is {job['status']}; do not resubmit.", 'error': job.get('error')} + collected = collect_remote_job_outputs(job_id, destination_path, tool_context) + if collected.get('status') != 'collected': + return {**collected, 'status': 'needs_replanning'} + workspace = Path(tool_context.state['workspace_dir']).resolve() + try: + directory = _output_directory(collected['artifacts'], workspace) + except ValueError as exc: + return {'status': 'invalid', 'job_id': job_id, 'message': str(exc)} + report = {'job_id': job_id, 'batchjob_id': job['external_id'], 'calculation_type': calculation_type, + 'output_path': str(directory), 'archived': False} + try: + spec = _service().store.get_job(job_id)['specification'] + report['submission'] = {k: spec[k] for k in ('image', 'machine_type', 'sku_id', 'project_id', + 'command', 'input_path', 'out_files', 'max_run_time', 'max_wait_time') if k in spec} + input_directory = (Path(spec['input_root']) / spec['input_path']).resolve() + if not input_directory.is_relative_to(workspace): + raise ValueError('Original inputs are outside this workspace.') + report.update(_validate(directory, calculation_type, input_directory)) + if calculation_type == 'static': + if not relaxation_job_id or relaxation_job_id == job_id: + raise ValueError('Provide the preceding verified relaxation_job_id.') + parent = get_remote_job_status(relaxation_job_id, tool_context) + if parent.get('status') != 'collected' or parent.get('provider') != 'bohr_batchjob' or parent.get('user_control'): + raise ValueError('Preceding relaxation is not collected in this session.') + parent_job = _service().store.get_job(relaxation_job_id) + parent_dir = _output_directory(parent_job['artifacts'], workspace) + parent_spec = parent_job['specification'] + parent_input = (Path(parent_spec['input_root']) / parent_spec['input_path']).resolve() + if not parent_input.is_relative_to(workspace): + raise ValueError('Relaxation inputs are outside this workspace.') + parent_result = _validate(parent_dir, 'relaxation', parent_input) + import numpy as np + from pymatgen.core import Structure + relaxed = Structure.from_dict(parent_result['structure']) + static_input = Structure.from_dict(report['initial_structure']) + if relaxed.species != static_input.species or not np.allclose(relaxed.lattice.matrix, static_input.lattice.matrix, atol=1e-5) or not np.allclose(relaxed.frac_coords, static_input.frac_coords, atol=1e-5): + raise ValueError('Static input does not match the verified relaxed structure.') + if report['conditions']['potcar_sha256'] != parent_result['conditions']['potcar_sha256']: + raise ValueError('Static and relaxation pseudopotentials differ.') + report['relaxation_source'] = {'job_id': relaxation_job_id, 'batchjob_id': parent['external_id'], + 'output_path': str(parent_dir), 'total_energy': parent_result['total_energy']} + report['status'] = 'success' + except Exception as exc: + report.update(status='invalid', message=str(exc)) + report_path = directory / 'silicon-result.json' + report_path.write_text(json.dumps(report, cls=MontyEncoder, ensure_ascii=False, indent=2), encoding='utf-8') + return {key: report[key] for key in ('status', 'job_id', 'output_path', 'archived')} | { + 'report_path': str(report_path), 'message': report.get('message', 'Scientific completion verified; ready for T3, not archived.')} diff --git a/tests/test_aidb_skill_hook.py b/tests/test_aidb_skill_hook.py index 37a9284..8df6775 100644 --- a/tests/test_aidb_skill_hook.py +++ b/tests/test_aidb_skill_hook.py @@ -42,7 +42,7 @@ def local_aidb(tmp_path, monkeypatch): pass -def run_task(tmp_path, request, calls, state=None, entry=None, followup=None, parallel=False): +def run_task(tmp_path, request, calls, state=None, entry=None, followup=None, parallel=False, extra_tools=()): class Model(BaseLlm): _index: int = PrivateAttr(default=0) async def generate_content_async(self, llm_request, stream=False): @@ -78,19 +78,19 @@ async def run(): session = await service.create_session(app_name='preflight_test', user_id='test', state={'workspace_dir': str(tmp_path), **(state or {})}) agent = LlmAgent(name='task_agent', model=Model(model='test'), - tools=[load_skill, run_python, submit_bohr_batchjob], + tools=[load_skill, run_python, submit_bohr_batchjob, *extra_tools], before_tool_callback=hook.before_aidb_skill_load, after_tool_callback=hook.after_aidb_skill_load) if entry == 'thinking': from matcreator.agents.thinking_agent.agent import thinking_agent agent = thinking_agent.model_copy(update={'name': 'thinking_test', 'model': Model(model='test'), - 'tools': [load_skill, run_python, submit_bohr_batchjob], 'instruction': '', + 'tools': [load_skill, run_python, submit_bohr_batchjob, *extra_tools], 'instruction': '', 'before_agent_callback': None, 'sub_agents': []}) elif entry == 'step': from matcreator.agents.execution_agent.step_executor import build_step_executor_agent from matcreator.llm_cards import LLMCard agent = build_step_executor_agent(LLMCard(name='test', model='openai/test')).model_copy( - update={'model': Model(model='test'), 'tools': [load_skill, run_python, submit_bohr_batchjob], + update={'model': Model(model='test'), 'tools': [load_skill, run_python, submit_bohr_batchjob, *extra_tools], 'instruction': '', 'input_schema': None, 'sub_agents': []}) runner = Runner(agent=agent, app_name='preflight_test', session_service=service) responses = [] diff --git a/tests/test_silicon_vasp_workflow.py b/tests/test_silicon_vasp_workflow.py new file mode 100644 index 0000000..65559cc --- /dev/null +++ b/tests/test_silicon_vasp_workflow.py @@ -0,0 +1,178 @@ +from types import SimpleNamespace +from pathlib import Path +import json + +import pytest + +from matcreator.tools.silicon_vasp import collect_silicon_vasp_result +from matcreator.agents.execution_agent import remote_job_tools +from matcreator.control_plane.remote_jobs import RemoteJobStore +from matcreator.control_plane.remote_job_service import RemoteJobService +from matcreator.control_plane.providers.base import RemoteJobAdapter, RemoteJobCapability, RemoteJobStatus +from test_aidb_skill_hook import local_aidb, run_task + + +class Provider(RemoteJobAdapter): + provider = 'bohr_batchjob' + capabilities = frozenset({RemoteJobCapability.BATCH_COLLECT}) + + def create(self, spec): + return 'test-batch' + + def status(self, external_id): + return RemoteJobStatus(normalized_status='running', snapshot={}) + + def cancel(self, external_id): + pass + + def collect_outputs(self, external_id, destination_dir): + destination_dir.mkdir() + for name, content in getattr(self, 'outputs', {}).items(): + (destination_dir / name).write_text(content) + return [{'destination': str(p)} for p in destination_dir.iterdir()] or [{'destination': str(destination_dir)}] + + +def job_context(tmp_path, monkeypatch): + service = RemoteJobService(RemoteJobStore(tmp_path / 'jobs.db'), + adapter_overrides={'bohr_batchjob': Provider()}) + monkeypatch.setattr(remote_job_tools, '_service', lambda: service) + context = SimpleNamespace(state={'workspace_dir': str(tmp_path), 'session_id': 'session'}, + _invocation_context=SimpleNamespace(user_id='test')) + job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', + idempotency_key='silicon-relax', spec={'input_path': 'relax', 'input_root': str(tmp_path)}) + return service, context, job['job_id'] + + +def test_running_job_is_not_scientific_success_or_resubmitted(tmp_path, monkeypatch): + service, context, job_id = job_context(tmp_path, monkeypatch) + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'needs_replanning' + assert result['job_id'] == job_id + assert not (tmp_path / 'outputs').exists() + + +def test_platform_success_with_missing_outputs_is_not_scientific_success(tmp_path, monkeypatch): + service, context, job_id = job_context(tmp_path, monkeypatch) + service.store.transition_job(job_id, 'succeeded') + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' + assert 'vasprun.xml' in result['message'] + assert (tmp_path / 'outputs').exists() + + +def silicon_outputs(nelm=60, electronic_steps=2, ionic_marker=True, stage='relaxation'): + # Deliberately synthetic provider output; never evidence of real compute. + from pymatgen.core import Lattice, Structure + from pymatgen.io.vasp.inputs import Poscar + structure = Structure(Lattice([[0, 2.715, 2.715], [2.715, 0, 2.715], [2.715, 2.715, 0]]), + ['Si', 'Si'], [[0, 0, 0], [.25, .25, .25]]) + positions = '0 0 0.25 .25 .25' + cell = '0 2.715 2.7152.715 0 2.7152.715 2.715 0' + nsw, ibrion = (99, 2) if stage == 'relaxation' else (0, -1) + params = f'{nelm}{nsw}{ibrion}-.02520' + energy = '-10.1-10.1-10.1' + xml = f'''6.4 + {params}{params} + SiSi + 2Si28.0854PAW_PBE Si 05Jan2001 + {cell}{positions} + {''.join(''+energy+'' for _ in range(electronic_steps))}{energy} + {cell}{positions}0 0 00 0 0 + {cell}{positions}''' + return {'vasprun.xml': xml, 'POSCAR': str(Poscar(structure)), 'CONTCAR': str(Poscar(structure)), + 'INCAR': f'NELM = {nelm}\nNSW = {nsw}\nIBRION = {ibrion}\nENCUT = 520\nEDIFFG = -0.02\n', + 'OUTCAR': ('reached required accuracy - stopping structural energy minimisation\n' if ionic_marker else '') + 'General timing and accounting informations for this job:\n', + 'OSZICAR': ' 1 F= -.10100000E+02 E0= -.10100000E+02\n'} + + +def completed_job(tmp_path, monkeypatch, outputs): + service, context, job_id = job_context(tmp_path, monkeypatch) + service.adapter_for('bohr_batchjob').outputs = outputs + source = tmp_path / 'relax' + source.mkdir() + for name in ('INCAR', 'POSCAR'): + (source / name).write_text(outputs[name]) + (source / 'POTCAR').write_text('synthetic licensed-potential boundary') + service.store.transition_job(job_id, 'succeeded') + return service, context, job_id + + +def test_verified_relaxation_has_energy_structure_provenance_and_replays(tmp_path, monkeypatch): + service, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'success' + report = json.loads(Path(result['report_path']).read_text()) + assert report['total_energy'] == {'value': -10.1, 'unit': 'eV'} + assert report['job_id'] == job_id and report['batchjob_id'] == 'test-batch' + assert report['structure']['sites'][0]['species'][0]['element'] == 'Si' + assert report['conditions']['parameters']['ENCUT'] == 520 + assert report['archived'] is False + # Same durable store in a new service instance simulates reconnect. + monkeypatch.setattr(remote_job_tools, '_service', lambda: RemoteJobService(service.store)) + replay = collect_silicon_vasp_result(job_id, 'other-output-path', 'relaxation', context) + assert replay['report_path'] == result['report_path'] + assert not (tmp_path / 'other-output-path').exists() + + +def test_static_without_verified_relaxation_is_not_a_completed_two_step_task(tmp_path, monkeypatch): + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs(stage='static')) + result = collect_silicon_vasp_result(job_id, 'outputs', 'static', context) + assert result['status'] == 'invalid' + assert 'relaxation_job_id' in result['message'] + + +@pytest.mark.parametrize('failure', ['electronic', 'ionic', 'truncated', 'wrong_method', 'wrong_structure', 'changed_input']) +def test_bad_scientific_outputs_are_preserved_but_rejected(tmp_path, monkeypatch, failure): + outputs = silicon_outputs(nelm=2) if failure == 'electronic' else silicon_outputs(ionic_marker=failure != 'ionic') + if failure == 'truncated': + outputs['vasprun.xml'] = outputs['vasprun.xml'][:-12] + if failure == 'wrong_method': + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace('PAW_PBE Si', 'PAW_LDA Si') + if failure == 'wrong_structure': + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace('Si', 'Ge') + _, context, job_id = completed_job(tmp_path, monkeypatch, outputs) + if failure == 'changed_input': + (tmp_path / 'relax/INCAR').write_text('NSW=0') + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' + assert (tmp_path / 'outputs/vasprun.xml').read_text() == outputs['vasprun.xml'] + report = json.loads(Path(result['report_path']).read_text()) + assert report['status'] == 'invalid' and report['archived'] is False + + +def test_static_total_energy_keeps_verified_relaxation_source(tmp_path, monkeypatch): + service, context, parent_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + assert collect_silicon_vasp_result(parent_id, 'relax-outputs', 'relaxation', context)['status'] == 'success' + outputs = silicon_outputs(stage='static') + source = tmp_path / 'static' + source.mkdir() + for name in ('INCAR', 'POSCAR'): + (source / name).write_text(outputs[name]) + (source / 'POTCAR').write_bytes((tmp_path / 'relax/POTCAR').read_bytes()) + service.adapter_for('bohr_batchjob').outputs = outputs + job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', + idempotency_key='silicon-static', spec={'input_path': 'static', 'input_root': str(tmp_path)}) + service.store.transition_job(job['job_id'], 'succeeded') + result = collect_silicon_vasp_result(job['job_id'], 'static-outputs', 'static', context, parent_id) + assert result['status'] == 'success' + report = json.loads(Path(result['report_path']).read_text()) + assert report['relaxation_source']['job_id'] == parent_id + assert report['completion']['ionic_converged'] is None + # Source outputs are checked again; editing a cached JSON cannot confer success. + (tmp_path / 'relax-outputs/OUTCAR').write_text('incomplete output') + replay = collect_silicon_vasp_result(job['job_id'], 'ignored', 'static', context, parent_id) + assert replay['status'] == 'invalid' + + +@pytest.mark.parametrize('entry', ['thinking', 'step']) +def test_natural_agent_request_queries_then_reads_scientific_result(local_aidb, tmp_path, monkeypatch, entry): + _, _, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + calls = [('load_skill', {'skill_name': 'vasp-pymatgen'}), + ('run_python', {'code': 'prepare'}), + ('collect_silicon_vasp_result', {'job_id': job_id, 'destination_path': 'outputs', 'calculation_type': 'relaxation'})] + responses, prepared = run_task(tmp_path, '重新计算金刚石单质硅体相普通 PBE 弛豫和总能', + calls, state={'session_id': 'session'}, entry=entry, extra_tools=[collect_silicon_vasp_result]) + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert prepared == ['prepare'] + assert responses[-1]['status'] == 'success' + assert Path(responses[-1]['report_path']).is_file() From 3c742dea29c4942daceeb72da98ddd7a582375b0 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:35:36 +0800 Subject: [PATCH 05/22] fix: bind silicon handoff to submitted VASP inputs --- docs/silicon_vasp_t2.md | 25 +++++- .../execution_agent/remote_job_tools.py | 8 ++ src/matcreator/skills/vasp-pymatgen/SKILL.md | 4 + src/matcreator/tools/silicon_vasp.py | 80 +++++++++++++------ tests/test_silicon_vasp_workflow.py | 75 ++++++++++++++--- 5 files changed, 156 insertions(+), 36 deletions(-) diff --git a/docs/silicon_vasp_t2.md b/docs/silicon_vasp_t2.md index f01b004..a7409cf 100644 --- a/docs/silicon_vasp_t2.md +++ b/docs/silicon_vasp_t2.md @@ -19,7 +19,13 @@ unsupported jobs return a needs-replanning receipt without submitting anything. Already-collected jobs reuse their durable artifact paths after reconnect. The parser requires complete XML, original and retained INCAR/POSCAR, CONTCAR, -OUTCAR, OSZICAR and original POTCAR provenance. It checks electronic convergence, +OUTCAR, OSZICAR and original POTCAR provenance. Tracked Batch Job submission +captures SHA-256 checksums for materialized VASP inputs in its durable +specification before invoking the provider. Collection verifies local/retained +inputs against that snapshot, and matches submitted POTCAR TITEL metadata with +the actual XML potential identity. Old jobs without submission evidence remain +unverified. Replaying a submit does not overwrite existing provenance. +It checks electronic convergence, ionic convergence and the OUTCAR accuracy/termination evidence for relaxation, separate static-step settings, PBE pseudopotential/method evidence, diamond Si symmetry, finite energy, and agreement between XML and structure files. These @@ -56,6 +62,23 @@ Private run evidence is retained under `.workspace/silicon-t2/`, including `events.jsonl`, `trajectories/`, `.aidb/runs/` reports and audits. Real Bohrium completion is still pending; no successful job ID or real T3 handoff is claimed. +The corrected launch session `silicon-t2-20261007-env2` also queried the real +local database successfully (empty candidates) and generated diamond Si2 using +the existing ASE skill. MatCreator stopped before submission after an actual +MPRelaxSet POTCAR check raised `PmgVaspPspDirError`: no licensed PBE potential +directory is configured. `compute.vasp_image` / `BOHRIUM_VASP_IMAGE` is also +empty. Bohrium CLI authentication and machine discovery succeeded. The specific +diagnostic is `.workspace/silicon-t2/preflight_env_report_step2.json`; no tracked +job was submitted and no successful real calculation is claimed. + +Automated validation: the initial targeted run passed 117 tests; new workflow +tests also passed in both production agent entry points. Full-suite execution +passed 756 tests, with 39 failures and two collection errors. A snapshot of +the starting commit reproduced exactly the same failure names (no new failed +tests). The baseline snapshot skipped aidb integration because it was outside +the adjacent-checkout layout. Final review added regression coverage for mutable +POTCAR inputs, dispersion corrections, and submission replay provenance. + The initial noninteractive launcher omitted the NVM bohr path; the installed CLI is `/home/shik-mechrevo-wsl/.nvm/versions/node/v24.14.1/bin/bohr` (2.6.100). Future launches should prepend both the repository `.venv/bin` and this Node diff --git a/src/matcreator/agents/execution_agent/remote_job_tools.py b/src/matcreator/agents/execution_agent/remote_job_tools.py index 089f6b7..d3eeeb9 100644 --- a/src/matcreator/agents/execution_agent/remote_job_tools.py +++ b/src/matcreator/agents/execution_agent/remote_job_tools.py @@ -349,6 +349,7 @@ def submit_bohr_batchjob( if out_files_error: return {"status": "error", "message": out_files_error} input_root: str | None = None + vasp_input_hashes: dict[str, str] = {} if input_path is not None: if not isinstance(input_path, str) or not input_path.strip(): return { @@ -379,6 +380,11 @@ def submit_bohr_batchjob( } input_path = str(source.relative_to(workspace)) input_root = str(workspace) + from ...tools.silicon_vasp import capture_vasp_input_hashes + try: + vasp_input_hashes = capture_vasp_input_hashes(source) + except (OSError, ValueError) as exc: + return {'status': 'error', 'message': f'VASP input provenance failed: {exc}'} spec = { "project_id": resolved_project_id, "name": name, @@ -392,6 +398,8 @@ def submit_bohr_batchjob( "max_run_time": max_run_time, "max_wait_time": max_wait_time, } + if vasp_input_hashes: + spec['vasp_input_sha256'] = vasp_input_hashes result = _submit( tool_context, provider="bohr_batchjob", diff --git a/src/matcreator/skills/vasp-pymatgen/SKILL.md b/src/matcreator/skills/vasp-pymatgen/SKILL.md index 9c122c2..b1f5662 100644 --- a/src/matcreator/skills/vasp-pymatgen/SKILL.md +++ b/src/matcreator/skills/vasp-pymatgen/SKILL.md @@ -129,6 +129,10 @@ outputs in the Batch Job reference. Neither a platform `succeeded` nor a partial XML is scientific completion. On `invalid` / `needs_replanning`, report the evidence and preserve the job and outputs; never auto-resubmit. On reconnect use the same job_id and existing collected artifacts. +Tracked submission saves input checksums; preserve the local input directory. +Old jobs without that evidence remain unverified. Ordinary PBE excludes U, +hybrid, meta-GGA and dispersion corrections. The report retains k-point inputs +and the actual XML sampling when available. For **MLFF energy/force labeling**, use `prepare_label` as a standalone step — it replaces the relaxation → SCF chain with a single static calculation optimized for dataset generation (no charge density, no relaxation, KSPACING-based k-points). diff --git a/src/matcreator/tools/silicon_vasp.py b/src/matcreator/tools/silicon_vasp.py index 9060a0b..3b645b3 100644 --- a/src/matcreator/tools/silicon_vasp.py +++ b/src/matcreator/tools/silicon_vasp.py @@ -1,4 +1,4 @@ -"""Fixed silicon recipe and scientific handoff for tracked Bohrium jobs. +"""Scientific silicon handoff for tracked Bohrium jobs. Submission, monitoring and collection remain owned by remote-job. This module does not archive results or upload to a cloud database. @@ -8,6 +8,7 @@ import hashlib import json import math +import re from pathlib import Path from typing import Literal @@ -17,7 +18,32 @@ def _digest(path: Path) -> str: - return hashlib.sha256(path.read_bytes()).hexdigest() + with path.open('rb') as stream: + return hashlib.file_digest(stream, 'sha256').hexdigest() + + +def capture_vasp_input_hashes(source: Path) -> dict: + """Capture materialized VASP inputs before tracked submission, without contents.""" + if not source.is_dir() or not all((source / name).is_file() for name in ('INCAR', 'POSCAR', 'POTCAR')): + return {} + files = [source / name for name in ('INCAR', 'POSCAR', 'POTCAR', 'KPOINTS') if (source / name).exists()] + if any(p.is_symlink() for p in files): + raise ValueError('VASP inputs must be regular files, not symbolic links.') + return {p.name: _digest(p) for p in files} + + +def _input_directory(spec: dict, workspace: Path) -> Path: + path = (Path(spec['input_root']) / spec['input_path']).resolve() + if not path.is_relative_to(workspace): + raise ValueError('Original inputs are outside this workspace.') + return path + + +def _same_structure(actual, expected) -> bool: + import numpy as np + return (actual.species == expected.species + and np.allclose(actual.lattice.matrix, expected.lattice.matrix, atol=1e-5, rtol=0) + and np.allclose(actual.frac_coords, expected.frac_coords, atol=1e-5, rtol=0)) def _output_directory(artifacts: list, workspace: Path) -> Path: @@ -33,21 +59,24 @@ def _output_directory(artifacts: list, workspace: Path) -> Path: raise ValueError('Cannot identify a unique VASP output directory.') -def _validate(directory: Path, calculation_type: str, input_directory: Path) -> dict: +def _validate(directory: Path, calculation_type: str, input_directory: Path, input_hashes: dict) -> dict: from pymatgen.io.vasp.inputs import Incar, Poscar from pymatgen.io.vasp.outputs import Vasprun from pymatgen.symmetry.analyzer import SpacegroupAnalyzer - import numpy as np required = ['vasprun.xml', 'OUTCAR', 'OSZICAR', 'CONTCAR', 'INCAR', 'POSCAR'] missing = [name for name in required if not (directory / name).is_file()] if missing: raise ValueError(f'Missing VASP outputs: {", ".join(missing)}; scientific completion unknown.') - for name in ('INCAR', 'POSCAR', 'POTCAR'): - if not (input_directory / name).is_file(): - raise ValueError(f'Original {name} unavailable; input provenance unknown.') - if name != 'POTCAR' and _digest(directory / name) != _digest(input_directory / name): - raise ValueError(f'Retained {name} differs from original job input.') + if not all(input_hashes.get(name) for name in ('INCAR', 'POSCAR', 'POTCAR')): + raise ValueError('Submission-time VASP input hashes unavailable; provenance unknown.') + for name, digest in input_hashes.items(): + if name not in {'INCAR', 'POSCAR', 'POTCAR', 'KPOINTS'}: + raise ValueError('Unrecognized VASP input provenance.') + if not (input_directory / name).is_file() or _digest(input_directory / name) != digest: + raise ValueError(f'Original {name} changed since tracked submission.') + if (name != 'POTCAR' or (directory / name).exists()) and (not (directory / name).is_file() or _digest(directory / name) != digest): + raise ValueError(f'Retained {name} differs from submitted job input.') run = Vasprun(directory / 'vasprun.xml', parse_dos=False, parse_eigen=False, parse_projected_eigen=False, parse_potcar_file=False, exception_on_bad_xml=True) incar = Incar.from_file(directory / 'INCAR') @@ -65,19 +94,25 @@ def _validate(directory: Path, calculation_type: str, input_directory: Path) -> raise ValueError('Ionic relaxation convergence not established.') elif nsw != 0 or ibrion != -1: raise ValueError('Expected a separate static total-energy step (NSW=0, IBRION=-1).') - if run.parameters.get('LDAU', False) or run.parameters.get('LHFCALC', False) or run.incar.get('ML_LMLFF', False) or run.parameters.get('METAGGA', '') not in {'', 'None', 'NONE'}: + if (run.parameters.get('LDAU', False) or run.parameters.get('LHFCALC', False) + or run.incar.get('ML_LMLFF', False) or run.parameters.get('METAGGA', '') not in {'', 'None', 'NONE'} + or run.parameters.get('IVDW', 0) or run.parameters.get('LUSE_VDW', False) + or incar.get('IVDW', 0) or incar.get('LUSE_VDW', False)): raise ValueError('Only ordinary PBE is supported; corrections/hybrid/meta-GGA detected.') if str(run.parameters.get('GGA', 'PE')).upper() != 'PE' or not run.potcar_symbols or any(not s.startswith('PAW_PBE Si ') for s in run.potcar_symbols): raise ValueError('Ordinary PBE Si pseudopotential evidence missing or incompatible.') + titles = re.findall(r'^\s*TITEL\s*=\s*(.+?)\s*$', (input_directory / 'POTCAR').read_text(errors='replace'), re.M) + if [' '.join(t.split()) for t in titles] != [' '.join(t.split()) for t in run.potcar_symbols]: + raise ValueError('Submitted POTCAR identity disagrees with actual VASP output.') for structure in (run.initial_structure, run.final_structure): if not structure.is_ordered or set(str(s) for s in structure.species) != {'Si'} or SpacegroupAnalyzer(structure, symprec=0.01).get_space_group_number() != 227: raise ValueError('Output is outside fixed diamond bulk silicon scope.') initial = Poscar.from_file(directory / 'POSCAR').structure final = Poscar.from_file(directory / 'CONTCAR').structure for actual, expected in ((initial, run.initial_structure), (final, run.final_structure)): - if actual.species != expected.species or not np.allclose(actual.lattice.matrix, expected.lattice.matrix, atol=1e-5) or not np.allclose(actual.frac_coords, expected.frac_coords, atol=1e-5): + if not _same_structure(actual, expected): raise ValueError('POSCAR/CONTCAR disagrees with vasprun.xml structure.') - if calculation_type == 'static' and (not np.allclose(initial.lattice.matrix, final.lattice.matrix, atol=1e-5) or not np.allclose(initial.frac_coords, final.frac_coords, atol=1e-5)): + if calculation_type == 'static' and not _same_structure(initial, final): raise ValueError('Static calculation changed its input structure.') energy = float(run.final_energy) if not math.isfinite(energy): @@ -87,8 +122,10 @@ def _validate(directory: Path, calculation_type: str, input_directory: Path) -> 'conditions': {'functional': 'PBE', 'domain': 'bulk', 'structure_model': 'diamond', 'software': 'VASP', 'version': run.vasp_version, 'incar': dict(incar), 'parameters': run.parameters, 'potcar_symbols': run.potcar_symbols, - 'potcar_sha256': _digest(input_directory / 'POTCAR'), - 'kpoints': (directory / 'KPOINTS').read_text() if (directory / 'KPOINTS').is_file() else None}, + 'potcar_sha256': input_hashes['POTCAR'], 'input_sha256': input_hashes, + 'kpoints': (directory / 'KPOINTS').read_text() if (directory / 'KPOINTS').is_file() else None, + 'actual_kpoints': getattr(run, 'actual_kpoints', None), + 'actual_kpoint_weights': getattr(run, 'actual_kpoints_weights', None)}, 'completion': {'electronic_converged': True, 'ionic_converged': True if calculation_type == 'relaxation' else None, 'ionic_steps': len(run.ionic_steps), 'outcar_termination': True}, @@ -126,10 +163,8 @@ def collect_silicon_vasp_result(job_id: str, destination_path: str, spec = _service().store.get_job(job_id)['specification'] report['submission'] = {k: spec[k] for k in ('image', 'machine_type', 'sku_id', 'project_id', 'command', 'input_path', 'out_files', 'max_run_time', 'max_wait_time') if k in spec} - input_directory = (Path(spec['input_root']) / spec['input_path']).resolve() - if not input_directory.is_relative_to(workspace): - raise ValueError('Original inputs are outside this workspace.') - report.update(_validate(directory, calculation_type, input_directory)) + input_directory = _input_directory(spec, workspace) + report.update(_validate(directory, calculation_type, input_directory, spec.get('vasp_input_sha256', {}))) if calculation_type == 'static': if not relaxation_job_id or relaxation_job_id == job_id: raise ValueError('Provide the preceding verified relaxation_job_id.') @@ -139,15 +174,12 @@ def collect_silicon_vasp_result(job_id: str, destination_path: str, parent_job = _service().store.get_job(relaxation_job_id) parent_dir = _output_directory(parent_job['artifacts'], workspace) parent_spec = parent_job['specification'] - parent_input = (Path(parent_spec['input_root']) / parent_spec['input_path']).resolve() - if not parent_input.is_relative_to(workspace): - raise ValueError('Relaxation inputs are outside this workspace.') - parent_result = _validate(parent_dir, 'relaxation', parent_input) - import numpy as np + parent_input = _input_directory(parent_spec, workspace) + parent_result = _validate(parent_dir, 'relaxation', parent_input, parent_spec.get('vasp_input_sha256', {})) from pymatgen.core import Structure relaxed = Structure.from_dict(parent_result['structure']) static_input = Structure.from_dict(report['initial_structure']) - if relaxed.species != static_input.species or not np.allclose(relaxed.lattice.matrix, static_input.lattice.matrix, atol=1e-5) or not np.allclose(relaxed.frac_coords, static_input.frac_coords, atol=1e-5): + if not _same_structure(relaxed, static_input): raise ValueError('Static input does not match the verified relaxed structure.') if report['conditions']['potcar_sha256'] != parent_result['conditions']['potcar_sha256']: raise ValueError('Static and relaxation pseudopotentials differ.') diff --git a/tests/test_silicon_vasp_workflow.py b/tests/test_silicon_vasp_workflow.py index 65559cc..83e7f31 100644 --- a/tests/test_silicon_vasp_workflow.py +++ b/tests/test_silicon_vasp_workflow.py @@ -1,6 +1,7 @@ from types import SimpleNamespace from pathlib import Path import json +import hashlib import pytest @@ -20,7 +21,7 @@ def create(self, spec): return 'test-batch' def status(self, external_id): - return RemoteJobStatus(normalized_status='running', snapshot={}) + return RemoteJobStatus(normalized_status='succeeded' if getattr(self, 'finished', False) else 'running', snapshot={}) def cancel(self, external_id): pass @@ -32,14 +33,22 @@ def collect_outputs(self, external_id, destination_dir): return [{'destination': str(p)} for p in destination_dir.iterdir()] or [{'destination': str(destination_dir)}] -def job_context(tmp_path, monkeypatch): +def job_context(tmp_path, monkeypatch, outputs=None): service = RemoteJobService(RemoteJobStore(tmp_path / 'jobs.db'), adapter_overrides={'bohr_batchjob': Provider()}) monkeypatch.setattr(remote_job_tools, '_service', lambda: service) context = SimpleNamespace(state={'workspace_dir': str(tmp_path), 'session_id': 'session'}, _invocation_context=SimpleNamespace(user_id='test')) + spec = {'input_path': 'relax', 'input_root': str(tmp_path)} + if outputs is not None: + source = tmp_path / 'relax' + source.mkdir() + for name in ('INCAR', 'POSCAR', 'KPOINTS'): + (source / name).write_text(outputs[name]) + (source / 'POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\nsynthetic test potential, not for computation\n') + spec['vasp_input_sha256'] = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in source.iterdir()} job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', - idempotency_key='silicon-relax', spec={'input_path': 'relax', 'input_root': str(tmp_path)}) + idempotency_key='silicon-relax', spec=spec) return service, context, job['job_id'] @@ -81,18 +90,14 @@ def silicon_outputs(nelm=60, electronic_steps=2, ionic_marker=True, stage='relax {cell}{positions}''' return {'vasprun.xml': xml, 'POSCAR': str(Poscar(structure)), 'CONTCAR': str(Poscar(structure)), 'INCAR': f'NELM = {nelm}\nNSW = {nsw}\nIBRION = {ibrion}\nENCUT = 520\nEDIFFG = -0.02\n', + 'KPOINTS': 'Synthetic 4x4x4\n0\nGamma\n4 4 4\n0 0 0\n', 'OUTCAR': ('reached required accuracy - stopping structural energy minimisation\n' if ionic_marker else '') + 'General timing and accounting informations for this job:\n', 'OSZICAR': ' 1 F= -.10100000E+02 E0= -.10100000E+02\n'} def completed_job(tmp_path, monkeypatch, outputs): - service, context, job_id = job_context(tmp_path, monkeypatch) + service, context, job_id = job_context(tmp_path, monkeypatch, outputs) service.adapter_for('bohr_batchjob').outputs = outputs - source = tmp_path / 'relax' - source.mkdir() - for name in ('INCAR', 'POSCAR'): - (source / name).write_text(outputs[name]) - (source / 'POTCAR').write_text('synthetic licensed-potential boundary') service.store.transition_job(job_id, 'succeeded') return service, context, job_id @@ -146,12 +151,13 @@ def test_static_total_energy_keeps_verified_relaxation_source(tmp_path, monkeypa outputs = silicon_outputs(stage='static') source = tmp_path / 'static' source.mkdir() - for name in ('INCAR', 'POSCAR'): + for name in ('INCAR', 'POSCAR', 'KPOINTS'): (source / name).write_text(outputs[name]) (source / 'POTCAR').write_bytes((tmp_path / 'relax/POTCAR').read_bytes()) service.adapter_for('bohr_batchjob').outputs = outputs job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', - idempotency_key='silicon-static', spec={'input_path': 'static', 'input_root': str(tmp_path)}) + idempotency_key='silicon-static', spec={'input_path': 'static', 'input_root': str(tmp_path), + 'vasp_input_sha256': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in source.iterdir()}}) service.store.transition_job(job['job_id'], 'succeeded') result = collect_silicon_vasp_result(job['job_id'], 'static-outputs', 'static', context, parent_id) assert result['status'] == 'success' @@ -164,6 +170,53 @@ def test_static_total_energy_keeps_verified_relaxation_source(tmp_path, monkeypa assert replay['status'] == 'invalid' +def test_modified_potential_cannot_change_submitted_provenance(tmp_path, monkeypatch): + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + (tmp_path / 'relax/POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\nchanged potential\n') + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' + assert 'POTCAR' in result['message'] + + +@pytest.mark.parametrize('correction', ['IVDW', 'LUSE_VDW']) +def test_dispersion_corrected_pbe_is_outside_ordinary_pbe(tmp_path, monkeypatch, correction): + outputs = silicon_outputs() + tag = '12' if correction == 'IVDW' else 'T' + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace('', tag + '') + _, context, job_id = completed_job(tmp_path, monkeypatch, outputs) + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' + assert 'ordinary PBE' in result['message'] + + +def test_tracked_submit_binds_potential_before_transfer_and_replay(tmp_path, monkeypatch): + provider = Provider() + provider.outputs = silicon_outputs() + provider.finished = True + service = RemoteJobService(RemoteJobStore(tmp_path / 'jobs.db'), adapter_overrides={'bohr_batchjob': provider}) + monkeypatch.setattr(remote_job_tools, '_service', lambda: service) + from matcreator.agents.execution_agent import recovery + monkeypatch.setattr(recovery, 'ADK_DIR', tmp_path) + context = SimpleNamespace(state={'workspace_dir': str(tmp_path), 'session_id': 'snapshot-test'}, + _invocation_context=SimpleNamespace(user_id='test')) + source = tmp_path / 'relax' + source.mkdir() + for name in ('INCAR', 'POSCAR', 'KPOINTS'): + (source / name).write_text(provider.outputs[name]) + (source / 'POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\nsynthetic potential\n') + args = dict(name='test-si', image='test-image', machine_type='test-cpu', project_id=1, + command='test provider', input_path='relax') + submission = remote_job_tools.submit_bohr_batchjob(context, **args) + result = collect_silicon_vasp_result(submission['job_id'], 'outputs', 'relaxation', context) + assert result['status'] == 'success' + (source / 'POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\nchanged potential\n') + repeated = remote_job_tools.submit_bohr_batchjob(context, **args) + assert repeated['job_id'] == submission['job_id'] + checked = collect_silicon_vasp_result(repeated['job_id'], 'unused', 'relaxation', context) + assert checked['status'] == 'invalid' and 'POTCAR' in checked['message'] + assert not (tmp_path / 'unused').exists() + + @pytest.mark.parametrize('entry', ['thinking', 'step']) def test_natural_agent_request_queries_then_reads_scientific_result(local_aidb, tmp_path, monkeypatch, entry): _, _, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) From 1397059444d6e915e7c5ffc02c3c557a8fe2c846 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:38:45 +0800 Subject: [PATCH 06/22] docs: record T2 review and blocked real acceptance --- docs/silicon_vasp_t2.md | 37 ++++++++++++++++++++++++++++++++++--- 1 file changed, 34 insertions(+), 3 deletions(-) diff --git a/docs/silicon_vasp_t2.md b/docs/silicon_vasp_t2.md index a7409cf..6142d3c 100644 --- a/docs/silicon_vasp_t2.md +++ b/docs/silicon_vasp_t2.md @@ -71,15 +71,46 @@ empty. Bohrium CLI authentication and machine discovery succeeded. The specific diagnostic is `.workspace/silicon-t2/preflight_env_report_step2.json`; no tracked job was submitted and no successful real calculation is claimed. -Automated validation: the initial targeted run passed 117 tests; new workflow -tests also passed in both production agent entry points. Full-suite execution -passed 756 tests, with 39 failures and two collection errors. A snapshot of +Automated validation: the final targeted run passed 252 tests, including the +workflow in both production agent entry points. Final full-suite execution +passed 760 tests, with 39 failures and two collection errors. A snapshot of the starting commit reproduced exactly the same failure names (no new failed tests). The baseline snapshot skipped aidb integration because it was outside the adjacent-checkout layout. Final review added regression coverage for mutable POTCAR inputs, dispersion corrections, and submission replay provenance. +The user will consult the host about Bohrium configuration. Real acceptance is +paused pending the licensed PBE directory and authorized VASP image; the existing +calculation authorization remains valid. Issue #3 is not marked complete. The initial noninteractive launcher omitted the NVM bohr path; the installed CLI is `/home/shik-mechrevo-wsl/.nvm/versions/node/v24.14.1/bin/bohr` (2.6.100). Future launches should prepend both the repository `.venv/bin` and this Node directory to PATH, so `run_python` and bohr use the intended environment. + +## Standards review + +Reviewed implementation commits `db7904a` and `3c742de` against starting commit +`071cfab6202b704704a27376d7f136dec8e5b884` with +`git diff 071cfab6202b704704a27376d7f136dec8e5b884...HEAD`. + +Final review: 0 documented-standard breaches and 0 remaining baseline smells. +The duplicated structure comparisons and input-path checks now use shared +helpers. The initial POTCAR provenance concern is resolved by submission-time +hashes, collection checks and XML identity matching. Synthetic tests do not +establish real scientific completion. + +## Spec review + +The initial P1 (mutable POTCAR provenance) and P2 (dispersion-corrected PBE +accepted as ordinary PBE) are resolved. Submission replay preserves the original +input evidence; modified potentials are rejected. IVDW/LUSE_VDW are rejected, +and k-point inputs plus available XML sampling are preserved. No additional +code-correctness or scope issue was identified in the final review. + +One acceptance requirement remains unmet: one real successful Bohrium silicon +task. Actual query, structure generation and environment discovery are evidenced, +but missing PBE potentials and VASP image prevented submission. No successful +job, archive or completed local loop is claimed. + +Review totals: Standards 0 findings; Spec 1 outstanding environment-blocked +acceptance requirement (real Bohrium completion). From f4b52f5e8cd53b6882c3ede7f5db1e55e22408cf Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 14:55:58 +0800 Subject: [PATCH 07/22] fix: preserve MatCreator subprocess Python environment Login-shell profiles displaced the harness venv and bare python used a stale interpreter. Pin Python tools to sys.executable and inherit the configured PATH in non-login Bash tools. Record authenticated bohr and private VASP image discovery; real Si acceptance remains blocked on verified PBE potentials. --- docs/silicon_vasp_t2.md | 14 ++- docs/vasp_runtime_diagnosis.md | 134 ++++++++++++++++++++++++ src/matcreator/tools/workspace_tools.py | 11 +- tests/test_workspace_tools.py | 46 ++++++++ 4 files changed, 197 insertions(+), 8 deletions(-) create mode 100644 docs/vasp_runtime_diagnosis.md diff --git a/docs/silicon_vasp_t2.md b/docs/silicon_vasp_t2.md index 6142d3c..05f7bac 100644 --- a/docs/silicon_vasp_t2.md +++ b/docs/silicon_vasp_t2.md @@ -78,14 +78,18 @@ the starting commit reproduced exactly the same failure names (no new failed tests). The baseline snapshot skipped aidb integration because it was outside the adjacent-checkout layout. Final review added regression coverage for mutable POTCAR inputs, dispersion corrections, and submission replay provenance. -The user will consult the host about Bohrium configuration. Real acceptance is -paused pending the licensed PBE directory and authorized VASP image; the existing -calculation authorization remains valid. Issue #3 is not marked complete. +Follow-up [runtime diagnosis](vasp_runtime_diagnosis.md) found and fixed tool +Python/PATH selection bugs and discovered the account-private VASP 6.3.0 image. +Its executable/potential paths cannot be established from stored metadata. +Real acceptance remains paused pending a verifiable PBE potential source and +usable runtime; the existing calculation authorization remains valid. Issue #3 +is not marked complete. The initial noninteractive launcher omitted the NVM bohr path; the installed CLI is `/home/shik-mechrevo-wsl/.nvm/versions/node/v24.14.1/bin/bohr` (2.6.100). Future launches should prepend both the repository `.venv/bin` and this Node -directory to PATH, so `run_python` and bohr use the intended environment. +directory to PATH. Python tools now pin the runtime interpreter and Bash tools +retain that startup environment. ## Standards review @@ -109,7 +113,7 @@ code-correctness or scope issue was identified in the final review. One acceptance requirement remains unmet: one real successful Bohrium silicon task. Actual query, structure generation and environment discovery are evidenced, -but missing PBE potentials and VASP image prevented submission. No successful +but unresolved PBE potentials and VASP runtime details prevent submission. No successful job, archive or completed local loop is claimed. Review totals: Standards 0 findings; Spec 1 outstanding environment-blocked diff --git a/docs/vasp_runtime_diagnosis.md b/docs/vasp_runtime_diagnosis.md new file mode 100644 index 0000000..6587b3d --- /dev/null +++ b/docs/vasp_runtime_diagnosis.md @@ -0,0 +1,134 @@ +# Local VASP runtime diagnosis — 2026-10-07 + +Issue #3 remains unaccepted until a real silicon workflow succeeds. This report +corrects the earlier environment report; it does not establish VASP completion. + +## Reproduction and causal probes + +From the WSL repository, with the project venv and NVM directory on PATH: + +```bash +PYTHONPATH=src .venv/bin/python .workspace/silicon-t2/debug-runtime/repro.py +``` + +Two runs before the fix failed identically: `run_python` used project Python +3.12.13, whereas `run_bash` resolved `~/.local/bin/python` (3.10.12) and raised +`ModuleNotFoundError: No module named 'ase'`. The Bash login profile prepended +`~/.local/bin`, replacing the intended interpreter. Independently, a shadow +`python` on PATH made both Python tools execute the wrong interpreter. Three +regression cases failed before the fix and pass afterward. + +Python tools now use `sys.executable`; Bash uses a non-login shell with inherited +configuration and the runtime interpreter directory prepended to PATH. Shell +profile initialization must happen at harness startup, rather than separately +inside each tool. The original reproduction passes after the fix. + +The provider PATH probe reproduces `The 'bohr' CLI is not installed or not on +PATH` with `/usr/local/bin:/usr/bin:/bin`. Changing only PATH to include +`~/.nvm/versions/node/v24.14.1/bin` makes the same provider call return 2.6.100. +This was a launcher environment error, not a missing installation. Future local +launches need the NVM directory on PATH; the tools now retain that startup PATH. + +## Environment comparison + +| Boundary | bohr | Python chosen by tools | Config and authentication | +| --- | --- | --- | --- | +| Interactive WSL | NVM bohr 2.6.100 | bare `python`: user shim 3.10.12; explicit project interpreter: 3.12.13 | bohr project query succeeds without either credential environment variable | +| MatCreator `run_python` | same path/version | project 3.12.13 | MatCreator config loads; project query succeeds | +| MatCreator `run_bash`, before fix | same path/version with corrected launcher | user shim 3.10.12; ASE missing | same HOME/config; project query succeeds | +| MatCreator `run_bash`, after fix | same path/version | project 3.12.13; ASE import succeeds | inherited configuration retained | +| Actual `run_bohr_json` provider | same path/version | parent project runtime | version and project query succeed with provider flags | + +MatCreator reads `/home/shik-mechrevo-wsl/.matcreator/config.yaml`. A file-open +trace shows bohr reads `/home/shik-mechrevo-wsl/.bohr-cli/config.yaml`. These are +different configurations. The CLI reports the default configuration while its +named profile list is empty; `profile show default` fails, which does not negate +the successful project authentication. In an isolated temporary HOME, both +`BOHRIUM_ACCESS_KEY` alone and adding `ACCESS_KEY` fail with `AUTH_REQUIRED` in +this installed version. Existing CLI config is the verified working credential +source. Credential values were neither displayed nor copied into artifacts. + +## PBE potentials + +Minimal independent check in the real project interpreter: + +```python +from pymatgen.core import Lattice, Structure +from pymatgen.io.vasp.sets import MPRelaxSet +MPRelaxSet(Structure.from_spacegroup( + "Fd-3m", Lattice.cubic(5.43), ["Si"], [[0, 0, 0]] +)).potcar +``` + +It raises `PmgVaspPspDirError: Set PMG_VASP_PSP_DIR= in +.pmgrc.yaml (needed to find POTCARs)`. INCAR generation succeeds. Interactive +and tool environments, MatCreator's `env` configuration, and pymatgen's loaded +settings contain no PSP directory. Neither default pymatgen YAML file exists; +there is no `PMG_CONFIG_FILE` override. Thus this is a genuinely absent +configuration, not a configured value silently ignored by the tool. + +The bounded home search (depth four, excluding caches, virtual environments, +Node dependencies and repositories' `.git`) found only +`~/.matcreator/workspace/vasp/relax_001/POTCAR.note`, an instruction file, not a +potential. `/opt` contains containerd/docker-desktop; `/data` is absent and +`/srv` empty. This establishes no usable source in the inspected scope, not +global absence on all host disks or inside remote images. No dummy POTCAR was +substituted. + +## VASP image source + +MatCreator's `compute.vasp_image` is empty and `BOHRIUM_VASP_IMAGE` unset in all +compared environments. The current account-private catalog nevertheless has +image **24280**, `vasp:6.3.0`, at +`registry.dp.tech/dptech/prod-28882/vasp:6.3.0`; `bohr image get 24280` succeeds. +The earlier diagnostic did not query this catalog and overstated the blocker. +The resource exists; its selection is not configured. No tag was invented. + +`bohr image search vasp` returns gvasp and VASPKIT, which do not prove presence +of the intended executable. `bohr image list -t VASP` is an invalid category in +2.6.100 and even exits zero with non-JSON diagnostic text. The private catalog +query is the correct discovery command. MatCreator independently queried the +image metadata and `bohr image dockerfile 24280`; no Dockerfile is stored +(`RESOURCE_NOT_FOUND`). Neither executable/MPI paths nor bundled potentials +can be established from that metadata. No extra paid probe job was created. + +The image-detail response reports zero CNY balance and `sufficient:false`. +This alone does not establish whether the selected project's photon payment +can fund the workflow; payment is not yet tested by a real submission. + +## Retained execution and remaining information + +The existing MatCreator session **silicon-t2-20261007-env2** was resumed with the +fixed tools. MatCreator owns the calculation plan and independently reproduced +the potential error, inspected the existing image, and recorded its decision +in `.workspace/silicon-t2/matcreator-runtime-diagnosis.json`. It stopped before +input preparation/submission. Development A remains active: query failures +must pause; the retained earlier real query succeeded with zero candidates. + +Retained evidence under `.workspace/silicon-t2/`: + +- `events-diagnosis.jsonl`, existing `events-env2.jsonl` and `trajectories/`; +- the existing `.aidb/runs/material-Si/1791354576-ebf79559/` report and audit; +- `debug-runtime/` clearly marked diagnostic harnesses, redacted environment + comparisons, PATH/config-source probes, and test log; +- the MatCreator report's historical legacy job ID **22947186**, group + **16319214**. This is not a new tracked T2 job or evidence of success. + +No new job was submitted; there is no new durable job ID or batchjob ID to +report. Any eventual submission must save both IDs and query existing status +before retrying. Issue #3 has not been closed or marked successful. + +The user-facing missing-information request is consolidated to a real PBE Si +potential source (local directory, documented in-image path, or accessible host +entry point), plus special VASP/MPI startup instructions if this image needs +them. Existing project and calculation authorization remains valid; no repeat +approval is required. Program availability and payment must be verified by the +eventual authorized workflow rather than assumed from the catalog entry. + +Validation after the fix: **231 passed** across workspace tools, silicon +workflow, remote-job tools and Bohrium Batch Job adapter tests. The original +Python environment reproduction passes. The independent POTCAR reproduction +still fails due to the unresolved resource configuration. + +Follow-up Standards and Spec reviews found zero new issues. They preserve the +same acceptance limitation: the real silicon calculation has not succeeded. diff --git a/src/matcreator/tools/workspace_tools.py b/src/matcreator/tools/workspace_tools.py index 67a6c94..7bda36f 100644 --- a/src/matcreator/tools/workspace_tools.py +++ b/src/matcreator/tools/workspace_tools.py @@ -18,6 +18,7 @@ import asyncio import os +import sys import textwrap from pathlib import Path @@ -330,7 +331,7 @@ async def run_python(code: str, tool_context: ToolContext) -> str: if session_id: cwd = str(get_session_workdir(session_id)) proc = await asyncio.create_subprocess_exec( - "python", "-c", code, + sys.executable, "-c", code, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, cwd=cwd, @@ -368,11 +369,15 @@ async def run_bash(script: str, tool_context: ToolContext) -> str: session_id = tool_context.state.get("session_id") if session_id: cwd = str(get_session_workdir(session_id)) + # Keep the harness configuration; login profiles can replace its venv PATH. + env = os.environ.copy() + env["PATH"] = str(Path(sys.executable).parent) + os.pathsep + env.get("PATH", "") proc = await asyncio.create_subprocess_exec( - "bash", "-lc", script, + "bash", "-c", script, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, cwd=cwd, + env=env, ) try: timeout = _execution_timeout_seconds() @@ -407,7 +412,7 @@ async def run_python_file(relative_path: str) -> str: if not target.exists(): return f"File not found: {target}" proc = await asyncio.create_subprocess_exec( - "python", str(target), + sys.executable, str(target), stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, ) diff --git a/tests/test_workspace_tools.py b/tests/test_workspace_tools.py index 0e35e7a..f7c76d7 100644 --- a/tests/test_workspace_tools.py +++ b/tests/test_workspace_tools.py @@ -1,8 +1,14 @@ from __future__ import annotations from concurrent.futures import ThreadPoolExecutor +import asyncio +import os +import shutil +import sys from pathlib import Path +import pytest + from matcreator.tools import workspace_tools @@ -11,6 +17,46 @@ def __init__(self): self.state = {} +@pytest.mark.parametrize("file_tool", [False, True]) +def test_python_tools_use_runtime_interpreter_not_path(tmp_path, monkeypatch, file_tool): + monkeypatch.setenv("MATCLAW_WORKSPACE", str(tmp_path)) + # A stale shell Python must not replace the interpreter running MatCreator. + shadow = tmp_path / "python" + shadow.write_text("#!/bin/sh\nprintf 'wrong-python'\n", encoding="utf-8") + shadow.chmod(0o755) + monkeypatch.setenv("PATH", str(tmp_path) + os.pathsep + os.environ["PATH"]) + code = "import sys; print(sys.executable)" + if file_tool: + (tmp_path / "probe.py").write_text(code, encoding="utf-8") + output = asyncio.run(workspace_tools.run_python_file("probe.py")) + else: + context = _FakeToolContext() + context.state["workspace_dir"] = str(tmp_path) + output = asyncio.run(workspace_tools.run_python(code, context)) + assert output.strip() == sys.executable + + +@pytest.mark.skipif(shutil.which("bash") is None, reason="bash required") +def test_bash_tool_preserves_runtime_environment_and_python(tmp_path, monkeypatch): + shadow = tmp_path / "python" + shadow.write_text("#!/bin/sh\nprintf 'wrong-python'\n", encoding="utf-8") + shadow.chmod(0o755) + parent_path = str(tmp_path) + os.pathsep + os.environ["PATH"] + monkeypatch.setenv("PATH", parent_path) + monkeypatch.setenv("PMG_VASP_PSP_DIR", str(tmp_path / "configured-potentials")) + context = _FakeToolContext() + context.state["workspace_dir"] = str(tmp_path) + output = asyncio.run(workspace_tools.run_bash( + "shopt -q login_shell && echo unexpected-login; " + "printf '%s\\n' \"$PATH\" \"$PMG_VASP_PSP_DIR\" \"$PWD\"; " + "python -c 'import sys; print(sys.executable)'", context + )) + assert output.splitlines() == [ + str(Path(sys.executable).parent) + os.pathsep + parent_path, + str(tmp_path / "configured-potentials"), str(tmp_path), sys.executable, + ] + + def test_execution_timeout_uses_valid_user_setting(monkeypatch): monkeypatch.delenv("MATCREATOR_EXEC_TIMEOUT_SECONDS", raising=False) assert workspace_tools._execution_timeout_seconds() == 3600 From b01acaa0c80840fbc2064014999077c897aa86ad Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 17:40:22 +0800 Subject: [PATCH 08/22] fix: preserve authorization and default PBE evidence in silicon workflow --- docs/vasp_runtime_diagnosis.md | 39 ++++++++++++ src/matcreator/agents/aidb_skill_hook.py | 34 +++++++--- src/matcreator/tools/silicon_vasp.py | 17 ++++- tests/test_aidb_skill_hook.py | 79 ++++++++++++++++++++++++ tests/test_silicon_vasp_workflow.py | 50 +++++++++++++-- 5 files changed, 205 insertions(+), 14 deletions(-) diff --git a/docs/vasp_runtime_diagnosis.md b/docs/vasp_runtime_diagnosis.md index 6587b3d..8f1c4da 100644 --- a/docs/vasp_runtime_diagnosis.md +++ b/docs/vasp_runtime_diagnosis.md @@ -132,3 +132,42 @@ still fails due to the unresolved resource configuration. Follow-up Standards and Spec reviews found zero new issues. They preserve the same acceptance limitation: the real silicon calculation has not succeeded. + +## Follow-up after the user supplied the PBE source + +The preceding blocked state is historical. The supplied library contains a +valid `PAW_PBE Si 05Jan2001` potential at +`~/projects/9f9b5b-main/psudopotential/paw_pbe/Si/POTCAR.Z`. It uses UNIX +compress; `gzip -cd` decodes it. The raw layout and compression differ from +current pymatgen's lookup layout. MatCreator staged only the required Si +potential under `pseudopotentials/POT_GGA_PAW_PBE/Si/POTCAR`, with a per-process +`PMG_VASP_PSP_DIR`. This was a missing configuration and layout mismatch, +not a missing potential resource. See [source research](pbe_bohrium_research.md). + +The account-private VASP image was selected explicitly in the resumed process. +The authorized real relaxation proved its VASP/MPI startup and payment path: +durable job `56072cf15fe94e639d7b71f563ed320a`, provider batchjob +`5ed5945230884cf88f30475c1afed598`, terminal success, exit code zero. +Scientific verification confirms electronic and ionic convergence, three ionic +steps and total energy -10.84124445 eV for the Si2 cell. The session remains +`silicon-t2-20261007-env2`; no relaxation resubmission was required. + +Three additional platform bugs were reproduced and repaired: + +- Broad `only`/`仅` authorization matching treated unrelated qualifiers as + input-only restrictions. Matching now ties restrictions to preparation/input + requests while retaining genuine input-only blocks at both agent entries. +- VASP 6.3.0 emitted XML `GGA=--` when INCAR omitted GGA. Official VASP rules + select the functional from POTCAR LEXCH in this case. The validator now accepts + this only with submission-hash-bound PBE potentials, matching XML identities + and `LEXCH=PE`; explicit non-PBE GGA or XC overrides remain rejected. Original + XML parameters are retained. [VASP GGA](https://vasp.at/wiki/index.php/GGA). +- Preflight feedback's nested `structure_path` caused the session artifact + collector to misclassify an existing input as a current step output. Exported + feedback now uses `input_structure_path`; internal task evidence and all + authorization/query checks remain intact. Output roots were not broadened. + +Each fix has red-to-green regression evidence in `debug-pbe/`. The real +relaxation validator reproduction also passed without changing scientific +inputs or outputs. Final related tests: **286 passed**; both review axes found +zero new issues. MatCreator resumed the static step after the final fix. diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index 520160e..94119a3 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -94,7 +94,17 @@ def _known_context(text, workspace): def _prepare_only(text): lower = text.lower() - return bool(re.search(r'不提交|不要提交|不要计算|先别计算|暂不.*(?:提交|计算)|do not submit|only.*input|(?:只|仅).*(?:输入|准备|生成)', lower)) or ( + # Restrict the requested task, not unrelated qualifiers such as "only after" + # or "record hashes only". Those occur in authorized executor actions too. + limited_preparation = re.search( + r'(?:只|仅)(?:要|需|需要|要求|允许|授权)?(?:帮我|为我)?(?:准备|生成|制作|创建|提供|输入)' + r'|\bonly\s+(?:(?:please|to)\s+)?(?:prepare|generate|create|write|provide|inputs?\b)' + r'|\b(?:prepare|generate|create|write|provide)\s+only\s+(?:[\w-]+\s+)*inputs?\b' + r'|\binputs?(?:\s+(?:preparation|generation))?\s+only\b', lower) + input_only_request = re.search( + r'(?:^|[.!?;\n])\s*(?:please\s+)?only\s+(?:the|an?)\s+' + r'(?:[\w-]+\s+)*inputs?\b', lower) + return bool(re.search(r'不提交|不要提交|不要计算|先别计算|暂不[^。；\n]*(?:提交|计算)|do not submit', lower) or limited_preparation or input_only_request) or ( bool(re.search(r'准备|prepare', lower)) and not re.search(r'提交|submit|执行计算|run calculation', lower)) @@ -102,7 +112,7 @@ def _task_context(text, workspace, previous=None): current, _, prior = text.partition('\n[AIDB_PRIOR_CONTEXT]\n') lower = current.lower() continuation = bool(_CONTINUATION_RE.search(current)) - calculation = continuation or bool(re.search(r'弛豫|优化|总能|计算|准备|生成|执行|提交|relax|total energy|calculat|prepare|generate|submit|\brun\b', lower)) + calculation = continuation or _prepare_only(current) or bool(re.search(r'弛豫|优化|总能|计算|准备|生成|执行|提交|relax|total energy|calculat|prepare|generate|submit|\brun\b', lower)) conceptual = bool(re.search(r'解释|介绍|什么是|概念|explain|what is|概念说明', lower)) if not calculation or (conceptual and not re.search(r'请.*(?:准备|生成|执行)|please.*(?:prepare|run|calculate)', lower)): return None @@ -199,6 +209,16 @@ async def _query(task, task_id): diagnostic_path.write_text(json.dumps(diagnostic, ensure_ascii=False), encoding='utf-8') +def _export_feedback(feedback): + """Keep input evidence visible without declaring it a generated artifact.""" + task = feedback.get('task_context') + if not task or 'structure_path' not in task.get('known', {}): + return feedback + known = dict(task['known']) + known['input_structure_path'] = known.pop('structure_path') + return {**feedback, 'task_context': {**task, 'known': known}} + + async def before_aidb_skill_load(tool, args, tool_context): name = getattr(tool, 'name', '') skill_name = str(args.get('skill_name') or '').strip().lower() @@ -252,7 +272,7 @@ async def before_aidb_skill_load(tool, args, tool_context): feedback['message'] = '本地查库失败，开发期 A 暂停新增计算；这不是未命中。请检查诊断产物。' tool_context.state['aidb_preflight'] = {'key': key, 'invocation_id': tool_context.invocation_id, 'feedback': feedback} logger.info('[aidb preflight] %s', feedback) - append_session_log_entry(tool_context, {'kind': 'aidb_hook_feedback', **feedback}) + append_session_log_entry(tool_context, {'kind': 'aidb_hook_feedback', **_export_feedback(feedback)}) except Exception as exc: logger.warning('aidb preflight failed', exc_info=True) feedback = {'status': 'failed', 'query_executed': False, 'detail': str(exc), @@ -269,15 +289,15 @@ async def before_aidb_skill_load(tool, args, tool_context): append_session_log_entry(tool_context, {'kind': 'aidb_hook_feedback', **feedback}) except Exception: logger.warning('Could not persist aidb failure feedback', exc_info=True) - tool_context.state[feedback_key] = feedback + tool_context.state[feedback_key] = _export_feedback(feedback) if name == 'load_skill': return None if feedback['status'] not in {'found', 'not_found'}: - return {'status': 'blocked', 'aidb_preflight': feedback} + return {'status': 'blocked', 'aidb_preflight': _export_feedback(feedback)} if 'structure_model' in task['unknown'] and not task['known'].get('structure_path'): - return {'status': 'blocked', 'message': '已查候选；计算准备前请明确结构模型，检索默认值不是计算授权。', 'aidb_preflight': feedback} + return {'status': 'blocked', 'message': '已查候选；计算准备前请明确结构模型，检索默认值不是计算授权。', 'aidb_preflight': _export_feedback(feedback)} if name in _REMOTE_TOOLS and task['prepare_only']: - return {'status': 'blocked', 'message': '当前请求仅授权准备，未授权提交计算。', 'aidb_preflight': feedback} + return {'status': 'blocked', 'message': '当前请求仅授权准备，未授权提交计算。', 'aidb_preflight': _export_feedback(feedback)} return None diff --git a/src/matcreator/tools/silicon_vasp.py b/src/matcreator/tools/silicon_vasp.py index 3b645b3..4dff1cc 100644 --- a/src/matcreator/tools/silicon_vasp.py +++ b/src/matcreator/tools/silicon_vasp.py @@ -97,11 +97,21 @@ def _validate(directory: Path, calculation_type: str, input_directory: Path, inp if (run.parameters.get('LDAU', False) or run.parameters.get('LHFCALC', False) or run.incar.get('ML_LMLFF', False) or run.parameters.get('METAGGA', '') not in {'', 'None', 'NONE'} or run.parameters.get('IVDW', 0) or run.parameters.get('LUSE_VDW', False) - or incar.get('IVDW', 0) or incar.get('LUSE_VDW', False)): + or incar.get('IVDW', 0) or incar.get('LUSE_VDW', False) + or incar.get('XC') or run.incar.get('XC') or run.parameters.get('XC')): raise ValueError('Only ordinary PBE is supported; corrections/hybrid/meta-GGA detected.') - if str(run.parameters.get('GGA', 'PE')).upper() != 'PE' or not run.potcar_symbols or any(not s.startswith('PAW_PBE Si ') for s in run.potcar_symbols): + potcar_text = (input_directory / 'POTCAR').read_text(errors='replace') + gga = str(run.parameters.get('GGA', 'PE')).strip().upper() + lexch = re.findall(r'\bLEXCH\s*=\s*(\S+)', potcar_text) + # VASP emits '--' when GGA is not selected explicitly: the functional then + # comes from the immutable submitted POTCAR's LEXCH, not the placeholder. + default_pbe = (gga == '--' and len(lexch) == len(run.potcar_symbols) + and bool(lexch) and all(value.upper() == 'PE' for value in lexch)) + if (gga != 'PE' and not default_pbe + or any(str(values.get('GGA', 'PE')).strip().upper() != 'PE' for values in (incar, run.incar)) + or not run.potcar_symbols or any(not s.startswith('PAW_PBE Si ') for s in run.potcar_symbols)): raise ValueError('Ordinary PBE Si pseudopotential evidence missing or incompatible.') - titles = re.findall(r'^\s*TITEL\s*=\s*(.+?)\s*$', (input_directory / 'POTCAR').read_text(errors='replace'), re.M) + titles = re.findall(r'^\s*TITEL\s*=\s*(.+?)\s*$', potcar_text, re.M) if [' '.join(t.split()) for t in titles] != [' '.join(t.split()) for t in run.potcar_symbols]: raise ValueError('Submitted POTCAR identity disagrees with actual VASP output.') for structure in (run.initial_structure, run.final_structure): @@ -120,6 +130,7 @@ def _validate(directory: Path, calculation_type: str, input_directory: Path, inp return {'structure': run.final_structure.as_dict(), 'initial_structure': run.initial_structure.as_dict(), 'total_energy': {'value': energy, 'unit': 'eV'}, 'conditions': {'functional': 'PBE', 'domain': 'bulk', 'structure_model': 'diamond', + 'functional_selection': 'POTCAR LEXCH' if default_pbe else 'GGA', 'software': 'VASP', 'version': run.vasp_version, 'incar': dict(incar), 'parameters': run.parameters, 'potcar_symbols': run.potcar_symbols, 'potcar_sha256': input_hashes['POTCAR'], 'input_sha256': input_hashes, diff --git a/tests/test_aidb_skill_hook.py b/tests/test_aidb_skill_hook.py index 8df6775..cd70bb2 100644 --- a/tests/test_aidb_skill_hook.py +++ b/tests/test_aidb_skill_hook.py @@ -296,6 +296,25 @@ def test_given_structure_supplies_identity_without_manual_json(local_aidb, tmp_p assert prepared == ['prepare'] +def test_preflight_input_structure_is_not_a_step_output(local_aidb, tmp_path, monkeypatch): + from ase.build import bulk + from ase.io import write + from matcreator.agents.session_log import collect_artifact_paths + write(tmp_path / 'POSCAR', bulk('Si', 'diamond', a=5.43)) + entries = [] + original = hook.append_session_log_entry + def record(context, entry): + entries.append(entry) + return original(context, entry) + monkeypatch.setattr(hook, 'append_session_log_entry', record) + responses, prepared = run_task(tmp_path, '用现有POSCAR准备VASP弛豫和总能输入', + [('run_python', {'code': 'prepare'})], entry='step') + assert prepared == ['prepare'] + assert str(tmp_path / 'POSCAR') not in collect_artifact_paths(responses[0]) + assert all(str(tmp_path / 'POSCAR') not in collect_artifact_paths(entry) for entry in entries) + assert responses[0]['aidb_preflight']['task_context']['known']['input_structure_path'] == str(tmp_path / 'POSCAR') + + def test_structure_read_failure_keeps_diagnostic_and_blocks_tools(local_aidb, tmp_path): (tmp_path / 'broken.cif').write_text('not a CIF') responses, prepared = run_task(tmp_path, '用broken.cif准备VASP弛豫输入', @@ -321,3 +340,63 @@ def test_explicit_new_submission_authorization_overrides_prior_prepare_only(loca assert responses[1]['status'] == 'submitted' assert responses[1]['aidb_preflight']['status'] == 'not_found' assert prepared == ['remote submission'] + + +@pytest.mark.parametrize('action', [ + 'ONLY after relaxation succeeds: prepare static VASP inputs and submit the job.', + 'Record POTCAR path and hash only, no body. Submit VASP with input_path=vasp-Si.', + '环境设置仅本次运行有效。准备并提交金刚石硅 PBE VASP 输入。', +]) +def test_authorized_step_only_qualifiers_do_not_forbid_submission(local_aidb, tmp_path, action): + step = json.dumps({'action': action, 'prior_context': 'diamond bulk Si PBE relaxation total energy'}) + responses, prepared = run_task(tmp_path, step, + [('submit_bohr_batchjob', {'name': 'Si'})], entry='step', + state={'goal': '继续准备并提交金刚石硅体相 PBE 弛豫和总能重算', + 'aidb_user_request': '继续准备并提交金刚石硅体相 PBE 弛豫和总能重算', + 'aidb_user_prepare_only': False}) + assert responses[0]['status'] == 'submitted' + assert prepared == ['remote submission'] + + +@pytest.mark.parametrize('action', [ + 'Only prepare diamond bulk Si PBE VASP inputs.', + 'Generate only diamond bulk Si PBE VASP inputs.', + '只生成金刚石硅体相 PBE VASP 输入。', +]) +def test_actual_prepare_only_step_still_blocks_submission(local_aidb, tmp_path, action): + step = json.dumps({'action': action, 'prior_context': 'diamond bulk Si PBE'}) + responses, prepared = run_task(tmp_path, step, + [('submit_bohr_batchjob', {'name': 'Si'})], entry='step', + state={'goal': '计算金刚石硅体相 PBE 弛豫和总能'}) + assert responses[0]['status'] == 'blocked' + assert prepared == [] + + +@pytest.mark.parametrize('user_text', [ + '我只要生成金刚石硅体相普通 PBE VASP 输入文件。', + '只需生成金刚石硅体相 PBE VASP 输入文件。', + 'Only the diamond bulk Si PBE VASP input files, please.', + 'Only a VASP input set for diamond bulk Si PBE, please.', +]) +def test_user_input_only_restriction_survives_executor_submit_action(local_aidb, tmp_path, user_text): + step = json.dumps({'action': 'Submit the VASP Batch Job for diamond bulk Si PBE relaxation.', + 'prior_context': user_text}) + responses, prepared = run_task(tmp_path, step, + [('submit_bohr_batchjob', {'name': 'Si'})], entry='step', + state={'goal': user_text, 'aidb_user_request': user_text, + 'aidb_user_prepare_only': False}) + assert responses[0]['status'] == 'blocked' + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert prepared == [] + + +@pytest.mark.parametrize('user_text', [ + 'Only the diamond bulk Si PBE VASP input files, please.', + 'Only a VASP input set for diamond bulk Si PBE, please.', +]) +def test_input_only_noun_request_blocks_submission_at_main_entry(local_aidb, tmp_path, user_text): + responses, prepared = run_task(tmp_path, user_text, + [('submit_bohr_batchjob', {'name': 'Si'})]) + assert responses[0]['status'] == 'blocked' + assert responses[0]['aidb_preflight']['status'] == 'not_found' + assert prepared == [] diff --git a/tests/test_silicon_vasp_workflow.py b/tests/test_silicon_vasp_workflow.py index 83e7f31..3108e58 100644 --- a/tests/test_silicon_vasp_workflow.py +++ b/tests/test_silicon_vasp_workflow.py @@ -33,7 +33,7 @@ def collect_outputs(self, external_id, destination_dir): return [{'destination': str(p)} for p in destination_dir.iterdir()] or [{'destination': str(destination_dir)}] -def job_context(tmp_path, monkeypatch, outputs=None): +def job_context(tmp_path, monkeypatch, outputs=None, lexch='PE'): service = RemoteJobService(RemoteJobStore(tmp_path / 'jobs.db'), adapter_overrides={'bohr_batchjob': Provider()}) monkeypatch.setattr(remote_job_tools, '_service', lambda: service) @@ -45,7 +45,8 @@ def job_context(tmp_path, monkeypatch, outputs=None): source.mkdir() for name in ('INCAR', 'POSCAR', 'KPOINTS'): (source / name).write_text(outputs[name]) - (source / 'POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\nsynthetic test potential, not for computation\n') + header = f'LEXCH = {lexch}\n' if lexch is not None else '' + (source / 'POTCAR').write_text('TITEL = PAW_PBE Si 05Jan2001\n' + header + 'synthetic test potential, not for computation\n') spec['vasp_input_sha256'] = {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in source.iterdir()} job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', idempotency_key='silicon-relax', spec=spec) @@ -95,13 +96,54 @@ def silicon_outputs(nelm=60, electronic_steps=2, ionic_marker=True, stage='relax 'OSZICAR': ' 1 F= -.10100000E+02 E0= -.10100000E+02\n'} -def completed_job(tmp_path, monkeypatch, outputs): - service, context, job_id = job_context(tmp_path, monkeypatch, outputs) +def completed_job(tmp_path, monkeypatch, outputs, lexch='PE'): + service, context, job_id = job_context(tmp_path, monkeypatch, outputs, lexch) service.adapter_for('bohr_batchjob').outputs = outputs service.store.transition_job(job_id, 'succeeded') return service, context, job_id +@pytest.mark.parametrize('gga,explicit,lexch,expected', [ + ('--', None, 'PE', 'success'), + ('--', 'PE', 'PE', 'success'), + ('--', 'PS', 'PE', 'invalid'), + ('--', None, 'CA', 'invalid'), + ('--', None, None, 'invalid'), + ('PS', None, 'PE', 'invalid'), + ('XX', None, 'PE', 'invalid'), +]) +def test_vasp_default_gga_requires_verified_pbe_potential_and_no_override( + tmp_path, monkeypatch, gga, explicit, lexch, expected +): + outputs = silicon_outputs() + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace('', + f'{gga}') + if explicit: + outputs['INCAR'] += f'GGA = {explicit}\n' + _, context, job_id = completed_job(tmp_path, monkeypatch, outputs, lexch) + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == expected + if expected == 'success': + report = json.loads(Path(result['report_path']).read_text()) + assert report['conditions']['parameters']['GGA'] == '--' + assert report['conditions']['functional_selection'] == 'POTCAR LEXCH' + + +@pytest.mark.parametrize('location', ['INCAR', 'xml_incar', 'xml_parameters']) +def test_exchange_correlation_override_is_not_inferred_from_pbe_potcar(tmp_path, monkeypatch, location): + outputs = silicon_outputs() + if location == 'INCAR': + outputs['INCAR'] += 'XC = PBE0\n' + else: + closing = '' if location == 'xml_incar' else '' + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace(closing, + 'PBE0' + closing) + _, context, job_id = completed_job(tmp_path, monkeypatch, outputs) + result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' + assert 'ordinary PBE' in result['message'] + + def test_verified_relaxation_has_energy_structure_provenance_and_replays(tmp_path, monkeypatch): service, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) result = collect_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) From 67777e657f8164b4c9cd1fda3f3ea7d4339c4b9a Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 17:45:48 +0800 Subject: [PATCH 09/22] docs: explain verified PBE potential setup for Bohrium VASP --- docs/learning/vasp/MISSION.md | 17 +++++ docs/learning/vasp/NOTES.md | 3 + docs/learning/vasp/RESOURCES.md | 30 ++++++++ docs/learning/vasp/assets/course.css | 2 + docs/learning/vasp/assets/quiz.js | 9 +++ .../vasp/lessons/0001-pbe-potcar-bohrium.html | 13 ++++ docs/learning/vasp/reference/vasp-inputs.html | 7 ++ docs/pbe_bohrium_research.md | 73 +++++++++++++++++++ 8 files changed, 154 insertions(+) create mode 100644 docs/learning/vasp/MISSION.md create mode 100644 docs/learning/vasp/NOTES.md create mode 100644 docs/learning/vasp/RESOURCES.md create mode 100644 docs/learning/vasp/assets/course.css create mode 100644 docs/learning/vasp/assets/quiz.js create mode 100644 docs/learning/vasp/lessons/0001-pbe-potcar-bohrium.html create mode 100644 docs/learning/vasp/reference/vasp-inputs.html create mode 100644 docs/pbe_bohrium_research.md diff --git a/docs/learning/vasp/MISSION.md b/docs/learning/vasp/MISSION.md new file mode 100644 index 0000000..ed8bc69 --- /dev/null +++ b/docs/learning/vasp/MISSION.md @@ -0,0 +1,17 @@ +# Mission: 理解 PBE 赝势如何进入 Bohrium VASP 作业 + +## Why +用户正在用 MatCreator 完成真实硅计算，已下载 PBE 文件，但尚不清楚它们与 VASP 的联动。学习目标是能判断输入是否齐全、文件应该放在哪里，以及计算实际使用了哪套势。 + +## Success looks like +- 区分 PBE 泛函、PAW 数据集、VASP 程序和 Bohrium 提交工具。 +- 指出本机赝势库、单次作业输入目录、远端运行目录的区别。 +- 能从 Si POTCAR 的 TITEL、ZVAL、ENMAX 读出基本含义。 + +## Constraints +- 从本次 Si 文件入手，用短课和即时反馈练习解释。 +- 实际计算继续由 MatCreator 主导，保留轨迹和作业身份。 +- 本目录是独立教学工作区，避免占用软件仓库根目录的教学状态文件。 + +## Out of scope +- 首课不推导 PAW 方程，不设计新的收敛研究。 diff --git a/docs/learning/vasp/NOTES.md b/docs/learning/vasp/NOTES.md new file mode 100644 index 0000000..23d982e --- /dev/null +++ b/docs/learning/vasp/NOTES.md @@ -0,0 +1,3 @@ +# 教学笔记 + +用户明确表示不清楚下载的 PBE 如何与 VASP 联动，希望解释文件位置、参与方式及作用。先建立文件流和角色，再介绍最少的物理概念。尚未收到练习答案，不记录为已经掌握。 diff --git a/docs/learning/vasp/RESOURCES.md b/docs/learning/vasp/RESOURCES.md new file mode 100644 index 0000000..516f601 --- /dev/null +++ b/docs/learning/vasp/RESOURCES.md @@ -0,0 +1,30 @@ +# VASP / PBE 资料 + +## Knowledge + +- [VASP Wiki：POTCAR](https://vasp.at/wiki/index.php/POTCAR) + 文件内容、元素顺序、TITEL/ZVAL/ENMAX；解释实际 Si 文件的首选。 +- [VASP Wiki：GGA](https://vasp.at/wiki/index.php/GGA) + 计算泛函的选择及 POTCAR 默认值；用于区分 PBE 方法和 PAW 数据。 +- [VASP Wiki：PAW](https://vasp.at/wiki/index.php/Projector-augmented-wave_formalism) + 平滑波函数与原子附近的重构；进阶解释赝势为何提高效率。 +- [VASP Wiki：Preparing a POTCAR](https://vasp.at/wiki/index.php/Preparing_a_POTCAR) + 从库挑选并组合 POTCAR 的官方操作说明。 +- [VASP Wiki：ENCUT](https://vasp.at/wiki/index.php/ENCUT) + 截断能与收敛；ENMAX 不等于所有体系都已验证的精度。 +- [pymatgen：安装与 POTCAR 配置](https://pymatgen.org/installation.html) + 本机库供输入生成器查找；实际路径仍以当前安装版本源码为准。 +- [本仓库的 Bohrium adapter](../../../src/matcreator/control_plane/providers/bohr_batchjob.py) + `input_path` 到 CLI `--input` 的真实调用边界。此相对链接从本目录指向项目源代码。 +- [本机文件实测](../../../.workspace/silicon-t2/debug-pbe/check.json) + 用户下载的压缩 Si 文件及 pymatgen 解析结果；不包含 POTCAR 正文。 + +## Wisdom (Communities) + +- [VASP 官方论坛](https://www.vasp.at/forum/) + 用于向维护团队确认特定势版本和程序报错；没有代用户发帖。 + +## Gaps + +- 官方知识和文件校验不替代本次真实 VASP 成功及收敛验证。 +- 镜像目录名不能证明其内部程序路径；以实际作业启动记录为准。 diff --git a/docs/learning/vasp/assets/course.css b/docs/learning/vasp/assets/course.css new file mode 100644 index 0000000..b73f6c9 --- /dev/null +++ b/docs/learning/vasp/assets/course.css @@ -0,0 +1,2 @@ +:root{color-scheme:light;--ink:#17313c;--muted:#536771;--paper:#faf8f2;--line:#d9dfda;--accent:#08786b;--pale:#e8f3ef} +*{box-sizing:border-box}body{margin:0;background:var(--paper);color:var(--ink);font:17px/1.8 system-ui,"Microsoft YaHei",sans-serif}main{max-width:940px;margin:auto;padding:46px 30px 72px}header{border-bottom:2px solid var(--ink);padding-bottom:26px}.eyebrow{font-size:12px;letter-spacing:.16em;color:var(--accent);font-weight:700}h1{font-size:clamp(28px,4vw,42px);line-height:1.35;margin:12px 0}h2{font-size:23px;margin:34px 0 12px}h3{font-size:18px;margin:0 0 8px}p{margin:12px 0}a{color:var(--accent);text-underline-offset:4px}code{font-size:.88em;background:#edf0e8;padding:2px 5px;border-radius:4px;overflow-wrap:anywhere}pre{overflow:auto;padding:16px;background:#edf0e8;font-size:14px;line-height:1.65;border-left:3px solid var(--accent)}pre code{padding:0}.lead{font-size:20px;max-width:770px}.note{border-left:3px solid var(--accent);background:var(--pale);padding:14px 18px}.flow{display:grid;grid-template-columns:1fr auto 1fr auto 1fr;gap:12px;align-items:center;margin:22px 0}.card{border:1px solid var(--line);background:white;padding:18px;min-height:175px;border-radius:8px}.arrow{font-size:28px;color:var(--accent)}.small{font-size:14px;color:var(--muted)}table{width:100%;border-collapse:collapse;font-size:15px}td,th{text-align:left;padding:12px;border-bottom:1px solid var(--line);vertical-align:top}th{background:#edf0e8}button{font:inherit;color:var(--ink);border:1px solid #9aaca5;background:white;border-radius:6px;padding:9px 15px;cursor:pointer}button:hover,button:focus-visible{background:var(--pale);outline:2px solid var(--accent);outline-offset:2px}button[aria-pressed=true]{background:var(--accent);color:white}.quiz{padding:20px;border:1px solid var(--line);margin:16px 0;background:white;border-radius:8px}.choices{display:flex;gap:10px;flex-wrap:wrap}.feedback{min-height:2.2em;color:var(--accent);font-weight:600}footer{border-top:1px solid var(--line);margin-top:32px;padding-top:20px;font-size:14px;color:var(--muted)}@media(max-width:700px){main{padding:25px 18px}.flow{grid-template-columns:1fr}.arrow{text-align:center;transform:rotate(90deg)}.card{min-height:0}table{font-size:13px}td,th{padding:8px}}@media print{body{background:white;font-size:11pt}main{padding:0;max-width:none}h1{font-size:24pt}a{color:inherit}.card,.quiz{break-inside:avoid}.flow{grid-template-columns:1fr auto 1fr auto 1fr}.choices button{font-size:10pt;padding:5px}footer{font-size:9pt}} diff --git a/docs/learning/vasp/assets/quiz.js b/docs/learning/vasp/assets/quiz.js new file mode 100644 index 0000000..dc8f4e2 --- /dev/null +++ b/docs/learning/vasp/assets/quiz.js @@ -0,0 +1,9 @@ +document.querySelectorAll('.quiz').forEach(quiz => { + quiz.querySelectorAll('button').forEach(button => { + button.setAttribute('aria-pressed', 'false'); + button.addEventListener('click', () => { + quiz.querySelectorAll('button').forEach(b => b.setAttribute('aria-pressed', String(b === button))); + quiz.querySelector('.feedback').textContent = button.dataset.correct === 'true' ? quiz.dataset.success : quiz.dataset.retry; + }); + }); +}); diff --git a/docs/learning/vasp/lessons/0001-pbe-potcar-bohrium.html b/docs/learning/vasp/lessons/0001-pbe-potcar-bohrium.html new file mode 100644 index 0000000..f6029f1 --- /dev/null +++ b/docs/learning/vasp/lessons/0001-pbe-potcar-bohrium.html @@ -0,0 +1,13 @@ +PBE 文件怎样进入 Bohrium VASP
+
MATCREATOR · VASP 首课 · 约 5 分钟

下载的 PBE，怎样进入一次 VASP 计算？

对这次纯硅计算，需要从本机库取出 Si 的数据，形成一个未压缩的 POTCAR，随作业输入上传。VASP 在远端作业目录里读取它。

+

先认清四个角色

角色做什么
PBE 泛函一种计算电子交换与关联能的近似方法。它由 VASP 实现，不是下载后运行的程序。
你下载的 PAW_PBE 数据每种元素的一套原子数据，用 PBE 生成；被选入 POTCAR，供 VASP 的 PAW 计算使用。
VASP / vasp_std读取结构、参数和原子数据，迭代计算电子态、能量与力的程序。
bohr把输入交给 Bohrium 的机器和镜像，并管理作业；它本身不执行电子结构求解。

依据：VASP GGA、POTCAR；bohr 的角色来自本仓库 Batch Job adapter 与安装版本命令帮助。

+

三个位置，一次文件传递

① 本机原始库

9f9b5b-main/
psudopotential/
paw_pbe/Si/POTCAR.Z

保留原件。MatCreator 解压并验证所需的 Si 文件。

② 本机作业目录

INCAR
POSCAR
POTCAR
KPOINTS · run.sh

POTCAR 是完整数据文件；不是写着库路径的说明。

③ Bohrium 作业目录

上传的输入文件
＋ 镜像中的 vasp_std
＋ 分配的 CPU

启动脚本在这里运行 VASP，并生成结果。

镜像存程序，作业输入带数据。采用本次上传方案，整个 PBE 库可以留在本机，不必安装进 Bohrium 镜像。远端 VASP 需要的是当前作业的 POTCAR，而不是本机 ~/projects/... 路径。已有镜像也可能自带库，但本次没有这方面的证据。

文件读取要求见 VASP Input；上传及解包行为见项目研究报告中的 bohr 2.6.100 与 adapter 证据。

+

本机 PMG_VASP_PSP_DIR 有什么用？

它告诉 pymatgen 输入生成器去哪里找库。当前版本默认期望如下布局：

pseudopotentials/             ← PMG_VASP_PSP_DIR 指向这里
+└── POT_GGA_PAW_PBE/
+    └── Si/
+        └── POTCAR           ← 解压后的完整文件

你下载的文件夹叫 paw_pbe，里面是 POTCAR.Z。所以资源已经存在，但目录约定、压缩格式和配置仍需接好。MatCreator 会在工作区暂存所需文件；原库保持原样。

准备完成后，pymatgen 把选中的数据写进单次作业目录的 POTCAR。VASP 的常规启动直接读这个文件，不依赖 pymatgen 的配置变量。依据：pymatgen 配置文档与当前安装版本 PotcarSingle.from_symbol_and_functional 的源码。

+

它在物理上起什么作用？

原子核附近的电子波函数变化很快，直接用平面波描述会很昂贵。PAW 用较平滑的辅助波函数计算，并利用原子数据在核附近重构所需信息；VASP 的实现采用冻结芯近似。POTCAR 提供相关原子数据，帮助建立计算中的原子描述。PBE 则决定交换与关联能采用哪种近似。官方 PAW 说明

因此，换 POTCAR 会影响价电子选择、所需平面波基组和能量基准；比较能量或接续静态计算时，要记录并保持势的一致性。它不会把体系能量预先写死：晶体的能量仍需 VASP 根据结构和参数实际求解。

+

读懂你这份 Si 文件

本机实测含义
PAW_PBE Si 05Jan2001Si 的 PBE PAW 数据集及生成日期。日期不等于 VASP 程序版本。
ZVAL = 4每个 Si 原子的价电子数。中性 Si₂ 晶胞有 8 个价电子；不是每个 Si 只有 4 个总电子。
ENMAX = 245.345 eV这份势推荐的平面波截断能。实际 INCAR 的 ENCUT 可以更高，精度仍需按任务验证。

以上数值来自本机文件解压后的 pymatgen 实测，is_valid=True。这证明数据可识别，不证明某次计算已收敛。字段定义见 POTCAR 和 ENCUT。

+

用三个判断把文件流记牢

1. 本次方案中，VASP 运行时从哪里读取 POTCAR？

2. PBE 泛函主要描述哪一项？

3. 如果 POSCAR 的元素顺序是 Si O，POTCAR 应怎么拼？

+
下次复习时，先不看图，试着说出“三个目录各放什么”。
速查：VASP 输入与 PBE 文件卡片。首选延伸阅读：官方 Preparing a POTCAR。
任何不清楚的地方都可以继续问我；练习接触不等于已经掌握，我们可根据你的回答调整下一课。
+
diff --git a/docs/learning/vasp/reference/vasp-inputs.html b/docs/learning/vasp/reference/vasp-inputs.html new file mode 100644 index 0000000..4599f9e --- /dev/null +++ b/docs/learning/vasp/reference/vasp-inputs.html @@ -0,0 +1,7 @@ +VASP 输入与 PBE 速查
可打印参考卡片

一次 VASP 作业，需要哪些输入？

文件 / 设置职责本次要点
POSCAR晶格、元素、数量与坐标金刚石结构 Si
POTCAR按元素顺序组成的 PAW / 赝势数据只需一份 Si 数据块；两个 Si 原子也不是拼两份
INCAR计算方法、精度与优化参数PBE 泛函可通过 GGA=PE 明确选择；不改 POTCAR 的 LEXCH
KPOINTS / KSPACING倒空间采样采用 KSPACING 的输入可不提供 KPOINTS
run.sh初始化程序环境、启动 MPI / VASP属于运行方式，不能替代 POTCAR
PMG_VASP_PSP_DIR让 pymatgen 找到本机势库指库根目录；不是单次 POTCAR 文件，也不会让远端自动访问本机路径

记住这条路径

原始库：paw_pbe/Si/POTCAR.Z
+  ↓ 解压、校验、适配当前 pymatgen 目录
+工作库：pseudopotentials/POT_GGA_PAW_PBE/Si/POTCAR
+  ↓ MPRelaxSet / MPStaticSet 生成输入
+作业：INCAR + POSCAR + POTCAR + KPOINTS（按需）+ run.sh
+  ↓ MatCreator tracked submit → bohr --input
+远端：在作业目录运行镜像中的 VASP

三个字段

TITEL：势身份。ZVAL：每原子价电子数。ENMAX：势建议的截断能，INCAR 的 ENCUT 可覆盖它。

更换势后不能直接混用不同势的总能量基准。静态计算应沿用松弛步骤的势，保留 TITEL 与 SHA256。POTCAR 原始数据不应随意改写。

diff --git a/docs/pbe_bohrium_research.md b/docs/pbe_bohrium_research.md new file mode 100644 index 0000000..cde749a --- /dev/null +++ b/docs/pbe_bohrium_research.md @@ -0,0 +1,73 @@ +# 本机 PBE PAW 数据与 Bohrium VASP 联动 + +核验日期：2026-10-07。资料来自 VASP / pymatgen 官方文档、已安装的 bohr 命令帮助及 MatCreator 代码。本报告解释输入联动；云端 VASP 启动和真实计算是否成功，以会话工具轨迹和作业结果为准。 + +## 当前已确认的本机资源 + +本机检查报告 [check.json](../.workspace/silicon-t2/debug-pbe/check.json) 记录：`/home/shik-mechrevo-wsl/projects/9f9b5b-main/psudopotential/paw_pbe/Si/POTCAR.Z` 是 UNIX compress 数据，`gzip -cd` 可以解压。真实 `PotcarSingle` 解析结果为 Si / PBE，`TITEL = PAW_PBE Si 05Jan2001`，`ZVAL = 4`，`ENMAX = 245.345 eV`，具有完整的数据集结束标记，`is_valid = True`。解压字节的 SHA256 为 `52dbe99da884e0191b2d348dfcc281aaba36e2eacd3019a49d93cf07d84da86b`。这支持文件身份与解析完整性，不证明下载来源授权，也不把它称为最新推荐版本。[本机检查证据](../.workspace/silicon-t2/debug-pbe/check.json) + +此前不是完全没有 Si 赝势：资源放在另一个项目，MatCreator 的 pymatgen 配置没有指向可读取的目录；原始 `paw_pbe/Si/POTCAR.Z` 与当前 pymatgen 所期待的目录、文件命名还不同。把原始目录路径直接赋给 `PMG_VASP_PSP_DIR` 不一定解决问题。官方方法先通过 `pmg config -p <原始库> <转换后的库>` 重排数据，再设置 `PMG_VASP_PSP_DIR` 为转换后根目录。本次可以只准备 Si，具体路径以 MatCreator 留存的准备记录为准。[pymatgen POTCAR setup](https://pymatgen.org/installation.html#potcar-setup) + +## PBE 到底是什么 + +PBE（Perdew–Burke–Ernzerhof）是一种描述电子交换与关联能的近似方法，属于 GGA。VASP 本身实现这种计算方法；`INCAR` 中的 `GGA = PE` 选择 PBE。如果没有其它交换关联方法设置，VASP 会依据 POTCAR 的 `LEXCH` 选择默认方法。[VASP GGA](https://vasp.at/wiki/GGA) + +你下载的“PBE 库”是用 PBE 构建的各元素 PAW 原子数据。它不是 PBE 算法插件，也不是硅晶体的计算结果。PAW 用平滑的辅助波函数减少平面波计算量，再在原子周围恢复所需的全电子信息；VASP 的实现采用冻结芯电子近似。这里的数据提供原子参考态、投影函数等信息，让 VASP 在求解晶体电子状态时正确处理核附近的贡献。[VASP PAW formalism](https://vasp.at/wiki/index.php/Projector-augmented-wave_formalism) + +对于你这个 Si 数据，`TITEL` 识别数据类型、元素和生成日期；`ZVAL = 4` 表示每个 Si 原子显式处理四个价电子；`ENMAX` 是该数据推荐的平面波截断能，实际 `ENCUT` 可以更高，仍须依所需精度判断收敛。不要手改 POTCAR 内部数值或 `LEXCH` 来“切换方法”。[VASP POTCAR](https://vasp.at/wiki/POTCAR) + +## 从本地库到云端计算 + +```text +WSL 中保存的 PBE PAW 库 + │ pymatgen 读取本次结构需要的元素数据 + ▼ +本次作业输入目录：INCAR、POSCAR、POTCAR、KPOINTS + │ MatCreator tracked submit → bohr --input 打包上传 + ▼ +Bohrium 作业工作目录中的同名输入文件 + │ 容器镜像提供 VASP 程序和运行环境 + ▼ +VASP 读取输入 → 计算能量/力 → 输出 OUTCAR、vasprun.xml 等 +``` + +这是根据官方输入定义和本仓库提交实现得出的联动流程：镜像负责程序，作业目录负责本次数据。因此，使用输入上传方式时，无需把整套 PBE 库放入镜像；本次 Si 的 POTCAR 随输入送到作业工作目录即可。WSL 的库路径不会自动变成远端容器路径。[VASP input](https://vasp.at/wiki/index.php/Input)、[MatCreator 提交适配器](../src/matcreator/control_plane/providers/bohr_batchjob.py)、[本仓库 Bohrium 技能](../src/matcreator/skills/bohrium/SKILL.md) + +已实际运行 `bohr batchjob submit --help`：安装版本的 `--input` 说明是本地文件或目录打包并上传；`--image` 指容器镜像，`--command` 指远端执行命令，`--dry-run` 只校验不执行。适配器把输入目录传给 `--input`，先做同参数 dry-run，再提交。库目录与 VASP 可执行文件路径互相独立；镜像内到底如何调用 VASP / MPI，应通过该镜像的启动信息核验，不能仅凭镜像名字推断。[MatCreator 提交适配器](../src/matcreator/control_plane/providers/bohr_batchjob.py)、[VASP Batch Job 参考](../src/matcreator/skills/vasp-pymatgen/references/bohr-batchjob.md) + +## 四个输入如何协作 + +| 文件 | 告诉 VASP 什么 | 硅任务中的含义 | +|---|---|---| +| POSCAR | 晶胞、元素数量、原子位置 | 这次研究哪一个硅结构 | +| INCAR | 计算任务、电子求解与精度参数 | 结构弛豫还是静态能量、PBE、收敛标准等 | +| POTCAR | 每种元素所用 PAW 原子数据 | 本次只有 Si，所以只含一份 Si 数据集 | +| KPOINTS | 周期体系布里渊区的采样 | 电子状态在倒空间如何采样；也可以用 INCAR 的 KSPACING 自动生成 | + +这些文件相互配合才构成完整的标准任务；KSPACING 的自动采样只在没有 KPOINTS 时使用。[VASP input](https://vasp.at/wiki/index.php/Input)、[VASP KSPACING](https://vasp.at/wiki/KSPACING) + +POTCAR 按 POSCAR 中的元素类型顺序排列，按元素类型一次拼接，不按每个原子重复。比如纯 Si 有两个原子也只需一份 Si 数据；含 Si、O 的结构若 POSCAR 元素顺序是 Si O，则 POTCAR 是 Si 数据接 O 数据。选错顺序会使结构与原子数据对应错误。[VASP Preparing a POTCAR](https://vasp.at/wiki/Preparing_a_POTCAR) + +可以把这四份文件理解为“地点与原子布局、计算要求、元素资料、采样方案”；VASP 是实际计算者，bohr 是把工作送到 Bohrium 并管理作业的入口，MatCreator 负责准备、提交、检查与留存。 + +## 复现与留存 + +结果应留存实际 POTCAR 的 TITEL 和字节哈希、INCAR、POSCAR、采样、镜像与启动命令，以及 MatCreator 的会话、工具轨迹和 durable job ID / provider batchjob ID。更换 POTCAR 版本可能改变结果，比较能量时要确认输入方法一致；当前输入校验工具会绑定提交时的哈希。[VASP POTCAR](https://vasp.at/wiki/POTCAR)、[本仓库结果验证工具](../src/matcreator/tools/silicon_vasp.py) + +本报告与提交记录只记录身份和核验信息，不把整套库放入代码仓库。pymatgen 官方也明确不能随软件分发 VASP 赝势。文件可解析与用户的授权来源是两项不同事实。[pymatgen POTCAR setup](https://pymatgen.org/installation.html#potcar-setup) + +后续仍由 MatCreator 在已有计算授权下执行硅工作流；输入准备成功、CLI 提交成功、科学计算收敛是三个不同的检查点。这里的库核验并不等于 issue #3 已通过真实计算验收。 + +## 实际输入准备结果 + +MatCreator 已在原会话 `silicon-t2-20261007-env2` 解压并核验所需 Si 文件，暂存于 +`.workspace/silicon-t2/pseudopotentials/POT_GGA_PAW_PBE/Si/POTCAR`，随后用 +`MPRelaxSet` 生成 `vasp-relax-Si/` 的 INCAR、POSCAR、POTCAR、KPOINTS 和 run.sh。 +作业 POTCAR 与原文件解压结果的 SHA256 相同。实际 INCAR 的 ENCUT 为 520 eV， +与 POTCAR 的 ENMAX 245.345 eV 是两个不同数值：前者是本次输入选择，后者是势的建议值。 +这项输入准备已解决此前 `PmgVaspPspDirError`，但不能据此宣称云端启动或收敛成功。 +证据：[MatCreator 输入准备报告](../.workspace/silicon-t2/pbe-input-preparation-report.json)、 +[势准备证据](../.workspace/silicon-t2/pseudopotential-evidence.json)。 + +配套教学：[互动首课](learning/vasp/lessons/0001-pbe-potcar-bohrium.html)与 +[输入速查卡片](learning/vasp/reference/vasp-inputs.html)。 From c0169fef352cc8110e701faf0eca65b92ffbaac6 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 18:02:44 +0800 Subject: [PATCH 10/22] docs: record verified real silicon relaxation and static acceptance --- docs/pbe_bohrium_research.md | 11 +++ docs/silicon_vasp_t2.md | 136 ++++++++++++++++----------------- docs/vasp_runtime_diagnosis.md | 12 +++ 3 files changed, 87 insertions(+), 72 deletions(-) diff --git a/docs/pbe_bohrium_research.md b/docs/pbe_bohrium_research.md index cde749a..1cef9a8 100644 --- a/docs/pbe_bohrium_research.md +++ b/docs/pbe_bohrium_research.md @@ -71,3 +71,14 @@ MatCreator 已在原会话 `silicon-t2-20261007-env2` 解压并核验所需 Si 配套教学：[互动首课](learning/vasp/lessons/0001-pbe-potcar-bohrium.html)与 [输入速查卡片](learning/vasp/reference/vasp-inputs.html)。 + +## 真实运行核验 + +原会话中的 Si 松弛与静态总能均已完成科学验证：松弛 durable job +`56072cf15fe94e639d7b71f563ed320a` / batchjob +`5ed5945230884cf88f30475c1afed598`，能量 -10.84124445 eV；静态 durable job +`c266a691a9874ab9ad097bfd09dca9b6` / batchjob +`6ce96bb830764a01976fec7ef9d16a99`，能量 -10.84601452 eV（均为 Si2 晶胞总能）。 +实际输出证明该私有镜像可以运行 VASP 6.3.0，所暂存的 Si POTCAR 确实参与两步计算。 +静态结构来自成功松弛结构，势哈希相同。完整证据与测试见 +[T2 验收记录](silicon_vasp_t2.md)。本研究不宣称数据库归档或完整本地闭环已完成。 diff --git a/docs/silicon_vasp_t2.md b/docs/silicon_vasp_t2.md index 05f7bac..0b89334 100644 --- a/docs/silicon_vasp_t2.md +++ b/docs/silicon_vasp_t2.md @@ -46,75 +46,67 @@ T2 never claims a record was saved or a local loop was completed. ## Validation and real-run evidence -```bash -PYTHONPATH=src .venv/bin/python -m pytest tests/test_silicon_vasp_workflow.py tests/test_aidb_skill_hook.py tests/test_remote_job_tools.py -q -``` - -The new tests use a controlled provider and explicitly synthetic VASP XML, -parsed by the real pymatgen parser. Agent-runner cases use a real isolated aidb -public bridge/database for query ordering and user-visible receipts. These -tests prove orchestration and scientific failure handling, not real computation. - -On 2026-10-07 a real natural request was dispatched to MatCreator with session -`silicon-t2-20261007`, using the existing user configuration and local aidb. -The real preflight succeeded with no Si/PBE/bulk candidates (no records deleted). -Private run evidence is retained under `.workspace/silicon-t2/`, including -`events.jsonl`, `trajectories/`, `.aidb/runs/` reports and audits. Real Bohrium -completion is still pending; no successful job ID or real T3 handoff is claimed. - -The corrected launch session `silicon-t2-20261007-env2` also queried the real -local database successfully (empty candidates) and generated diamond Si2 using -the existing ASE skill. MatCreator stopped before submission after an actual -MPRelaxSet POTCAR check raised `PmgVaspPspDirError`: no licensed PBE potential -directory is configured. `compute.vasp_image` / `BOHRIUM_VASP_IMAGE` is also -empty. Bohrium CLI authentication and machine discovery succeeded. The specific -diagnostic is `.workspace/silicon-t2/preflight_env_report_step2.json`; no tracked -job was submitted and no successful real calculation is claimed. - -Automated validation: the final targeted run passed 252 tests, including the -workflow in both production agent entry points. Final full-suite execution -passed 760 tests, with 39 failures and two collection errors. A snapshot of -the starting commit reproduced exactly the same failure names (no new failed -tests). The baseline snapshot skipped aidb integration because it was outside -the adjacent-checkout layout. Final review added regression coverage for mutable -POTCAR inputs, dispersion corrections, and submission replay provenance. -Follow-up [runtime diagnosis](vasp_runtime_diagnosis.md) found and fixed tool -Python/PATH selection bugs and discovered the account-private VASP 6.3.0 image. -Its executable/potential paths cannot be established from stored metadata. -Real acceptance remains paused pending a verifiable PBE potential source and -usable runtime; the existing calculation authorization remains valid. Issue #3 -is not marked complete. - -The initial noninteractive launcher omitted the NVM bohr path; the installed -CLI is `/home/shik-mechrevo-wsl/.nvm/versions/node/v24.14.1/bin/bohr` (2.6.100). -Future launches should prepend both the repository `.venv/bin` and this Node -directory to PATH. Python tools now pin the runtime interpreter and Bash tools -retain that startup environment. - -## Standards review - -Reviewed implementation commits `db7904a` and `3c742de` against starting commit -`071cfab6202b704704a27376d7f136dec8e5b884` with -`git diff 071cfab6202b704704a27376d7f136dec8e5b884...HEAD`. - -Final review: 0 documented-standard breaches and 0 remaining baseline smells. -The duplicated structure comparisons and input-path checks now use shared -helpers. The initial POTCAR provenance concern is resolved by submission-time -hashes, collection checks and XML identity matching. Synthetic tests do not -establish real scientific completion. - -## Spec review - -The initial P1 (mutable POTCAR provenance) and P2 (dispersion-corrected PBE -accepted as ordinary PBE) are resolved. Submission replay preserves the original -input evidence; modified potentials are rejected. IVDW/LUSE_VDW are rejected, -and k-point inputs plus available XML sampling are preserved. No additional -code-correctness or scope issue was identified in the final review. - -One acceptance requirement remains unmet: one real successful Bohrium silicon -task. Actual query, structure generation and environment discovery are evidenced, -but unresolved PBE potentials and VASP runtime details prevent submission. No successful -job, archive or completed local loop is claimed. - -Review totals: Standards 0 findings; Spec 1 outstanding environment-blocked -acceptance requirement (real Bohrium completion). +Final related regression run: **286 passed**, covering ADK preflight callbacks, +workspace subprocesses, scientific collection, tracked jobs and the Bohrium +Batch Job adapter. Synthetic XML tests use the actual pymatgen parser; agent +cases use a real isolated aidb bridge/database. They are not real calculations. + +The final full suite had **786 passed, 39 failed and 2 collection errors**. +Failure names exactly match the initial baseline snapshot. Logs are retained in +`.workspace/silicon-t2/debug-pbe/final-full-fixed.log` and +`.workspace/silicon-t2/baseline-tests.log`. Unrelated legacy failures were not +changed. Standards and Spec reviews found zero new implementation findings. + +The real run remains in session `silicon-t2-20261007-env2`. A real preflight +succeeded with empty Si/PBE/bulk candidates; no database records were deleted. +The user-supplied potential and the account-private image resolved the original +resource/configuration blockers. MatCreator owns input generation and submission; +Codex only diagnosed the runtime, repaired platform validation and resumed the +existing session. See [runtime diagnosis](vasp_runtime_diagnosis.md) and +[PBE research](pbe_bohrium_research.md). + +Verified real relaxation: + +- Durable job ID: `56072cf15fe94e639d7b71f563ed320a`. +- Provider batchjob ID: `5ed5945230884cf88f30475c1afed598`. +- VASP image: `registry.dp.tech/dptech/prod-28882/vasp:6.3.0`. +- POTCAR: `PAW_PBE Si 05Jan2001`, SHA256 + `52dbe99da884e0191b2d348dfcc281aaba36e2eacd3019a49d93cf07d84da86b`. +- Electronic and ionic convergence, three ionic steps, completed OUTCAR. +- Total energy: **-10.84124445 eV per Si2 cell**. +- Verified report: `silicon-t2-results/relax-collected/silicon-result.json`. + +All paths above are relative to `.workspace/silicon-t2/`. Actual source outputs, +submission input hashes, query reports/audits, `events-pbe.jsonl` and trajectories +are retained. The relaxation was never resubmitted. MatCreator completed the +static continuation using the verified CONTCAR and the same potential. +No archive or full local-loop completion is claimed. Issue #3 remains open; +both steps now have real scientific completion evidence. + +Verified real static continuation: + +- Durable job ID: `c266a691a9874ab9ad097bfd09dca9b6`. +- Provider batchjob ID: `6ce96bb830764a01976fec7ef9d16a99`. +- Total energy: **-10.84601452 eV per Si2 cell**. +- Electronic convergence, one static step and completed OUTCAR. +- `NSW=0`; effective `IBRION=-1`; actual ENCUT 520 eV. +- Static input matches the verified relaxed structure; unchanged POTCAR SHA256. +- Verified report: `silicon-t2-results/static-collected/silicon-result.json`, + with the relaxation IDs and energy retained in `relaxation_source`. + +Both authorized steps now have real scientific completion evidence. No third +job, probe computation, archive or cloud database upload was performed. The +first collection round produced the successful report but ended with ADK +`StaleSessionError`; a subsequent child collection reported a workspace-path +error. A public-tool replay in the original workspace returned success from +both original VASP output sets without altering inputs or outputs. These +runner incidents and recovery logs remain in `debug-pbe/`; this does not claim +a general fix for ADK concurrent session updates. + +T2's real-computation acceptance is evidenced. T3 archive/requery and the final +A-to-B switch in the main specification remain separate work. Issue #3 was not +closed by this session. + +MatCreator's final report is +`.workspace/silicon-t2/silicon-t2-results/silicon-t2-final-report.json`. +Its listed output and scientific-report checksums match the retained files. diff --git a/docs/vasp_runtime_diagnosis.md b/docs/vasp_runtime_diagnosis.md index 8f1c4da..49c9eba 100644 --- a/docs/vasp_runtime_diagnosis.md +++ b/docs/vasp_runtime_diagnosis.md @@ -171,3 +171,15 @@ Each fix has red-to-green regression evidence in `debug-pbe/`. The real relaxation validator reproduction also passed without changing scientific inputs or outputs. Final related tests: **286 passed**; both review axes found zero new issues. MatCreator resumed the static step after the final fix. + +The static continuation also completed and passed scientific checks: +durable job `c266a691a9874ab9ad097bfd09dca9b6`, provider batchjob +`6ce96bb830764a01976fec7ef9d16a99`, total energy -10.84601452 eV per Si2 +cell, electronic convergence, one static step and completed OUTCAR. Static +input matches the verified relaxed structure and uses the same POTCAR hash. +Both output sets and the parent-child source link are retained. Final complete +suite: 786 passed, 39 failures and two collection errors, with the exact same +failure names as the starting baseline. No new job was needed for recovery. +ADK session-version conflicts and a child replay workspace-path incident remain +recorded separately; a public-tool replay in the original workspace succeeded. +This is real T2 computation evidence, not proof of T3 archiving or final B behavior. From a1e74996dda4706fd3154c130795a2614284b25b Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:36:30 +0800 Subject: [PATCH 11/22] feat: archive verified silicon outputs through public aidb round trip --- docs/silicon_vasp_t3.md | 92 +++++++++ src/matcreator/agents/aidb_skill_hook.py | 3 + .../agents/execution_agent/step_executor.py | 2 + src/matcreator/agents/thinking_agent/agent.py | 2 + src/matcreator/skills/vasp-pymatgen/SKILL.md | 16 +- src/matcreator/tools/silicon_archive.py | 181 ++++++++++++++++++ tests/test_silicon_archive.py | 140 ++++++++++++++ 7 files changed, 435 insertions(+), 1 deletion(-) create mode 100644 docs/silicon_vasp_t3.md create mode 100644 src/matcreator/tools/silicon_archive.py create mode 100644 tests/test_silicon_archive.py diff --git a/docs/silicon_vasp_t3.md b/docs/silicon_vasp_t3.md new file mode 100644 index 0000000..a813d25 --- /dev/null +++ b/docs/silicon_vasp_t3.md @@ -0,0 +1,92 @@ +# Silicon VASP T3: automatic local archive and content verification + +Scope: [Ai-ready_Database #4](https://github.com/Zikkying/Ai-ready_Database/issues/4), +using [#1](https://github.com/Zikkying/Ai-ready_Database/issues/1) as the main spec. +Development A remains active. Natural candidate reuse (T4) and final B are +separate work. This implementation submits no calculation and uploads nothing +to a cloud database. No database internals are imported by MatCreator. + +## T2 gate and workflow + +Before implementation, both existing T2 jobs were verified against the durable +job store, immutable submitted-input checksums, complete original VASP outputs +and the T2 final report. Electronic/ionic completion and the static-to-relaxed +structure/potential link passed. Original report/output checksums still match +after T3. Issue #3 remains open; its real successful artifacts, rather than its +issue state alone, establish this prerequisite. See [T2 evidence](silicon_vasp_t2.md). + +The registered main-agent and step-executor after-tool callback automatically +archives successful `collect_silicon_vasp_result` responses. The public +`archive_silicon_vasp_result` tool retries storage for an existing job. Both +revalidate raw outputs and tracked provenance, then use the existing aidb Skill +adapter, client/server, query, archive-and-requery and JSONL export interfaces. +Failed/running/unknown/unconverged calculations retain outputs and do not write +successful records. Scientific success and storage verification have separate +receipts; `silicon-result.json` remains the unchanged scientific handoff. + +Records contain final structure, cell total energy (eV), actual INCAR and +effective conditions, successful completion evidence, tracked/provider IDs, +session, submission settings, source paths and hashes. Additional properties +are not invented. Static metadata links the verified parent record and its +original relaxation job; the two energies remain separate. + +Each attempt queries and exports first. Matching job identities are compared +before reuse; differing content is reported without overwriting. A new result +uses archive-and-requery, verifies the returned ID and compares exported +structure, energy/unit, conditions, completion and provenance. Per-job file +locks serialize duplicate notifications. A timed-out write stays unverified; +the next attempt queries durable state before another write. Adapter failures, +receipts and exported content remain under `.aidb/archives//`. Only a +verified readback is reported as archived. A UTC `archived_at` value persists in +record metadata and is preserved on duplicate processing. + +## Real acceptance, 2026-10-07 + +MatCreator resumed the original session `silicon-t2-20261007-env2` and called +the public archive tool for each original collected job, relaxation first. +Its tool events, graph/trajectories and final report remain in the original +workspace. Both real archive receipts passed ID requery and content comparison. + +| Step | Tracked job ID | Local record ID | Cell energy (eV) | Archive time (UTC) | +| --- | --- | --- | ---: | --- | +| Relaxation | `56072cf15fe94e639d7b71f563ed320a` | `694772dca0777de504b25ba70366fc3d` | -10.84124445 | 2026-10-07 13:30:20.838212 | +| Static | `c266a691a9874ab9ad097bfd09dca9b6` | `1a1e16e62710d4bec6721ad350ee1f05` | -10.84601452 | 2026-10-07 13:30:46.863940 | + +A separate process subsequently queried/exported these same records through +the public adapter. Its independent checks matched CONTCAR, INCAR, the known +T2 energy values and units, completion, parent record ID and every retained +output hash. This proves persistent round trips, not T4 natural-language reuse. + +Evidence paths relative to `.workspace/silicon-t2/`: + +- `t3-t2-verification.json`: prerequisite reparse and hash verification. +- `silicon-t2-results/silicon-t3-final-report.json`: MatCreator's real report, + with both archive IDs and complete receipt/query/export/audit paths. +- `events-pbe.jsonl`, `t3-runner.log`, `trajectories/`: original-session execution. +- `t3-independent-query.json`, `t3-independent-export/`, + `t3-independent-verification.json`: independent persistent readback. + +## Regression and limitations + +Nine new full-tool/callback tests use synthetic completed VASP outputs and real +isolated aidb databases/public bridges. They cover both registered ADK agents, +automatic saving, duplicates, static parent links, invalid scientific outputs, +lookup failure/recovery, uncertain write recovery and changed readback units. +They are regression evidence, not real calculation evidence. + +Related regression: **356 passed**. Full suite: **795 passed, 39 failed, +2 collection errors**; the exact failure/error names match the T2 baseline +(`debug-pbe/final-full-fixed.log`). Logs: `t3-targeted-tests.log` and +`t3-full-tests.log`. Changed Python modules compile successfully. No repository +typechecking command is configured. No aidb code/schema change was needed: +its public record metadata/export contract retained the required content. + +One regression corruption call initially omitted its isolated config and wrote +synthetic record `e506b6d0b06ebb6b3a866195dec9a4b8` to the default local database. +The test now supplies the config explicitly. That exact record was corrected +through public archive-local to `rejected_records`, `status=invalid`, +`test_only=true`, `real_computation=false`. It was not counted as real evidence; +no user records were deleted. Recovery payload: `t3-accidental-fixture.json`. + +Standards/Spec review results are recorded after implementation review. #4 and +#3 have not been closed or commented on by this session. diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index 94119a3..b05c9f8 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -304,6 +304,9 @@ async def before_aidb_skill_load(tool, args, tool_context): def after_aidb_skill_load(tool, args, tool_context, tool_response): """Put lookup evidence in the actual tool response seen by the agent/user.""" name = getattr(tool, 'name', '') + if name == 'collect_silicon_vasp_result' and isinstance(tool_response, dict) and tool_response.get('status') == 'success': + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + return archive_silicon_vasp_result(**args, tool_context=tool_context) if name in _PREPARATION_TOOLS or (name == 'load_skill' and str(args.get('skill_name') or '').strip().lower() in AIDB_PREFLIGHT_SKILLS): feedback = tool_context.state.get(f'temp:aidb_skill_feedback_{tool_context.function_call_id}') if feedback and isinstance(tool_response, dict): diff --git a/src/matcreator/agents/execution_agent/step_executor.py b/src/matcreator/agents/execution_agent/step_executor.py index 2daf4c2..cd433ae 100644 --- a/src/matcreator/agents/execution_agent/step_executor.py +++ b/src/matcreator/agents/execution_agent/step_executor.py @@ -18,6 +18,7 @@ from ...tools.util_tools import show_artifact, show_plot, show_structure from ...tools.workspace_tools import get_user_skills_root, run_bash, run_python from ...tools.silicon_vasp import collect_silicon_vasp_result +from ...tools.silicon_archive import archive_silicon_vasp_result from .remote_job_tools import ( attach_bohr_batchjob, collect_remote_job_outputs, @@ -322,6 +323,7 @@ def build_step_executor_agent(llm_card: LLMCard) -> LlmAgent: FunctionTool(download_remote_job_output), FunctionTool(collect_remote_job_outputs), FunctionTool(collect_silicon_vasp_result), + FunctionTool(archive_silicon_vasp_result), FunctionTool(pause_remote_job), FunctionTool(terminate_remote_job), ALL_SKILLS_TOOLSET, diff --git a/src/matcreator/agents/thinking_agent/agent.py b/src/matcreator/agents/thinking_agent/agent.py index 1a3271d..b847fa6 100644 --- a/src/matcreator/agents/thinking_agent/agent.py +++ b/src/matcreator/agents/thinking_agent/agent.py @@ -48,6 +48,7 @@ from .history_tools import read_session_log from ..execution_agent.remote_job_tools import list_remote_jobs from ...tools.silicon_vasp import collect_silicon_vasp_result +from ...tools.silicon_archive import archive_silicon_vasp_result logger = logging.getLogger(__name__) @@ -502,6 +503,7 @@ def before_agent_callback(callback_context: CallbackContext) -> None: FunctionTool(read_session_log), FunctionTool(list_remote_jobs), FunctionTool(collect_silicon_vasp_result), + FunctionTool(archive_silicon_vasp_result), FunctionTool(load_skill), show_artifact, show_plot, diff --git a/src/matcreator/skills/vasp-pymatgen/SKILL.md b/src/matcreator/skills/vasp-pymatgen/SKILL.md index b1f5662..af3c283 100644 --- a/src/matcreator/skills/vasp-pymatgen/SKILL.md +++ b/src/matcreator/skills/vasp-pymatgen/SKILL.md @@ -16,6 +16,7 @@ metadata: - run_bash - load_skill_resource - collect_silicon_vasp_result + - archive_silicon_vasp_result dependent_skills: - bohrium - remote-job @@ -114,7 +115,7 @@ Run exactly **one property step at a time**. Do not chain relaxation + SCF in a For diamond bulk Si / ordinary PBE relaxation and total energy, inspect the actual `aidb_preflight` tool response before creating a structure or inputs. Failed lookup pauses new computation. Missing identity/model must be clarified; -retrieval defaults are not calculation authorization. Do not archive in T2. +retrieval defaults are not calculation authorization. Use the existing ASE diamond structure recipe and `MPRelaxSet`, then tracked Batch Job submission. On completion call `collect_silicon_vasp_result(job_id, @@ -134,6 +135,19 @@ Old jobs without that evidence remain unverified. Ordinary PBE excludes U, hybrid, meta-GGA and dispersion corrections. The report retains k-point inputs and the actual XML sampling when available. +After successful collection, MatCreator automatically saves the verified result +to local aidb using the public archive-and-requery adapter and compares the +exported structure, cell energy/unit, actual conditions, completion and source. +Inspect `local_archive`: only `status="verified"`, `archived=true`, the returned +record ID and `content_compared=true` prove local storage. Scientific success +alone and a write receipt do not prove this. Archive the relaxation first; +the static record retains that parent ID and its own job/energy identity. +On archive failure retain original outputs and diagnostics. Retry with +`archive_silicon_vasp_result` for the same collected job; it revalidates outputs +and queries before writing, including after timeout. Never resubmit a completed +calculation to repair storage. Do not request confirmation for local saving. +No cloud database upload is performed. Development A remains active. + For **MLFF energy/force labeling**, use `prepare_label` as a standalone step — it replaces the relaxation → SCF chain with a single static calculation optimized for dataset generation (no charge density, no relaxation, KSPACING-based k-points). --- diff --git a/src/matcreator/tools/silicon_archive.py b/src/matcreator/tools/silicon_archive.py new file mode 100644 index 0000000..4c4f284 --- /dev/null +++ b/src/matcreator/tools/silicon_archive.py @@ -0,0 +1,181 @@ +"""Local Si result round trips through the aidb Skill's public adapter only.""" +from __future__ import annotations + +from datetime import datetime, timezone +import json +import os +from pathlib import Path +import subprocess +import sys +import uuid + +from google.adk.tools.tool_context import ToolContext +from monty.json import MontyEncoder + + +def _write(path, payload): + temporary = path.with_name(f'.{path.name}.{uuid.uuid4().hex}.tmp') + temporary.write_text(json.dumps(payload, cls=MontyEncoder, indent=2), encoding='utf-8') + temporary.replace(path) + + +def _compare(saved, source): + from pymatgen.core import Structure + from .silicon_vasp import _same_structure + source = json.loads(json.dumps(source, cls=MontyEncoder)) + mismatches = [] + if not _same_structure(Structure.from_dict(saved['structure']), Structure.from_dict(source['structure'])): + mismatches.append('structure') + for key in ('source_dataset', 'source_id', 'energy', 'functional', 'calculation_type', 'calculation_parameters'): + if saved.get(key) != source.get(key): + mismatches.append(key) + metadata = saved.get('source_metadata') or {} + for key, value in source['source_metadata'].items(): + if metadata.get(key) != value: + mismatches.append(f'source_metadata.{key}') + if not metadata.get('archived_at'): + mismatches.append('source_metadata.archived_at') + if mismatches: + raise ValueError('Archived content differs from verified output: ' + ', '.join(mismatches)) + + +def archive_verified_result(report: dict, tool_context: ToolContext) -> dict: + """Archive a freshly scientifically verified handoff; never submit a job. + + Query/export before each write, including recovery after an uncertain + timeout. Never overwrite a differing record for the same tracked job. + """ + from matcreator.agents.aidb_skill_hook import _adapter_path + workspace = Path(tool_context.state['workspace_dir']).resolve() + directory = workspace / '.aidb' / 'archives' / report['job_id'] + directory.mkdir(parents=True, exist_ok=True) + attempt = directory / uuid.uuid4().hex + attempt.mkdir() + receipt = {'status': 'failed', 'archived': False, 'job_id': report['job_id'], + 'source_report_path': str(Path(report['output_path']) / 'silicon-result.json'), + 'receipt_path': str(attempt / 'receipt.json'), 'cloud_upload': False} + try: + if report.get('status') != 'success': + raise ValueError('Scientific success is required before local archive.') + # This Unix lock matches the supported Unix-socket aidb bridge. It also + # prevents concurrent completion callbacks from racing the same write. + import fcntl + with (directory / 'archive.lock').open('a') as lock: + fcntl.flock(lock, fcntl.LOCK_EX) + task_path = attempt / 'task.json' + _write(task_path, {'task_id': f"silicon-{report['job_id']}", 'target': { + 'formula': 'Si', 'functional': 'PBE', 'domain': 'bulk', 'min_records': 1}, + 'related_materials': []}) + adapter = _adapter_path() + + def command(stage, arguments): + env = {**os.environ, 'AI_READY_DB_BRIDGE_TIMEOUT': '20', 'AI_READY_DB_BRIDGE_STARTUP_TIMEOUT': '15'} + try: + proc = subprocess.run([sys.executable, str(adapter), '--workspace', str(workspace), + *arguments], capture_output=True, text=True, env=env, timeout=75) + except subprocess.TimeoutExpired as exc: + receipt['write_completion_unknown'] = stage == 'archive' + _write(attempt / f'{stage}.json', {'timed_out': True, 'stage': stage, + 'stdout': str(exc.stdout or ''), 'stderr': str(exc.stderr or '')}) + raise + _write(attempt / f'{stage}.json', {'returncode': proc.returncode, + 'stdout': proc.stdout, 'stderr': proc.stderr}) + payload = json.loads(proc.stdout) + receipt[f'{stage}_audit_path'] = payload.get('audit_path') + receipt[f'{stage}_report_path'] = payload.get('report_path') + if proc.returncode or payload.get('ok') is not True: + raise ValueError(f'{stage} failed: {payload}') + return payload + + preflight = command('recovery_query', ['preflight', '--task-json', str(task_path), + '--command-timeout', '20', '--export-dir', str(attempt / 'before'), '--export-format', 'jsonl']) + query = json.loads(Path(preflight['report_path']).read_text()) + records = [] + for export in query['exports']: + records.extend(json.loads(line) for line in Path(export['output_path']).read_text().splitlines() if line.strip()) + existing = [r for r in records if r.get('source_dataset') == 'matcreator-bohrium' + and r.get('source_id') == report['job_id']] + if len(existing) > 1: + raise ValueError('Multiple archive identities for this tracked job; resolve conflict before writing.') + conditions = report['conditions'] + metadata = {'job_id': report['job_id'], 'batchjob_id': report['batchjob_id'], + 'session_id': str(tool_context.state.get('session_id') or ''), + 'status': 'success', 'completion': report['completion'], 'energy_unit': report['total_energy']['unit'], + 'energy_basis': 'cell', 'conditions': conditions, 'artifacts': report['artifacts'], + 'submission': report['submission'], 'output_path': report['output_path'], + 'relaxation_source': report.get('relaxation_source'), + 'parent_record_id': None, 'origin': 'local_calculation', 'sync_status': 'local_only'} + parent = report.get('relaxation_source') + if parent: + parents = [r for r in records if r.get('source_dataset') == 'matcreator-bohrium' + and r.get('source_id') == parent['job_id'] and r.get('calculation_type') == 'relaxation'] + if len(parents) != 1: + raise ValueError('Archive and verify preceding relaxation before static result.') + from matcreator.agents.execution_agent.remote_job_tools import _service + from .silicon_vasp import _validate, _input_directory + parent_spec = _service().store.get_job(parent['job_id'])['specification'] + parent_report = _validate(Path(parent['output_path']), 'relaxation', + _input_directory(parent_spec, workspace), parent_spec.get('vasp_input_sha256', {})) + parent_source = {'source_dataset': 'matcreator-bohrium', 'source_id': parent['job_id'], + 'structure': parent_report['structure'], 'energy': parent_report['total_energy']['value'], + 'functional': 'PBE', 'calculation_type': 'relaxation', + 'calculation_parameters': parent_report['conditions']['incar'], + 'source_metadata': {'conditions': parent_report['conditions'], 'status': 'success', + 'completion': parent_report['completion'], 'energy_unit': 'eV', 'job_id': parent['job_id']}} + _compare(parents[0], parent_source) + metadata['parent_record_id'] = parents[0]['record_id'] + source = {'source_dataset': 'matcreator-bohrium', 'source_id': report['job_id'], + 'structure': report['structure'], 'energy': report['total_energy']['value'], + 'functional': 'PBE', 'calculation_type': report['calculation_type'], + 'calculation_parameters': conditions['incar'], 'source_metadata': metadata, + 'provenance': {'provider': 'bohr_batchjob', 'job_id': report['job_id'], 'batchjob_id': report['batchjob_id']}} + # Normalize JSON exactly as it travels through the public contract. + source = json.loads(json.dumps(source, cls=MontyEncoder)) + if existing: + _compare(existing[0], source) + saved = existing[0] + export_path = attempt / 'existing.jsonl' + _write(export_path, saved) + receipt['previously_present'] = True + else: + record_path = directory / 'record.json' + if record_path.exists(): + pending = json.loads(record_path.read_text()) + _compare(pending, source) + source['source_metadata']['archived_at'] = datetime.now(timezone.utc).isoformat() + _write(record_path, source) + archived = command('archive', ['archive-and-requery', '--task-json', str(task_path), + '--record-json', str(record_path), '--origin', 'local_calculation', '--sync-status', 'local_only', + '--export-dir', str(attempt / 'after'), '--export-format', 'jsonl']) + receipt['record_id'] = archived['record_id'] + export_path = Path(archived['export_path']) + readback = [json.loads(line) for line in export_path.read_text().splitlines() if line.strip()] + if len(readback) != 1 or readback[0]['record_id'] != archived['record_id']: + raise ValueError('Archive export does not contain exactly the returned record ID.') + saved = readback[0] + _compare(saved, source) + receipt['previously_present'] = False + receipt.update(status='verified', archived=True, record_id=saved['record_id'], + export_path=str(export_path), archived_at=saved['source_metadata']['archived_at'], + verified_in_requery=True, content_compared=True) + except Exception as exc: + receipt['error'] = str(exc) or type(exc).__name__ + receipt['message'] = 'Local archive unverified; retain outputs and retry lookup before writing. Do not resubmit calculation.' + _write(attempt / 'receipt.json', receipt) + return receipt + + +def archive_silicon_vasp_result(job_id: str, destination_path: str, calculation_type: str, + tool_context: ToolContext, relaxation_job_id: str = '') -> dict: + """Retry local archive of an existing Si job after rechecking original outputs. + + Does not submit any computation or upload to a cloud database. + """ + from .silicon_vasp import collect_silicon_vasp_result + collected = collect_silicon_vasp_result(job_id, destination_path, calculation_type, tool_context, relaxation_job_id) + if collected.get('status') != 'success': + return collected + report = json.loads(Path(collected['report_path']).read_text()) + archive = archive_verified_result(report, tool_context) + return {**collected, 'archived': archive['archived'], 'local_archive': archive, + 'message': 'Scientific result verified; local archive ' + archive['status'] + '.'} diff --git a/tests/test_silicon_archive.py b/tests/test_silicon_archive.py new file mode 100644 index 0000000..3672007 --- /dev/null +++ b/tests/test_silicon_archive.py @@ -0,0 +1,140 @@ +"""Full tool/callback round trips; synthetic outputs are not real computation.""" +import json +from pathlib import Path +from types import SimpleNamespace +import pytest +import hashlib +import subprocess +import os +import sys + +from matcreator.agents.aidb_skill_hook import after_aidb_skill_load +from matcreator.tools.silicon_vasp import collect_silicon_vasp_result +from test_aidb_skill_hook import local_aidb, run_task +from test_silicon_vasp_workflow import completed_job, silicon_outputs + + +def test_successful_collection_automatically_archives_and_reads_back(local_aidb, tmp_path, monkeypatch): + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + args = {'job_id': job_id, 'destination_path': 'outputs', 'calculation_type': 'relaxation'} + response = collect_silicon_vasp_result(**args, tool_context=context) + response = after_aidb_skill_load(SimpleNamespace(name='collect_silicon_vasp_result'), args, context, response) + assert response['archived'] is True + archive = response['local_archive'] + assert archive['status'] == 'verified' + saved = json.loads(Path(archive['export_path']).read_text()) + assert saved['record_id'] == archive['record_id'] + assert saved['energy'] == -10.1 + assert saved['source_metadata']['energy_unit'] == 'eV' + assert saved['source_metadata']['completion']['electronic_converged'] is True + assert saved['source_metadata']['job_id'] == job_id + assert saved['calculation_parameters']['ENCUT'] == 520 + assert saved['source_metadata']['archived_at'] + assert saved['source_metadata']['sync_status'] == 'local_only' + assert saved['forces'] is None and saved['stress'] is None + + +@pytest.mark.parametrize('entry', ['thinking', 'step']) +def test_registered_agent_collection_automatically_archives(local_aidb, tmp_path, monkeypatch, entry): + _, _, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + responses, prepared = run_task(tmp_path, '取回已有金刚石硅 PBE 弛豫成功产物并自动本地归档，不重新计算', + [('collect_silicon_vasp_result', {'job_id': job_id, 'destination_path': 'outputs', 'calculation_type': 'relaxation'})], + state={'session_id': 'session'}, entry=entry, extra_tools=(collect_silicon_vasp_result,)) + assert responses[0]['local_archive']['content_compared'] is True + assert responses[0]['archived'] is True + assert prepared == [] + + +def test_repeated_completion_keeps_record_and_archive_time(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + first = archive_silicon_vasp_result(job_id, 'outputs', 'relaxation', context)['local_archive'] + second = archive_silicon_vasp_result(job_id, 'ignored', 'relaxation', context)['local_archive'] + assert first['status'] == second['status'] == 'verified' + assert second['previously_present'] is True + assert first['record_id'] == second['record_id'] + assert first['archived_at'] == second['archived_at'] + + +def test_unknown_scientific_completion_does_not_archive(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs(ionic_marker=False)) + result = archive_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['status'] == 'invalid' and result['archived'] is False + assert not (tmp_path / '.aidb/archives').exists() + assert (tmp_path / 'outputs/vasprun.xml').is_file() + + +def test_static_archive_keeps_parent_record_and_separate_energy(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + service, context, parent = completed_job(tmp_path, monkeypatch, silicon_outputs()) + relaxed = archive_silicon_vasp_result(parent, 'relax-outputs', 'relaxation', context) + outputs = silicon_outputs(stage='static') + outputs['vasprun.xml'] = outputs['vasprun.xml'].replace('-10.1', '-10.2') + source = tmp_path / 'static' + source.mkdir() + for name in ('INCAR', 'POSCAR', 'KPOINTS'): + (source / name).write_text(outputs[name]) + (source / 'POTCAR').write_bytes((tmp_path / 'relax/POTCAR').read_bytes()) + service.adapter_for('bohr_batchjob').outputs = outputs + job = service.submit_job(owner_id='test', session_id='session', provider='bohr_batchjob', + idempotency_key='static', spec={'input_root': str(tmp_path), 'input_path': 'static', + 'vasp_input_sha256': {p.name: hashlib.sha256(p.read_bytes()).hexdigest() for p in source.iterdir()}}) + service.store.transition_job(job['job_id'], 'succeeded') + result = archive_silicon_vasp_result(job['job_id'], 'static-outputs', 'static', context, parent) + assert result['archived'] is True + saved = json.loads(Path(result['local_archive']['export_path']).read_text()) + assert saved['energy'] == -10.2 + assert saved['source_metadata']['parent_record_id'] == relaxed['local_archive']['record_id'] + assert saved['source_metadata']['relaxation_source']['job_id'] == parent + assert saved['source_metadata']['completion']['ionic_converged'] is None + + +def test_lookup_failure_retains_outputs_and_can_recover_without_resubmission(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + with monkeypatch.context() as missing: + missing.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + result = archive_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert result['archived'] is False + assert result['local_archive']['status'] == 'failed' + assert not list((tmp_path / '.aidb/runs').glob('**/archive-report.json')) + assert (tmp_path / 'outputs/OUTCAR').is_file() + recovered = archive_silicon_vasp_result(job_id, 'ignored', 'relaxation', context) + assert recovered['archived'] is True + + +def test_uncertain_write_is_queried_and_read_back_before_retry(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + real_run = subprocess.run + def lose_receipt(argv, **kwargs): + result = real_run(argv, **kwargs) + if 'archive-and-requery' in argv: + raise subprocess.TimeoutExpired(argv, 75, output=result.stdout, stderr=result.stderr) + return result + with monkeypatch.context() as network: + network.setattr(subprocess, 'run', lose_receipt) + first = archive_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + assert first['archived'] is False + second = archive_silicon_vasp_result(job_id, 'ignored', 'relaxation', context) + assert second['archived'] is True + assert second['local_archive']['previously_present'] is True + assert len(list((tmp_path / '.aidb/runs').glob('**/archive-report.json'))) == 1 + + +def test_readback_difference_is_reported_without_overwriting_record(local_aidb, tmp_path, monkeypatch): + from matcreator.tools.silicon_archive import archive_silicon_vasp_result + _, context, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + first = archive_silicon_vasp_result(job_id, 'outputs', 'relaxation', context) + saved = json.loads(Path(first['local_archive']['export_path']).read_text()) + saved['source_metadata']['energy_unit'] = 'Ha' + edited = tmp_path / 'edited.json' + edited.write_text(json.dumps(saved)) + subprocess.run([sys.executable, str(local_aidb / 'skills/ai-ready-db/scripts/aidb_client.py'), + 'archive-local', '--record-json', str(edited), '--namespace', 'bulk_base_pbe', + '--origin', 'local_calculation', '--sync-status', 'local_only', '--config', os.environ['AI_READY_DB_CONFIG']], check=True, capture_output=True) + result = archive_silicon_vasp_result(job_id, 'ignored', 'relaxation', context) + assert result['archived'] is False + assert 'energy_unit' in result['local_archive']['error'] + assert len(list((tmp_path / '.aidb/runs').glob('**/archive-report.json'))) == 1 From 7ea0698e3cbacf5f3fffe7e4e1df202f15b0271e Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 21:41:39 +0800 Subject: [PATCH 12/22] docs: record silicon archive standards and spec review --- docs/silicon_vasp_t3.md | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/docs/silicon_vasp_t3.md b/docs/silicon_vasp_t3.md index a813d25..51148d5 100644 --- a/docs/silicon_vasp_t3.md +++ b/docs/silicon_vasp_t3.md @@ -88,5 +88,21 @@ through public archive-local to `rejected_records`, `status=invalid`, `test_only=true`, `real_computation=false`. It was not counted as real evidence; no user records were deleted. Recovery payload: `t3-accidental-fixture.json`. -Standards/Spec review results are recorded after implementation review. #4 and -#3 have not been closed or commented on by this session. +## Standards + +Zero findings in `c0169fef352cc8110e701faf0eca65b92ffbaac6...a1e7499`. +Source layout, skill metadata and testing conventions conform to AGENTS.md. +The archive/recovery/readback workflow is cohesive; no material baseline smell +was found. Pre-existing uncommitted instructions were excluded from review. + +## Spec + +Pass, zero actionable findings for #4. The review verified automatic registered +callbacks, raw scientific revalidation, public interfaces, content comparison, +parent linkage, timestamps, duplicate/timeout recovery and retained errors. +The real MatCreator report and independent public export support acceptance. +Synthetic regression fixtures are clearly separated from real Bohrium evidence. +T4 and B remain outside this task as explicitly requested. + +Review total: Standards 0, Spec 0; neither axis has an outstanding issue. +#4 and #3 have not been closed or commented on by this session. From 8351169f0dfa3b909cc2f13777a06aa2a04dede1 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:06:52 +0800 Subject: [PATCH 13/22] Implement verified silicon result reuse (Ai-ready_Database #5) --- docs/aidb_skill_hook.md | 16 ++ docs/silicon_vasp_t4.md | 91 +++++++ src/matcreator/agents/aidb_skill_hook.py | 71 ++++- .../agents/execution_agent/step_executor.py | 4 +- src/matcreator/agents/silicon_reuse.py | 146 ++++++++++ src/matcreator/agents/thinking_agent/agent.py | 4 +- tests/test_silicon_reuse.py | 253 ++++++++++++++++++ 7 files changed, 573 insertions(+), 12 deletions(-) create mode 100644 docs/silicon_vasp_t4.md create mode 100644 src/matcreator/agents/silicon_reuse.py create mode 100644 tests/test_silicon_reuse.py diff --git a/docs/aidb_skill_hook.md b/docs/aidb_skill_hook.md index bfd6504..e31b857 100644 --- a/docs/aidb_skill_hook.md +++ b/docs/aidb_skill_hook.md @@ -54,3 +54,19 @@ PYTHONPATH=src .venv/bin/python -m pytest tests/test_aidb_skill_hook.py -q 和相邻 Ai-ready_Database 的真实公开桥接及隔离本地库。命中样本仅用于测试， 不是本任务提交的真实计算。没有相邻 aidb checkout/runtime 时集成用例跳过。 不提交 Bohrium 作业、不归档未完成计算、不自动上传云端数据库。 + +## T4 verified candidate reuse + +[T4 evidence and rules](silicon_vasp_t4.md) extend the original T1 discovery +behavior. Public preflight now exports the exact candidate IDs and verifies +scientific content before selecting. `found` remains lookup status; +`reuse.status` separately reports `reused`, `recalculate`, `unavailable`, or +`needs_clarification`. The response includes candidate diagnostics, selected +structures and separate eV/cell energies, source metadata and choice reason. + +A compatible selection prevents preparation and new submission, including the +Flash executor entry. Explicit recalculation still queries first and respects +preparation-only authorization. Same-condition duplicates use persistent +archive times; ambiguity or meaningful condition differences require +clarification. Task changes trigger a new lookup. Development A and no cloud +database upload remain in force. This does not implement T5's final B behavior. diff --git a/docs/silicon_vasp_t4.md b/docs/silicon_vasp_t4.md new file mode 100644 index 0000000..44b1071 --- /dev/null +++ b/docs/silicon_vasp_t4.md @@ -0,0 +1,91 @@ +# Silicon T4: verified reuse in a new MatCreator session + +Scope: [Ai-ready_Database #5](https://github.com/Zikkying/Ai-ready_Database/issues/5), +main specification [#1](https://github.com/Zikkying/Ai-ready_Database/issues/1). +Development A remains active; final B belongs to T5. No new real computation +or cloud database upload was performed. Database access uses only the existing +Skill adapter/client/server and public query/export contracts. + +## T3 prerequisite verified before implementation + +Issue #4 is still open with no comments; that alone is not evidence of failure +or completion. A fresh public preflight/export read both real T3 records from +the default local database. The verifier reparsed retained raw VASP outputs, +checked durable collected-job/provider identities and submitted inputs, original +report and artifact hashes, archive receipts, parent linkage, structure, energy, +units, conditions and scientific completion. All checks passed. Evidence: +`.workspace/silicon-t4/t3-gate.json`, `gate-query.json`, `gate-export/`, +`verify-t3.py`. Original T2/T3 artifacts were retained without resubmission. + +## Behavior + +Preflight now exports and checks the exact current candidate set, then separates +candidate discovery from scientific applicability. Reuse requires ordered, +periodic diamond Si (space group 227), ordinary PBE without corrections, +successful electronic/ionic completion, final structure, finite eV/cell energy, +source identity, potential, cutoff and k-point evidence. Known input structure +and explicit INCAR/k-mesh constraints must agree. Retrieval defaults remain +visible and do not silently become reuse requirements. + +The current task and all extracted conditions bind the lookup/selection. A +fresh turn queries again; a changed material, method, cutoff or target cannot +reuse an old choice. Preparation, tracked submission and Flash executor entry +return existing results without starting work when reuse succeeds. Explicit +recalculation still queries first; internal executors inherit the human's +recalculation intent and preparation-only limits. Negated recalculation is not +interpreted as permission to recompute. + +Exact same-condition options are ordered by persistent timezone-aware archive +time. Missing/tied times or materially different complete conditions require +scientific clarification. Recency does not imply higher precision. Linked +relaxation/static records require the persisted parent and job relationship, +common potential/cutoff/spin and matching final structure. Both stages keep +their own settings and energies. Explicit static goals cannot be satisfied by +a relaxation energy. No general structural-equivalence or ranking system was +introduced. + +## Real natural-language acceptance + +A new process created a fresh SQLite ADK session +`silicon-t4-natural-final-20261007`, with no inherited conversation, receipt or +record ID in the user request. It ran the production MatCreator app/model in +Flash mode. Request: + +> 请给我金刚石结构单质硅体相的普通 PBE 弛豫后结构和总能。如果已有兼容且成功的结果，请直接使用并说明结构、能量单位、来源和选择依据。本次仅获取已有结果，不提交任何新计算，也不上传云端数据库。 + +The agent naturally reached the registered hook, queried/exported the same +local database, selected both associated real T3 records and reported their +structure, separate energies, conditions, sources and selection basis: + +| Stage | Record ID | Job ID | Energy (eV / 2-atom cell) | +| --- | --- | --- | ---: | +| Relaxation | `694772dca0777de504b25ba70366fc3d` | `56072cf15fe94e639d7b71f563ed320a` | -10.84124445 | +| Static | `1a1e16e62710d4bec6721ad350ee1f05` | `c266a691a9874ab9ad097bfd09dca9b6` | -10.84601452 | + +The primitive cell has vectors (0, 2.73483634, 2.73483634) and permutations, +fractional Si positions (0,0,0), (1/4,1/4,1/4), volume 40.90948591 ų. +The final agent response explicitly says selection does not establish highest +precision and no convergence comparison was performed. The initial acceptance +run had overstated static accuracy; shared agent guidance was corrected and +this second independent session verified the final feedback. + +Evidence under `.workspace/silicon-t4-natural-final/`: `request.txt`, `run.py`, +`session.db`, `events.jsonl`, `runner.log`, `result.json` and fresh `.aidb/` +query/export/audit artifacts. The harness retained the production model and +hook, with an additional no-computation/no-archive guard for acceptance only. +No guard denial occurred. This is real persistent reuse of the already completed +Bohrium results, not a replay or another real calculation. Synthetic regression +fixtures below are separate evidence. + +## Regression + +25 new task-level tests pass using ADK, both registered agents, controlled +remote tools and real isolated aidb databases/public bridges. They cover reuse +without preparation/submission, explicit/negated recalculation, parent intent, +recent archive order, ambiguous order, different conditions, linked/unlinked +stages, explicit static targets, missing/invalid scientific evidence, query +failure, parameter/target changes and Flash entry. The original archive suite +retains failure, missing output, unconverged, duplicate and uncertain-write +recovery coverage; archive errors preserve outputs rather than resubmitting. +Related T1–T4 run: 102 passed before the last four additional regression cases. +Changed modules compile. Full-suite results and independent review follow. diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index b05c9f8..f29fa00 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -1,7 +1,7 @@ """Development-stage material preflight through the public aidb Skill adapter. -Only candidate discovery is implemented here. A successful query is not proof -of scientific compatibility, and never authorizes a new remote calculation. +Candidate discovery and fixed Si/PBE reuse use public exports. Lookup never +authorizes a new remote calculation. """ from __future__ import annotations @@ -18,6 +18,20 @@ from .session_log import append_session_log_entry +AIDB_REUSE_INSTRUCTION = """ +For a silicon calculation request, consult the current `aidb_preflight` response +before preparing or submitting work. On reuse.status="reused", return its +verified results and record/source IDs; do not run preparation or computation. +Report relaxation and linked static energies separately with eV/cell units, +structure and actual conditions. Use the selection reason from the receipt. +Archive recency and a denser k mesh do not establish greater accuracy: never +claim a selected/static record is more accurate or highest precision. Current +public readback is authoritative; old memory/logs are not this turn's lookup. +On needs_clarification explain scientific differences or missing evidence; +on failed pause new computation (development A). Explicit recalculation still +requires lookup and stays within the user's existing authorization. +""" + logger = logging.getLogger(__name__) # Locks live only while callbacks are running; no Futures enter ADK state. _preflight_locks = weakref.WeakValueDictionary() @@ -25,7 +39,7 @@ AIDB_PREFLIGHT_SKILLS = frozenset({'vasp-pymatgen', 'atomic-structure'}) _PREPARATION_TOOLS = frozenset({'run_python', 'run_bash', 'run_skill_script', 'submit_bohr_batchjob', 'submit_bohr_sandbox', 'upload_remote_job_input', - 'start_remote_job_command', 'run_remote_job_command', 'run_sub_agent'}) + 'start_remote_job_command', 'run_remote_job_command', 'run_sub_agent', 'run_flash_step'}) _REMOTE_TOOLS = frozenset({'submit_bohr_batchjob', 'submit_bohr_sandbox', 'upload_remote_job_input', 'start_remote_job_command', 'run_remote_job_command'}) @@ -78,6 +92,22 @@ def _known_context(text, workspace): method = re.search(r'(? LlmAgent: "Executes a single plan step in an isolated session. " "Receives structured input with action and skill name; loads skill instructions autonomously." ), - instruction=_STEP_EXECUTOR_INSTRUCTION, + instruction=_STEP_EXECUTOR_INSTRUCTION + AIDB_REUSE_INSTRUCTION, input_schema=StepExecutorInput, before_tool_callback=before_aidb_skill_load, after_tool_callback=after_aidb_skill_load, diff --git a/src/matcreator/agents/silicon_reuse.py b/src/matcreator/agents/silicon_reuse.py new file mode 100644 index 0000000..a27806e --- /dev/null +++ b/src/matcreator/agents/silicon_reuse.py @@ -0,0 +1,146 @@ +"""Conservative reuse of the fixed diamond-Si/PBE workflow from public exports.""" +from datetime import datetime +import json +import math + +from pymatgen.core import Structure +from pymatgen.symmetry.analyzer import SpacegroupAnalyzer + +from matcreator.tools.silicon_vasp import _same_structure + + +def _validate(record, task): + meta = record.get('source_metadata') or {} + conditions = meta.get('conditions') or {} + completion = meta.get('completion') or {} + stage = record.get('calculation_type') + if meta.get('test_only') or meta.get('status') != 'success': + raise ValueError('successful scientific status absent') + if completion.get('electronic_converged') is not True or completion.get('outcar_termination') is not True: + raise ValueError('electronic completion evidence absent') + if stage not in {'relaxation', 'static'} or (stage == 'relaxation' and completion.get('ionic_converged') is not True): + raise ValueError('requested calculation completion absent') + structure = Structure.from_dict(record['structure']) + if (not structure.is_ordered or set(str(s) for s in structure.species) != {'Si'} + or not all(structure.lattice.pbc) + or SpacegroupAnalyzer(structure, symprec=0.01).get_space_group_number() != 227): + raise ValueError('structure is not diamond bulk Si') + if record.get('functional') != 'PBE' or any(conditions.get(k) != v for k, v in + {'functional': 'PBE', 'domain': 'bulk', 'structure_model': 'diamond'}.items()): + raise ValueError('method or system differs from ordinary PBE bulk diamond') + incar = conditions.get('incar') or {} + parameters = conditions.get('parameters') or {} + if not incar or record.get('calculation_parameters') != incar: + raise ValueError('basic calculation conditions absent or inconsistent') + for values in (incar, parameters): + if (values.get('LDAU') or values.get('LHFCALC') or values.get('ML_LMLFF') + or values.get('METAGGA', '') not in {'', 'None', 'NONE'} + or values.get('IVDW') or values.get('LUSE_VDW') or values.get('XC') + or str(values.get('GGA', 'PE')).upper() not in {'PE', '--'}): + raise ValueError('ordinary PBE excludes U, hybrid and other corrections') + if not conditions.get('potcar_sha256') or not conditions.get('potcar_symbols') or not all( + s.startswith('PAW_PBE Si ') for s in conditions['potcar_symbols']): + raise ValueError('PBE potential evidence absent') + if not conditions.get('kpoints') and not conditions.get('actual_kpoints'): + raise ValueError('k-point conditions absent') + if not isinstance(incar.get('ENCUT'), (int, float)) or incar['ENCUT'] <= 0: + raise ValueError('cutoff condition absent') + if 'encut' in task['known'] and incar['ENCUT'] != task['known']['encut']: + raise ValueError('cutoff differs from current request') + for key, value in task['known'].get('incar_constraints', {}).items(): + if incar.get(key, parameters.get(key)) != value: + raise ValueError(f'{key} differs from current request or is unknown') + if 'kpoint_mesh' in task['known']: + from pymatgen.io.vasp.inputs import Kpoints + mesh = Kpoints.from_str(conditions.get('kpoints') or '').kpts + if len(mesh) != 1 or list(mesh[0]) != task['known']['kpoint_mesh']: + raise ValueError('k-point mesh differs from current request') + if task['known'].get('structure_path'): + if not _same_structure(structure, Structure.from_file(task['known']['structure_path'])): + raise ValueError('given structure differs; no generic equivalence assumed') + if not isinstance(record.get('energy'), (int, float)) or isinstance(record['energy'], bool) or not math.isfinite(record['energy']): + raise ValueError('total energy absent or non-finite') + if meta.get('energy_unit') != 'eV' or meta.get('energy_basis') != 'cell': + raise ValueError('energy unit or basis unsupported/absent') + if not record.get('source_dataset') or not record.get('source_id') or not meta.get('job_id'): + raise ValueError('source identity absent') + return structure + + +def _compatible_pair(parent, child): + pm, cm = parent['source_metadata'], child['source_metadata'] + relation = cm.get('relaxation_source') or {} + if cm.get('parent_record_id') != parent['record_id'] or relation.get('job_id') != pm['job_id']: + return False + pc, cc = pm['conditions'], cm['conditions'] + # These stages deliberately have different ionic/smearing/k-point settings. + # Preserve both; require common method/potential/cutoff/spin and final cell. + return (pc['potcar_sha256'] == cc['potcar_sha256'] + and all(pc['incar'].get(k) == cc['incar'].get(k) for k in ('ENCUT', 'ISPIN', 'LDAU', 'LHFCALC')) + and _same_structure(Structure.from_dict(parent['structure']), Structure.from_dict(child['structure']))) + + +def _signature(records): + return json.dumps([{'structure': r['structure'], 'stage': r['calculation_type'], + 'incar': r['source_metadata']['conditions']['incar'], + 'kpoints': r['source_metadata']['conditions'].get('kpoints'), + 'actual_kpoints': r['source_metadata']['conditions'].get('actual_kpoints'), + 'potential': r['source_metadata']['conditions']['potcar_sha256']} for r in records], sort_keys=True) + + +def select_reuse(records, task): + """Separate candidates from applicability, without a database connection.""" + valid, candidates = [], [] + for record in records: + candidate = {'record_id': record.get('record_id'), 'calculation_type': record.get('calculation_type'), + 'energy': record.get('energy'), 'functional': record.get('functional'), + 'source_dataset': record.get('source_dataset'), 'source_id': record.get('source_id'), + 'structure': record.get('structure'), 'conditions': (record.get('source_metadata') or {}).get('conditions'), + 'completion': (record.get('source_metadata') or {}).get('completion'), + 'energy_unit': (record.get('source_metadata') or {}).get('energy_unit')} + try: + _validate(record, task) + valid.append(record) + candidate.update(compatible=True, reason='scientific content and basic conditions verified') + except (ValueError, KeyError, TypeError, AttributeError) as exc: + candidate.update(compatible=False, reason=str(exc)) + candidates.append(candidate) + base = {'candidates': candidates, 'record_ids': [], 'new_calculation_needed': False} + if not task['targets'] or any(k in task['unknown'] for k in ('structure_model', 'domain', 'functional')): + return {**base, 'status': 'needs_clarification', 'reason': '请明确目标、结构、体系及方法；检索默认值不能作为复用条件。'} + relax = [r for r in valid if r['calculation_type'] == 'relaxation'] + static = [r for r in valid if r['calculation_type'] == 'static'] + options = [] + if 'relaxation' in task['targets']: + if 'total_energy' in task['targets']: + options = [[p, c] for p in relax for c in static if _compatible_pair(p, c)] + if not options and not task.get('require_static'): + options = [[p] for p in relax] + elif 'total_energy' in task['targets']: + options = [[c] for c in static] + if not options and not task.get('require_static'): + options = [[p] for p in relax] + if not options: + return {**base, 'status': 'unavailable', 'new_calculation_needed': True, + 'reason': '候选尚未核验为满足当前目标的可复用结果；保留缺失/不符证据。'} + if len({_signature(option) for option in options}) > 1: + return {**base, 'status': 'needs_clarification', 'reason': '完整候选的结构或计算条件有实质差异，请选择科学条件。'} + if len(options) > 1: + try: + times = [datetime.fromisoformat(option[-1]['source_metadata']['archived_at']) for option in options] + if any(t.tzinfo is None for t in times) or len(set(times)) != len(times): + raise ValueError('archive order ambiguous') + options = [options[max(range(len(times)), key=times.__getitem__)]] + except (KeyError, ValueError, TypeError): + return {**base, 'status': 'needs_clarification', 'reason': '持久归档时间不足以确定最近成功结果，请澄清。'} + chosen = options[0] + energy = chosen[-1] + return {**base, 'status': 'recalculate' if task['recalculate'] else 'reused', + 'record_ids': [r['record_id'] for r in chosen], 'structure': chosen[0]['structure'], + 'total_energy': {'value': energy['energy'], 'unit': 'eV', 'basis': 'cell'}, + 'results': [{'record_id': r['record_id'], 'calculation_type': r['calculation_type'], + 'structure': r['structure'], 'total_energy': {'value': r['energy'], 'unit': 'eV', 'basis': 'cell'}, + 'source_dataset': r['source_dataset'], 'source_id': r['source_id'], + 'source_metadata': r['source_metadata']} for r in chosen], + 'reason': '公开读回核验成功、结构/方法/目标兼容；同条件优先最近成功归档，不代表精度最高。', + 'new_calculation_needed': task['recalculate']} diff --git a/src/matcreator/agents/thinking_agent/agent.py b/src/matcreator/agents/thinking_agent/agent.py index b847fa6..93c79c1 100644 --- a/src/matcreator/agents/thinking_agent/agent.py +++ b/src/matcreator/agents/thinking_agent/agent.py @@ -12,7 +12,7 @@ from google.adk.tools.tool_context import ToolContext from google.adk.agents.callback_context import CallbackContext -from ..aidb_skill_hook import before_aidb_skill_load, after_aidb_skill_load +from ..aidb_skill_hook import AIDB_REUSE_INSTRUCTION, before_aidb_skill_load, after_aidb_skill_load from ...constants import LLM_API_KEY, LLM_BASE_URL, LLM_MODEL from ...adk_compat import install_lenient_tool_argument_parsing from .planning import validate_plan, validate_graph @@ -408,7 +408,7 @@ async def run_flash_step( or `node_id="..."` for one executor of interest; do not request bulk detail by default. """ -_MATCREATOR_INSTRUCTION = "{instruction_body}" +_MATCREATOR_INSTRUCTION = "{instruction_body}" + AIDB_REUSE_INSTRUCTION # --------------------------------------------------------------------------- # before_agent_callback: inject dynamic context into session state diff --git a/tests/test_silicon_reuse.py b/tests/test_silicon_reuse.py new file mode 100644 index 0000000..c94c101 --- /dev/null +++ b/tests/test_silicon_reuse.py @@ -0,0 +1,253 @@ +"""Task-level reuse tests through ADK and the public isolated local database.""" +import json +import os +import subprocess +from pathlib import Path + +from test_aidb_skill_hook import local_aidb, run_task +from test_silicon_vasp_workflow import completed_job, silicon_outputs +from matcreator.tools.silicon_archive import archive_silicon_vasp_result + + +def seed_relaxation(tmp_path, monkeypatch): + _, context, job = completed_job(tmp_path, monkeypatch, silicon_outputs()) + result = archive_silicon_vasp_result(job, 'outputs', 'relaxation', context) + assert result['archived'], result + return json.loads(Path(result['local_archive']['export_path']).read_text()) + + +def test_compatible_success_reused_before_preparation_or_submission(local_aidb, tmp_path, monkeypatch): + saved = seed_relaxation(tmp_path, monkeypatch) + responses, prepared = run_task(tmp_path, '请计算金刚石硅体相普通 PBE 弛豫和总能', + [('load_skill', {'skill_name': 'vasp-pymatgen'}), ('run_python', {'code': 'prepare'}), + ('submit_bohr_batchjob', {'name': 'Si'})]) + feedback = responses[0]['aidb_preflight'] + assert feedback['reuse']['status'] == 'reused' + assert feedback['reuse']['record_ids'] == [saved['record_id']] + assert feedback['reuse']['total_energy'] == {'value': -10.1, 'unit': 'eV', 'basis': 'cell'} + assert feedback['reuse']['structure'] == saved['structure'] + assert prepared == [] + +def public_archive(local_aidb, tmp_path, record): + path = tmp_path / 'candidate.json' + path.write_text(json.dumps(record)) + proc = subprocess.run([os.environ['AI_READY_DB_PYTHON'], + str(local_aidb / 'skills/ai-ready-db/scripts/aidb_client.py'), 'archive-local', + '--record-json', str(path), '--namespace', 'bulk_base_pbe', '--origin', 'local_calculation', + '--sync-status', 'local_only', '--config', os.environ['AI_READY_DB_CONFIG']], capture_output=True, text=True) + assert proc.returncode == 0, proc.stderr + + +def query(tmp_path, request='请计算金刚石硅体相普通 PBE 弛豫和总能', **kwargs): + return run_task(tmp_path, request, [('load_skill', {'skill_name': 'vasp-pymatgen'}), + ('submit_bohr_batchjob', {'name': 'Si'})], **kwargs) + + +def test_explicit_recalculation_queries_first_and_preserves_authorization(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, '请重新计算并提交金刚石硅体相 PBE 弛豫和总能') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'recalculate' + assert Path(responses[0]['aidb_preflight']['report_path']).is_file() + assert prepared == ['remote submission'] + responses, prepared = query(tmp_path, '只准备金刚石硅体相 PBE 弛豫重算输入，不提交') + assert prepared == [] + + +def test_recent_archive_selected_without_claiming_accuracy(local_aidb, tmp_path, monkeypatch): + import copy + first = seed_relaxation(tmp_path, monkeypatch) + second = copy.deepcopy(first) + second.pop('record_id') + second['source_id'] = 'later-job' + second['source_metadata']['job_id'] = 'later-job' + second['source_metadata']['archived_at'] = '2026-10-08T00:00:00+00:00' + second['energy'] = -10.3 + public_archive(local_aidb, tmp_path, second) + responses, prepared = query(tmp_path) + reuse = responses[0]['aidb_preflight']['reuse'] + assert reuse['status'] == 'reused' + assert reuse['total_energy']['value'] == -10.3 + assert '不代表精度最高' in reuse['reason'] + assert prepared == [] + + +def test_missing_archive_order_requires_clarification(local_aidb, tmp_path, monkeypatch): + import copy + first = seed_relaxation(tmp_path, monkeypatch) + second = copy.deepcopy(first) + second.pop('record_id') + second['source_id'] = 'undated-job' + second['source_metadata']['job_id'] = 'undated-job' + second['source_metadata'].pop('archived_at') + public_archive(local_aidb, tmp_path, second) + responses, prepared = query(tmp_path) + assert responses[0]['aidb_preflight']['reuse']['status'] == 'needs_clarification' + assert prepared == [] + + +def test_different_conditions_require_scientific_choice(local_aidb, tmp_path, monkeypatch): + import copy + first = seed_relaxation(tmp_path, monkeypatch) + second = copy.deepcopy(first) + second.pop('record_id') + second['source_id'] = 'other-cutoff' + second['source_metadata']['job_id'] = 'other-cutoff' + second['source_metadata']['conditions']['incar']['ENCUT'] = 600 + second['calculation_parameters']['ENCUT'] = 600 + public_archive(local_aidb, tmp_path, second) + responses, prepared = query(tmp_path) + assert responses[0]['aidb_preflight']['reuse']['status'] == 'needs_clarification' + assert prepared == [] + responses, prepared = query(tmp_path, '计算金刚石硅体相 PBE ENCUT=520 弛豫和总能') + assert responses[0]['aidb_preflight']['reuse']['record_ids'] == [first['record_id']] + assert prepared == [] + + +def test_linked_static_and_relaxation_keep_distinct_energies(local_aidb, tmp_path, monkeypatch): + import copy + parent = seed_relaxation(tmp_path, monkeypatch) + child = copy.deepcopy(parent) + child.pop('record_id') + child['source_id'] = 'static-job' + child['calculation_type'] = 'static' + child['energy'] = -10.2 + meta = child['source_metadata'] + meta['job_id'] = 'static-job' + meta['parent_record_id'] = parent['record_id'] + meta['relaxation_source'] = {'job_id': parent['source_id']} + meta['completion']['ionic_converged'] = None + meta['conditions']['incar']['NSW'] = 0 + meta['conditions']['incar']['IBRION'] = -1 + child['calculation_parameters'] = meta['conditions']['incar'] + public_archive(local_aidb, tmp_path, child) + responses, prepared = query(tmp_path) + reuse = responses[0]['aidb_preflight']['reuse'] + assert reuse['status'] == 'reused' and len(reuse['record_ids']) == 2 + assert [r['total_energy']['value'] for r in reuse['results']] == [-10.1, -10.2] + assert prepared == [] + meta['parent_record_id'] = 'unrelated' + public_archive(local_aidb, tmp_path, child) + responses, prepared = query(tmp_path) + assert responses[0]['aidb_preflight']['reuse']['record_ids'] == [parent['record_id']] + assert prepared == [] + + +def test_target_change_does_not_reuse_previous_selection(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = run_task(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], followup='改用 HSE06 计算金刚石硅体相总能') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' + assert responses[1]['aidb_preflight']['status'] == 'needs_clarification' + assert not responses[1]['aidb_preflight'].get('reuse') + assert prepared == [] + + +def test_negative_recompute_intent_still_reuses(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能，不重新计算') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' + assert prepared == [] +import pytest + +@pytest.mark.parametrize('defect', ['electronic', 'ionic', 'unit', 'energy', 'hybrid', 'potential', 'kpoints', 'phase', 'failed']) +def test_incomplete_or_incompatible_records_remain_candidates(local_aidb, tmp_path, monkeypatch, defect): + saved = seed_relaxation(tmp_path, monkeypatch) + meta = saved['source_metadata'] + if defect == 'electronic': + meta['completion']['electronic_converged'] = False + elif defect == 'ionic': + meta['completion']['ionic_converged'] = None + elif defect == 'unit': + meta['energy_unit'] = 'Ha' + elif defect == 'energy': + saved['energy'] = None + elif defect == 'hybrid': + meta['conditions']['parameters']['LHFCALC'] = True + elif defect == 'potential': + meta['conditions'].pop('potcar_sha256') + elif defect == 'kpoints': + meta['conditions']['kpoints'] = None + meta['conditions']['actual_kpoints'] = None + elif defect == 'phase': + saved['structure']['sites'][1]['abc'] = [.3, .25, .25] + else: + meta['status'] = 'failed' + # A fresh isolated database lets the public contract store this malformed + # candidate without merging it with the already valid archived record. + config_path = Path(os.environ['AI_READY_DB_CONFIG']) + config = json.loads(config_path.read_text()) + config['ase_sqlite'] = {'path': str(tmp_path / 'malformed.db'), 'json_sidecar_path': str(tmp_path / 'malformed.json')} + config_path.write_text(json.dumps(config)) + public_archive(local_aidb, tmp_path, saved) + responses, prepared = query(tmp_path, '只准备金刚石硅体相 PBE 弛豫和总能输入，不提交') + feedback = responses[0]['aidb_preflight'] + assert feedback['status'] == 'found' + assert feedback['reuse']['status'] == 'unavailable' + assert feedback['reuse']['candidates'][0]['compatible'] is False + assert prepared == [] + + +@pytest.mark.parametrize('entry', ['thinking', 'step']) +def test_registered_entry_reuses_and_respects_parent_recalculation(local_aidb, tmp_path, monkeypatch, entry): + seed_relaxation(tmp_path, monkeypatch) + if entry == 'step': + request = json.dumps({'action': '准备硅弛豫与总能输入', 'prior_context': ''}) + state = {'goal': '计算金刚石硅体相 PBE 弛豫和总能'} + else: + request, state = '计算金刚石硅体相 PBE 弛豫和总能', {} + responses, prepared = query(tmp_path, request, entry=entry, state=state) + assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' + assert prepared == [] + if entry == 'step': + state = {'goal': '重新计算并提交金刚石硅体相 PBE 弛豫和总能'} + request = json.dumps({'action': '提交硅弛豫计算', 'prior_context': ''}) + responses, prepared = query(tmp_path, request, entry=entry, state=state) + assert responses[0]['aidb_preflight']['reuse']['status'] == 'recalculate' + assert prepared == ['remote submission'] + + +def test_missing_identity_conditions_and_query_error_never_submit(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, '请计算硅弛豫和总能') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'needs_clarification' + assert prepared == [] + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, prepared = query(tmp_path) + assert responses[0]['aidb_preflight']['status'] == 'failed' + assert prepared == [] + + +def test_changed_cutoff_does_not_keep_old_result(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = run_task(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能', + [('load_skill', {'skill_name': 'vasp-pymatgen'})], followup='改用 ENCUT=600 计算总能') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' + assert responses[1]['aidb_preflight']['reuse']['status'] == 'unavailable' + assert responses[0]['aidb_preflight']['report_path'] != responses[1]['aidb_preflight']['report_path'] + assert prepared == [] + +def test_explicit_static_goal_never_substitutes_relaxation_energy(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, '只准备金刚石硅体相 PBE 弛豫和静态总能输入，不提交') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'unavailable' + assert prepared == [] + + +def test_flash_entry_does_not_spawn_computation_when_reused(local_aidb, tmp_path, monkeypatch): + seed_relaxation(tmp_path, monkeypatch) + called = [] + def run_flash_step(action: str) -> dict: + called.append(action) + return {'status': 'executed'} + responses, prepared = run_task(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能', + [('run_flash_step', {'action': 'compute Si'})], extra_tools=(run_flash_step,)) + assert responses[0]['status'] == 'reused' + assert called == prepared == [] + + +@pytest.mark.parametrize('setting', ['EDIFF=0.000001', 'k网格=8x8x8']) +def test_requested_parameters_are_not_replaced_with_candidate_conditions(local_aidb, tmp_path, monkeypatch, setting): + seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, f'只准备金刚石硅体相 PBE 弛豫总能 {setting} 输入，不提交') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'unavailable' + assert prepared == [] From 19d37b899465b82d401568cd588eda66037712bf Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:14:35 +0800 Subject: [PATCH 14/22] Respect matching KPOINTS constraints during Si reuse --- src/matcreator/agents/aidb_skill_hook.py | 2 ++ tests/test_silicon_reuse.py | 46 ++++++++++++------------ 2 files changed, 25 insertions(+), 23 deletions(-) diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index f29fa00..aab987d 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -97,6 +97,8 @@ def _known_context(text, workspace): known['encut'] = float(cutoff.group(1)) constraints = {} for setting in re.finditer(r'\b([A-Z][A-Z0-9_]*)\s*=\s*([^\s,，;；]+)', text): + if setting.group(1) == 'KPOINTS': + continue # Mesh is a separate VASP input, not an INCAR tag. raw = setting.group(2) try: value = float(raw) diff --git a/tests/test_silicon_reuse.py b/tests/test_silicon_reuse.py index c94c101..6e98e99 100644 --- a/tests/test_silicon_reuse.py +++ b/tests/test_silicon_reuse.py @@ -1,9 +1,12 @@ """Task-level reuse tests through ADK and the public isolated local database.""" +import copy import json import os import subprocess from pathlib import Path +import pytest + from test_aidb_skill_hook import local_aidb, run_task from test_silicon_vasp_workflow import completed_job, silicon_outputs from matcreator.tools.silicon_archive import archive_silicon_vasp_result @@ -28,6 +31,14 @@ def test_compatible_success_reused_before_preparation_or_submission(local_aidb, assert feedback['reuse']['structure'] == saved['structure'] assert prepared == [] +def derive_candidate(record, source_id): + candidate = copy.deepcopy(record) + candidate.pop('record_id') + candidate['source_id'] = source_id + candidate['source_metadata']['job_id'] = source_id + return candidate + + def public_archive(local_aidb, tmp_path, record): path = tmp_path / 'candidate.json' path.write_text(json.dumps(record)) @@ -54,12 +65,8 @@ def test_explicit_recalculation_queries_first_and_preserves_authorization(local_ def test_recent_archive_selected_without_claiming_accuracy(local_aidb, tmp_path, monkeypatch): - import copy first = seed_relaxation(tmp_path, monkeypatch) - second = copy.deepcopy(first) - second.pop('record_id') - second['source_id'] = 'later-job' - second['source_metadata']['job_id'] = 'later-job' + second = derive_candidate(first, 'later-job') second['source_metadata']['archived_at'] = '2026-10-08T00:00:00+00:00' second['energy'] = -10.3 public_archive(local_aidb, tmp_path, second) @@ -72,12 +79,8 @@ def test_recent_archive_selected_without_claiming_accuracy(local_aidb, tmp_path, def test_missing_archive_order_requires_clarification(local_aidb, tmp_path, monkeypatch): - import copy first = seed_relaxation(tmp_path, monkeypatch) - second = copy.deepcopy(first) - second.pop('record_id') - second['source_id'] = 'undated-job' - second['source_metadata']['job_id'] = 'undated-job' + second = derive_candidate(first, 'undated-job') second['source_metadata'].pop('archived_at') public_archive(local_aidb, tmp_path, second) responses, prepared = query(tmp_path) @@ -86,12 +89,8 @@ def test_missing_archive_order_requires_clarification(local_aidb, tmp_path, monk def test_different_conditions_require_scientific_choice(local_aidb, tmp_path, monkeypatch): - import copy first = seed_relaxation(tmp_path, monkeypatch) - second = copy.deepcopy(first) - second.pop('record_id') - second['source_id'] = 'other-cutoff' - second['source_metadata']['job_id'] = 'other-cutoff' + second = derive_candidate(first, 'other-cutoff') second['source_metadata']['conditions']['incar']['ENCUT'] = 600 second['calculation_parameters']['ENCUT'] = 600 public_archive(local_aidb, tmp_path, second) @@ -104,15 +103,11 @@ def test_different_conditions_require_scientific_choice(local_aidb, tmp_path, mo def test_linked_static_and_relaxation_keep_distinct_energies(local_aidb, tmp_path, monkeypatch): - import copy parent = seed_relaxation(tmp_path, monkeypatch) - child = copy.deepcopy(parent) - child.pop('record_id') - child['source_id'] = 'static-job' + child = derive_candidate(parent, 'static-job') child['calculation_type'] = 'static' child['energy'] = -10.2 meta = child['source_metadata'] - meta['job_id'] = 'static-job' meta['parent_record_id'] = parent['record_id'] meta['relaxation_source'] = {'job_id': parent['source_id']} meta['completion']['ionic_converged'] = None @@ -147,8 +142,6 @@ def test_negative_recompute_intent_still_reuses(local_aidb, tmp_path, monkeypatc responses, prepared = query(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能，不重新计算') assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' assert prepared == [] -import pytest - @pytest.mark.parametrize('defect', ['electronic', 'ionic', 'unit', 'energy', 'hybrid', 'potential', 'kpoints', 'phase', 'failed']) def test_incomplete_or_incompatible_records_remain_candidates(local_aidb, tmp_path, monkeypatch, defect): saved = seed_relaxation(tmp_path, monkeypatch) @@ -250,4 +243,11 @@ def test_requested_parameters_are_not_replaced_with_candidate_conditions(local_a seed_relaxation(tmp_path, monkeypatch) responses, prepared = query(tmp_path, f'只准备金刚石硅体相 PBE 弛豫总能 {setting} 输入，不提交') assert responses[0]['aidb_preflight']['reuse']['status'] == 'unavailable' - assert prepared == [] + assert prepared == [] + +def test_matching_uppercase_kpoints_condition_reuses(local_aidb, tmp_path, monkeypatch): + saved = seed_relaxation(tmp_path, monkeypatch) + responses, prepared = query(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能 KPOINTS=4x4x4') + assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' + assert responses[0]['aidb_preflight']['reuse']['record_ids'] == [saved['record_id']] + assert prepared == [] From 100b17a5f3a35ed03ce8763b10567792f4e29a51 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:19:41 +0800 Subject: [PATCH 15/22] Keep all recognized k-mesh syntax separate from INCAR --- src/matcreator/agents/aidb_skill_hook.py | 4 ++-- tests/test_silicon_reuse.py | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index aab987d..907f6e7 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -95,9 +95,10 @@ def _known_context(text, workspace): cutoff = re.search(r'ENCUT\s*[=:]?\s*(\d+(?:\.\d+)?)', text, re.I) if cutoff: known['encut'] = float(cutoff.group(1)) + mesh = re.search(r'(?:k网格|k点|kpoints?|k[- ]?mesh)\s*[=:]?\s*(\d+)\s*[x×]\s*(\d+)\s*[x×]\s*(\d+)', text, re.I) constraints = {} for setting in re.finditer(r'\b([A-Z][A-Z0-9_]*)\s*=\s*([^\s,，;；]+)', text): - if setting.group(1) == 'KPOINTS': + if mesh and mesh.start() <= setting.start() < mesh.end(): continue # Mesh is a separate VASP input, not an INCAR tag. raw = setting.group(2) try: @@ -107,7 +108,6 @@ def _known_context(text, workspace): constraints[setting.group(1)] = value if constraints: known['incar_constraints'] = constraints - mesh = re.search(r'(?:k网格|k点|kpoints?|k[- ]?mesh)\s*[=:]?\s*(\d+)\s*[x×]\s*(\d+)\s*[x×]\s*(\d+)', text, re.I) if mesh: known['kpoint_mesh'] = [int(v) for v in mesh.groups()] # Read explicitly supplied existing structures; never create one at lookup. diff --git a/tests/test_silicon_reuse.py b/tests/test_silicon_reuse.py index 6e98e99..33c9c2b 100644 --- a/tests/test_silicon_reuse.py +++ b/tests/test_silicon_reuse.py @@ -245,9 +245,10 @@ def test_requested_parameters_are_not_replaced_with_candidate_conditions(local_a assert responses[0]['aidb_preflight']['reuse']['status'] == 'unavailable' assert prepared == [] -def test_matching_uppercase_kpoints_condition_reuses(local_aidb, tmp_path, monkeypatch): +@pytest.mark.parametrize('mesh_input', ['KPOINTS=4x4x4', 'KMESH=4x4x4', 'K-MESH=4x4x4']) +def test_matching_uppercase_kpoints_condition_reuses(local_aidb, tmp_path, monkeypatch, mesh_input): saved = seed_relaxation(tmp_path, monkeypatch) - responses, prepared = query(tmp_path, '计算金刚石硅体相 PBE 弛豫和总能 KPOINTS=4x4x4') + responses, prepared = query(tmp_path, f'计算金刚石硅体相 PBE 弛豫和总能 {mesh_input}') assert responses[0]['aidb_preflight']['reuse']['status'] == 'reused' assert responses[0]['aidb_preflight']['reuse']['record_ids'] == [saved['record_id']] assert prepared == [] From df6116dcb9886e85f287702812bf44be80bcfe10 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 22:24:14 +0800 Subject: [PATCH 16/22] Record T4 independent-session acceptance and final reviews --- docs/silicon_vasp_t4.md | 41 ++++++++++++++++++++++++++-- docs/silicon_vasp_t4_acceptance.json | 22 +++++++++++++++ 2 files changed, 60 insertions(+), 3 deletions(-) create mode 100644 docs/silicon_vasp_t4_acceptance.json diff --git a/docs/silicon_vasp_t4.md b/docs/silicon_vasp_t4.md index 44b1071..7f6a159 100644 --- a/docs/silicon_vasp_t4.md +++ b/docs/silicon_vasp_t4.md @@ -79,7 +79,7 @@ fixtures below are separate evidence. ## Regression -25 new task-level tests pass using ADK, both registered agents, controlled +28 new task-level tests pass using ADK, both registered agents, controlled remote tools and real isolated aidb databases/public bridges. They cover reuse without preparation/submission, explicit/negated recalculation, parent intent, recent archive order, ambiguous order, different conditions, linked/unlinked @@ -87,5 +87,40 @@ stages, explicit static targets, missing/invalid scientific evidence, query failure, parameter/target changes and Flash entry. The original archive suite retains failure, missing output, unconverged, duplicate and uncertain-write recovery coverage; archive errors preserve outputs rather than resubmitting. -Related T1–T4 run: 102 passed before the last four additional regression cases. -Changed modules compile. Full-suite results and independent review follow. +Related T1–T4 run: 102 passed before the last seven additional regression cases. +Changed modules compile. Full-suite results and independent review follow. + +Full suite: **823 passed, 39 failed, 2 collection errors**. Failure/error names +exactly match the T3 baseline; no new failures. Machine-readable verification: +`.workspace/silicon-t4/acceptance.json` and `verify-acceptance.py` compare the +current public query, tool feedback and final answer against the verified T3 +records, prove the request contains no IDs, and find zero jobs owned by the new +session. Full log: `.workspace/silicon-t4/full-final-alias.log`. No typechecking command is +configured in this repository; changed modules pass compileall and diff checks. + + +Final related preflight/reuse regression: **73 passed**, including all 28 reuse +cases. KPOINTS, KMESH and K-MESH matching syntax is covered by the same task +regression. The uppercase parser collision was reproduced, then fixed by +excluding the already recognized mesh span from INCAR constraints. + +## Standards + +Final review of `7ea0698...100b17a`: zero documented-standard violations and +zero unresolved judgment-call suggestions. The initial duplicated candidate +setup suggestion was resolved by `derive_candidate`; mesh aliases share one +recognizer and no per-alias special cases. Source/test layout follows AGENTS. + +## Spec + +Final review against #1 and the live #5 body: zero remaining actionable +findings. The initial P2 matching-KPOINTS rejection was fixed and checked for +all three supported spellings. All eight #5 acceptance criteria are supported +by the public-interface implementation, isolated regression and real independent +session evidence. The extra acceptance guard denied no calls and did not +manufacture reuse. Development A remains active; B has not been delivered here. + +Review total: Standards 0 unresolved, Spec 0 unresolved; neither axis has an +outstanding issue. Current evidence summary is committed as +[silicon_vasp_t4_acceptance.json](silicon_vasp_t4_acceptance.json). +No issue comment or closure was performed in this implementation session. diff --git a/docs/silicon_vasp_t4_acceptance.json b/docs/silicon_vasp_t4_acceptance.json new file mode 100644 index 0000000..8263960 --- /dev/null +++ b/docs/silicon_vasp_t4_acceptance.json @@ -0,0 +1,22 @@ +{ + "phase": "A", + "t3_prerequisite": "verified", + "independent_session": "silicon-t4-natural-final-20261007", + "natural_request_has_no_record_ids": true, + "current_public_query": "/home/shik-mechrevo-wsl/projects/MatCreator/.workspace/silicon-t4-natural-final/.aidb/runs/5c62b6a6ec9d22b746615a1a/1791381714-0e5724e9/preflight-report.json", + "public_exported_reuse_ids": [ + "694772dca0777de504b25ba70366fc3d", + "1a1e16e62710d4bec6721ad350ee1f05" + ], + "tool_feedback_and_final_response_verified": true, + "new_real_jobs_in_session": 0, + "cloud_database_upload": false, + "new_task_tests_passed": 28, + "full_suite": { + "passed": 823, + "failed": 39, + "collection_errors": 2, + "new_failures": [], + "failure_names_equal_t3_baseline": true + } +} \ No newline at end of file From 587becd411f3601c1a8876b3b7a68695e7f681fd Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 23:08:53 +0800 Subject: [PATCH 17/22] Implement final B lookup-failure consent (Ai-ready_Database #6) --- docs/aidb_skill_hook.md | 29 ++- docs/silicon_vasp_t5.md | 76 ++++++++ src/matcreator/agents/aidb_skill_hook.py | 117 ++++++++++-- .../execution_agent/step_executor_runner.py | 5 + src/matcreator/skills/vasp-pymatgen/SKILL.md | 16 +- tests/test_aidb_skill_hook.py | 16 +- tests/test_silicon_lookup_consent.py | 174 ++++++++++++++++++ 7 files changed, 406 insertions(+), 27 deletions(-) create mode 100644 docs/silicon_vasp_t5.md create mode 100644 tests/test_silicon_lookup_consent.py diff --git a/docs/aidb_skill_hook.md b/docs/aidb_skill_hook.md index e31b857..e681d39 100644 --- a/docs/aidb_skill_hook.md +++ b/docs/aidb_skill_hook.md @@ -1,7 +1,9 @@ -# aidb 计算前预检（issue #2，开发期 A） +# aidb 计算前预检（最终 B） 主规格为 [Ai-ready_Database #1](https://github.com/Zikkying/Ai-ready_Database/issues/1)， -本次只实现 [T1 / #2](https://github.com/Zikkying/Ai-ready_Database/issues/2)。 +初始实现为 [T1 / #2](https://github.com/Zikkying/Ai-ready_Database/issues/2)， +当前默认行为已按 [T5 / #6](https://github.com/Zikkying/Ai-ready_Database/issues/6) +切换为最终 B；[切换验收记录](silicon_vasp_t5.md)。 主 thinking agent 和内部 step executor 共用工具回调。加载 `vasp-pymatgen`、 硅任务的 `atomic-structure`，以及受支持任务直接调用准备或远程工具时， @@ -33,16 +35,22 @@ min_records=1，无相关材料扩展。每个 CLI 调用等待最多 20 秒， Skill 正文正常加载；after_tool_callback 将 `aidb_preflight` 加入实际工具响应， 模型能够读取并向用户反馈；直接调用准备工具时其响应也携带同样证据。反馈同时写入 session_log 和运行日志。 成功响应读取完整 `preflight-report.json`，保留完整候选 ID、报告、审计位置。 -`found` 是候选命中，不能当成已经核验可复用；本票不实现候选复用/归档。 +`found` 是候选命中，适用性与复用由独立的 `reuse` 回执给出。 `not_found` 是成功查询空候选；`failed` 是真实配置、连接、超时或协议失败。 任务上下文与原始 adapter 交换记录保存在工作区 `.aidb/requests/`，完整查询 报告和 audit 保存在 adapter 的 `.aidb/runs/` 下。失败诊断也保留位置。 -开发期 A 中，失败或材料澄清状态阻止新结构/输入准备及 tracked remote-job -提交/上传/执行。只准备的请求即使查询成功也不授权提交。状态/结果读取工具 -仍可使用。同任务同次 invocation 的重复入口共用回执；新轮次重新查库， -条件变化不会复用旧回执。并发入口通过同调用锁共用查询；自然“继续/重算”沿用任务条件与原授权，新一轮仍重新查库，明确提交授权才能扩大原准备范围。此约束覆盖登记的工具入口，不承诺约束任意自由 -shell/Python 入口。后续 T5 才切换最终 B，不能把本票称为完整闭环验收。 +最终 B 中，查询失败先说明数据存在性未知与重复计算风险，再保持准备/提交 +暂停。用户可回复回执给出的 `确认继续 `，明确接受此次失败风险， +仅继续原授权范围。仅准备授权仍不允许提交。拒绝、普通“继续/yes”、未确认、 +旧会话或旧失败事件的确认保持暂停；提取错误与材料澄清不能绕过。 +确认绑定原任务、条件及此次失败，已引用结构文件变化也会使其失效。 +受支持 executor 只能由 runner 继承本轮已确认范围，不能用内部 action 伪造同意。 +查询回执始终为 `failed`，`loop_complete=false`；计算完成仍自动尝试本地归档， +分别报告科学结果和实际保存结果。状态/结果读取工具仍可使用。 +同任务同次 invocation 的重复入口共用回执；独立新请求与条件变化重新查库。 +该确认针对已有失败事件，所以确认轮次沿用该事件而不制造新的查询失败。 +此约束覆盖登记的工具入口，不承诺约束任意自由 shell/Python 入口。 ## 验证 @@ -68,5 +76,6 @@ A compatible selection prevents preparation and new submission, including the Flash executor entry. Explicit recalculation still queries first and respects preparation-only authorization. Same-condition duplicates use persistent archive times; ambiguity or meaningful condition differences require -clarification. Task changes trigger a new lookup. Development A and no cloud -database upload remain in force. This does not implement T5's final B behavior. +clarification. Task changes trigger a new lookup. T4's historical A evidence is +retained; the current default is [T5 final B](silicon_vasp_t5.md). No cloud +database upload is performed. diff --git a/docs/silicon_vasp_t5.md b/docs/silicon_vasp_t5.md new file mode 100644 index 0000000..0f17bbd --- /dev/null +++ b/docs/silicon_vasp_t5.md @@ -0,0 +1,76 @@ +# Silicon T5: final B after verified T4 + +Scope: [Ai-ready_Database #6](https://github.com/Zikkying/Ai-ready_Database/issues/6), +main specification [#1](https://github.com/Zikkying/Ai-ready_Database/issues/1). +The implementation lives in MatCreator. The default is now final B; no A +configuration switch remains. Historical T1–T4 evidence stays unchanged. + +## Prerequisite and switch + +Starting point: `df6116dcb9886e85f287702812bf44be80bcfe10`. +Before changing behavior, checked its `docs/silicon_vasp_t4.md` and committed +`docs/silicon_vasp_t4_acceptance.json` against the retained acceptance JSON. +Re-ran `.workspace/silicon-t4/verify-acceptance.py` and `verify-t3.py`. +Both passed: the independent natural-language session had no supplied record +IDs, found both real T3 records, and owned zero new jobs. Fresh public +query/export, reparsed raw VASP outputs and artifact hashes again verified: + +| Stage | Record ID | Job ID | Energy (eV/cell) | +| --- | --- | --- | ---: | +| Relaxation | `694772dca0777de504b25ba70366fc3d` | `56072cf15fe94e639d7b71f563ed320a` | -10.84124445 | +| Static | `1a1e16e62710d4bec6721ad350ee1f05` | `c266a691a9874ab9ad097bfd09dca9b6` | -10.84601452 | + +T4's two-axis review had zero unresolved findings; its 39 test failures and +two collection errors matched the pre-existing T3 baseline. Issue status alone +was not used to decide completion. No real calculation was resubmitted and +no cloud database was accessed for this implementation. + +## Current behavior + +Lookup failure explains that data existence is unknown and continuing may +duplicate computation. Preparation/submission pauses until the human responds +with the displayed `确认继续 ` (English `confirm continue ` +also works). The ordinary-language reply contains no internal query JSON. +The event ID makes consent unambiguous; plain “continue”, “yes”, negated replies +and internal executor text do not grant it. The session must contain the actual +warning response from an earlier invocation, and the saved task must still +match, including known conditions and the digest of an explicitly supplied file. +New targets, methods, cutoff, k mesh or failure events invalidate old consent. + +Confirmation resumes the original scope without querying again for the same +failed event. Preparation-only remains preparation-only. Registered Flash/Plan +step runners may delegate the current invocation's grant to a fresh isolated +executor session; changed step conditions revoke that grant. A copied previous +session receipt cannot grant consent in a new session. + +The lookup stays `failed`, `query_executed=false`, with failure ID and diagnostics; +the feedback and collected result retain `loop_complete=false`. Scientific +completion still triggers the existing local archive-and-requery/readback. +Archive failure is reported with retained artifacts; recovery can save and +compare the local record without resubmission. Even successful local saving +after bypass is not a successful complete lookup/compute/archive loop. +Material clarification and context extraction errors remain blocking. + +## Verification + +The first red task test failed because A lacked a confirmation event. After +the switch, 23 task cases passed using ADK, real isolated aidb configuration, +SQLite and public bridge calls, and controlled calculation tools. They cover +confirmation, refusal/absence, target/condition/event/session changes, original +preparation-only scope, internal text, both agent entries, actual Flash-to-step +delegation, archive failure and recovered local readback. The controlled +provider's synthetic VASP outputs are orchestration evidence only. The real +calculation/reuse prerequisite remains the separate T3/T4 evidence above. + +Commands: + +```bash +.venv/bin/python -m pytest tests/test_silicon_lookup_consent.py -q +.venv/bin/python -m pytest tests/ -q --continue-on-collection-errors +``` + +Logs are retained in `.workspace/silicon-t5-{red,current,related,full}.log`. +The full-suite comparison and final independent review are recorded below +after completion. There is no configured typechecking command; changed modules +are checked with compileall. Diff checks cover only this change, preserving +the user's unrelated AGENTS.md and untracked files. diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index 907f6e7..4b4f7bc 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -1,4 +1,4 @@ -"""Development-stage material preflight through the public aidb Skill adapter. +"""Material preflight with final B consent through the public aidb Skill adapter. Candidate discovery and fixed Si/PBE reuse use public exports. Lookup never authorizes a new remote calculation. @@ -28,8 +28,11 @@ claim a selected/static record is more accurate or highest precision. Current public readback is authoritative; old memory/logs are not this turn's lookup. On needs_clarification explain scientific differences or missing evidence; -on failed pause new computation (development A). Explicit recalculation still -requires lookup and stays within the user's existing authorization. +On failed explain that existing data is unknown and computation may duplicate +existing work. Pause until the human replies with the exact confirmation shown +in this failure's message. Never generate that confirmation as an executor +action. Confirmation allows only the original scope and never marks lookup or +the full loop successful. Explicit recalculation still requires lookup. """ logger = logging.getLogger(__name__) @@ -42,6 +45,70 @@ 'start_remote_job_command', 'run_remote_job_command', 'run_sub_agent', 'run_flash_step'}) _REMOTE_TOOLS = frozenset({'submit_bohr_batchjob', 'submit_bohr_sandbox', 'upload_remote_job_input', 'start_remote_job_command', 'run_remote_job_command'}) +_CONFIRM_RE = re.compile(r'(?:确认继续|confirm continue)\s+([0-9a-f]{32})[。.!]?\s*', re.I) + + +def _failure_warning(feedback, tool_context, *, can_confirm): + failure_id = uuid.uuid4().hex + feedback.update(failure_id=failure_id, loop_complete=False, + bypass={'status': 'pending', 'can_confirm': can_confirm}, + message='本地查库失败，无法判断已有数据是否存在，继续可能重复计算；新增计算保持暂停。') + if can_confirm: + feedback['message'] += f'若明确接受风险并继续原授权范围，请回复「确认继续 {failure_id}」。拒绝或未确认保持暂停；仅准备授权不允许提交。' + else: + feedback['message'] += '任务信息提取失败，请先修复或澄清；不能绕过此错误。' + feedback['failure_session_id'] = tool_context._invocation_context.session.id + + +def _confirm_failure(text, task, saved, tool_context): + """Consent comes only from a later human turn after the exact warning.""" + feedback = saved.get('feedback') or {} + match = _CONFIRM_RE.fullmatch(text.partition('\n[AIDB_PRIOR_CONTEXT]\n')[0].strip()) + if (not match or tool_context.state['temp:aidb_internal_step'] + or feedback.get('status') != 'failed' + or not feedback.get('bypass', {}).get('can_confirm') + or match[1] != feedback.get('failure_id') + or feedback.get('failure_session_id') != tool_context._invocation_context.session.id + or task != feedback.get('task_context')): + return None + # State alone, including copied state from another session, is not evidence + # that this human has received the warning for this event. + events = tool_context._invocation_context.session.events + if not any(event.invocation_id != tool_context.invocation_id and event.content + and any(part.function_response and + (part.function_response.response.get('aidb_preflight') or {}).get('failure_id') == match[1] + for part in (event.content.parts or [])) for event in events): + return None + return {**feedback, 'bypass': {**feedback['bypass'], 'status': 'confirmed'}, + 'message': '已明确确认此次查询失败及重复计算风险；仅继续原授权范围。查询仍失败，不代表未命中、预检成功或完整闭环通过；计算后仍尝试本地归档并如实报告。'} + + +def inherit_aidb_confirmation(state, tool_context, child_session_id): + """The executor runner delegates this turn's grant to one fresh child.""" + saved = tool_context.state.get('aidb_preflight') or {} + feedback = saved.get('feedback') or {} + state.pop('aidb_parent_bypass', None) + if (saved.get('invocation_id') == tool_context.invocation_id + and feedback.get('bypass', {}).get('status') == 'confirmed'): + state['aidb_parent_bypass'] = {'child_session_id': child_session_id, + 'feedback': feedback, 'request': saved['request']} + + +def _delegated_failure(text, workspace, tool_context): + grant = tool_context.state.get('aidb_parent_bypass') or {} + if (not tool_context.state['temp:aidb_internal_step'] + or grant.get('child_session_id') != tool_context._invocation_context.session.id): + return None + feedback = grant['feedback'] + parent = feedback['task_context'] + task = _task_context(text, workspace, parent) + if (not task or task['known'] != parent['known'] + or not set(task['targets']).issubset(parent['targets']) + or (task['require_static'] and 'total_energy' not in parent['targets'])): + tool_context.state['aidb_parent_bypass'] = None + return None + task['prepare_only'] = task['prepare_only'] or parent['prepare_only'] + return {**feedback, 'task_context': task} def _request(tool_context): @@ -280,10 +347,15 @@ async def before_aidb_skill_load(tool, args, tool_context): referential = _CONTINUATION_RE.search(text.partition('\n[AIDB_PRIOR_CONTEXT]\n')[0]) if name != 'load_skill' or skill_name == 'atomic-structure': active = (tool_context.state.get('aidb_preflight') or {}).get('invocation_id') == tool_context.invocation_id - if not active and not (referential and saved_task) and not re.search(r'vasp|硅|\bsilicon\b|(?` and pause preparation/submission until the human replies +with it. Continue only the original scope: preparation-only remains preparation. +The confirmation belongs to this session, task/conditions and failure event; +a new failure or changed task requires a fresh warning and confirmation. +Executor text is not human consent. Missing identity/model must be clarified; +retrieval defaults are not calculation authorization. A confirmed bypass keeps +lookup `failed` and the full loop incomplete, even if local archive later succeeds. Use the existing ASE diamond structure recipe and `MPRelaxSet`, then tracked Batch Job submission. On completion call `collect_silicon_vasp_result(job_id, @@ -146,7 +153,8 @@ On archive failure retain original outputs and diagnostics. Retry with `archive_silicon_vasp_result` for the same collected job; it revalidates outputs and queries before writing, including after timeout. Never resubmit a completed calculation to repair storage. Do not request confirmation for local saving. -No cloud database upload is performed. Development A remains active. +No cloud database upload is performed. Final B is the default; see +`docs/silicon_vasp_t5.md` for the prerequisite and switch evidence. For **MLFF energy/force labeling**, use `prepare_label` as a standalone step — it replaces the relaxation → SCF chain with a single static calculation optimized for dataset generation (no charge density, no relaxation, KSPACING-based k-points). diff --git a/tests/test_aidb_skill_hook.py b/tests/test_aidb_skill_hook.py index cd70bb2..6b0d524 100644 --- a/tests/test_aidb_skill_hook.py +++ b/tests/test_aidb_skill_hook.py @@ -42,7 +42,7 @@ def local_aidb(tmp_path, monkeypatch): pass -def run_task(tmp_path, request, calls, state=None, entry=None, followup=None, parallel=False, extra_tools=()): +def run_task(tmp_path, request, calls, state=None, entry=None, followup=None, parallel=False, extra_tools=(), turns=(), final_state=None): class Model(BaseLlm): _index: int = PrivateAttr(default=0) async def generate_content_async(self, llm_request, stream=False): @@ -74,6 +74,7 @@ def submit_bohr_batchjob(name: str) -> dict: return {'status': 'submitted'} async def run(): + nonlocal calls service = InMemorySessionService() session = await service.create_session(app_name='preflight_test', user_id='test', state={'workspace_dir': str(tmp_path), **(state or {})}) @@ -105,6 +106,19 @@ async def run(): for part in (event.content.parts if event.content else []): if part.function_response: responses.append(part.function_response.response) + for message, turn_calls in turns: + calls = turn_calls + agent.model._index = 0 + if callable(message): + message = message(responses) + async for event in runner.run_async(user_id='test', session_id=session.id, + new_message=types.Content(role='user', parts=[types.Part(text=message)])): + for part in (event.content.parts if event.content else []): + if part.function_response: + responses.append(part.function_response.response) + if final_state is not None: + current = await service.get_session(app_name='preflight_test', user_id='test', session_id=session.id) + final_state.update(current.state) return responses return asyncio.run(run()), prepared diff --git a/tests/test_silicon_lookup_consent.py b/tests/test_silicon_lookup_consent.py new file mode 100644 index 0000000..7a80b63 --- /dev/null +++ b/tests/test_silicon_lookup_consent.py @@ -0,0 +1,174 @@ +"""Final B at the ADK task boundary, with real isolated public aidb access.""" +import json +from pathlib import Path + +import pytest + +from test_aidb_skill_hook import local_aidb, run_task + + +REQUEST = '计算金刚石单质硅体相普通 PBE VASP 弛豫和总能' +LOAD = ('load_skill', {'skill_name': 'vasp-pymatgen'}) +PREPARE = ('run_python', {'code': 'prepare'}) +SUBMIT = ('submit_bohr_batchjob', {'name': 'Si'}) + + +def confirm(responses): + return '确认继续 ' + responses[-1]['aidb_preflight']['failure_id'] + + +def test_confirmation_continues_original_scope_without_claiming_lookup_success(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, performed = run_task(tmp_path, REQUEST, [LOAD, PREPARE, SUBMIT], + turns=[(confirm, [LOAD, PREPARE, SUBMIT])]) + warning = responses[0]['aidb_preflight'] + assert '无法判断' in warning['message'] and '重复计算' in warning['message'] + assert performed == ['prepare', 'remote submission'] + continued = responses[-1]['aidb_preflight'] + assert continued['status'] == 'failed' and continued['query_executed'] is False + assert continued['failure_id'] == warning['failure_id'] + assert continued['bypass']['status'] == 'confirmed' + assert continued['loop_complete'] is False + + +@pytest.mark.parametrize('reply', ['拒绝', '不确认', '继续', 'yes', '解释一下风险', '我不确认继续']) +def test_rejection_or_no_explicit_confirmation_keeps_tools_paused(local_aidb, tmp_path, monkeypatch, reply): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, performed = run_task(tmp_path, REQUEST, [LOAD], turns=[(reply, [PREPARE, SUBMIT])]) + assert performed == [] + assert all(response['status'] == 'blocked' for response in responses[1:]) + + +def test_confirmation_preserves_preparation_only(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, performed = run_task(tmp_path, '仅准备金刚石硅体相 PBE VASP 弛豫输入，不提交计算', + [LOAD], turns=[(confirm, [PREPARE, SUBMIT])]) + assert performed == ['prepare'] + assert responses[-1]['status'] == 'blocked' + assert '仅授权准备' in responses[-1]['message'] + + +@pytest.mark.parametrize('change', ['改用金刚石碳体相 PBE 总能', '改用 HSE06 计算硅总能', + '继续，ENCUT=600', '继续，KPOINTS=6x6x6', '继续，只计算静态总能']) +def test_changed_target_or_conditions_invalidates_failure_confirmation(local_aidb, tmp_path, monkeypatch, change): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + token = {} + def remember(responses): + token['old'] = confirm(responses) + return change + responses, performed = run_task(tmp_path, REQUEST, [LOAD], + turns=[(remember, [LOAD, PREPARE]), (lambda _: token['old'], [PREPARE, SUBMIT])]) + assert performed == [] + assert responses[-1]['status'] == 'blocked' + + +def test_old_event_token_cannot_confirm_a_new_failure(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + token = {} + def remember(responses): + token['old'] = confirm(responses) + return REQUEST + responses, performed = run_task(tmp_path, REQUEST, [LOAD], + turns=[(remember, [LOAD]), (lambda _: token['old'], [PREPARE, SUBMIT])]) + assert responses[0]['aidb_preflight']['failure_id'] != responses[1]['aidb_preflight']['failure_id'] + assert performed == [] + + +def test_copied_session_state_does_not_authorize_old_consent(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + state = {} + responses, _ = run_task(tmp_path, REQUEST, [LOAD], final_state=state) + responses, performed = run_task(tmp_path, confirm(responses), [PREPARE, SUBMIT], state=state) + assert performed == [] + + +def test_executor_text_cannot_supply_human_consent(local_aidb, tmp_path, monkeypatch): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, performed = run_task(tmp_path, REQUEST, [LOAD], turns=[ + (lambda responses: json.dumps({'action': confirm(responses), 'prior_context': REQUEST}), [PREPARE, SUBMIT])]) + assert performed == [] + + +@pytest.mark.parametrize('entry', ['thinking', 'step']) +def test_final_b_is_registered_on_both_agent_entries(local_aidb, tmp_path, monkeypatch, entry): + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + responses, performed = run_task(tmp_path, REQUEST, [LOAD], entry=entry, + turns=[(confirm, [PREPARE, SUBMIT])]) + assert performed == ['prepare', 'remote submission'] + + +@pytest.mark.parametrize('recover', [False, True]) +def test_failed_lookup_bypass_still_attempts_local_archive(local_aidb, tmp_path, monkeypatch, recover): + from test_silicon_vasp_workflow import completed_job, silicon_outputs + from matcreator.tools.silicon_vasp import collect_silicon_vasp_result + _, _, job_id = completed_job(tmp_path, monkeypatch, silicon_outputs()) + config = str(tmp_path / 'config.json') + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + collect = ('collect_silicon_vasp_result', {'job_id': job_id, + 'destination_path': 'outputs', 'calculation_type': 'relaxation'}) + def reply(responses): + if recover: + monkeypatch.setenv('AI_READY_DB_CONFIG', config) + return confirm(responses) + responses, performed = run_task(tmp_path, REQUEST, [LOAD], state={'session_id': 'session'}, + turns=[(reply, [PREPARE, collect])], extra_tools=[collect_silicon_vasp_result]) + assert performed == ['prepare'] + assert responses[-1]['archived'] is recover + assert responses[-1]['local_archive']['status'] == ('verified' if recover else 'failed') + assert responses[-1]['loop_complete'] is False + if recover: + saved = json.loads(Path(responses[-1]['local_archive']['export_path']).read_text()) + assert saved['energy'] == -10.1 + assert saved['source_metadata']['sync_status'] == 'local_only' + else: + assert 'error' in responses[-1]['local_archive'] + assert (tmp_path / 'outputs/OUTCAR').is_file() + + +@pytest.mark.parametrize('prepare_only,changed', [(False, False), (True, False), (False, True)]) +def test_confirmed_parent_can_delegate_only_original_scope_to_real_executor( + local_aidb, tmp_path, monkeypatch, prepare_only, changed +): + from google.adk.models.base_llm import BaseLlm + from google.adk.models.llm_response import LlmResponse + from google.genai import types + from pydantic import PrivateAttr + from matcreator.agents.execution_agent import step_executor_runner, recovery + from matcreator.agents.execution_agent.step_executor import build_step_executor_agent, submit_step_result + from matcreator.agents.thinking_agent.agent import run_flash_step + from matcreator.agents import graph_logger + from matcreator.llm_cards import LLMCard + + monkeypatch.setenv('AI_READY_DB_CONFIG', str(tmp_path / 'missing.json')) + monkeypatch.setattr(recovery, 'ADK_DIR', tmp_path) + monkeypatch.setattr(graph_logger, 'ADK_DIR', tmp_path) + performed = [] + def run_python(code: str) -> dict: + performed.append('prepare') + return {'status': 'prepared'} + def submit_bohr_batchjob(name: str) -> dict: + performed.append('submission') + return {'status': 'submitted'} + child_calls = [PREPARE, SUBMIT, ('submit_step_result', {'status': 'success', + 'key_results': 'Controlled tools completed.', 'concise_summary': 'Controlled tools completed.'})] + class Model(BaseLlm): + _index: int = PrivateAttr(default=0) + async def generate_content_async(self, llm_request, stream=False): + if self._index < len(child_calls): + name, args = child_calls[self._index] + part = types.Part(function_call=types.FunctionCall(name=name, args=args)) + else: + part = types.Part(text='finished') + self._index += 1 + yield LlmResponse(content=types.Content(role='model', parts=[part])) + card = LLMCard(name='test', model='openai/test') + monkeypatch.setattr(step_executor_runner, 'select_executor_llm_card', lambda **_: card) + monkeypatch.setattr(step_executor_runner, 'build_step_executor_agent', lambda _: build_step_executor_agent(card).model_copy( + update={'model': Model(model='test'), 'tools': [run_python, submit_bohr_batchjob, submit_step_result], + 'instruction': '', 'input_schema': None, 'sub_agents': []})) + action = REQUEST + (' ENCUT=600' if changed else '') + request = '仅准备' + REQUEST if prepare_only else REQUEST + run_task(tmp_path, request, [LOAD], state={'session_id': 'controlled-child'}, + turns=[(confirm, [('run_flash_step', {'action': action, 'suggested_skills': ['vasp-pymatgen']})])], + extra_tools=[run_flash_step]) + assert performed == ([] if changed else ['prepare'] if prepare_only else ['prepare', 'submission']) From 4dc04d38a90027e347ccb6835a2215b7a2834b6a Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Wed, 7 Oct 2026 23:17:26 +0800 Subject: [PATCH 18/22] Invalidate stale consent after standalone task updates --- src/matcreator/agents/aidb_skill_hook.py | 13 ++++++- tests/test_aidb_skill_hook.py | 8 +++- tests/test_silicon_lookup_consent.py | 47 ++++++++++++++++++++++-- 3 files changed, 63 insertions(+), 5 deletions(-) diff --git a/src/matcreator/agents/aidb_skill_hook.py b/src/matcreator/agents/aidb_skill_hook.py index 4b4f7bc..abf04ba 100644 --- a/src/matcreator/agents/aidb_skill_hook.py +++ b/src/matcreator/agents/aidb_skill_hook.py @@ -58,6 +58,7 @@ def _failure_warning(feedback, tool_context, *, can_confirm): else: feedback['message'] += '任务信息提取失败，请先修复或澄清；不能绕过此错误。' feedback['failure_session_id'] = tool_context._invocation_context.session.id + feedback['failure_invocation_id'] = tool_context.invocation_id def _confirm_failure(text, task, saved, tool_context): @@ -74,6 +75,10 @@ def _confirm_failure(text, task, saved, tool_context): # State alone, including copied state from another session, is not evidence # that this human has received the warning for this event. events = tool_context._invocation_context.session.events + previous_user = next((event for event in reversed(events) if event.content + and event.content.role == 'user' and event.invocation_id != tool_context.invocation_id), None) + if not previous_user or previous_user.invocation_id != feedback.get('failure_invocation_id'): + return None if not any(event.invocation_id != tool_context.invocation_id and event.content and any(part.function_response and (part.function_response.response.get('aidb_preflight') or {}).get('failure_id') == match[1] @@ -351,6 +356,12 @@ async def before_aidb_skill_load(tool, args, tool_context): if not active and not pending_failure and not (referential and saved_task) and not re.search(r'vasp|硅|\bsilicon\b|(? Date: Wed, 7 Oct 2026 23:22:17 +0800 Subject: [PATCH 19/22] Record verified A-to-B switch and final dual-axis review --- docs/silicon_vasp_t5.md | 60 +++++++++++++++++++++++----- docs/silicon_vasp_t5_acceptance.json | 44 ++++++++++++++++++++ 2 files changed, 95 insertions(+), 9 deletions(-) create mode 100644 docs/silicon_vasp_t5_acceptance.json diff --git a/docs/silicon_vasp_t5.md b/docs/silicon_vasp_t5.md index 0f17bbd..61b1aca 100644 --- a/docs/silicon_vasp_t5.md +++ b/docs/silicon_vasp_t5.md @@ -33,9 +33,12 @@ with the displayed `确认继续 ` (English `confirm continue Date: Thu, 8 Oct 2026 00:42:50 +0800 Subject: [PATCH 20/22] Make silicon loop checks repeatable and fail on missing integration --- .github/workflows/test.yml | 42 ++++ CODING_STANDARDS.md | 10 + docs/aidb_skill_hook.md | 4 + docs/testing/known-failures.json | 336 +++++++++++++++++++++++++++++ docs/testing/silicon-local-loop.md | 60 ++++++ script/check_silicon_loop.ps1 | 10 + script/check_silicon_loop.py | 242 +++++++++++++++++++++ script/silicon_test_report.py | 26 +++ tests/test_aidb_skill_hook.py | 12 +- tests/test_silicon_checks.py | 104 +++++++++ 10 files changed, 842 insertions(+), 4 deletions(-) create mode 100644 CODING_STANDARDS.md create mode 100644 docs/testing/known-failures.json create mode 100644 docs/testing/silicon-local-loop.md create mode 100644 script/check_silicon_loop.ps1 create mode 100644 script/check_silicon_loop.py create mode 100644 script/silicon_test_report.py create mode 100644 tests/test_silicon_checks.py diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 902a485..6d92ebf 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -59,3 +59,45 @@ jobs: - name: Run frontend checks run: npm run check + + silicon-local-loop: + name: Silicon local loop (public aidb bridge) + runs-on: ubuntu-latest + timeout-minutes: 20 + permissions: + contents: read + steps: + - uses: actions/checkout@v4 + with: + path: MatCreator + - name: Checkout pinned aidb integration dependency + uses: actions/checkout@v4 + with: + repository: Zikkying/Ai-ready_Database + # Prepared in MatCreator/.workspace/aidb-ci; publish this dependency + # commit before expecting this job to run on GitHub. Never use latest. + ref: 87148777fc9995a6d2e3bd85571ee8e20ff53181 + path: Ai-ready_Database + persist-credentials: false + - uses: actions/setup-python@v5 + with: + python-version: '3.12' + - name: Install both public packages + run: | + python -m pip install --upgrade pip + python -m venv Ai-ready_Database/.venv + Ai-ready_Database/.venv/bin/python -m pip install -e ./Ai-ready_Database + python -m pip install -e ./MatCreator pytest + - name: Run isolated loop tests with zero skips + working-directory: MatCreator + env: + AI_READY_DB_ROOT: ${{ github.workspace }}/Ai-ready_Database + AI_READY_DB_PYTHON: ${{ github.workspace }}/Ai-ready_Database/.venv/bin/python + run: python script/check_silicon_loop.py tests --output .workspace/ci/silicon-check.json + - name: Retain failure and test artifacts + if: always() + uses: actions/upload-artifact@v4 + with: + name: silicon-local-loop-results + path: MatCreator/.workspace/ci/ + if-no-files-found: warn diff --git a/CODING_STANDARDS.md b/CODING_STANDARDS.md new file mode 100644 index 0000000..cbd3429 --- /dev/null +++ b/CODING_STANDARDS.md @@ -0,0 +1,10 @@ +# Scientific result review + +Review scientific feedback against the current structured tool receipt and retained calculation evidence. Automated receipt checks establish provenance and consistency; they do not establish the scientific accuracy of a narrative. + +- Check that the selected structures, calculation stages, methods, parameters, completion status and source jobs support the requested reuse. Explain material compatibility limits when they affect the answer. +- Keep relaxation and static results distinct. State energy units and basis (for example eV per cell), and do not infer comparative physical meaning solely from a numerical difference between stages. +- Assess accuracy and precision claims from convergence evidence, the model and method, and the intended use. Printed decimal places, a successful job, or a verified archive alone do not justify a claim such as “high precision.” +- Keep retrieval defaults distinct from user requirements. A successful lookup is not permission to submit a calculation, and historical development policy A is not the current final B policy. + +The repeatable checks and evidence navigation are documented in [the silicon local-loop guide](docs/testing/silicon-local-loop.md). diff --git a/docs/aidb_skill_hook.md b/docs/aidb_skill_hook.md index e681d39..65c0ab7 100644 --- a/docs/aidb_skill_hook.md +++ b/docs/aidb_skill_hook.md @@ -79,3 +79,7 @@ archive times; ambiguity or meaningful condition differences require clarification. Task changes trigger a new lookup. T4's historical A evidence is retained; the current default is [T5 final B](silicon_vasp_t5.md). No cloud database upload is performed. + +## 重复验证入口 + +隔离回归、全量失败基线和真实记录只读验收统一使用 [silicon local-loop checks](testing/silicon-local-loop.md)。科学结论由 [review 标准](../CODING_STANDARDS.md)核对；工具校验不代替科学判断。 diff --git a/docs/testing/known-failures.json b/docs/testing/known-failures.json new file mode 100644 index 0000000..5eea000 --- /dev/null +++ b/docs/testing/known-failures.json @@ -0,0 +1,336 @@ +{ + "recorded_on": "2026-10-08", + "source_commit": "5f88f95", + "source_log": ".workspace/silicon-t5-full-final.log", + "policy": "Exact identity comparison only; this is not a clean full-suite result or a resolved defect.", + "failures": [ + { + "outcome": "failed", + "nodeid": "tests/test_benchmark_config.py::test_benchmark_client_registers_and_persists_missing_development_token", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_disabled_skills.py::test_set_disabled_skill_names_updates_graph_nodes", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_evaluation_routes.py::test_toggle_skill_graph_node_updates_and_reports_graph_state", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_extractor.py::test_extractor_skips_unchanged_session_and_processes_only_delta", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_extractor.py::test_extractor_advances_cursor_for_empty_valid_response", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_extractor.py::test_extractor_migrates_legacy_extracted_session", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_progressive_retrieval.py::test_graph_disabled_skills_are_hidden_from_discovery_and_reads", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_routes_through_protected_reviewer_policy", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_kdg_progressive_retrieval.py::test_graph_agent_tool_scopes_memory_review_to_current_session", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_skill_graph_seed.py::test_seed_reuses_disabled_skill_node_and_merges_legacy_duplicate", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_skill_graph_seed.py::test_seed_removes_stale_managed_skill_and_uses_virtual_dependency", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_step_executor_runner.py::test_step_executor_retries_only_malformed_streamed_tool_arguments", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_structure_modeling.py::test_supercell_supports_general_integer_matrix", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_structure_modeling.py::test_molecule_can_be_created_or_appended", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_structure_modeling.py::test_crystal_is_created_through_matcraft_bulk_builder", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_structure_modeling.py::test_smiles_molecule_can_be_appended_at_requested_center", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_structure_modeling.py::test_matcraft_perturbation_is_seeded_and_changes_positions", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_frontend_tracks_requests_per_session", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_session_creation_does_not_start_knowledge_review", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_sse_request_uses_captured_session_context", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_completed_request_handoffs_live_turn_before_durable_refresh", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_roadmap_fits_after_the_popup_has_a_stable_layout", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_running_session_switch_discovers_and_reconnects_managed_run", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_stop_and_plan_refreshes_preserve_open_node_dialogs", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_bottom_attachment_and_node_toggle_use_one_scroll_policy", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_all_chat_disclosures_share_the_reading_position_controller", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_image_and_all_timeline_updates_preserve_node_disclosures", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_plain_text_blocks_keep_history_order_and_stable_identity", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_chat_messages_use_one_model_scheduler_and_stable_regions", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_step_cancellation_identifies_the_active_session_owner", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_remote_job_polling_is_scoped_to_the_active_session", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_remote_jobs_are_collapsed_and_keep_lifecycle_status_visible", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_session_switch_loads_page_scoped_graph_after_the_transcript_page", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_session_switch_renders_cached_snapshot_immediately", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_reselecting_the_active_session_does_not_rebuild_the_view", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_new_session_ids_are_not_limited_to_one_second_resolution", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_active_session_transitions_recompute_composer_state", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_startup_restores_only_an_accessible_session_owner_tuple", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "failed", + "nodeid": "tests/test_web_concurrent_sessions.py::test_startup_keeps_session_api_ready_while_maintenance_runs_in_background", + "reason": "Existing failure recorded at T5; root cause still requires diagnosis.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "error", + "nodeid": "tests/test_execution_grouping.py", + "reason": "Legacy agents.MatCreator import no longer matches the current src package.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + }, + { + "outcome": "error", + "nodeid": "tests/test_matcreator_phase_and_skills.py", + "reason": "Legacy agents.MatCreator import no longer matches the current src package.", + "owner": "unassigned", + "tracking_issue": null, + "status": "needs-triage" + } + ] +} diff --git a/docs/testing/silicon-local-loop.md b/docs/testing/silicon-local-loop.md new file mode 100644 index 0000000..52b8c1f --- /dev/null +++ b/docs/testing/silicon-local-loop.md @@ -0,0 +1,60 @@ +# Silicon local-loop checks + +This is the repeatable validation entry point for MatCreator's silicon local loop. The current runtime policy is [final B / T5](../silicon_vasp_t5.md). [T4](../silicon_vasp_t4.md) records historical acceptance under development policy A; these checks do not change either policy. + +## Isolated regression + +From the repository root, with both Python environments installed: + +```bash +.venv/bin/python script/check_silicon_loop.py tests --output .workspace/checks/loop.json +``` + +From the Windows PowerShell checkout on WSL: + +```powershell +./script/check_silicon_loop.ps1 tests --output .workspace/checks/loop.json +``` + +Use a new output filename for every run. The command refuses an existing pytest report, writes JSON counts and exact failure identities, a log and JUnit XML, and fails on skipped tests or any failure. Missing aidb dependencies fail rather than quietly skipping. `AI_READY_DB_ROOT` and `AI_READY_DB_PYTHON` select the public bridge checkout and its Python; defaults use the adjacent `Ai-ready_Database` checkout and `.venv/bin/python`. + +The `silicon-local-loop` job in [the workflow](../../.github/workflows/test.yml) pins the aidb commit, installs both packages, runs this same command against isolated test databases and retains its artifacts. The dependency must be available on GitHub before Actions can fetch it. The prepared version is `87148777fc9995a6d2e3bd85571ee8e20ff53181`, retained locally in `.workspace/aidb-ci`; it contains the public bridge and bounded preflight contract. A checkout of GitHub's older aidb main alone lacks that bridge. Publication is tracked separately from local validation. + +## Full-suite failure baseline + +```bash +.venv/bin/python script/check_silicon_loop.py tests --full \ + --baseline docs/testing/known-failures.json --output .workspace/checks/full.json +``` + +[known-failures.json](known-failures.json) records the exact failed test IDs and collection errors observed at T5's verified commit `5f88f95`. Comparison uses identity and outcome, not an old test count. New failures, skips, interrupted runs and inconsistent reports fail. Resolved baseline entries appear in the report. A run accepted against this baseline still reports `all_tests_passed: false` while failures remain; the isolated CI job never uses this baseline. + +Each baseline entry carries reason, owner, tracking issue and status. Existing failures remain `needs-triage`, with owner `unassigned` and tracking issue `null` until someone takes responsibility. Assign the owner and link a diagnosis issue during triage; remove an entry only after its test passes. Do not add new entries just to make a check green. The two collection errors currently refer to obsolete `agents.MatCreator` imports. Fixing the existing unrelated failures is separate work. + +## Read-only real reuse audit + +```bash +.venv/bin/python script/check_silicon_loop.py verify-reuse \ + --workspace .workspace/silicon-t4-natural-final \ + --source-report .workspace/silicon-t2/silicon-t2-results/silicon-t3-final-report.json \ + --output .workspace/checks/reuse.json +``` + +Supply `--config` if the retained real records live in a nondefault aidb configuration. Both real source files and the existing tracked-job store must still be present. This command audits the retained independent MatCreator session; it does not start a new LLM session. It derives identities from the session's tool feedback and original request, performs fresh public query/export and reuse selection, compares the complete selected receipt, rereads raw VASP evidence and verifies artifact hashes and archive receipts. No record ID is supplied on the command line. + +The audit writes reports, public exports and the actual final response into a unique subdirectory beside `--output`. It neither submits computations nor archives records nor uploads to a cloud database. Its job-store check verifies that the retained acceptance session owns zero tracked jobs; it is not a claim about unrelated sessions or all provider history. + +Read the saved final response with [the scientific review standard](../../CODING_STANDARDS.md). The report deliberately marks `scientific_claim_review: required`; matching hashes and energy values cannot decide whether a prose claim of scientific accuracy is justified. + +For the evidence chain, start with [T3 real archive verification](../silicon_vasp_t3.md), then [T4 independent discovery and reuse](../silicon_vasp_t4.md), and [T5 policy switch](../silicon_vasp_t5.md). The runtime interface and configuration remain documented in [aidb_skill_hook.md](../aidb_skill_hook.md). +## Implementation verification, 2026-10-08 + +Validated against a clean checkout of the pinned dependency, rather than its developer working tree: + +- Isolated loop: 150 passed, no failures, errors or skips (`.workspace/retro-checks/loop-v4.json`). +- Full suite: 864 passed, 39 failed, two collection errors, no skips. Exact failures match the recorded baseline; the suite is still not clean (`full.json`). +- Public bridge and normalization identity contract: 15 passed in the prepared aidb checkout. The pin includes bounded preflight support and the missing Hubbard-U identity argument; no unrelated developer changes were copied. +- Fresh readback of both retained real records passed (`reuse-final.json`), with zero tracked jobs in the independent session. A changed structure was rejected; a missing store was rejected without creating it. +- PowerShell wrapper, workflow YAML loading, Python compilation and authored-file whitespace checks passed. + +Reports are local under `.workspace/retro-checks/`; original T3/T4 evidence is retained. GitHub Actions has not run this new job: the prepared dependency commit must first be published. Local validation does not establish remote CI success. diff --git a/script/check_silicon_loop.ps1 b/script/check_silicon_loop.ps1 new file mode 100644 index 0000000..e052bda --- /dev/null +++ b/script/check_silicon_loop.ps1 @@ -0,0 +1,10 @@ +param([Parameter(ValueFromRemainingArguments=$true)][string[]]$CheckArgs) +$ErrorActionPreference = 'Stop' +$repoPath = (Resolve-Path (Join-Path $PSScriptRoot '..')).ProviderPath +if ($repoPath -notmatch '^\\\\wsl(?:\$|\.localhost)\\([^\\]+)\\(.+)$') { + throw 'Run this wrapper from a WSL checkout; on Linux use python script/check_silicon_loop.py.' +} +$distribution = $Matches[1] +$linuxPath = '/' + $Matches[2].Replace('\', '/') +& wsl.exe -d $distribution --cd $linuxPath --exec ./.venv/bin/python script/check_silicon_loop.py @CheckArgs +exit $LASTEXITCODE diff --git a/script/check_silicon_loop.py b/script/check_silicon_loop.py new file mode 100644 index 0000000..57d78a7 --- /dev/null +++ b/script/check_silicon_loop.py @@ -0,0 +1,242 @@ +#!/usr/bin/env python3 +"""Repeatable local-loop regression and read-only persistent reuse acceptance.""" +from __future__ import annotations +import argparse +import hashlib +import json +import os +from pathlib import Path +import subprocess +import sys +import uuid + +REPO = Path(__file__).resolve().parents[1] +LOOP_TESTS = ['tests/test_aidb_skill_hook.py', 'tests/test_silicon_vasp_workflow.py', + 'tests/test_silicon_archive.py', 'tests/test_silicon_reuse.py', + 'tests/test_silicon_lookup_consent.py', 'tests/test_silicon_checks.py'] + + +class CheckError(ValueError): + pass + + +def read_json(path): + return json.loads(Path(path).read_text(encoding='utf-8-sig')) + + +def require(condition, message): + if not condition: + raise CheckError(message) + + +def digest(path): + with Path(path).open('rb') as stream: + return hashlib.file_digest(stream, 'sha256').hexdigest() + + +def emit(path, payload): + path = Path(path).resolve() + path.parent.mkdir(parents=True, exist_ok=True) + temporary = path.with_name(path.name + '.' + uuid.uuid4().hex + '.tmp') + temporary.write_text(json.dumps(payload, indent=2, ensure_ascii=False), encoding='utf-8') + temporary.replace(path) + print(json.dumps({'ok': payload.get('ok'), 'report_path': str(path), + 'error': payload.get('error')}, ensure_ascii=False)) + + +def check_results(report, baseline=None): + require(report['exit_code'] in (0, 1), f"pytest did not complete: exit {report['exit_code']}") + counts = report['counts'] + require(all(type(counts[k]) is int and counts[k] >= 0 for k in ('passed', 'failed', 'error', 'skipped')), 'invalid test counts') + require((report['exit_code'] == 0) == (counts['failed'] + counts['error'] == 0), 'pytest exit code contradicts failures') + require(counts['passed'] > 0, 'no tests passed') + require(counts['skipped'] == 0, 'unexpected skipped tests') + actual = {(r['outcome'], r['nodeid']) for r in report['failures']} + expected = {(r['outcome'], r['nodeid']) for r in (baseline or {}).get('failures', [])} + new = sorted(actual - expected) + require(not new, f'new failures: {new}') + require(counts['failed'] + counts['error'] == len(report['failures']), 'failure tally inconsistent') + return {'counts': counts, 'new_failures': new, 'resolved_baseline_failures': sorted(expected - actual), + 'baseline_used': baseline is not None, + 'all_tests_passed': counts['failed'] + counts['error'] == 0} + + +def runtime(args): + root = Path(args.aidb_root or os.environ.get('AI_READY_DB_ROOT') or REPO.parent / 'Ai-ready_Database').expanduser().resolve() + python = Path(args.aidb_python or os.environ.get('AI_READY_DB_PYTHON') or root / '.venv/bin/python').expanduser().absolute() + scripts = root / 'skills/ai-ready-db/scripts' + require(python.is_file(), f'aidb Python missing: {python}') + require(all((scripts / name).is_file() for name in ('aidb_loop_adapter.py', 'aidb_client.py', 'aidb_server.py')), + f'public bridge missing: {scripts}') + env = {**os.environ, 'AI_READY_DB_ROOT': str(root), 'AI_READY_DB_PYTHON': str(python)} + help_result = subprocess.run([str(python), str(scripts / 'aidb_loop_adapter.py'), 'preflight', '--help'], + capture_output=True, text=True, env=env, timeout=20) + require(help_result.returncode == 0 and '--command-timeout' in help_result.stdout, + 'aidb adapter lacks bounded preflight; use the pinned integration version') + return root, python, env + + +def run_tests(args): + _, _, env = runtime(args) + output = args.output.resolve() + output.parent.mkdir(parents=True, exist_ok=True) + report = output.with_suffix('.pytest.json') + require(not report.exists(), f'refuse stale pytest report: {report}; choose a new output path') + env.update(MATCREATOR_REQUIRE_AIDB='1', MATCREATOR_TEST_REPORT=str(report), + PYTHONPATH=os.pathsep.join([str(REPO), str(REPO / 'src'), env.get('PYTHONPATH', '')])) + command = [sys.executable, '-m', 'pytest', *( ['tests/'] if args.full else LOOP_TESTS), + '-q', '--continue-on-collection-errors', '-p', 'script.silicon_test_report', + '--junitxml', str(output.with_suffix('.xml'))] + with output.with_suffix('.log').open('w', encoding='utf-8') as log: + process = subprocess.run(command, cwd=REPO, env=env, stdout=log, stderr=subprocess.STDOUT) + require(report.exists(), f'pytest produced no report; inspect {output.with_suffix(".log")}') + results = read_json(report) + require(results['exit_code'] == process.returncode, 'pytest exit code mismatch') + baseline = read_json(args.baseline) if args.baseline else None + return {'ok': True, 'scope': 'full' if args.full else 'silicon-local-loop', + **check_results(results, baseline), 'pytest_report': str(report), + 'junit': str(output.with_suffix('.xml')), 'log': str(output.with_suffix('.log'))} + + +def verify_reuse(args): + root, python, env = runtime(args) + # The bridge may initialize a missing store even during local-status. + # Validate configuration paths before starting a supposedly read-only audit. + config_path = Path(args.config or env.get('AI_READY_DB_CONFIG') or root / 'configs/base_database.yaml').resolve() + import yaml + config = yaml.safe_load(config_path.read_text(encoding='utf-8-sig')) + store = Path(config['ase_sqlite']['path']).expanduser() + if not store.is_absolute(): + store = root / store + require(store.is_file(), f'existing local store required for read-only audit: {store}') + env['AI_READY_DB_CONFIG'] = str(config_path) + workspace = args.workspace.resolve() + evidence = read_json(workspace / 'result.json') + feedback = evidence['feedback'] + reuse = feedback['reuse'] + require(evidence.get('independent_session') is True, 'independent-session evidence absent') + require(not evidence.get('guard_denied'), 'acceptance guard intervened; cannot claim natural reuse') + require(feedback.get('query_executed') is True and reuse.get('status') == 'reused', 'current evidence is not successful reuse') + require(reuse.get('new_calculation_needed') is False, 'receipt requests computation') + require(len(set(reuse['record_ids'])) == len(reuse['record_ids']) > 0, 'invalid selected record identities') + request = (workspace / 'request.txt').read_text(encoding='utf-8-sig') + require(all(r not in request for r in reuse['record_ids']), 'request contains selected record ID') + events = [json.loads(line) for line in (workspace / 'events.jsonl').read_text().splitlines() if line.strip()] + responses = [p['function_response'] for e in events for p in (e.get('content') or {}).get('parts', []) if p.get('function_response')] + require(any((r.get('response') or {}).get('aidb_preflight', {}).get('reuse', {}).get('record_ids') == reuse['record_ids'] for r in responses), + 'no matching live tool feedback in session events') + messages = [e for e in events if (e.get('content') or {}).get('role') == 'model' + and any(p.get('text') and not p.get('thought') for p in e['content'].get('parts', []))] + require(bool(messages), 'final response absent') + final = '\n'.join(p['text'] for p in messages[-1]['content']['parts'] if p.get('text') and not p.get('thought')) + require(all(r in final for r in reuse['record_ids']), 'final response omits selected IDs') + require('eV' in final, 'final response omits energy unit') + # Scientific accuracy claims require reviewer judgement; no keyword test declares + # a statement scientifically sound. Save the actual final response for it. + source = read_json(args.source_report) + stages = {step['archive']['record_id']: step for step in source['steps']} + require(all(r in stages for r in reuse['record_ids']), 'selection is not in the source archive evidence') + verification_dir = args.output.resolve().parent / ('reuse-check-' + uuid.uuid4().hex) + verification_dir.mkdir(parents=True) + task_path = verification_dir / 'task.json' + task = feedback['task_context'] + task_path.write_text(json.dumps({'task_id': 'verify-persistent-reuse', 'target': { + 'formula': task['known']['formula'], 'functional': task['known']['functional'], + 'domain': task['known']['domain'], 'min_records': 1}})) + if args.config: + env['AI_READY_DB_CONFIG'] = str(args.config.resolve()) + command = [str(python), str(root / 'skills/ai-ready-db/scripts/aidb_loop_adapter.py'), + '--workspace', str(verification_dir), 'preflight', '--task-json', str(task_path), + '--command-timeout', '20', '--export-dir', str(verification_dir / 'export'), '--export-format', 'jsonl'] + process = subprocess.run(command, capture_output=True, text=True, env=env, timeout=90) + (verification_dir / 'adapter-result.json').write_text(json.dumps({'returncode': process.returncode, + 'stdout': process.stdout, 'stderr': process.stderr}, indent=2)) + require(process.returncode == 0, 'fresh public preflight failed; see adapter-result.json') + receipt = json.loads(process.stdout) + require(receipt.get('ok') is True, 'public preflight returned failure') + query = read_json(receipt['report_path']) + records = [json.loads(line) for export in query['exports'] for line in Path(export['output_path']).read_text().splitlines() if line.strip()] + require(sorted(r['record_id'] for r in records) == sorted(query['queries'][0]['record_ids']), 'public export identities incomplete') + sys.path.insert(0, str(REPO / 'src')) + from matcreator.agents.silicon_reuse import select_reuse + from matcreator.tools.silicon_vasp import _validate + from matcreator.tools.silicon_archive import _compare + selected = select_reuse(records, task) + require(selected.get('status') == 'reused' and selected['record_ids'] == reuse['record_ids'], + 'fresh public selection changed; review current candidates') + compared = [] + for record_id in selected['record_ids']: + saved = next(r for r in records if r['record_id'] == record_id) + step = stages[record_id] + report_path = Path(step['source_result_json']['path']) + require(digest(report_path) == step['source_result_json']['sha256'], 'source result report changed') + report = read_json(report_path) + require(report.get('status') == 'success', 'source scientific completion is not success') + input_dir = Path(source['workspace']) / report['submission']['input_path'] + parsed = _validate(report_path.parent, step['calculation_type'], input_dir, report['conditions']['input_sha256']) + expected = {'structure': parsed['structure'], 'energy': parsed['total_energy']['value'], + 'functional': 'PBE', 'calculation_type': step['calculation_type'], + 'calculation_parameters': parsed['conditions']['incar'], 'source_dataset': 'matcreator-bohrium', + 'source_id': step['job_id'], 'source_metadata': {'status': 'success', 'completion': parsed['completion'], + 'conditions': parsed['conditions'], 'energy_unit': 'eV', 'energy_basis': 'cell'}} + _compare(saved, expected) + previous = next(r for r in reuse['results'] if r['record_id'] == record_id) + current = next(r for r in selected['results'] if r['record_id'] == record_id) + require(previous == current, 'old tool receipt differs from current saved result') + require(str(saved['energy']) in final, 'final response omits actual energy') + archive = read_json(step['archive']['receipt_path']) + require(archive.get('status') == 'verified' and archive['record_id'] == record_id, 'source archive unverified') + for name, artifact in saved['source_metadata']['artifacts'].items(): + require(digest(report_path.parent / name) == artifact['sha256'], f'source artifact changed: {name}') + compared.append({'record_id': record_id, 'job_id': step['job_id'], 'energy_eV': saved['energy']}) + from types import SimpleNamespace + from matcreator.workspace import ADK_DIR + from matcreator.agents.execution_agent.remote_job_tools import list_remote_jobs + require((ADK_DIR / 'remote-jobs.db').is_file(), 'tracked-job store missing; cannot verify zero jobs') + context = SimpleNamespace(state={'session_id': evidence['session_id']}, _invocation_context=SimpleNamespace(user_id=args.user_id)) + jobs = list_remote_jobs(context) + require(jobs.get('status') == 'ok' and jobs['job_count'] == 0, 'acceptance session owns remote jobs') + final_path = verification_dir / 'final-response.txt' + final_path.write_text(final, encoding='utf-8') + return {'ok': True, 'verification': 'fresh-public-readback-and-retained-real-artifacts', + 'session_id': evidence['session_id'], 'records': compared, 'new_jobs_in_session': 0, + 'cloud_database_upload': False, 'query_report': receipt['report_path'], + 'final_response': str(final_path), 'scientific_claim_review': 'required'} + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest='command', required=True) + test = sub.add_parser('tests', help='run isolated regression; fail on skips or new failures') + test.add_argument('--full', action='store_true') + test.add_argument('--baseline', type=Path) + verify = sub.add_parser('verify-reuse', help='read back existing real records; never submit or archive') + verify.add_argument('--workspace', type=Path, required=True) + verify.add_argument('--source-report', type=Path, required=True) + verify.add_argument('--config', type=Path) + verify.add_argument('--user-id', default='user') + compare = sub.add_parser('compare-results', help='validate a pytest JSON report without running tests') + compare.add_argument('--pytest-report', type=Path, required=True) + compare.add_argument('--baseline', type=Path) + for command in (test, verify): + command.add_argument('--aidb-root') + command.add_argument('--aidb-python') + for command in (test, verify, compare): + command.add_argument('--output', type=Path, required=True) + args = parser.parse_args() + try: + if args.command == 'tests': + payload = run_tests(args) + elif args.command == 'verify-reuse': + payload = verify_reuse(args) + else: + payload = {'ok': True, **check_results(read_json(args.pytest_report), read_json(args.baseline) if args.baseline else None)} + except (CheckError, OSError, ValueError, KeyError, TypeError, subprocess.TimeoutExpired) as exc: + emit(args.output, {'ok': False, 'error': str(exc)}) + return 1 + emit(args.output, payload) + return 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/script/silicon_test_report.py b/script/silicon_test_report.py new file mode 100644 index 0000000..6d7e4ac --- /dev/null +++ b/script/silicon_test_report.py @@ -0,0 +1,26 @@ +"""Pytest JSON reporting: counts and exact failure IDs, never a copied tally.""" +import json +import os +from pathlib import Path + +_results = [] + + +def pytest_runtest_logreport(report): + if report.when == 'call' or report.failed or report.skipped: + kind = 'error' if report.failed and report.when != 'call' else report.outcome + _results.append({'nodeid': report.nodeid, 'outcome': kind}) + + +def pytest_collectreport(report): + if report.failed or report.skipped: + _results.append({'nodeid': report.nodeid, 'outcome': 'error' if report.failed else 'skipped'}) + + +def pytest_sessionfinish(session, exitstatus): + output = os.environ.get('MATCREATOR_TEST_REPORT') + if output: + counts = {kind: sum(r['outcome'] == kind for r in _results) + for kind in ('passed', 'failed', 'error', 'skipped')} + Path(output).write_text(json.dumps({'exit_code': int(exitstatus), 'counts': counts, + 'failures': [r for r in _results if r['outcome'] in {'failed', 'error'}]}, indent=2)) diff --git a/tests/test_aidb_skill_hook.py b/tests/test_aidb_skill_hook.py index 478e920..a77c0c6 100644 --- a/tests/test_aidb_skill_hook.py +++ b/tests/test_aidb_skill_hook.py @@ -19,11 +19,15 @@ @pytest.fixture def local_aidb(tmp_path, monkeypatch): - root = Path(__file__).resolve().parents[2] / 'Ai-ready_Database' - if not (root / '.venv/bin/python').exists(): - pytest.skip('adjacent aidb checkout and runtime required for public bridge integration') + root = Path(os.environ.get('AI_READY_DB_ROOT') or Path(__file__).resolve().parents[2] / 'Ai-ready_Database').expanduser().resolve() + python = Path(os.environ.get('AI_READY_DB_PYTHON') or root / '.venv/bin/python').expanduser().absolute() + if not python.is_file() or not (root / 'skills/ai-ready-db/scripts/aidb_loop_adapter.py').is_file(): + message = 'aidb public bridge/runtime required; configure AI_READY_DB_ROOT and AI_READY_DB_PYTHON' + if os.environ.get('MATCREATOR_REQUIRE_AIDB') == '1': + pytest.fail(message) + pytest.skip(message) monkeypatch.setenv('AI_READY_DB_ROOT', str(root)) - monkeypatch.setenv('AI_READY_DB_PYTHON', str(root / '.venv/bin/python')) + monkeypatch.setenv('AI_READY_DB_PYTHON', str(python)) monkeypatch.setenv('AI_READY_DB_BRIDGE_SOCK', str(tmp_path / 'bridge.sock')) config = tmp_path / 'config.json' config.write_text(json.dumps({'ase_sqlite': {'path': str(tmp_path / 'local.db'), diff --git a/tests/test_silicon_checks.py b/tests/test_silicon_checks.py new file mode 100644 index 0000000..7546a1d --- /dev/null +++ b/tests/test_silicon_checks.py @@ -0,0 +1,104 @@ +"""Command-level checks of the regression guard; no database or remote jobs.""" +import json +from pathlib import Path +import subprocess +import sys +import pytest + +SCRIPT = Path(__file__).resolve().parents[1] / 'script/check_silicon_loop.py' + + +def compare(tmp_path, counts, failures=(), baseline=None, exit_code=0): + report = tmp_path / 'pytest.json' + report.write_text(json.dumps({'exit_code': exit_code, 'counts': counts, 'failures': failures})) + output = tmp_path / 'acceptance.json' + args = [sys.executable, str(SCRIPT), 'compare-results', '--pytest-report', str(report), '--output', str(output)] + if baseline is not None: + path = tmp_path / 'baseline.json' + path.write_text(json.dumps({'failures': baseline})) + args += ['--baseline', str(path)] + process = subprocess.run(args, capture_output=True, text=True) + return process.returncode, json.loads(output.read_text()) + + +def test_counts_come_from_completed_pytest_report(tmp_path): + code, result = compare(tmp_path, {'passed': 7, 'failed': 0, 'error': 0, 'skipped': 0}) + assert code == 0 and result['counts']['passed'] == 7 + assert result['all_tests_passed'] is True + + +@pytest.mark.parametrize('counts, exit_code', [ + ({'passed': 7, 'failed': 0, 'error': 0, 'skipped': 1}, 0), + ({'passed': 0, 'failed': 0, 'error': 0, 'skipped': 0}, 0), + ({'passed': 7, 'failed': 0, 'error': 0, 'skipped': 0}, 3), +]) +def test_incomplete_or_skipped_run_cannot_pass(tmp_path, counts, exit_code): + code, result = compare(tmp_path, counts, exit_code=exit_code) + assert code == 1 and result['ok'] is False + + +def test_exact_baseline_is_reported_without_claiming_clean_suite(tmp_path): + known = [{'nodeid': 'tests/test_old.py::test_old', 'outcome': 'failed'}] + code, result = compare(tmp_path, {'passed': 7, 'failed': 1, 'error': 0, 'skipped': 0}, known, known, 1) + assert code == 0 and result['all_tests_passed'] is False + assert result['new_failures'] == [] + + +def test_equal_failure_count_cannot_hide_new_failure(tmp_path): + known = [{'nodeid': 'tests/test_old.py::test_old', 'outcome': 'failed'}] + new = [{'nodeid': 'tests/test_new.py::test_new', 'outcome': 'failed'}] + code, result = compare(tmp_path, {'passed': 7, 'failed': 1, 'error': 0, 'skipped': 0}, new, known, 1) + assert code == 1 and 'test_new' in result['error'] + + +def test_collection_error_is_a_failure_without_baseline(tmp_path): + errors = [{'nodeid': 'tests/test_import.py', 'outcome': 'error'}] + code, result = compare(tmp_path, {'passed': 7, 'failed': 0, 'error': 1, 'skipped': 0}, errors, exit_code=1) + assert code == 1 and 'test_import' in result['error'] + + +def test_missing_dependency_fails_before_tests_can_skip(tmp_path): + output = tmp_path / 'report.json' + process = subprocess.run([sys.executable, str(SCRIPT), 'tests', '--aidb-root', str(tmp_path / 'missing'), + '--aidb-python', str(tmp_path / 'missing-python'), '--output', str(output)], capture_output=True, text=True) + assert process.returncode == 1 + assert 'aidb Python missing' in json.loads(output.read_text())['error'] + + +def test_pytest_plugin_reports_failures_errors_and_collection_skips(tmp_path): + import os + (tmp_path / 'test_examples.py').write_text('''import pytest + +def test_pass(): + assert True + +def test_fail(): + assert False + +@pytest.fixture +def broken(): + raise ValueError("setup failed") + +def test_setup(broken): + pass +''') + (tmp_path / 'test_skipped.py').write_text('import pytest\npytest.skip("unavailable", allow_module_level=True)\n') + report_path = tmp_path / 'report.json' + env = {**os.environ, 'MATCREATOR_TEST_REPORT': str(report_path), + 'PYTHONPATH': str(SCRIPT.parents[1])} + process = subprocess.run([sys.executable, '-m', 'pytest', str(tmp_path), '-q', + '-p', 'script.silicon_test_report'], capture_output=True, text=True, env=env, cwd=tmp_path) + assert process.returncode == 1 + report = json.loads(report_path.read_text()) + assert report['counts'] == {'passed': 1, 'failed': 1, 'error': 1, 'skipped': 1} + assert {r['outcome'] for r in report['failures']} == {'failed', 'error'} + + +@pytest.mark.parametrize('counts,exit_code', [ + ({'passed': 1, 'failed': 0, 'error': 0, 'skipped': 0}, 1), + ({'passed': 1, 'failed': 1, 'error': 0, 'skipped': 0}, 0), + ({'passed': 1, 'failed': -1, 'error': 0, 'skipped': 0}, 0), +]) +def test_contradictory_counts_are_rejected(tmp_path, counts, exit_code): + code, result = compare(tmp_path, counts, exit_code=exit_code) + assert code == 1 and result['ok'] is False From fb6f474b173c1acd102a0281d13586e69f344bf7 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Thu, 8 Oct 2026 01:57:03 +0800 Subject: [PATCH 21/22] Record published CI bridge dependency and T5 compatibility --- .github/workflows/test.yml | 4 ++-- docs/testing/silicon-local-loop.md | 4 ++-- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6d92ebf..280aafb 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -74,8 +74,8 @@ jobs: uses: actions/checkout@v4 with: repository: Zikkying/Ai-ready_Database - # Prepared in MatCreator/.workspace/aidb-ci; publish this dependency - # commit before expecting this job to run on GitHub. Never use latest. + # Public bridge contract verified at this immutable integration commit. + # Published on codex/matcreator-ci-bridge; never use latest. ref: 87148777fc9995a6d2e3bd85571ee8e20ff53181 path: Ai-ready_Database persist-credentials: false diff --git a/docs/testing/silicon-local-loop.md b/docs/testing/silicon-local-loop.md index 52b8c1f..f6e7fbc 100644 --- a/docs/testing/silicon-local-loop.md +++ b/docs/testing/silicon-local-loop.md @@ -18,7 +18,7 @@ From the Windows PowerShell checkout on WSL: Use a new output filename for every run. The command refuses an existing pytest report, writes JSON counts and exact failure identities, a log and JUnit XML, and fails on skipped tests or any failure. Missing aidb dependencies fail rather than quietly skipping. `AI_READY_DB_ROOT` and `AI_READY_DB_PYTHON` select the public bridge checkout and its Python; defaults use the adjacent `Ai-ready_Database` checkout and `.venv/bin/python`. -The `silicon-local-loop` job in [the workflow](../../.github/workflows/test.yml) pins the aidb commit, installs both packages, runs this same command against isolated test databases and retains its artifacts. The dependency must be available on GitHub before Actions can fetch it. The prepared version is `87148777fc9995a6d2e3bd85571ee8e20ff53181`, retained locally in `.workspace/aidb-ci`; it contains the public bridge and bounded preflight contract. A checkout of GitHub's older aidb main alone lacks that bridge. Publication is tracked separately from local validation. +The `silicon-local-loop` job in [the workflow](../../.github/workflows/test.yml) pins the aidb commit, installs both packages, runs this same command against isolated test databases and retains its artifacts. The published dependency version is `87148777fc9995a6d2e3bd85571ee8e20ff53181`, available on [codex/matcreator-ci-bridge](https://github.com/Zikkying/Ai-ready_Database/tree/codex/matcreator-ci-bridge) and retained locally in `.workspace/aidb-ci`; it contains the public bridge and bounded preflight contract. A checkout of GitHub's older aidb main alone lacks that bridge. Its remote branch SHA was verified on 2026-10-08; publication is separate from a successful Actions run. ## Full-suite failure baseline @@ -57,4 +57,4 @@ Validated against a clean checkout of the pinned dependency, rather than its dev - Fresh readback of both retained real records passed (`reuse-final.json`), with zero tracked jobs in the independent session. A changed structure was rejected; a missing store was rejected without creating it. - PowerShell wrapper, workflow YAML loading, Python compilation and authored-file whitespace checks passed. -Reports are local under `.workspace/retro-checks/`; original T3/T4 evidence is retained. GitHub Actions has not run this new job: the prepared dependency commit must first be published. Local validation does not establish remote CI success. +Reports are local under `.workspace/retro-checks/`; original T3/T4 evidence is retained. The pinned dependency was published to `codex/matcreator-ci-bridge` on 2026-10-08 and its remote SHA verified. GitHub Actions has not run this new MatCreator job; local validation and dependency publication do not establish remote CI success. From 692bf7c8fb70bf9c636d70fda5bce72801374543 Mon Sep 17 00:00:00 2001 From: Shik-Utau <129398280+Shik-Utau@users.noreply.github.com> Date: Thu, 8 Oct 2026 09:01:34 +0800 Subject: [PATCH 22/22] Retain T5 retro regression proof for silicon delivery --- docs/testing/silicon-consent-retro-proof.json | 47 +++++++++++++++ docs/testing/silicon-consent-retro.md | 58 +++++++++++++++++++ 2 files changed, 105 insertions(+) create mode 100644 docs/testing/silicon-consent-retro-proof.json create mode 100644 docs/testing/silicon-consent-retro.md diff --git a/docs/testing/silicon-consent-retro-proof.json b/docs/testing/silicon-consent-retro-proof.json new file mode 100644 index 0000000..b4a641d --- /dev/null +++ b/docs/testing/silicon-consent-retro-proof.json @@ -0,0 +1,47 @@ +{ + "historical_hook_commit": "587becd", + "repair_commit": "4dc04d3", + "results": { + "historical": { + "exit_code": 1, + "counts": { + "passed": 2, + "failed": 4, + "error": 0, + "skipped": 0 + }, + "failures": [ + { + "nodeid": "tests/test_silicon_lookup_consent.py::test_changed_target_or_conditions_invalidates_failure_confirmation[ENCUT=600]", + "outcome": "failed" + }, + { + "nodeid": "tests/test_silicon_lookup_consent.py::test_changed_target_or_conditions_invalidates_failure_confirmation[KPOINTS=6x6x6]", + "outcome": "failed" + }, + { + "nodeid": "tests/test_silicon_lookup_consent.py::test_changed_target_or_conditions_invalidates_failure_confirmation[HSE06]", + "outcome": "failed" + }, + { + "nodeid": "tests/test_silicon_lookup_consent.py::test_condition_update_without_tools_still_invalidates_old_confirmation", + "outcome": "failed" + } + ] + }, + "current": { + "exit_code": 0, + "counts": { + "passed": 6, + "failed": 0, + "error": 0, + "skipped": 0 + }, + "failures": [] + } + }, + "checkout_runtime_changed": false, + "shared_ci_or_git_changed": false, + "real_jobs_submitted": 0, + "cloud_database_upload": false +} diff --git a/docs/testing/silicon-consent-retro.md b/docs/testing/silicon-consent-retro.md new file mode 100644 index 0000000..faf4516 --- /dev/null +++ b/docs/testing/silicon-consent-retro.md @@ -0,0 +1,58 @@ +# T5 consent retrospective + +This supplements [the T4 retrospective tooling](silicon-local-loop.md) for +[Ai-ready_Database #6](https://github.com/Zikkying/Ai-ready_Database/issues/6), +under [the main specification](https://github.com/Zikkying/Ai-ready_Database/issues/1). + +The initial implementation tested `继续，ENCUT=600`, but missed standalone +`ENCUT=600`: task classification returned no calculation task and retained the +old failure receipt. A later exact confirmation resumed the old authorization. +An intervening user update without tool calls exposed the same lifecycle gap. +The independent Spec review reproduced this at the ADK task boundary; +`4dc04d3` fixed both paths. Confirmation now checks the preceding human +invocation, the exact failure event, session and complete task conditions. + +## Deterministic regression evidence + +On 2026-10-08, the same six public task-boundary tests were run in independent +pytest processes. One process loaded the historical `587becd` hook from Git; +the other used the current hook. The checkout and shared branch were unchanged. + +| Cases | Historical hook | Current hook | +| --- | --- | --- | +| Standalone ENCUT, KPOINTS and HSE06 updates, followed by old confirmation | 3 failures | 3 passes | +| Condition update with no tool calls, followed by old confirmation | 1 failure | 1 pass | +| Current valid confirmation and original preparation-only authorization | 2 passes | 2 passes | +| Errors / skipped tests | 0 / 0 | 0 / 0 | + +The four historical failure IDs exactly match the negative cases. Positive +controls show the guard still allows valid confirmation and respects original +preparation-only scope. Tests observe controlled preparation/submission tools +and actual ADK feedback, rather than private guard calls or prompt strings. +The public aidb bridge intentionally reports a missing isolated config to +produce lookup failure; `_query` is not replaced with a mock. No real +calculation or cloud upload was performed. + +Exact counts and failure identities are in +[silicon-consent-retro-proof.json](silicon-consent-retro-proof.json). +The temporary replay driver and detailed logs remain under +`.workspace/silicon-t5-retro-20261008/`. This historical replay establishes +regression sensitivity; it is not another permanent CI harness. + +## Deduplication with T4 retro + +The existing `script/check_silicon_loop.py tests` command and the +`silicon-local-loop` CI job already run all 29 consent cases, require the +public integration dependency, reject unexpected skips/failures and retain +reports. Reuse them; no second CI job, dependency branch, baseline file or +global steering paragraph is needed for T5. The T4 retro also owns scientific +claim review, navigation and the PowerShell/WSL entry point. + +For future authorization changes, keep standalone condition updates and human +turns without callbacks in the task suite. Permission lifetime crosses language +classification, session history and executor delegation; a syntax linter alone +cannot establish it. Retain the independent Spec review that detected the gap. + +Publication/local validation of the pinned aidb dependency is separate from +successful remote CI. The combined delivery PR reports the actual remote +checks separately from these historical/local results.