Files
workbuddy_skills/session-mechanism/graph/graphctl.py
T
admin ce05207aa2 graph: 补齐图模式剩余四件(验收机判/回退边/运行期改图/跨图子图),163 条测试全绿
- F1 accepts 机判:report done 前按 artifact:/evidence: 逐条校验,缺什么报什么,不许绕过
- F2 rework_to 回退边:判不过回到指定上游重做,attempts/reworks/事件齐备,人审驳回同路
- F3 运行期改图:add-node / remove-edge,目录锁+唯一临时名+原子替换+_rev 回读,并发不丢更新
- F4 跨图子图:subgraph.gid 引用同工作区另一张图,deps/when/rework_to 加前缀,父等收敛;缺图/自引用/跨图环/嵌套均硬报错
- 测试:122 → 163,新增 T20-T23;顺带修测试进程被本机批量删除守卫误杀的假红
2026-10-19 23:57:40 +08:00

2298 lines
97 KiB
Python
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""graphctl.py —— 「图模式」的独立实现(2026-10-18;2026-10-19 补批量能力)。
## 它是什么
把目标拆成节点、按依赖推进的**独立**运行模式。**⛔ 不修改任何现有机制**:
不读也不写 `goal.json` / `tasks.json` / `collabd-state.json`,不参与六道闸,
不接管常驻循环,不改变现有会话的任何行为。
## 数据落点(独立目录,与现有状态完全分开)
<ws>/.workbuddy/graph/<gid>/graph.json 图的唯一真源
<ws>/.workbuddy/graph/<gid>/events.jsonl 每次状态变更的流水
## 图的结构
{
"gid": "20261018-2230-ui改造",
"title": "给三块功能各加一份文档",
"created_at": "2026-10-18T22:30:00",
"meta": {"max_parallel": 3,
"lease": {"timeout_sec": 3600, "on_timeout": "requeue"},
"retry": {"max": 0, "delay_sec": 0, "backoff": "fixed", "factor": 2}},
"nodes": {
"N1": {"title": "...", "deps": [], "domain": ["docs/ui/"],
"accepts": ["产物在位"], "status": "pending",
"evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": "all_success",
"retry": {"max": 2, "delay_sec": 30, "backoff": "exponential"},
"map": {"over": ["a", "b"], "child_domain": "docs/ui/{i}/"}}
}
}
`status` 五态:`pending` → `running` → `done` / `blocked`;
另有 `gating`——展开出来的父节点,等子节点全部完成(父节点自动转 `done`)。
## 命令
init --ws <工作区> --title <目标> (--kpi "判据A;判据B" | --nodes-json <文件|->)
[--max-parallel N] [--lease-sec S] [--on-timeout requeue|block|keep]
[--retry-max N] [--retry-delay S] [--retry-backoff fixed|linear|exponential]
check --ws <ws> --gid <gid> 环/悬空依赖/空域/同批域冲突/非法规则
ready --ws <ws> --gid <gid> 就绪集(含为什么没被选);顺带回收超时租约、展开 map
start --ws <ws> --gid <gid> --node N1 [--by ...] [--lease-sec S]
report --ws <ws> --gid <gid> --node N1 --state done|blocked [--evidence "..."] [--artifact P ...]
rework --ws <ws> --gid <gid> --node N1 [--to 上游] [--reason ...] 回退到上游重做
add-node --ws <ws> --gid <gid> --node N9 [--deps A,B] [--domain P] [--accepts C]
remove-edge --ws <ws> --gid <gid> --from A --to B 删掉「B 依赖 A」这条边
expand --ws <ws> --gid <gid> --node P [--force] 手动触发展开
show --ws <ws> --gid <gid>
list --ws <ws>
tick --ws <ws> --gid <gid> [--by ...] [--limit N] [--dry-run] 自动派活(算完就开)
approve --ws <ws> --gid <gid> --node N [--reject] [--reason ...] 人审
cancel --ws <ws> --gid <gid> (--node N | --all) [--reason ...] 取消
plan --ws <ws> --gid <gid> [--scope ...] [--after-sec N] 出一行排期
progress --ws <ws> --gid <gid> 进度/关键路径
## 复杂任务能力(2026-10-19 补,C 方案)
条件分支:节点 `when: [{"node": "A", "status": "blocked"}]` —— 条件被证伪即
`skipped`;`on_dep_fail: wait|skip|block` 决定上游跑不成时下游怎么走
(默认 wait,与旧行为一致)。
子图:节点 `subgraph: {"nodes": [...]}` —— 内联子流程展开成 `<父>/<子id>`,
父节点 `gating`,子节点全成功才 `done`,有不成功的 ⇒ 父 `blocked`。
人审:节点 `approval: true` —— report done 后进 `awaiting`,`approve` 才放行
(`--reject` ⇒ blocked)。
取消:`cancelled` 是终态,连坐子节点;下游按 `on_dep_fail` 处置。
分池:`meta.pools: {"域前缀": 并发上限}` —— 同域并发受限,与 `max_parallel` 叠加。
进度:`progress` 给出 by_status/percent/关键路径(未完成节点的最长依赖链)。
## 验收判据(accepts)怎么机判(2026-10-19 补)
`report --state done` 时**先按 `accepts` 校验产物与证据**,不满足直接拒收(rc≠0,节点
不置 done,并在 stderr 逐条列出缺什么)。条目两种写法,**纯文本=描述性判据、不参与机判**:
"artifact:docs/*.md" 产物里至少一条匹配该通配(反斜杠与 `/` 等价)
"artifact:>=2" 产物至少 2 条
"evidence:测试通过" 证据里必须出现「测试通过」
"evidence:全绿,零回归" 多个关键词用 `,` 分隔 ⇒ 全部都要出现(AND)
"evidence:>=20" 证据至少 20 个字符
{"kind": "artifact", "glob": "..."} / {"kind": "artifact", "min": 2}
{"kind": "evidence", "contains": ["a", "b"]} / {"kind": "evidence", "min": 20}
校验读的是**本次 + 历史**的合并视图(`--artifact` 是追加语义)。
## 回退边(rework_to,2026-10-19 补)
节点写 `rework_to: "上游节点"`(或列表,取第一个)。判不过时用 `rework --node X`:
① 目标必须是 X 的**上游**(传递依赖),⛔ 不能是自己或下游;
② 目标及其**全部传递下游**复位 `pending`,清租约/完成时刻/驳回原因;
③ 复位时把 `evidence` / `artifacts` 挪到 `prev_evidence` / `prev_artifacts` 再清空
⇒ 重做之后必须重新交产物(否则验收闸会被旧产物蒙过);
④ 目标 `attempts +1`、全链 `reworks +1`,写一条 `rework` 事件。
⑤ `approve --reject` 时若节点带 `rework_to`,自动走同一套回退(人审判不过=判不过)。
⑥ 链上有已展开(`map`/子图)的节点 ⇒ **收回它的子节点并清 `expanded`**,
下次 `ready`/`tick` 按当时的输入重新展开(⛔ 不留半截子节点)。
## 运行期改图(2026-10-19 补)
add-node --node N9 --deps A,B --domain docs/x/ --accepts "artifact:docs/x/a.md"
remove-edge --from A --to B 删掉「B 依赖 A」
两条都走 `_update`(目录锁 + 唯一临时名 + 原子替换 + `_rev` 回读)⇒ 与并发的
`start`/`report` 互不覆盖;边不存在、节点重复、依赖悬空 ⇒ **报错**,⛔ 不静默空操作。
## 跨图子图(2026-10-19 补)
`subgraph: {"gid": "<同工作区另一张图的 gid>"}` —— 除内联 `{"nodes": [...]}` 外,
可引用 `<ws>/.workbuddy/graph/<gid>/graph.json`,把它的**节点定义**展开成本图的
`<父>/<子id>`,父节点照旧 `gating`、等子节点全部收敛。
① 只搬定义(title/deps/domain/accepts/deps_rule/on_dep_fail/when/retry/
lease_sec/approval/rework_to),⛔ 不搬运行状态(引用即重跑一遍);
② 子图内部依赖与被引用图的 `when`/`rework_to` 引用都加父前缀;
③ ⛔ 被引用的图里不允许再有 `map`/`subgraph`(不嵌套),`check` 报硬错;
④ 自引用、A→B→A 成环 ⇒ `check` 与展开时都报错。
## 就绪的判据
① 依赖按 `deps_rule` 满足(默认 all_success)
② 自己还是 pending,且不在重试等待窗内(`not_before` 未到 ⇒ 等)
③ 域不与任何 running 节点重叠,也不与本批已选节点重叠
④ 不超 max_parallel(running 占名额)
## 三件批量能力(2026-10-19 补)
租约:start 记 `lease_at` / `lease_at_ts`;ready 时超过 `lease.timeout_sec`
的 running 节点按 `on_timeout` 处置 —— requeue(默认,回 pending 且
attempts+1)/block(进 blocked)/keep(只提示不动)。
重试:`retry.max` 给了才自动重试,report blocked 后 attempts≤max ⇒ 回 pending
并写 `not_before`(按 backoff 算),超过 ⇒ blocked 且 `retry_exhausted`。
默认 `max=0` ⇒ 与旧行为一致(report blocked 直接 blocked)。
展开:`map.over`(建图给列表)或 `map.from`(运行时读上游 artifacts / evidence),
ready 命中即生成子节点 `P#1..P#N`,父节点转 `gating`,子节点全 done ⇒ 父 done。
## 域的用法
`domain` 是一组路径前缀(相对工作区根)。两节点域重叠 ⇒ 不能同批跑。
判定:任一前缀互为前缀即视为重叠(`docs/` 与 `docs/ui/` 算重叠)。
展开出的子节点**默认不继承父域**(同一批展开通常写同一批文件,继承会把并发挡死);
需要分片写盘时用 `map.child_domain` 模板(`{i}` / `{item}`)。
"""
from __future__ import annotations
import argparse
import fnmatch
import io
import json
import math
import os
import random
import sys
import time
from pathlib import Path
MARK = ".workbuddy/graph"
LOCK_STALE_SEC = 30.0 # 锁目录超过这么久没动静 ⇒ 认定持锁进程已死,接管
STATUSES = ("pending", "running", "done", "blocked", "gating",
"awaiting", "skipped", "cancelled")
SETTLED = ("done", "blocked", "skipped", "cancelled") # 不会再自己变了
DEP_RULES = ("all_success", "all_done", "one_success")
TIMEOUT_POLICIES = ("requeue", "block", "keep")
BACKOFFS = ("fixed", "linear", "exponential")
ON_DEP_FAIL = ("wait", "skip", "block") # 上游跑不成时下游怎么办
DEFAULT_LEASE_SEC = 3600.0
# ── 基础设施 ────────────────────────────────────────────────────────────
def _now() -> str:
return time.strftime("%Y-%m-%dT%H:%M:%S")
def _iso(ts: float) -> str:
return time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(ts))
def _parse_ts(v):
"""ISO 字符串 → epoch;失败返回 None(⛔ 不抛,坏数据当作"没有租约")。"""
if v in (None, ""):
return None
try:
return float(v)
except (TypeError, ValueError):
pass
for fmt in ("%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S"):
try:
return time.mktime(time.strptime(str(v), fmt))
except ValueError:
continue
return None
def _root(ws: str) -> Path:
return Path(ws).resolve() / MARK
def _gdir(ws: str, gid: str) -> Path:
return _root(ws) / gid
def _load(ws: str, gid: str) -> dict:
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
raise SystemExit("找不到图:%s" % p)
return json.loads(p.read_text(encoding="utf-8"))
def _save(ws: str, gid: str, g: dict) -> None:
"""整份写回(新建图时用)。临时名带 pid + 随机数 ⇒ 两进程不会撞同一个名字。"""
p = _gdir(ws, gid) / "graph.json"
tmp = p.with_suffix(".%d.%d.json.tmp" % (os.getpid(), random.randint(0, 1 << 30)))
tmp.write_text(json.dumps(g, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8", newline="\n")
os.replace(tmp, p) # 原子替换,避免半截文件
class _Lock:
"""写串行化:`os.mkdir` 是原子的,用它当互斥量(⛔ 不用 OS 文件锁,本机会卡死)。
⚠️ 2026-10-19 实测:只靠 `_rev` + token 回读核对仍然丢更新 ——
A 写完回读确认后、B 再覆盖,B 回读看到的也是自己的 token,两边都"成功"。
那次实测里 4 轮测试抖 1 轮(`pending` != `running`)。所以写必须串行。
⚠️ 僵死锁:持锁进程崩了锁目录会留着 ⇒ 超过 `LOCK_STALE_SEC` 直接接管,
⛔ 不会永久卡住;等 `wait` 秒仍拿不到则**报错退出**(带手工清理提示)。
"""
def __init__(self, p: Path, wait: float = 10.0):
self.p = p
self.wait = wait
self.held = False
def __enter__(self):
t0 = time.time()
while True:
try:
os.mkdir(self.p)
self.held = True
try:
(self.p / "owner").write_text(str(os.getpid()), encoding="utf-8")
except OSError:
pass
return self
except FileExistsError:
try:
age = time.time() - self.p.stat().st_mtime
except OSError:
age = 0.0
if age > LOCK_STALE_SEC:
for f in ("owner",):
try:
(self.p / f).unlink()
except OSError:
pass
try:
os.rmdir(self.p)
except OSError:
pass
continue
if time.time() - t0 > self.wait:
raise SystemExit(
"写锁等待超时:%s(等了 %.0f 秒;确认没有僵死进程后可直接删这个目录)"
% (self.p, self.wait))
time.sleep(0.02)
def __exit__(self, *exc):
if self.held:
try:
(self.p / "owner").unlink()
except OSError:
pass
try:
os.rmdir(self.p)
except OSError:
pass
return False
def _update(ws: str, gid: str, mutate, retries: int = 3) -> dict:
"""读-改-写:**先拿目录锁再动文件**,锁内重放变更并原子替换。
① 临时名带 pid + 随机数 ⇒ 两进程不会撞同一个名字(早期固定名会 `PermissionError`)。
② 写整体串行(`_Lock`)⇒ 不会"后写的整份覆盖前写的"导致节点状态静默消失。
③ `_rev` + `_token` 是第二道网:写后回读核对,对不上就重做;
连续 `retries` 次仍被覆盖则**报错退出**(⛔ 不静默丢状态)。
④ Windows 上 `os.replace` 与别的进程的读并发时会短暂 `PermissionError`
⇒ 替换动作本身小步重试,⛔ 不把它当成"并发冲突"直接失败。
"""
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
raise SystemExit("找不到图:%s" % p)
with _Lock(p.parent / ".lock"):
for _ in range(retries):
g = json.loads(p.read_text(encoding="utf-8"))
rev = int(g.get("_rev") or 0)
mutate(g)
token = "%d-%d" % (os.getpid(), random.randint(0, 1 << 30))
g["_rev"] = rev + 1
g["_token"] = token
tmp = p.with_suffix(".%d.%d.json.tmp" % (os.getpid(), random.randint(0, 1 << 30)))
tmp.write_text(json.dumps(g, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8", newline="\n")
for _try in range(6): # 目标被读打开时 replace 会短暂被拒
try:
os.replace(tmp, p)
break
except OSError:
time.sleep(0.05)
else:
try:
tmp.unlink()
except OSError:
pass
continue
try:
back = json.loads(p.read_text(encoding="utf-8"))
except Exception: # noqa: BLE001
back = {}
if int(back.get("_rev") or 0) == rev + 1 and back.get("_token") == token:
return back
raise SystemExit("并发冲突:%s 连续 %d 次被别的进程覆盖,稍后重试" % (p, retries))
def _event(ws: str, gid: str, obj: dict) -> None:
p = _gdir(ws, gid) / "events.jsonl"
obj = dict(obj)
obj.setdefault("at", _now())
with io.open(p, "a", encoding="utf-8", newline="\n") as f:
f.write(json.dumps(obj, ensure_ascii=False) + "\n")
def _out(obj) -> None:
data = json.dumps(obj, ensure_ascii=False, indent=2) + "\n"
sys.stdout.buffer.write(data.encode("utf-8"))
# ── 验收判据(accepts 机判)──────────────────────────────────────────────
def _accepts_problem(vals):
"""校验 accepts 语法;有问题返回一句人读的原因,没问题返回 None。"""
for raw in (vals or []):
if isinstance(raw, dict):
kind = str(raw.get("kind") or "").strip()
if kind not in ("artifact", "evidence"):
return 'accepts 的对象条目要写 kind=artifact 或 evidence(实际 %r)' % (raw,)
if kind == "artifact":
if raw.get("glob") in (None, "") and raw.get("min") in (None, ""):
return 'accepts 的 artifact 条目要写 glob 或 min:%r' % (raw,)
if raw.get("min") not in (None, ""):
try:
if int(raw["min"]) <= 0:
return "accepts 的 min 必须 ≥1:%r" % (raw,)
except (TypeError, ValueError):
return "accepts 的 min 不是整数:%r" % (raw,)
else:
cs = raw.get("contains")
if cs is not None and not isinstance(cs, list):
return 'accepts 的 evidence.contains 要写成列表:%r' % (raw,)
if not cs and raw.get("min") in (None, ""):
return 'accepts 的 evidence 条目要写 contains 或 min:%r' % (raw,)
if raw.get("min") not in (None, ""):
try:
if int(raw["min"]) <= 0:
return "accepts 的 min 必须 ≥1:%r" % (raw,)
except (TypeError, ValueError):
return "accepts 的 min 不是整数:%r" % (raw,)
continue
if not isinstance(raw, str):
return "accepts 条目只认字符串或对象(实际 %r)" % (raw,)
s = raw.strip()
low = s.lower()
if not (low.startswith("artifact:") or low.startswith("evidence:")):
continue # 纯文本 = 描述性判据,不机判
head, _, arg = s.partition(":")
arg = arg.strip()
if not arg:
return "accepts 条目缺参数:%r(写成 %s:<通配|>=N|关键词>)" % (s, head)
if arg.startswith(">="):
try:
if int(arg[2:]) <= 0:
return "accepts 的 >=N 要 ≥1:%r" % s
except ValueError:
return "accepts 的 >=N 不是整数:%r" % s
elif low.startswith("evidence:") and not any(
x.strip() for x in arg.replace(",", ",").split(",")):
return "accepts 的 evidence 关键词为空:%r" % s
return None
def _norm_accepts(vals) -> list:
p = _accepts_problem(vals)
if p:
raise SystemExit(p)
out = []
for raw in (vals or []):
if isinstance(raw, dict):
kind = str(raw.get("kind")).strip()
it = {"kind": kind}
if kind == "artifact":
if raw.get("glob") not in (None, ""):
it["glob"] = str(raw["glob"]).strip()
if raw.get("min") not in (None, ""):
it["min"] = int(raw["min"])
else:
if raw.get("contains"):
it["contains"] = [str(x) for x in raw["contains"]]
if raw.get("min") not in (None, ""):
it["min"] = int(raw["min"])
out.append(it)
else:
out.append(str(raw).strip())
return out
def _norm_path(s) -> str:
return str(s or "").replace("\\", "/").strip()
def _accepts_missing(accepts, artifacts, evidence) -> list:
"""逐条判 accepts,返回没达标的描述(空列表=全过)。"""
arts = [_norm_path(x) for x in (artifacts or [])]
ev = str(evidence or "")
miss = []
for raw in (accepts or []):
if isinstance(raw, dict):
kind = str(raw.get("kind") or "")
glob = raw.get("glob")
mn = raw.get("min")
need = [str(x) for x in (raw.get("contains") or [])]
else:
s = str(raw).strip()
head, _, arg = s.partition(":")
low = head.strip().lower()
arg = arg.strip()
if low not in ("artifact", "evidence"):
continue # 描述性判据,不机判
kind = low
glob = None
mn = None
need = []
if arg.startswith(">="):
mn = int(arg[2:])
elif kind == "artifact":
glob = arg
else:
need = [x.strip() for x in arg.replace(",", ",").split(",") if x.strip()]
if kind == "artifact":
if glob:
if not any(fnmatch.fnmatch(a, _norm_path(glob)) for a in arts):
miss.append("缺产物 artifact:%s(现有产物:%s)"
% (glob, "、".join(arts) if arts else "无"))
if mn is not None and len(arts) < mn:
miss.append("产物数量不足 artifact:>=%d(现有 %d 条)" % (mn, len(arts)))
elif kind == "evidence":
if mn is not None and len(ev.strip()) < mn:
miss.append("证据太短 evidence:>=%d(现有 %d 字)" % (mn, len(ev.strip())))
if need:
absent = [k for k in need if k not in ev]
if absent:
miss.append("证据缺关键词 evidence:%s(现缺:%s)"
% (",".join(need), "、".join(absent)))
return miss
# ── 回退边(rework_to)──────────────────────────────────────────────────
def _ancestors(nodes: dict, nid: str) -> set:
seen, stack = set(), list(_deps(nodes.get(nid) or {}))
while stack:
x = stack.pop()
if x in seen or x not in nodes:
continue
seen.add(x)
stack.extend(_deps(nodes[x]))
return seen
def _descendants(nodes: dict, nid: str) -> set:
"""所有传递下游(不含自己)。"""
kids = {}
for k, n in nodes.items():
for d in _deps(n):
kids.setdefault(d, []).append(k)
seen, stack = set(), list(kids.get(nid) or [])
while stack:
x = stack.pop()
if x in seen or x not in nodes:
continue
seen.add(x)
stack.extend(kids.get(x) or [])
return seen
def _rework_target(n: dict):
"""节点自己的 rework_to(字符串或列表)⇒ 第一个目标 id(可能为空)。"""
rt = n.get("rework_to")
if isinstance(rt, (list, tuple)):
for x in rt:
if str(x).strip():
return str(x).strip()
return ""
return str(rt or "").strip()
def _drop_subtree(nodes: dict, nid: str) -> list:
"""收回一个已展开节点的全部子节点(展开产物,重新展开时按需重建)。"""
out, stack = [], list((nodes.get(nid) or {}).get("children") or [])
while stack:
c = stack.pop()
if c not in nodes:
continue
out.append(c)
stack.extend((nodes[c] or {}).get("children") or [])
for c in out:
nodes.pop(c, None)
return out
def _rework_reset(nodes: dict, from_nid: str, tgt: str, reason: str = "") -> dict:
"""把 tgt 及其全部传递下游复位成 pending(调用方保证已通过前置校验)。
链上有已展开(map/subgraph)的节点 ⇒ **收回它的子节点、清掉 expanded**,
下次 ready/tick 会按当时的输入重新展开(⛔ 不留半截子节点)。
"""
down = _descendants(nodes, tgt) | {tgt}
bad = [x for x in down if str((nodes.get(x) or {}).get("status")) == "running"]
if bad:
raise SystemExit("回退链上有正在跑的节点:%s(先 cancel 或等它结束)"
% ", ".join(sorted(bad)))
dropped = []
for x in sorted(down):
m = nodes.get(x)
if m is not None and (m.get("expanded") or str(m.get("status")) == "gating"):
dropped.extend(_drop_subtree(nodes, x))
now = _now()
reset = []
for x in sorted(down):
m = nodes.get(x)
if m is None: # 已被当作子节点收回
continue
if m.get("artifacts"):
m["prev_artifacts"] = list(m["artifacts"])
m["artifacts"] = []
if m.get("evidence"):
m["prev_evidence"] = m["evidence"]
m["evidence"] = ""
m["status"] = "pending"
for k in ("lease_at", "lease_at_ts", "lease_sec", "done_at", "awaiting_at",
"approved_at", "approved_by", "not_before", "retry_at",
"retry_exhausted", "rejected_at", "blocked_by", "blocked_reason",
"skipped_at", "skipped_reason", "completed_at",
"expanded", "expanded_at", "item_count", "children"):
m.pop(k, None)
m["reworked_at"] = now
m["rework_of"] = tgt
m["reworks"] = int(m.get("reworks") or 0) + 1
reset.append(x)
nodes[tgt]["attempts"] = int(nodes[tgt].get("attempts") or 0) + 1
nodes[tgt]["rework_from"] = from_nid
if reason:
nodes[tgt]["rework_reason"] = reason
res = {"target": tgt, "reset": reset,
"attempts": nodes[tgt]["attempts"],
"reworks": nodes[tgt]["reworks"]}
if dropped:
res["dropped_children"] = dropped
return res
# ── 跨图子图(subgraph.gid)─────────────────────────────────────────────
def _sg_chain(g: dict, nid: str) -> list:
"""从自己往上走 parent 链,收集沿途声明的 subgraph.gid(防跨图成环)。"""
nodes = g.get("nodes") or {}
out, seen, cur = [], set(), nid
while cur and cur in nodes and cur not in seen:
seen.add(cur)
sg = (nodes.get(cur) or {}).get("subgraph")
if isinstance(sg, dict) and str(sg.get("gid") or "").strip():
out.append(str(sg["gid"]).strip())
cur = (nodes.get(cur) or {}).get("parent")
return out
def _sg_specs_from_gid(ws: str, ref_gid: str, chain: list) -> list:
"""读另一张图,把它的节点定义翻成内联子图规格。"""
if not ws:
raise SystemExit("内部错误:展开跨图子图时缺工作区路径(gid=%s)" % ref_gid)
if ref_gid in chain:
raise SystemExit("子图引用成环:%s" % " → ".join(list(chain) + [ref_gid]))
p = _gdir(ws, ref_gid) / "graph.json"
if not p.is_file():
raise SystemExit("子图引用的图不存在:%s(gid=%s)" % (p, ref_gid))
sg = json.loads(p.read_text(encoding="utf-8"))
specs = []
for sid, sn in (sg.get("nodes") or {}).items():
if not isinstance(sn, dict):
continue
if sn.get("map") or sn.get("subgraph"):
raise SystemExit("子图 %s 的节点 %s 自己又声明了 map/subgraph ⇒ 不支持嵌套"
% (ref_gid, sid))
specs.append({
"id": sid,
"title": sn.get("title") or sid,
"deps": list(sn.get("deps") or []),
"domain": _norm_dom(sn.get("domain")),
"accepts": list(sn.get("accepts") or []),
"deps_rule": sn.get("deps_rule"),
"on_dep_fail": sn.get("on_dep_fail"),
"when": sn.get("when"),
"retry": sn.get("retry"),
"lease_sec": sn.get("lease_sec"),
"approval": sn.get("approval"),
"rework_to": sn.get("rework_to"),
})
return specs
# ── 域与依赖 ────────────────────────────────────────────────────────────
def _norm_dom(d) -> list:
if isinstance(d, str):
d = [d]
out = []
for x in (d or []):
s = str(x).strip().replace("\\", "/").lstrip("./")
if s:
out.append(s if s.endswith("/") else s + "/")
return out
def _dom_overlap(a, b) -> bool:
"""任一前缀互为前缀 ⇒ 重叠(docs/ 与 docs/ui/ 算重叠)。"""
for x in a:
for y in b:
if x.startswith(y) or y.startswith(x):
return True
return False
def _deps(n: dict) -> list:
v = n.get("deps") or []
return [str(x) for x in v] if isinstance(v, list) else []
def _is_done(nodes: dict, nid: str) -> bool:
return str((nodes.get(nid) or {}).get("status")) == "done"
def _st(nodes: dict, nid: str) -> str:
return str((nodes.get(nid) or {}).get("status"))
def _dep_verdict(nodes: dict, n: dict) -> dict:
"""依赖判定 ⇒ {"verdict": go|wait|skip|block, "why": 说明}
① `when` 条件子句先判(条件分支):全部成立 ⇒ 继续;任一被证伪 ⇒ skip。
② 再按 `deps_rule` 判:all_success(默认)/all_done/one_success。
③ 上游停在 skipped / cancelled / blocked 时按 `on_dep_fail` 处置:
wait(默认,与旧行为一致)/skip(连坐跳过)/block(整枝停住)。
"""
deps = _deps(n)
rule = str(n.get("deps_rule") or "all_success")
on_fail = str(n.get("on_dep_fail") or "wait")
if on_fail not in ON_DEP_FAIL:
on_fail = "wait"
st = {d: _st(nodes, d) for d in deps}
for c in (n.get("when") or []):
if not isinstance(c, dict):
continue
cnode = str(c.get("node") or "")
want = str(c.get("status") or "done")
got = _st(nodes, cnode)
if got == want:
continue
if got in SETTLED:
return {"verdict": "skip",
"why": "条件不成立:%s 是 %s,要 %s" % (cnode, got, want)}
if not deps:
return {"verdict": "wait", "why": "条件节点 %s 还没定论(%s)" % (cnode, got)}
if not deps:
return {"verdict": "go", "why": ""}
dead = [(d, s) for d, s in st.items() if s in ("skipped", "cancelled")] \
or [(d, s) for d, s in st.items() if s == "blocked"]
if rule == "all_done":
lack = [d for d, s in st.items() if s not in SETTLED]
if lack:
return {"verdict": "wait", "why": "依赖未跑完:%s" % ", ".join(lack)}
return {"verdict": "go", "why": ""}
if rule == "one_success":
if any(s == "done" for s in st.values()):
return {"verdict": "go", "why": ""}
if dead and all(s in SETTLED for s in st.values()):
return {"verdict": on_fail if on_fail != "wait" else "wait",
"why": "没有任一上游成功:%s"
% ", ".join("%s(%s)" % (d, s) for d, s in st.items())}
return {"verdict": "wait",
"why": "等待任一上游成功:%s"
% ", ".join("%s(%s)" % (d, s) for d, s in st.items())}
lack = [d for d, s in st.items() if s != "done"]
if not lack:
return {"verdict": "go", "why": ""}
if dead:
why = "上游不会成功了:%s" % ", ".join("%s(%s)" % (d, s) for d, s in dead)
if on_fail != "wait":
return {"verdict": on_fail, "why": why}
return {"verdict": "wait",
"why": "依赖未完成:%s" % ", ".join("%s(%s)" % (d, st[d]) for d in lack)}
return {"verdict": "wait",
"why": "依赖未完成:%s" % ", ".join("%s(%s)" % (d, st[d]) for d in lack)}
def _deps_ok(nodes: dict, n: dict):
"""按 `deps_rule` 判依赖是否放行,返回 (bool, 没满足的原因)。"""
v = _dep_verdict(nodes, n)
return v["verdict"] == "go", v["why"]
# ── 三件批量能力的配置读法 ──────────────────────────────────────────────
def _meta(g: dict) -> dict:
m = g.get("meta")
return m if isinstance(m, dict) else {}
def _lease_cfg(g: dict) -> dict:
d = _meta(g).get("lease")
d = d if isinstance(d, dict) else {}
try:
sec = float(d.get("timeout_sec", DEFAULT_LEASE_SEC))
except (TypeError, ValueError):
sec = DEFAULT_LEASE_SEC
pol = str(d.get("on_timeout") or "requeue")
return {"timeout_sec": sec, "on_timeout": pol if pol in TIMEOUT_POLICIES else "requeue"}
def _node_lease_sec(g: dict, n: dict) -> float:
"""节点级 `lease_sec` 优先于全局;0 表示该节点不判超时。"""
v = n.get("lease_sec")
if v in (None, ""):
return _lease_cfg(g)["timeout_sec"]
try:
return float(v)
except (TypeError, ValueError):
return _lease_cfg(g)["timeout_sec"]
def _retry_cfg(g: dict, n: dict) -> dict:
d = {}
for src in (_meta(g).get("retry"), n.get("retry")):
if isinstance(src, dict):
d.update(src)
def _num(k, dv):
try:
return float(d.get(k, dv))
except (TypeError, ValueError):
return float(dv)
bk = str(d.get("backoff") or "fixed")
return {"max": max(0, int(_num("max", 0))),
"delay_sec": max(0.0, _num("delay_sec", 0)),
"backoff": bk if bk in BACKOFFS else "fixed",
"factor": max(1.0, _num("factor", 2))}
def _delay_of(rc: dict, attempts: int) -> float:
k = max(1, int(attempts))
if rc["backoff"] == "linear":
return rc["delay_sec"] * k
if rc["backoff"] == "exponential":
return rc["delay_sec"] * (rc["factor"] ** (k - 1))
return rc["delay_sec"]
# ── 校验 ────────────────────────────────────────────────────────────────
def _find_cycle(nodes: dict) -> list:
color, stack, hit = {}, [], []
def dfs(u):
color[u] = 1
stack.append(u)
for v in _deps(nodes.get(u) or {}):
if v not in nodes:
continue
if color.get(v) == 1:
i = stack.index(v)
hit.extend(stack[i:] + [v])
return True
if color.get(v, 0) == 0 and dfs(v):
return True
stack.pop()
color[u] = 2
return False
for k in nodes:
if color.get(k, 0) == 0 and dfs(k):
return hit
return []
def _ref_graph(ws: str, gid: str):
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
return None
try:
return json.loads(p.read_text(encoding="utf-8"))
except Exception: # noqa: BLE001
return None
def _check_sg_ref(ws: str, chain: list, nid: str, ref: str, errs: list,
depth: int = 0) -> list:
"""校验 `subgraph.gid`:存在性/自引用/跨图成环/不许嵌套;返回子图节点规格。"""
if ref in chain:
errs.append("%s 的 subgraph 引用成环:%s" % (nid, " → ".join(chain + [ref])))
return []
if depth > 6:
errs.append("%s 的 subgraph 引用链超过 6 层(gid=%s)" % (nid, ref))
return []
rg = _ref_graph(ws, ref)
if rg is None:
errs.append("%s 的 subgraph 引用的图不存在:gid=%s(%s)"
% (nid, ref, _gdir(ws, ref) / "graph.json"))
return []
rnodes = {k: v for k, v in (rg.get("nodes") or {}).items() if isinstance(v, dict)}
out = []
for sid, sn in rnodes.items():
if sn.get("map") or sn.get("subgraph"):
errs.append("%s 引用的图 %s 里节点 %s 又声明了 map/subgraph ⇒ 不支持嵌套"
% (nid, ref, sid))
out.append({"id": sid, "deps": [str(x) for x in (sn.get("deps") or [])]})
for sid, sn in rnodes.items():
sub2 = sn.get("subgraph")
g2 = str((sub2 or {}).get("gid") or "").strip() if isinstance(sub2, dict) else ""
if g2:
_check_sg_ref(ws, chain + [ref], "%s/%s" % (nid, sid), g2, errs, depth + 1)
return out
def cmd_check(a) -> int:
g = _load(a.ws, a.gid)
nodes = g.get("nodes") or {}
errs, warns = [], []
cyc = _find_cycle(nodes)
if cyc:
errs.append("存在环:%s" % " → ".join(cyc))
for nid, n in nodes.items():
for d in _deps(n):
if d not in nodes:
errs.append("%s 的依赖 %s 不存在" % (nid, d))
if str(n.get("deps_rule") or "all_success") not in DEP_RULES:
errs.append("%s 的 deps_rule 非法:%r(只认 %s)"
% (nid, n.get("deps_rule"), " / ".join(DEP_RULES)))
m = n.get("map")
if m is not None:
if not isinstance(m, dict) or not ("over" in m or "from" in m):
errs.append("%s 的 map 需要 over(建图给列表)或 from(运行时读上游)" % nid)
else:
src = (m.get("from") or {}).get("node") if isinstance(m.get("from"), dict) else None
if src and src not in nodes:
errs.append("%s 的 map.from.node %s 不存在" % (nid, src))
elif src and src not in _deps(n):
warns.append("%s 的展开源 %s 不在 deps 里 ⇒ 上游没完成时会被当成"
"「展开源未就绪」空等" % (nid, src))
if n.get("map") and n.get("subgraph"):
errs.append("%s 同时声明了 map 与 subgraph(只能选一个)" % nid)
sg = n.get("subgraph")
if sg is not None:
kids = sg.get("nodes") if isinstance(sg, dict) else None
ref = str((sg or {}).get("gid") or "").strip() if isinstance(sg, dict) else ""
if isinstance(kids, list) and ref:
errs.append("%s 的 subgraph 同时给了 nodes 与 gid(只能选一种)" % nid)
if not isinstance(kids, list) and not ref:
errs.append('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "..."}' % nid)
kids = []
if ref:
kids = _check_sg_ref(a.ws, [a.gid], nid, ref, errs)
extra = [s["id"] for s in kids]
for s in kids:
for d in s["deps"]:
if d not in extra:
errs.append("%s 引用的图 %s 里 %s 依赖 %s 不在那张图里"
% (nid, ref, s["id"], d))
seen = set()
for s in kids:
sid = str((s or {}).get("id") or "")
if not sid:
warns.append("%s 的子图节点缺 id ⇒ 会按顺序补 S1/S2,deps 引用会指不到"
% nid)
elif sid in seen:
errs.append("%s 的子图里有重复 id:%s" % (nid, sid))
else:
seen.add(sid)
for s in kids:
for d in ((s or {}).get("deps") or []):
if str(d) not in seen and str(d) not in nodes:
errs.append("%s/%s 的依赖 %s 既不在子图里也不在图里"
% (nid, (s or {}).get("id") or "?", d))
if str(n.get("on_dep_fail") or "wait") not in ON_DEP_FAIL:
errs.append("%s 的 on_dep_fail 非法:%r(只认 %s)"
% (nid, n.get("on_dep_fail"), " / ".join(ON_DEP_FAIL)))
for c in (n.get("when") or []):
if not isinstance(c, dict) or not str(c.get("node") or ""):
errs.append('%s 的 when 子句要写 {"node": "A", "status": "done"}' % nid)
continue
if str(c["node"]) not in nodes:
errs.append("%s 的 when 条件节点 %s 不存在" % (nid, c["node"]))
if str(c.get("status") or "done") not in STATUSES:
errs.append("%s 的 when 期望状态非法:%r" % (nid, c.get("status")))
if n.get("expanded"):
for cid in (n.get("children") or []):
if cid not in nodes:
errs.append("%s 的子节点 %s 不见了(图被手改过?)" % (nid, cid))
if not _norm_dom(n.get("domain")) and not n.get("parent"):
warns.append("%s 没有声明域 ⇒ 域比较视为不冲突,可与任意节点同批;"
"保守做法是让它单独跑" % nid)
if str(n.get("status")) not in STATUSES:
errs.append("%s 的状态非法:%r" % (nid, n.get("status")))
ap = _accepts_problem(n.get("accepts"))
if ap:
errs.append("%s:%s" % (nid, ap))
rt = _rework_target(n)
if rt:
if rt not in nodes:
errs.append("%s 的 rework_to 指向不存在的节点:%s" % (nid, rt))
elif rt == nid:
errs.append("%s 的 rework_to 指向自己" % nid)
elif rt not in _ancestors(nodes, nid):
errs.append("%s 的 rework_to %s 不是它的上游(回退只能回到上游)"
% (nid, rt))
if not (n.get("accepts") or []) and not n.get("map") and not n.get("parent"):
warns.append("%s 没有验收判据 ⇒ 判不出完成" % nid)
pools = _meta(g).get("pools")
if pools is not None:
if not isinstance(pools, dict):
errs.append('meta.pools 要写成 {"域前缀": 并发上限}')
else:
doms = [(_norm_dom(n.get("domain")), nid) for nid, n in nodes.items()]
for k, v in pools.items():
try:
cap = int(v)
except (TypeError, ValueError):
errs.append("meta.pools[%s] 不是数字:%r" % (k, v))
continue
if cap <= 0:
errs.append("meta.pools[%s] 必须是正整数" % k)
pre = _norm_dom(k)
if not pre:
errs.append("meta.pools 的域不能为空")
elif not any(d and _dom_overlap(pre, d) for d, _n in doms):
warns.append("域池 %s 没有匹配任何节点的域" % k)
ids = list(nodes)
for i in range(len(ids)):
for j in range(i + 1, len(ids)):
a1, a2 = ids[i], ids[j]
if _norm_dom(nodes[a1].get("domain")) and \
_dom_overlap(_norm_dom(nodes[a1].get("domain")),
_norm_dom(nodes[a2].get("domain"))):
warns.append("%s 与 %s 的域重叠 ⇒ 永不同批" % (a1, a2))
_out({"ok": not errs, "errors": errs, "warnings": warns,
"nodes": len(nodes)})
return 0 if not errs else 1
# ── 租约超时回收 ────────────────────────────────────────────────────────
def _lease_over(g: dict, n: dict, now: float):
"""超时则返回超时秒数,否则 None。没有租约时刻 ⇒ 不判超时(兼容手改的旧图)。"""
if str(n.get("status")) != "running":
return None
sec = _node_lease_sec(g, n)
if sec <= 0:
return None
ts = _parse_ts(n.get("lease_at_ts"))
if ts is None:
ts = _parse_ts(n.get("lease_at"))
if ts is None:
return None
over = now - ts - sec
return over if over > 0 else None
def _reclaim_leases(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
lc = _lease_cfg(g)
now = time.time()
changed = 0
for nid, n in list(nodes.items()):
over = _lease_over(g, n, now)
if over is None:
continue
pol = str(n.get("on_timeout") or lc["on_timeout"])
if pol == "keep":
continue # 只提示,不动(notice 由外层补)
n["attempts"] = int(n.get("attempts") or 0) + 1
n["reclaimed_at"] = _now()
n["reclaim_count"] = int(n.get("reclaim_count") or 0) + 1
for k in ("lease_at", "lease_at_ts", "lease_sec"):
n.pop(k, None)
if pol == "block":
n["status"] = "blocked"
n["blocked_by"] = "lease_timeout"
else:
n["status"] = "pending"
acts.append({"node": nid, "event": "lease_timeout", "to": n["status"],
"over_sec": int(over), "attempts": n["attempts"],
"policy": pol})
changed += 1
return changed
def _lease_notices(g: dict) -> list:
lc = _lease_cfg(g)
if lc["on_timeout"] != "keep":
return []
now = time.time()
out = []
for nid, n in (g.get("nodes") or {}).items():
over = _lease_over(g, n, now)
if over is not None:
out.append({"node": nid, "over_sec": int(over),
"why": "租约超时但策略为 keep(只提示,不回收)"})
return out
# ── 动态展开(map / foreach)─────────────────────────────────────────────
def _as_text(x) -> str:
if isinstance(x, (dict, list)):
return json.dumps(x, ensure_ascii=False)
return str(x)
def _parse_list(v) -> list:
"""证据/产物 → 列表。JSON 数组优先,否则按行与分号切。"""
if isinstance(v, list):
return list(v)
s = str(v or "").strip()
if not s:
return []
if s[0] in "[(":
try:
got = json.loads(s)
if isinstance(got, list):
return list(got)
except Exception: # noqa: BLE001
pass
out = []
for line in s.replace("\r\n", "\n").replace(";", "\n").replace(";", "\n").split("\n"):
t = line.strip().lstrip("-*").strip()
if t:
out.append(t)
return out
def _map_items(g: dict, nodes: dict, nid: str):
"""取展开项;源还没就绪 ⇒ None(区别于空列表 = N=0)。"""
n = nodes.get(nid) or {}
m = n.get("map")
if not isinstance(m, dict):
return None
if "over" in m:
items = list(m.get("over") or [])
else:
f = m.get("from") or {}
up = nodes.get(str(f.get("node") or ""))
if not up or str(up.get("status")) != "done":
return None
pick = str(f.get("pick") or "artifacts")
if pick == "evidence":
items = _parse_list(up.get("evidence"))
elif pick == "children":
items = list(up.get("children") or [])
else:
items = list(up.get("artifacts") or [])
cap = int(m.get("max_items") or 0)
if cap > 0:
items = items[:cap]
return items
def _child_domain(m: dict, i: int, item) -> list:
cd = m.get("child_domain")
if not cd:
return []
if isinstance(cd, str):
cd = [cd]
out = []
for x in cd:
s = str(x).replace("{i}", str(i)).replace("{item}", _as_text(item))
out.append(s)
return _norm_dom(out)
def _expand_one(g: dict, nodes: dict, nid: str, items: list) -> dict:
n = nodes[nid]
if items and int((n.get("map") or {}).get("max_items") or 0) > 0:
items = items[:int((n.get("map") or {}).get("max_items"))]
n["expanded"] = True
n["expanded_at"] = _now()
n["item_count"] = len(items)
if not items: # N=0 ⇒ 无需执行,父节点直接完成
n["status"] = "done"
n["children"] = []
n["evidence"] = n.get("evidence") or "展开 0 项,无需执行"
return {"node": nid, "event": "expand", "count": 0, "children": [],
"to": "done"}
kids = []
for i, it in enumerate(items, 1):
cid = "%s#%d" % (nid, i)
if cid in nodes:
raise SystemExit("子节点 id 已被占用:%s(父节点 %s)" % (cid, nid))
title = "%s[%d]" % (str(n.get("title") or nid), i)
if not isinstance(it, (dict, list)):
title = "%s %s" % (title, str(it)[:40])
nodes[cid] = {
"title": title,
"deps": _deps(n),
"domain": _child_domain(n.get("map") or {}, i, it),
"accepts": list(n.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"parent": nid, "index": i, "item": it,
}
kids.append(cid)
n["status"] = "gating"
n["children"] = kids
return {"node": nid, "event": "expand", "count": len(kids), "children": kids,
"to": "gating"}
def _expand_maps(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
st = str(n.get("status"))
if st not in ("pending", "blocked") or n.get("expanded") or not n.get("map"):
continue
if st == "blocked":
continue # 失败节点不自动展开,等人工 start/重试
okd, _why = _deps_ok(nodes, n)
if not okd:
continue
items = _map_items(g, nodes, nid)
if items is None:
continue
acts.append(_expand_one(g, nodes, nid, items))
changed += 1
return changed
def _complete_parents(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "gating":
continue
kids = [c for c in (n.get("children") or []) if c in nodes]
if not kids or not all(_st(nodes, c) in SETTLED for c in kids):
continue
arts = []
for c in kids:
arts.extend(list((nodes.get(c) or {}).get("artifacts") or []))
if arts:
n["artifacts"] = arts
if all(_is_done(nodes, c) for c in kids):
n["status"] = "done"
n["completed_at"] = _now()
n["evidence"] = n.get("evidence") or ("展开 %d 项全部完成" % len(kids))
acts.append({"node": nid, "event": "children_done", "to": "done",
"count": len(kids)})
else: # 子节点全定下来了,但不全成功 ⇒ 父节点不算成功
bad = ["%s(%s)" % (c, _st(nodes, c)) for c in kids if not _is_done(nodes, c)]
n["status"] = "blocked"
n["blocked_by"] = "child_failed"
n["blocked_reason"] = "子节点未全部成功:%s" % ", ".join(bad)
acts.append({"node": nid, "event": "children_failed", "to": "blocked",
"count": len(kids), "bad": bad})
changed += 1
return changed
# ── 条件分支(when / on_dep_fail)───────────────────────────────────────
def _apply_conditions(g: dict, acts: list) -> int:
"""把「条件不成立 ⇒ skipped」「上游跑不成且要求 block ⇒ blocked」落到状态上。"""
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "pending":
continue
if n.get("map") and not n.get("expanded"):
continue
if n.get("subgraph") and not n.get("expanded"):
continue
v = _dep_verdict(nodes, n)
if v["verdict"] == "skip":
n["status"] = "skipped"
n["skipped_at"] = _now()
n["skipped_reason"] = v["why"]
acts.append({"node": nid, "event": "skip", "to": "skipped",
"why": v["why"]})
changed += 1
elif v["verdict"] == "block":
n["status"] = "blocked"
n["blocked_by"] = "dep"
n["blocked_reason"] = v["why"]
acts.append({"node": nid, "event": "block", "to": "blocked",
"why": v["why"]})
changed += 1
return changed
# ── 子图(节点内部再拆子流程)───────────────────────────────────────────
def _expand_subgraphs(g: dict, acts: list, ws: str = "") -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "pending" or n.get("expanded") or not n.get("subgraph"):
continue
if not _deps_ok(nodes, n)[0]:
continue
acts.append(_expand_subgraph(g, nodes, nid, ws))
changed += 1
return changed
def _expand_subgraph(g: dict, nodes: dict, nid: str, ws: str = "") -> dict:
"""子图 ⇒ 子节点 `<父>/<子id>`,父节点转 gating,子节点全成功才 done。
`subgraph.nodes` 内联;`subgraph.gid` 引用同工作区另一张图的节点定义。
"""
n = nodes[nid]
sub = n.get("subgraph") or {}
ref_gid = str(sub.get("gid") or "").strip()
if ref_gid:
if ref_gid == str(g.get("gid") or ""):
raise SystemExit("子图引用了当前图自己:%s" % ref_gid)
par = n.get("parent")
chain = list(reversed(_sg_chain(g, par))) if par else []
specs = _sg_specs_from_gid(ws, ref_gid, chain)
n["subgraph_ref"] = ref_gid
else:
specs = [s for s in (sub.get("nodes") or []) if isinstance(s, dict)]
prefix = nid + "/"
local, kids = {}, []
for i, s in enumerate(specs, 1):
raw = str(s.get("id") or ("S%d" % i)).strip()
cid = prefix + raw
if cid in nodes:
raise SystemExit("子图节点 id 已被占用:%s" % cid)
local[raw] = cid
kids.append((cid, s))
n["expanded"] = True
n["expanded_at"] = _now()
if not kids: # 空子图 ⇒ 无需执行
n["status"] = "done"
n["children"] = []
n["evidence"] = n.get("evidence") or "子图为空,无需执行"
return {"node": nid, "event": "expand", "kind": "subgraph", "count": 0, "to": "done"}
base = _norm_dom(n.get("domain")) or _norm_dom(sub.get("child_domain"))
outer = _deps(n)
for cid, s in kids:
inner = []
for d in (s.get("deps") or []):
tgt = local.get(str(d)) or str(d)
if tgt not in inner:
inner.append(tgt)
deps = list(outer) + [d for d in inner if d not in outer]
dom = _norm_dom(s.get("domain"))
if not dom:
cd = sub.get("child_domain")
dom = _norm_dom(str(cd).replace("{id}", cid[len(prefix):])) if cd else list(base)
kid = {"title": str(s.get("title") or cid), "deps": deps, "domain": dom,
"accepts": _norm_accepts(s.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [],
"attempts": 0, "parent": nid, "sub": True}
for k in ("deps_rule", "on_dep_fail", "approval", "lease_sec"):
if s.get(k) not in (None, "", []):
kid[k] = s[k]
# `when` / `rework_to` 里的节点引用要跟着加父前缀,否则指向不存在的 id
if s.get("when"):
kid["when"] = [{"node": local.get(str(c.get("node")), str(c.get("node"))),
"status": str(c.get("status") or "done")}
for c in s["when"] if isinstance(c, dict)]
rt = _rework_target(s)
if rt:
kid["rework_to"] = local.get(rt, rt)
if s.get("retry"):
kid["retry"] = _retry_cfg({"meta": {}}, {"retry": s["retry"]})
if ref_gid:
kid["origin_graph"] = ref_gid
nodes[cid] = kid
n["status"] = "gating"
n["children"] = [c for c, _ in kids]
return {"node": nid, "event": "expand", "kind": "subgraph",
"count": len(kids), "children": n["children"], "to": "gating",
**({"from_graph": ref_gid} if ref_gid else {})}
# ── 按域分池 ────────────────────────────────────────────────────────────
def _pool_caps(g: dict) -> list:
p = _meta(g).get("pools")
out = []
if isinstance(p, dict):
for k, v in p.items():
try:
cap = int(v)
except (TypeError, ValueError):
continue
dom = _norm_dom(k)
if cap > 0 and dom:
out.append((dom[0], cap))
return out
def _pool_of(g: dict, n: dict) -> list:
dom = _norm_dom((n or {}).get("domain"))
if not dom:
return []
return [pre for pre, _c in _pool_caps(g) if _dom_overlap(dom, [pre])]
def _pool_full(g: dict, n: dict, occupied: list) -> dict:
"""节点所在池是否已满;满了返回 {"pool","cap","used"},否则 {}。"""
caps = dict(_pool_caps(g))
for pre in _pool_of(g, n):
cap = caps.get(pre) or 0
if cap <= 0:
continue
used = sum(1 for x in occupied if pre in _pool_of(g, x))
if used >= cap:
return {"pool": pre, "cap": cap, "used": used}
return {}
def _advance(g: dict, ws: str = "") -> list:
"""把「超时回收/子节点收敛/展开/条件判定」推到不动点;返回动了哪些(供写事件)。"""
acts: list = []
for _ in range(6):
moved = 0
moved += _reclaim_leases(g, acts)
moved += _complete_parents(g, acts)
moved += _expand_maps(g, acts)
moved += _expand_subgraphs(g, acts, ws)
moved += _apply_conditions(g, acts)
if not moved:
break
return acts
def _maybe_advance(g: dict) -> bool:
"""便宜的预判:没有可推进的事就不写盘(保持 ready 的只读语义)。"""
nodes = g.get("nodes") or {}
lc = _lease_cfg(g)
now = time.time()
if lc["on_timeout"] != "keep" and lc["timeout_sec"] > 0:
for n in nodes.values():
if _lease_over(g, n, now) is not None:
return True
for nid, n in nodes.items():
st = str(n.get("status"))
if st == "gating":
kids = [c for c in (n.get("children") or []) if c in nodes]
if kids and all(_st(nodes, c) in SETTLED for c in kids):
return True
elif st == "pending":
if n.get("expanded"):
continue
if n.get("map"):
if _deps_ok(nodes, n)[0] and _map_items(g, nodes, nid) is not None:
return True
continue
if n.get("subgraph"):
if _deps_ok(nodes, n)[0]:
return True
continue
v = _dep_verdict(nodes, n)
if v["verdict"] in ("skip", "block"):
return True
return False
def _advance_graph(ws: str, gid: str, g: dict):
acts: list = []
if not _maybe_advance(g):
return g, acts
def _mut(gg):
acts.extend(_advance(gg, ws))
g2 = _update(ws, gid, _mut)
for x in acts:
_event(ws, gid, x)
return g2, acts
# ── 就绪集 ──────────────────────────────────────────────────────────────
def _ready(g: dict) -> dict:
nodes = g.get("nodes") or {}
cap = int(_meta(g).get("max_parallel") or 3)
running = [k for k, n in nodes.items() if str(n.get("status")) == "running"]
ready, waiting = [], []
now = time.time()
for nid, n in nodes.items():
st = str(n.get("status"))
if st == "gating":
kids = [c for c in (n.get("children") or []) if c in nodes]
left = [c for c in kids if not _is_done(nodes, c)]
waiting.append({"node": nid, "why": "等待 %d 个子节点完成(还剩 %d)"
% (len(kids), len(left))})
continue
if st != "pending":
continue
if n.get("map") and not n.get("expanded"):
waiting.append({"node": nid, "why": "展开源未就绪(上游没完成或没有产物)"})
continue
nb = n.get("not_before")
if nb not in (None, ""):
try:
left = float(nb) - now
except (TypeError, ValueError):
left = 0
if left > 0:
waiting.append({"node": nid, "why": "重试等待中:还需 %d 秒"
% int(math.ceil(left))})
continue
okd, why = _deps_ok(nodes, n)
if not okd:
waiting.append({"node": nid, "why": why})
continue
dom = _norm_dom(n.get("domain"))
clash = [r for r in running
if dom and _norm_dom((nodes.get(r) or {}).get("domain"))
and _dom_overlap(dom, _norm_dom((nodes.get(r) or {}).get("domain")))]
if clash:
waiting.append({"node": nid, "why": "域与在跑的冲突:%s" % ", ".join(clash)})
continue
pf = _pool_full(g, n, [nodes.get(r) or {} for r in running])
if pf:
waiting.append({"node": nid, "why": "域池 %s 已满(%d/%d)"
% (pf["pool"], pf["used"], pf["cap"])})
continue
ready.append(nid)
slots = max(0, cap - len(running))
picked, backlog = [], []
for nid in ready:
if len(picked) >= slots:
backlog.append(nid)
waiting.append({"node": nid, "why": "并发上限 %d 已占满" % cap})
continue
dom = _norm_dom((nodes.get(nid) or {}).get("domain"))
clash = [p for p in picked
if dom and _norm_dom((nodes.get(p) or {}).get("domain"))
and _dom_overlap(dom, _norm_dom((nodes.get(p) or {}).get("domain")))]
if clash:
# 同批也不能撞域:check 已警告「永不同批」,ready 必须兑现
waiting.append({"node": nid, "why": "域与本批已选节点重叠:%s" % ", ".join(clash)})
continue
pf = _pool_full(g, nodes.get(nid) or {},
[nodes.get(r) or {} for r in running] +
[nodes.get(p) or {} for p in picked])
if pf:
waiting.append({"node": nid, "why": "域池 %s 已满(%d/%d)"
% (pf["pool"], pf["used"], pf["cap"])})
continue
picked.append(nid)
awaiting = [k for k, n in nodes.items() if str(n.get("status")) == "awaiting"]
return {"gid": g.get("gid"), "running": running, "ready": picked,
"backlog": backlog, "waiting": waiting, "awaiting": awaiting,
"pools": dict(_pool_caps(g)),
"cap": cap, "slots": slots}
def cmd_ready(a) -> int:
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
r["reclaimed"] = [x for x in acts if x.get("event") == "lease_timeout"]
r["expanded"] = [x for x in acts if x.get("event") == "expand"]
r["completed_parents"] = [x["node"] for x in acts if x.get("event") == "children_done"]
r["lease_notices"] = _lease_notices(g)
_out(r)
return 0
# ── 状态流转 ────────────────────────────────────────────────────────────
def cmd_start(a) -> int:
def _mut(g):
n = (g.get("nodes") or {}).get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
st = str(n.get("status"))
if st == "running":
raise SystemExit("节点 %s 已在跑,不能重复开工" % a.node)
if st == "done":
raise SystemExit("节点 %s 已完成,不能开工" % a.node)
if st == "gating" or (n.get("map") and not n.get("expanded")):
raise SystemExit("节点 %s 是展开节点,不能直接开工(由 ready 展开后跑子节点)"
% a.node)
# pending 与 blocked 都可开工:blocked 是「这轮没过」,重试时 attempts 保留
n["status"] = "running"
now = time.time()
n["lease_at"] = _iso(now)
n["lease_at_ts"] = now
if a.lease_sec is not None:
n["lease_sec"] = float(a.lease_sec)
n["starts"] = int(n.get("starts") or 0) + 1
n.pop("not_before", None)
n.pop("retry_at", None)
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"node": a.node, "to": "running", "by": a.by or ""})
n = (g.get("nodes") or {}).get(a.node) or {}
_out({"ok": True, "node": a.node, "status": "running",
"lease_sec": _node_lease_sec(g, n)})
return 0
def cmd_report(a) -> int:
if a.state not in ("done", "blocked"):
raise SystemExit("--state 只认 done / blocked")
retry_info = {}
def _mut(g):
n = (g.get("nodes") or {}).get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
if n.get("map") or str(n.get("status")) == "gating":
raise SystemExit("节点 %s 是展开父节点,本身不回报(子节点全完成自动 done)"
% a.node)
if a.evidence:
n["evidence"] = a.evidence
for art in (a.artifact or []):
n.setdefault("artifacts", []).append(str(art))
if a.state == "done": # F1:报 done 先过验收判据
gaps = _accepts_missing(n.get("accepts"), n.get("artifacts"), n.get("evidence"))
if gaps:
raise SystemExit("节点 %s 验收不通过(不置 done):\n - %s\n"
"补齐产物/证据后重新 report(accepts 不可绕过)"
% (a.node, "\n - ".join(gaps)))
n["accepts_checked_at"] = _now()
n["accepts_gaps"] = []
for k in ("lease_at", "lease_at_ts", "lease_sec"):
n.pop(k, None)
if a.state == "blocked":
n["attempts"] = int(n.get("attempts") or 0) + 1
n["blocked_by"] = "report"
rc = _retry_cfg(g, n)
left = rc["max"] - n["attempts"] + 1 # 还能再试几次
if rc["max"] > 0 and n["attempts"] <= rc["max"]:
delay = _delay_of(rc, n["attempts"])
at = time.time() + delay
n["status"] = "pending"
n["not_before"] = at
n["retry_at"] = _iso(at)
n["retry_exhausted"] = False
retry_info.update({"policy_max": rc["max"], "attempts": n["attempts"],
"requeued": True, "delay_sec": delay,
"retry_at": n["retry_at"],
"retries_left": max(0, left - 1)})
else:
n["status"] = "blocked"
n["retry_exhausted"] = rc["max"] > 0
n.pop("not_before", None)
n.pop("retry_at", None)
retry_info.update({"policy_max": rc["max"], "attempts": n["attempts"],
"requeued": False, "retry_at": None,
"exhausted": True})
else:
n.pop("not_before", None)
n.pop("retry_at", None)
if n.get("approval") and not n.get("approved_at"):
n["status"] = "awaiting" # 跑完了,等人审过目
n["awaiting_at"] = _now()
else:
n["status"] = "done"
n["done_at"] = _now()
g = _update(a.ws, a.gid, _mut)
final = _st(g.get("nodes") or {}, a.node)
_event(a.ws, a.gid, {"node": a.node,
"to": final if final == "awaiting"
else (a.state if a.state == "done" else "blocked"),
"evidence": a.evidence or "", "by": a.by or "",
"artifacts": list(a.artifact or [])})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node,
"status": str((g.get("nodes") or {}).get(a.node, {}).get("status")),
"next_ready": r["ready"],
"retry": retry_info or None,
"completed_parents": [x["node"] for x in acts
if x.get("event") == "children_done"]})
return 0
def cmd_expand(a) -> int:
acts = []
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
if not n.get("map"):
raise SystemExit("节点 %s 没有 map 声明,无可展开" % a.node)
if n.get("expanded"):
raise SystemExit("节点 %s 已展开过(children=%s)"
% (a.node, n.get("children")))
okd, why = _deps_ok(nodes, n)
if not okd and not a.force:
raise SystemExit("展开源未就绪:%s(要强制展开加 --force)" % why)
items = _map_items(g, nodes, a.node)
if items is None:
raise SystemExit("展开源未就绪:上游节点没完成或没有产物")
acts.append(_expand_one(g, nodes, a.node, items))
g = _update(a.ws, a.gid, _mut)
for x in acts:
_event(a.ws, a.gid, x)
g, more = _advance_graph(a.ws, a.gid, g)
acts.extend(more)
n = (g.get("nodes") or {}).get(a.node) or {}
_out({"ok": True, "node": a.node, "status": n.get("status"),
"children": n.get("children") or [],
"count": len(n.get("children") or [])})
return 0
# ── 人审 / 取消 / 派活 / 排期 / 进度 ─────────────────────────────────────
def cmd_approve(a) -> int:
"""人审节点:`awaiting` ⇒ done(默认)或 blocked(--reject)。
驳回时若节点带 `rework_to` ⇒ 直接走回退(人审判不过 = 判不过)。
"""
rw = {}
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
st = str(n.get("status"))
if st != "awaiting":
raise SystemExit("节点 %s 不在等人审(当前 %s)" % (a.node, st))
if a.reject:
tgt = _rework_target(n)
if tgt:
if tgt not in nodes:
raise SystemExit("回退目标不存在:%s" % tgt)
if tgt not in _ancestors(nodes, a.node):
raise SystemExit("回退目标 %s 不是 %s 的上游" % (tgt, a.node))
n["status"] = "blocked"
n["blocked_by"] = "rejected"
n["blocked_reason"] = a.reason or "人审驳回"
n["rejected_at"] = _now()
rw.update(_rework_reset(nodes, a.node, tgt, a.reason or "人审驳回"))
rw["auto"] = True
return
n["status"] = "blocked"
n["blocked_by"] = "rejected"
n["blocked_reason"] = a.reason or "人审驳回"
n["rejected_at"] = _now()
else:
n["status"] = "done"
n["done_at"] = _now()
n["approved_at"] = _now()
n["approved_by"] = a.by or "user"
g = _update(a.ws, a.gid, _mut)
st = _st(g.get("nodes") or {}, a.node)
_event(a.ws, a.gid, {"node": a.node, "to": st, "by": a.by or "",
"reject": bool(a.reject), "reason": a.reason or "",
**({"rework": rw} if rw else {})})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node, "status": _st(g.get("nodes") or {}, a.node),
"next_ready": r["ready"], "rework": rw or None})
return 0
def cmd_cancel(a) -> int:
"""取消节点(连坐它的子节点);`--all` 取消整图。"""
if not a.node and not a.all:
raise SystemExit("要给 --node <节点> 或 --all")
hit: list = []
def _mut(g):
nodes = g.get("nodes") or {}
if a.node and a.node not in nodes:
raise SystemExit("没有这个节点:%s" % a.node)
stack = [a.node] if a.node else list(nodes)
while stack:
nid = stack.pop()
n = nodes.get(nid)
if not n or str(n.get("status")) in ("done", "cancelled"):
continue
n["status"] = "cancelled"
n["cancelled_at"] = _now()
if a.by:
n["cancelled_by"] = a.by
if a.reason:
n["cancel_reason"] = a.reason
hit.append(nid)
stack.extend(n.get("children") or [])
g = _update(a.ws, a.gid, _mut)
for nid in hit:
_event(a.ws, a.gid, {"node": nid, "to": "cancelled",
"by": a.by or "", "reason": a.reason or ""})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "cancelled": hit,
"skipped_by_cancel": [x["node"] for x in acts if x.get("event") == "skip"],
"next_ready": r["ready"]})
return 0
def cmd_rework(a) -> int:
"""回退到上游重做:目标及其全部传递下游复位 pending,记 attempts/reworks 与事件。"""
res = {}
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
tgt = str(a.to or "").strip() or _rework_target(n)
if not tgt:
raise SystemExit("节点 %s 没有 rework_to,也没给 --to" % a.node)
if tgt not in nodes:
raise SystemExit("回退目标不存在:%s" % tgt)
if tgt == a.node:
raise SystemExit("回退目标不能是自己(要重做自己直接 start 该节点)")
if tgt not in _ancestors(nodes, a.node):
raise SystemExit("回退目标 %s 不是 %s 的上游(回退只能回到上游)"
% (tgt, a.node))
res.update(_rework_reset(nodes, a.node, tgt, a.reason or ""))
res["node"] = a.node
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "rework", "node": a.node, "to": res.get("target"),
"reset": res.get("reset") or [], "reason": a.reason or "",
"by": a.by or "", "attempts": res.get("attempts"),
"reworks": res.get("reworks")})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node, "target": res.get("target"),
"reset": res.get("reset") or [],
"dropped_children": res.get("dropped_children") or [],
"attempts": res.get("attempts"),
"reworks": res.get("reworks"),
"skipped_by_rework": [x["node"] for x in acts if x.get("event") == "skip"],
"next_ready": r["ready"]})
return 0
def cmd_add_node(a) -> int:
"""运行期加节点(与并发的 start/report 共用同一把目录锁,不会互相覆盖)。"""
nid = str(a.node or "").strip()
if not nid:
raise SystemExit("--node 不能为空")
deps = [x.strip() for x in str(a.deps or "").replace(",", ",").split(",") if x.strip()]
rule = str(a.deps_rule or "all_success")
if rule not in DEP_RULES:
raise SystemExit("--deps-rule 只认 %s" % " / ".join(DEP_RULES))
accepts = _norm_accepts(a.accepts or [])
rework_to = str(a.rework_to or "").strip()
def _mut(g):
nodes = g.get("nodes") or {}
if nid in nodes:
raise SystemExit("节点已存在:%s(换一个 id 或先 remove-edge)" % nid)
miss = [d for d in deps if d not in nodes]
if miss:
raise SystemExit("依赖不存在:%s" % ", ".join(miss))
if rework_to and rework_to not in nodes:
raise SystemExit("rework_to 指向的节点不存在:%s" % rework_to)
if rework_to == nid:
raise SystemExit("rework_to 不能指向自己")
nodes[nid] = {
"title": str(a.title or nid), "deps": deps,
"domain": _norm_dom(a.domain or []), "accepts": accepts,
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": rule, "added_at": _now(),
}
if rework_to:
nodes[nid]["rework_to"] = rework_to
if a.approval:
nodes[nid]["approval"] = True
g["nodes"] = nodes
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "add_node", "node": nid, "deps": deps,
"by": a.by or "", "rev": g.get("_rev")})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": nid, "deps": deps,
"status": _st(g.get("nodes") or {}, nid),
"rev": g.get("_rev"), "next_ready": r["ready"]})
return 0
def cmd_remove_edge(a) -> int:
"""删边:删掉「--to 依赖 --from」这一条依赖(边不存在即报错,⛔ 不静默空操作)。"""
src, dst = str(a.src or "").strip(), str(a.to or "").strip()
if not src or not dst:
raise SystemExit("要给 --from <上游> 和 --to <下游>")
def _mut(g):
nodes = g.get("nodes") or {}
for x in (src, dst):
if x not in nodes:
raise SystemExit("没有这个节点:%s" % x)
d = _deps(nodes[dst])
if src not in d:
raise SystemExit("边不存在:%s → %s(%s 的 deps=%s 里没有 %s)"
% (src, dst, dst, d, src))
nodes[dst]["deps"] = [x for x in d if x != src]
nodes[dst]["edge_removed_at"] = _now()
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "remove_edge", "from": src, "to": dst,
"by": a.by or "", "rev": g.get("_rev")})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "from": src, "to": dst, "deps": _deps(
(g.get("nodes") or {}).get(dst) or {}), "rev": g.get("_rev"),
"next_ready": r["ready"]})
return 0
def cmd_tick(a) -> int:
"""自动派活:推进到不动点 → 算就绪集 → 直接把就绪节点开成 running。"""
started, missed = [], []
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
cand = list(r["ready"])
if a.limit:
cand = cand[:int(a.limit)]
if cand and not a.dry_run:
def _mut(gg):
nodes = gg.get("nodes") or {}
for nid in cand:
n = nodes.get(nid)
if not n or str(n.get("status")) != "pending":
missed.append(nid)
continue
if _dep_verdict(nodes, n)["verdict"] != "go":
missed.append(nid)
continue
now = time.time()
n["status"] = "running"
n["lease_at"] = _iso(now)
n["lease_at_ts"] = now
if a.lease_sec is not None:
n["lease_sec"] = float(a.lease_sec)
n["starts"] = int(n.get("starts") or 0) + 1
n.pop("not_before", None)
n.pop("retry_at", None)
n["started_by"] = a.by or "tick"
started.append(nid)
g = _update(a.ws, a.gid, _mut)
for nid in started:
_event(a.ws, a.gid, {"node": nid, "to": "running", "by": a.by or "tick"})
g, more = _advance_graph(a.ws, a.gid, g)
acts.extend(more)
r2 = _ready(g)
_out({"gid": a.gid, "started": started, "missed": missed,
"dry_run": bool(a.dry_run), "advanced": acts,
"ready_now": r2["ready"], "running": r2["running"],
"awaiting": r2["awaiting"], "cap": r2["cap"], "slots": r2["slots"]})
return 0
def cmd_plan(a) -> int:
"""出一行排期(四条硬规则已按规范形态写好),⛔ 不写宿主库。"""
ws = str(Path(a.ws).resolve()).replace("\\", "/")
g = _load(a.ws, a.gid)
r = _ready(g)
after = 240 if a.after_sec is None else int(a.after_sec)
at_ms = int((time.time() + after) * 1000)
scope = a.scope or (ws.rstrip("/").rsplit("/", 1)[-1] + "/")
row = {
"name": "[图模式派活]-%s-%s" % (a.gid, time.strftime("%Y%m%d-%H%M")),
"prompt": ("跑一次图模式派活:python %s tick --ws %s --gid %s --by tick;"
"本轮只做这一件事,做完即停。"
% (Path(__file__).resolve(), ws, a.gid)),
"cwds": ws,
"workspace_scope": scope,
"next_run_at": at_ms,
"scheduled_at": time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(at_ms / 1000.0)),
}
_out({"ok": True, "gid": a.gid, "rows": [row],
"ready": r["ready"], "running": r["running"],
"rules": {"cwds": "正斜杠绝对路径", "workspace_scope": "线名/",
"next_run_at": "毫秒 epoch", "scheduled_at": "ISO 本地时间"},
"note": "只产出行,⛔ 不写宿主库(图模式零侵入)"})
return 0
def _remaining_chain(nodes: dict) -> list:
"""未完成节点之间的最长依赖链(关键路径)。"""
todo = {k: v for k, v in nodes.items()
if str(v.get("status")) not in ("done", "skipped", "cancelled")}
memo = {}
def best(nid):
if nid in memo:
return memo[nid]
memo[nid] = [nid]
cand = [nid]
for d in _deps(todo[nid]):
if d in todo:
chain = best(d) + [nid]
if len(chain) > len(cand):
cand = chain
memo[nid] = cand
return cand
out: list = []
for nid in todo:
c = best(nid)
if len(c) > len(out):
out = c
return out
def cmd_progress(a) -> int:
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
nodes = g.get("nodes") or {}
cnt: dict = {}
for n in nodes.values():
s = str(n.get("status"))
cnt[s] = cnt.get(s, 0) + 1
active = len(nodes) - cnt.get("cancelled", 0) - cnt.get("skipped", 0)
done = cnt.get("done", 0)
percent = round(100.0 * done / active, 1) if active else 100.0
r = _ready(g)
cp = _remaining_chain(nodes)
_out({"gid": g.get("gid"), "title": g.get("title"), "total": len(nodes),
"by_status": cnt, "active": active, "done": done, "percent": percent,
"running": r["running"], "ready": r["ready"], "awaiting": r["awaiting"],
"blocked": [k for k, n in nodes.items() if str(n.get("status")) == "blocked"],
"skipped": [k for k, n in nodes.items() if str(n.get("status")) == "skipped"],
"cancelled": [k for k, n in nodes.items() if str(n.get("status")) == "cancelled"],
"critical_path": cp, "critical_len": len(cp),
"advanced": acts})
return 0
# ── 建图 ────────────────────────────────────────────────────────────────
def _gid_of(title: str) -> str:
safe = "".join(c for c in title if c not in '\\/:*?"<>|\r\n\t')[:24].strip()
return "%s-%s" % (time.strftime("%Y%m%d-%H%M"), safe or "graph")
def _norm_map(m) -> dict:
if not isinstance(m, dict):
raise SystemExit("map 必须是对象")
out = {}
if "over" in m:
if not isinstance(m["over"], list):
raise SystemExit("map.over 必须是列表")
out["over"] = list(m["over"])
if "from" in m:
f = m["from"]
if not isinstance(f, dict) or not str(f.get("node") or "").strip():
raise SystemExit('map.from 需要 {"node": "上游节点"} 形态')
out["from"] = {"node": str(f["node"]).strip(),
"pick": str(f.get("pick") or "artifacts")}
if not out:
raise SystemExit("map 需要 over(建图给列表)或 from(运行时读上游)")
if m.get("child_domain"):
out["child_domain"] = m["child_domain"]
if m.get("max_items"):
out["max_items"] = int(m["max_items"])
if m.get("as"):
out["as"] = str(m["as"])
return out
def cmd_init(a) -> int:
if a.nodes_json:
raw = sys.stdin.read() if a.nodes_json == "-" else \
Path(a.nodes_json).read_text(encoding="utf-8")
spec = json.loads(raw)
elif a.kpi:
spec = {"nodes": [
{"id": "N%d" % (i + 1), "title": k.strip(), "deps": [],
"domain": [], "accepts": [k.strip()]}
for i, k in enumerate([x for x in a.kpi.replace(";", ";").split(";") if x.strip()])
]}
else:
raise SystemExit("要给 --kpi 或 --nodes-json")
if a.on_timeout not in TIMEOUT_POLICIES:
raise SystemExit("--on-timeout 只认 %s" % " / ".join(TIMEOUT_POLICIES))
if a.retry_backoff not in BACKOFFS:
raise SystemExit("--retry-backoff 只认 %s" % " / ".join(BACKOFFS))
nodes = {}
for i, n in enumerate(spec.get("nodes") or []):
raw = n.get("id")
nid = str(raw).strip() if raw not in (None, "") else ""
if not nid:
k = i + 1
while ("N%d" % k) in nodes: # 自动编号要避开已占用的 id
k += 1
nid = "N%d" % k
if nid in nodes: # 显式重复 ⇒ 硬错,⛔ 不静默覆盖前一个
raise SystemExit("节点 id 重复:%s" % nid)
rule = str(n.get("deps_rule") or a.deps_rule or "all_success")
if rule not in DEP_RULES:
raise SystemExit("%s 的 deps_rule 非法:%r(只认 %s)"
% (nid, rule, " / ".join(DEP_RULES)))
nodes[nid] = {
"title": str(n.get("title") or nid),
"deps": [str(x) for x in (n.get("deps") or [])],
"domain": _norm_dom(n.get("domain")),
"accepts": _norm_accepts(n.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": rule,
}
if n.get("retry"):
if not isinstance(n["retry"], dict):
raise SystemExit("%s 的 retry 必须是对象" % nid)
nodes[nid]["retry"] = _retry_cfg({"meta": {}}, {"retry": n["retry"]})
if n.get("lease_sec") not in (None, ""):
nodes[nid]["lease_sec"] = float(n["lease_sec"])
if n.get("on_timeout"):
if str(n["on_timeout"]) not in TIMEOUT_POLICIES:
raise SystemExit("%s 的 on_timeout 非法:%r" % (nid, n["on_timeout"]))
nodes[nid]["on_timeout"] = str(n["on_timeout"])
if n.get("map"):
nodes[nid]["map"] = _norm_map(n["map"])
if n.get("subgraph"):
sg = n["subgraph"]
if not isinstance(sg, dict):
raise SystemExit('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "另一张图"}'
% nid)
has_nodes = isinstance(sg.get("nodes"), list)
has_gid = bool(str(sg.get("gid") or "").strip())
if has_nodes and has_gid:
raise SystemExit("%s 的 subgraph 只能选一种:内联 nodes 或引用 gid" % nid)
if not has_nodes and not has_gid:
raise SystemExit('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "..."}'
% nid)
if n.get("map"):
raise SystemExit("%s 同时声明了 map 与 subgraph(只能选一个)" % nid)
sub = {}
if has_nodes:
sub["nodes"] = [dict(s) for s in sg["nodes"] if isinstance(s, dict)]
else:
sub["gid"] = str(sg["gid"]).strip()
if sg.get("child_domain"):
sub["child_domain"] = str(sg["child_domain"])
nodes[nid]["subgraph"] = sub
if n.get("rework_to"):
rt = _rework_target({"rework_to": n["rework_to"]})
if not rt:
raise SystemExit("%s 的 rework_to 不能为空" % nid)
if rt == nid:
raise SystemExit("%s 的 rework_to 不能指向自己" % nid)
nodes[nid]["rework_to"] = rt
if n.get("when"):
if not isinstance(n["when"], list):
raise SystemExit("%s 的 when 必须是列表" % nid)
clauses = []
for c in n["when"]:
if not isinstance(c, dict) or not str(c.get("node") or "").strip():
raise SystemExit('%s 的 when 子句要写 {"node": "A", "status": "done"}'
% nid)
clauses.append({"node": str(c["node"]).strip(),
"status": str(c.get("status") or "done")})
nodes[nid]["when"] = clauses
if n.get("on_dep_fail"):
if str(n["on_dep_fail"]) not in ON_DEP_FAIL:
raise SystemExit("%s 的 on_dep_fail 非法:%r(只认 %s)"
% (nid, n["on_dep_fail"], " / ".join(ON_DEP_FAIL)))
nodes[nid]["on_dep_fail"] = str(n["on_dep_fail"])
if n.get("approval"):
nodes[nid]["approval"] = True
gid = a.gid or _gid_of(a.title)
g = {"gid": gid, "title": a.title, "created_at": _now(),
"meta": {"max_parallel": int(spec.get("max_parallel") or a.max_parallel or 3),
"lease": {"timeout_sec": float(a.lease_sec) if a.lease_sec is not None
else DEFAULT_LEASE_SEC,
"on_timeout": a.on_timeout},
"retry": {"max": int(a.retry_max or 0),
"delay_sec": float(a.retry_delay or 0),
"backoff": a.retry_backoff,
"factor": float(a.retry_factor or 2)}},
"nodes": nodes}
d = _gdir(a.ws, gid)
pools = {}
if isinstance(spec.get("pools"), dict):
pools.update(spec["pools"])
for item in (a.pool or []):
s = str(item)
if "=" not in s:
raise SystemExit("--pool 要写成 域前缀=上限(如 docs/ui/=1)")
k, v = s.split("=", 1)
pools[k.strip()] = v
if pools:
g["meta"]["pools"] = {}
for k, v in pools.items():
dom = _norm_dom(k)
if not dom:
raise SystemExit("域池的域不能为空")
try:
cap = int(v)
except (TypeError, ValueError):
raise SystemExit("域池上限不是整数:%s=%r" % (k, v))
if cap <= 0:
raise SystemExit("域池上限必须 ≥1:%s=%r" % (k, v))
g["meta"]["pools"][dom[0]] = cap
d.mkdir(parents=True, exist_ok=True)
_save(a.ws, gid, g)
_event(a.ws, gid, {"event": "init", "nodes": list(nodes)})
_out({"ok": True, "gid": gid, "dir": str(d), "nodes": len(nodes),
"note": "独立图:⛔ 不参与现有机制(不写 goal.json / tasks.json)"})
return 0
def cmd_show(a) -> int:
_out(_load(a.ws, a.gid))
return 0
def cmd_list(a) -> int:
root = _root(a.ws)
rows = []
if root.is_dir():
for d in sorted(root.iterdir()):
p = d / "graph.json"
if not p.is_file():
continue
try:
g = json.loads(p.read_text(encoding="utf-8"))
except Exception:
continue
nodes = g.get("nodes") or {}
cnt = {}
for n in nodes.values():
cnt[str(n.get("status"))] = cnt.get(str(n.get("status")), 0) + 1
rows.append({"gid": g.get("gid"), "title": g.get("title"),
"nodes": len(nodes), "by_status": cnt,
"created_at": g.get("created_at")})
_out({"ws": str(Path(a.ws).resolve()), "graphs": rows})
return 0
# ── 入口 ────────────────────────────────────────────────────────────────
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="graphctl", description="图模式独立实现(不碰现有机制)")
sub = ap.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("init", help="按判据或节点定义建图")
p.add_argument("--ws", required=True)
p.add_argument("--title", required=True)
p.add_argument("--gid", default="")
p.add_argument("--kpi", default="", help="判据,分号分隔;每条生成一个节点")
p.add_argument("--nodes-json", default="", help="节点定义文件(- 表示标准输入)")
p.add_argument("--max-parallel", type=int, default=3)
p.add_argument("--lease-sec", type=float, default=None,
help="租约多少秒算超时(默认 3600;0 = 不判超时)")
p.add_argument("--on-timeout", default="requeue",
help="超时处置:requeue(默认)/block/keep")
p.add_argument("--retry-max", type=int, default=0, help="默认自动重试次数(默认 0)")
p.add_argument("--retry-delay", type=float, default=0, help="重试基础间隔秒(默认 0)")
p.add_argument("--retry-backoff", default="fixed", help="fixed(默认)/linear/exponential")
p.add_argument("--retry-factor", type=float, default=2, help="指数退避倍数(默认 2)")
p.add_argument("--deps-rule", default="all_success",
help="默认依赖规则:all_success(默认)/all_done/one_success")
p.add_argument("--pool", action="append", default=[],
help="按域分池:域前缀=并发上限,可重复(如 docs/ui/=1)")
p.set_defaults(fn=cmd_init)
for name, fn in (("check", cmd_check), ("ready", cmd_ready), ("show", cmd_show)):
p = sub.add_parser(name)
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.set_defaults(fn=fn)
p = sub.add_parser("start", help="节点开工(记租约)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--by", default="")
p.add_argument("--lease-sec", type=float, default=None, help="本次开工的租约秒数(覆盖默认)")
p.set_defaults(fn=cmd_start)
p = sub.add_parser("report", help="节点回报")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--state", required=True)
p.add_argument("--evidence", default="")
p.add_argument("--artifact", action="append", default=[],
help="产物清单,可重复(供 map.from pick=artifacts 读取)")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_report)
p = sub.add_parser("rework", help="回退到上游重做(目标及其下游复位 pending)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--to", default="", help="回退目标(默认取节点自己的 rework_to)")
p.add_argument("--reason", default="")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_rework)
p = sub.add_parser("add-node", help="运行期加节点")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--title", default="")
p.add_argument("--deps", default="", help="依赖节点,逗号分隔")
p.add_argument("--domain", action="append", default=[], help="域前缀,可重复")
p.add_argument("--accepts", action="append", default=[],
help="验收判据,可重复(artifact:… / evidence:… / 纯文本)")
p.add_argument("--deps-rule", default="all_success")
p.add_argument("--rework-to", default="")
p.add_argument("--approval", action="store_true")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_add_node)
p = sub.add_parser("remove-edge", help="删边:删掉「--to 依赖 --from」")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--from", dest="src", required=True, help="上游节点")
p.add_argument("--to", required=True, help="下游节点(它的 deps 里删掉上游)")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_remove_edge)
p = sub.add_parser("expand", help="手动触发展开")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--force", action="store_true")
p.set_defaults(fn=cmd_expand)
p = sub.add_parser("approve", help="人审通过/驳回(awaiting ⇒ done/blocked)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--by", default="")
p.add_argument("--reject", action="store_true")
p.add_argument("--reason", default="")
p.set_defaults(fn=cmd_approve)
p = sub.add_parser("cancel", help="取消节点(连坐子节点)或整图")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", default="")
p.add_argument("--all", action="store_true")
p.add_argument("--by", default="")
p.add_argument("--reason", default="")
p.set_defaults(fn=cmd_cancel)
p = sub.add_parser("tick", help="自动派活:算就绪集并直接开工")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--by", default="tick")
p.add_argument("--limit", type=int, default=0, help="本次最多开几个(默认不额外限)")
p.add_argument("--dry-run", action="store_true", help="只算不开")
p.add_argument("--lease-sec", type=float, default=None)
p.set_defaults(fn=cmd_tick)
p = sub.add_parser("plan", help="出一行排期(四条硬规则已规范化)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--scope", default="", help="workspace_scope,默认取工作区目录名")
p.add_argument("--after-sec", type=int, default=None, help="多久后触发(默认 240 秒)")
p.set_defaults(fn=cmd_plan)
p = sub.add_parser("progress", help="进度与关键路径")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.set_defaults(fn=cmd_progress)
p = sub.add_parser("list")
p.add_argument("--ws", required=True)
p.set_defaults(fn=cmd_list)
a = ap.parse_args(argv)
return a.fn(a)
if __name__ == "__main__":
sys.exit(main())