Files
workbuddy_skills/session-mechanism/graph/graphctl.py
T

2297 lines
97 KiB
Python
Raw Normal View History

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""graphctl.py —— 「图模式」的独立实现(2026-10-18;2026-10-19 补批量能力)。
## 它是什么
把目标拆成节点、按依赖推进的**独立**运行模式。**⛔ 不修改任何现有机制**:
不读也不写 `goal.json` / `tasks.json` / `collabd-state.json`,不参与六道闸,
不接管常驻循环,不改变现有会话的任何行为。
## 数据落点(独立目录,与现有状态完全分开)
<ws>/.workbuddy/graph/<gid>/graph.json 图的唯一真源
<ws>/.workbuddy/graph/<gid>/events.jsonl 每次状态变更的流水
## 图的结构
{
"gid": "20261018-2230-ui改造",
"title": "给三块功能各加一份文档",
"created_at": "2026-10-18T22:30:00",
"meta": {"max_parallel": 3,
"lease": {"timeout_sec": 3600, "on_timeout": "requeue"},
"retry": {"max": 0, "delay_sec": 0, "backoff": "fixed", "factor": 2}},
"nodes": {
"N1": {"title": "...", "deps": [], "domain": ["docs/ui/"],
"accepts": ["产物在位"], "status": "pending",
"evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": "all_success",
"retry": {"max": 2, "delay_sec": 30, "backoff": "exponential"},
"map": {"over": ["a", "b"], "child_domain": "docs/ui/{i}/"}}
}
}
`status` 五态:`pending` → `running` → `done` / `blocked`;
另有 `gating`——展开出来的父节点,等子节点全部完成(父节点自动转 `done`)。
## 命令
init --ws <工作区> --title <目标> (--kpi "判据A;判据B" | --nodes-json <文件|->)
[--max-parallel N] [--lease-sec S] [--on-timeout requeue|block|keep]
[--retry-max N] [--retry-delay S] [--retry-backoff fixed|linear|exponential]
check --ws <ws> --gid <gid> 环/悬空依赖/空域/同批域冲突/非法规则
ready --ws <ws> --gid <gid> 就绪集(含为什么没被选);顺带回收超时租约、展开 map
start --ws <ws> --gid <gid> --node N1 [--by ...] [--lease-sec S]
report --ws <ws> --gid <gid> --node N1 --state done|blocked [--evidence "..."] [--artifact P ...]
rework --ws <ws> --gid <gid> --node N1 [--to 上游] [--reason ...] 回退到上游重做
add-node --ws <ws> --gid <gid> --node N9 [--deps A,B] [--domain P] [--accepts C]
remove-edge --ws <ws> --gid <gid> --from A --to B 删掉「B 依赖 A」这条边
expand --ws <ws> --gid <gid> --node P [--force] 手动触发展开
show --ws <ws> --gid <gid>
list --ws <ws>
tick --ws <ws> --gid <gid> [--by ...] [--limit N] [--dry-run] 自动派活(算完就开)
approve --ws <ws> --gid <gid> --node N [--reject] [--reason ...] 人审
cancel --ws <ws> --gid <gid> (--node N | --all) [--reason ...] 取消
plan --ws <ws> --gid <gid> [--scope ...] [--after-sec N] 出一行排期
progress --ws <ws> --gid <gid> 进度/关键路径
## 复杂任务能力(2026-10-19 补,C 方案)
条件分支:节点 `when: [{"node": "A", "status": "blocked"}]` —— 条件被证伪即
`skipped`;`on_dep_fail: wait|skip|block` 决定上游跑不成时下游怎么走
(默认 wait,与旧行为一致)。
子图:节点 `subgraph: {"nodes": [...]}` —— 内联子流程展开成 `<父>/<子id>`,
父节点 `gating`,子节点全成功才 `done`,有不成功的 ⇒ 父 `blocked`。
人审:节点 `approval: true` —— report done 后进 `awaiting`,`approve` 才放行
(`--reject` ⇒ blocked)。
取消:`cancelled` 是终态,连坐子节点;下游按 `on_dep_fail` 处置。
分池:`meta.pools: {"域前缀": 并发上限}` —— 同域并发受限,与 `max_parallel` 叠加。
进度:`progress` 给出 by_status/percent/关键路径(未完成节点的最长依赖链)。
## 验收判据(accepts)怎么机判(2026-10-19 补)
`report --state done` 时**先按 `accepts` 校验产物与证据**,不满足直接拒收(rc≠0,节点
不置 done,并在 stderr 逐条列出缺什么)。条目两种写法,**纯文本=描述性判据、不参与机判**:
"artifact:docs/*.md" 产物里至少一条匹配该通配(反斜杠与 `/` 等价)
"artifact:>=2" 产物至少 2 条
"evidence:测试通过" 证据里必须出现「测试通过」
"evidence:全绿,零回归" 多个关键词用 `,` 分隔 ⇒ 全部都要出现(AND)
"evidence:>=20" 证据至少 20 个字符
{"kind": "artifact", "glob": "..."} / {"kind": "artifact", "min": 2}
{"kind": "evidence", "contains": ["a", "b"]} / {"kind": "evidence", "min": 20}
校验读的是**本次 + 历史**的合并视图(`--artifact` 是追加语义)。
## 回退边(rework_to,2026-10-19 补)
节点写 `rework_to: "上游节点"`(或列表,取第一个)。判不过时用 `rework --node X`:
① 目标必须是 X 的**上游**(传递依赖),⛔ 不能是自己或下游;
② 目标及其**全部传递下游**复位 `pending`,清租约/完成时刻/驳回原因;
③ 复位时把 `evidence` / `artifacts` 挪到 `prev_evidence` / `prev_artifacts` 再清空
⇒ 重做之后必须重新交产物(否则验收闸会被旧产物蒙过);
④ 目标 `attempts +1`、全链 `reworks +1`,写一条 `rework` 事件。
⑤ `approve --reject` 时若节点带 `rework_to`,自动走同一套回退(人审判不过=判不过)。
⑥ 链上有已展开(`map`/子图)的节点 ⇒ **收回它的子节点并清 `expanded`**,
下次 `ready`/`tick` 按当时的输入重新展开(⛔ 不留半截子节点)。
## 运行期改图(2026-10-19 补)
add-node --node N9 --deps A,B --domain docs/x/ --accepts "artifact:docs/x/a.md"
remove-edge --from A --to B 删掉「B 依赖 A」
两条都走 `_update`(目录锁 + 唯一临时名 + 原子替换 + `_rev` 回读)⇒ 与并发的
`start`/`report` 互不覆盖;边不存在、节点重复、依赖悬空 ⇒ **报错**,⛔ 不静默空操作。
## 跨图子图(2026-10-19 补)
`subgraph: {"gid": "<同工作区另一张图的 gid>"}` —— 除内联 `{"nodes": [...]}` 外,
可引用 `<ws>/.workbuddy/graph/<gid>/graph.json`,把它的**节点定义**展开成本图的
`<父>/<子id>`,父节点照旧 `gating`、等子节点全部收敛。
① 只搬定义(title/deps/domain/accepts/deps_rule/on_dep_fail/when/retry/
lease_sec/approval/rework_to),⛔ 不搬运行状态(引用即重跑一遍);
② 子图内部依赖与被引用图的 `when`/`rework_to` 引用都加父前缀;
③ ⛔ 被引用的图里不允许再有 `map`/`subgraph`(不嵌套),`check` 报硬错;
④ 自引用、A→B→A 成环 ⇒ `check` 与展开时都报错。
## 就绪的判据
① 依赖按 `deps_rule` 满足(默认 all_success)
② 自己还是 pending,且不在重试等待窗内(`not_before` 未到 ⇒ 等)
③ 域不与任何 running 节点重叠,也不与本批已选节点重叠
④ 不超 max_parallel(running 占名额)
## 三件批量能力(2026-10-19 补)
租约:start 记 `lease_at` / `lease_at_ts`;ready 时超过 `lease.timeout_sec`
的 running 节点按 `on_timeout` 处置 —— requeue(默认,回 pending 且
attempts+1)/block(进 blocked)/keep(只提示不动)。
重试:`retry.max` 给了才自动重试,report blocked 后 attempts≤max ⇒ 回 pending
并写 `not_before`(按 backoff 算),超过 ⇒ blocked 且 `retry_exhausted`。
默认 `max=0` ⇒ 与旧行为一致(report blocked 直接 blocked)。
展开:`map.over`(建图给列表)或 `map.from`(运行时读上游 artifacts / evidence),
ready 命中即生成子节点 `P#1..P#N`,父节点转 `gating`,子节点全 done ⇒ 父 done。
## 域的用法
`domain` 是一组路径前缀(相对工作区根)。两节点域重叠 ⇒ 不能同批跑。
判定:任一前缀互为前缀即视为重叠(`docs/` 与 `docs/ui/` 算重叠)。
展开出的子节点**默认不继承父域**(同一批展开通常写同一批文件,继承会把并发挡死);
需要分片写盘时用 `map.child_domain` 模板(`{i}` / `{item}`)。
"""
from __future__ import annotations
import argparse
import fnmatch
import io
import json
import math
import os
import random
import sys
import time
from pathlib import Path
MARK = ".workbuddy/graph"
LOCK_STALE_SEC = 30.0 # 锁目录超过这么久没动静 ⇒ 认定持锁进程已死,接管
STATUSES = ("pending", "running", "done", "blocked", "gating",
"awaiting", "skipped", "cancelled")
SETTLED = ("done", "blocked", "skipped", "cancelled") # 不会再自己变了
DEP_RULES = ("all_success", "all_done", "one_success")
TIMEOUT_POLICIES = ("requeue", "block", "keep")
BACKOFFS = ("fixed", "linear", "exponential")
ON_DEP_FAIL = ("wait", "skip", "block") # 上游跑不成时下游怎么办
DEFAULT_LEASE_SEC = 3600.0
# ── 基础设施 ────────────────────────────────────────────────────────────
def _now() -> str:
return time.strftime("%Y-%m-%dT%H:%M:%S")
def _iso(ts: float) -> str:
return time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(ts))
def _parse_ts(v):
"""ISO 字符串 → epoch;失败返回 None(⛔ 不抛,坏数据当作"没有租约")。"""
if v in (None, ""):
return None
try:
return float(v)
except (TypeError, ValueError):
pass
for fmt in ("%Y-%m-%dT%H:%M:%S", "%Y-%m-%d %H:%M:%S"):
try:
return time.mktime(time.strptime(str(v), fmt))
except ValueError:
continue
return None
def _root(ws: str) -> Path:
return Path(ws).resolve() / MARK
def _gdir(ws: str, gid: str) -> Path:
return _root(ws) / gid
def _load(ws: str, gid: str) -> dict:
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
raise SystemExit("找不到图:%s" % p)
return json.loads(p.read_text(encoding="utf-8"))
def _save(ws: str, gid: str, g: dict) -> None:
"""整份写回(新建图时用)。临时名带 pid + 随机数 ⇒ 两进程不会撞同一个名字。"""
p = _gdir(ws, gid) / "graph.json"
tmp = p.with_suffix(".%d.%d.json.tmp" % (os.getpid(), random.randint(0, 1 << 30)))
tmp.write_text(json.dumps(g, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8", newline="\n")
os.replace(tmp, p) # 原子替换,避免半截文件
class _Lock:
"""写串行化:`os.mkdir` 是原子的,用它当互斥量(⛔ 不用 OS 文件锁,本机会卡死)。
⚠️ 2026-10-19 实测:只靠 `_rev` + token 回读核对仍然丢更新 ——
A 写完回读确认后、B 再覆盖,B 回读看到的也是自己的 token,两边都"成功"。
那次实测里 4 轮测试抖 1 轮(`pending` != `running`)。所以写必须串行。
⚠️ 僵死锁:持锁进程崩了锁目录会留着 ⇒ 超过 `LOCK_STALE_SEC` 直接接管,
⛔ 不会永久卡住;等 `wait` 秒仍拿不到则**报错退出**(带手工清理提示)。
"""
def __init__(self, p: Path, wait: float = 10.0):
self.p = p
self.wait = wait
self.held = False
def __enter__(self):
t0 = time.time()
while True:
try:
os.mkdir(self.p)
self.held = True
try:
(self.p / "owner").write_text(str(os.getpid()), encoding="utf-8")
except OSError:
pass
return self
except FileExistsError:
try:
age = time.time() - self.p.stat().st_mtime
except OSError:
age = 0.0
if age > LOCK_STALE_SEC:
for f in ("owner",):
try:
(self.p / f).unlink()
except OSError:
pass
try:
os.rmdir(self.p)
except OSError:
pass
continue
if time.time() - t0 > self.wait:
raise SystemExit(
"写锁等待超时:%s(等了 %.0f 秒;确认没有僵死进程后可直接删这个目录)"
% (self.p, self.wait))
time.sleep(0.02)
def __exit__(self, *exc):
if self.held:
try:
(self.p / "owner").unlink()
except OSError:
pass
try:
os.rmdir(self.p)
except OSError:
pass
return False
def _update(ws: str, gid: str, mutate, retries: int = 3) -> dict:
"""读-改-写:**先拿目录锁再动文件**,锁内重放变更并原子替换。
① 临时名带 pid + 随机数 ⇒ 两进程不会撞同一个名字(早期固定名会 `PermissionError`)。
② 写整体串行(`_Lock`)⇒ 不会"后写的整份覆盖前写的"导致节点状态静默消失。
③ `_rev` + `_token` 是第二道网:写后回读核对,对不上就重做;
连续 `retries` 次仍被覆盖则**报错退出**(⛔ 不静默丢状态)。
④ Windows 上 `os.replace` 与别的进程的读并发时会短暂 `PermissionError`
⇒ 替换动作本身小步重试,⛔ 不把它当成"并发冲突"直接失败。
"""
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
raise SystemExit("找不到图:%s" % p)
with _Lock(p.parent / ".lock"):
for _ in range(retries):
g = json.loads(p.read_text(encoding="utf-8"))
rev = int(g.get("_rev") or 0)
mutate(g)
token = "%d-%d" % (os.getpid(), random.randint(0, 1 << 30))
g["_rev"] = rev + 1
g["_token"] = token
tmp = p.with_suffix(".%d.%d.json.tmp" % (os.getpid(), random.randint(0, 1 << 30)))
tmp.write_text(json.dumps(g, ensure_ascii=False, indent=2) + "\n",
encoding="utf-8", newline="\n")
for _try in range(6): # 目标被读打开时 replace 会短暂被拒
try:
os.replace(tmp, p)
break
except OSError:
time.sleep(0.05)
else:
try:
tmp.unlink()
except OSError:
pass
continue
try:
back = json.loads(p.read_text(encoding="utf-8"))
except Exception: # noqa: BLE001
back = {}
if int(back.get("_rev") or 0) == rev + 1 and back.get("_token") == token:
return back
raise SystemExit("并发冲突:%s 连续 %d 次被别的进程覆盖,稍后重试" % (p, retries))
def _event(ws: str, gid: str, obj: dict) -> None:
p = _gdir(ws, gid) / "events.jsonl"
obj = dict(obj)
obj.setdefault("at", _now())
with io.open(p, "a", encoding="utf-8", newline="\n") as f:
f.write(json.dumps(obj, ensure_ascii=False) + "\n")
def _out(obj) -> None:
data = json.dumps(obj, ensure_ascii=False, indent=2) + "\n"
sys.stdout.buffer.write(data.encode("utf-8"))
# ── 验收判据(accepts 机判)──────────────────────────────────────────────
def _accepts_problem(vals):
"""校验 accepts 语法;有问题返回一句人读的原因,没问题返回 None。"""
for raw in (vals or []):
if isinstance(raw, dict):
kind = str(raw.get("kind") or "").strip()
if kind not in ("artifact", "evidence"):
return 'accepts 的对象条目要写 kind=artifact 或 evidence(实际 %r)' % (raw,)
if kind == "artifact":
if raw.get("glob") in (None, "") and raw.get("min") in (None, ""):
return 'accepts 的 artifact 条目要写 glob 或 min:%r' % (raw,)
if raw.get("min") not in (None, ""):
try:
if int(raw["min"]) <= 0:
return "accepts 的 min 必须 ≥1:%r" % (raw,)
except (TypeError, ValueError):
return "accepts 的 min 不是整数:%r" % (raw,)
else:
cs = raw.get("contains")
if cs is not None and not isinstance(cs, list):
return 'accepts 的 evidence.contains 要写成列表:%r' % (raw,)
if not cs and raw.get("min") in (None, ""):
return 'accepts 的 evidence 条目要写 contains 或 min:%r' % (raw,)
if raw.get("min") not in (None, ""):
try:
if int(raw["min"]) <= 0:
return "accepts 的 min 必须 ≥1:%r" % (raw,)
except (TypeError, ValueError):
return "accepts 的 min 不是整数:%r" % (raw,)
continue
if not isinstance(raw, str):
return "accepts 条目只认字符串或对象(实际 %r)" % (raw,)
s = raw.strip()
low = s.lower()
if not (low.startswith("artifact:") or low.startswith("evidence:")):
continue # 纯文本 = 描述性判据,不机判
head, _, arg = s.partition(":")
arg = arg.strip()
if not arg:
return "accepts 条目缺参数:%r(写成 %s:<通配|>=N|关键词>)" % (s, head)
if arg.startswith(">="):
try:
if int(arg[2:]) <= 0:
return "accepts 的 >=N 要 ≥1:%r" % s
except ValueError:
return "accepts 的 >=N 不是整数:%r" % s
elif low.startswith("evidence:") and not any(
x.strip() for x in arg.replace(",", ",").split(",")):
return "accepts 的 evidence 关键词为空:%r" % s
return None
def _norm_accepts(vals) -> list:
p = _accepts_problem(vals)
if p:
raise SystemExit(p)
out = []
for raw in (vals or []):
if isinstance(raw, dict):
kind = str(raw.get("kind")).strip()
it = {"kind": kind}
if kind == "artifact":
if raw.get("glob") not in (None, ""):
it["glob"] = str(raw["glob"]).strip()
if raw.get("min") not in (None, ""):
it["min"] = int(raw["min"])
else:
if raw.get("contains"):
it["contains"] = [str(x) for x in raw["contains"]]
if raw.get("min") not in (None, ""):
it["min"] = int(raw["min"])
out.append(it)
else:
out.append(str(raw).strip())
return out
def _norm_path(s) -> str:
return str(s or "").replace("\\", "/").strip()
def _accepts_missing(accepts, artifacts, evidence) -> list:
"""逐条判 accepts,返回没达标的描述(空列表=全过)。"""
arts = [_norm_path(x) for x in (artifacts or [])]
ev = str(evidence or "")
miss = []
for raw in (accepts or []):
if isinstance(raw, dict):
kind = str(raw.get("kind") or "")
glob = raw.get("glob")
mn = raw.get("min")
need = [str(x) for x in (raw.get("contains") or [])]
else:
s = str(raw).strip()
head, _, arg = s.partition(":")
low = head.strip().lower()
arg = arg.strip()
if low not in ("artifact", "evidence"):
continue # 描述性判据,不机判
kind = low
glob = None
mn = None
need = []
if arg.startswith(">="):
mn = int(arg[2:])
elif kind == "artifact":
glob = arg
else:
need = [x.strip() for x in arg.replace(",", ",").split(",") if x.strip()]
if kind == "artifact":
if glob:
if not any(fnmatch.fnmatch(a, _norm_path(glob)) for a in arts):
miss.append("缺产物 artifact:%s(现有产物:%s)"
% (glob, "、".join(arts) if arts else "无"))
if mn is not None and len(arts) < mn:
miss.append("产物数量不足 artifact:>=%d(现有 %d 条)" % (mn, len(arts)))
elif kind == "evidence":
if mn is not None and len(ev.strip()) < mn:
miss.append("证据太短 evidence:>=%d(现有 %d 字)" % (mn, len(ev.strip())))
if need:
absent = [k for k in need if k not in ev]
if absent:
miss.append("证据缺关键词 evidence:%s(现缺:%s)"
% (",".join(need), "、".join(absent)))
return miss
# ── 回退边(rework_to)──────────────────────────────────────────────────
def _ancestors(nodes: dict, nid: str) -> set:
seen, stack = set(), list(_deps(nodes.get(nid) or {}))
while stack:
x = stack.pop()
if x in seen or x not in nodes:
continue
seen.add(x)
stack.extend(_deps(nodes[x]))
return seen
def _descendants(nodes: dict, nid: str) -> set:
"""所有传递下游(不含自己)。"""
kids = {}
for k, n in nodes.items():
for d in _deps(n):
kids.setdefault(d, []).append(k)
seen, stack = set(), list(kids.get(nid) or [])
while stack:
x = stack.pop()
if x in seen or x not in nodes:
continue
seen.add(x)
stack.extend(kids.get(x) or [])
return seen
def _rework_target(n: dict):
"""节点自己的 rework_to(字符串或列表)⇒ 第一个目标 id(可能为空)。"""
rt = n.get("rework_to")
if isinstance(rt, (list, tuple)):
for x in rt:
if str(x).strip():
return str(x).strip()
return ""
return str(rt or "").strip()
def _drop_subtree(nodes: dict, nid: str) -> list:
"""收回一个已展开节点的全部子节点(展开产物,重新展开时按需重建)。"""
out, stack = [], list((nodes.get(nid) or {}).get("children") or [])
while stack:
c = stack.pop()
if c not in nodes:
continue
out.append(c)
stack.extend((nodes[c] or {}).get("children") or [])
for c in out:
nodes.pop(c, None)
return out
def _rework_reset(nodes: dict, from_nid: str, tgt: str, reason: str = "") -> dict:
"""把 tgt 及其全部传递下游复位成 pending(调用方保证已通过前置校验)。
链上有已展开(map/subgraph)的节点 ⇒ **收回它的子节点、清掉 expanded**,
下次 ready/tick 会按当时的输入重新展开(⛔ 不留半截子节点)。
"""
down = _descendants(nodes, tgt) | {tgt}
bad = [x for x in down if str((nodes.get(x) or {}).get("status")) == "running"]
if bad:
raise SystemExit("回退链上有正在跑的节点:%s(先 cancel 或等它结束)"
% ", ".join(sorted(bad)))
dropped = []
for x in sorted(down):
m = nodes.get(x)
if m is not None and (m.get("expanded") or str(m.get("status")) == "gating"):
dropped.extend(_drop_subtree(nodes, x))
now = _now()
reset = []
for x in sorted(down):
m = nodes.get(x)
if m is None: # 已被当作子节点收回
continue
if m.get("artifacts"):
m["prev_artifacts"] = list(m["artifacts"])
m["artifacts"] = []
if m.get("evidence"):
m["prev_evidence"] = m["evidence"]
m["evidence"] = ""
m["status"] = "pending"
for k in ("lease_at", "lease_at_ts", "lease_sec", "done_at", "awaiting_at",
"approved_at", "approved_by", "not_before", "retry_at",
"retry_exhausted", "rejected_at", "blocked_by", "blocked_reason",
"skipped_at", "skipped_reason", "completed_at",
"expanded", "expanded_at", "item_count", "children"):
m.pop(k, None)
m["reworked_at"] = now
m["rework_of"] = tgt
m["reworks"] = int(m.get("reworks") or 0) + 1
reset.append(x)
nodes[tgt]["attempts"] = int(nodes[tgt].get("attempts") or 0) + 1
nodes[tgt]["rework_from"] = from_nid
if reason:
nodes[tgt]["rework_reason"] = reason
res = {"target": tgt, "reset": reset,
"attempts": nodes[tgt]["attempts"],
"reworks": nodes[tgt]["reworks"]}
if dropped:
res["dropped_children"] = dropped
return res
# ── 跨图子图(subgraph.gid)─────────────────────────────────────────────
def _sg_chain(g: dict, nid: str) -> list:
"""从自己往上走 parent 链,收集沿途声明的 subgraph.gid(防跨图成环)。"""
nodes = g.get("nodes") or {}
out, seen, cur = [], set(), nid
while cur and cur in nodes and cur not in seen:
seen.add(cur)
sg = (nodes.get(cur) or {}).get("subgraph")
if isinstance(sg, dict) and str(sg.get("gid") or "").strip():
out.append(str(sg["gid"]).strip())
cur = (nodes.get(cur) or {}).get("parent")
return out
def _sg_specs_from_gid(ws: str, ref_gid: str, chain: list) -> list:
"""读另一张图,把它的节点定义翻成内联子图规格。"""
if not ws:
raise SystemExit("内部错误:展开跨图子图时缺工作区路径(gid=%s)" % ref_gid)
if ref_gid in chain:
raise SystemExit("子图引用成环:%s" % " → ".join(list(chain) + [ref_gid]))
p = _gdir(ws, ref_gid) / "graph.json"
if not p.is_file():
raise SystemExit("子图引用的图不存在:%s(gid=%s)" % (p, ref_gid))
sg = json.loads(p.read_text(encoding="utf-8"))
specs = []
for sid, sn in (sg.get("nodes") or {}).items():
if not isinstance(sn, dict):
continue
if sn.get("map") or sn.get("subgraph"):
raise SystemExit("子图 %s 的节点 %s 自己又声明了 map/subgraph ⇒ 不支持嵌套"
% (ref_gid, sid))
specs.append({
"id": sid,
"title": sn.get("title") or sid,
"deps": list(sn.get("deps") or []),
"domain": _norm_dom(sn.get("domain")),
"accepts": list(sn.get("accepts") or []),
"deps_rule": sn.get("deps_rule"),
"on_dep_fail": sn.get("on_dep_fail"),
"when": sn.get("when"),
"retry": sn.get("retry"),
"lease_sec": sn.get("lease_sec"),
"approval": sn.get("approval"),
"rework_to": sn.get("rework_to"),
})
return specs
# ── 域与依赖 ────────────────────────────────────────────────────────────
def _norm_dom(d) -> list:
if isinstance(d, str):
d = [d]
out = []
for x in (d or []):
s = str(x).strip().replace("\\", "/").lstrip("./")
if s:
out.append(s if s.endswith("/") else s + "/")
return out
def _dom_overlap(a, b) -> bool:
"""任一前缀互为前缀 ⇒ 重叠(docs/ 与 docs/ui/ 算重叠)。"""
for x in a:
for y in b:
if x.startswith(y) or y.startswith(x):
return True
return False
def _deps(n: dict) -> list:
v = n.get("deps") or []
return [str(x) for x in v] if isinstance(v, list) else []
def _is_done(nodes: dict, nid: str) -> bool:
return str((nodes.get(nid) or {}).get("status")) == "done"
def _st(nodes: dict, nid: str) -> str:
return str((nodes.get(nid) or {}).get("status"))
def _dep_verdict(nodes: dict, n: dict) -> dict:
"""依赖判定 ⇒ {"verdict": go|wait|skip|block, "why": 说明}
① `when` 条件子句先判(条件分支):全部成立 ⇒ 继续;任一被证伪 ⇒ skip。
② 再按 `deps_rule` 判:all_success(默认)/all_done/one_success。
③ 上游停在 skipped / cancelled / blocked 时按 `on_dep_fail` 处置:
wait(默认,与旧行为一致)/skip(连坐跳过)/block(整枝停住)。
"""
deps = _deps(n)
rule = str(n.get("deps_rule") or "all_success")
on_fail = str(n.get("on_dep_fail") or "wait")
if on_fail not in ON_DEP_FAIL:
on_fail = "wait"
st = {d: _st(nodes, d) for d in deps}
for c in (n.get("when") or []):
if not isinstance(c, dict):
continue
cnode = str(c.get("node") or "")
want = str(c.get("status") or "done")
got = _st(nodes, cnode)
if got == want:
continue
if got in SETTLED:
return {"verdict": "skip",
"why": "条件不成立:%s 是 %s,要 %s" % (cnode, got, want)}
if not deps:
return {"verdict": "wait", "why": "条件节点 %s 还没定论(%s)" % (cnode, got)}
if not deps:
return {"verdict": "go", "why": ""}
dead = [(d, s) for d, s in st.items() if s in ("skipped", "cancelled")] \
or [(d, s) for d, s in st.items() if s == "blocked"]
if rule == "all_done":
lack = [d for d, s in st.items() if s not in SETTLED]
if lack:
return {"verdict": "wait", "why": "依赖未跑完:%s" % ", ".join(lack)}
return {"verdict": "go", "why": ""}
if rule == "one_success":
if any(s == "done" for s in st.values()):
return {"verdict": "go", "why": ""}
if dead and all(s in SETTLED for s in st.values()):
return {"verdict": on_fail if on_fail != "wait" else "wait",
"why": "没有任一上游成功:%s"
% ", ".join("%s(%s)" % (d, s) for d, s in st.items())}
return {"verdict": "wait",
"why": "等待任一上游成功:%s"
% ", ".join("%s(%s)" % (d, s) for d, s in st.items())}
lack = [d for d, s in st.items() if s != "done"]
if not lack:
return {"verdict": "go", "why": ""}
if dead:
why = "上游不会成功了:%s" % ", ".join("%s(%s)" % (d, s) for d, s in dead)
if on_fail != "wait":
return {"verdict": on_fail, "why": why}
return {"verdict": "wait",
"why": "依赖未完成:%s" % ", ".join("%s(%s)" % (d, st[d]) for d in lack)}
return {"verdict": "wait",
"why": "依赖未完成:%s" % ", ".join("%s(%s)" % (d, st[d]) for d in lack)}
def _deps_ok(nodes: dict, n: dict):
"""按 `deps_rule` 判依赖是否放行,返回 (bool, 没满足的原因)。"""
v = _dep_verdict(nodes, n)
return v["verdict"] == "go", v["why"]
# ── 三件批量能力的配置读法 ──────────────────────────────────────────────
def _meta(g: dict) -> dict:
m = g.get("meta")
return m if isinstance(m, dict) else {}
def _lease_cfg(g: dict) -> dict:
d = _meta(g).get("lease")
d = d if isinstance(d, dict) else {}
try:
sec = float(d.get("timeout_sec", DEFAULT_LEASE_SEC))
except (TypeError, ValueError):
sec = DEFAULT_LEASE_SEC
pol = str(d.get("on_timeout") or "requeue")
return {"timeout_sec": sec, "on_timeout": pol if pol in TIMEOUT_POLICIES else "requeue"}
def _node_lease_sec(g: dict, n: dict) -> float:
"""节点级 `lease_sec` 优先于全局;0 表示该节点不判超时。"""
v = n.get("lease_sec")
if v in (None, ""):
return _lease_cfg(g)["timeout_sec"]
try:
return float(v)
except (TypeError, ValueError):
return _lease_cfg(g)["timeout_sec"]
def _retry_cfg(g: dict, n: dict) -> dict:
d = {}
for src in (_meta(g).get("retry"), n.get("retry")):
if isinstance(src, dict):
d.update(src)
def _num(k, dv):
try:
return float(d.get(k, dv))
except (TypeError, ValueError):
return float(dv)
bk = str(d.get("backoff") or "fixed")
return {"max": max(0, int(_num("max", 0))),
"delay_sec": max(0.0, _num("delay_sec", 0)),
"backoff": bk if bk in BACKOFFS else "fixed",
"factor": max(1.0, _num("factor", 2))}
def _delay_of(rc: dict, attempts: int) -> float:
k = max(1, int(attempts))
if rc["backoff"] == "linear":
return rc["delay_sec"] * k
if rc["backoff"] == "exponential":
return rc["delay_sec"] * (rc["factor"] ** (k - 1))
return rc["delay_sec"]
# ── 校验 ────────────────────────────────────────────────────────────────
def _find_cycle(nodes: dict) -> list:
color, stack, hit = {}, [], []
def dfs(u):
color[u] = 1
stack.append(u)
for v in _deps(nodes.get(u) or {}):
if v not in nodes:
continue
if color.get(v) == 1:
i = stack.index(v)
hit.extend(stack[i:] + [v])
return True
if color.get(v, 0) == 0 and dfs(v):
return True
stack.pop()
color[u] = 2
return False
for k in nodes:
if color.get(k, 0) == 0 and dfs(k):
return hit
return []
def _ref_graph(ws: str, gid: str):
p = _gdir(ws, gid) / "graph.json"
if not p.is_file():
return None
try:
return json.loads(p.read_text(encoding="utf-8"))
except Exception: # noqa: BLE001
return None
def _check_sg_ref(ws: str, chain: list, nid: str, ref: str, errs: list,
depth: int = 0) -> list:
"""校验 `subgraph.gid`:存在性/自引用/跨图成环/不许嵌套;返回子图节点规格。"""
if ref in chain:
errs.append("%s 的 subgraph 引用成环:%s" % (nid, " → ".join(chain + [ref])))
return []
if depth > 6:
errs.append("%s 的 subgraph 引用链超过 6 层(gid=%s)" % (nid, ref))
return []
rg = _ref_graph(ws, ref)
if rg is None:
errs.append("%s 的 subgraph 引用的图不存在:gid=%s(%s)"
% (nid, ref, _gdir(ws, ref) / "graph.json"))
return []
rnodes = {k: v for k, v in (rg.get("nodes") or {}).items() if isinstance(v, dict)}
out = []
for sid, sn in rnodes.items():
if sn.get("map") or sn.get("subgraph"):
errs.append("%s 引用的图 %s 里节点 %s 又声明了 map/subgraph ⇒ 不支持嵌套"
% (nid, ref, sid))
out.append({"id": sid, "deps": [str(x) for x in (sn.get("deps") or [])]})
for sid, sn in rnodes.items():
sub2 = sn.get("subgraph")
g2 = str((sub2 or {}).get("gid") or "").strip() if isinstance(sub2, dict) else ""
if g2:
_check_sg_ref(ws, chain + [ref], "%s/%s" % (nid, sid), g2, errs, depth + 1)
return out
def cmd_check(a) -> int:
g = _load(a.ws, a.gid)
nodes = g.get("nodes") or {}
errs, warns = [], []
cyc = _find_cycle(nodes)
if cyc:
errs.append("存在环:%s" % " → ".join(cyc))
for nid, n in nodes.items():
for d in _deps(n):
if d not in nodes:
errs.append("%s 的依赖 %s 不存在" % (nid, d))
if str(n.get("deps_rule") or "all_success") not in DEP_RULES:
errs.append("%s 的 deps_rule 非法:%r(只认 %s)"
% (nid, n.get("deps_rule"), " / ".join(DEP_RULES)))
m = n.get("map")
if m is not None:
if not isinstance(m, dict) or not ("over" in m or "from" in m):
errs.append("%s 的 map 需要 over(建图给列表)或 from(运行时读上游)" % nid)
else:
src = (m.get("from") or {}).get("node") if isinstance(m.get("from"), dict) else None
if src and src not in nodes:
errs.append("%s 的 map.from.node %s 不存在" % (nid, src))
elif src and src not in _deps(n):
warns.append("%s 的展开源 %s 不在 deps 里 ⇒ 上游没完成时会被当成"
"「展开源未就绪」空等" % (nid, src))
if n.get("map") and n.get("subgraph"):
errs.append("%s 同时声明了 map 与 subgraph(只能选一个)" % nid)
sg = n.get("subgraph")
if sg is not None:
kids = sg.get("nodes") if isinstance(sg, dict) else None
ref = str((sg or {}).get("gid") or "").strip() if isinstance(sg, dict) else ""
if isinstance(kids, list) and ref:
errs.append("%s 的 subgraph 同时给了 nodes 与 gid(只能选一种)" % nid)
if not isinstance(kids, list) and not ref:
errs.append('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "..."}' % nid)
kids = []
if ref:
kids = _check_sg_ref(a.ws, [a.gid], nid, ref, errs)
extra = [s["id"] for s in kids]
for s in kids:
for d in s["deps"]:
if d not in extra:
errs.append("%s 引用的图 %s 里 %s 依赖 %s 不在那张图里"
% (nid, ref, s["id"], d))
seen = set()
for s in kids:
sid = str((s or {}).get("id") or "")
if not sid:
warns.append("%s 的子图节点缺 id ⇒ 会按顺序补 S1/S2,deps 引用会指不到"
% nid)
elif sid in seen:
errs.append("%s 的子图里有重复 id:%s" % (nid, sid))
else:
seen.add(sid)
for s in kids:
for d in ((s or {}).get("deps") or []):
if str(d) not in seen and str(d) not in nodes:
errs.append("%s/%s 的依赖 %s 既不在子图里也不在图里"
% (nid, (s or {}).get("id") or "?", d))
if str(n.get("on_dep_fail") or "wait") not in ON_DEP_FAIL:
errs.append("%s 的 on_dep_fail 非法:%r(只认 %s)"
% (nid, n.get("on_dep_fail"), " / ".join(ON_DEP_FAIL)))
for c in (n.get("when") or []):
if not isinstance(c, dict) or not str(c.get("node") or ""):
errs.append('%s 的 when 子句要写 {"node": "A", "status": "done"}' % nid)
continue
if str(c["node"]) not in nodes:
errs.append("%s 的 when 条件节点 %s 不存在" % (nid, c["node"]))
if str(c.get("status") or "done") not in STATUSES:
errs.append("%s 的 when 期望状态非法:%r" % (nid, c.get("status")))
if n.get("expanded"):
for cid in (n.get("children") or []):
if cid not in nodes:
errs.append("%s 的子节点 %s 不见了(图被手改过?)" % (nid, cid))
if not _norm_dom(n.get("domain")) and not n.get("parent"):
warns.append("%s 没有声明域 ⇒ 域比较视为不冲突,可与任意节点同批;"
"保守做法是让它单独跑" % nid)
if str(n.get("status")) not in STATUSES:
errs.append("%s 的状态非法:%r" % (nid, n.get("status")))
ap = _accepts_problem(n.get("accepts"))
if ap:
errs.append("%s:%s" % (nid, ap))
rt = _rework_target(n)
if rt:
if rt not in nodes:
errs.append("%s 的 rework_to 指向不存在的节点:%s" % (nid, rt))
elif rt == nid:
errs.append("%s 的 rework_to 指向自己" % nid)
elif rt not in _ancestors(nodes, nid):
errs.append("%s 的 rework_to %s 不是它的上游(回退只能回到上游)"
% (nid, rt))
if not (n.get("accepts") or []) and not n.get("map") and not n.get("parent"):
warns.append("%s 没有验收判据 ⇒ 判不出完成" % nid)
pools = _meta(g).get("pools")
if pools is not None:
if not isinstance(pools, dict):
errs.append('meta.pools 要写成 {"域前缀": 并发上限}')
else:
doms = [(_norm_dom(n.get("domain")), nid) for nid, n in nodes.items()]
for k, v in pools.items():
try:
cap = int(v)
except (TypeError, ValueError):
errs.append("meta.pools[%s] 不是数字:%r" % (k, v))
continue
if cap <= 0:
errs.append("meta.pools[%s] 必须是正整数" % k)
pre = _norm_dom(k)
if not pre:
errs.append("meta.pools 的域不能为空")
elif not any(d and _dom_overlap(pre, d) for d, _n in doms):
warns.append("域池 %s 没有匹配任何节点的域" % k)
ids = list(nodes)
for i in range(len(ids)):
for j in range(i + 1, len(ids)):
a1, a2 = ids[i], ids[j]
if _norm_dom(nodes[a1].get("domain")) and \
_dom_overlap(_norm_dom(nodes[a1].get("domain")),
_norm_dom(nodes[a2].get("domain"))):
warns.append("%s 与 %s 的域重叠 ⇒ 永不同批" % (a1, a2))
_out({"ok": not errs, "errors": errs, "warnings": warns,
"nodes": len(nodes)})
return 0 if not errs else 1
# ── 租约超时回收 ────────────────────────────────────────────────────────
def _lease_over(g: dict, n: dict, now: float):
"""超时则返回超时秒数,否则 None。没有租约时刻 ⇒ 不判超时(兼容手改的旧图)。"""
if str(n.get("status")) != "running":
return None
sec = _node_lease_sec(g, n)
if sec <= 0:
return None
ts = _parse_ts(n.get("lease_at_ts"))
if ts is None:
ts = _parse_ts(n.get("lease_at"))
if ts is None:
return None
over = now - ts - sec
return over if over > 0 else None
def _reclaim_leases(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
lc = _lease_cfg(g)
now = time.time()
changed = 0
for nid, n in list(nodes.items()):
over = _lease_over(g, n, now)
if over is None:
continue
pol = str(n.get("on_timeout") or lc["on_timeout"])
if pol == "keep":
continue # 只提示,不动(notice 由外层补)
n["attempts"] = int(n.get("attempts") or 0) + 1
n["reclaimed_at"] = _now()
n["reclaim_count"] = int(n.get("reclaim_count") or 0) + 1
for k in ("lease_at", "lease_at_ts", "lease_sec"):
n.pop(k, None)
if pol == "block":
n["status"] = "blocked"
n["blocked_by"] = "lease_timeout"
else:
n["status"] = "pending"
acts.append({"node": nid, "event": "lease_timeout", "to": n["status"],
"over_sec": int(over), "attempts": n["attempts"],
"policy": pol})
changed += 1
return changed
def _lease_notices(g: dict) -> list:
lc = _lease_cfg(g)
if lc["on_timeout"] != "keep":
return []
now = time.time()
out = []
for nid, n in (g.get("nodes") or {}).items():
over = _lease_over(g, n, now)
if over is not None:
out.append({"node": nid, "over_sec": int(over),
"why": "租约超时但策略为 keep(只提示,不回收)"})
return out
# ── 动态展开(map / foreach)─────────────────────────────────────────────
def _as_text(x) -> str:
if isinstance(x, (dict, list)):
return json.dumps(x, ensure_ascii=False)
return str(x)
def _parse_list(v) -> list:
"""证据/产物 → 列表。JSON 数组优先,否则按行与分号切。"""
if isinstance(v, list):
return list(v)
s = str(v or "").strip()
if not s:
return []
if s[0] in "[(":
try:
got = json.loads(s)
if isinstance(got, list):
return list(got)
except Exception: # noqa: BLE001
pass
out = []
for line in s.replace("\r\n", "\n").replace(";", "\n").replace(";", "\n").split("\n"):
t = line.strip().lstrip("-*").strip()
if t:
out.append(t)
return out
def _map_items(g: dict, nodes: dict, nid: str):
"""取展开项;源还没就绪 ⇒ None(区别于空列表 = N=0)。"""
n = nodes.get(nid) or {}
m = n.get("map")
if not isinstance(m, dict):
return None
if "over" in m:
items = list(m.get("over") or [])
else:
f = m.get("from") or {}
up = nodes.get(str(f.get("node") or ""))
if not up or str(up.get("status")) != "done":
return None
pick = str(f.get("pick") or "artifacts")
if pick == "evidence":
items = _parse_list(up.get("evidence"))
elif pick == "children":
items = list(up.get("children") or [])
else:
items = list(up.get("artifacts") or [])
cap = int(m.get("max_items") or 0)
if cap > 0:
items = items[:cap]
return items
def _child_domain(m: dict, i: int, item) -> list:
cd = m.get("child_domain")
if not cd:
return []
if isinstance(cd, str):
cd = [cd]
out = []
for x in cd:
s = str(x).replace("{i}", str(i)).replace("{item}", _as_text(item))
out.append(s)
return _norm_dom(out)
def _expand_one(g: dict, nodes: dict, nid: str, items: list) -> dict:
n = nodes[nid]
if items and int((n.get("map") or {}).get("max_items") or 0) > 0:
items = items[:int((n.get("map") or {}).get("max_items"))]
n["expanded"] = True
n["expanded_at"] = _now()
n["item_count"] = len(items)
if not items: # N=0 ⇒ 无需执行,父节点直接完成
n["status"] = "done"
n["children"] = []
n["evidence"] = n.get("evidence") or "展开 0 项,无需执行"
return {"node": nid, "event": "expand", "count": 0, "children": [],
"to": "done"}
kids = []
for i, it in enumerate(items, 1):
cid = "%s#%d" % (nid, i)
if cid in nodes:
raise SystemExit("子节点 id 已被占用:%s(父节点 %s)" % (cid, nid))
title = "%s[%d]" % (str(n.get("title") or nid), i)
if not isinstance(it, (dict, list)):
title = "%s %s" % (title, str(it)[:40])
nodes[cid] = {
"title": title,
"deps": _deps(n),
"domain": _child_domain(n.get("map") or {}, i, it),
"accepts": list(n.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"parent": nid, "index": i, "item": it,
}
kids.append(cid)
n["status"] = "gating"
n["children"] = kids
return {"node": nid, "event": "expand", "count": len(kids), "children": kids,
"to": "gating"}
def _expand_maps(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
st = str(n.get("status"))
if st not in ("pending", "blocked") or n.get("expanded") or not n.get("map"):
continue
if st == "blocked":
continue # 失败节点不自动展开,等人工 start/重试
okd, _why = _deps_ok(nodes, n)
if not okd:
continue
items = _map_items(g, nodes, nid)
if items is None:
continue
acts.append(_expand_one(g, nodes, nid, items))
changed += 1
return changed
def _complete_parents(g: dict, acts: list) -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "gating":
continue
kids = [c for c in (n.get("children") or []) if c in nodes]
if not kids or not all(_st(nodes, c) in SETTLED for c in kids):
continue
arts = []
for c in kids:
arts.extend(list((nodes.get(c) or {}).get("artifacts") or []))
if arts:
n["artifacts"] = arts
if all(_is_done(nodes, c) for c in kids):
n["status"] = "done"
n["completed_at"] = _now()
n["evidence"] = n.get("evidence") or ("展开 %d 项全部完成" % len(kids))
acts.append({"node": nid, "event": "children_done", "to": "done",
"count": len(kids)})
else: # 子节点全定下来了,但不全成功 ⇒ 父节点不算成功
bad = ["%s(%s)" % (c, _st(nodes, c)) for c in kids if not _is_done(nodes, c)]
n["status"] = "blocked"
n["blocked_by"] = "child_failed"
n["blocked_reason"] = "子节点未全部成功:%s" % ", ".join(bad)
acts.append({"node": nid, "event": "children_failed", "to": "blocked",
"count": len(kids), "bad": bad})
changed += 1
return changed
# ── 条件分支(when / on_dep_fail)───────────────────────────────────────
def _apply_conditions(g: dict, acts: list) -> int:
"""把「条件不成立 ⇒ skipped」「上游跑不成且要求 block ⇒ blocked」落到状态上。"""
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "pending":
continue
if n.get("map") and not n.get("expanded"):
continue
if n.get("subgraph") and not n.get("expanded"):
continue
v = _dep_verdict(nodes, n)
if v["verdict"] == "skip":
n["status"] = "skipped"
n["skipped_at"] = _now()
n["skipped_reason"] = v["why"]
acts.append({"node": nid, "event": "skip", "to": "skipped",
"why": v["why"]})
changed += 1
elif v["verdict"] == "block":
n["status"] = "blocked"
n["blocked_by"] = "dep"
n["blocked_reason"] = v["why"]
acts.append({"node": nid, "event": "block", "to": "blocked",
"why": v["why"]})
changed += 1
return changed
# ── 子图(节点内部再拆子流程)───────────────────────────────────────────
def _expand_subgraphs(g: dict, acts: list, ws: str = "") -> int:
nodes = g.get("nodes") or {}
changed = 0
for nid, n in list(nodes.items()):
if str(n.get("status")) != "pending" or n.get("expanded") or not n.get("subgraph"):
continue
if not _deps_ok(nodes, n)[0]:
continue
acts.append(_expand_subgraph(g, nodes, nid, ws))
changed += 1
return changed
def _expand_subgraph(g: dict, nodes: dict, nid: str, ws: str = "") -> dict:
"""子图 ⇒ 子节点 `<父>/<子id>`,父节点转 gating,子节点全成功才 done。
`subgraph.nodes` 内联;`subgraph.gid` 引用同工作区另一张图的节点定义。
"""
n = nodes[nid]
sub = n.get("subgraph") or {}
ref_gid = str(sub.get("gid") or "").strip()
if ref_gid:
if ref_gid == str(g.get("gid") or ""):
raise SystemExit("子图引用了当前图自己:%s" % ref_gid)
par = n.get("parent")
chain = list(reversed(_sg_chain(g, par))) if par else []
specs = _sg_specs_from_gid(ws, ref_gid, chain)
n["subgraph_ref"] = ref_gid
else:
specs = [s for s in (sub.get("nodes") or []) if isinstance(s, dict)]
prefix = nid + "/"
local, kids = {}, []
for i, s in enumerate(specs, 1):
raw = str(s.get("id") or ("S%d" % i)).strip()
cid = prefix + raw
if cid in nodes:
raise SystemExit("子图节点 id 已被占用:%s" % cid)
local[raw] = cid
kids.append((cid, s))
n["expanded"] = True
n["expanded_at"] = _now()
if not kids: # 空子图 ⇒ 无需执行
n["status"] = "done"
n["children"] = []
n["evidence"] = n.get("evidence") or "子图为空,无需执行"
return {"node": nid, "event": "expand", "kind": "subgraph", "count": 0, "to": "done"}
base = _norm_dom(n.get("domain")) or _norm_dom(sub.get("child_domain"))
outer = _deps(n)
for cid, s in kids:
inner = []
for d in (s.get("deps") or []):
tgt = local.get(str(d)) or str(d)
if tgt not in inner:
inner.append(tgt)
deps = list(outer) + [d for d in inner if d not in outer]
dom = _norm_dom(s.get("domain"))
if not dom:
cd = sub.get("child_domain")
dom = _norm_dom(str(cd).replace("{id}", cid[len(prefix):])) if cd else list(base)
kid = {"title": str(s.get("title") or cid), "deps": deps, "domain": dom,
"accepts": _norm_accepts(s.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [],
"attempts": 0, "parent": nid, "sub": True}
for k in ("deps_rule", "on_dep_fail", "approval", "lease_sec"):
if s.get(k) not in (None, "", []):
kid[k] = s[k]
# `when` / `rework_to` 里的节点引用要跟着加父前缀,否则指向不存在的 id
if s.get("when"):
kid["when"] = [{"node": local.get(str(c.get("node")), str(c.get("node"))),
"status": str(c.get("status") or "done")}
for c in s["when"] if isinstance(c, dict)]
rt = _rework_target(s)
if rt:
kid["rework_to"] = local.get(rt, rt)
if s.get("retry"):
kid["retry"] = _retry_cfg({"meta": {}}, {"retry": s["retry"]})
if ref_gid:
kid["origin_graph"] = ref_gid
nodes[cid] = kid
n["status"] = "gating"
n["children"] = [c for c, _ in kids]
return {"node": nid, "event": "expand", "kind": "subgraph",
"count": len(kids), "children": n["children"], "to": "gating",
**({"from_graph": ref_gid} if ref_gid else {})}
# ── 按域分池 ────────────────────────────────────────────────────────────
def _pool_caps(g: dict) -> list:
p = _meta(g).get("pools")
out = []
if isinstance(p, dict):
for k, v in p.items():
try:
cap = int(v)
except (TypeError, ValueError):
continue
dom = _norm_dom(k)
if cap > 0 and dom:
out.append((dom[0], cap))
return out
def _pool_of(g: dict, n: dict) -> list:
dom = _norm_dom((n or {}).get("domain"))
if not dom:
return []
return [pre for pre, _c in _pool_caps(g) if _dom_overlap(dom, [pre])]
def _pool_full(g: dict, n: dict, occupied: list) -> dict:
"""节点所在池是否已满;满了返回 {"pool","cap","used"},否则 {}。"""
caps = dict(_pool_caps(g))
for pre in _pool_of(g, n):
cap = caps.get(pre) or 0
if cap <= 0:
continue
used = sum(1 for x in occupied if pre in _pool_of(g, x))
if used >= cap:
return {"pool": pre, "cap": cap, "used": used}
return {}
def _advance(g: dict, ws: str = "") -> list:
"""把「超时回收/子节点收敛/展开/条件判定」推到不动点;返回动了哪些(供写事件)。"""
acts: list = []
for _ in range(6):
moved = 0
moved += _reclaim_leases(g, acts)
moved += _complete_parents(g, acts)
moved += _expand_maps(g, acts)
moved += _expand_subgraphs(g, acts, ws)
moved += _apply_conditions(g, acts)
if not moved:
break
return acts
def _maybe_advance(g: dict) -> bool:
"""便宜的预判:没有可推进的事就不写盘(保持 ready 的只读语义)。"""
nodes = g.get("nodes") or {}
lc = _lease_cfg(g)
now = time.time()
if lc["on_timeout"] != "keep" and lc["timeout_sec"] > 0:
for n in nodes.values():
if _lease_over(g, n, now) is not None:
return True
for nid, n in nodes.items():
st = str(n.get("status"))
if st == "gating":
kids = [c for c in (n.get("children") or []) if c in nodes]
if kids and all(_st(nodes, c) in SETTLED for c in kids):
return True
elif st == "pending":
if n.get("expanded"):
continue
if n.get("map"):
if _deps_ok(nodes, n)[0] and _map_items(g, nodes, nid) is not None:
return True
continue
if n.get("subgraph"):
if _deps_ok(nodes, n)[0]:
return True
continue
v = _dep_verdict(nodes, n)
if v["verdict"] in ("skip", "block"):
return True
return False
def _advance_graph(ws: str, gid: str, g: dict):
acts: list = []
if not _maybe_advance(g):
return g, acts
def _mut(gg):
acts.extend(_advance(gg, ws))
g2 = _update(ws, gid, _mut)
for x in acts:
_event(ws, gid, x)
return g2, acts
# ── 就绪集 ──────────────────────────────────────────────────────────────
def _ready(g: dict) -> dict:
nodes = g.get("nodes") or {}
cap = int(_meta(g).get("max_parallel") or 3)
running = [k for k, n in nodes.items() if str(n.get("status")) == "running"]
ready, waiting = [], []
now = time.time()
for nid, n in nodes.items():
st = str(n.get("status"))
if st == "gating":
kids = [c for c in (n.get("children") or []) if c in nodes]
left = [c for c in kids if not _is_done(nodes, c)]
waiting.append({"node": nid, "why": "等待 %d 个子节点完成(还剩 %d)"
% (len(kids), len(left))})
continue
if st != "pending":
continue
if n.get("map") and not n.get("expanded"):
waiting.append({"node": nid, "why": "展开源未就绪(上游没完成或没有产物)"})
continue
nb = n.get("not_before")
if nb not in (None, ""):
try:
left = float(nb) - now
except (TypeError, ValueError):
left = 0
if left > 0:
waiting.append({"node": nid, "why": "重试等待中:还需 %d 秒"
% int(math.ceil(left))})
continue
okd, why = _deps_ok(nodes, n)
if not okd:
waiting.append({"node": nid, "why": why})
continue
dom = _norm_dom(n.get("domain"))
clash = [r for r in running
if dom and _norm_dom((nodes.get(r) or {}).get("domain"))
and _dom_overlap(dom, _norm_dom((nodes.get(r) or {}).get("domain")))]
if clash:
waiting.append({"node": nid, "why": "域与在跑的冲突:%s" % ", ".join(clash)})
continue
pf = _pool_full(g, n, [nodes.get(r) or {} for r in running])
if pf:
waiting.append({"node": nid, "why": "域池 %s 已满(%d/%d)"
% (pf["pool"], pf["used"], pf["cap"])})
continue
ready.append(nid)
slots = max(0, cap - len(running))
picked, backlog = [], []
for nid in ready:
if len(picked) >= slots:
backlog.append(nid)
waiting.append({"node": nid, "why": "并发上限 %d 已占满" % cap})
continue
dom = _norm_dom((nodes.get(nid) or {}).get("domain"))
clash = [p for p in picked
if dom and _norm_dom((nodes.get(p) or {}).get("domain"))
and _dom_overlap(dom, _norm_dom((nodes.get(p) or {}).get("domain")))]
if clash:
# 同批也不能撞域:check 已警告「永不同批」,ready 必须兑现
waiting.append({"node": nid, "why": "域与本批已选节点重叠:%s" % ", ".join(clash)})
continue
pf = _pool_full(g, nodes.get(nid) or {},
[nodes.get(r) or {} for r in running] +
[nodes.get(p) or {} for p in picked])
if pf:
waiting.append({"node": nid, "why": "域池 %s 已满(%d/%d)"
% (pf["pool"], pf["used"], pf["cap"])})
continue
picked.append(nid)
awaiting = [k for k, n in nodes.items() if str(n.get("status")) == "awaiting"]
return {"gid": g.get("gid"), "running": running, "ready": picked,
"backlog": backlog, "waiting": waiting, "awaiting": awaiting,
"pools": dict(_pool_caps(g)),
"cap": cap, "slots": slots}
def cmd_ready(a) -> int:
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
r["reclaimed"] = [x for x in acts if x.get("event") == "lease_timeout"]
r["expanded"] = [x for x in acts if x.get("event") == "expand"]
r["completed_parents"] = [x["node"] for x in acts if x.get("event") == "children_done"]
r["lease_notices"] = _lease_notices(g)
_out(r)
return 0
# ── 状态流转 ────────────────────────────────────────────────────────────
def cmd_start(a) -> int:
def _mut(g):
n = (g.get("nodes") or {}).get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
st = str(n.get("status"))
if st == "running":
raise SystemExit("节点 %s 已在跑,不能重复开工" % a.node)
if st == "done":
raise SystemExit("节点 %s 已完成,不能开工" % a.node)
if st == "gating" or (n.get("map") and not n.get("expanded")):
raise SystemExit("节点 %s 是展开节点,不能直接开工(由 ready 展开后跑子节点)"
% a.node)
# pending 与 blocked 都可开工:blocked 是「这轮没过」,重试时 attempts 保留
n["status"] = "running"
now = time.time()
n["lease_at"] = _iso(now)
n["lease_at_ts"] = now
if a.lease_sec is not None:
n["lease_sec"] = float(a.lease_sec)
n["starts"] = int(n.get("starts") or 0) + 1
n.pop("not_before", None)
n.pop("retry_at", None)
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"node": a.node, "to": "running", "by": a.by or ""})
n = (g.get("nodes") or {}).get(a.node) or {}
_out({"ok": True, "node": a.node, "status": "running",
"lease_sec": _node_lease_sec(g, n)})
return 0
def cmd_report(a) -> int:
if a.state not in ("done", "blocked"):
raise SystemExit("--state 只认 done / blocked")
retry_info = {}
def _mut(g):
n = (g.get("nodes") or {}).get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
if n.get("map") or str(n.get("status")) == "gating":
raise SystemExit("节点 %s 是展开父节点,本身不回报(子节点全完成自动 done)"
% a.node)
if a.evidence:
n["evidence"] = a.evidence
for art in (a.artifact or []):
n.setdefault("artifacts", []).append(str(art))
if a.state == "done": # F1:报 done 先过验收判据
gaps = _accepts_missing(n.get("accepts"), n.get("artifacts"), n.get("evidence"))
if gaps:
raise SystemExit("节点 %s 验收不通过(不置 done):\n - %s\n"
"补齐产物/证据后重新 report(accepts 不可绕过)"
% (a.node, "\n - ".join(gaps)))
n["accepts_checked_at"] = _now()
n["accepts_gaps"] = []
for k in ("lease_at", "lease_at_ts", "lease_sec"):
n.pop(k, None)
if a.state == "blocked":
n["attempts"] = int(n.get("attempts") or 0) + 1
n["blocked_by"] = "report"
rc = _retry_cfg(g, n)
left = rc["max"] - n["attempts"] + 1 # 还能再试几次
if rc["max"] > 0 and n["attempts"] <= rc["max"]:
delay = _delay_of(rc, n["attempts"])
at = time.time() + delay
n["status"] = "pending"
n["not_before"] = at
n["retry_at"] = _iso(at)
n["retry_exhausted"] = False
retry_info.update({"policy_max": rc["max"], "attempts": n["attempts"],
"requeued": True, "delay_sec": delay,
"retry_at": n["retry_at"],
"retries_left": max(0, left - 1)})
else:
n["status"] = "blocked"
n["retry_exhausted"] = rc["max"] > 0
n.pop("not_before", None)
n.pop("retry_at", None)
retry_info.update({"policy_max": rc["max"], "attempts": n["attempts"],
"requeued": False, "retry_at": None,
"exhausted": True})
else:
n.pop("not_before", None)
n.pop("retry_at", None)
if n.get("approval") and not n.get("approved_at"):
n["status"] = "awaiting" # 跑完了,等人审过目
n["awaiting_at"] = _now()
else:
n["status"] = "done"
n["done_at"] = _now()
g = _update(a.ws, a.gid, _mut)
final = _st(g.get("nodes") or {}, a.node)
_event(a.ws, a.gid, {"node": a.node,
"to": final if final == "awaiting"
else (a.state if a.state == "done" else "blocked"),
"evidence": a.evidence or "", "by": a.by or "",
"artifacts": list(a.artifact or [])})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node,
"status": str((g.get("nodes") or {}).get(a.node, {}).get("status")),
"next_ready": r["ready"],
"retry": retry_info or None,
"completed_parents": [x["node"] for x in acts
if x.get("event") == "children_done"]})
return 0
def cmd_expand(a) -> int:
acts = []
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
if not n.get("map"):
raise SystemExit("节点 %s 没有 map 声明,无可展开" % a.node)
if n.get("expanded"):
raise SystemExit("节点 %s 已展开过(children=%s)"
% (a.node, n.get("children")))
okd, why = _deps_ok(nodes, n)
if not okd and not a.force:
raise SystemExit("展开源未就绪:%s(要强制展开加 --force)" % why)
items = _map_items(g, nodes, a.node)
if items is None:
raise SystemExit("展开源未就绪:上游节点没完成或没有产物")
acts.append(_expand_one(g, nodes, a.node, items))
g = _update(a.ws, a.gid, _mut)
for x in acts:
_event(a.ws, a.gid, x)
g, more = _advance_graph(a.ws, a.gid, g)
acts.extend(more)
n = (g.get("nodes") or {}).get(a.node) or {}
_out({"ok": True, "node": a.node, "status": n.get("status"),
"children": n.get("children") or [],
"count": len(n.get("children") or [])})
return 0
# ── 人审 / 取消 / 派活 / 排期 / 进度 ─────────────────────────────────────
def cmd_approve(a) -> int:
"""人审节点:`awaiting` ⇒ done(默认)或 blocked(--reject)。
驳回时若节点带 `rework_to` ⇒ 直接走回退(人审判不过 = 判不过)。
"""
rw = {}
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
st = str(n.get("status"))
if st != "awaiting":
raise SystemExit("节点 %s 不在等人审(当前 %s)" % (a.node, st))
if a.reject:
tgt = _rework_target(n)
if tgt:
if tgt not in nodes:
raise SystemExit("回退目标不存在:%s" % tgt)
if tgt not in _ancestors(nodes, a.node):
raise SystemExit("回退目标 %s 不是 %s 的上游" % (tgt, a.node))
n["status"] = "blocked"
n["blocked_by"] = "rejected"
n["blocked_reason"] = a.reason or "人审驳回"
n["rejected_at"] = _now()
rw.update(_rework_reset(nodes, a.node, tgt, a.reason or "人审驳回"))
rw["auto"] = True
return
n["status"] = "blocked"
n["blocked_by"] = "rejected"
n["blocked_reason"] = a.reason or "人审驳回"
n["rejected_at"] = _now()
else:
n["status"] = "done"
n["done_at"] = _now()
n["approved_at"] = _now()
n["approved_by"] = a.by or "user"
g = _update(a.ws, a.gid, _mut)
st = _st(g.get("nodes") or {}, a.node)
_event(a.ws, a.gid, {"node": a.node, "to": st, "by": a.by or "",
"reject": bool(a.reject), "reason": a.reason or "",
**({"rework": rw} if rw else {})})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node, "status": _st(g.get("nodes") or {}, a.node),
"next_ready": r["ready"], "rework": rw or None})
return 0
def cmd_cancel(a) -> int:
"""取消节点(连坐它的子节点);`--all` 取消整图。"""
if not a.node and not a.all:
raise SystemExit("要给 --node <节点> 或 --all")
hit: list = []
def _mut(g):
nodes = g.get("nodes") or {}
if a.node and a.node not in nodes:
raise SystemExit("没有这个节点:%s" % a.node)
stack = [a.node] if a.node else list(nodes)
while stack:
nid = stack.pop()
n = nodes.get(nid)
if not n or str(n.get("status")) in ("done", "cancelled"):
continue
n["status"] = "cancelled"
n["cancelled_at"] = _now()
if a.by:
n["cancelled_by"] = a.by
if a.reason:
n["cancel_reason"] = a.reason
hit.append(nid)
stack.extend(n.get("children") or [])
g = _update(a.ws, a.gid, _mut)
for nid in hit:
_event(a.ws, a.gid, {"node": nid, "to": "cancelled",
"by": a.by or "", "reason": a.reason or ""})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "cancelled": hit,
"skipped_by_cancel": [x["node"] for x in acts if x.get("event") == "skip"],
"next_ready": r["ready"]})
return 0
def cmd_rework(a) -> int:
"""回退到上游重做:目标及其全部传递下游复位 pending,记 attempts/reworks 与事件。"""
res = {}
def _mut(g):
nodes = g.get("nodes") or {}
n = nodes.get(a.node)
if not n:
raise SystemExit("没有这个节点:%s" % a.node)
tgt = str(a.to or "").strip() or _rework_target(n)
if not tgt:
raise SystemExit("节点 %s 没有 rework_to,也没给 --to" % a.node)
if tgt not in nodes:
raise SystemExit("回退目标不存在:%s" % tgt)
if tgt == a.node:
raise SystemExit("回退目标不能是自己(要重做自己直接 start 该节点)")
if tgt not in _ancestors(nodes, a.node):
raise SystemExit("回退目标 %s 不是 %s 的上游(回退只能回到上游)"
% (tgt, a.node))
res.update(_rework_reset(nodes, a.node, tgt, a.reason or ""))
res["node"] = a.node
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "rework", "node": a.node, "to": res.get("target"),
"reset": res.get("reset") or [], "reason": a.reason or "",
"by": a.by or "", "attempts": res.get("attempts"),
"reworks": res.get("reworks")})
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": a.node, "target": res.get("target"),
"reset": res.get("reset") or [],
"dropped_children": res.get("dropped_children") or [],
"attempts": res.get("attempts"),
"reworks": res.get("reworks"),
"skipped_by_rework": [x["node"] for x in acts if x.get("event") == "skip"],
"next_ready": r["ready"]})
return 0
def cmd_add_node(a) -> int:
"""运行期加节点(与并发的 start/report 共用同一把目录锁,不会互相覆盖)。"""
nid = str(a.node or "").strip()
if not nid:
raise SystemExit("--node 不能为空")
deps = [x.strip() for x in str(a.deps or "").replace(",", ",").split(",") if x.strip()]
rule = str(a.deps_rule or "all_success")
if rule not in DEP_RULES:
raise SystemExit("--deps-rule 只认 %s" % " / ".join(DEP_RULES))
accepts = _norm_accepts(a.accepts or [])
rework_to = str(a.rework_to or "").strip()
def _mut(g):
nodes = g.get("nodes") or {}
if nid in nodes:
raise SystemExit("节点已存在:%s(换一个 id 或先 remove-edge)" % nid)
miss = [d for d in deps if d not in nodes]
if miss:
raise SystemExit("依赖不存在:%s" % ", ".join(miss))
if rework_to and rework_to not in nodes:
raise SystemExit("rework_to 指向的节点不存在:%s" % rework_to)
if rework_to == nid:
raise SystemExit("rework_to 不能指向自己")
nodes[nid] = {
"title": str(a.title or nid), "deps": deps,
"domain": _norm_dom(a.domain or []), "accepts": accepts,
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": rule, "added_at": _now(),
}
if rework_to:
nodes[nid]["rework_to"] = rework_to
if a.approval:
nodes[nid]["approval"] = True
g["nodes"] = nodes
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "add_node", "node": nid, "deps": deps,
"by": a.by or "", "rev": g.get("_rev")})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "node": nid, "deps": deps,
"status": _st(g.get("nodes") or {}, nid),
"rev": g.get("_rev"), "next_ready": r["ready"]})
return 0
def cmd_remove_edge(a) -> int:
"""删边:删掉「--to 依赖 --from」这一条依赖(边不存在即报错,⛔ 不静默空操作)。"""
src, dst = str(a.src or "").strip(), str(a.to or "").strip()
if not src or not dst:
raise SystemExit("要给 --from <上游> 和 --to <下游>")
def _mut(g):
nodes = g.get("nodes") or {}
for x in (src, dst):
if x not in nodes:
raise SystemExit("没有这个节点:%s" % x)
d = _deps(nodes[dst])
if src not in d:
raise SystemExit("边不存在:%s → %s(%s 的 deps=%s 里没有 %s)"
% (src, dst, dst, d, src))
nodes[dst]["deps"] = [x for x in d if x != src]
nodes[dst]["edge_removed_at"] = _now()
g = _update(a.ws, a.gid, _mut)
_event(a.ws, a.gid, {"event": "remove_edge", "from": src, "to": dst,
"by": a.by or "", "rev": g.get("_rev")})
g, _acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
_out({"ok": True, "from": src, "to": dst, "deps": _deps(
(g.get("nodes") or {}).get(dst) or {}), "rev": g.get("_rev"),
"next_ready": r["ready"]})
return 0
def cmd_tick(a) -> int:
"""自动派活:推进到不动点 → 算就绪集 → 直接把就绪节点开成 running。"""
started, missed = [], []
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
r = _ready(g)
cand = list(r["ready"])
if a.limit:
cand = cand[:int(a.limit)]
if cand and not a.dry_run:
def _mut(gg):
nodes = gg.get("nodes") or {}
for nid in cand:
n = nodes.get(nid)
if not n or str(n.get("status")) != "pending":
missed.append(nid)
continue
if _dep_verdict(nodes, n)["verdict"] != "go":
missed.append(nid)
continue
now = time.time()
n["status"] = "running"
n["lease_at"] = _iso(now)
n["lease_at_ts"] = now
if a.lease_sec is not None:
n["lease_sec"] = float(a.lease_sec)
n["starts"] = int(n.get("starts") or 0) + 1
n.pop("not_before", None)
n.pop("retry_at", None)
n["started_by"] = a.by or "tick"
started.append(nid)
g = _update(a.ws, a.gid, _mut)
for nid in started:
_event(a.ws, a.gid, {"node": nid, "to": "running", "by": a.by or "tick"})
g, more = _advance_graph(a.ws, a.gid, g)
acts.extend(more)
r2 = _ready(g)
_out({"gid": a.gid, "started": started, "missed": missed,
"dry_run": bool(a.dry_run), "advanced": acts,
"ready_now": r2["ready"], "running": r2["running"],
"awaiting": r2["awaiting"], "cap": r2["cap"], "slots": r2["slots"]})
return 0
def cmd_plan(a) -> int:
"""出一行排期(四条硬规则已按规范形态写好),⛔ 不写宿主库。"""
ws = str(Path(a.ws).resolve()).replace("\\", "/")
g = _load(a.ws, a.gid)
r = _ready(g)
after = 240 if a.after_sec is None else int(a.after_sec)
at_ms = int((time.time() + after) * 1000)
scope = a.scope or (ws.rstrip("/").rsplit("/", 1)[-1] + "/")
row = {
"name": "[图模式派活]-%s-%s" % (a.gid, time.strftime("%Y%m%d-%H%M")),
"prompt": ("跑一次图模式派活:python %s tick --ws %s --gid %s --by tick;"
"本轮只做这一件事,做完即停。"
% (Path(__file__).resolve(), ws, a.gid)),
"cwds": ws,
"workspace_scope": scope,
"next_run_at": at_ms,
"scheduled_at": time.strftime("%Y-%m-%dT%H:%M:%S", time.localtime(at_ms / 1000.0)),
}
_out({"ok": True, "gid": a.gid, "rows": [row],
"ready": r["ready"], "running": r["running"],
"rules": {"cwds": "正斜杠绝对路径", "workspace_scope": "线名/",
"next_run_at": "毫秒 epoch", "scheduled_at": "ISO 本地时间"},
"note": "只产出行,⛔ 不写宿主库(图模式零侵入)"})
return 0
def _remaining_chain(nodes: dict) -> list:
"""未完成节点之间的最长依赖链(关键路径)。"""
todo = {k: v for k, v in nodes.items()
if str(v.get("status")) not in ("done", "skipped", "cancelled")}
memo = {}
def best(nid):
if nid in memo:
return memo[nid]
memo[nid] = [nid]
cand = [nid]
for d in _deps(todo[nid]):
if d in todo:
chain = best(d) + [nid]
if len(chain) > len(cand):
cand = chain
memo[nid] = cand
return cand
out: list = []
for nid in todo:
c = best(nid)
if len(c) > len(out):
out = c
return out
def cmd_progress(a) -> int:
g = _load(a.ws, a.gid)
g, acts = _advance_graph(a.ws, a.gid, g)
nodes = g.get("nodes") or {}
cnt: dict = {}
for n in nodes.values():
s = str(n.get("status"))
cnt[s] = cnt.get(s, 0) + 1
active = len(nodes) - cnt.get("cancelled", 0) - cnt.get("skipped", 0)
done = cnt.get("done", 0)
percent = round(100.0 * done / active, 1) if active else 100.0
r = _ready(g)
cp = _remaining_chain(nodes)
_out({"gid": g.get("gid"), "title": g.get("title"), "total": len(nodes),
"by_status": cnt, "active": active, "done": done, "percent": percent,
"running": r["running"], "ready": r["ready"], "awaiting": r["awaiting"],
"blocked": [k for k, n in nodes.items() if str(n.get("status")) == "blocked"],
"skipped": [k for k, n in nodes.items() if str(n.get("status")) == "skipped"],
"cancelled": [k for k, n in nodes.items() if str(n.get("status")) == "cancelled"],
"critical_path": cp, "critical_len": len(cp),
"advanced": acts})
return 0
# ── 建图 ────────────────────────────────────────────────────────────────
def _gid_of(title: str) -> str:
safe = "".join(c for c in title if c not in '\\/:*?"<>|\r\n\t')[:24].strip()
return "%s-%s" % (time.strftime("%Y%m%d-%H%M"), safe or "graph")
def _norm_map(m) -> dict:
if not isinstance(m, dict):
raise SystemExit("map 必须是对象")
out = {}
if "over" in m:
if not isinstance(m["over"], list):
raise SystemExit("map.over 必须是列表")
out["over"] = list(m["over"])
if "from" in m:
f = m["from"]
if not isinstance(f, dict) or not str(f.get("node") or "").strip():
raise SystemExit('map.from 需要 {"node": "上游节点"} 形态')
out["from"] = {"node": str(f["node"]).strip(),
"pick": str(f.get("pick") or "artifacts")}
if not out:
raise SystemExit("map 需要 over(建图给列表)或 from(运行时读上游)")
if m.get("child_domain"):
out["child_domain"] = m["child_domain"]
if m.get("max_items"):
out["max_items"] = int(m["max_items"])
if m.get("as"):
out["as"] = str(m["as"])
return out
def cmd_init(a) -> int:
if a.nodes_json:
raw = sys.stdin.read() if a.nodes_json == "-" else \
Path(a.nodes_json).read_text(encoding="utf-8")
spec = json.loads(raw)
elif a.kpi:
spec = {"nodes": [
{"id": "N%d" % (i + 1), "title": k.strip(), "deps": [],
"domain": [], "accepts": [k.strip()]}
for i, k in enumerate([x for x in a.kpi.replace(";", ";").split(";") if x.strip()])
]}
else:
raise SystemExit("要给 --kpi 或 --nodes-json")
if a.on_timeout not in TIMEOUT_POLICIES:
raise SystemExit("--on-timeout 只认 %s" % " / ".join(TIMEOUT_POLICIES))
if a.retry_backoff not in BACKOFFS:
raise SystemExit("--retry-backoff 只认 %s" % " / ".join(BACKOFFS))
nodes = {}
for i, n in enumerate(spec.get("nodes") or []):
raw = n.get("id")
nid = str(raw).strip() if raw not in (None, "") else ""
if not nid:
k = i + 1
while ("N%d" % k) in nodes: # 自动编号要避开已占用的 id
k += 1
nid = "N%d" % k
if nid in nodes: # 显式重复 ⇒ 硬错,⛔ 不静默覆盖前一个
raise SystemExit("节点 id 重复:%s" % nid)
rule = str(n.get("deps_rule") or a.deps_rule or "all_success")
if rule not in DEP_RULES:
raise SystemExit("%s 的 deps_rule 非法:%r(只认 %s)"
% (nid, rule, " / ".join(DEP_RULES)))
nodes[nid] = {
"title": str(n.get("title") or nid),
"deps": [str(x) for x in (n.get("deps") or [])],
"domain": _norm_dom(n.get("domain")),
"accepts": _norm_accepts(n.get("accepts") or []),
"status": "pending", "evidence": "", "artifacts": [], "attempts": 0,
"deps_rule": rule,
}
if n.get("retry"):
if not isinstance(n["retry"], dict):
raise SystemExit("%s 的 retry 必须是对象" % nid)
nodes[nid]["retry"] = _retry_cfg({"meta": {}}, {"retry": n["retry"]})
if n.get("lease_sec") not in (None, ""):
nodes[nid]["lease_sec"] = float(n["lease_sec"])
if n.get("on_timeout"):
if str(n["on_timeout"]) not in TIMEOUT_POLICIES:
raise SystemExit("%s 的 on_timeout 非法:%r" % (nid, n["on_timeout"]))
nodes[nid]["on_timeout"] = str(n["on_timeout"])
if n.get("map"):
nodes[nid]["map"] = _norm_map(n["map"])
if n.get("subgraph"):
sg = n["subgraph"]
if not isinstance(sg, dict):
raise SystemExit('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "另一张图"}'
% nid)
has_nodes = isinstance(sg.get("nodes"), list)
has_gid = bool(str(sg.get("gid") or "").strip())
if has_nodes and has_gid:
raise SystemExit("%s 的 subgraph 只能选一种:内联 nodes 或引用 gid" % nid)
if not has_nodes and not has_gid:
raise SystemExit('%s 的 subgraph 要写 {"nodes": [...]} 或 {"gid": "..."}'
% nid)
if n.get("map"):
raise SystemExit("%s 同时声明了 map 与 subgraph(只能选一个)" % nid)
sub = {}
if has_nodes:
sub["nodes"] = [dict(s) for s in sg["nodes"] if isinstance(s, dict)]
else:
sub["gid"] = str(sg["gid"]).strip()
if sg.get("child_domain"):
sub["child_domain"] = str(sg["child_domain"])
nodes[nid]["subgraph"] = sub
if n.get("rework_to"):
rt = _rework_target({"rework_to": n["rework_to"]})
if not rt:
raise SystemExit("%s 的 rework_to 不能为空" % nid)
if rt == nid:
raise SystemExit("%s 的 rework_to 不能指向自己" % nid)
nodes[nid]["rework_to"] = rt
if n.get("when"):
if not isinstance(n["when"], list):
raise SystemExit("%s 的 when 必须是列表" % nid)
clauses = []
for c in n["when"]:
if not isinstance(c, dict) or not str(c.get("node") or "").strip():
raise SystemExit('%s 的 when 子句要写 {"node": "A", "status": "done"}'
% nid)
clauses.append({"node": str(c["node"]).strip(),
"status": str(c.get("status") or "done")})
nodes[nid]["when"] = clauses
if n.get("on_dep_fail"):
if str(n["on_dep_fail"]) not in ON_DEP_FAIL:
raise SystemExit("%s 的 on_dep_fail 非法:%r(只认 %s)"
% (nid, n["on_dep_fail"], " / ".join(ON_DEP_FAIL)))
nodes[nid]["on_dep_fail"] = str(n["on_dep_fail"])
if n.get("approval"):
nodes[nid]["approval"] = True
gid = a.gid or _gid_of(a.title)
g = {"gid": gid, "title": a.title, "created_at": _now(),
"meta": {"max_parallel": int(spec.get("max_parallel") or a.max_parallel or 3),
"lease": {"timeout_sec": float(a.lease_sec) if a.lease_sec is not None
else DEFAULT_LEASE_SEC,
"on_timeout": a.on_timeout},
"retry": {"max": int(a.retry_max or 0),
"delay_sec": float(a.retry_delay or 0),
"backoff": a.retry_backoff,
"factor": float(a.retry_factor or 2)}},
"nodes": nodes}
d = _gdir(a.ws, gid)
pools = {}
if isinstance(spec.get("pools"), dict):
pools.update(spec["pools"])
for item in (a.pool or []):
s = str(item)
if "=" not in s:
raise SystemExit("--pool 要写成 域前缀=上限(如 docs/ui/=1)")
k, v = s.split("=", 1)
pools[k.strip()] = v
if pools:
g["meta"]["pools"] = {}
for k, v in pools.items():
dom = _norm_dom(k)
if not dom:
raise SystemExit("域池的域不能为空")
try:
cap = int(v)
except (TypeError, ValueError):
raise SystemExit("域池上限不是整数:%s=%r" % (k, v))
if cap <= 0:
raise SystemExit("域池上限必须 ≥1:%s=%r" % (k, v))
g["meta"]["pools"][dom[0]] = cap
d.mkdir(parents=True, exist_ok=True)
_save(a.ws, gid, g)
_event(a.ws, gid, {"event": "init", "nodes": list(nodes)})
_out({"ok": True, "gid": gid, "dir": str(d), "nodes": len(nodes),
"note": "独立图:⛔ 不参与现有机制(不写 goal.json / tasks.json)"})
return 0
def cmd_show(a) -> int:
_out(_load(a.ws, a.gid))
return 0
def cmd_list(a) -> int:
root = _root(a.ws)
rows = []
if root.is_dir():
for d in sorted(root.iterdir()):
p = d / "graph.json"
if not p.is_file():
continue
try:
g = json.loads(p.read_text(encoding="utf-8"))
except Exception:
continue
nodes = g.get("nodes") or {}
cnt = {}
for n in nodes.values():
cnt[str(n.get("status"))] = cnt.get(str(n.get("status")), 0) + 1
rows.append({"gid": g.get("gid"), "title": g.get("title"),
"nodes": len(nodes), "by_status": cnt,
"created_at": g.get("created_at")})
_out({"ws": str(Path(a.ws).resolve()), "graphs": rows})
return 0
# ── 入口 ────────────────────────────────────────────────────────────────
def main(argv=None) -> int:
ap = argparse.ArgumentParser(prog="graphctl", description="图模式独立实现(不碰现有机制)")
sub = ap.add_subparsers(dest="cmd", required=True)
p = sub.add_parser("init", help="按判据或节点定义建图")
p.add_argument("--ws", required=True)
p.add_argument("--title", required=True)
p.add_argument("--gid", default="")
p.add_argument("--kpi", default="", help="判据,分号分隔;每条生成一个节点")
p.add_argument("--nodes-json", default="", help="节点定义文件(- 表示标准输入)")
p.add_argument("--max-parallel", type=int, default=3)
p.add_argument("--lease-sec", type=float, default=None,
help="租约多少秒算超时(默认 3600;0 = 不判超时)")
p.add_argument("--on-timeout", default="requeue",
help="超时处置:requeue(默认)/block/keep")
p.add_argument("--retry-max", type=int, default=0, help="默认自动重试次数(默认 0)")
p.add_argument("--retry-delay", type=float, default=0, help="重试基础间隔秒(默认 0)")
p.add_argument("--retry-backoff", default="fixed", help="fixed(默认)/linear/exponential")
p.add_argument("--retry-factor", type=float, default=2, help="指数退避倍数(默认 2)")
p.add_argument("--deps-rule", default="all_success",
help="默认依赖规则:all_success(默认)/all_done/one_success")
p.add_argument("--pool", action="append", default=[],
help="按域分池:域前缀=并发上限,可重复(如 docs/ui/=1)")
p.set_defaults(fn=cmd_init)
for name, fn in (("check", cmd_check), ("ready", cmd_ready), ("show", cmd_show)):
p = sub.add_parser(name)
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.set_defaults(fn=fn)
p = sub.add_parser("start", help="节点开工(记租约)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--by", default="")
p.add_argument("--lease-sec", type=float, default=None, help="本次开工的租约秒数(覆盖默认)")
p.set_defaults(fn=cmd_start)
p = sub.add_parser("report", help="节点回报")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--state", required=True)
p.add_argument("--evidence", default="")
p.add_argument("--artifact", action="append", default=[],
help="产物清单,可重复(供 map.from pick=artifacts 读取)")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_report)
p = sub.add_parser("rework", help="回退到上游重做(目标及其下游复位 pending)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--to", default="", help="回退目标(默认取节点自己的 rework_to)")
p.add_argument("--reason", default="")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_rework)
p = sub.add_parser("add-node", help="运行期加节点")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--title", default="")
p.add_argument("--deps", default="", help="依赖节点,逗号分隔")
p.add_argument("--domain", action="append", default=[], help="域前缀,可重复")
p.add_argument("--accepts", action="append", default=[],
help="验收判据,可重复(artifact:… / evidence:… / 纯文本)")
p.add_argument("--deps-rule", default="all_success")
p.add_argument("--rework-to", default="")
p.add_argument("--approval", action="store_true")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_add_node)
p = sub.add_parser("remove-edge", help="删边:删掉「--to 依赖 --from」")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--from", dest="src", required=True, help="上游节点")
p.add_argument("--to", required=True, help="下游节点(它的 deps 里删掉上游)")
p.add_argument("--by", default="")
p.set_defaults(fn=cmd_remove_edge)
p = sub.add_parser("expand", help="手动触发展开")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--force", action="store_true")
p.set_defaults(fn=cmd_expand)
p = sub.add_parser("approve", help="人审通过/驳回(awaiting ⇒ done/blocked)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", required=True)
p.add_argument("--by", default="")
p.add_argument("--reject", action="store_true")
p.add_argument("--reason", default="")
p.set_defaults(fn=cmd_approve)
p = sub.add_parser("cancel", help="取消节点(连坐子节点)或整图")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--node", default="")
p.add_argument("--all", action="store_true")
p.add_argument("--by", default="")
p.add_argument("--reason", default="")
p.set_defaults(fn=cmd_cancel)
p = sub.add_parser("tick", help="自动派活:算就绪集并直接开工")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--by", default="tick")
p.add_argument("--limit", type=int, default=0, help="本次最多开几个(默认不额外限)")
p.add_argument("--dry-run", action="store_true", help="只算不开")
p.add_argument("--lease-sec", type=float, default=None)
p.set_defaults(fn=cmd_tick)
p = sub.add_parser("plan", help="出一行排期(四条硬规则已规范化)")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.add_argument("--scope", default="", help="workspace_scope,默认取工作区目录名")
p.add_argument("--after-sec", type=int, default=None, help="多久后触发(默认 240 秒)")
p.set_defaults(fn=cmd_plan)
p = sub.add_parser("progress", help="进度与关键路径")
p.add_argument("--ws", required=True)
p.add_argument("--gid", required=True)
p.set_defaults(fn=cmd_progress)
p = sub.add_parser("list")
p.add_argument("--ws", required=True)
p.set_defaults(fn=cmd_list)
a = ap.parse_args(argv)
return a.fn(a)
if __name__ == "__main__":
sys.exit(main())