diff --git a/AGENTS.md b/AGENTS.md index 291bd45..76eb3d5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -31,16 +31,19 @@ - P4定义发布使用schema_version=1有限结构，词法作用域禁止跨分支/前向读取，任务策略显式冻结；SDK类型独立。发布按认证→应用额度→操作身份→定义→排序队列持锁；原键与同版本去重仍校验当前权限。定义/版本/revision/审计/回执同事务；同版本不同内容不得覆盖。保留精确数字，校验存储原文与结构摘要；不得把发布通过或DSL支持误称为实例执行已实现。 -- P4定义子项已通过真实MySQL、HTTP丢回执、独立SDK和全量回归（docs/reports/phase-4-definitions.md）。实例启动/固定版本/首批任务同事务的内部存储已补齐；内部事件消费、顺序与组合推进及根接续已接入；TIMER与后续POLL时钟已接入，继续SIGNAL及实例公共SDK；P4整体和P5–P7仍未完成。 +- P4定义子项已通过真实MySQL、HTTP丢回执、独立SDK和全量回归（docs/reports/phase-4-definitions.md）。实例启动/固定版本/首批任务同事务的内部存储已补齐；内部事件消费、顺序与组合推进及根接续已接入；TIMER与后续POLL时钟已接入，SIGNAL及最终查询竞争已接入，继续完整控制及实例公共SDK；P4整体和P5–P7仍未完成。 - 内部结果交付使用workflow_activation所有者并固定激活ID，TASK经activation.task_id、POLL经轮次校验归属；weave_inbox来源及独立快照与Pulse结果同事务，先于外部结果流行锁，不在结果路径反锁实例/步骤/订阅。迁移000017隔离缺快照历史，禁止用当前任务重建旧事件。 - weave/engine仅计算有限决策；实例受理、Inbox消费与根接续已接入，完整等待/控制运行器继续开发。Plan最多32个变更，More必须持久化待续；耗尽变更预算仍须遍历其他已活动分支，不能误判在途工作不可达。冻结分支选择，精确数值，不以PENDING查询结果完成POLL。后续消费确认/当前激活校验/新任务或检查点必须同事务，不能把纯快照往返测试当作真实进程恢复。 -- 实例内部受理已实现，公共实例HTTP/SDK及信号运行器尚未开放；结果消费和根待续扫描已接入后台。Start按认证→应用额度→原身份→排序队列建立门控/固定版本/全部步骤/首批工作，所有失败同事务回滚。首批推进递增revision至2，原键返回冻结回执且仍核验当前权限，不重算截止。 +- 实例内部受理已实现，信号内部运行器已接入，公共实例/信号HTTP及独立SDK尚未开放；结果消费和根待续扫描已接入后台。Start按认证→应用额度→原身份→排序队列建立门控/固定版本/全部步骤/首批工作，所有失败同事务回滚。首批推进递增revision至2，原键返回冻结回执且仍核验当前权限，不重算截止。 - 实例根步骤`$root`/JOIN不属于用户定义，Steps查询排除它；More对应根步骤版本化持久检查点。TASK直接关联激活task_id，POLL通过轮次关联，实际任务截止必须受实例/查询绝对截止限制。Wait名称及契约版本冻结，旧NULL等待须核实，不能盲目解释；新关联512字节、存储兼容历史764字节。 - 结果消费按身份→应用额度→排序队列→实例→门控→步骤/Inbox持锁，禁止先锁Inbox再锁父资源。来源使用独立冻结快照，旧激活/低版本留痕；数据库故障不归类为业务失败。消费确认、步骤、后继或根待续、事件及审计同事务。 - 后继满额用SAVEPOINT回滚整轮新工作，保留已接受结果和待续；恢复必须重读持久图。多次根推进使用独立系统审计操作身份；迟到证据也递增实例历史版本，禁止复用已有workflow.advanced唯一身份。 -- 本轮消费运行器和真实Orbit强杀/双节点接管已验证，见docs/reports/phase-4-runtime.md。TIMER到期与POLL后续轮次已接入，见docs/13-workflow-clock.md；SIGNAL及最终信号竞争、FAIL_AFTER_SETTLED分支结清、控制API和实例SDK仍须完成，不得按部分自动运行宣称P4/P5完成。 +- 本轮消费运行器和真实Orbit强杀/双节点接管已验证，见docs/reports/phase-4-runtime.md。TIMER到期与POLL后续轮次已接入，见docs/13-workflow-clock.md；SIGNAL及最终信号竞争已接入（docs/14-workflow-signals.md）；FAIL_AFTER_SETTLED分支结清、控制API和实例SDK仍须完成，不得按部分自动运行宣称P4/P5完成。 - TIMER/POLL复用weave_pending_advances及到期索引，时钟固定激活ID；迁移000019只回填原等待/查询时钟，不改截止和预算。后续POLL只由已确认POLL_PENDING开启，首轮精确输入/资源键冻结，每轮Pulse尝试独立计数；满额/暂停不消耗轮数。截止遇到已落库结果先等待Inbox，仍在途则关闭门控但保留ACTIVE和当前任务，许可由Pulse结清；不得由时钟假定外部请求已结束。 + +- SIGNAL受理要求显式workflows:signal、固定目标步骤/激活代次/内容契约/精确关联及原因；每实例全部历史信号上限1024，原键先返回冻结回执。accepted_at/expires_at及摘要与不可变审计核验；available_at只用于到期检查退避。 +- 最终信号与当前查询按原受理/事件时间裁决，PENDING不争夺终结权；保存信号赢家后复用Pulse取消规则，在途任务保留许可直至真实结果。未知/冲突结果转核查，已有核查不得自动清除。迁移000020隔离旧PENDING身份并保留历史原文，不引入外键。 diff --git a/README.md b/README.md index cc1faab..a533d48 100644 --- a/README.md +++ b/README.md @@ -10,7 +10,7 @@ Orbit 是面向多个业务项目的通用任务调度与流程编排平台， 已完成 P0–P3 平台功能阶段验收：35 个数据模型、版本化迁移、凭据及范围校验、队列与任务 API、WSS 执行、独立 SDK、自动 Worker 和 [执行尝试/证据查询](docs/api/execution-history.md)。[Worker Schema](docs/api/worker-schema.md) 与真实进程强杀恢复回归已通过；[安全重试](docs/api/retries.md)、[定时计划](docs/api/schedules.md)、[运行中取消](docs/api/cancellation.md) 、[队列资源控制](docs/api/queues.md) 、[应用共享配额](docs/api/application-resources.md) 、[资源键互斥](docs/api/keyed-concurrency.md) 和 [优先级/节点公平派发](docs/api/priority-and-fairness.md) 已接入服务端和 SDK，[可靠结果订阅](docs/api/subscriptions.md) 已实现持久化核心、HTTP/WSS、管理 SDK 与示例，[消费 SDK](docs/sdk/event-consumer.md) 和 [独立 MySQL Inbox 示例](docs/examples/result-inbox.md) 已实现；[批量任务受理](docs/api/batch-tasks.md) 已提供逐项幂等、失败分类与 SDK 示例；[人工核查与重试](docs/api/task-recovery.md) 已提供带证据的管理接口、关联新任务、审计查询及运行示例；[业务键与时间范围查询](docs/api/task-queries.md) 已提供过滤、分页与SDK场景；[任务来源查询](docs/api/task-source.md) 已提供人工重试父任务、计划触发记录与固定版本；[业务事务Outbox](docs/examples/task-outbox.md) 已提供独立业务库、稳定提交身份、发送恢复与CLI；[相对延迟执行](docs/examples/delayed-task.md)已提供数据库计时及原请求恢复；完整范围核对见[P3验收](docs/reports/phase-3.md)，流程引擎与生产验证按 [持续开发计划](docs/08-development-plan.md) 继续实现。 -P4已实现[流程定义与不可变版本发布](docs/api/workflow-definitions.md)、[结构版本1](docs/api/workflow-definition-format.md)及[SDK发布例子](docs/examples/workflow-definition.md)。[有限步骤决策器与内部结果交付](docs/10-workflow-engine-core.md)已实现；已补[实例受理及首批工作存储](docs/11-workflow-admission.md)。[结果消费、顺序/分支/并行推进及根检查点接续](docs/12-workflow-runtime.md)已接入后台运行器。[定时等待、后续查询轮次及原预算恢复](docs/13-workflow-clock.md)已接入；信号、失败结清及实例公共API/SDK继续开发，发布定义不会启动任务。 +P4已实现[流程定义与不可变版本发布](docs/api/workflow-definitions.md)、[结构版本1](docs/api/workflow-definition-format.md)及[SDK发布例子](docs/examples/workflow-definition.md)。[有限步骤决策器与内部结果交付](docs/10-workflow-engine-core.md)已实现；已补[实例受理及首批工作存储](docs/11-workflow-admission.md)。[结果消费、顺序/分支/并行推进及根检查点接续](docs/12-workflow-runtime.md)已接入后台运行器。[定时等待、后续查询轮次及原预算恢复](docs/13-workflow-clock.md)已接入；[外部信号和查询最终信号竞争](docs/14-workflow-signals.md)已接入内部运行器；失败结清及实例公共API/SDK继续开发，发布定义不会启动任务。 数据库禁止外键，关联一致性通过服务端事务校验，索引、唯一约束和 CHECK 保留；readiness 同时检查迁移及零外键要求。这不表示完整平台已具备生产服务能力。验证证据见 [P0](docs/reports/phase-0.md)、[P1](docs/reports/phase-1.md)、[P2 阶段验收](docs/reports/phase-2.md)、[安全重试](docs/reports/phase-3-safe-retry.md)、[计划调度](docs/reports/phase-3-schedules.md)、[取消与无外键回归](docs/reports/phase-3-cancellation.md) 及 [队列资源回归](docs/reports/phase-3-queue-resources.md)。 @@ -33,6 +33,7 @@ weave/model/ 定义、实例、步骤、信号和查询轮次 weave/definition/ 有限结构/类型校验与定义发布契约 weave/engine/ 不可变规则的有限步骤决策与精确结果映射 weave/instance/ 内部实例受理/查询契约，公共运行API继续开发 +weave/signal/ 内部可靠信号受理/查询契约及显式激活身份 internal/config/ 配置校验 internal/storage/mysql/ 连接、事务、迁移及集成测试 internal/transport/httpserver/ HTTP 接入 diff --git a/docs/06-workflow-spec.md b/docs/06-workflow-spec.md index 2e71198..e37c60c 100644 --- a/docs/06-workflow-spec.md +++ b/docs/06-workflow-spec.md @@ -1,6 +1,6 @@ # Weave 流程编排功能与设计 -> 版本 0.4 · 2026-09-18。首版功能范围已确认。定义结构、发布校验与不可变版本已实现，见[定义格式](api/workflow-definition-format.md)和[发布API](api/workflow-definitions.md)；[实例受理](11-workflow-admission.md)、[结果消费与根推进](12-workflow-runtime.md)、[TIMER与POLL时钟](13-workflow-clock.md)已接入。信号、完整管理及实例公共API/SDK仍待P4–P5实现验收。 +> 版本 0.5 · 2026-09-18。首版功能范围已确认。定义结构、发布校验与不可变版本已实现，见[定义格式](api/workflow-definition-format.md)和[发布API](api/workflow-definitions.md)；[实例受理](11-workflow-admission.md)、[结果消费与根推进](12-workflow-runtime.md)、[TIMER与POLL时钟](13-workflow-clock.md)已接入。[信号等待及最终查询竞争](14-workflow-signals.md)已接入内部运行器；完整管理及实例公共API/SDK仍待P4–P5实现验收。 ## 1. 职责与边界 diff --git a/docs/07-data-model.md b/docs/07-data-model.md index 9e7d226..3c9c0ca 100644 --- a/docs/07-data-model.md +++ b/docs/07-data-model.md @@ -75,7 +75,7 @@ | weave | `InboxEvent` | `weave_inbox` | | weave | `PendingAdvance` | `weave_pending_advances` | -`orbit_events` 保存原始状态事件，`orbit_result_events` 使用相同事件 ID 保存队列、提交序号和冻结快照，`orbit_event_streams` 串行分配命名空间内的结果序号；对外投递使用 `orbit_deliveries`，内部流程结果原子写入 `weave_inbox`及其独立冻结快照，已接入[实例消费推进](12-workflow-runtime.md)。`weave_pending_advances` 保存根步骤续接和TIMER/POLL时钟检查点。 +`orbit_events` 保存原始状态事件，`orbit_result_events` 使用相同事件 ID 保存队列、提交序号和冻结快照，`orbit_event_streams` 串行分配命名空间内的结果序号；对外投递使用 `orbit_deliveries`，内部流程结果原子写入 `weave_inbox`及其独立冻结快照，已接入[实例消费推进](12-workflow-runtime.md)。`weave_pending_advances` 保存根步骤续接和TIMER/SIGNAL/POLL时钟检查点。 ## 4. 已落实的数据库约束 @@ -199,4 +199,8 @@ Wait新增SignalName和PayloadVersion两个中文注释字段；correlation_key ## 等待时钟回填（000019） -复用PendingAdvance保存ROOT、TIMER、POLL检查点；新增内部JSON结构字段均有中文注释，不新增持久化列或实体，仍为35个实体/295个直接持久化字段。TIMER/POLL检查点固定激活ID，实际推进核验同命名空间归属、原截止及剩余预算。迁移只回填历史有效等待的唤醒身份和时间，保留已有检查点；无外键、无级联，见[时钟运行约定](13-workflow-clock.md)。 +复用PendingAdvance保存ROOT、TIMER、POLL检查点；新增内部JSON结构字段均有中文注释，不新增持久化列或实体，截至该迁移为35个实体/295个直接持久化字段。TIMER/POLL检查点固定激活ID，实际推进核验同命名空间归属、原截止及剩余预算。迁移只回填历史有效等待的唤醒身份和时间，保留已有检查点；无外键、无级联，见[时钟运行约定](13-workflow-clock.md)。 + +## 信号身份与赢家增量（000020） + +Signal新增payload_version、target_activation_sequence、evidence_ref、disposition、available_at；Wait新增winner_signal_id。当前35个实体、含公共Record的301个直接持久化字段声明，全部中文注释。新信号精确关联最多512字节，存储兼容历史764字节；过期检查退避不改变expires_at。赢家唯一约束只防重复，实例/等待/激活/信号归属及不可变受理证明仍由事务验证，外键保持0。历史不完整身份隔离和升级回填见[信号约定](14-workflow-signals.md)。 diff --git a/docs/08-development-plan.md b/docs/08-development-plan.md index 98d1b77..d60a7ae 100644 --- a/docs/08-development-plan.md +++ b/docs/08-development-plan.md @@ -28,14 +28,22 @@ | P1 任务受理闭环 | 应用凭据与作用域权限、队列、幂等提交、查询、待执行取消、受理事件；SDK 客户端 | API/SDK 创建查询闭环；重复键与内容冲突、响应未确认重报、跨命名空间拒绝、参数边界 | 已完成并通过阶段回归（见 P1 报告；真实断路/提交 ACK 丢失注入留后续故障阶段） | | P2 Worker 执行闭环 | WSS 接入、能力/容量、领取与开始、租约、续租、结果确认、排空与恢复；Worker SDK | 真 Worker 执行；失联/重连/重复确认/旧尝试拒绝；业务效果未知转核查；节点故障接管 | 已完成并通过阶段回归：服务端、自动 SDK、历史查询、完整 Schema 与真实进程强杀恢复；测试拓扑和剩余生产故障矩阵见 P2 验收报告 | | P3 Pulse 完整能力 | 安全重试、延时、Cron/固定间隔、漏跑策略、计划版本、运行中取消、并发许可、配额/背压、可靠结果订阅、批量受理、独立任务核查/人工重试、查询筛选与事务Outbox接入 | 多实例领取/触发竞争、重试上限、取消竞争、队列隔离、积压恢复、事件重复交付 | 已完成平台功能阶段验收：[原范围逐项核对](reports/phase-3.md)、[本轮完整回归](reports/phase-3-delay-recovery.md)。补齐相对延迟、改参数关联示例和四种重试全停恢复场景；20个可运行示例全部通过。全平台生产门槛及SDK首版交付剩余项仍按P6–P7推进 | -| P4 Weave 主流程 | SDK 定义构建与发布校验、实例、顺序/分支/结构化并行汇合、信号/定时等待、版本绑定 | 重启接续、早到/重复信号、并行竞争、内部事件重放、发布版本兼容；端到端通用流程 | 定义格式/词法类型校验、不可变发布、独立SDK及六类节点定义例子已实现，已通过[子项验收](reports/phase-4-definitions.md)；已补[有限决策器及内部结果交付](reports/phase-4-engine-core.md)；[实例受理及首批事务](reports/phase-4-admission.md)已实现；[结果消费和根接续](reports/phase-4-runtime.md)已接入后台，顺序/分支/并行及真实进程恢复已验证；[TIMER与后续POLL轮次](13-workflow-clock.md)已接入；信号、完整控制和实例公共API/SDK继续开发 | -| P5 Weave 恢复与查询 | 有界查询、暂停继续、取消、人工核查恢复、控制门控与审计 | 轮次与尝试独立预算；暂停/开始/回执/超时竞争；恢复保留预算；未知结果不盲目重复 | 有界查询轮次/尝试独立预算及原截止恢复已接入内部运行器；信号竞争、完整控制/人工恢复及公共API/SDK继续开发 | +| P4 Weave 主流程 | SDK 定义构建与发布校验、实例、顺序/分支/结构化并行汇合、信号/定时等待、版本绑定 | 重启接续、早到/重复信号、并行竞争、内部事件重放、发布版本兼容；端到端通用流程 | 定义格式/词法类型校验、不可变发布、独立SDK及六类节点定义例子已实现，已通过[子项验收](reports/phase-4-definitions.md)；已补[有限决策器及内部结果交付](reports/phase-4-engine-core.md)；[实例受理及首批事务](reports/phase-4-admission.md)已实现；[结果消费和根接续](reports/phase-4-runtime.md)已接入后台，顺序/分支/并行及真实进程恢复已验证；[TIMER与后续POLL轮次](13-workflow-clock.md)已接入；[信号消费和最终查询竞争](14-workflow-signals.md)已接入内部运行器；完整控制和实例公共API/SDK继续开发 | +| P5 Weave 恢复与查询 | 有界查询、暂停继续、取消、人工核查恢复、控制门控与审计 | 轮次与尝试独立预算；暂停/开始/回执/超时竞争；恢复保留预算；未知结果不盲目重复 | 有界查询轮次/尝试独立预算及原截止恢复已接入内部运行器；最终信号竞争及在途查询结清已接入；完整控制/人工恢复及公共API/SDK继续开发 | | P6 生产验证与运维 | 完整监控、凭据轮换、备份恢复、滚动升级、运行手册、可复现负载和故障工具 | 对照生产验收清单执行，报告环境/指标/遗留项；容量、RPO/RTO 按真实拓扑验证 | 待开发 | | P7 首版交付核对 | SDK 兼容与发布准备、完整 API 手册、场景示例、全量回归及交付清单 | 新接入者按文档跑通全部示例；所有首版能力可追溯到实现与测试；报告实际生产边界 | 待开发 | SDK、文档与示例随 P1–P5 同步开发，P7 是全量核对，不是最后才开始补文档。 -P4/P5定时等待与后续查询轮次的实现及回归记录见[时钟报告](reports/phase-4-clock.md)。实例公共入口继续与信号/控制能力同步交付，不能将内部存储链路作为完整SDK验收。 +P4/P5定时等待与后续查询轮次的实现及回归记录见[时钟报告](reports/phase-4-clock.md)。外部信号与最终查询竞争见[信号报告](reports/phase-4-signals.md)。实例公共入口继续与控制能力同步交付，不能将内部存储链路作为完整SDK验收。 + +### P4/P5 下一步实施顺序 + +1. 完成FAIL_AFTER_SETTLED与显式取消的分支结清：持久取消意图、复用Pulse取消、收集已发出查询/任务结果、等待全部已激活工作结清后才进入失败或取消终态。效果未知仍保留核查。 +2. 完成实例暂停/继续/取消内部操作，以control_revision、操作幂等身份、明确原因及审计保护控制决定；暂停不延长原截止，继续不覆盖核查或取消状态。 +3. 完成人工核查恢复与旧激活隔离，保留原预算、结果和证据，不能通过换激活/新键绕过未知效果或执行上限。 +4. 同步开放实例/步骤/信号/来源/审计HTTP、独立SDK和端到端场景，覆盖原键恢复、权限、版本冲突、竞争与进程全停；公共能力逐项记录实际验收结果。 +5. 转入P6/P7的生产故障、容量、运维、发布与全部文档核对。内部存储阶段通过不会提前勾选公共接口或生产验收。 ### P3 功能范围核对 diff --git a/docs/10-workflow-engine-core.md b/docs/10-workflow-engine-core.md index f49bacc..deb5e10 100644 --- a/docs/10-workflow-engine-core.md +++ b/docs/10-workflow-engine-core.md @@ -74,7 +74,7 @@ v16及以前的未处理记录缺少冻结来源，升级后统一标为QUARANTI ## 4. 后续存储适配必须满足的条件 -以下保留存储运行器的完整要求。第1项已按[受理报告](reports/phase-4-admission.md)验证；第2–4项结果消费和根接续已接入[实际运行器](12-workflow-runtime.md)，第5项的TIMER/POLL已接入[时钟运行器](13-workflow-clock.md)，其余信号、完整控制、接口和SDK仍继续实现： +以下保留存储运行器的完整要求。第1项已按[受理报告](reports/phase-4-admission.md)验证；第2–4项结果消费和根接续已接入[实际运行器](12-workflow-runtime.md)，第5项的TIMER/POLL已接入[时钟运行器](13-workflow-clock.md)，第5项的信号等待已接入[信号运行器](14-workflow-signals.md)，完整控制、接口和SDK仍继续实现： 1. 启动时认证、固定不可变版本/输入和策略、校验全部队列、计算实例截止，并在同事务创建实例、门控、首批激活/任务及审计回执。相同身份返回原实例。 2. 消费者领取内部事件后核对命名空间、来源任务版本、激活/轮次及当前步骤版本；消费确认、步骤变化、后续任务或持久化待续记录同事务提交。 diff --git a/docs/11-workflow-admission.md b/docs/11-workflow-admission.md index 4382dab..ab3d111 100644 --- a/docs/11-workflow-admission.md +++ b/docs/11-workflow-admission.md @@ -1,6 +1,6 @@ # Weave 实例受理与首批工作 -状态：内部Go服务与MySQL存储适配已实现实例Start/Get/Steps、固定版本、首批决策落库和可靠结果交付。[后台Inbox消费和根待续扫描](12-workflow-runtime.md)已接通。**尚未开放实例HTTP路由、启动SDK或命令行入口；信号运行器和完整控制仍继续开发。** Weave相关公开SDK目前仅支持定义发布。[TIMER和后续POLL轮次](13-workflow-clock.md)已接入，TASK及组合节点已可自动推进，不能将此解释成全部等待/控制能力已实现。 +状态：内部Go服务与MySQL存储适配已实现实例Start/Get/Steps、固定版本、首批决策落库和可靠结果交付。[后台Inbox消费和根待续扫描](12-workflow-runtime.md)已接通。**尚未开放实例HTTP路由、启动SDK或命令行入口；完整控制仍继续开发；[信号运行器](14-workflow-signals.md)已接入。** Weave相关公开SDK目前仅支持定义发布。[TIMER和后续POLL轮次](13-workflow-clock.md)已接入，TASK及组合节点已可自动推进，不能将此解释成全部等待/控制能力已实现。 ## 1. 受理事务 @@ -49,9 +49,9 @@ Get返回当前实例视图；Start重报返回受理时冻结的原始视图， | 活跃实例 | 每个namespace最多10000个RUNNING/WAITING/CANCELLING/NEEDS_REVIEW实例；终态不计入；满额仍可恢复原受理回执 | | 信号关联 | 新受理最多512字节；数据库兼容旧191字符最长764字节，升级不截断历史 | | 截止 | 数据库时间生成；任务、查询与等待均不超过实例总截止 | -| 自动运行 | Inbox消费、根待续、TIMER到期及后续POLL轮次已接入；信号消费及完整控制仍待实现 | +| 自动运行 | Inbox消费、根待续、TIMER到期及后续POLL轮次已接入；SIGNAL与最终查询竞争已接入，完整控制仍待实现 | -迁移000018为Wait增加signal_name和payload_version两个带中文注释的可空字段；NULL仅表示TIMER或尚未核实的历史元数据，未来消费者不得把历史NULL自动解释为有效的新信号等待。correlation_key转为精确字节存储；实例business_key改为utf8mb4_0900_bin。35个实体、含公共Record的295个直接持久化字段声明，全部有中文注释，继续保持零外键。 +迁移000018为Wait增加signal_name和payload_version两个带中文注释的可空字段；NULL仅表示TIMER或尚未核实的历史元数据，未来消费者不得把历史NULL自动解释为有效的新信号等待。correlation_key转为精确字节存储；实例business_key改为utf8mb4_0900_bin。该迁移时为35个实体、含公共Record的295个直接持久化字段声明，全部有中文注释，继续保持零外键。 ## 5. 下一步接入 diff --git a/docs/12-workflow-runtime.md b/docs/12-workflow-runtime.md index 736db3b..5754eb9 100644 --- a/docs/12-workflow-runtime.md +++ b/docs/12-workflow-runtime.md @@ -1,6 +1,6 @@ # Weave 结果消费与自动推进 -状态：内部实例的TASK结果消费、顺序、分支、有限并行汇合和根检查点接续已接入Orbit后台运行器。[定时等待和后续查询轮次](13-workflow-clock.md)也已接入。实例公共HTTP/SDK仍未开放，信号、失败后的分支取消结清及人工恢复继续开发，不能将本阶段等同于完整Weave交付。 +状态：内部实例的TASK结果消费、顺序、分支、有限并行汇合和根检查点接续已接入Orbit后台运行器。[定时等待和后续查询轮次](13-workflow-clock.md)也已接入。[信号等待与最终查询竞争](14-workflow-signals.md)已接入。实例公共HTTP/SDK仍未开放，失败后的分支取消结清及人工恢复继续开发，不能将本阶段等同于完整Weave交付。 ## 1. 后台运行和事务边界 @@ -45,7 +45,7 @@ flowchart LR 暂停、关闭门控或停用应用/命名空间时，继续收集已经落库的结果，但不激活新步骤；若还有后继，将其保存为待续。重新允许推进后，从持久记录继续。测试通过直接设置控制状态验证该行为；暂停/继续公共接口尚未交付。 -已过实例总截止时不创建后继，未能全部完成则进入核查。最后的有效任务结果在截止前已由Pulse接受，而消费发生较晚时，可以整理完整结果并完成实例。暂停或重启不延长截止。TIMER与POLL的时钟已接入；SIGNAL及其超期裁决仍需后续实现。 +已过实例总截止时不创建后继，未能全部完成则进入核查。最后的有效任务结果在截止前已由Pulse接受，而消费发生较晚时，可以整理完整结果并完成实例。暂停或重启不延长截止。TIMER与POLL的时钟已接入；[SIGNAL及其超期/最终查询裁决](14-workflow-signals.md)已接入。 ## 4. POLL与失败策略的当前边界 @@ -56,7 +56,7 @@ flowchart LR - 轮次耗尽、查询截止、未知/非法值或单轮请求失败：进入核查，不新建轮次绕过Pulse重试预算。 - 业务明确失败：按固定实例失败策略处理；REVIEW进入核查，FAIL_AFTER_SETTLED进入CANCELLING并关闭门控。 -下一轮查询派发及到期扫描已实现，详见[时钟、预算与恢复](13-workflow-clock.md)。**尚未实现FAIL_AFTER_SETTLED的剩余分支取消/收集/失败终态判定。** CANCELLING保留未结清事实，不提前宣称FAILED。信号抢先完成时与当前在途查询的结清竞争，也属于后续必须实现的功能。 +下一轮查询派发及到期扫描已实现，详见[时钟、预算与恢复](13-workflow-clock.md)。**尚未实现FAIL_AFTER_SETTLED的剩余分支取消/收集/失败终态判定。** CANCELLING保留未结清事实，不提前宣称FAILED。信号抢先到达时保留当前查询，实际结果结清及竞争见[信号运行约定](14-workflow-signals.md)。 ## 5. 回归与后续工作 @@ -64,4 +64,4 @@ flowchart LR 进程回归由`make integration`构建二进制后显式执行。缺少一次性数据库或`ORBIT_TEST_SERVER_BINARY`时测试会跳过；跳过不构成进程恢复证据。验证记录见[本阶段报告](reports/phase-4-runtime.md)。 -下一步仍包括持久化信号等待及查询最终信号竞争、完整控制与人工恢复、实例/来源公共API、独立SDK和通用端到端场景。P4整体及P5–P7继续按[完整计划](08-development-plan.md)执行。 +持久化信号等待及查询最终信号竞争已按[信号运行约定](14-workflow-signals.md)实现。下一步仍包括完整控制与人工恢复、实例/来源公共API、独立SDK和通用端到端场景。P4整体及P5–P7继续按[完整计划](08-development-plan.md)执行。 diff --git a/docs/13-workflow-clock.md b/docs/13-workflow-clock.md index 5ba0434..56025d7 100644 --- a/docs/13-workflow-clock.md +++ b/docs/13-workflow-clock.md @@ -1,10 +1,10 @@ # Weave 持久化定时等待与查询轮次 -状态：TIMER到期推进和POLL后续业务轮次已接入Orbit后台运行器。复用MySQL和既有待续索引，不增加外部调度服务或数据库外键。实例公共HTTP/SDK、SIGNAL消费、查询最终信号竞争以及完整控制/人工恢复仍未交付。 +状态：TIMER到期推进和POLL后续业务轮次已接入Orbit后台运行器。复用MySQL和既有待续索引，不增加外部调度服务或数据库外键。[SIGNAL消费与查询最终信号竞争](14-workflow-signals.md)已进一步接入；实例公共HTTP/SDK以及完整控制/人工恢复仍未交付。 ## 1. 时钟和恢复身份 -`weave_pending_advances`同时保存ROOT、TIMER、POLL三类检查点。唯一范围为命名空间、实例和步骤；时钟检查点额外固定当前激活ID，避免旧激活唤醒新工作。扫描只提供候选，实际推进仍须持锁核验固定版本、父记录、步骤/激活状态、任务所有者、门控和原始预算。 +`weave_pending_advances`同时保存ROOT、TIMER、SIGNAL、POLL四类检查点。唯一范围为命名空间、实例和步骤；时钟检查点额外固定当前激活ID，避免旧激活唤醒新工作。扫描只提供候选，实际推进仍须持锁核验固定版本、父记录、步骤/激活状态、任务所有者、门控和原始预算。 | 检查点 | 到期时间来源 | 推进结果 | | --- | --- | --- | @@ -50,7 +50,7 @@ TIMER由待续表直接唤醒，不创建Pulse业务任务，也不占Worker执 当前任务仍在途且查询总截止已到时，实例转NEEDS_REVIEW并关闭新执行门控；步骤保持ACTIVE、轮次保持当前任务引用，继续收集Pulse结果。时钟不释放执行许可，也不假定外部请求已经停止。Pulse恢复将失联执行记录为UNKNOWN后，内部消费再结清当前查询记录，保留核查状态。 -最终信号与在途查询的竞争尚未实现。定义可以描述`final_signal`并建立等待，但当前只能通过查询结果结清；不能据此承诺外部信号已经可以消费。 +最终信号与在途查询的竞争已按[信号运行约定](14-workflow-signals.md)接入：保存赢家后等待Pulse查询结清，不能提前释放许可或覆盖未知效果。实例及信号公共HTTP/SDK尚未开放。 ## 5. 升级和验证 @@ -60,4 +60,4 @@ TIMER由待续表直接唤醒，不创建Pulse业务任务，也不占Worker执 [存储回归](../internal/storage/mysql/workflow_clock_integration_test.go)覆盖到期、竞争、预算、暂停/满额、回滚、归属损坏与未知结果；[v18升级回归](../internal/storage/mysql/workflow_clock_upgrade_integration_test.go)验证回填及原预算保留；[真实进程回归](../internal/storage/mysql/workflow_runtime_process_integration_test.go)覆盖全部控制进程退出后的定时和查询接续。执行记录见[阶段报告](reports/phase-4-clock.md)。 -下一阶段继续SIGNAL早到/重复/超期处理、信号与查询竞争、失败分支结清、公共控制及实例SDK，完整范围见[持续计划](08-development-plan.md)。 +SIGNAL早到/重复/超期处理及最终查询竞争已接入；下一阶段继续失败分支结清、公共控制及实例SDK，完整范围见[持续计划](08-development-plan.md)。 diff --git a/docs/14-workflow-signals.md b/docs/14-workflow-signals.md new file mode 100644 index 0000000..4e5bf51 --- /dev/null +++ b/docs/14-workflow-signals.md @@ -0,0 +1,68 @@ +# Weave 外部信号与最终查询结果 + +状态：内部 Go 信号受理/查询、SIGNAL 等待、早到保存、截止处理及 POLL 最终信号竞争已接入 MySQL 和后台运行器。**实例与信号的公共 HTTP/独立 SDK 尚未开放**；完整控制、人工恢复和失败分支结清继续按[持续计划](08-development-plan.md)实现。定义发布 SDK 不等于实例运行 SDK。 + +## 1. 受理身份与权限 + +[内部契约](../weave/signal/service.go)为 `Send(token, workflowID, idempotencyKey, input)` 与 `Get(token, workflowID, signalID)`。输入绑定固定定义中的信号名称、内容版本、步骤键、显式目标激活序号和精确操作关联，附内容、受理原因与可选证据引用。 + +| 输入 | 限制及语义 | +| --- | --- | +| name / payload_version | 定义中冻结的名称/内容契约；ASCII，最多128/32字符 | +| target_step_key | 固定定义的目标步骤，最多64字符，必须为SIGNAL或带最终信号的POLL | +| target_activation_sequence | 显式正数；首次和早到信号为1，旧信号不能跟随人工恢复后的新激活；尚不存在的未来代次拒绝 | +| correlation_key | 1至512字节UTF8，精确匹配，保留尾部空格；不由调用者随意省略 | +| payload | 固定类型的精确JSON，最多256KiB；显式null与缺失不同；POLL最终信号不能是PENDING | +| reason / evidence_ref | 必填原因最多1024字符；可选证据引用最多512字符、不含空白/控制符；平台不访问引用地址 | + +提交要求显式 `workflows:signal` 权限；查询要求 `workflows:read`。两者都核对当前凭据、命名空间、实例固定版本及全部相关队列权限。actor来自认证凭据，不能请求自报。现有bootstrap的定义发布授权不会自动授予信号操作权限。 + +提交按认证 → 应用额度 → 原幂等身份 → 排序队列 → 实例/门控/步骤持锁。首次受理将信号、检查点、事件、审计和冻结回执同事务写入。审计包含原请求摘要、受理时间和总截止；运行器会核对受理证明，参数被替换时转核查，数据库连接/SQL错误则保留原来源并退避重试。 + +同一实例最多保存1024条信号，包含拒绝及终结历史；原幂等身份恢复先于上限检查。同键同内容返回首次冻结回执，同键不同内容冲突。重报仍校验当前权限。受理回执中的PENDING不跟随消费变化，当前状态用Get读取。 + +## 2. 早到、迟到与消费 + +目标步骤尚未激活时，信号保持PENDING；步骤激活会创建立即到期的检查点扫描早到信号。每个等待固定名称、版本、操作关联、激活序号和原截止，消费选择按受理时间、信号ID排序的第一条匹配记录。 + +- 信号必须在等待截止之前受理，等于截止不算按时。按时受理但控制进程稍后恢复，可以继续收集该事实。 +- 暂停或应用停用仍可收集结果；后继工作受原门控、额度及实例总截止约束。 +- 已激活时的关联不符、旧代次、已结清步骤或实例会保存明确拒绝原因；内容类型/定义契约无效的请求直接拒绝，不产生有效信号。 +- 同一等待只记录一个winner_signal_id；其余同代次信号保存WAIT_SETTLED或QUERY_SETTLED等原因。所有关联由事务核验，不使用外键。 +- SIGNAL在没有按时匹配信号的情况下到期，将等待标为EXPIRED、步骤和实例转NEEDS_REVIEW，并关闭新执行门控。 + +信号的expires_at固定为首次受理时的实例总截止。尚未匹配的早到信号通过单独到期索引结清；available_at仅表示下次检查时间，故障退避不能改变expires_at、accepted_at或等待截止。若实例已过总截止且仍需新工作，保留已收到事实并进入核查。 + +消费确认、等待赢家、步骤结果、后继任务或根检查点及审计同事务。审计失败不会留下“信号已消费但后继未保存”的部分状态。 + +## 3. 最终信号与POLL查询竞争 + +查询请求由Pulse执行，最终信号和查询结果由同一个实例事务裁决。PENDING只是一次成功查询观察，不争夺最终结果；请求失败也不允许创建新业务轮次绕过Pulse尝试预算。 + +| 条件 | 处理 | +| --- | --- | +| 最终查询结果早于信号受理，且在原查询截止内 | 查询结果胜出，信号保存QUERY_SETTLED；精度相同的时间优先查询 | +| 已有最终信号，当前无在途查询 | 结清POLL，不再创建下一轮查询 | +| 最终信号先到，查询尚未开始 | 保存信号赢家，使用Pulse取消规则结清未开始任务，再收集其Inbox结果 | +| 最终信号先到，查询正在执行 | 保存赢家和取消意图；保留当前任务、ACTIVE步骤及执行许可，等待Pulse实际结果 | +| 在途查询随后返回PENDING、相同最终结果或已知失败/取消 | 收集该查询结清后采用信号结果，不新增查询轮次 | +| 在途查询效果未知、结果不合法或返回不同最终结果 | 保留信号和查询证据，转NEEDS_REVIEW；不能让信号覆盖未知事实 | +| 原查询总截止已到，查询仍在途 | 关闭新执行门控，保留当前任务等待结清；后续结果不能自动清除已建立的核查状态 | + +`CONSUMED`表示信号已被等待采用，不表示整个实例成功。带在途查询时，即使等待记录为SATISFIED，步骤仍保持ACTIVE直到Pulse确认结束。取消复用Pulse既有许可释放逻辑，不在Weave中实现另一套释放计数。 + +控制时钟发现查询结果已落库但未消费时，读取原Inbox事件参与裁决；不由扫描器先后决定赢家，也不绕过Inbox故障退避。结果、信号和许可预算均在MySQL持久保存，进程重启不重新选择已保存的赢家。 + +## 4. 存储升级和运行边界 + +[迁移000020](../internal/storage/mysql/migrations/000020_workflow_signal_identity.up.sql)增加信号契约/代次/证据/处置/到期检查时间，以及等待的赢家引用。名称扩至128字符，关联保持精确字节，并兼容历史764字节值。35个实体、301个直接持久化字段声明均提供中文说明；数据库无外键、无级联删除。 + +历史信号没有足够身份和受理证明：旧PENDING标记REJECTED/LEGACY_UNVERIFIED，保留原载荷和时间；已消费历史不被重新解释。旧NULL等待元数据不能推测成有效新契约。迁移为当前SIGNAL等待回填原截止的检查点，保留已有检查点，排除终态实例及旧激活。 + +升级仍采用停止旧写入节点 → 应用全部迁移 → 启动新版；新版信号字段和运行语义不支持与旧节点混跑。后台扫描三类候选：结果Inbox、根/等待检查点和未匹配信号到期；每类最多100项，最终批次最多100项，每个候选短事务处理。 + +## 5. 验证与后续 + +[受理回归](../internal/storage/mysql/workflow_signals_integration_test.go)、[消费回归](../internal/storage/mysql/workflow_signal_runtime_integration_test.go)、[真实v19升级](../internal/storage/mysql/workflow_signal_upgrade_integration_test.go)和[实际进程恢复](../internal/storage/mysql/workflow_runtime_process_integration_test.go)对应受理/并发/故障/升级边界，实际运行结果记录在[阶段报告](reports/phase-4-signals.md)。 + +下一步完成失败分支取消结清、实例暂停/继续/取消/人工核查恢复，再交付实例与信号HTTP、独立SDK及完整场景。当前内部存储测试不替代尚未开放的公共接口验收，也不替代生产MySQL切换与容量测试。 diff --git a/docs/reports/phase-4-signals.md b/docs/reports/phase-4-signals.md new file mode 100644 index 0000000..2224d36 --- /dev/null +++ b/docs/reports/phase-4-signals.md @@ -0,0 +1,49 @@ +# P4/P5 外部信号与最终查询竞争回归 + +日期：2026-09-18。实现和验收范围为内部信号服务与持久化运行器。实例/信号公共HTTP和独立SDK、完整控制与人工恢复仍待交付，持续目标未结束。 + +## 变化与边界 + +- 固定目标步骤、激活代次、信号名称/契约、精确操作关联、原因及证据，显式信号授权；每实例历史总上限1024。 +- 原键冻结回执，参数冲突拒绝；受理证明与不可变审计核对，不接受修改后的信号参数授权后继。 +- SIGNAL早到、等待、按时受理延迟消费、重复与截止处理；暂停收集并持久保存根接续。 +- POLL最终信号与查询按原事实时间竞争，PENDING不作为最终结果；在途查询先请求取消并保留许可，未知或冲突转核查。 +- Pulse取消逻辑提取为公共内部事务函数，原公共取消权限、幂等和审计边界保留；信号调用前验证流程所有权。 +- 迁移000020增加5个Signal字段和1个Wait字段。当前35个实体、301个直接持久化字段声明，中文注释、零外键及禁止级联约束继续执行。 + +详细规则及升级顺序见[信号文档](../14-workflow-signals.md)。此处CONSUMED表示等待采用了信号，不代表流程成功或外部查询已结束。 + +## 验证记录 + +环境：darwin/arm64、Go 1.27.1、Docker MySQL 8.4.10；临时隔离数据库，未访问业务数据库或生产数据库。 + +| 检查 | 结果 | +| --- | --- | +| 基础格式、单元、竞态、静态检查、构建 | `make fmt test vet build`通过，退出0，包含服务端、独立SDK和业务示例module | +| 信号受理与运行器专项 | 受理4组、运行器20组，连同TIMER/POLL、原流程运行器、Pulse取消、新建结构和v7/v9升级，`-race`通过，51.926秒 | +| 真实v19升级 | 全量存储阶段已通过，3.90秒；原764字节关联、精确内容、已消费证据保留，旧PENDING隔离，当前SIGNAL在原截止回填 | +| 全量隔离MySQL与真实进程/SDK/示例 | 已通过，脚本退出0；全存储172.485秒、流程进程20.127秒、Worker SDK 94.716秒、SDK进程故障93.118秒；19个SDK示例和2个独立业务示例全部通过 | + +专项入口： + +```sh +go test -race -count=1 -run '^TestIntegration(WorkflowSignalAdmission|WorkflowSignalRuntime|WorkflowSignalUpgrade|WorkflowClock|WorkflowRuntime|WorkerCancellation|SchemaAndConstraints|NoForeignKeysUpgrade|ApplicationResourcesUpgrade)$' -v ./internal/storage/mysql +go build -o /tmp/orbit-signal-test-server ./cmd/orbit +ORBIT_TEST_SERVER_BINARY=/tmp/orbit-signal-test-server go test -race -count=1 -run '^TestIntegrationWorkflowRuntimeProcess$' -v ./internal/storage/mysql +``` + +必须设置`ORBIT_TEST_MYSQL_DSN`为一次性数据库。未配置环境变量导致跳过时不能记作通过。[全量脚本](../../scripts/test-mysql.sh)创建隔离MySQL并显式构建二进制、再运行实际进程测试。 + +存储专项覆盖12个并发受理只生成一条信号、12个并发消费者只创建一个后继、早到保存、暂停后接续、按时受理晚消费、真实等待超时、审计故障回滚、原回执冻结、载荷/actor/受理时间篡改进入核查、过期退避不改截止，以及最终信号在轮次间/未开始/执行中到达。执行中的查询分别返回取消、PENDING、相同最终值、未知、已知请求失败及冲突最终值；核查和许可释放按实际结果核验。查询较早完成时，先运行时钟或先运行Inbox都保留查询胜出。 + +真实进程用例进一步扩展了早到信号在全停期间保存、两个新控制进程激活消费，以及最终信号选中后再强杀全部进程、查询结清入库后由两个新进程恢复赢家的场景。该用例已由实际Orbit二进制执行通过，20.127秒；不是仅重建Go对象或内存快照的模拟。 + +数据库直接核验外键0、缺少COMMENT的业务字段0；代码301个直接持久化字段声明均有中文注释。文档检查870个本地链接、53个JSON代码块，错误0，业务项目标识检查通过；`git diff --check`通过。v7升级中70个历史外键全部移除，323项原索引组件保留；v20的两个信号索引和赢家唯一索引另行核验定义。 + +## 仍须完成 + +1. FAIL_AFTER_SETTLED分支取消与完整结清、实例暂停/继续/取消/人工核查恢复。 +2. 实例/步骤/来源/信号/审计HTTP、独立SDK和可运行的端到端通用示例。 +3. P6–P7的监控、完整故障矩阵、性能容量、MySQL切换/备份恢复和SDK发布核对。 + +这些测试不等同于生产高可用、容量或RPO/RTO验收；运行器采用单MySQL持久事务，生产边界继续单独验证。 diff --git a/internal/storage/mysql/application_resources_upgrade_integration_test.go b/internal/storage/mysql/application_resources_upgrade_integration_test.go index ae07a42..bcd82a3 100644 --- a/internal/storage/mysql/application_resources_upgrade_integration_test.go +++ b/internal/storage/mysql/application_resources_upgrade_integration_test.go @@ -64,6 +64,8 @@ func TestIntegrationApplicationResourcesUpgrade(t *testing.T) { retryFinish(t, g, finished, "SUCCEEDED", false) assertApplicationCounts(t, f, 4, 2, 1) // 还原后续新增表和字段，得到真实 v9 结构；现有任务内容保持不变。 + f.exec(`ALTER TABLE weave_signals DROP CHECK ck_signal_version_sequence, DROP CHECK ck_signal_pending_identity, DROP INDEX ix_signal_expiry, DROP INDEX ix_signal_activation, DROP COLUMN payload_version, DROP COLUMN target_activation_sequence, DROP COLUMN evidence_ref, DROP COLUMN disposition, DROP COLUMN available_at, MODIFY COLUMN type VARCHAR(64) NOT NULL COMMENT '事件或信号的稳定类型标识', MODIFY COLUMN correlation_key VARCHAR(191) NOT NULL COMMENT '外部业务操作关联键，防止旧信号误用于新工作'`) + f.exec(`ALTER TABLE weave_waits DROP INDEX uq_wait_winner_signal, DROP COLUMN winner_signal_id`) f.exec(`ALTER TABLE weave_waits DROP COLUMN signal_name, DROP COLUMN payload_version, MODIFY COLUMN correlation_key VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs NOT NULL COMMENT '外部业务操作关联键，防止旧信号误用于新工作'`) f.exec(`ALTER TABLE weave_instances MODIFY COLUMN business_key VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs NULL COMMENT '业务侧关联键，用于检索追踪，不自动提供业务幂等'`) f.exec(`ALTER TABLE weave_inbox DROP CHECK ck_weave_inbox_source, DROP INDEX uq_weave_task_result, DROP COLUMN source_task_id, DROP COLUMN source_activation_id, DROP COLUMN source_revision, DROP COLUMN snapshot`) diff --git a/internal/storage/mysql/execution_cancel.go b/internal/storage/mysql/execution_cancel.go new file mode 100644 index 0000000..9ff6bde --- /dev/null +++ b/internal/storage/mysql/execution_cancel.go @@ -0,0 +1,82 @@ +package mysql + +import ( + "context" + "database/sql" + "errors" + "time" + + core "git.yuntuo.hk/k2safe/orbit/model" + pulse "git.yuntuo.hk/k2safe/orbit/pulse/model" + "git.yuntuo.hk/k2safe/orbit/pulse/service" +) + +// cancelExecutionTask 由已验证所有权的调用方复用；调用前必须持有应用、队列和任务锁。 +// 仅已知未开始的任务立即结清，在途执行保留许可并等待结果，历史未知效果不能变成已取消。 +func cancelExecutionTask(ctx context.Context, tx *sql.Tx, ns core.ID, task executionTask, now time.Time, reason string) (service.TaskView, error) { + var out service.TaskView + view := task.view + if view.State.Terminal() || view.CancelRequestedAt != nil { + return view, nil + } + id := view.ID + var err error + var attempt *pulse.Attempt + if view.State == pulse.TaskLeased || view.State == pulse.TaskRunning { + if task.current == nil { + return out, errors.New("active task lacks attempt") + } + a, err := readAttempt(ctx, tx, ns, *task.current) + if err != nil { + return out, err + } + if a.TaskID != id || a.Epoch != task.epoch || string(a.State) != string(view.State) { + return out, errors.New("task attempt identity or state mismatch") + } + attempt = &a + } + next := view.State + lease := task.lease + var completed *time.Time + if view.State == pulse.TaskWaiting || view.State == pulse.TaskLeased { + next = pulse.TaskCancelled + completed = &now + lease = nil + if view.EffectUncertain { + next, completed = pulse.TaskNeedsReview, nil + } + if attempt != nil { + if attempt.AuthorizedAt != nil { + return out, errors.New("leased attempt already authorized") + } + result, err := tx.ExecContext(ctx, `UPDATE pulse_attempts SET state='CANCELLED',ended_at=?,error_code='CANCELLED_BEFORE_START',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, now, ns, attempt.ID, attempt.Revision) + if err := exactlyOne(result, err); err != nil { + return out, err + } + if err := releasePermits(ctx, tx, ns, attempt.ID, now); err != nil { + return out, err + } + } + } + result, err := tx.ExecContext(ctx, `UPDATE pulse_tasks SET state=?,cancel_requested_at=?,lease_until=?,completed_at=?,updated_at=?,revision=revision+1 WHERE namespace_id=? AND id=? AND revision=?`, next, now, lease, completed, now, ns, id, view.Revision) + if err := exactlyOne(result, err); err != nil { + return out, err + } + if next.Terminal() { + if err := releaseQueueTask(ctx, tx, ns, view.QueueID); err != nil { + return out, err + } + } + out, err = readTask(ctx, tx, ns, id, false) + if err != nil { + return out, err + } + event := resultEvent(next) + if next == view.State { + event = "task.cancel_requested" + } + if err := emitEvent(ctx, tx, ns, id, out.Revision, "task", event, map[string]any{"task_id": id, "state": next, "reason": reason, "effect_uncertain": view.EffectUncertain}); err != nil { + return out, err + } + return out, nil +} diff --git a/internal/storage/mysql/migrations/000020_workflow_signal_identity.up.sql b/internal/storage/mysql/migrations/000020_workflow_signal_identity.up.sql new file mode 100644 index 0000000..3cd3a7a --- /dev/null +++ b/internal/storage/mysql/migrations/000020_workflow_signal_identity.up.sql @@ -0,0 +1,33 @@ +-- 外部信号固定参数版本与目标激活代次；旧记录无此身份，不能推测为新等待的有效输入。 +ALTER TABLE weave_signals + MODIFY COLUMN `type` VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_bin NOT NULL COMMENT '固定定义中的信号名称，新受理限制为128个ASCII字符', + MODIFY COLUMN correlation_key VARBINARY(764) NOT NULL COMMENT '精确UTF8操作关联，保留尾部空格；新信号最多512字节，历史兼容764字节', + ADD COLUMN payload_version VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '信号内容契约版本；为空仅表示未核实的历史记录', + ADD COLUMN target_activation_sequence INT UNSIGNED NULL COMMENT '固定目标步骤激活序号，早到首轮为1；为空仅表示未核实历史', + ADD COLUMN evidence_ref VARCHAR(512) NULL COMMENT '可选的受控证据引用；不保存秘密，也不由平台请求该地址', + ADD COLUMN disposition VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '稳定消费或拒绝原因；PENDING时为空，不包含业务参数', + ADD COLUMN available_at DATETIME(6) NULL COMMENT '未匹配信号下次到期检查时间；故障退避可延后检查，但不改变原截止', + ADD KEY ix_signal_expiry (state,available_at,namespace_id,id), + ADD KEY ix_signal_activation (namespace_id,workflow_id,target_step_key,target_activation_sequence,state,accepted_at,id); + +UPDATE weave_signals SET state='REJECTED',disposition='LEGACY_UNVERIFIED',revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE state='PENDING'; +UPDATE weave_signals SET available_at=expires_at; + +ALTER TABLE weave_signals + MODIFY COLUMN available_at DATETIME(6) NOT NULL COMMENT '未匹配信号下次到期检查时间；故障退避可延后检查，但不改变原截止', + ADD CONSTRAINT ck_signal_version_sequence CHECK ((payload_version IS NULL AND target_activation_sequence IS NULL) OR (payload_version IS NOT NULL AND payload_version<>'' AND target_activation_sequence IS NOT NULL AND target_activation_sequence>0)), + ADD CONSTRAINT ck_signal_pending_identity CHECK (state<>'PENDING' OR (payload_version IS NOT NULL AND target_activation_sequence IS NOT NULL AND disposition IS NULL)); + +ALTER TABLE weave_waits + ADD COLUMN winner_signal_id BINARY(16) NULL COMMENT '胜出的外部信号标识；仅时间或查询结果结清时为空，所有者由同事务校验', + ADD UNIQUE KEY uq_wait_winner_signal (namespace_id,winner_signal_id); + +-- SIGNAL到期扫描与TIMER/POLL共用待续表；旧元数据缺失的等待交给运行器核查，不凭空授权。 +INSERT INTO weave_pending_advances(namespace_id,id,workflow_id,step_id,expected_step_revision,checkpoint,available_at) + SELECT s.namespace_id,RANDOM_BYTES(16),s.workflow_id,s.id,s.revision, + JSON_OBJECT('schema_version',1,'kind','SIGNAL','activation_id',LOWER(HEX(a.id))),w.deadline_at + FROM weave_steps s JOIN weave_activations a ON a.namespace_id=s.namespace_id AND a.id=s.current_activation_id AND a.workflow_id=s.workflow_id AND a.step_id=s.id + JOIN weave_instances i ON i.namespace_id=s.namespace_id AND i.id=s.workflow_id + JOIN weave_waits w ON w.namespace_id=s.namespace_id AND w.workflow_id=s.workflow_id AND w.activation_id=a.id + WHERE s.kind='SIGNAL' AND s.state='WAITING' AND a.state='WAITING' AND w.kind='SIGNAL' AND w.state='WAITING' AND i.state IN ('RUNNING','WAITING','CANCELLING','NEEDS_REVIEW') + ON DUPLICATE KEY UPDATE id=weave_pending_advances.id; diff --git a/internal/storage/mysql/no_foreign_keys_integration_test.go b/internal/storage/mysql/no_foreign_keys_integration_test.go index 8e893ea..b4919e3 100644 --- a/internal/storage/mysql/no_foreign_keys_integration_test.go +++ b/internal/storage/mysql/no_foreign_keys_integration_test.go @@ -88,8 +88,8 @@ func TestIntegrationNoForeignKeysUpgrade(t *testing.T) { } indexes := func() []string { t.Helper() - // v15替换retry_of索引、v17增加内部结果唯一索引，下方单独验证定义。 - rows, err := db.QueryContext(ctx, `SELECT CONCAT_WS(':',TABLE_NAME,INDEX_NAME,NON_UNIQUE,SEQ_IN_INDEX,COLUMN_NAME,COALESCE(SUB_PART,0),INDEX_TYPE,IS_VISIBLE) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME NOT IN ('orbit_application_resources','orbit_event_streams','orbit_result_events') AND INDEX_NAME NOT IN ('ix_task_priority_dispatch','ix_delivery_subscription_due','ix_delivery_subscription_lease','ix_delivery_history','ix_delivery_state_history','uq_task_retry_parent','fk_pulse_tasks_3','ix_task_queue_business_created','ix_task_queue_business_state_created','uq_weave_task_result') ORDER BY TABLE_NAME,INDEX_NAME,SEQ_IN_INDEX`) + // v15替换retry_of索引、v17增加内部结果唯一索引、v20增加信号索引，下方单独验证定义。 + rows, err := db.QueryContext(ctx, `SELECT CONCAT_WS(':',TABLE_NAME,INDEX_NAME,NON_UNIQUE,SEQ_IN_INDEX,COLUMN_NAME,COALESCE(SUB_PART,0),INDEX_TYPE,IS_VISIBLE) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME NOT IN ('orbit_application_resources','orbit_event_streams','orbit_result_events') AND INDEX_NAME NOT IN ('ix_task_priority_dispatch','ix_delivery_subscription_due','ix_delivery_subscription_lease','ix_delivery_history','ix_delivery_state_history','uq_task_retry_parent','fk_pulse_tasks_3','ix_task_queue_business_created','ix_task_queue_business_state_created','uq_weave_task_result','ix_signal_expiry','ix_signal_activation','uq_wait_winner_signal') ORDER BY TABLE_NAME,INDEX_NAME,SEQ_IN_INDEX`) if err != nil { t.Fatal(err) } @@ -178,6 +178,16 @@ func TestIntegrationNoForeignKeysUpgrade(t *testing.T) { t.Fatalf("workflow result uniqueness=%q err=%v", resultIndex, err) } var collation string + for _, index := range []struct{ table, name, want string }{ + {"weave_signals", "ix_signal_expiry", "state:1,available_at:1,namespace_id:1,id:1"}, + {"weave_signals", "ix_signal_activation", "namespace_id:1,workflow_id:1,target_step_key:1,target_activation_sequence:1,state:1,accepted_at:1,id:1"}, + {"weave_waits", "uq_wait_winner_signal", "namespace_id:0,winner_signal_id:0"}, + } { + var got string + if err := db.QueryRowContext(ctx, `SELECT GROUP_CONCAT(CONCAT(COLUMN_NAME,':',NON_UNIQUE) ORDER BY SEQ_IN_INDEX) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME=? AND INDEX_NAME=?`, index.table, index.name).Scan(&got); err != nil || got != index.want { + t.Fatalf("signal index %s=%q err=%v", index.name, got, err) + } + } if err := db.QueryRowContext(ctx, `SELECT COLLATION_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='pulse_tasks' AND COLUMN_NAME='business_key'`).Scan(&collation); err != nil || collation != "utf8mb4_0900_bin" { t.Fatalf("business key collation=%q err=%v", collation, err) } diff --git a/internal/storage/mysql/task_cancel.go b/internal/storage/mysql/task_cancel.go index 20cdf38..9a8970e 100644 --- a/internal/storage/mysql/task_cancel.go +++ b/internal/storage/mysql/task_cancel.go @@ -4,14 +4,11 @@ import ( "context" "database/sql" "encoding/json" - "errors" "fmt" - "time" "git.yuntuo.hk/k2safe/orbit/internal/fault" core "git.yuntuo.hk/k2safe/orbit/model" "git.yuntuo.hk/k2safe/orbit/platform/auth" - pulse "git.yuntuo.hk/k2safe/orbit/pulse/model" "git.yuntuo.hk/k2safe/orbit/pulse/service" ) @@ -56,68 +53,15 @@ func (s *TaskStore) CancelTask(ctx context.Context, token auth.Token, id core.ID if owner.Valid && owner.String != "schedule" { return fault.Conflict } - var attempt *pulse.Attempt - if view.State == pulse.TaskLeased || view.State == pulse.TaskRunning { - if task.current == nil { - return errors.New("active task lacks attempt") - } - a, err := readAttempt(ctx, tx, token.NamespaceID, *task.current) - if err != nil { - return err - } - if a.TaskID != id || a.Epoch != task.epoch || string(a.State) != string(view.State) { - return errors.New("task attempt identity or state mismatch") - } - attempt = &a - } now, err := dbNow(ctx, tx) if err != nil { return err } - next := view.State - lease := task.lease - var completed *time.Time - if view.State == pulse.TaskWaiting || view.State == pulse.TaskLeased { - next = pulse.TaskCancelled - completed = &now - lease = nil - if view.EffectUncertain { - next, completed = pulse.TaskNeedsReview, nil - } - if attempt != nil { - if attempt.AuthorizedAt != nil { - return errors.New("leased attempt already authorized") - } - result, err := tx.ExecContext(ctx, `UPDATE pulse_attempts SET state='CANCELLED',ended_at=?,error_code='CANCELLED_BEFORE_START',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, now, token.NamespaceID, attempt.ID, attempt.Revision) - if err := exactlyOne(result, err); err != nil { - return err - } - if err := releasePermits(ctx, tx, token.NamespaceID, attempt.ID, now); err != nil { - return err - } - } - } - result, err := tx.ExecContext(ctx, `UPDATE pulse_tasks SET state=?,cancel_requested_at=?,lease_until=?,completed_at=?,updated_at=?,revision=revision+1 WHERE namespace_id=? AND id=? AND revision=?`, next, now, lease, completed, now, token.NamespaceID, id, in.ExpectedRevision) - if err := exactlyOne(result, err); err != nil { - return err - } - if next.Terminal() { - if err := releaseQueueTask(ctx, tx, token.NamespaceID, view.QueueID); err != nil { - return err - } - } - out.Task, err = readTask(ctx, tx, token.NamespaceID, id, false) + out.Task, err = cancelExecutionTask(ctx, tx, token.NamespaceID, task, now, "CANCEL_REQUESTED") if err != nil { return err } - event := resultEvent(next) - if next == view.State { - event = "task.cancel_requested" - } - if err := emitEvent(ctx, tx, token.NamespaceID, id, out.Task.Revision, "task", event, map[string]any{"task_id": id, "state": next, "reason": "CANCEL_REQUESTED", "effect_uncertain": view.EffectUncertain}); err != nil { - return err - } - if err := appendAudit(ctx, tx, token, identity, id, "task", "task.cancel", in.Reason, map[string]any{"state": view.State, "revision": fmt.Sprint(view.Revision)}, map[string]any{"state": next, "revision": fmt.Sprint(out.Task.Revision), "cancel_requested_at": now}); err != nil { + if err := appendAudit(ctx, tx, token, identity, id, "task", "task.cancel", in.Reason, map[string]any{"state": view.State, "revision": fmt.Sprint(view.Revision)}, map[string]any{"state": out.Task.State, "revision": fmt.Sprint(out.Task.Revision), "cancel_requested_at": now}); err != nil { return err } return saveIdentity(ctx, tx, token.NamespaceID, identity, out) diff --git a/internal/storage/mysql/workflow_apply.go b/internal/storage/mysql/workflow_apply.go index d7d6314..0e79a4a 100644 --- a/internal/storage/mysql/workflow_apply.go +++ b/internal/storage/mysql/workflow_apply.go @@ -255,6 +255,11 @@ func activateWorkflowStep(ctx context.Context, tx *sql.Tx, g *workflowGraph, row if err := saveWorkflowClock(ctx, tx, g, row, "TIMER", row.view.Revision+1, until); err != nil { return err } + } else if c.Wait.Kind == "SIGNAL" { + // 先扫描早到信号；未匹配时运行器按原等待截止继续持久化。 + if err := saveWorkflowClock(ctx, tx, g, row, "SIGNAL", row.view.Revision+1, now); err != nil { + return err + } } } if c.Task != nil { @@ -274,7 +279,11 @@ func activateWorkflowStep(ctx context.Context, tx *sql.Tx, g *workflowGraph, row } } if c.Kind == "POLL" { - return saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision+1, deadline) + due := deadline + if c.Wait != nil { + due = now + } + return saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision+1, due) } return nil } diff --git a/internal/storage/mysql/workflow_clock.go b/internal/storage/mysql/workflow_clock.go index 455cc85..9ed0399 100644 --- a/internal/storage/mysql/workflow_clock.go +++ b/internal/storage/mysql/workflow_clock.go @@ -14,7 +14,7 @@ import ( // workflowCheckpoint 是待续表中版本化的内部唤醒身份；不保存内存执行栈或新的业务预算。 type workflowCheckpoint struct { SchemaVersion uint32 `json:"schema_version"` // SchemaVersion 当前结构版本为1，未知版本禁止推进。 - Kind string `json:"kind"` // Kind 为ROOT、TIMER或POLL，决定实际权威记录的读取方式。 + Kind string `json:"kind"` // Kind 为ROOT、TIMER、SIGNAL或POLL，决定实际权威记录的读取方式。 ActivationID *core.ID `json:"activation_id,omitempty"` // ActivationID 固定等待激活；ROOT不使用该字段。 } @@ -23,7 +23,7 @@ func saveWorkflowClock(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *w if err != nil { return err } - if row.view.ActivationID == nil || kind != "TIMER" && kind != "POLL" { + if row.view.ActivationID == nil || kind != "TIMER" && kind != "POLL" && kind != "SIGNAL" { return errors.New("workflow clock identity invalid") } raw, err := json.Marshal(workflowCheckpoint{1, kind, row.view.ActivationID}) @@ -60,8 +60,24 @@ func (r *WorkflowRuntime) advanceClock(ctx context.Context, tx *sql.Tx, g *workf return false, err } if checkpoint.Kind == "POLL" { + node, _ := g.program.Step(row.view.Key) + if node.Poll.FinalSignal != nil { + handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now) + if err != nil { + return false, err + } + if handled { + if changed && reason != "" { + return true, r.audit(ctx, tx, g, c.id, reason, now) + } + return changed, nil + } + } return r.advancePoll(ctx, tx, g, row, c, activated, allow, now) } + if checkpoint.Kind == "SIGNAL" { + return r.advanceSignalClock(ctx, tx, g, row, c, allow, now) + } if checkpoint.Kind != "TIMER" || row.view.State != weave.StepWaiting { return false, errors.New("workflow timer state invalid") } diff --git a/internal/storage/mysql/workflow_consume.go b/internal/storage/mysql/workflow_consume.go index 5f05ff5..e9d366d 100644 --- a/internal/storage/mysql/workflow_consume.go +++ b/internal/storage/mysql/workflow_consume.go @@ -177,6 +177,15 @@ func (r *WorkflowRuntime) consume(ctx context.Context, tx *sql.Tx, g *workflowGr if resolved != nil || current == nil || *current != source.task { return true, r.confirmWorkflowResult(ctx, tx, g, c, "PROCESSED", "STALE_POLL_ROUND", now) } + if node.Poll.FinalSignal != nil { + handled, _, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, source, allow, now) + if err != nil { + return false, err + } + if handled { + return true, r.confirmWorkflowResult(ctx, tx, g, c, "PROCESSED", reason, now) + } + } // 单轮请求失败不能通过新建业务轮次绕过Pulse预算。 if state != weave.StepSucceeded { state, reason = weave.StepNeedsReview, "POLL_REQUEST_REVIEW" @@ -216,6 +225,15 @@ func (r *WorkflowRuntime) consume(ctx context.Context, tx *sql.Tx, g *workflowGr return false, err } if settled != nil { + if node.Poll.FinalSignal != nil { + w, err := readWorkflowSignalWait(ctx, tx, g, row) + if err != nil { + return false, err + } + if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "QUERY_SETTLED", now); err != nil { + return false, err + } + } waitState := "CANCELLED" if state == weave.StepSucceeded { waitState = "SATISFIED" diff --git a/internal/storage/mysql/workflow_poll_clock.go b/internal/storage/mysql/workflow_poll_clock.go index d3ef1c5..5d06fb8 100644 --- a/internal/storage/mysql/workflow_poll_clock.go +++ b/internal/storage/mysql/workflow_poll_clock.go @@ -16,31 +16,42 @@ import ( weave "git.yuntuo.hk/k2safe/orbit/weave/model" ) -func (r *WorkflowRuntime) advancePoll(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, activated time.Time, allow bool, now time.Time) (bool, error) { +// readWorkflowPollState 校验当前激活的完整轮次与原预算；时钟和最终信号共用相同检查。 +func readWorkflowPollState(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, activated time.Time) (weave.PollState, core.ID, error) { var p weave.PollState - err := tx.QueryRowContext(ctx, `SELECT id,revision,max_rounds,used_rounds,max_attempts_per_round,interval_ms,deadline_at,next_poll_at,current_task_id,resolved_at FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, c.ns, c.workflow, row.view.ActivationID).Scan(&p.ID, &p.Revision, &p.MaxRounds, &p.UsedRounds, &p.MaxAttemptsPerRound, &p.IntervalMS, &p.DeadlineAt, &p.NextPollAt, &p.CurrentTaskID, &p.ResolvedAt) + err := tx.QueryRowContext(ctx, `SELECT id,revision,max_rounds,used_rounds,max_attempts_per_round,interval_ms,deadline_at,next_poll_at,current_task_id,resolved_at FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&p.ID, &p.Revision, &p.MaxRounds, &p.UsedRounds, &p.MaxAttemptsPerRound, &p.IntervalMS, &p.DeadlineAt, &p.NextPollAt, &p.CurrentTaskID, &p.ResolvedAt) + if err != nil { + return p, "", err + } + node, _ := g.program.Step(row.view.Key) + policy := node.Poll + if policy == nil || p.MaxRounds != policy.MaxRounds || p.MaxAttemptsPerRound != policy.Task.Retry.MaxAttempts || p.IntervalMS != policy.IntervalMS || p.UsedRounds < 1 || !p.DeadlineAt.Equal(minimumTime(g.view.DeadlineAt, activated.Add(time.Duration(policy.TimeoutMS)*time.Millisecond))) { + return p, "", errors.New("workflow poll policy mismatch") + } + if p.ResolvedAt != nil { + return p, "", errors.New("unsettled poll step has resolved budget") + } + var count, minRound, maxRound uint32 + if err := tx.QueryRowContext(ctx, "SELECT COUNT(*),COALESCE(MIN(`round`),0),COALESCE(MAX(`round`),0) FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=?", g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&count, &minRound, &maxRound); err != nil { + return p, "", err + } + if count != p.UsedRounds || minRound != 1 || maxRound != p.UsedRounds { + return p, "", errors.New("workflow poll rounds incomplete") + } + var lastTask core.ID + if err := tx.QueryRowContext(ctx, "SELECT task_id FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=? AND `round`=?", g.view.NamespaceID, g.view.ID, row.view.ActivationID, p.UsedRounds).Scan(&lastTask); err != nil { + return p, "", err + } + return p, lastTask, nil +} + +func (r *WorkflowRuntime) advancePoll(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, activated time.Time, allow bool, now time.Time) (bool, error) { + p, lastTask, err := readWorkflowPollState(ctx, tx, g, row, activated) if err != nil { return false, err } node, _ := g.program.Step(row.view.Key) policy := node.Poll - if policy == nil || p.MaxRounds != policy.MaxRounds || p.MaxAttemptsPerRound != policy.Task.Retry.MaxAttempts || p.IntervalMS != policy.IntervalMS || p.UsedRounds < 1 || !p.DeadlineAt.Equal(minimumTime(g.view.DeadlineAt, activated.Add(time.Duration(policy.TimeoutMS)*time.Millisecond))) { - return false, errors.New("workflow poll policy mismatch") - } - if p.ResolvedAt != nil { - return false, errors.New("unsettled poll step has resolved budget") - } - var count, minRound, maxRound uint32 - if err := tx.QueryRowContext(ctx, "SELECT COUNT(*),COALESCE(MIN(`round`),0),COALESCE(MAX(`round`),0) FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=?", c.ns, c.workflow, row.view.ActivationID).Scan(&count, &minRound, &maxRound); err != nil { - return false, err - } - if count != p.UsedRounds || minRound != 1 || maxRound != p.UsedRounds { - return false, errors.New("workflow poll rounds incomplete") - } - var lastTask core.ID - if err := tx.QueryRowContext(ctx, "SELECT task_id FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=? AND `round`=?", c.ns, c.workflow, row.view.ActivationID, p.UsedRounds).Scan(&lastTask); err != nil { - return false, err - } if p.CurrentTaskID != nil { if *p.CurrentTaskID != lastTask || p.NextPollAt != nil || row.view.State != weave.StepActive { return false, errors.New("workflow in-flight poll mismatch") diff --git a/internal/storage/mysql/workflow_runtime.go b/internal/storage/mysql/workflow_runtime.go index f005ec2..3146577 100644 --- a/internal/storage/mysql/workflow_runtime.go +++ b/internal/storage/mysql/workflow_runtime.go @@ -29,8 +29,8 @@ func NewWorkflowRuntime(db *sql.DB, node core.ID) (*WorkflowRuntime, error) { type WorkflowRuntimeStats struct { Examined int // Examined 本批取出的持久化候选数，包含被其他节点先处理或跳过的记录。 - Consumed int // Consumed 已提交确认或隔离的内部结果数，事务回滚不计入。 - Advanced int // Advanced 已提交推进的根或等待检查点数，不等于业务任务执行次数。 + Consumed int // Consumed 以Inbox候选为入口提交确认或隔离的数量；时钟内消费计入Advanced，回滚不计入。 + Advanced int // Advanced 已提交推进的根、等待检查点或过期信号数，不等于业务任务执行次数。 } type workflowCandidate struct { @@ -39,7 +39,7 @@ type workflowCandidate struct { revision uint64 } -// Tick 同时读取两种到期索引，各取最多limit项再全局选前limit项；不先锁Inbox再反锁父资源。 +// Tick 同时读取三种到期索引，各取最多limit项再全局选前limit项；不先锁Inbox或信号再反锁父资源。 func (r *WorkflowRuntime) Tick(ctx context.Context, limit int) (WorkflowRuntimeStats, error) { var stats WorkflowRuntimeStats if limit < 1 || limit > 100 { @@ -47,7 +47,8 @@ func (r *WorkflowRuntime) Tick(ctx context.Context, limit int) (WorkflowRuntimeS } rows, err := r.db.QueryContext(ctx, `(SELECT 'INBOX' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_inbox WHERE state='PENDING' AND available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?) UNION ALL (SELECT 'ROOT' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_pending_advances WHERE available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?) - ORDER BY available_at,namespace_id,id LIMIT ?`, limit, limit, limit) + UNION ALL (SELECT 'SIGNAL' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_signals WHERE state='PENDING' AND available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?) + ORDER BY available_at,namespace_id,id LIMIT ?`, limit, limit, limit, limit) if err != nil { return stats, err } @@ -95,6 +96,8 @@ func (r *WorkflowRuntime) deferCandidate(ctx context.Context, c workflowCandidat q := `UPDATE weave_pending_advances SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=?` if c.kind == "INBOX" { q = `UPDATE weave_inbox SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),error_code='RUNTIME_UNAVAILABLE',revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=? AND state='PENDING'` + } else if c.kind == "SIGNAL" { + q = `UPDATE weave_signals SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=? AND state='PENDING'` } _, err := r.db.ExecContext(ctx, q, c.ns, c.id, c.revision) return err @@ -115,6 +118,10 @@ func (r *WorkflowRuntime) process(ctx context.Context, c workflowCandidate, skip done, err = r.consume(ctx, tx, g, c, allow, now) return err } + if c.kind == "SIGNAL" { + done, err = r.expireWorkflowSignal(ctx, tx, g, c, allow, now) + return err + } var step core.ID var expected uint64 var raw []byte @@ -133,7 +140,7 @@ func (r *WorkflowRuntime) process(ctx context.Context, c workflowCandidate, skip if json.Unmarshal(raw, &checkpoint) != nil || checkpoint.SchemaVersion != 1 { return errors.New("workflow checkpoint invalid") } - if checkpoint.Kind == "TIMER" || checkpoint.Kind == "POLL" { + if checkpoint.Kind == "TIMER" || checkpoint.Kind == "POLL" || checkpoint.Kind == "SIGNAL" { done, err = r.advanceClock(ctx, tx, g, c, step, expected, checkpoint, allow, now) return err } diff --git a/internal/storage/mysql/workflow_runtime_graph.go b/internal/storage/mysql/workflow_runtime_graph.go index 69574e8..b424f27 100644 --- a/internal/storage/mysql/workflow_runtime_graph.go +++ b/internal/storage/mysql/workflow_runtime_graph.go @@ -11,8 +11,8 @@ import ( weave "git.yuntuo.hk/k2safe/orbit/weave/model" ) -// lockWorkflowGraph 只由内部运行器调用。身份→应用额度→排序队列→实例→门控， -// 不用启动凭据继续授权；停用身份仍收集在途结果，但禁止创建新工作。 +// lockWorkflowGraph 由内部运行器和已经认证的信号受理事务复用。身份→应用额度→排序队列→实例→门控， +// 运行器不用启动凭据继续授权；停用身份仍收集在途结果，但禁止创建新工作。 func lockWorkflowGraph(ctx context.Context, tx *sql.Tx, ns, id core.ID, skip bool) (*workflowGraph, bool, error) { var enabled bool if err := tx.QueryRowContext(ctx, `SELECT n.enabled AND a.enabled FROM orbit_namespaces n JOIN orbit_applications a ON a.id=n.application_id WHERE n.id=? FOR SHARE`, ns).Scan(&enabled); err != nil { diff --git a/internal/storage/mysql/workflow_runtime_process_integration_test.go b/internal/storage/mysql/workflow_runtime_process_integration_test.go index 951e2ab..be0acd9 100644 --- a/internal/storage/mysql/workflow_runtime_process_integration_test.go +++ b/internal/storage/mysql/workflow_runtime_process_integration_test.go @@ -211,4 +211,55 @@ func TestIntegrationWorkflowRuntimeProcess(t *testing.T) { t.Fatal("poll takeover lost budget or duplicated round") } t.Log("all controllers exited before timer expiry and poll PENDING delivery; two fresh processes preserved original wake/deadline and completed timer plus exactly two query rounds") + for _, p := range processes { + stop(p) + } + early := runtimeStart(t, f, signalSpec(f.resources.QueueID, true)) + grantWorkflowSignal(f) + earlySignal := sendSignal(t, f, early, signalInput(early)) + ref = retryStart(t, f) + retryFinish(t, f, ref, "SUCCEEDED", false) + if f.count("weave_waits", "workflow_id=?", early.ID) != 0 || f.count("weave_signals", "id=? AND state='PENDING'", earlySignal.Signal.ID) != 1 { + t.Fatal("early signal did not remain durable during all-stop") + } + start("signal-replacement-a") + start("signal-replacement-b") + wait(func() bool { + current, err := api.Get(ctx, f.token, early.ID) + return err == nil && current.State == "SUCCEEDED" + }, "early signal after process restart") + if f.count("weave_signals", "id=? AND state='CONSUMED'", earlySignal.Signal.ID) != 1 || f.count("pulse_tasks", "1=1") != 7 { + t.Fatal("early signal restart duplicated work") + } + for _, p := range processes { + stop(p) + } + final := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID)) + grantWorkflowSignal(f) + ref = retryStart(t, f) + permits := f.used() + sendSignal(t, f, final, pollSignalInput(final)) + start("final-signal-selection-a") + start("final-signal-selection-b") + wait(func() bool { return f.get(ref.TaskID).CancelRequestedAt != nil }, "signal cancellation intent") + if f.used() != permits || f.get(ref.TaskID).State != "RUNNING" { + t.Fatal("process released running query on final signal") + } + for _, p := range processes { + stop(p) + } + finishQueryOnly(t, f, ref, "PENDING") + if f.count("weave_inbox", "source_task_id=? AND state='PENDING'", ref.TaskID) != 1 { + t.Fatal("query settlement missing with all controllers stopped") + } + start("final-signal-settlement-a") + start("final-signal-settlement-b") + wait(func() bool { + current, err := api.Get(ctx, f.token, final.ID) + return err == nil && current.State == "SUCCEEDED" + }, "selected final signal and query settlement after restart") + if f.used() != 0 || f.count("pulse_tasks", "1=1") != 8 || f.count("weave_poll_states", "workflow_id=? AND used_rounds=1 AND resolved_at IS NOT NULL", final.ID) != 1 || f.count("weave_pending_advances", "1=1") != 0 { + t.Fatal("final signal restart lost winner or issued an extra round") + } + t.Log("early signal persisted before wait activation across all-stop; final signal selected with running query kept permits, then survived another all-stop and accepted PENDING settlement without issuing a second query") } diff --git a/internal/storage/mysql/workflow_signal_runtime.go b/internal/storage/mysql/workflow_signal_runtime.go new file mode 100644 index 0000000..7951b01 --- /dev/null +++ b/internal/storage/mysql/workflow_signal_runtime.go @@ -0,0 +1,315 @@ +package mysql + +import ( + "bytes" + "context" + "database/sql" + "errors" + "time" + + "git.yuntuo.hk/k2safe/orbit/internal/jsonvalue" + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/weave/engine" + weave "git.yuntuo.hk/k2safe/orbit/weave/model" +) + +// driveWorkflowSignal 同时用于等待时钟和当前查询结果；handled=false时继续原查询分类。 +// reason为空表示内部已消费Inbox并写入审计，调用方不能再次确认同一来源。 +func (r *WorkflowRuntime) driveWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, source *workflowResult, allow bool, now time.Time) (handled, changed bool, reason string, err error) { + // 来源矛盾单独转核查；任何SQL故障仍回滚整个事务并保留原来源。 + if _, err := tx.ExecContext(ctx, `SAVEPOINT workflow_signal_source`); err != nil { + return false, false, "", err + } + before := cloneWorkflowGraph(g) + handled, changed, reason, err = r.applyWorkflowSignal(ctx, tx, g, row, c, source, allow, now) + var invalid workflowSourceError + if !errors.As(err, &invalid) { + return + } + if _, err := tx.ExecContext(ctx, `ROLLBACK TO SAVEPOINT workflow_signal_source`); err != nil { + return false, false, "", err + } + *g = *before + row = g.steps[row.view.Key] + if g.view.State != weave.InstanceNeedsReview { + if err := holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, string(invalid), now); err != nil { + return false, false, "", err + } + } + var current *core.ID + if row.view.Kind == "POLL" { + if err := tx.QueryRowContext(ctx, `SELECT current_task_id FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(¤t); err != nil { + return false, false, "", err + } + if source != nil || current == nil { + if _, err := tx.ExecContext(ctx, `UPDATE weave_poll_states SET current_task_id=NULL,next_poll_at=NULL,resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, now, now, g.view.NamespaceID, g.view.ID, row.view.ActivationID); err != nil { + return false, false, "", err + } + } + } + // 已发出的查询仍由Pulse结清，不能因信号损坏而假设外部执行结束。 + if row.view.Kind != "POLL" || source != nil || current == nil { + if err := saveWorkflowResultStep(ctx, tx, g, row, weave.StepNeedsReview, nil, now); err != nil { + return false, false, "", err + } + } + if _, err := tx.ExecContext(ctx, `UPDATE weave_signals s JOIN weave_activations a ON a.namespace_id=s.namespace_id AND a.workflow_id=s.workflow_id AND a.sequence=s.target_activation_sequence SET s.state='REJECTED',s.disposition=?,s.revision=s.revision+1,s.updated_at=? WHERE s.namespace_id=? AND s.workflow_id=? AND s.target_step_key=? AND a.id=? AND a.step_id=? AND s.state='PENDING'`, string(invalid), now, g.view.NamespaceID, g.view.ID, row.view.Key, row.view.ActivationID, row.view.ID); err != nil { + return false, false, "", err + } + if err := deleteWorkflowClock(ctx, tx, g, row); err != nil { + return false, false, "", err + } + return true, true, string(invalid), nil +} + +func (r *WorkflowRuntime) applyWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, source *workflowResult, allow bool, now time.Time) (handled, changed bool, reason string, err error) { + w, err := readWorkflowSignalWait(ctx, tx, g, row) + if err != nil { + return false, false, "", err + } + node, _ := g.program.Step(row.view.Key) + var poll weave.PollState + if node.Poll != nil { + var last core.ID + poll, last, err = readWorkflowPollState(ctx, tx, g, row, w.activated) + if err != nil { + return false, false, "", err + } + if err := validatePollTaskOwner(ctx, tx, g, row, last); err != nil { + return false, false, "", err + } + if _, err := pollTaskInput(ctx, tx, g, row, node.Poll.Task); err != nil { + return false, false, "", err + } + if poll.CurrentTaskID != nil { + if *poll.CurrentTaskID != last || row.view.State != weave.StepActive || poll.NextPollAt != nil { + return false, false, "", errors.New("signal poll current round invalid") + } + if err := validatePollTaskOwner(ctx, tx, g, row, last); err != nil { + return false, false, "", err + } + if source != nil && source.task != last { + return false, false, "", nil + } + } else { + if source != nil { + return false, false, "", nil + } + if row.view.State != weave.StepWaiting || poll.NextPollAt == nil { + return false, false, "", errors.New("signal poll continuation missing") + } + var disposition string + if err := tx.QueryRowContext(ctx, `SELECT error_code FROM weave_inbox WHERE namespace_id=? AND workflow_id=? AND source_task_id=? AND source_activation_id=? AND state='PROCESSED' ORDER BY source_revision DESC LIMIT 1`, g.view.NamespaceID, g.view.ID, last, row.view.ActivationID).Scan(&disposition); err != nil { + return false, false, "", err + } + if disposition != "POLL_PENDING" { + return false, false, "", errors.New("signal poll previous round not pending") + } + } + } + v, err := selectedWorkflowSignal(ctx, tx, g, row, w, now) + if err != nil { + return false, false, "", err + } + if v == nil { + if node.Poll != nil { + return false, false, "", nil + } + if w.state != "WAITING" { + return false, false, "", errors.New("signal step has settled wait without winner") + } + if now.Before(w.deadline) { + return true, false, "", saveWorkflowClock(ctx, tx, g, row, "SIGNAL", row.view.Revision, w.deadline) + } + res, err := tx.ExecContext(ctx, `UPDATE weave_waits SET state='EXPIRED',resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='WAITING'`, now, now, g.view.NamespaceID, w.id) + if err := exactlyOne(res, err); err != nil { + return true, false, "", err + } + if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "SIGNAL_DEADLINE", now); err != nil { + return true, false, "", err + } + if err := saveWorkflowResultStep(ctx, tx, g, row, weave.StepNeedsReview, nil, now); err != nil { + return true, false, "", err + } + if err := deleteWorkflowClock(ctx, tx, g, row); err != nil { + return true, false, "", err + } + return true, true, "SIGNAL_DEADLINE", holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, "SIGNAL_DEADLINE", now) + } + if node.Poll != nil && source == nil && poll.CurrentTaskID != nil { + // 已落库查询结果必须按原事件时间参与竞争，不能由扫描先后决定赢家。 + task, err := readTask(ctx, tx, g.view.NamespaceID, *poll.CurrentTaskID, false) + if err != nil { + return false, false, "", err + } + if task.QueueID != node.Poll.Task.QueueID { + return false, false, "", errors.New("signal query queue invalid") + } + if task.State.Terminal() || task.State == "NEEDS_REVIEW" { + pending := workflowCandidate{kind: "INBOX", ns: g.view.NamespaceID, workflow: g.view.ID} + var due time.Time + if err := tx.QueryRowContext(ctx, `SELECT id,revision,available_at FROM weave_inbox WHERE namespace_id=? AND workflow_id=? AND source_task_id=? AND source_activation_id=? AND state='PENDING' ORDER BY source_revision DESC LIMIT 1`, g.view.NamespaceID, g.view.ID, poll.CurrentTaskID, row.view.ActivationID).Scan(&pending.id, &pending.revision, &due); err != nil { + return false, false, "", err + } + if due.After(now) { + return true, false, "", saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision, due) + } + done, err := r.consume(ctx, tx, g, pending, allow, now) + return true, done, "", err + } + } + if node.Poll != nil && source != nil && w.winner == nil && source.snapshot.State == "SUCCEEDED" && !source.snapshot.EffectUncertain && source.occurred.Before(w.deadline) && !source.occurred.After(v.AcceptedAt) { + outcome, err := g.program.ClassifyPoll(row.view.Key, source.snapshot.Result) + if err == nil && outcome != engine.PollPending { + return false, false, "", nil + } // 较早的最终查询先胜出；PENDING不争夺终结权。 + } + newWinner := w.winner == nil + if err := claimWorkflowSignal(ctx, tx, g, &w, v, now); err != nil { + return true, false, "", err + } + if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "WAIT_SETTLED", now); err != nil { + return true, false, "", err + } + if node.Poll != nil && poll.CurrentTaskID != nil && source == nil { + task, err := readExecutionTask(ctx, tx, g.view.NamespaceID, *poll.CurrentTaskID) + if err != nil { + return true, false, "", err + } + if _, err := cancelExecutionTask(ctx, tx, g.view.NamespaceID, task, now, "WORKFLOW_SIGNAL_WON"); err != nil { + return true, false, "", err + } + if !now.Before(poll.DeadlineAt) && g.view.State != weave.InstanceNeedsReview { + if err := holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, "POLL_SIGNAL_INFLIGHT_DEADLINE", now); err != nil { + return true, false, "", err + } + newWinner = true + } + if err := saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision, now.Add(time.Second)); err != nil { + return true, false, "", err + } + if newWinner { + return true, true, "POLL_SIGNAL_SELECTED", nil + } + return true, false, "", nil + } + state, output, reason := weave.StepSucceeded, v.Payload, "SIGNAL_CONSUMED" + if node.Poll != nil { + outcome, err := g.program.ClassifyPoll(row.view.Key, v.Payload) + if err != nil || outcome == engine.PollPending { + return true, false, "", workflowSourceError("POLL_FINAL_SIGNAL_INVALID") + } + switch outcome { + case engine.PollFailed: + state, output, reason = weave.StepFailed, nil, "POLL_SIGNAL_FAILED" + case engine.PollNeedsReview: + state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_REVIEW" + default: + reason = "POLL_SIGNAL_SUCCEEDED" + } + if source != nil { + if source.snapshot.EffectUncertain || source.snapshot.State == "NEEDS_REVIEW" { + state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_UNCERTAIN" + } else if source.snapshot.State == "SUCCEEDED" { + query, err := g.program.ClassifyPoll(row.view.Key, source.snapshot.Result) + if err != nil { + state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_INVALID" + } else if query != engine.PollPending { + left, err := jsonvalue.Canonical(source.snapshot.Result) + if err != nil { + return true, false, "", err + } + right, err := jsonvalue.Canonical(v.Payload) + if err != nil { + return true, false, "", err + } + if !bytes.Equal(left, right) { + state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_CONFLICT" + } + } + } + } + res, err := tx.ExecContext(ctx, `UPDATE weave_poll_states SET current_task_id=NULL,next_poll_at=NULL,resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND resolved_at IS NULL`, now, now, g.view.NamespaceID, poll.ID) + if err := exactlyOne(res, err); err != nil { + return true, false, "", err + } + } + if err := saveWorkflowResultStep(ctx, tx, g, row, state, output, now); err != nil { + return true, false, "", err + } + if err := deleteWorkflowClock(ctx, tx, g, row); err != nil { + return true, false, "", err + } + if state == weave.StepNeedsReview || state == weave.StepFailed { + next := weave.InstanceNeedsReview + if state == weave.StepFailed && (g.version.Specification.FailurePolicy == "FAIL_AFTER_SETTLED" || g.view.CancelRequestedAt != nil) { + next = weave.InstanceCancelling + } + if err := holdWorkflow(ctx, tx, g, next, reason, now); err != nil { + return true, false, "", err + } + } else if err := r.progress(ctx, tx, g, allow, now); err != nil { + return true, false, "", err + } + return true, true, reason, nil +} + +func (r *WorkflowRuntime) advanceSignalClock(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, allow bool, now time.Time) (bool, error) { + handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now) + if err != nil { + return false, err + } + if !handled { + return false, errors.New("signal clock did not identify a wait") + } + if changed && reason != "" { + return true, r.audit(ctx, tx, g, c.id, reason, now) + } + return changed, nil +} + +// expireWorkflowSignal 对早到但未匹配的信号使用总截止索引；已按时匹配的信号仍先完成实际消费。 +func (r *WorkflowRuntime) expireWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, c workflowCandidate, allow bool, now time.Time) (bool, error) { + var due time.Time + if err := tx.QueryRowContext(ctx, `SELECT available_at FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, c.ns, c.workflow, c.id).Scan(&due); err != nil { + return false, err + } + if due.After(now) { + return false, nil + } + v, err := readWorkflowSignal(ctx, tx, c.ns, c.workflow, c.id) + if err != nil { + return false, err + } + if v.State != "PENDING" || now.Before(v.ExpiresAt) { + return false, nil + } + row := g.steps[v.TargetStepKey] + if row != nil && row.view.ActivationID != nil && !g.view.State.Terminal() && (row.view.State == weave.StepWaiting || row.view.State == weave.StepActive) { + w, err := readWorkflowSignalWait(ctx, tx, g, row) + if err != nil { + return false, err + } + if w.sequence == v.TargetActivationSequence && w.correlation == v.CorrelationKey && w.contract.Name == v.Name && w.contract.PayloadVersion == v.PayloadVersion && v.AcceptedAt.Before(w.deadline) { + handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now) + if err != nil { + return false, err + } + if handled { + // 若匹配到另一条更早的信号或正等待结果退避，避免过期索引持续抢占批次。 + if _, err := tx.ExecContext(ctx, `UPDATE weave_signals SET available_at=DATE_ADD(?,INTERVAL 30 SECOND),revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='PENDING'`, now, now, c.ns, c.id); err != nil { + return false, err + } + if changed && reason != "" { + return true, r.audit(ctx, tx, g, c.id, reason, now) + } + return changed, nil + } + } + } + res, err := tx.ExecContext(ctx, `UPDATE weave_signals SET state='EXPIRED',disposition='UNMATCHED_DEADLINE',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='PENDING' AND revision=?`, now, c.ns, c.id, v.Revision) + if err := exactlyOne(res, err); err != nil { + return false, err + } + return true, r.audit(ctx, tx, g, c.id, "SIGNAL_EXPIRED", now) +} diff --git a/internal/storage/mysql/workflow_signal_runtime_integration_test.go b/internal/storage/mysql/workflow_signal_runtime_integration_test.go new file mode 100644 index 0000000..77dd9ea --- /dev/null +++ b/internal/storage/mysql/workflow_signal_runtime_integration_test.go @@ -0,0 +1,361 @@ +package mysql + +import ( + "context" + "encoding/json" + "os" + "strings" + "sync" + "testing" + "time" + + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/pulse/worker" + "git.yuntuo.hk/k2safe/orbit/weave/definition" + "git.yuntuo.hk/k2safe/orbit/weave/instance" + "git.yuntuo.hk/k2safe/orbit/weave/signal" +) + +func sendSignal(t *testing.T, f *executionFixture, v instance.View, in signal.Input) signal.Result { + t.Helper() + out, err := signal.New(NewWorkflowSignalStore(f.db)).Send(f.ctx, f.token, v.ID, string(executionID(t)), in) + if err != nil { + t.Fatal(err) + } + return out +} +func finalSignalSpec(queue core.ID) definition.Specification { + s := pollingSpec(queue) + p := s.Body.Steps[0].Poll + p.Task.ResultType.Enum = append(p.Task.ResultType.Enum, "FAILED") + p.Failed = []string{"FAILED"} + p.FinalSignal = &definition.Signal{Name: "external.done", PayloadVersion: "v1", Correlation: definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "workflow", Path: []string{"business_key"}}}, PayloadType: p.Task.ResultType, TimeoutMS: p.TimeoutMS} + s.Body.OutputType = p.Task.ResultType + return s +} +func pollSignalInput(v instance.View) signal.Input { + in := signalInput(v) + in.TargetStepKey = "query" + in.Payload = json.RawMessage(`"DONE"`) + return in +} +func finishQueryOnly(t *testing.T, f *executionFixture, ref worker.AttemptRef, result string) { + t.Helper() + raw, _ := json.Marshal(result) + if _, err := f.a.Finish(f.ctx, f.token, f.binding, executionID(t), worker.FinishInput{Ref: ref, Outcome: "SUCCEEDED", Result: raw}); err != nil { + t.Fatal(err) + } +} +func TestIntegrationWorkflowSignalRuntime(t *testing.T) { + dsn := os.Getenv("ORBIT_TEST_MYSQL_DSN") + if dsn == "" { + t.Skip("requires disposable MySQL") + } + ctx, stop := context.WithTimeout(context.Background(), 150*time.Second) + defer stop() + db, err := Open(ctx, dsn, 32, 8) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if err := Migrate(ctx, db); err != nil { + t.Fatal(err) + } + t.Run("wait_then_twelve_consumers_one_successor", func(t *testing.T) { + f, v, api := signalStart(t, ctx, db, false) + c := clockCandidate(t, f, v.ID, "signal") + if runClock(t, f, c) || f.count("pulse_tasks", "1=1") != 0 { + t.Fatal("empty wait advanced") + } + in := signalInput(v) + key := string(executionID(t)) + out, err := api.Send(ctx, f.token, v.ID, key, in) + if err != nil { + t.Fatal(err) + } + other := sendSignal(t, f, v, in) + c = clockCandidate(t, f, v.ID, "signal") + var wg sync.WaitGroup + errs := make(chan error, 12) + results := make(chan bool, 12) + for i := 0; i < 12; i++ { + wg.Add(1) + go func() { + defer wg.Done() + done, err := newRuntime(t, db).process(ctx, c, false) + errs <- err + results <- done + }() + } + wg.Wait() + close(errs) + close(results) + for err := range errs { + if err != nil { + t.Fatal(err) + } + } + changed := 0 + for done := range results { + if done { + changed++ + } + } + if changed != 1 || f.count("pulse_tasks", "1=1") != 1 || f.count("weave_signals", "state='CONSUMED'") != 1 || f.count("weave_signals", "state='REJECTED'") != 1 { + t.Fatal("duplicate signal advancement", changed) + } + current, err := api.Get(ctx, f.token, v.ID, out.Signal.ID) + if err != nil || current.State != "CONSUMED" || current.ConsumedWaitID == nil { + t.Fatal("signal not consumed", err) + } + rejected, err := api.Get(ctx, f.token, v.ID, other.Signal.ID) + if err != nil || rejected.Disposition != "WAIT_SETTLED" { + t.Fatal("competing signal not retained", err) + } + replay, err := api.Send(ctx, f.token, v.ID, key, in) + if err != nil || !replay.Deduplicated || replay.Signal.State != "PENDING" || replay.Signal.Revision != 1 { + t.Fatal("original receipt was mutated", err) + } + var output string + if err := db.QueryRowContext(ctx, `SELECT output FROM weave_activations WHERE namespace_id=? AND id=(SELECT current_activation_id FROM weave_steps WHERE namespace_id=? AND workflow_id=? AND `+"`key`"+`='signal')`, f.resources.NamespaceID, f.resources.NamespaceID, v.ID).Scan(&output); err != nil || !strings.Contains(output, "9007199254740993") { + t.Fatal("signal number changed", err) + } + }) + t.Run("early_signal_survives_activation_and_fresh_runtime", func(t *testing.T) { + f, v, _ := signalStart(t, ctx, db, true) + sendSignal(t, f, v, signalInput(v)) + if f.count("weave_waits", "1=1") != 0 { + t.Fatal("early signal activated blocked step") + } + ref := retryStart(t, f) + retryFinish(t, f, ref, "SUCCEEDED", false) + runtimeConsume(t, f, newRuntime(t, db), ref.TaskID) + if !runClock(t, f, clockCandidate(t, f, v.ID, "signal")) { + t.Fatal("early signal not resumed") + } + out := runtimeView(t, f, v.ID, "SUCCEEDED") + if !strings.Contains(string(out.Output), "9007199254740993") || f.count("pulse_tasks", "1=1") != 1 { + t.Fatal("early result lost or new work created") + } + }) + t.Run("paused_collects_and_persists_root", func(t *testing.T) { + f, v, _ := signalStart(t, ctx, db, false) + f.exec(`UPDATE weave_instances SET paused=TRUE WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, v.ID) + f.exec(`UPDATE pulse_execution_gates SET paused=TRUE WHERE namespace_id=?`, f.resources.NamespaceID) + sendSignal(t, f, v, signalInput(v)) + runClock(t, f, clockCandidate(t, f, v.ID, "signal")) + if f.count("weave_signals", "state='CONSUMED'") != 1 || f.count("pulse_tasks", "1=1") != 0 || f.count("weave_pending_advances", "1=1") != 1 { + t.Fatal("pause dropped evidence or started successor") + } + f.exec(`UPDATE weave_instances SET paused=FALSE WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, v.ID) + f.exec(`UPDATE pulse_execution_gates SET paused=FALSE WHERE namespace_id=?`, f.resources.NamespaceID) + runtimeAdvance(t, f, newRuntime(t, db), v.ID) + if f.count("pulse_tasks", "1=1") != 1 { + t.Fatal("root did not resume") + } + }) + for _, timely := range []bool{false, true} { + name := "deadline_without_signal" + if timely { + name = "timely_acceptance_after_consumer_deadline" + } + t.Run(name, func(t *testing.T) { + f := newExecutionFixture(t, ctx, db) + spec := signalSpec(f.resources.QueueID, false) + spec.Body.Steps[0].Signal.TimeoutMS = 1000 + v := runtimeStart(t, f, spec) + grantWorkflowSignal(f) + if timely { + sendSignal(t, f, v, signalInput(v)) + } + c := clockCandidate(t, f, v.ID, "signal") + if !timely { + runClock(t, f, c) + } + var due time.Time + if err := db.QueryRowContext(ctx, `SELECT deadline_at FROM weave_waits WHERE namespace_id=?`, f.resources.NamespaceID).Scan(&due); err != nil { + t.Fatal(err) + } + for time.Now().Before(due.Add(20 * time.Millisecond)) { + time.Sleep(20 * time.Millisecond) + } + if !runClock(t, f, c) { + t.Fatal("deadline not handled") + } + if timely { + runtimeView(t, f, v.ID, "RUNNING") + if f.count("pulse_tasks", "1=1") != 1 { + t.Fatal("timely signal lost") + } + } else { + runtimeView(t, f, v.ID, "NEEDS_REVIEW") + if f.count("weave_waits", "state='EXPIRED'") != 1 || f.count("pulse_tasks", "1=1") != 0 { + t.Fatal("expired wait progressed") + } + late := sendSignal(t, f, v, signalInput(v)) + if late.Signal.State != "REJECTED" { + t.Fatal("settled wait accepted late signal") + } + } + }) + } + t.Run("audit_failure_rolls_back_consumption_wait_successor", func(t *testing.T) { + f, v, _ := signalStart(t, ctx, db, false) + sendSignal(t, f, v, signalInput(v)) + c := clockCandidate(t, f, v.ID, "signal") + f.exec(`CREATE TRIGGER test_signal_consume_failure BEFORE INSERT ON orbit_audit_events FOR EACH ROW SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='injected signal consume failure'`) + defer f.exec(`DROP TRIGGER IF EXISTS test_signal_consume_failure`) + if done, err := newRuntime(t, db).process(ctx, c, false); err == nil || done { + t.Fatal("audit failure committed") + } + if f.count("weave_signals", "state='PENDING'") != 1 || f.count("weave_waits", "state='WAITING' AND winner_signal_id IS NULL") != 1 || f.count("pulse_tasks", "1=1") != 0 { + t.Fatal("partial signal transaction") + } + f.exec(`DROP TRIGGER test_signal_consume_failure`) + if !runClock(t, f, c) || f.count("pulse_tasks", "1=1") != 1 { + t.Fatal("rollback not recoverable") + } + }) + for _, mutation := range []string{"payload", "actor", "accepted_at"} { + t.Run("tampered_"+mutation+"_requires_review", func(t *testing.T) { + f, v, _ := signalStart(t, ctx, db, false) + out := sendSignal(t, f, v, signalInput(v)) + switch mutation { + case "payload": + f.exec(`UPDATE weave_signals SET payload='{"n":1}' WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID) + case "actor": + f.exec(`UPDATE weave_signals SET actor_id='spoofed' WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID) + default: + f.exec(`UPDATE weave_signals SET accepted_at=DATE_SUB(accepted_at,INTERVAL 1 SECOND) WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID) + } + if !runClock(t, f, clockCandidate(t, f, v.ID, "signal")) { + t.Fatal("invalid source not handled") + } + runtimeView(t, f, v.ID, "NEEDS_REVIEW") + if f.count("weave_signals", "state='REJECTED'") != 1 || f.count("weave_steps", "state='NEEDS_REVIEW'") != 1 || f.count("pulse_tasks", "1=1") != 0 { + t.Fatal("mutable signal authorized work") + } + }) + } + t.Run("early_mismatch_expires_without_changing_deadline_on_backoff", func(t *testing.T) { + f, v, _ := signalStart(t, ctx, db, true) + in := signalInput(v) + in.CorrelationKey += "wrong" + out := sendSignal(t, f, v, in) + c := workflowCandidate{kind: "SIGNAL", ns: f.resources.NamespaceID, workflow: v.ID, id: out.Signal.ID, revision: 1} + // 仅故障注入过期扫描条件，不把篡改过的信号用于匹配或推进。 + f.exec(`UPDATE weave_signals SET expires_at=DATE_SUB(UTC_TIMESTAMP(6),INTERVAL 1 SECOND),available_at=DATE_SUB(UTC_TIMESTAMP(6),INTERVAL 1 SECOND) WHERE namespace_id=? AND id=?`, c.ns, c.id) + var expires time.Time + if err := db.QueryRowContext(ctx, `SELECT expires_at FROM weave_signals WHERE namespace_id=? AND id=?`, c.ns, c.id).Scan(&expires); err != nil { + t.Fatal(err) + } + r := newRuntime(t, db) + if err := r.deferCandidate(ctx, c); err != nil { + t.Fatal(err) + } + var current, due time.Time + if err := db.QueryRowContext(ctx, `SELECT expires_at,available_at FROM weave_signals WHERE namespace_id=? AND id=?`, c.ns, c.id).Scan(¤t, &due); err != nil || !current.Equal(expires) || !due.After(time.Now()) { + t.Fatal("backoff extended deadline", err) + } + if done, err := r.process(ctx, c, false); err != nil || done { + t.Fatal("ignored backoff", err) + } + f.exec(`UPDATE weave_signals SET available_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=?`, c.ns, c.id) + if done, err := r.process(ctx, c, false); err != nil || !done { + t.Fatal("unmatched expiry failed", err) + } + if f.count("weave_signals", "state='EXPIRED' AND disposition='UNMATCHED_DEADLINE'") != 1 || f.count("pulse_tasks", "1=1") != 1 { + t.Fatal("expiry authorized work") + } + }) + t.Run("poll_signal_between_rounds_does_not_issue_query", func(t *testing.T) { + f := newClockFixture(t, ctx, db) + v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID)) + grantWorkflowSignal(f) + ref := retryStart(t, f) + finishPoll(t, f, ref, "PENDING") + sendSignal(t, f, v, pollSignalInput(v)) + runClock(t, f, clockCandidate(t, f, v.ID, "query")) + runtimeView(t, f, v.ID, "SUCCEEDED") + if f.count("pulse_tasks", "1=1") != 1 || f.count("weave_poll_states", "used_rounds=1 AND resolved_at IS NOT NULL") != 1 || f.used() != 0 { + t.Fatal("final signal consumed extra budget") + } + }) + t.Run("poll_signal_cancels_unstarted_query", func(t *testing.T) { + f := newClockFixture(t, ctx, db) + v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID)) + grantWorkflowSignal(f) + sendSignal(t, f, v, pollSignalInput(v)) + runClock(t, f, clockCandidate(t, f, v.ID, "query")) + var task core.ID + if err := db.QueryRowContext(ctx, `SELECT id FROM pulse_tasks WHERE namespace_id=?`, f.resources.NamespaceID).Scan(&task); err != nil { + t.Fatal(err) + } + if f.get(task).State != "CANCELLED" || f.count("weave_steps", "`key`='query' AND state='ACTIVE'") != 1 { + t.Fatal("unstarted query not safely cancelled") + } + runtimeConsume(t, f, newRuntime(t, db), task) + runtimeView(t, f, v.ID, "SUCCEEDED") + if f.count("weave_inbox", "state='PROCESSED'") != 1 || f.count("weave_pending_advances", "1=1") != 0 { + t.Fatal("cancel result not collected") + } + }) + for _, outcome := range []string{"CANCELLED", "PENDING", "DONE", "UNKNOWN", "FAILED", "CONFLICT"} { + t.Run("poll_running_signal_then_"+outcome, func(t *testing.T) { + f := newClockFixture(t, ctx, db) + v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID)) + grantWorkflowSignal(f) + ref := retryStart(t, f) + before := f.used() + sendSignal(t, f, v, pollSignalInput(v)) + runClock(t, f, clockCandidate(t, f, v.ID, "query")) + task := f.get(ref.TaskID) + if task.State != "RUNNING" || task.CancelRequestedAt == nil || f.used() != before || f.count("weave_poll_states", "current_task_id IS NOT NULL AND resolved_at IS NULL") != 1 { + t.Fatal("signal prematurely released execution") + } + switch outcome { + case "PENDING", "DONE": + finishQueryOnly(t, f, ref, outcome) + case "CONFLICT": + finishQueryOnly(t, f, ref, "FAILED") + default: + retryFinish(t, f, ref, outcome, false) + } + runtimeConsume(t, f, newRuntime(t, db), ref.TaskID) + want := "SUCCEEDED" + if outcome == "UNKNOWN" || outcome == "CONFLICT" { + want = "NEEDS_REVIEW" + } + runtimeView(t, f, v.ID, want) + if f.used() != 0 || f.count("pulse_tasks", "1=1") != 1 || f.count("weave_poll_states", "current_task_id IS NULL AND resolved_at IS NOT NULL") != 1 { + t.Fatal("query not settled exactly once") + } + }) + } + for _, clockFirst := range []bool{true, false} { + name := "query_earlier_inbox_first" + if clockFirst { + name = "query_earlier_clock_first" + } + t.Run(name, func(t *testing.T) { + f := newClockFixture(t, ctx, db) + v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID)) + grantWorkflowSignal(f) + ref := retryStart(t, f) + finishQueryOnly(t, f, ref, "DONE") + in := pollSignalInput(v) + in.Payload = json.RawMessage(`"FAILED"`) + sendSignal(t, f, v, in) + if clockFirst { + runClock(t, f, clockCandidate(t, f, v.ID, "query")) + } else { + runtimeConsume(t, f, newRuntime(t, db), ref.TaskID) + } + runtimeView(t, f, v.ID, "SUCCEEDED") + if f.count("weave_signals", "state='REJECTED' AND disposition='QUERY_SETTLED'") != 1 || f.count("weave_waits", "state='SATISFIED' AND winner_signal_id IS NULL") != 1 { + t.Fatal("scan order changed winner") + } + }) + } + t.Log("signal wait/early/duplicate/deadline/pause/rollback/proof and POLL final-signal settlement verified on real MySQL") +} diff --git a/internal/storage/mysql/workflow_signal_source.go b/internal/storage/mysql/workflow_signal_source.go new file mode 100644 index 0000000..081c0ea --- /dev/null +++ b/internal/storage/mysql/workflow_signal_source.go @@ -0,0 +1,166 @@ +package mysql + +import ( + "context" + "database/sql" + "encoding/hex" + "encoding/json" + "errors" + "time" + "unicode/utf8" + + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/weave/definition" + "git.yuntuo.hk/k2safe/orbit/weave/signal" +) + +type workflowSignalWait struct { + id core.ID + sequence uint32 + state, correlation string + deadline, activated time.Time + winner *core.ID + contract *definition.Signal +} + +// readWorkflowSignalWait 读取已经冻结的等待身份，不能把历史NULL元数据解释成有效信号契约。 +func readWorkflowSignalWait(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep) (workflowSignalWait, error) { + var w workflowSignalWait + node, ok := g.program.Step(row.view.Key) + w.contract = signalContract(node) + if !ok || w.contract == nil || row.view.ActivationID == nil { + return w, errors.New("signal step definition invalid") + } + var count int + if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_waits WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&count); err != nil { + return w, err + } + if count != 1 { + return w, errors.New("signal wait set invalid") + } + var name, version *string + var kind string + err := tx.QueryRowContext(ctx, `SELECT w.id,w.kind,w.state,w.signal_name,w.payload_version,w.correlation_key,w.deadline_at,w.winner_signal_id,a.sequence,a.created_at FROM weave_waits w JOIN weave_activations a ON a.namespace_id=w.namespace_id AND a.workflow_id=w.workflow_id AND a.id=w.activation_id AND a.step_id=? WHERE w.namespace_id=? AND w.workflow_id=? AND w.activation_id=? FOR UPDATE`, row.view.ID, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&w.id, &kind, &w.state, &name, &version, &w.correlation, &w.deadline, &w.winner, &w.sequence, &w.activated) + if err != nil { + return w, err + } + if name == nil || version == nil { + return w, workflowSourceError("SIGNAL_WAIT_UNVERIFIED") + } + if w.sequence == 0 || kind != string(row.view.Kind) || *name != w.contract.Name || *version != w.contract.PayloadVersion || !utf8.ValidString(w.correlation) || len(w.correlation) == 0 || len(w.correlation) > 512 || !w.deadline.Equal(minimumTime(g.view.DeadlineAt, w.activated.Add(time.Duration(w.contract.TimeoutMS)*time.Millisecond))) { + return w, workflowSourceError("SIGNAL_WAIT_INVALID") + } + if node.Signal != nil { + var input struct { + Name string `json:"name"` + Version string `json:"payload_version"` + Correlation string `json:"correlation"` + } + if json.Unmarshal(row.input, &input) != nil || input.Name != *name || input.Version != *version || input.Correlation != w.correlation { + return w, workflowSourceError("SIGNAL_ACTIVATION_INPUT_INVALID") + } + } + return w, nil +} + +// verifyWorkflowSignal 核对不可改写受理审计及请求摘要，防止可变状态读取替换原受理参数或时间。 +func verifyWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, w workflowSignalWait, v signal.View, now time.Time) error { + if v.NamespaceID != g.view.NamespaceID || v.WorkflowID != g.view.ID || v.TargetStepKey != row.view.Key || v.TargetActivationSequence != w.sequence || v.CorrelationKey != w.correlation || v.Name != w.contract.Name || v.PayloadVersion != w.contract.PayloadVersion || v.AcceptedAt.After(now) || !v.AcceptedAt.Before(w.deadline) || !v.ExpiresAt.Equal(g.view.DeadlineAt) || definition.ValidateData(w.contract.PayloadType, v.Payload) != nil { + return workflowSourceError("SIGNAL_IDENTITY_INVALID") + } + var digest core.Digest + if err := tx.QueryRowContext(ctx, `SELECT request_digest FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, g.view.NamespaceID, g.view.ID, v.ID).Scan(&digest); err != nil { + return err + } + var count int + if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM orbit_audit_events WHERE namespace_id=? AND resource_kind='workflow_signal' AND resource_id=? AND action='workflow.signal'`, g.view.NamespaceID, v.ID).Scan(&count); err != nil { + return err + } + if count != 1 { + return workflowSourceError("SIGNAL_ACCEPTANCE_PROOF_MISSING") + } + var actor, reason string + var evidence sql.NullString + var raw []byte + if err := tx.QueryRowContext(ctx, "SELECT actor_id,reason,evidence_ref,`after` FROM orbit_audit_events WHERE namespace_id=? AND resource_kind='workflow_signal' AND resource_id=? AND action='workflow.signal'", g.view.NamespaceID, v.ID).Scan(&actor, &reason, &evidence, &raw); err != nil { + return err + } + var meta struct { + SignalID core.ID `json:"signal_id"` + WorkflowID core.ID `json:"workflow_id"` + Key string `json:"target_step_key"` + Sequence uint32 `json:"target_activation_sequence"` + Digest string `json:"request_digest"` + AcceptedAt time.Time `json:"accepted_at"` + ExpiresAt time.Time `json:"expires_at"` + } + if json.Unmarshal(raw, &meta) != nil || meta.SignalID != v.ID || meta.WorkflowID != g.view.ID || meta.Key != row.view.Key || meta.Sequence != w.sequence || meta.Digest != hex.EncodeToString(digest[:]) || !meta.AcceptedAt.Equal(v.AcceptedAt) || !meta.ExpiresAt.Equal(v.ExpiresAt) || actor != v.ActorID || evidence.String != v.EvidenceRef { + return workflowSourceError("SIGNAL_ACCEPTANCE_PROOF_INVALID") + } + expected, err := requestDigest(signal.Input{Name: v.Name, PayloadVersion: v.PayloadVersion, TargetStepKey: v.TargetStepKey, TargetActivationSequence: v.TargetActivationSequence, CorrelationKey: v.CorrelationKey, Payload: v.Payload, Reason: reason, EvidenceRef: v.EvidenceRef}) + if err != nil { + return err + } + if expected != digest { + return workflowSourceError("SIGNAL_REQUEST_CHANGED") + } + return nil +} + +// selectedWorkflowSignal 返回已胜出信号或最早按时受理的匹配信号；查询结果竞争由调用方在同一实例锁下裁决。 +func selectedWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, w workflowSignalWait, now time.Time) (*signal.View, error) { + var id core.ID + if w.winner != nil { + if w.state != "SATISFIED" { + return nil, workflowSourceError("SIGNAL_WINNER_WAIT_INVALID") + } + id = *w.winner + } else { + if w.state != "WAITING" { + return nil, nil + } + err := tx.QueryRowContext(ctx, `SELECT id FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND target_step_key=? AND target_activation_sequence=? AND state='PENDING' AND type=? AND payload_version=? AND correlation_key=? AND accepted_at 19 { + break + } + statements, err := splitSQL(m.SQL) + if err != nil { + t.Fatal(err) + } + for _, q := range statements { + exec(q) + } + exec(`INSERT INTO orbit_schema_migrations(version,name,checksum,dirty,applied_at) VALUES (?,?,?,FALSE,UTC_TIMESTAMP(6))`, m.Version, m.Name, m.Checksum) + } + var now time.Time + if err := db.QueryRowContext(ctx, `SELECT UTC_TIMESTAMP(6)`).Scan(&now); err != nil { + t.Fatal(err) + } + ns := executionID(t) + deadline := now.Add(time.Hour) + type fixture struct{ workflow, step, activation core.ID } + seed := func(key string, terminal, stale bool) fixture { + t.Helper() + f := fixture{executionID(t), executionID(t), executionID(t)} + state := "WAITING" + var completed *time.Time + if terminal { + state = "CANCELLED" + completed = &now + } + exec(`INSERT INTO weave_instances(namespace_id,id,definition_id,definition_version_id,gate_id,state,input,policy_snapshot,deadline_at,completed_at) VALUES (?,?,?,?,?,?,'{}','{}',?,?)`, ns, f.workflow, executionID(t), executionID(t), executionID(t), state, deadline, completed) + current := f.activation + if stale { + current = executionID(t) + } + exec("INSERT INTO weave_steps(namespace_id,id,workflow_id,`key`,kind,state,current_activation_id,revision) VALUES (?,?,?,?,'SIGNAL','WAITING',?,7)", ns, f.step, f.workflow, key, current) + exec(`INSERT INTO weave_activations(namespace_id,id,workflow_id,step_id,sequence,state,input) VALUES (?,?,?,?,1,'WAITING','null')`, ns, f.activation, f.workflow, f.step) + exec(`INSERT INTO weave_waits(namespace_id,id,workflow_id,activation_id,kind,correlation_key,deadline_at) VALUES (?,?,?,?,'SIGNAL','historical',?)`, ns, executionID(t), f.workflow, f.activation, deadline) + return f + } + active := seed("signal", false, false) + existing := seed("existing", false, false) + seed("terminal", true, false) + seed("stale", false, true) + existingID := executionID(t) + original := `{"schema_version":1,"kind":"SIGNAL","activation_id":"` + string(existing.activation) + `"}` + exec(`INSERT INTO weave_pending_advances(namespace_id,id,workflow_id,step_id,expected_step_revision,checkpoint,available_at,revision) VALUES (?,?,?,?,7,?,?,11)`, ns, existingID, existing.workflow, existing.step, original, now) + pending, consumed, wait := executionID(t), executionID(t), executionID(t) + correlation := strings.Repeat("😀", 191) + name64 := strings.Repeat("a", 64) + exec(`INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,target_step_key,correlation_key,payload,actor_id,accepted_at,expires_at) VALUES (?,?,?,RANDOM_BYTES(32),RANDOM_BYTES(32),?,'signal',?,'{"n":9007199254740993}','legacy',?,?)`, ns, pending, active.workflow, name64, correlation, now, deadline) + exec(`INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,target_step_key,correlation_key,payload,actor_id,state,consumed_wait_id,accepted_at,expires_at) VALUES (?,?,?,RANDOM_BYTES(32),RANDOM_BYTES(32),'done','signal','history ','null','legacy','CONSUMED',?,?,?)`, ns, consumed, active.workflow, wait, now, deadline) + if err := Migrate(ctx, db); err != nil { + t.Fatal(err) + } + if err := CheckSchema(ctx, db); err != nil { + t.Fatal(err) + } + var payload, corr, signalName, state, disposition string + var sequence *uint32 + var version *string + var revision uint64 + var expires, available time.Time + if err := db.QueryRowContext(ctx, `SELECT payload,correlation_key,type,state,disposition,target_activation_sequence,payload_version,revision,expires_at,available_at FROM weave_signals WHERE namespace_id=? AND id=?`, ns, pending).Scan(&payload, &corr, &signalName, &state, &disposition, &sequence, &version, &revision, &expires, &available); err != nil { + t.Fatal(err) + } + if !strings.Contains(payload, "9007199254740993") || corr != correlation || signalName != name64 || state != "REJECTED" || disposition != "LEGACY_UNVERIFIED" || sequence != nil || version != nil || revision != 2 || !expires.Equal(deadline) || !available.Equal(deadline) { + t.Fatal("legacy signal rewritten or treated as verified") + } + var count int + if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_signals WHERE namespace_id=? AND id=? AND state='CONSUMED' AND consumed_wait_id=? AND revision=1 AND payload_version IS NULL AND target_activation_sequence IS NULL AND correlation_key=?`, ns, consumed, wait, []byte("history ")).Scan(&count); err != nil || count != 1 { + t.Fatal("historical consumed evidence changed", err) + } + var raw []byte + var due time.Time + var expected uint64 + if err := db.QueryRowContext(ctx, `SELECT checkpoint,available_at,expected_step_revision FROM weave_pending_advances WHERE namespace_id=? AND workflow_id=? AND step_id=?`, ns, active.workflow, active.step).Scan(&raw, &due, &expected); err != nil { + t.Fatal(err) + } + var checkpoint workflowCheckpoint + if err := json.Unmarshal(raw, &checkpoint); err != nil || checkpoint.Kind != "SIGNAL" || checkpoint.SchemaVersion != 1 || checkpoint.ActivationID == nil || *checkpoint.ActivationID != active.activation || !due.Equal(deadline) || expected != 7 { + t.Fatal("signal checkpoint reset original wait deadline", err) + } + if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_pending_advances`).Scan(&count); err != nil || count != 2 { + t.Fatal("stale/terminal or duplicate checkpoint", count, err) + } + if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_pending_advances WHERE id=? AND revision=11 AND available_at=? AND checkpoint=?`, existingID, now, original).Scan(&count); err != nil || count != 1 { + t.Fatal("existing checkpoint changed", err) + } + if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM information_schema.REFERENTIAL_CONSTRAINTS WHERE CONSTRAINT_SCHEMA=DATABASE()`).Scan(&count); err != nil || count != 0 { + t.Fatal("foreign keys introduced", err) + } + if err := Migrate(ctx, db); err != nil { + t.Fatal(err) + } + if err := db.QueryRowContext(ctx, `SELECT revision FROM weave_signals WHERE namespace_id=? AND id=?`, ns, pending).Scan(&revision); err != nil || revision != 2 { + t.Fatal("migration replay modified history", err) + } + t.Log("real v19 upgrade preserves 764-byte legacy correlation, exact payload and consumed evidence; isolates unverifiable pending signals; backfills current SIGNAL at original deadline; zero foreign keys") +} diff --git a/internal/storage/mysql/workflow_signals.go b/internal/storage/mysql/workflow_signals.go new file mode 100644 index 0000000..9c52a62 --- /dev/null +++ b/internal/storage/mysql/workflow_signals.go @@ -0,0 +1,207 @@ +package mysql + +import ( + "context" + "crypto/sha256" + "database/sql" + "encoding/hex" + "encoding/json" + "errors" + "time" + + "git.yuntuo.hk/k2safe/orbit/internal/fault" + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/platform/auth" + "git.yuntuo.hk/k2safe/orbit/weave/definition" + "git.yuntuo.hk/k2safe/orbit/weave/engine" + weave "git.yuntuo.hk/k2safe/orbit/weave/model" + "git.yuntuo.hk/k2safe/orbit/weave/signal" +) + +type WorkflowSignalStore struct{ *TaskStore } + +func NewWorkflowSignalStore(db *sql.DB) *WorkflowSignalStore { + return &WorkflowSignalStore{NewTaskStore(db)} +} + +var _ signal.Store = (*WorkflowSignalStore)(nil) + +func signalContract(node definition.Step) *definition.Signal { + if node.Signal != nil { + return node.Signal + } + if node.Poll != nil { + return node.Poll.FinalSignal + } + return nil +} + +const signalColumns = `id,namespace_id,workflow_id,revision,type,COALESCE(payload_version,''),target_step_key,COALESCE(target_activation_sequence,0),correlation_key,payload,actor_id,COALESCE(evidence_ref,''),state,COALESCE(disposition,''),accepted_at,expires_at,consumed_wait_id` + +func readWorkflowSignal(ctx context.Context, tx *sql.Tx, ns, workflow, id core.ID) (signal.View, error) { + var v signal.View + var payload []byte + err := tx.QueryRowContext(ctx, `SELECT `+signalColumns+` FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, ns, workflow, id).Scan(&v.ID, &v.NamespaceID, &v.WorkflowID, &v.Revision, &v.Name, &v.PayloadVersion, &v.TargetStepKey, &v.TargetActivationSequence, &v.CorrelationKey, &payload, &v.ActorID, &v.EvidenceRef, &v.State, &v.Disposition, &v.AcceptedAt, &v.ExpiresAt, &v.ConsumedWaitID) + v.Payload = payload + if errors.Is(err, sql.ErrNoRows) { + err = fault.NotFound + } + return v, err +} + +func (s *WorkflowSignalStore) Send(ctx context.Context, t auth.Token, workflow core.ID, key string, in signal.Input) (signal.Result, error) { + var out signal.Result + err := s.transaction(ctx, t, signal.SendAction, func(tx *sql.Tx, p auth.Permissions) error { + if _, err := lockApplicationResources(ctx, tx, t.NamespaceID, false); err != nil { + return err + } + id, err := core.NewID() + if err != nil { + return err + } + identity, cached, err := reserveIdentity(ctx, tx, t.NamespaceID, "workflow.signal", string(workflow), key, "workflow_signal", id, in) + if err != nil { + return err + } + g, _, err := lockWorkflowGraph(ctx, tx, t.NamespaceID, workflow, false) + if err != nil { + return err + } + for _, queue := range g.version.Analysis.Queues { + if !p.AllowsQueue(queue) { + return fault.NotFound + } + } + node, ok := g.program.Step(in.TargetStepKey) + contract := signalContract(node) + if !ok || contract == nil || contract.Name != in.Name || contract.PayloadVersion != in.PayloadVersion || definition.ValidateData(contract.PayloadType, in.Payload) != nil { + return fault.Invalid("signal does not match fixed workflow contract") + } + if node.Poll != nil { + outcome, err := g.program.ClassifyPoll(in.TargetStepKey, in.Payload) + if err != nil || outcome == engine.PollPending { + return fault.Invalid("final signal requires a terminal business result") + } + } + if cached != nil { + if err := json.Unmarshal(cached, &out); err != nil { + return err + } + current, err := readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, out.Signal.ID) + if err != nil { + return err + } + if out.Signal.NamespaceID != t.NamespaceID || out.Signal.WorkflowID != workflow || !current.AcceptedAt.Equal(out.Signal.AcceptedAt) { + return errors.New("workflow signal receipt mismatch") + } + out.Deduplicated = true + return nil + } + var count int + if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM (SELECT id FROM weave_signals WHERE namespace_id=? AND workflow_id=? LIMIT ?) admitted_signals`, t.NamespaceID, workflow, signal.MaxPerWorkflow).Scan(&count); err != nil { + return err + } + if count >= signal.MaxPerWorkflow { + return fault.New("WORKFLOW_SIGNAL_LIMIT", "workflow signal limit reached") + } + row := g.steps[in.TargetStepKey] + sequence := uint32(1) + if row.view.ActivationID != nil { + if err := tx.QueryRowContext(ctx, `SELECT sequence FROM weave_activations WHERE namespace_id=? AND workflow_id=? AND step_id=? AND id=?`, t.NamespaceID, workflow, row.view.ID, row.view.ActivationID).Scan(&sequence); err != nil { + return err + } + } + if in.TargetActivationSequence > sequence { + return fault.Conflict + } + now, err := dbNow(ctx, tx) + if err != nil { + return err + } + state, disposition := weave.SignalState("PENDING"), "" + switch { + case in.TargetActivationSequence < sequence: + state, disposition = "REJECTED", "STALE_ACTIVATION" + case g.view.State.Terminal(): + state, disposition = "REJECTED", "WORKFLOW_SETTLED" + case !now.Before(g.view.DeadlineAt): + state, disposition = "EXPIRED", "WORKFLOW_DEADLINE" + case row.view.State == weave.StepSkipped || row.view.State == weave.StepSucceeded || row.view.State == weave.StepFailed || row.view.State == weave.StepCancelled || row.view.State == weave.StepNeedsReview: + state, disposition = "REJECTED", "STEP_SETTLED" + case row.view.State == weave.StepBlocked && g.view.CancelRequestedAt != nil: + state, disposition = "REJECTED", "WORKFLOW_CANCELLING" + } + if disposition == "" && row.view.ActivationID != nil { + var name, version *string + var correlation, waitState string + var deadline time.Time + if err := tx.QueryRowContext(ctx, `SELECT signal_name,payload_version,correlation_key,state,deadline_at FROM weave_waits WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, t.NamespaceID, workflow, row.view.ActivationID).Scan(&name, &version, &correlation, &waitState, &deadline); err != nil { + return err + } + if name == nil || version == nil || *name != in.Name || *version != in.PayloadVersion { + return errors.New("workflow signal wait contract invalid") + } + switch { + case waitState != "WAITING": + state, disposition = "REJECTED", "WAIT_SETTLED" + case correlation != in.CorrelationKey: + state, disposition = "REJECTED", "CORRELATION_MISMATCH" + case !now.Before(deadline): + state, disposition = "EXPIRED", "SIGNAL_DEADLINE" + } + } + digest, err := requestDigest(in) + if err != nil { + return err + } + keyDigest := core.Digest(sha256.Sum256([]byte(key))) + _, err = tx.ExecContext(ctx, `INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,payload_version,target_step_key,target_activation_sequence,correlation_key,payload,actor_id,evidence_ref,state,disposition,accepted_at,expires_at,available_at,created_at,updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`, t.NamespaceID, id, workflow, keyDigest, digest, in.Name, in.PayloadVersion, in.TargetStepKey, in.TargetActivationSequence, []byte(in.CorrelationKey), string(in.Payload), string(t.CredentialID), businessKeyValue(in.EvidenceRef), state, businessKeyValue(disposition), now, g.view.DeadlineAt, g.view.DeadlineAt, now, now) + if err != nil { + return err + } + if state == "PENDING" && row.view.ActivationID != nil { + if err := saveWorkflowClock(ctx, tx, g, row, string(row.view.Kind), row.view.Revision, now); err != nil { + return err + } + } + res, err := tx.ExecContext(ctx, `UPDATE weave_instances SET revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, t.NamespaceID, workflow, g.view.Revision) + if err := exactlyOne(res, err); err != nil { + return err + } + g.view.Revision++ + meta := map[string]any{"signal_id": id, "workflow_id": workflow, "target_step_key": in.TargetStepKey, "target_activation_sequence": in.TargetActivationSequence, "state": state, "disposition": disposition, "request_digest": hex.EncodeToString(digest[:]), "accepted_at": now, "expires_at": g.view.DeadlineAt} + if err := emitEvent(ctx, tx, t.NamespaceID, workflow, g.view.Revision, "workflow", "workflow.signal.accepted", meta); err != nil { + return err + } + if _, err := appendAuditWithEvidence(ctx, tx, t, identity, id, "workflow_signal", "workflow.signal", in.Reason, in.EvidenceRef, nil, meta); err != nil { + return err + } + out.Signal, err = readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, id) + if err != nil { + return err + } + return saveIdentity(ctx, tx, t.NamespaceID, identity, out) + }) + return out, err +} +func (s *WorkflowSignalStore) Get(ctx context.Context, t auth.Token, workflow, id core.ID) (signal.View, error) { + var out signal.View + err := s.transaction(ctx, t, signal.ReadAction, func(tx *sql.Tx, p auth.Permissions) error { + v, _, err := readWorkflowInstance(ctx, tx, t.NamespaceID, workflow, false) + if err != nil { + return err + } + version, err := instanceVersion(ctx, tx, v) + if err != nil { + return err + } + for _, queue := range version.Analysis.Queues { + if !p.AllowsQueue(queue) { + return fault.NotFound + } + } + out, err = readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, id) + return err + }) + return out, err +} diff --git a/internal/storage/mysql/workflow_signals_integration_test.go b/internal/storage/mysql/workflow_signals_integration_test.go new file mode 100644 index 0000000..4439967 --- /dev/null +++ b/internal/storage/mysql/workflow_signals_integration_test.go @@ -0,0 +1,161 @@ +package mysql + +import ( + "context" + "database/sql" + "encoding/json" + "os" + "sync" + "testing" + "time" + + "git.yuntuo.hk/k2safe/orbit/internal/fault" + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/weave/definition" + "git.yuntuo.hk/k2safe/orbit/weave/instance" + "git.yuntuo.hk/k2safe/orbit/weave/signal" +) + +func signalSpec(queue core.ID, early bool) definition.Specification { + spec := runtimeSpec(queue, 1) + node := definition.Step{Key: "signal", Kind: "SIGNAL", Signal: &definition.Signal{Name: "external.done", PayloadVersion: "v1", Correlation: definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "workflow", Path: []string{"business_key"}}}, PayloadType: spec.InputType, TimeoutMS: 30000}} + if early { + spec.Body.Steps = append(spec.Body.Steps, node) + spec.Body.Output = definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "signal"}} + } else { + spec.Body.Steps = append([]definition.Step{node}, spec.Body.Steps...) + } + return spec +} +func grantWorkflowSignal(f *executionFixture) { + f.exec(`UPDATE orbit_credentials SET permissions=JSON_ARRAY_APPEND(permissions,'$.actions','workflows:signal') WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, f.resources.CredentialID) +} +func signalInput(v instance.View) signal.Input { + return signal.Input{Name: "external.done", PayloadVersion: "v1", TargetStepKey: "signal", TargetActivationSequence: 1, CorrelationKey: v.BusinessKey, Payload: json.RawMessage(`{"n":9007199254740993}`), Reason: "verified external completion", EvidenceRef: "proof:event-1"} +} +func signalStart(t *testing.T, ctx context.Context, db *sql.DB, early bool) (*executionFixture, instance.View, *signal.Service) { + t.Helper() + f := newExecutionFixture(t, ctx, db) + v := runtimeStart(t, f, signalSpec(f.resources.QueueID, early)) + grantWorkflowSignal(f) + return f, v, signal.New(NewWorkflowSignalStore(db)) +} +func TestIntegrationWorkflowSignalAdmission(t *testing.T) { + dsn := os.Getenv("ORBIT_TEST_MYSQL_DSN") + if dsn == "" { + t.Skip("requires disposable MySQL") + } + ctx, stop := context.WithTimeout(context.Background(), 90*time.Second) + defer stop() + db, err := Open(ctx, dsn, 32, 8) + if err != nil { + t.Fatal(err) + } + defer db.Close() + if err := Migrate(ctx, db); err != nil { + t.Fatal(err) + } + t.Run("concurrent_identity_exact_input_and_frozen_receipt", func(t *testing.T) { + f, v, api := signalStart(t, ctx, db, false) + in := signalInput(v) + key := string(executionID(t)) + var wg sync.WaitGroup + results := make(chan signal.Result, 12) + errs := make(chan error, 12) + for i := 0; i < 12; i++ { + wg.Add(1) + go func() { + defer wg.Done() + out, err := api.Send(ctx, f.token, v.ID, key, in) + results <- out + errs <- err + }() + } + wg.Wait() + close(results) + close(errs) + for err := range errs { + if err != nil { + t.Fatal(err) + } + } + var id core.ID + fresh := 0 + for out := range results { + if id == "" { + id = out.Signal.ID + } + if out.Signal.ID != id || out.Signal.State != "PENDING" || out.Signal.CorrelationKey != in.CorrelationKey || string(out.Signal.Payload) != string(in.Payload) { + t.Fatal("signal replay changed identity") + } + if !out.Deduplicated { + fresh++ + } + } + if fresh != 1 || f.count("weave_signals", "1=1") != 1 || f.count("orbit_audit_events", "action='workflow.signal'") != 1 { + t.Fatal("duplicate signal admission") + } + changed := in + changed.Payload = json.RawMessage(`{"n":2}`) + if _, err := api.Send(ctx, f.token, v.ID, key, changed); fault.Code(err) != "IDEMPOTENCY_CONFLICT" { + t.Fatal("same key changed content", err) + } + current, err := api.Get(ctx, f.token, v.ID, id) + if err != nil || current.ActorID != string(f.resources.CredentialID) || current.EvidenceRef != in.EvidenceRef { + t.Fatal("signal query lost provenance", err) + } + }) + t.Run("early_signal_is_durable_without_premature_activation", func(t *testing.T) { + f, v, api := signalStart(t, ctx, db, true) + out, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), signalInput(v)) + if err != nil || out.Signal.State != "PENDING" { + t.Fatal(err) + } + if f.count("pulse_tasks", "1=1") != 1 || f.count("weave_steps", "kind='SIGNAL' AND state='BLOCKED' AND current_activation_id IS NULL") != 1 { + t.Fatal("early signal activated successor") + } + }) + t.Run("contract_generation_and_scope_are_checked", func(t *testing.T) { + f, v, api := signalStart(t, ctx, db, false) + in := signalInput(v) + for _, change := range []func(*signal.Input){func(in *signal.Input) { in.Name = "wrong" }, func(in *signal.Input) { in.PayloadVersion = "v2" }, func(in *signal.Input) { in.Payload = json.RawMessage(`"bad"`) }, func(in *signal.Input) { in.TargetActivationSequence = 2 }} { + bad := in + change(&bad) + if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), bad); err == nil { + t.Fatal("invalid signal contract or generation admitted") + } + } + if f.count("weave_signals", "1=1") != 0 { + t.Fatal("rejected request was partially stored") + } + wrong := in + wrong.CorrelationKey = "different" + out, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), wrong) + if err != nil || out.Signal.State != "REJECTED" || out.Signal.Disposition != "CORRELATION_MISMATCH" { + t.Fatal("mismatched correlation granted authority", err) + } + grantWorkflowInstance(t, f) + if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), in); fault.Code(err) != "PERMISSION_DENIED" { + t.Fatal("signal action not enforced", err) + } + grantWorkflowSignal(f) + grantWorkflowInstance(t, f, string(executionID(t))) + grantWorkflowSignal(f) + if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), in); fault.Code(err) != "NOT_FOUND" { + t.Fatal("signal queue scope not enforced", err) + } + }) + t.Run("audit_failure_rolls_back_signal_identity_and_clock", func(t *testing.T) { + f, v, api := signalStart(t, ctx, db, false) + key := string(executionID(t)) + f.exec(`CREATE TRIGGER test_signal_admission_failure BEFORE INSERT ON orbit_audit_events FOR EACH ROW SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='injected signal audit failure'`) + _, err := api.Send(ctx, f.token, v.ID, key, signalInput(v)) + f.exec(`DROP TRIGGER test_signal_admission_failure`) + if err == nil || f.count("weave_signals", "1=1") != 0 || f.count("orbit_idempotency_keys", "operation='workflow.signal'") != 0 { + t.Fatal("signal admission partially committed") + } + if out, err := api.Send(ctx, f.token, v.ID, key, signalInput(v)); err != nil || out.Deduplicated { + t.Fatal("original identity did not recover", err) + } + }) +} diff --git a/platform/auth/token.go b/platform/auth/token.go index 291cd21..3f0031b 100644 --- a/platform/auth/token.go +++ b/platform/auth/token.go @@ -58,7 +58,7 @@ type Permissions struct { Queues []string `json:"queues"` } -var supported = map[string]bool{"workflows:start": true, "workflows:read": true, "workflows:definitions:read": true, "workflows:definitions:publish": true, "tasks:review": true, "tasks:retry": true, "tasks:audit:read": true, "events:read": true, "events:manage": true, "events:consume": true, "application:resources:read": true, "application:resources:manage": true, "tasks:submit": true, "tasks:read": true, "tasks:cancel": true, "queues:read": true, "queues:manage": true, "workers:connect": true, "workers:consume": true, "schedules:read": true, "schedules:manage": true} +var supported = map[string]bool{"workflows:signal": true, "workflows:start": true, "workflows:read": true, "workflows:definitions:read": true, "workflows:definitions:publish": true, "tasks:review": true, "tasks:retry": true, "tasks:audit:read": true, "events:read": true, "events:manage": true, "events:consume": true, "application:resources:read": true, "application:resources:manage": true, "tasks:submit": true, "tasks:read": true, "tasks:cancel": true, "queues:read": true, "queues:manage": true, "workers:connect": true, "workers:consume": true, "schedules:read": true, "schedules:manage": true} func (p Permissions) Validate() error { if len(p.Actions) == 0 || len(p.Actions) > 32 || len(p.Queues) == 0 || len(p.Queues) > 128 { diff --git a/weave/model/signal.go b/weave/model/signal.go index c151525..e27be10 100644 --- a/weave/model/signal.go +++ b/weave/model/signal.go @@ -19,20 +19,30 @@ type Signal struct { RequestDigest core.Digest `db:"request_digest"` // Type 事件或信号的稳定类型标识。 Type string `db:"type"` + // PayloadVersion 固定信号契约版本；为空仅表示不能参与新等待匹配的历史记录。 + PayloadVersion *string `db:"payload_version"` // TargetStepKey 信号目标步骤键，用于早到信号的后续匹配。 TargetStepKey string `db:"target_step_key"` - // CorrelationKey 外部业务操作关联键，防止旧信号误用于新工作。 + // TargetActivationSequence 目标激活序号，早到首轮为1；为空仅表示身份未核实的历史信号。 + TargetActivationSequence *uint32 `db:"target_activation_sequence"` + // CorrelationKey 精确UTF8操作关联，新信号最多512字节，历史兼容764字节，保留尾部空格。 CorrelationKey string `db:"correlation_key"` // Payload 版本化输入或事件内容；仅承载通用 JSON，不解释行业规则。 Payload core.JSON `db:"payload"` // ActorID 经认证的操作主体标识，不能直接信任请求自报值。 ActorID string `db:"actor_id"` + // EvidenceRef 可选受控证据引用，最多512个无空白字符；平台不请求该地址。 + EvidenceRef *string `db:"evidence_ref"` // State 当前状态；只允许定义的状态转换并配合事务校验。 State SignalState `db:"state"` + // Disposition 稳定消费/拒绝分类；PENDING时为空，不记录原始错误或业务参数。 + Disposition *string `db:"disposition"` // AcceptedAt 信号可靠受理时间，不能用稍后的消费时间替代。 AcceptedAt time.Time `db:"accepted_at"` // ExpiresAt 过期时间，统一 UTC；到期后的行为由对应资源策略决定。 ExpiresAt time.Time `db:"expires_at"` + // AvailableAt 未匹配信号下次到期检查时间；数据库故障只推迟检查，不延长原受理截止。 + AvailableAt time.Time `db:"available_at"` // ConsumedWaitID 消费该信号的等待记录；尚未消费时为空。 ConsumedWaitID *core.ID `db:"consumed_wait_id"` } diff --git a/weave/model/wait.go b/weave/model/wait.go index de58751..e087f38 100644 --- a/weave/model/wait.go +++ b/weave/model/wait.go @@ -23,6 +23,8 @@ type Wait struct { SignalName *string `db:"signal_name"` // PayloadVersion 激活时冻结的信号内容契约版本；TIMER及待核实历史记录为空。 PayloadVersion *string `db:"payload_version"` + // WinnerSignalID 胜出的外部信号；时间或查询结果胜出时为空，须核对双向归属。 + WinnerSignalID *core.ID `db:"winner_signal_id"` // CorrelationKey 精确UTF8业务关联且保留尾部空格；新受理最多512字节，历史兼容764字节，TIMER为空串。 CorrelationKey string `db:"correlation_key"` // DeadlineAt 等待截止时间，用可靠受理时间判断信号是否按时到达。 diff --git a/weave/signal/service.go b/weave/signal/service.go new file mode 100644 index 0000000..4c9fc40 --- /dev/null +++ b/weave/signal/service.go @@ -0,0 +1,94 @@ +// Package signal 定义可靠外部信号的受理与查询；受理成功不等于已经推进流程。 +package signal + +import ( + "context" + "encoding/json" + "strings" + "time" + "unicode" + "unicode/utf8" + + "git.yuntuo.hk/k2safe/orbit/internal/fault" + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/platform/auth" + "git.yuntuo.hk/k2safe/orbit/pulse/service" + "git.yuntuo.hk/k2safe/orbit/weave/definition" + weave "git.yuntuo.hk/k2safe/orbit/weave/model" +) + +const SendAction = "workflows:signal" +const ReadAction = "workflows:read" +const MaxPerWorkflow = 1024 // MaxPerWorkflow 限制实例全部受理信号数，终结/拒绝记录也保留占用，原键恢复不受上限影响。 + +type Input struct { + Name string `json:"name"` // Name 固定定义信号名称，最多128个ASCII字符。 + PayloadVersion string `json:"payload_version"` // PayloadVersion 固定内容契约版本，最多32个ASCII字符。 + TargetStepKey string `json:"target_step_key"` // TargetStepKey 固定定义的目标步骤，最多64个ASCII字符。 + TargetActivationSequence uint32 `json:"target_activation_sequence"` // TargetActivationSequence 显式目标激活代次，首次及早到信号为1，不能自动跟随人工恢复。 + CorrelationKey string `json:"correlation_key"` // CorrelationKey 1至512字节精确UTF8操作关联，保留尾部空格。 + Payload json.RawMessage `json:"payload" nullable:"true"` // Payload 固定契约内容，最多256KiB，显式null与缺失不同。 + Reason string `json:"reason"` // Reason 受理原因，1至1024个Unicode字符，进入审计。 + EvidenceRef string `json:"evidence_ref,omitempty"` // EvidenceRef 可选受控证据引用，最多512个无空白/控制字符，不携带秘密。 +} + +type View struct { + ID core.ID `json:"id"` // ID 信号持久身份。 + NamespaceID core.ID `json:"namespace_id"` // NamespaceID 授权和数据隔离范围。 + WorkflowID core.ID `json:"workflow_id"` // WorkflowID 固定目标实例。 + Revision uint64 `json:"revision,string"` // Revision 当前信号记录版本。 + Name string `json:"name"` // Name 固定信号类型。 + PayloadVersion string `json:"payload_version"` // PayloadVersion 固定契约版本。 + TargetStepKey string `json:"target_step_key"` // TargetStepKey 目标步骤名称。 + TargetActivationSequence uint32 `json:"target_activation_sequence"` // TargetActivationSequence 固定激活序号。 + CorrelationKey string `json:"correlation_key"` // CorrelationKey 原始操作关联。 + Payload json.RawMessage `json:"payload"` // Payload 原始精确内容。 + ActorID string `json:"actor_id"` // ActorID 平台认证主体，不接受请求自报身份。 + EvidenceRef string `json:"evidence_ref,omitempty"` // EvidenceRef 原证据引用。 + State weave.SignalState `json:"state"` // State PENDING/CONSUMED/EXPIRED/REJECTED。 + Disposition string `json:"disposition,omitempty"` // Disposition 稳定消费或拒绝原因。 + AcceptedAt time.Time `json:"accepted_at"` // AcceptedAt 数据库可靠受理时间，截止不含等号。 + ExpiresAt time.Time `json:"expires_at"` // ExpiresAt 固定实例总截止，不因重报延长。 + ConsumedWaitID *core.ID `json:"consumed_wait_id,omitempty"` // ConsumedWaitID 消费它的唯一等待身份。 +} +type Result struct { + Signal View `json:"signal"` // Signal 首次受理的冻结视图，当前状态通过Get查询。 + Deduplicated bool `json:"deduplicated"` // Deduplicated 原身份恢复，不重报新事件或重新计时。 +} +type Store interface { + Send(context.Context, auth.Token, core.ID, string, Input) (Result, error) + Get(context.Context, auth.Token, core.ID, core.ID) (View, error) +} +type Service struct{ store Store } + +func New(store Store) *Service { return &Service{store: store} } +func (s *Service) Send(ctx context.Context, t auth.Token, workflow core.ID, key string, in Input) (Result, error) { + if err := service.ValidateKey(key); err != nil { + return Result{}, err + } + if workflow.Validate() != nil || !validName(in.Name, 128) || !validName(in.PayloadVersion, 32) || !validName(in.TargetStepKey, 64) || in.TargetActivationSequence == 0 { + return Result{}, fault.Invalid("valid signal name, contract, step and activation sequence required") + } + if !utf8.ValidString(in.CorrelationKey) || len(in.CorrelationKey) == 0 || len(in.CorrelationKey) > 512 || strings.ContainsRune(in.CorrelationKey, 0) { + return Result{}, fault.Invalid("signal correlation must be 1 to 512 UTF-8 bytes") + } + if err := definition.ValidateData(definition.Type{Kind: "JSON"}, in.Payload); err != nil { + return Result{}, fault.Invalid("valid signal payload required, maximum 256 KiB") + } + if !utf8.ValidString(in.Reason) || strings.TrimSpace(in.Reason) == "" || utf8.RuneCountInString(in.Reason) > 1024 || strings.ContainsRune(in.Reason, 0) { + return Result{}, fault.Invalid("signal reason required, maximum 1024 characters") + } + if !utf8.ValidString(in.EvidenceRef) || utf8.RuneCountInString(in.EvidenceRef) > 512 || strings.IndexFunc(in.EvidenceRef, func(r rune) bool { return unicode.IsSpace(r) || unicode.IsControl(r) }) >= 0 { + return Result{}, fault.Invalid("invalid signal evidence reference") + } + return s.store.Send(ctx, t, workflow, key, in) +} +func validName(value string, max int) bool { + return service.ValidName(value, max) && !strings.ContainsRune(value, ':') +} +func (s *Service) Get(ctx context.Context, t auth.Token, workflow, id core.ID) (View, error) { + if workflow.Validate() != nil || id.Validate() != nil { + return View{}, fault.Invalid("invalid workflow or signal id") + } + return s.store.Get(ctx, t, workflow, id) +} diff --git a/weave/signal/service_test.go b/weave/signal/service_test.go new file mode 100644 index 0000000..8661ed0 --- /dev/null +++ b/weave/signal/service_test.go @@ -0,0 +1,55 @@ +package signal + +import ( + "context" + "encoding/json" + "strings" + "testing" + + core "git.yuntuo.hk/k2safe/orbit/model" + "git.yuntuo.hk/k2safe/orbit/platform/auth" +) + +type captureStore struct { + calls int + input Input +} + +func (s *captureStore) Send(_ context.Context, _ auth.Token, _ core.ID, _ string, in Input) (Result, error) { + s.calls++ + s.input = in + return Result{}, nil +} +func (s *captureStore) Get(context.Context, auth.Token, core.ID, core.ID) (View, error) { + s.calls++ + return View{}, nil +} +func TestSignalInputBoundary(t *testing.T) { + store := &captureStore{} + api := New(store) + id := core.ID("11111111111111111111111111111111") + base := Input{Name: "completed", PayloadVersion: "v1", TargetStepKey: "wait", TargetActivationSequence: 1, CorrelationKey: "关联 ", Payload: json.RawMessage(`{"n":9007199254740993}`), Reason: "verified source", EvidenceRef: "proof:record-1"} + for _, raw := range []string{`{"n":9007199254740993}`, `null`} { + in := base + in.Payload = json.RawMessage(raw) + if _, err := api.Send(context.Background(), auth.Token{}, id, "original", in); err != nil || string(store.input.Payload) != raw || store.input.CorrelationKey != base.CorrelationKey { + t.Fatal("signal input changed", err) + } + } + for _, change := range []func(*Input){ + func(in *Input) { in.TargetActivationSequence = 0 }, func(in *Input) { in.CorrelationKey = "" }, func(in *Input) { in.CorrelationKey = strings.Repeat("中", 171) }, func(in *Input) { in.CorrelationKey = "x\x00y" }, + func(in *Input) { in.Name = "a:b" }, func(in *Input) { in.Name = strings.Repeat("n", 129) }, func(in *Input) { in.PayloadVersion = "" }, func(in *Input) { in.TargetStepKey = "$root" }, + func(in *Input) { in.Payload = nil }, func(in *Input) { in.Payload = json.RawMessage(`{"n":1,"n":2}`) }, func(in *Input) { in.Payload = json.RawMessage(`"\ud800"`) }, + func(in *Input) { in.Reason = " " }, func(in *Input) { in.Reason = strings.Repeat("中", 1025) }, func(in *Input) { in.EvidenceRef = "proof:secret\nvalue" }, func(in *Input) { in.EvidenceRef = strings.Repeat("x", 513) }, + } { + in := base + change(&in) + before := store.calls + if _, err := api.Send(context.Background(), auth.Token{}, id, "original", in); err == nil || before != store.calls { + t.Fatal("invalid signal reached store") + } + } + if _, err := api.Send(context.Background(), auth.Token{}, id, "", base); err == nil { + t.Fatal("missing identity accepted") + } +}