feat: persist workflow signals and arbitrate poll completion
This commit is contained in:
@@ -31,16 +31,19 @@
|
||||
|
||||
- P4定义发布使用schema_version=1有限结构，词法作用域禁止跨分支/前向读取，任务策略显式冻结；SDK类型独立。发布按认证→应用额度→操作身份→定义→排序队列持锁；原键与同版本去重仍校验当前权限。定义/版本/revision/审计/回执同事务；同版本不同内容不得覆盖。保留精确数字，校验存储原文与结构摘要；不得把发布通过或DSL支持误称为实例执行已实现。
|
||||
|
||||
- P4定义子项已通过真实MySQL、HTTP丢回执、独立SDK和全量回归（docs/reports/phase-4-definitions.md）。实例启动/固定版本/首批任务同事务的内部存储已补齐；内部事件消费、顺序与组合推进及根接续已接入；TIMER与后续POLL时钟已接入，继续SIGNAL及实例公共SDK；P4整体和P5–P7仍未完成。
|
||||
- P4定义子项已通过真实MySQL、HTTP丢回执、独立SDK和全量回归（docs/reports/phase-4-definitions.md）。实例启动/固定版本/首批任务同事务的内部存储已补齐；内部事件消费、顺序与组合推进及根接续已接入；TIMER与后续POLL时钟已接入，SIGNAL及最终查询竞争已接入，继续完整控制及实例公共SDK；P4整体和P5–P7仍未完成。
|
||||
|
||||
- 内部结果交付使用workflow_activation所有者并固定激活ID，TASK经activation.task_id、POLL经轮次校验归属；weave_inbox来源及独立快照与Pulse结果同事务，先于外部结果流行锁，不在结果路径反锁实例/步骤/订阅。迁移000017隔离缺快照历史，禁止用当前任务重建旧事件。
|
||||
- weave/engine仅计算有限决策；实例受理、Inbox消费与根接续已接入，完整等待/控制运行器继续开发。Plan最多32个变更，More必须持久化待续；耗尽变更预算仍须遍历其他已活动分支，不能误判在途工作不可达。冻结分支选择，精确数值，不以PENDING查询结果完成POLL。后续消费确认/当前激活校验/新任务或检查点必须同事务，不能把纯快照往返测试当作真实进程恢复。
|
||||
|
||||
- 实例内部受理已实现，公共实例HTTP/SDK及信号运行器尚未开放；结果消费和根待续扫描已接入后台。Start按认证→应用额度→原身份→排序队列建立门控/固定版本/全部步骤/首批工作，所有失败同事务回滚。首批推进递增revision至2，原键返回冻结回执且仍核验当前权限，不重算截止。
|
||||
- 实例内部受理已实现，信号内部运行器已接入，公共实例/信号HTTP及独立SDK尚未开放；结果消费和根待续扫描已接入后台。Start按认证→应用额度→原身份→排序队列建立门控/固定版本/全部步骤/首批工作，所有失败同事务回滚。首批推进递增revision至2，原键返回冻结回执且仍核验当前权限，不重算截止。
|
||||
- 实例根步骤`$root`/JOIN不属于用户定义，Steps查询排除它；More对应根步骤版本化持久检查点。TASK直接关联激活task_id，POLL通过轮次关联，实际任务截止必须受实例/查询绝对截止限制。Wait名称及契约版本冻结，旧NULL等待须核实，不能盲目解释；新关联512字节、存储兼容历史764字节。
|
||||
|
||||
- 结果消费按身份→应用额度→排序队列→实例→门控→步骤/Inbox持锁，禁止先锁Inbox再锁父资源。来源使用独立冻结快照，旧激活/低版本留痕；数据库故障不归类为业务失败。消费确认、步骤、后继或根待续、事件及审计同事务。
|
||||
- 后继满额用SAVEPOINT回滚整轮新工作，保留已接受结果和待续；恢复必须重读持久图。多次根推进使用独立系统审计操作身份；迟到证据也递增实例历史版本，禁止复用已有workflow.advanced唯一身份。
|
||||
- 本轮消费运行器和真实Orbit强杀/双节点接管已验证，见docs/reports/phase-4-runtime.md。TIMER到期与POLL后续轮次已接入，见docs/13-workflow-clock.md；SIGNAL及最终信号竞争、FAIL_AFTER_SETTLED分支结清、控制API和实例SDK仍须完成，不得按部分自动运行宣称P4/P5完成。
|
||||
- 本轮消费运行器和真实Orbit强杀/双节点接管已验证，见docs/reports/phase-4-runtime.md。TIMER到期与POLL后续轮次已接入，见docs/13-workflow-clock.md；SIGNAL及最终信号竞争已接入（docs/14-workflow-signals.md）；FAIL_AFTER_SETTLED分支结清、控制API和实例SDK仍须完成，不得按部分自动运行宣称P4/P5完成。
|
||||
|
||||
- TIMER/POLL复用weave_pending_advances及到期索引，时钟固定激活ID；迁移000019只回填原等待/查询时钟，不改截止和预算。后续POLL只由已确认POLL_PENDING开启，首轮精确输入/资源键冻结，每轮Pulse尝试独立计数；满额/暂停不消耗轮数。截止遇到已落库结果先等待Inbox，仍在途则关闭门控但保留ACTIVE和当前任务，许可由Pulse结清；不得由时钟假定外部请求已结束。
|
||||
|
||||
- SIGNAL受理要求显式workflows:signal、固定目标步骤/激活代次/内容契约/精确关联及原因；每实例全部历史信号上限1024，原键先返回冻结回执。accepted_at/expires_at及摘要与不可变审计核验；available_at只用于到期检查退避。
|
||||
- 最终信号与当前查询按原受理/事件时间裁决，PENDING不争夺终结权；保存信号赢家后复用Pulse取消规则，在途任务保留许可直至真实结果。未知/冲突结果转核查，已有核查不得自动清除。迁移000020隔离旧PENDING身份并保留历史原文，不引入外键。
|
||||
|
||||
@@ -10,7 +10,7 @@ Orbit 是面向多个业务项目的通用任务调度与流程编排平台，
|
||||
|
||||
已完成 P0–P3 平台功能阶段验收：35 个数据模型、版本化迁移、凭据及范围校验、队列与任务 API、WSS 执行、独立 SDK、自动 Worker 和 [执行尝试/证据查询](docs/api/execution-history.md)。[Worker Schema](docs/api/worker-schema.md) 与真实进程强杀恢复回归已通过；[安全重试](docs/api/retries.md)、[定时计划](docs/api/schedules.md)、[运行中取消](docs/api/cancellation.md) 、[队列资源控制](docs/api/queues.md) 、[应用共享配额](docs/api/application-resources.md) 、[资源键互斥](docs/api/keyed-concurrency.md) 和 [优先级/节点公平派发](docs/api/priority-and-fairness.md) 已接入服务端和 SDK，[可靠结果订阅](docs/api/subscriptions.md) 已实现持久化核心、HTTP/WSS、管理 SDK 与示例，[消费 SDK](docs/sdk/event-consumer.md) 和 [独立 MySQL Inbox 示例](docs/examples/result-inbox.md) 已实现；[批量任务受理](docs/api/batch-tasks.md) 已提供逐项幂等、失败分类与 SDK 示例；[人工核查与重试](docs/api/task-recovery.md) 已提供带证据的管理接口、关联新任务、审计查询及运行示例；[业务键与时间范围查询](docs/api/task-queries.md) 已提供过滤、分页与SDK场景；[任务来源查询](docs/api/task-source.md) 已提供人工重试父任务、计划触发记录与固定版本；[业务事务Outbox](docs/examples/task-outbox.md) 已提供独立业务库、稳定提交身份、发送恢复与CLI；[相对延迟执行](docs/examples/delayed-task.md)已提供数据库计时及原请求恢复；完整范围核对见[P3验收](docs/reports/phase-3.md)，流程引擎与生产验证按 [持续开发计划](docs/08-development-plan.md) 继续实现。
|
||||
|
||||
P4已实现[流程定义与不可变版本发布](docs/api/workflow-definitions.md)、[结构版本1](docs/api/workflow-definition-format.md)及[SDK发布例子](docs/examples/workflow-definition.md)。[有限步骤决策器与内部结果交付](docs/10-workflow-engine-core.md)已实现；已补[实例受理及首批工作存储](docs/11-workflow-admission.md)。[结果消费、顺序/分支/并行推进及根检查点接续](docs/12-workflow-runtime.md)已接入后台运行器。[定时等待、后续查询轮次及原预算恢复](docs/13-workflow-clock.md)已接入；信号、失败结清及实例公共API/SDK继续开发，发布定义不会启动任务。
|
||||
P4已实现[流程定义与不可变版本发布](docs/api/workflow-definitions.md)、[结构版本1](docs/api/workflow-definition-format.md)及[SDK发布例子](docs/examples/workflow-definition.md)。[有限步骤决策器与内部结果交付](docs/10-workflow-engine-core.md)已实现；已补[实例受理及首批工作存储](docs/11-workflow-admission.md)。[结果消费、顺序/分支/并行推进及根检查点接续](docs/12-workflow-runtime.md)已接入后台运行器。[定时等待、后续查询轮次及原预算恢复](docs/13-workflow-clock.md)已接入；[外部信号和查询最终信号竞争](docs/14-workflow-signals.md)已接入内部运行器；失败结清及实例公共API/SDK继续开发，发布定义不会启动任务。
|
||||
|
||||
数据库禁止外键，关联一致性通过服务端事务校验，索引、唯一约束和 CHECK 保留；readiness 同时检查迁移及零外键要求。这不表示完整平台已具备生产服务能力。验证证据见 [P0](docs/reports/phase-0.md)、[P1](docs/reports/phase-1.md)、[P2 阶段验收](docs/reports/phase-2.md)、[安全重试](docs/reports/phase-3-safe-retry.md)、[计划调度](docs/reports/phase-3-schedules.md)、[取消与无外键回归](docs/reports/phase-3-cancellation.md) 及 [队列资源回归](docs/reports/phase-3-queue-resources.md)。
|
||||
|
||||
@@ -33,6 +33,7 @@ weave/model/ 定义、实例、步骤、信号和查询轮次
|
||||
weave/definition/ 有限结构/类型校验与定义发布契约
|
||||
weave/engine/ 不可变规则的有限步骤决策与精确结果映射
|
||||
weave/instance/ 内部实例受理/查询契约，公共运行API继续开发
|
||||
weave/signal/ 内部可靠信号受理/查询契约及显式激活身份
|
||||
internal/config/ 配置校验
|
||||
internal/storage/mysql/ 连接、事务、迁移及集成测试
|
||||
internal/transport/httpserver/ HTTP 接入
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Weave 流程编排功能与设计
|
||||
|
||||
> 版本 0.4 · 2026-09-18。首版功能范围已确认。定义结构、发布校验与不可变版本已实现，见[定义格式](api/workflow-definition-format.md)和[发布API](api/workflow-definitions.md)；[实例受理](11-workflow-admission.md)、[结果消费与根推进](12-workflow-runtime.md)、[TIMER与POLL时钟](13-workflow-clock.md)已接入。信号、完整管理及实例公共API/SDK仍待P4–P5实现验收。
|
||||
> 版本 0.5 · 2026-09-18。首版功能范围已确认。定义结构、发布校验与不可变版本已实现，见[定义格式](api/workflow-definition-format.md)和[发布API](api/workflow-definitions.md)；[实例受理](11-workflow-admission.md)、[结果消费与根推进](12-workflow-runtime.md)、[TIMER与POLL时钟](13-workflow-clock.md)已接入。[信号等待及最终查询竞争](14-workflow-signals.md)已接入内部运行器；完整管理及实例公共API/SDK仍待P4–P5实现验收。
|
||||
|
||||
## 1. 职责与边界
|
||||
|
||||
|
||||
@@ -75,7 +75,7 @@
|
||||
| weave | `InboxEvent` | `weave_inbox` |
|
||||
| weave | `PendingAdvance` | `weave_pending_advances` |
|
||||
|
||||
`orbit_events` 保存原始状态事件，`orbit_result_events` 使用相同事件 ID 保存队列、提交序号和冻结快照，`orbit_event_streams` 串行分配命名空间内的结果序号；对外投递使用 `orbit_deliveries`，内部流程结果原子写入 `weave_inbox`及其独立冻结快照，已接入[实例消费推进](12-workflow-runtime.md)。`weave_pending_advances` 保存根步骤续接和TIMER/POLL时钟检查点。
|
||||
`orbit_events` 保存原始状态事件，`orbit_result_events` 使用相同事件 ID 保存队列、提交序号和冻结快照，`orbit_event_streams` 串行分配命名空间内的结果序号；对外投递使用 `orbit_deliveries`，内部流程结果原子写入 `weave_inbox`及其独立冻结快照，已接入[实例消费推进](12-workflow-runtime.md)。`weave_pending_advances` 保存根步骤续接和TIMER/SIGNAL/POLL时钟检查点。
|
||||
|
||||
## 4. 已落实的数据库约束
|
||||
|
||||
@@ -199,4 +199,8 @@ Wait新增SignalName和PayloadVersion两个中文注释字段；correlation_key
|
||||
|
||||
## 等待时钟回填（000019）
|
||||
|
||||
复用PendingAdvance保存ROOT、TIMER、POLL检查点；新增内部JSON结构字段均有中文注释，不新增持久化列或实体，仍为35个实体/295个直接持久化字段。TIMER/POLL检查点固定激活ID，实际推进核验同命名空间归属、原截止及剩余预算。迁移只回填历史有效等待的唤醒身份和时间，保留已有检查点；无外键、无级联，见[时钟运行约定](13-workflow-clock.md)。
|
||||
复用PendingAdvance保存ROOT、TIMER、POLL检查点；新增内部JSON结构字段均有中文注释，不新增持久化列或实体，截至该迁移为35个实体/295个直接持久化字段。TIMER/POLL检查点固定激活ID，实际推进核验同命名空间归属、原截止及剩余预算。迁移只回填历史有效等待的唤醒身份和时间，保留已有检查点；无外键、无级联，见[时钟运行约定](13-workflow-clock.md)。
|
||||
|
||||
## 信号身份与赢家增量（000020）
|
||||
|
||||
Signal新增payload_version、target_activation_sequence、evidence_ref、disposition、available_at；Wait新增winner_signal_id。当前35个实体、含公共Record的301个直接持久化字段声明，全部中文注释。新信号精确关联最多512字节，存储兼容历史764字节；过期检查退避不改变expires_at。赢家唯一约束只防重复，实例/等待/激活/信号归属及不可变受理证明仍由事务验证，外键保持0。历史不完整身份隔离和升级回填见[信号约定](14-workflow-signals.md)。
|
||||
|
||||
@@ -28,14 +28,22 @@
|
||||
| P1 任务受理闭环 | 应用凭据与作用域权限、队列、幂等提交、查询、待执行取消、受理事件；SDK 客户端 | API/SDK 创建查询闭环；重复键与内容冲突、响应未确认重报、跨命名空间拒绝、参数边界 | 已完成并通过阶段回归（见 P1 报告；真实断路/提交 ACK 丢失注入留后续故障阶段） |
|
||||
| P2 Worker 执行闭环 | WSS 接入、能力/容量、领取与开始、租约、续租、结果确认、排空与恢复；Worker SDK | 真 Worker 执行；失联/重连/重复确认/旧尝试拒绝；业务效果未知转核查；节点故障接管 | 已完成并通过阶段回归：服务端、自动 SDK、历史查询、完整 Schema 与真实进程强杀恢复；测试拓扑和剩余生产故障矩阵见 P2 验收报告 |
|
||||
| P3 Pulse 完整能力 | 安全重试、延时、Cron/固定间隔、漏跑策略、计划版本、运行中取消、并发许可、配额/背压、可靠结果订阅、批量受理、独立任务核查/人工重试、查询筛选与事务Outbox接入 | 多实例领取/触发竞争、重试上限、取消竞争、队列隔离、积压恢复、事件重复交付 | 已完成平台功能阶段验收：[原范围逐项核对](reports/phase-3.md)、[本轮完整回归](reports/phase-3-delay-recovery.md)。补齐相对延迟、改参数关联示例和四种重试全停恢复场景；20个可运行示例全部通过。全平台生产门槛及SDK首版交付剩余项仍按P6–P7推进 |
|
||||
| P4 Weave 主流程 | SDK 定义构建与发布校验、实例、顺序/分支/结构化并行汇合、信号/定时等待、版本绑定 | 重启接续、早到/重复信号、并行竞争、内部事件重放、发布版本兼容；端到端通用流程 | 定义格式/词法类型校验、不可变发布、独立SDK及六类节点定义例子已实现，已通过[子项验收](reports/phase-4-definitions.md)；已补[有限决策器及内部结果交付](reports/phase-4-engine-core.md)；[实例受理及首批事务](reports/phase-4-admission.md)已实现；[结果消费和根接续](reports/phase-4-runtime.md)已接入后台，顺序/分支/并行及真实进程恢复已验证；[TIMER与后续POLL轮次](13-workflow-clock.md)已接入；信号、完整控制和实例公共API/SDK继续开发 |
|
||||
| P5 Weave 恢复与查询 | 有界查询、暂停继续、取消、人工核查恢复、控制门控与审计 | 轮次与尝试独立预算；暂停/开始/回执/超时竞争；恢复保留预算；未知结果不盲目重复 | 有界查询轮次/尝试独立预算及原截止恢复已接入内部运行器；信号竞争、完整控制/人工恢复及公共API/SDK继续开发 |
|
||||
| P4 Weave 主流程 | SDK 定义构建与发布校验、实例、顺序/分支/结构化并行汇合、信号/定时等待、版本绑定 | 重启接续、早到/重复信号、并行竞争、内部事件重放、发布版本兼容；端到端通用流程 | 定义格式/词法类型校验、不可变发布、独立SDK及六类节点定义例子已实现，已通过[子项验收](reports/phase-4-definitions.md)；已补[有限决策器及内部结果交付](reports/phase-4-engine-core.md)；[实例受理及首批事务](reports/phase-4-admission.md)已实现；[结果消费和根接续](reports/phase-4-runtime.md)已接入后台，顺序/分支/并行及真实进程恢复已验证；[TIMER与后续POLL轮次](13-workflow-clock.md)已接入；[信号消费和最终查询竞争](14-workflow-signals.md)已接入内部运行器；完整控制和实例公共API/SDK继续开发 |
|
||||
| P5 Weave 恢复与查询 | 有界查询、暂停继续、取消、人工核查恢复、控制门控与审计 | 轮次与尝试独立预算；暂停/开始/回执/超时竞争；恢复保留预算；未知结果不盲目重复 | 有界查询轮次/尝试独立预算及原截止恢复已接入内部运行器；最终信号竞争及在途查询结清已接入；完整控制/人工恢复及公共API/SDK继续开发 |
|
||||
| P6 生产验证与运维 | 完整监控、凭据轮换、备份恢复、滚动升级、运行手册、可复现负载和故障工具 | 对照生产验收清单执行，报告环境/指标/遗留项；容量、RPO/RTO 按真实拓扑验证 | 待开发 |
|
||||
| P7 首版交付核对 | SDK 兼容与发布准备、完整 API 手册、场景示例、全量回归及交付清单 | 新接入者按文档跑通全部示例；所有首版能力可追溯到实现与测试；报告实际生产边界 | 待开发 |
|
||||
|
||||
SDK、文档与示例随 P1–P5 同步开发，P7 是全量核对，不是最后才开始补文档。
|
||||
|
||||
P4/P5定时等待与后续查询轮次的实现及回归记录见[时钟报告](reports/phase-4-clock.md)。实例公共入口继续与信号/控制能力同步交付，不能将内部存储链路作为完整SDK验收。
|
||||
P4/P5定时等待与后续查询轮次的实现及回归记录见[时钟报告](reports/phase-4-clock.md)。外部信号与最终查询竞争见[信号报告](reports/phase-4-signals.md)。实例公共入口继续与控制能力同步交付，不能将内部存储链路作为完整SDK验收。
|
||||
|
||||
### P4/P5 下一步实施顺序
|
||||
|
||||
1. 完成FAIL_AFTER_SETTLED与显式取消的分支结清：持久取消意图、复用Pulse取消、收集已发出查询/任务结果、等待全部已激活工作结清后才进入失败或取消终态。效果未知仍保留核查。
|
||||
2. 完成实例暂停/继续/取消内部操作，以control_revision、操作幂等身份、明确原因及审计保护控制决定；暂停不延长原截止，继续不覆盖核查或取消状态。
|
||||
3. 完成人工核查恢复与旧激活隔离，保留原预算、结果和证据，不能通过换激活/新键绕过未知效果或执行上限。
|
||||
4. 同步开放实例/步骤/信号/来源/审计HTTP、独立SDK和端到端场景，覆盖原键恢复、权限、版本冲突、竞争与进程全停；公共能力逐项记录实际验收结果。
|
||||
5. 转入P6/P7的生产故障、容量、运维、发布与全部文档核对。内部存储阶段通过不会提前勾选公共接口或生产验收。
|
||||
|
||||
### P3 功能范围核对
|
||||
|
||||
|
||||
@@ -74,7 +74,7 @@ v16及以前的未处理记录缺少冻结来源，升级后统一标为QUARANTI
|
||||
|
||||
## 4. 后续存储适配必须满足的条件
|
||||
|
||||
以下保留存储运行器的完整要求。第1项已按[受理报告](reports/phase-4-admission.md)验证；第2–4项结果消费和根接续已接入[实际运行器](12-workflow-runtime.md)，第5项的TIMER/POLL已接入[时钟运行器](13-workflow-clock.md)，其余信号、完整控制、接口和SDK仍继续实现：
|
||||
以下保留存储运行器的完整要求。第1项已按[受理报告](reports/phase-4-admission.md)验证；第2–4项结果消费和根接续已接入[实际运行器](12-workflow-runtime.md)，第5项的TIMER/POLL已接入[时钟运行器](13-workflow-clock.md)，第5项的信号等待已接入[信号运行器](14-workflow-signals.md)，完整控制、接口和SDK仍继续实现：
|
||||
|
||||
1. 启动时认证、固定不可变版本/输入和策略、校验全部队列、计算实例截止，并在同事务创建实例、门控、首批激活/任务及审计回执。相同身份返回原实例。
|
||||
2. 消费者领取内部事件后核对命名空间、来源任务版本、激活/轮次及当前步骤版本；消费确认、步骤变化、后续任务或持久化待续记录同事务提交。
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Weave 实例受理与首批工作
|
||||
|
||||
状态：内部Go服务与MySQL存储适配已实现实例Start/Get/Steps、固定版本、首批决策落库和可靠结果交付。[后台Inbox消费和根待续扫描](12-workflow-runtime.md)已接通。**尚未开放实例HTTP路由、启动SDK或命令行入口；信号运行器和完整控制仍继续开发。** Weave相关公开SDK目前仅支持定义发布。[TIMER和后续POLL轮次](13-workflow-clock.md)已接入，TASK及组合节点已可自动推进，不能将此解释成全部等待/控制能力已实现。
|
||||
状态：内部Go服务与MySQL存储适配已实现实例Start/Get/Steps、固定版本、首批决策落库和可靠结果交付。[后台Inbox消费和根待续扫描](12-workflow-runtime.md)已接通。**尚未开放实例HTTP路由、启动SDK或命令行入口；完整控制仍继续开发；[信号运行器](14-workflow-signals.md)已接入。** Weave相关公开SDK目前仅支持定义发布。[TIMER和后续POLL轮次](13-workflow-clock.md)已接入，TASK及组合节点已可自动推进，不能将此解释成全部等待/控制能力已实现。
|
||||
|
||||
## 1. 受理事务
|
||||
|
||||
@@ -49,9 +49,9 @@ Get返回当前实例视图；Start重报返回受理时冻结的原始视图，
|
||||
| 活跃实例 | 每个namespace最多10000个RUNNING/WAITING/CANCELLING/NEEDS_REVIEW实例；终态不计入；满额仍可恢复原受理回执 |
|
||||
| 信号关联 | 新受理最多512字节；数据库兼容旧191字符最长764字节，升级不截断历史 |
|
||||
| 截止 | 数据库时间生成；任务、查询与等待均不超过实例总截止 |
|
||||
| 自动运行 | Inbox消费、根待续、TIMER到期及后续POLL轮次已接入；信号消费及完整控制仍待实现 |
|
||||
| 自动运行 | Inbox消费、根待续、TIMER到期及后续POLL轮次已接入；SIGNAL与最终查询竞争已接入，完整控制仍待实现 |
|
||||
|
||||
迁移000018为Wait增加signal_name和payload_version两个带中文注释的可空字段；NULL仅表示TIMER或尚未核实的历史元数据，未来消费者不得把历史NULL自动解释为有效的新信号等待。correlation_key转为精确字节存储；实例business_key改为utf8mb4_0900_bin。35个实体、含公共Record的295个直接持久化字段声明，全部有中文注释，继续保持零外键。
|
||||
迁移000018为Wait增加signal_name和payload_version两个带中文注释的可空字段；NULL仅表示TIMER或尚未核实的历史元数据，未来消费者不得把历史NULL自动解释为有效的新信号等待。correlation_key转为精确字节存储；实例business_key改为utf8mb4_0900_bin。该迁移时为35个实体、含公共Record的295个直接持久化字段声明，全部有中文注释，继续保持零外键。
|
||||
|
||||
## 5. 下一步接入
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Weave 结果消费与自动推进
|
||||
|
||||
状态：内部实例的TASK结果消费、顺序、分支、有限并行汇合和根检查点接续已接入Orbit后台运行器。[定时等待和后续查询轮次](13-workflow-clock.md)也已接入。实例公共HTTP/SDK仍未开放，信号、失败后的分支取消结清及人工恢复继续开发，不能将本阶段等同于完整Weave交付。
|
||||
状态：内部实例的TASK结果消费、顺序、分支、有限并行汇合和根检查点接续已接入Orbit后台运行器。[定时等待和后续查询轮次](13-workflow-clock.md)也已接入。[信号等待与最终查询竞争](14-workflow-signals.md)已接入。实例公共HTTP/SDK仍未开放，失败后的分支取消结清及人工恢复继续开发，不能将本阶段等同于完整Weave交付。
|
||||
|
||||
## 1. 后台运行和事务边界
|
||||
|
||||
@@ -45,7 +45,7 @@ flowchart LR
|
||||
|
||||
暂停、关闭门控或停用应用/命名空间时，继续收集已经落库的结果，但不激活新步骤；若还有后继，将其保存为待续。重新允许推进后，从持久记录继续。测试通过直接设置控制状态验证该行为；暂停/继续公共接口尚未交付。
|
||||
|
||||
已过实例总截止时不创建后继，未能全部完成则进入核查。最后的有效任务结果在截止前已由Pulse接受，而消费发生较晚时，可以整理完整结果并完成实例。暂停或重启不延长截止。TIMER与POLL的时钟已接入；SIGNAL及其超期裁决仍需后续实现。
|
||||
已过实例总截止时不创建后继，未能全部完成则进入核查。最后的有效任务结果在截止前已由Pulse接受，而消费发生较晚时，可以整理完整结果并完成实例。暂停或重启不延长截止。TIMER与POLL的时钟已接入；[SIGNAL及其超期/最终查询裁决](14-workflow-signals.md)已接入。
|
||||
|
||||
## 4. POLL与失败策略的当前边界
|
||||
|
||||
@@ -56,7 +56,7 @@ flowchart LR
|
||||
- 轮次耗尽、查询截止、未知/非法值或单轮请求失败：进入核查，不新建轮次绕过Pulse重试预算。
|
||||
- 业务明确失败：按固定实例失败策略处理；REVIEW进入核查，FAIL_AFTER_SETTLED进入CANCELLING并关闭门控。
|
||||
|
||||
下一轮查询派发及到期扫描已实现，详见[时钟、预算与恢复](13-workflow-clock.md)。**尚未实现FAIL_AFTER_SETTLED的剩余分支取消/收集/失败终态判定。** CANCELLING保留未结清事实，不提前宣称FAILED。信号抢先完成时与当前在途查询的结清竞争，也属于后续必须实现的功能。
|
||||
下一轮查询派发及到期扫描已实现，详见[时钟、预算与恢复](13-workflow-clock.md)。**尚未实现FAIL_AFTER_SETTLED的剩余分支取消/收集/失败终态判定。** CANCELLING保留未结清事实，不提前宣称FAILED。信号抢先到达时保留当前查询，实际结果结清及竞争见[信号运行约定](14-workflow-signals.md)。
|
||||
|
||||
## 5. 回归与后续工作
|
||||
|
||||
@@ -64,4 +64,4 @@ flowchart LR
|
||||
|
||||
进程回归由`make integration`构建二进制后显式执行。缺少一次性数据库或`ORBIT_TEST_SERVER_BINARY`时测试会跳过；跳过不构成进程恢复证据。验证记录见[本阶段报告](reports/phase-4-runtime.md)。
|
||||
|
||||
下一步仍包括持久化信号等待及查询最终信号竞争、完整控制与人工恢复、实例/来源公共API、独立SDK和通用端到端场景。P4整体及P5–P7继续按[完整计划](08-development-plan.md)执行。
|
||||
持久化信号等待及查询最终信号竞争已按[信号运行约定](14-workflow-signals.md)实现。下一步仍包括完整控制与人工恢复、实例/来源公共API、独立SDK和通用端到端场景。P4整体及P5–P7继续按[完整计划](08-development-plan.md)执行。
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
# Weave 持久化定时等待与查询轮次
|
||||
|
||||
状态：TIMER到期推进和POLL后续业务轮次已接入Orbit后台运行器。复用MySQL和既有待续索引，不增加外部调度服务或数据库外键。实例公共HTTP/SDK、SIGNAL消费、查询最终信号竞争以及完整控制/人工恢复仍未交付。
|
||||
状态：TIMER到期推进和POLL后续业务轮次已接入Orbit后台运行器。复用MySQL和既有待续索引，不增加外部调度服务或数据库外键。[SIGNAL消费与查询最终信号竞争](14-workflow-signals.md)已进一步接入；实例公共HTTP/SDK以及完整控制/人工恢复仍未交付。
|
||||
|
||||
## 1. 时钟和恢复身份
|
||||
|
||||
`weave_pending_advances`同时保存ROOT、TIMER、POLL三类检查点。唯一范围为命名空间、实例和步骤；时钟检查点额外固定当前激活ID，避免旧激活唤醒新工作。扫描只提供候选，实际推进仍须持锁核验固定版本、父记录、步骤/激活状态、任务所有者、门控和原始预算。
|
||||
`weave_pending_advances`同时保存ROOT、TIMER、SIGNAL、POLL四类检查点。唯一范围为命名空间、实例和步骤；时钟检查点额外固定当前激活ID，避免旧激活唤醒新工作。扫描只提供候选，实际推进仍须持锁核验固定版本、父记录、步骤/激活状态、任务所有者、门控和原始预算。
|
||||
|
||||
| 检查点 | 到期时间来源 | 推进结果 |
|
||||
| --- | --- | --- |
|
||||
@@ -50,7 +50,7 @@ TIMER由待续表直接唤醒，不创建Pulse业务任务，也不占Worker执
|
||||
|
||||
当前任务仍在途且查询总截止已到时，实例转NEEDS_REVIEW并关闭新执行门控；步骤保持ACTIVE、轮次保持当前任务引用，继续收集Pulse结果。时钟不释放执行许可，也不假定外部请求已经停止。Pulse恢复将失联执行记录为UNKNOWN后，内部消费再结清当前查询记录，保留核查状态。
|
||||
|
||||
最终信号与在途查询的竞争尚未实现。定义可以描述`final_signal`并建立等待，但当前只能通过查询结果结清；不能据此承诺外部信号已经可以消费。
|
||||
最终信号与在途查询的竞争已按[信号运行约定](14-workflow-signals.md)接入：保存赢家后等待Pulse查询结清，不能提前释放许可或覆盖未知效果。实例及信号公共HTTP/SDK尚未开放。
|
||||
|
||||
## 5. 升级和验证
|
||||
|
||||
@@ -60,4 +60,4 @@ TIMER由待续表直接唤醒，不创建Pulse业务任务，也不占Worker执
|
||||
|
||||
[存储回归](../internal/storage/mysql/workflow_clock_integration_test.go)覆盖到期、竞争、预算、暂停/满额、回滚、归属损坏与未知结果；[v18升级回归](../internal/storage/mysql/workflow_clock_upgrade_integration_test.go)验证回填及原预算保留；[真实进程回归](../internal/storage/mysql/workflow_runtime_process_integration_test.go)覆盖全部控制进程退出后的定时和查询接续。执行记录见[阶段报告](reports/phase-4-clock.md)。
|
||||
|
||||
下一阶段继续SIGNAL早到/重复/超期处理、信号与查询竞争、失败分支结清、公共控制及实例SDK，完整范围见[持续计划](08-development-plan.md)。
|
||||
SIGNAL早到/重复/超期处理及最终查询竞争已接入；下一阶段继续失败分支结清、公共控制及实例SDK，完整范围见[持续计划](08-development-plan.md)。
|
||||
|
||||
68
docs/14-workflow-signals.md
Normal file
68
docs/14-workflow-signals.md
Normal file
@@ -0,0 +1,68 @@
|
||||
# Weave 外部信号与最终查询结果
|
||||
|
||||
状态：内部 Go 信号受理/查询、SIGNAL 等待、早到保存、截止处理及 POLL 最终信号竞争已接入 MySQL 和后台运行器。**实例与信号的公共 HTTP/独立 SDK 尚未开放**；完整控制、人工恢复和失败分支结清继续按[持续计划](08-development-plan.md)实现。定义发布 SDK 不等于实例运行 SDK。
|
||||
|
||||
## 1. 受理身份与权限
|
||||
|
||||
[内部契约](../weave/signal/service.go)为 `Send(token, workflowID, idempotencyKey, input)` 与 `Get(token, workflowID, signalID)`。输入绑定固定定义中的信号名称、内容版本、步骤键、显式目标激活序号和精确操作关联，附内容、受理原因与可选证据引用。
|
||||
|
||||
| 输入 | 限制及语义 |
|
||||
| --- | --- |
|
||||
| name / payload_version | 定义中冻结的名称/内容契约；ASCII，最多128/32字符 |
|
||||
| target_step_key | 固定定义的目标步骤，最多64字符，必须为SIGNAL或带最终信号的POLL |
|
||||
| target_activation_sequence | 显式正数；首次和早到信号为1，旧信号不能跟随人工恢复后的新激活；尚不存在的未来代次拒绝 |
|
||||
| correlation_key | 1至512字节UTF8，精确匹配，保留尾部空格；不由调用者随意省略 |
|
||||
| payload | 固定类型的精确JSON，最多256KiB；显式null与缺失不同；POLL最终信号不能是PENDING |
|
||||
| reason / evidence_ref | 必填原因最多1024字符；可选证据引用最多512字符、不含空白/控制符；平台不访问引用地址 |
|
||||
|
||||
提交要求显式 `workflows:signal` 权限；查询要求 `workflows:read`。两者都核对当前凭据、命名空间、实例固定版本及全部相关队列权限。actor来自认证凭据，不能请求自报。现有bootstrap的定义发布授权不会自动授予信号操作权限。
|
||||
|
||||
提交按认证 → 应用额度 → 原幂等身份 → 排序队列 → 实例/门控/步骤持锁。首次受理将信号、检查点、事件、审计和冻结回执同事务写入。审计包含原请求摘要、受理时间和总截止；运行器会核对受理证明，参数被替换时转核查，数据库连接/SQL错误则保留原来源并退避重试。
|
||||
|
||||
同一实例最多保存1024条信号，包含拒绝及终结历史；原幂等身份恢复先于上限检查。同键同内容返回首次冻结回执，同键不同内容冲突。重报仍校验当前权限。受理回执中的PENDING不跟随消费变化，当前状态用Get读取。
|
||||
|
||||
## 2. 早到、迟到与消费
|
||||
|
||||
目标步骤尚未激活时，信号保持PENDING；步骤激活会创建立即到期的检查点扫描早到信号。每个等待固定名称、版本、操作关联、激活序号和原截止，消费选择按受理时间、信号ID排序的第一条匹配记录。
|
||||
|
||||
- 信号必须在等待截止之前受理，等于截止不算按时。按时受理但控制进程稍后恢复，可以继续收集该事实。
|
||||
- 暂停或应用停用仍可收集结果；后继工作受原门控、额度及实例总截止约束。
|
||||
- 已激活时的关联不符、旧代次、已结清步骤或实例会保存明确拒绝原因；内容类型/定义契约无效的请求直接拒绝，不产生有效信号。
|
||||
- 同一等待只记录一个winner_signal_id；其余同代次信号保存WAIT_SETTLED或QUERY_SETTLED等原因。所有关联由事务核验，不使用外键。
|
||||
- SIGNAL在没有按时匹配信号的情况下到期，将等待标为EXPIRED、步骤和实例转NEEDS_REVIEW，并关闭新执行门控。
|
||||
|
||||
信号的expires_at固定为首次受理时的实例总截止。尚未匹配的早到信号通过单独到期索引结清；available_at仅表示下次检查时间，故障退避不能改变expires_at、accepted_at或等待截止。若实例已过总截止且仍需新工作，保留已收到事实并进入核查。
|
||||
|
||||
消费确认、等待赢家、步骤结果、后继任务或根检查点及审计同事务。审计失败不会留下“信号已消费但后继未保存”的部分状态。
|
||||
|
||||
## 3. 最终信号与POLL查询竞争
|
||||
|
||||
查询请求由Pulse执行，最终信号和查询结果由同一个实例事务裁决。PENDING只是一次成功查询观察，不争夺最终结果；请求失败也不允许创建新业务轮次绕过Pulse尝试预算。
|
||||
|
||||
| 条件 | 处理 |
|
||||
| --- | --- |
|
||||
| 最终查询结果早于信号受理，且在原查询截止内 | 查询结果胜出，信号保存QUERY_SETTLED；精度相同的时间优先查询 |
|
||||
| 已有最终信号，当前无在途查询 | 结清POLL，不再创建下一轮查询 |
|
||||
| 最终信号先到，查询尚未开始 | 保存信号赢家，使用Pulse取消规则结清未开始任务，再收集其Inbox结果 |
|
||||
| 最终信号先到，查询正在执行 | 保存赢家和取消意图；保留当前任务、ACTIVE步骤及执行许可，等待Pulse实际结果 |
|
||||
| 在途查询随后返回PENDING、相同最终结果或已知失败/取消 | 收集该查询结清后采用信号结果，不新增查询轮次 |
|
||||
| 在途查询效果未知、结果不合法或返回不同最终结果 | 保留信号和查询证据，转NEEDS_REVIEW；不能让信号覆盖未知事实 |
|
||||
| 原查询总截止已到，查询仍在途 | 关闭新执行门控，保留当前任务等待结清；后续结果不能自动清除已建立的核查状态 |
|
||||
|
||||
`CONSUMED`表示信号已被等待采用，不表示整个实例成功。带在途查询时，即使等待记录为SATISFIED，步骤仍保持ACTIVE直到Pulse确认结束。取消复用Pulse既有许可释放逻辑，不在Weave中实现另一套释放计数。
|
||||
|
||||
控制时钟发现查询结果已落库但未消费时，读取原Inbox事件参与裁决；不由扫描器先后决定赢家，也不绕过Inbox故障退避。结果、信号和许可预算均在MySQL持久保存，进程重启不重新选择已保存的赢家。
|
||||
|
||||
## 4. 存储升级和运行边界
|
||||
|
||||
[迁移000020](../internal/storage/mysql/migrations/000020_workflow_signal_identity.up.sql)增加信号契约/代次/证据/处置/到期检查时间，以及等待的赢家引用。名称扩至128字符，关联保持精确字节，并兼容历史764字节值。35个实体、301个直接持久化字段声明均提供中文说明；数据库无外键、无级联删除。
|
||||
|
||||
历史信号没有足够身份和受理证明：旧PENDING标记REJECTED/LEGACY_UNVERIFIED，保留原载荷和时间；已消费历史不被重新解释。旧NULL等待元数据不能推测成有效新契约。迁移为当前SIGNAL等待回填原截止的检查点，保留已有检查点，排除终态实例及旧激活。
|
||||
|
||||
升级仍采用停止旧写入节点 → 应用全部迁移 → 启动新版；新版信号字段和运行语义不支持与旧节点混跑。后台扫描三类候选：结果Inbox、根/等待检查点和未匹配信号到期；每类最多100项，最终批次最多100项，每个候选短事务处理。
|
||||
|
||||
## 5. 验证与后续
|
||||
|
||||
[受理回归](../internal/storage/mysql/workflow_signals_integration_test.go)、[消费回归](../internal/storage/mysql/workflow_signal_runtime_integration_test.go)、[真实v19升级](../internal/storage/mysql/workflow_signal_upgrade_integration_test.go)和[实际进程恢复](../internal/storage/mysql/workflow_runtime_process_integration_test.go)对应受理/并发/故障/升级边界，实际运行结果记录在[阶段报告](reports/phase-4-signals.md)。
|
||||
|
||||
下一步完成失败分支取消结清、实例暂停/继续/取消/人工核查恢复，再交付实例与信号HTTP、独立SDK及完整场景。当前内部存储测试不替代尚未开放的公共接口验收，也不替代生产MySQL切换与容量测试。
|
||||
49
docs/reports/phase-4-signals.md
Normal file
49
docs/reports/phase-4-signals.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# P4/P5 外部信号与最终查询竞争回归
|
||||
|
||||
日期：2026-09-18。实现和验收范围为内部信号服务与持久化运行器。实例/信号公共HTTP和独立SDK、完整控制与人工恢复仍待交付，持续目标未结束。
|
||||
|
||||
## 变化与边界
|
||||
|
||||
- 固定目标步骤、激活代次、信号名称/契约、精确操作关联、原因及证据，显式信号授权；每实例历史总上限1024。
|
||||
- 原键冻结回执，参数冲突拒绝；受理证明与不可变审计核对，不接受修改后的信号参数授权后继。
|
||||
- SIGNAL早到、等待、按时受理延迟消费、重复与截止处理；暂停收集并持久保存根接续。
|
||||
- POLL最终信号与查询按原事实时间竞争，PENDING不作为最终结果；在途查询先请求取消并保留许可，未知或冲突转核查。
|
||||
- Pulse取消逻辑提取为公共内部事务函数，原公共取消权限、幂等和审计边界保留；信号调用前验证流程所有权。
|
||||
- 迁移000020增加5个Signal字段和1个Wait字段。当前35个实体、301个直接持久化字段声明，中文注释、零外键及禁止级联约束继续执行。
|
||||
|
||||
详细规则及升级顺序见[信号文档](../14-workflow-signals.md)。此处CONSUMED表示等待采用了信号，不代表流程成功或外部查询已结束。
|
||||
|
||||
## 验证记录
|
||||
|
||||
环境：darwin/arm64、Go 1.27.1、Docker MySQL 8.4.10；临时隔离数据库，未访问业务数据库或生产数据库。
|
||||
|
||||
| 检查 | 结果 |
|
||||
| --- | --- |
|
||||
| 基础格式、单元、竞态、静态检查、构建 | `make fmt test vet build`通过，退出0，包含服务端、独立SDK和业务示例module |
|
||||
| 信号受理与运行器专项 | 受理4组、运行器20组，连同TIMER/POLL、原流程运行器、Pulse取消、新建结构和v7/v9升级，`-race`通过，51.926秒 |
|
||||
| 真实v19升级 | 全量存储阶段已通过，3.90秒；原764字节关联、精确内容、已消费证据保留，旧PENDING隔离，当前SIGNAL在原截止回填 |
|
||||
| 全量隔离MySQL与真实进程/SDK/示例 | 已通过，脚本退出0；全存储172.485秒、流程进程20.127秒、Worker SDK 94.716秒、SDK进程故障93.118秒；19个SDK示例和2个独立业务示例全部通过 |
|
||||
|
||||
专项入口：
|
||||
|
||||
```sh
|
||||
go test -race -count=1 -run '^TestIntegration(WorkflowSignalAdmission|WorkflowSignalRuntime|WorkflowSignalUpgrade|WorkflowClock|WorkflowRuntime|WorkerCancellation|SchemaAndConstraints|NoForeignKeysUpgrade|ApplicationResourcesUpgrade)$' -v ./internal/storage/mysql
|
||||
go build -o /tmp/orbit-signal-test-server ./cmd/orbit
|
||||
ORBIT_TEST_SERVER_BINARY=/tmp/orbit-signal-test-server go test -race -count=1 -run '^TestIntegrationWorkflowRuntimeProcess$' -v ./internal/storage/mysql
|
||||
```
|
||||
|
||||
必须设置`ORBIT_TEST_MYSQL_DSN`为一次性数据库。未配置环境变量导致跳过时不能记作通过。[全量脚本](../../scripts/test-mysql.sh)创建隔离MySQL并显式构建二进制、再运行实际进程测试。
|
||||
|
||||
存储专项覆盖12个并发受理只生成一条信号、12个并发消费者只创建一个后继、早到保存、暂停后接续、按时受理晚消费、真实等待超时、审计故障回滚、原回执冻结、载荷/actor/受理时间篡改进入核查、过期退避不改截止，以及最终信号在轮次间/未开始/执行中到达。执行中的查询分别返回取消、PENDING、相同最终值、未知、已知请求失败及冲突最终值；核查和许可释放按实际结果核验。查询较早完成时，先运行时钟或先运行Inbox都保留查询胜出。
|
||||
|
||||
真实进程用例进一步扩展了早到信号在全停期间保存、两个新控制进程激活消费，以及最终信号选中后再强杀全部进程、查询结清入库后由两个新进程恢复赢家的场景。该用例已由实际Orbit二进制执行通过，20.127秒；不是仅重建Go对象或内存快照的模拟。
|
||||
|
||||
数据库直接核验外键0、缺少COMMENT的业务字段0；代码301个直接持久化字段声明均有中文注释。文档检查870个本地链接、53个JSON代码块，错误0，业务项目标识检查通过；`git diff --check`通过。v7升级中70个历史外键全部移除，323项原索引组件保留；v20的两个信号索引和赢家唯一索引另行核验定义。
|
||||
|
||||
## 仍须完成
|
||||
|
||||
1. FAIL_AFTER_SETTLED分支取消与完整结清、实例暂停/继续/取消/人工核查恢复。
|
||||
2. 实例/步骤/来源/信号/审计HTTP、独立SDK和可运行的端到端通用示例。
|
||||
3. P6–P7的监控、完整故障矩阵、性能容量、MySQL切换/备份恢复和SDK发布核对。
|
||||
|
||||
这些测试不等同于生产高可用、容量或RPO/RTO验收；运行器采用单MySQL持久事务，生产边界继续单独验证。
|
||||
@@ -64,6 +64,8 @@ func TestIntegrationApplicationResourcesUpgrade(t *testing.T) {
|
||||
retryFinish(t, g, finished, "SUCCEEDED", false)
|
||||
assertApplicationCounts(t, f, 4, 2, 1)
|
||||
// 还原后续新增表和字段，得到真实 v9 结构；现有任务内容保持不变。
|
||||
f.exec(`ALTER TABLE weave_signals DROP CHECK ck_signal_version_sequence, DROP CHECK ck_signal_pending_identity, DROP INDEX ix_signal_expiry, DROP INDEX ix_signal_activation, DROP COLUMN payload_version, DROP COLUMN target_activation_sequence, DROP COLUMN evidence_ref, DROP COLUMN disposition, DROP COLUMN available_at, MODIFY COLUMN type VARCHAR(64) NOT NULL COMMENT '事件或信号的稳定类型标识', MODIFY COLUMN correlation_key VARCHAR(191) NOT NULL COMMENT '外部业务操作关联键，防止旧信号误用于新工作'`)
|
||||
f.exec(`ALTER TABLE weave_waits DROP INDEX uq_wait_winner_signal, DROP COLUMN winner_signal_id`)
|
||||
f.exec(`ALTER TABLE weave_waits DROP COLUMN signal_name, DROP COLUMN payload_version, MODIFY COLUMN correlation_key VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs NOT NULL COMMENT '外部业务操作关联键，防止旧信号误用于新工作'`)
|
||||
f.exec(`ALTER TABLE weave_instances MODIFY COLUMN business_key VARCHAR(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_as_cs NULL COMMENT '业务侧关联键，用于检索追踪，不自动提供业务幂等'`)
|
||||
f.exec(`ALTER TABLE weave_inbox DROP CHECK ck_weave_inbox_source, DROP INDEX uq_weave_task_result, DROP COLUMN source_task_id, DROP COLUMN source_activation_id, DROP COLUMN source_revision, DROP COLUMN snapshot`)
|
||||
|
||||
82
internal/storage/mysql/execution_cancel.go
Normal file
82
internal/storage/mysql/execution_cancel.go
Normal file
@@ -0,0 +1,82 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
pulse "git.yuntuo.hk/k2safe/orbit/pulse/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/pulse/service"
|
||||
)
|
||||
|
||||
// cancelExecutionTask 由已验证所有权的调用方复用；调用前必须持有应用、队列和任务锁。
|
||||
// 仅已知未开始的任务立即结清，在途执行保留许可并等待结果，历史未知效果不能变成已取消。
|
||||
func cancelExecutionTask(ctx context.Context, tx *sql.Tx, ns core.ID, task executionTask, now time.Time, reason string) (service.TaskView, error) {
|
||||
var out service.TaskView
|
||||
view := task.view
|
||||
if view.State.Terminal() || view.CancelRequestedAt != nil {
|
||||
return view, nil
|
||||
}
|
||||
id := view.ID
|
||||
var err error
|
||||
var attempt *pulse.Attempt
|
||||
if view.State == pulse.TaskLeased || view.State == pulse.TaskRunning {
|
||||
if task.current == nil {
|
||||
return out, errors.New("active task lacks attempt")
|
||||
}
|
||||
a, err := readAttempt(ctx, tx, ns, *task.current)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if a.TaskID != id || a.Epoch != task.epoch || string(a.State) != string(view.State) {
|
||||
return out, errors.New("task attempt identity or state mismatch")
|
||||
}
|
||||
attempt = &a
|
||||
}
|
||||
next := view.State
|
||||
lease := task.lease
|
||||
var completed *time.Time
|
||||
if view.State == pulse.TaskWaiting || view.State == pulse.TaskLeased {
|
||||
next = pulse.TaskCancelled
|
||||
completed = &now
|
||||
lease = nil
|
||||
if view.EffectUncertain {
|
||||
next, completed = pulse.TaskNeedsReview, nil
|
||||
}
|
||||
if attempt != nil {
|
||||
if attempt.AuthorizedAt != nil {
|
||||
return out, errors.New("leased attempt already authorized")
|
||||
}
|
||||
result, err := tx.ExecContext(ctx, `UPDATE pulse_attempts SET state='CANCELLED',ended_at=?,error_code='CANCELLED_BEFORE_START',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, now, ns, attempt.ID, attempt.Revision)
|
||||
if err := exactlyOne(result, err); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if err := releasePermits(ctx, tx, ns, attempt.ID, now); err != nil {
|
||||
return out, err
|
||||
}
|
||||
}
|
||||
}
|
||||
result, err := tx.ExecContext(ctx, `UPDATE pulse_tasks SET state=?,cancel_requested_at=?,lease_until=?,completed_at=?,updated_at=?,revision=revision+1 WHERE namespace_id=? AND id=? AND revision=?`, next, now, lease, completed, now, ns, id, view.Revision)
|
||||
if err := exactlyOne(result, err); err != nil {
|
||||
return out, err
|
||||
}
|
||||
if next.Terminal() {
|
||||
if err := releaseQueueTask(ctx, tx, ns, view.QueueID); err != nil {
|
||||
return out, err
|
||||
}
|
||||
}
|
||||
out, err = readTask(ctx, tx, ns, id, false)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
event := resultEvent(next)
|
||||
if next == view.State {
|
||||
event = "task.cancel_requested"
|
||||
}
|
||||
if err := emitEvent(ctx, tx, ns, id, out.Revision, "task", event, map[string]any{"task_id": id, "state": next, "reason": reason, "effect_uncertain": view.EffectUncertain}); err != nil {
|
||||
return out, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
-- 外部信号固定参数版本与目标激活代次；旧记录无此身份，不能推测为新等待的有效输入。
|
||||
ALTER TABLE weave_signals
|
||||
MODIFY COLUMN `type` VARCHAR(128) CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_bin NOT NULL COMMENT '固定定义中的信号名称，新受理限制为128个ASCII字符',
|
||||
MODIFY COLUMN correlation_key VARBINARY(764) NOT NULL COMMENT '精确UTF8操作关联，保留尾部空格；新信号最多512字节，历史兼容764字节',
|
||||
ADD COLUMN payload_version VARCHAR(32) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '信号内容契约版本；为空仅表示未核实的历史记录',
|
||||
ADD COLUMN target_activation_sequence INT UNSIGNED NULL COMMENT '固定目标步骤激活序号，早到首轮为1；为空仅表示未核实历史',
|
||||
ADD COLUMN evidence_ref VARCHAR(512) NULL COMMENT '可选的受控证据引用；不保存秘密，也不由平台请求该地址',
|
||||
ADD COLUMN disposition VARCHAR(64) CHARACTER SET ascii COLLATE ascii_bin NULL COMMENT '稳定消费或拒绝原因；PENDING时为空，不包含业务参数',
|
||||
ADD COLUMN available_at DATETIME(6) NULL COMMENT '未匹配信号下次到期检查时间；故障退避可延后检查，但不改变原截止',
|
||||
ADD KEY ix_signal_expiry (state,available_at,namespace_id,id),
|
||||
ADD KEY ix_signal_activation (namespace_id,workflow_id,target_step_key,target_activation_sequence,state,accepted_at,id);
|
||||
|
||||
UPDATE weave_signals SET state='REJECTED',disposition='LEGACY_UNVERIFIED',revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE state='PENDING';
|
||||
UPDATE weave_signals SET available_at=expires_at;
|
||||
|
||||
ALTER TABLE weave_signals
|
||||
MODIFY COLUMN available_at DATETIME(6) NOT NULL COMMENT '未匹配信号下次到期检查时间；故障退避可延后检查，但不改变原截止',
|
||||
ADD CONSTRAINT ck_signal_version_sequence CHECK ((payload_version IS NULL AND target_activation_sequence IS NULL) OR (payload_version IS NOT NULL AND payload_version<>'' AND target_activation_sequence IS NOT NULL AND target_activation_sequence>0)),
|
||||
ADD CONSTRAINT ck_signal_pending_identity CHECK (state<>'PENDING' OR (payload_version IS NOT NULL AND target_activation_sequence IS NOT NULL AND disposition IS NULL));
|
||||
|
||||
ALTER TABLE weave_waits
|
||||
ADD COLUMN winner_signal_id BINARY(16) NULL COMMENT '胜出的外部信号标识；仅时间或查询结果结清时为空，所有者由同事务校验',
|
||||
ADD UNIQUE KEY uq_wait_winner_signal (namespace_id,winner_signal_id);
|
||||
|
||||
-- SIGNAL到期扫描与TIMER/POLL共用待续表；旧元数据缺失的等待交给运行器核查，不凭空授权。
|
||||
INSERT INTO weave_pending_advances(namespace_id,id,workflow_id,step_id,expected_step_revision,checkpoint,available_at)
|
||||
SELECT s.namespace_id,RANDOM_BYTES(16),s.workflow_id,s.id,s.revision,
|
||||
JSON_OBJECT('schema_version',1,'kind','SIGNAL','activation_id',LOWER(HEX(a.id))),w.deadline_at
|
||||
FROM weave_steps s JOIN weave_activations a ON a.namespace_id=s.namespace_id AND a.id=s.current_activation_id AND a.workflow_id=s.workflow_id AND a.step_id=s.id
|
||||
JOIN weave_instances i ON i.namespace_id=s.namespace_id AND i.id=s.workflow_id
|
||||
JOIN weave_waits w ON w.namespace_id=s.namespace_id AND w.workflow_id=s.workflow_id AND w.activation_id=a.id
|
||||
WHERE s.kind='SIGNAL' AND s.state='WAITING' AND a.state='WAITING' AND w.kind='SIGNAL' AND w.state='WAITING' AND i.state IN ('RUNNING','WAITING','CANCELLING','NEEDS_REVIEW')
|
||||
ON DUPLICATE KEY UPDATE id=weave_pending_advances.id;
|
||||
@@ -88,8 +88,8 @@ func TestIntegrationNoForeignKeysUpgrade(t *testing.T) {
|
||||
}
|
||||
indexes := func() []string {
|
||||
t.Helper()
|
||||
// v15替换retry_of索引、v17增加内部结果唯一索引，下方单独验证定义。
|
||||
rows, err := db.QueryContext(ctx, `SELECT CONCAT_WS(':',TABLE_NAME,INDEX_NAME,NON_UNIQUE,SEQ_IN_INDEX,COLUMN_NAME,COALESCE(SUB_PART,0),INDEX_TYPE,IS_VISIBLE) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME NOT IN ('orbit_application_resources','orbit_event_streams','orbit_result_events') AND INDEX_NAME NOT IN ('ix_task_priority_dispatch','ix_delivery_subscription_due','ix_delivery_subscription_lease','ix_delivery_history','ix_delivery_state_history','uq_task_retry_parent','fk_pulse_tasks_3','ix_task_queue_business_created','ix_task_queue_business_state_created','uq_weave_task_result') ORDER BY TABLE_NAME,INDEX_NAME,SEQ_IN_INDEX`)
|
||||
// v15替换retry_of索引、v17增加内部结果唯一索引、v20增加信号索引，下方单独验证定义。
|
||||
rows, err := db.QueryContext(ctx, `SELECT CONCAT_WS(':',TABLE_NAME,INDEX_NAME,NON_UNIQUE,SEQ_IN_INDEX,COLUMN_NAME,COALESCE(SUB_PART,0),INDEX_TYPE,IS_VISIBLE) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME NOT IN ('orbit_application_resources','orbit_event_streams','orbit_result_events') AND INDEX_NAME NOT IN ('ix_task_priority_dispatch','ix_delivery_subscription_due','ix_delivery_subscription_lease','ix_delivery_history','ix_delivery_state_history','uq_task_retry_parent','fk_pulse_tasks_3','ix_task_queue_business_created','ix_task_queue_business_state_created','uq_weave_task_result','ix_signal_expiry','ix_signal_activation','uq_wait_winner_signal') ORDER BY TABLE_NAME,INDEX_NAME,SEQ_IN_INDEX`)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -178,6 +178,16 @@ func TestIntegrationNoForeignKeysUpgrade(t *testing.T) {
|
||||
t.Fatalf("workflow result uniqueness=%q err=%v", resultIndex, err)
|
||||
}
|
||||
var collation string
|
||||
for _, index := range []struct{ table, name, want string }{
|
||||
{"weave_signals", "ix_signal_expiry", "state:1,available_at:1,namespace_id:1,id:1"},
|
||||
{"weave_signals", "ix_signal_activation", "namespace_id:1,workflow_id:1,target_step_key:1,target_activation_sequence:1,state:1,accepted_at:1,id:1"},
|
||||
{"weave_waits", "uq_wait_winner_signal", "namespace_id:0,winner_signal_id:0"},
|
||||
} {
|
||||
var got string
|
||||
if err := db.QueryRowContext(ctx, `SELECT GROUP_CONCAT(CONCAT(COLUMN_NAME,':',NON_UNIQUE) ORDER BY SEQ_IN_INDEX) FROM information_schema.STATISTICS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME=? AND INDEX_NAME=?`, index.table, index.name).Scan(&got); err != nil || got != index.want {
|
||||
t.Fatalf("signal index %s=%q err=%v", index.name, got, err)
|
||||
}
|
||||
}
|
||||
if err := db.QueryRowContext(ctx, `SELECT COLLATION_NAME FROM information_schema.COLUMNS WHERE TABLE_SCHEMA=DATABASE() AND TABLE_NAME='pulse_tasks' AND COLUMN_NAME='business_key'`).Scan(&collation); err != nil || collation != "utf8mb4_0900_bin" {
|
||||
t.Fatalf("business key collation=%q err=%v", collation, err)
|
||||
}
|
||||
|
||||
@@ -4,14 +4,11 @@ import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/orbit/internal/fault"
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/platform/auth"
|
||||
pulse "git.yuntuo.hk/k2safe/orbit/pulse/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/pulse/service"
|
||||
)
|
||||
|
||||
@@ -56,68 +53,15 @@ func (s *TaskStore) CancelTask(ctx context.Context, token auth.Token, id core.ID
|
||||
if owner.Valid && owner.String != "schedule" {
|
||||
return fault.Conflict
|
||||
}
|
||||
var attempt *pulse.Attempt
|
||||
if view.State == pulse.TaskLeased || view.State == pulse.TaskRunning {
|
||||
if task.current == nil {
|
||||
return errors.New("active task lacks attempt")
|
||||
}
|
||||
a, err := readAttempt(ctx, tx, token.NamespaceID, *task.current)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if a.TaskID != id || a.Epoch != task.epoch || string(a.State) != string(view.State) {
|
||||
return errors.New("task attempt identity or state mismatch")
|
||||
}
|
||||
attempt = &a
|
||||
}
|
||||
now, err := dbNow(ctx, tx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
next := view.State
|
||||
lease := task.lease
|
||||
var completed *time.Time
|
||||
if view.State == pulse.TaskWaiting || view.State == pulse.TaskLeased {
|
||||
next = pulse.TaskCancelled
|
||||
completed = &now
|
||||
lease = nil
|
||||
if view.EffectUncertain {
|
||||
next, completed = pulse.TaskNeedsReview, nil
|
||||
}
|
||||
if attempt != nil {
|
||||
if attempt.AuthorizedAt != nil {
|
||||
return errors.New("leased attempt already authorized")
|
||||
}
|
||||
result, err := tx.ExecContext(ctx, `UPDATE pulse_attempts SET state='CANCELLED',ended_at=?,error_code='CANCELLED_BEFORE_START',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, now, token.NamespaceID, attempt.ID, attempt.Revision)
|
||||
if err := exactlyOne(result, err); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := releasePermits(ctx, tx, token.NamespaceID, attempt.ID, now); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
result, err := tx.ExecContext(ctx, `UPDATE pulse_tasks SET state=?,cancel_requested_at=?,lease_until=?,completed_at=?,updated_at=?,revision=revision+1 WHERE namespace_id=? AND id=? AND revision=?`, next, now, lease, completed, now, token.NamespaceID, id, in.ExpectedRevision)
|
||||
if err := exactlyOne(result, err); err != nil {
|
||||
return err
|
||||
}
|
||||
if next.Terminal() {
|
||||
if err := releaseQueueTask(ctx, tx, token.NamespaceID, view.QueueID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
out.Task, err = readTask(ctx, tx, token.NamespaceID, id, false)
|
||||
out.Task, err = cancelExecutionTask(ctx, tx, token.NamespaceID, task, now, "CANCEL_REQUESTED")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
event := resultEvent(next)
|
||||
if next == view.State {
|
||||
event = "task.cancel_requested"
|
||||
}
|
||||
if err := emitEvent(ctx, tx, token.NamespaceID, id, out.Task.Revision, "task", event, map[string]any{"task_id": id, "state": next, "reason": "CANCEL_REQUESTED", "effect_uncertain": view.EffectUncertain}); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := appendAudit(ctx, tx, token, identity, id, "task", "task.cancel", in.Reason, map[string]any{"state": view.State, "revision": fmt.Sprint(view.Revision)}, map[string]any{"state": next, "revision": fmt.Sprint(out.Task.Revision), "cancel_requested_at": now}); err != nil {
|
||||
if err := appendAudit(ctx, tx, token, identity, id, "task", "task.cancel", in.Reason, map[string]any{"state": view.State, "revision": fmt.Sprint(view.Revision)}, map[string]any{"state": out.Task.State, "revision": fmt.Sprint(out.Task.Revision), "cancel_requested_at": now}); err != nil {
|
||||
return err
|
||||
}
|
||||
return saveIdentity(ctx, tx, token.NamespaceID, identity, out)
|
||||
|
||||
@@ -255,6 +255,11 @@ func activateWorkflowStep(ctx context.Context, tx *sql.Tx, g *workflowGraph, row
|
||||
if err := saveWorkflowClock(ctx, tx, g, row, "TIMER", row.view.Revision+1, until); err != nil {
|
||||
return err
|
||||
}
|
||||
} else if c.Wait.Kind == "SIGNAL" {
|
||||
// 先扫描早到信号；未匹配时运行器按原等待截止继续持久化。
|
||||
if err := saveWorkflowClock(ctx, tx, g, row, "SIGNAL", row.view.Revision+1, now); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
if c.Task != nil {
|
||||
@@ -274,7 +279,11 @@ func activateWorkflowStep(ctx context.Context, tx *sql.Tx, g *workflowGraph, row
|
||||
}
|
||||
}
|
||||
if c.Kind == "POLL" {
|
||||
return saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision+1, deadline)
|
||||
due := deadline
|
||||
if c.Wait != nil {
|
||||
due = now
|
||||
}
|
||||
return saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision+1, due)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
// workflowCheckpoint 是待续表中版本化的内部唤醒身份；不保存内存执行栈或新的业务预算。
|
||||
type workflowCheckpoint struct {
|
||||
SchemaVersion uint32 `json:"schema_version"` // SchemaVersion 当前结构版本为1，未知版本禁止推进。
|
||||
Kind string `json:"kind"` // Kind 为ROOT、TIMER或POLL，决定实际权威记录的读取方式。
|
||||
Kind string `json:"kind"` // Kind 为ROOT、TIMER、SIGNAL或POLL，决定实际权威记录的读取方式。
|
||||
ActivationID *core.ID `json:"activation_id,omitempty"` // ActivationID 固定等待激活；ROOT不使用该字段。
|
||||
}
|
||||
|
||||
@@ -23,7 +23,7 @@ func saveWorkflowClock(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *w
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if row.view.ActivationID == nil || kind != "TIMER" && kind != "POLL" {
|
||||
if row.view.ActivationID == nil || kind != "TIMER" && kind != "POLL" && kind != "SIGNAL" {
|
||||
return errors.New("workflow clock identity invalid")
|
||||
}
|
||||
raw, err := json.Marshal(workflowCheckpoint{1, kind, row.view.ActivationID})
|
||||
@@ -60,8 +60,24 @@ func (r *WorkflowRuntime) advanceClock(ctx context.Context, tx *sql.Tx, g *workf
|
||||
return false, err
|
||||
}
|
||||
if checkpoint.Kind == "POLL" {
|
||||
node, _ := g.program.Step(row.view.Key)
|
||||
if node.Poll.FinalSignal != nil {
|
||||
handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if handled {
|
||||
if changed && reason != "" {
|
||||
return true, r.audit(ctx, tx, g, c.id, reason, now)
|
||||
}
|
||||
return changed, nil
|
||||
}
|
||||
}
|
||||
return r.advancePoll(ctx, tx, g, row, c, activated, allow, now)
|
||||
}
|
||||
if checkpoint.Kind == "SIGNAL" {
|
||||
return r.advanceSignalClock(ctx, tx, g, row, c, allow, now)
|
||||
}
|
||||
if checkpoint.Kind != "TIMER" || row.view.State != weave.StepWaiting {
|
||||
return false, errors.New("workflow timer state invalid")
|
||||
}
|
||||
|
||||
@@ -177,6 +177,15 @@ func (r *WorkflowRuntime) consume(ctx context.Context, tx *sql.Tx, g *workflowGr
|
||||
if resolved != nil || current == nil || *current != source.task {
|
||||
return true, r.confirmWorkflowResult(ctx, tx, g, c, "PROCESSED", "STALE_POLL_ROUND", now)
|
||||
}
|
||||
if node.Poll.FinalSignal != nil {
|
||||
handled, _, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, source, allow, now)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if handled {
|
||||
return true, r.confirmWorkflowResult(ctx, tx, g, c, "PROCESSED", reason, now)
|
||||
}
|
||||
}
|
||||
// 单轮请求失败不能通过新建业务轮次绕过Pulse预算。
|
||||
if state != weave.StepSucceeded {
|
||||
state, reason = weave.StepNeedsReview, "POLL_REQUEST_REVIEW"
|
||||
@@ -216,6 +225,15 @@ func (r *WorkflowRuntime) consume(ctx context.Context, tx *sql.Tx, g *workflowGr
|
||||
return false, err
|
||||
}
|
||||
if settled != nil {
|
||||
if node.Poll.FinalSignal != nil {
|
||||
w, err := readWorkflowSignalWait(ctx, tx, g, row)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "QUERY_SETTLED", now); err != nil {
|
||||
return false, err
|
||||
}
|
||||
}
|
||||
waitState := "CANCELLED"
|
||||
if state == weave.StepSucceeded {
|
||||
waitState = "SATISFIED"
|
||||
|
||||
@@ -16,31 +16,42 @@ import (
|
||||
weave "git.yuntuo.hk/k2safe/orbit/weave/model"
|
||||
)
|
||||
|
||||
func (r *WorkflowRuntime) advancePoll(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, activated time.Time, allow bool, now time.Time) (bool, error) {
|
||||
// readWorkflowPollState 校验当前激活的完整轮次与原预算；时钟和最终信号共用相同检查。
|
||||
func readWorkflowPollState(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, activated time.Time) (weave.PollState, core.ID, error) {
|
||||
var p weave.PollState
|
||||
err := tx.QueryRowContext(ctx, `SELECT id,revision,max_rounds,used_rounds,max_attempts_per_round,interval_ms,deadline_at,next_poll_at,current_task_id,resolved_at FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, c.ns, c.workflow, row.view.ActivationID).Scan(&p.ID, &p.Revision, &p.MaxRounds, &p.UsedRounds, &p.MaxAttemptsPerRound, &p.IntervalMS, &p.DeadlineAt, &p.NextPollAt, &p.CurrentTaskID, &p.ResolvedAt)
|
||||
err := tx.QueryRowContext(ctx, `SELECT id,revision,max_rounds,used_rounds,max_attempts_per_round,interval_ms,deadline_at,next_poll_at,current_task_id,resolved_at FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&p.ID, &p.Revision, &p.MaxRounds, &p.UsedRounds, &p.MaxAttemptsPerRound, &p.IntervalMS, &p.DeadlineAt, &p.NextPollAt, &p.CurrentTaskID, &p.ResolvedAt)
|
||||
if err != nil {
|
||||
return p, "", err
|
||||
}
|
||||
node, _ := g.program.Step(row.view.Key)
|
||||
policy := node.Poll
|
||||
if policy == nil || p.MaxRounds != policy.MaxRounds || p.MaxAttemptsPerRound != policy.Task.Retry.MaxAttempts || p.IntervalMS != policy.IntervalMS || p.UsedRounds < 1 || !p.DeadlineAt.Equal(minimumTime(g.view.DeadlineAt, activated.Add(time.Duration(policy.TimeoutMS)*time.Millisecond))) {
|
||||
return p, "", errors.New("workflow poll policy mismatch")
|
||||
}
|
||||
if p.ResolvedAt != nil {
|
||||
return p, "", errors.New("unsettled poll step has resolved budget")
|
||||
}
|
||||
var count, minRound, maxRound uint32
|
||||
if err := tx.QueryRowContext(ctx, "SELECT COUNT(*),COALESCE(MIN(`round`),0),COALESCE(MAX(`round`),0) FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=?", g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&count, &minRound, &maxRound); err != nil {
|
||||
return p, "", err
|
||||
}
|
||||
if count != p.UsedRounds || minRound != 1 || maxRound != p.UsedRounds {
|
||||
return p, "", errors.New("workflow poll rounds incomplete")
|
||||
}
|
||||
var lastTask core.ID
|
||||
if err := tx.QueryRowContext(ctx, "SELECT task_id FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=? AND `round`=?", g.view.NamespaceID, g.view.ID, row.view.ActivationID, p.UsedRounds).Scan(&lastTask); err != nil {
|
||||
return p, "", err
|
||||
}
|
||||
return p, lastTask, nil
|
||||
}
|
||||
|
||||
func (r *WorkflowRuntime) advancePoll(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, activated time.Time, allow bool, now time.Time) (bool, error) {
|
||||
p, lastTask, err := readWorkflowPollState(ctx, tx, g, row, activated)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
node, _ := g.program.Step(row.view.Key)
|
||||
policy := node.Poll
|
||||
if policy == nil || p.MaxRounds != policy.MaxRounds || p.MaxAttemptsPerRound != policy.Task.Retry.MaxAttempts || p.IntervalMS != policy.IntervalMS || p.UsedRounds < 1 || !p.DeadlineAt.Equal(minimumTime(g.view.DeadlineAt, activated.Add(time.Duration(policy.TimeoutMS)*time.Millisecond))) {
|
||||
return false, errors.New("workflow poll policy mismatch")
|
||||
}
|
||||
if p.ResolvedAt != nil {
|
||||
return false, errors.New("unsettled poll step has resolved budget")
|
||||
}
|
||||
var count, minRound, maxRound uint32
|
||||
if err := tx.QueryRowContext(ctx, "SELECT COUNT(*),COALESCE(MIN(`round`),0),COALESCE(MAX(`round`),0) FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=?", c.ns, c.workflow, row.view.ActivationID).Scan(&count, &minRound, &maxRound); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if count != p.UsedRounds || minRound != 1 || maxRound != p.UsedRounds {
|
||||
return false, errors.New("workflow poll rounds incomplete")
|
||||
}
|
||||
var lastTask core.ID
|
||||
if err := tx.QueryRowContext(ctx, "SELECT task_id FROM weave_poll_rounds WHERE namespace_id=? AND workflow_id=? AND activation_id=? AND `round`=?", c.ns, c.workflow, row.view.ActivationID, p.UsedRounds).Scan(&lastTask); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if p.CurrentTaskID != nil {
|
||||
if *p.CurrentTaskID != lastTask || p.NextPollAt != nil || row.view.State != weave.StepActive {
|
||||
return false, errors.New("workflow in-flight poll mismatch")
|
||||
|
||||
@@ -29,8 +29,8 @@ func NewWorkflowRuntime(db *sql.DB, node core.ID) (*WorkflowRuntime, error) {
|
||||
|
||||
type WorkflowRuntimeStats struct {
|
||||
Examined int // Examined 本批取出的持久化候选数，包含被其他节点先处理或跳过的记录。
|
||||
Consumed int // Consumed 已提交确认或隔离的内部结果数，事务回滚不计入。
|
||||
Advanced int // Advanced 已提交推进的根或等待检查点数，不等于业务任务执行次数。
|
||||
Consumed int // Consumed 以Inbox候选为入口提交确认或隔离的数量；时钟内消费计入Advanced，回滚不计入。
|
||||
Advanced int // Advanced 已提交推进的根、等待检查点或过期信号数，不等于业务任务执行次数。
|
||||
}
|
||||
|
||||
type workflowCandidate struct {
|
||||
@@ -39,7 +39,7 @@ type workflowCandidate struct {
|
||||
revision uint64
|
||||
}
|
||||
|
||||
// Tick 同时读取两种到期索引，各取最多limit项再全局选前limit项；不先锁Inbox再反锁父资源。
|
||||
// Tick 同时读取三种到期索引，各取最多limit项再全局选前limit项；不先锁Inbox或信号再反锁父资源。
|
||||
func (r *WorkflowRuntime) Tick(ctx context.Context, limit int) (WorkflowRuntimeStats, error) {
|
||||
var stats WorkflowRuntimeStats
|
||||
if limit < 1 || limit > 100 {
|
||||
@@ -47,7 +47,8 @@ func (r *WorkflowRuntime) Tick(ctx context.Context, limit int) (WorkflowRuntimeS
|
||||
}
|
||||
rows, err := r.db.QueryContext(ctx, `(SELECT 'INBOX' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_inbox WHERE state='PENDING' AND available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?)
|
||||
UNION ALL (SELECT 'ROOT' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_pending_advances WHERE available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?)
|
||||
ORDER BY available_at,namespace_id,id LIMIT ?`, limit, limit, limit)
|
||||
UNION ALL (SELECT 'SIGNAL' kind,namespace_id,id,workflow_id,revision,available_at FROM weave_signals WHERE state='PENDING' AND available_at<=UTC_TIMESTAMP(6) ORDER BY available_at,namespace_id,id LIMIT ?)
|
||||
ORDER BY available_at,namespace_id,id LIMIT ?`, limit, limit, limit, limit)
|
||||
if err != nil {
|
||||
return stats, err
|
||||
}
|
||||
@@ -95,6 +96,8 @@ func (r *WorkflowRuntime) deferCandidate(ctx context.Context, c workflowCandidat
|
||||
q := `UPDATE weave_pending_advances SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=?`
|
||||
if c.kind == "INBOX" {
|
||||
q = `UPDATE weave_inbox SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),error_code='RUNTIME_UNAVAILABLE',revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=? AND state='PENDING'`
|
||||
} else if c.kind == "SIGNAL" {
|
||||
q = `UPDATE weave_signals SET available_at=DATE_ADD(UTC_TIMESTAMP(6),INTERVAL 30 SECOND),revision=revision+1,updated_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=? AND revision=? AND state='PENDING'`
|
||||
}
|
||||
_, err := r.db.ExecContext(ctx, q, c.ns, c.id, c.revision)
|
||||
return err
|
||||
@@ -115,6 +118,10 @@ func (r *WorkflowRuntime) process(ctx context.Context, c workflowCandidate, skip
|
||||
done, err = r.consume(ctx, tx, g, c, allow, now)
|
||||
return err
|
||||
}
|
||||
if c.kind == "SIGNAL" {
|
||||
done, err = r.expireWorkflowSignal(ctx, tx, g, c, allow, now)
|
||||
return err
|
||||
}
|
||||
var step core.ID
|
||||
var expected uint64
|
||||
var raw []byte
|
||||
@@ -133,7 +140,7 @@ func (r *WorkflowRuntime) process(ctx context.Context, c workflowCandidate, skip
|
||||
if json.Unmarshal(raw, &checkpoint) != nil || checkpoint.SchemaVersion != 1 {
|
||||
return errors.New("workflow checkpoint invalid")
|
||||
}
|
||||
if checkpoint.Kind == "TIMER" || checkpoint.Kind == "POLL" {
|
||||
if checkpoint.Kind == "TIMER" || checkpoint.Kind == "POLL" || checkpoint.Kind == "SIGNAL" {
|
||||
done, err = r.advanceClock(ctx, tx, g, c, step, expected, checkpoint, allow, now)
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -11,8 +11,8 @@ import (
|
||||
weave "git.yuntuo.hk/k2safe/orbit/weave/model"
|
||||
)
|
||||
|
||||
// lockWorkflowGraph 只由内部运行器调用。身份→应用额度→排序队列→实例→门控，
|
||||
// 不用启动凭据继续授权；停用身份仍收集在途结果，但禁止创建新工作。
|
||||
// lockWorkflowGraph 由内部运行器和已经认证的信号受理事务复用。身份→应用额度→排序队列→实例→门控，
|
||||
// 运行器不用启动凭据继续授权；停用身份仍收集在途结果，但禁止创建新工作。
|
||||
func lockWorkflowGraph(ctx context.Context, tx *sql.Tx, ns, id core.ID, skip bool) (*workflowGraph, bool, error) {
|
||||
var enabled bool
|
||||
if err := tx.QueryRowContext(ctx, `SELECT n.enabled AND a.enabled FROM orbit_namespaces n JOIN orbit_applications a ON a.id=n.application_id WHERE n.id=? FOR SHARE`, ns).Scan(&enabled); err != nil {
|
||||
|
||||
@@ -211,4 +211,55 @@ func TestIntegrationWorkflowRuntimeProcess(t *testing.T) {
|
||||
t.Fatal("poll takeover lost budget or duplicated round")
|
||||
}
|
||||
t.Log("all controllers exited before timer expiry and poll PENDING delivery; two fresh processes preserved original wake/deadline and completed timer plus exactly two query rounds")
|
||||
for _, p := range processes {
|
||||
stop(p)
|
||||
}
|
||||
early := runtimeStart(t, f, signalSpec(f.resources.QueueID, true))
|
||||
grantWorkflowSignal(f)
|
||||
earlySignal := sendSignal(t, f, early, signalInput(early))
|
||||
ref = retryStart(t, f)
|
||||
retryFinish(t, f, ref, "SUCCEEDED", false)
|
||||
if f.count("weave_waits", "workflow_id=?", early.ID) != 0 || f.count("weave_signals", "id=? AND state='PENDING'", earlySignal.Signal.ID) != 1 {
|
||||
t.Fatal("early signal did not remain durable during all-stop")
|
||||
}
|
||||
start("signal-replacement-a")
|
||||
start("signal-replacement-b")
|
||||
wait(func() bool {
|
||||
current, err := api.Get(ctx, f.token, early.ID)
|
||||
return err == nil && current.State == "SUCCEEDED"
|
||||
}, "early signal after process restart")
|
||||
if f.count("weave_signals", "id=? AND state='CONSUMED'", earlySignal.Signal.ID) != 1 || f.count("pulse_tasks", "1=1") != 7 {
|
||||
t.Fatal("early signal restart duplicated work")
|
||||
}
|
||||
for _, p := range processes {
|
||||
stop(p)
|
||||
}
|
||||
final := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID))
|
||||
grantWorkflowSignal(f)
|
||||
ref = retryStart(t, f)
|
||||
permits := f.used()
|
||||
sendSignal(t, f, final, pollSignalInput(final))
|
||||
start("final-signal-selection-a")
|
||||
start("final-signal-selection-b")
|
||||
wait(func() bool { return f.get(ref.TaskID).CancelRequestedAt != nil }, "signal cancellation intent")
|
||||
if f.used() != permits || f.get(ref.TaskID).State != "RUNNING" {
|
||||
t.Fatal("process released running query on final signal")
|
||||
}
|
||||
for _, p := range processes {
|
||||
stop(p)
|
||||
}
|
||||
finishQueryOnly(t, f, ref, "PENDING")
|
||||
if f.count("weave_inbox", "source_task_id=? AND state='PENDING'", ref.TaskID) != 1 {
|
||||
t.Fatal("query settlement missing with all controllers stopped")
|
||||
}
|
||||
start("final-signal-settlement-a")
|
||||
start("final-signal-settlement-b")
|
||||
wait(func() bool {
|
||||
current, err := api.Get(ctx, f.token, final.ID)
|
||||
return err == nil && current.State == "SUCCEEDED"
|
||||
}, "selected final signal and query settlement after restart")
|
||||
if f.used() != 0 || f.count("pulse_tasks", "1=1") != 8 || f.count("weave_poll_states", "workflow_id=? AND used_rounds=1 AND resolved_at IS NOT NULL", final.ID) != 1 || f.count("weave_pending_advances", "1=1") != 0 {
|
||||
t.Fatal("final signal restart lost winner or issued an extra round")
|
||||
}
|
||||
t.Log("early signal persisted before wait activation across all-stop; final signal selected with running query kept permits, then survived another all-stop and accepted PENDING settlement without issuing a second query")
|
||||
}
|
||||
|
||||
315
internal/storage/mysql/workflow_signal_runtime.go
Normal file
315
internal/storage/mysql/workflow_signal_runtime.go
Normal file
@@ -0,0 +1,315 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/orbit/internal/jsonvalue"
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/engine"
|
||||
weave "git.yuntuo.hk/k2safe/orbit/weave/model"
|
||||
)
|
||||
|
||||
// driveWorkflowSignal 同时用于等待时钟和当前查询结果；handled=false时继续原查询分类。
|
||||
// reason为空表示内部已消费Inbox并写入审计，调用方不能再次确认同一来源。
|
||||
func (r *WorkflowRuntime) driveWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, source *workflowResult, allow bool, now time.Time) (handled, changed bool, reason string, err error) {
|
||||
// 来源矛盾单独转核查；任何SQL故障仍回滚整个事务并保留原来源。
|
||||
if _, err := tx.ExecContext(ctx, `SAVEPOINT workflow_signal_source`); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
before := cloneWorkflowGraph(g)
|
||||
handled, changed, reason, err = r.applyWorkflowSignal(ctx, tx, g, row, c, source, allow, now)
|
||||
var invalid workflowSourceError
|
||||
if !errors.As(err, &invalid) {
|
||||
return
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, `ROLLBACK TO SAVEPOINT workflow_signal_source`); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
*g = *before
|
||||
row = g.steps[row.view.Key]
|
||||
if g.view.State != weave.InstanceNeedsReview {
|
||||
if err := holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, string(invalid), now); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
}
|
||||
var current *core.ID
|
||||
if row.view.Kind == "POLL" {
|
||||
if err := tx.QueryRowContext(ctx, `SELECT current_task_id FROM weave_poll_states WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(¤t); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if source != nil || current == nil {
|
||||
if _, err := tx.ExecContext(ctx, `UPDATE weave_poll_states SET current_task_id=NULL,next_poll_at=NULL,resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, now, now, g.view.NamespaceID, g.view.ID, row.view.ActivationID); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
}
|
||||
}
|
||||
// 已发出的查询仍由Pulse结清，不能因信号损坏而假设外部执行结束。
|
||||
if row.view.Kind != "POLL" || source != nil || current == nil {
|
||||
if err := saveWorkflowResultStep(ctx, tx, g, row, weave.StepNeedsReview, nil, now); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
}
|
||||
if _, err := tx.ExecContext(ctx, `UPDATE weave_signals s JOIN weave_activations a ON a.namespace_id=s.namespace_id AND a.workflow_id=s.workflow_id AND a.sequence=s.target_activation_sequence SET s.state='REJECTED',s.disposition=?,s.revision=s.revision+1,s.updated_at=? WHERE s.namespace_id=? AND s.workflow_id=? AND s.target_step_key=? AND a.id=? AND a.step_id=? AND s.state='PENDING'`, string(invalid), now, g.view.NamespaceID, g.view.ID, row.view.Key, row.view.ActivationID, row.view.ID); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if err := deleteWorkflowClock(ctx, tx, g, row); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
return true, true, string(invalid), nil
|
||||
}
|
||||
|
||||
func (r *WorkflowRuntime) applyWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, source *workflowResult, allow bool, now time.Time) (handled, changed bool, reason string, err error) {
|
||||
w, err := readWorkflowSignalWait(ctx, tx, g, row)
|
||||
if err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
node, _ := g.program.Step(row.view.Key)
|
||||
var poll weave.PollState
|
||||
if node.Poll != nil {
|
||||
var last core.ID
|
||||
poll, last, err = readWorkflowPollState(ctx, tx, g, row, w.activated)
|
||||
if err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if err := validatePollTaskOwner(ctx, tx, g, row, last); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if _, err := pollTaskInput(ctx, tx, g, row, node.Poll.Task); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if poll.CurrentTaskID != nil {
|
||||
if *poll.CurrentTaskID != last || row.view.State != weave.StepActive || poll.NextPollAt != nil {
|
||||
return false, false, "", errors.New("signal poll current round invalid")
|
||||
}
|
||||
if err := validatePollTaskOwner(ctx, tx, g, row, last); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if source != nil && source.task != last {
|
||||
return false, false, "", nil
|
||||
}
|
||||
} else {
|
||||
if source != nil {
|
||||
return false, false, "", nil
|
||||
}
|
||||
if row.view.State != weave.StepWaiting || poll.NextPollAt == nil {
|
||||
return false, false, "", errors.New("signal poll continuation missing")
|
||||
}
|
||||
var disposition string
|
||||
if err := tx.QueryRowContext(ctx, `SELECT error_code FROM weave_inbox WHERE namespace_id=? AND workflow_id=? AND source_task_id=? AND source_activation_id=? AND state='PROCESSED' ORDER BY source_revision DESC LIMIT 1`, g.view.NamespaceID, g.view.ID, last, row.view.ActivationID).Scan(&disposition); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if disposition != "POLL_PENDING" {
|
||||
return false, false, "", errors.New("signal poll previous round not pending")
|
||||
}
|
||||
}
|
||||
}
|
||||
v, err := selectedWorkflowSignal(ctx, tx, g, row, w, now)
|
||||
if err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if v == nil {
|
||||
if node.Poll != nil {
|
||||
return false, false, "", nil
|
||||
}
|
||||
if w.state != "WAITING" {
|
||||
return false, false, "", errors.New("signal step has settled wait without winner")
|
||||
}
|
||||
if now.Before(w.deadline) {
|
||||
return true, false, "", saveWorkflowClock(ctx, tx, g, row, "SIGNAL", row.view.Revision, w.deadline)
|
||||
}
|
||||
res, err := tx.ExecContext(ctx, `UPDATE weave_waits SET state='EXPIRED',resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='WAITING'`, now, now, g.view.NamespaceID, w.id)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "SIGNAL_DEADLINE", now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if err := saveWorkflowResultStep(ctx, tx, g, row, weave.StepNeedsReview, nil, now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if err := deleteWorkflowClock(ctx, tx, g, row); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
return true, true, "SIGNAL_DEADLINE", holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, "SIGNAL_DEADLINE", now)
|
||||
}
|
||||
if node.Poll != nil && source == nil && poll.CurrentTaskID != nil {
|
||||
// 已落库查询结果必须按原事件时间参与竞争，不能由扫描先后决定赢家。
|
||||
task, err := readTask(ctx, tx, g.view.NamespaceID, *poll.CurrentTaskID, false)
|
||||
if err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if task.QueueID != node.Poll.Task.QueueID {
|
||||
return false, false, "", errors.New("signal query queue invalid")
|
||||
}
|
||||
if task.State.Terminal() || task.State == "NEEDS_REVIEW" {
|
||||
pending := workflowCandidate{kind: "INBOX", ns: g.view.NamespaceID, workflow: g.view.ID}
|
||||
var due time.Time
|
||||
if err := tx.QueryRowContext(ctx, `SELECT id,revision,available_at FROM weave_inbox WHERE namespace_id=? AND workflow_id=? AND source_task_id=? AND source_activation_id=? AND state='PENDING' ORDER BY source_revision DESC LIMIT 1`, g.view.NamespaceID, g.view.ID, poll.CurrentTaskID, row.view.ActivationID).Scan(&pending.id, &pending.revision, &due); err != nil {
|
||||
return false, false, "", err
|
||||
}
|
||||
if due.After(now) {
|
||||
return true, false, "", saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision, due)
|
||||
}
|
||||
done, err := r.consume(ctx, tx, g, pending, allow, now)
|
||||
return true, done, "", err
|
||||
}
|
||||
}
|
||||
if node.Poll != nil && source != nil && w.winner == nil && source.snapshot.State == "SUCCEEDED" && !source.snapshot.EffectUncertain && source.occurred.Before(w.deadline) && !source.occurred.After(v.AcceptedAt) {
|
||||
outcome, err := g.program.ClassifyPoll(row.view.Key, source.snapshot.Result)
|
||||
if err == nil && outcome != engine.PollPending {
|
||||
return false, false, "", nil
|
||||
} // 较早的最终查询先胜出；PENDING不争夺终结权。
|
||||
}
|
||||
newWinner := w.winner == nil
|
||||
if err := claimWorkflowSignal(ctx, tx, g, &w, v, now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if err := rejectOtherWorkflowSignals(ctx, tx, g, row, w.sequence, "WAIT_SETTLED", now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if node.Poll != nil && poll.CurrentTaskID != nil && source == nil {
|
||||
task, err := readExecutionTask(ctx, tx, g.view.NamespaceID, *poll.CurrentTaskID)
|
||||
if err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if _, err := cancelExecutionTask(ctx, tx, g.view.NamespaceID, task, now, "WORKFLOW_SIGNAL_WON"); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if !now.Before(poll.DeadlineAt) && g.view.State != weave.InstanceNeedsReview {
|
||||
if err := holdWorkflow(ctx, tx, g, weave.InstanceNeedsReview, "POLL_SIGNAL_INFLIGHT_DEADLINE", now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
newWinner = true
|
||||
}
|
||||
if err := saveWorkflowClock(ctx, tx, g, row, "POLL", row.view.Revision, now.Add(time.Second)); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if newWinner {
|
||||
return true, true, "POLL_SIGNAL_SELECTED", nil
|
||||
}
|
||||
return true, false, "", nil
|
||||
}
|
||||
state, output, reason := weave.StepSucceeded, v.Payload, "SIGNAL_CONSUMED"
|
||||
if node.Poll != nil {
|
||||
outcome, err := g.program.ClassifyPoll(row.view.Key, v.Payload)
|
||||
if err != nil || outcome == engine.PollPending {
|
||||
return true, false, "", workflowSourceError("POLL_FINAL_SIGNAL_INVALID")
|
||||
}
|
||||
switch outcome {
|
||||
case engine.PollFailed:
|
||||
state, output, reason = weave.StepFailed, nil, "POLL_SIGNAL_FAILED"
|
||||
case engine.PollNeedsReview:
|
||||
state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_REVIEW"
|
||||
default:
|
||||
reason = "POLL_SIGNAL_SUCCEEDED"
|
||||
}
|
||||
if source != nil {
|
||||
if source.snapshot.EffectUncertain || source.snapshot.State == "NEEDS_REVIEW" {
|
||||
state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_UNCERTAIN"
|
||||
} else if source.snapshot.State == "SUCCEEDED" {
|
||||
query, err := g.program.ClassifyPoll(row.view.Key, source.snapshot.Result)
|
||||
if err != nil {
|
||||
state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_INVALID"
|
||||
} else if query != engine.PollPending {
|
||||
left, err := jsonvalue.Canonical(source.snapshot.Result)
|
||||
if err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
right, err := jsonvalue.Canonical(v.Payload)
|
||||
if err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if !bytes.Equal(left, right) {
|
||||
state, output, reason = weave.StepNeedsReview, nil, "POLL_SIGNAL_QUERY_CONFLICT"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
res, err := tx.ExecContext(ctx, `UPDATE weave_poll_states SET current_task_id=NULL,next_poll_at=NULL,resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND resolved_at IS NULL`, now, now, g.view.NamespaceID, poll.ID)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
}
|
||||
if err := saveWorkflowResultStep(ctx, tx, g, row, state, output, now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if err := deleteWorkflowClock(ctx, tx, g, row); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
if state == weave.StepNeedsReview || state == weave.StepFailed {
|
||||
next := weave.InstanceNeedsReview
|
||||
if state == weave.StepFailed && (g.version.Specification.FailurePolicy == "FAIL_AFTER_SETTLED" || g.view.CancelRequestedAt != nil) {
|
||||
next = weave.InstanceCancelling
|
||||
}
|
||||
if err := holdWorkflow(ctx, tx, g, next, reason, now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
} else if err := r.progress(ctx, tx, g, allow, now); err != nil {
|
||||
return true, false, "", err
|
||||
}
|
||||
return true, true, reason, nil
|
||||
}
|
||||
|
||||
func (r *WorkflowRuntime) advanceSignalClock(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, c workflowCandidate, allow bool, now time.Time) (bool, error) {
|
||||
handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if !handled {
|
||||
return false, errors.New("signal clock did not identify a wait")
|
||||
}
|
||||
if changed && reason != "" {
|
||||
return true, r.audit(ctx, tx, g, c.id, reason, now)
|
||||
}
|
||||
return changed, nil
|
||||
}
|
||||
|
||||
// expireWorkflowSignal 对早到但未匹配的信号使用总截止索引；已按时匹配的信号仍先完成实际消费。
|
||||
func (r *WorkflowRuntime) expireWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, c workflowCandidate, allow bool, now time.Time) (bool, error) {
|
||||
var due time.Time
|
||||
if err := tx.QueryRowContext(ctx, `SELECT available_at FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, c.ns, c.workflow, c.id).Scan(&due); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if due.After(now) {
|
||||
return false, nil
|
||||
}
|
||||
v, err := readWorkflowSignal(ctx, tx, c.ns, c.workflow, c.id)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if v.State != "PENDING" || now.Before(v.ExpiresAt) {
|
||||
return false, nil
|
||||
}
|
||||
row := g.steps[v.TargetStepKey]
|
||||
if row != nil && row.view.ActivationID != nil && !g.view.State.Terminal() && (row.view.State == weave.StepWaiting || row.view.State == weave.StepActive) {
|
||||
w, err := readWorkflowSignalWait(ctx, tx, g, row)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if w.sequence == v.TargetActivationSequence && w.correlation == v.CorrelationKey && w.contract.Name == v.Name && w.contract.PayloadVersion == v.PayloadVersion && v.AcceptedAt.Before(w.deadline) {
|
||||
handled, changed, reason, err := r.driveWorkflowSignal(ctx, tx, g, row, c, nil, allow, now)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if handled {
|
||||
// 若匹配到另一条更早的信号或正等待结果退避，避免过期索引持续抢占批次。
|
||||
if _, err := tx.ExecContext(ctx, `UPDATE weave_signals SET available_at=DATE_ADD(?,INTERVAL 30 SECOND),revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='PENDING'`, now, now, c.ns, c.id); err != nil {
|
||||
return false, err
|
||||
}
|
||||
if changed && reason != "" {
|
||||
return true, r.audit(ctx, tx, g, c.id, reason, now)
|
||||
}
|
||||
return changed, nil
|
||||
}
|
||||
}
|
||||
}
|
||||
res, err := tx.ExecContext(ctx, `UPDATE weave_signals SET state='EXPIRED',disposition='UNMATCHED_DEADLINE',revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND state='PENDING' AND revision=?`, now, c.ns, c.id, v.Revision)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return false, err
|
||||
}
|
||||
return true, r.audit(ctx, tx, g, c.id, "SIGNAL_EXPIRED", now)
|
||||
}
|
||||
@@ -0,0 +1,361 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/pulse/worker"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/definition"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/instance"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/signal"
|
||||
)
|
||||
|
||||
func sendSignal(t *testing.T, f *executionFixture, v instance.View, in signal.Input) signal.Result {
|
||||
t.Helper()
|
||||
out, err := signal.New(NewWorkflowSignalStore(f.db)).Send(f.ctx, f.token, v.ID, string(executionID(t)), in)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
func finalSignalSpec(queue core.ID) definition.Specification {
|
||||
s := pollingSpec(queue)
|
||||
p := s.Body.Steps[0].Poll
|
||||
p.Task.ResultType.Enum = append(p.Task.ResultType.Enum, "FAILED")
|
||||
p.Failed = []string{"FAILED"}
|
||||
p.FinalSignal = &definition.Signal{Name: "external.done", PayloadVersion: "v1", Correlation: definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "workflow", Path: []string{"business_key"}}}, PayloadType: p.Task.ResultType, TimeoutMS: p.TimeoutMS}
|
||||
s.Body.OutputType = p.Task.ResultType
|
||||
return s
|
||||
}
|
||||
func pollSignalInput(v instance.View) signal.Input {
|
||||
in := signalInput(v)
|
||||
in.TargetStepKey = "query"
|
||||
in.Payload = json.RawMessage(`"DONE"`)
|
||||
return in
|
||||
}
|
||||
func finishQueryOnly(t *testing.T, f *executionFixture, ref worker.AttemptRef, result string) {
|
||||
t.Helper()
|
||||
raw, _ := json.Marshal(result)
|
||||
if _, err := f.a.Finish(f.ctx, f.token, f.binding, executionID(t), worker.FinishInput{Ref: ref, Outcome: "SUCCEEDED", Result: raw}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestIntegrationWorkflowSignalRuntime(t *testing.T) {
|
||||
dsn := os.Getenv("ORBIT_TEST_MYSQL_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL")
|
||||
}
|
||||
ctx, stop := context.WithTimeout(context.Background(), 150*time.Second)
|
||||
defer stop()
|
||||
db, err := Open(ctx, dsn, 32, 8)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
if err := Migrate(ctx, db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Run("wait_then_twelve_consumers_one_successor", func(t *testing.T) {
|
||||
f, v, api := signalStart(t, ctx, db, false)
|
||||
c := clockCandidate(t, f, v.ID, "signal")
|
||||
if runClock(t, f, c) || f.count("pulse_tasks", "1=1") != 0 {
|
||||
t.Fatal("empty wait advanced")
|
||||
}
|
||||
in := signalInput(v)
|
||||
key := string(executionID(t))
|
||||
out, err := api.Send(ctx, f.token, v.ID, key, in)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
other := sendSignal(t, f, v, in)
|
||||
c = clockCandidate(t, f, v.ID, "signal")
|
||||
var wg sync.WaitGroup
|
||||
errs := make(chan error, 12)
|
||||
results := make(chan bool, 12)
|
||||
for i := 0; i < 12; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
done, err := newRuntime(t, db).process(ctx, c, false)
|
||||
errs <- err
|
||||
results <- done
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(errs)
|
||||
close(results)
|
||||
for err := range errs {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
changed := 0
|
||||
for done := range results {
|
||||
if done {
|
||||
changed++
|
||||
}
|
||||
}
|
||||
if changed != 1 || f.count("pulse_tasks", "1=1") != 1 || f.count("weave_signals", "state='CONSUMED'") != 1 || f.count("weave_signals", "state='REJECTED'") != 1 {
|
||||
t.Fatal("duplicate signal advancement", changed)
|
||||
}
|
||||
current, err := api.Get(ctx, f.token, v.ID, out.Signal.ID)
|
||||
if err != nil || current.State != "CONSUMED" || current.ConsumedWaitID == nil {
|
||||
t.Fatal("signal not consumed", err)
|
||||
}
|
||||
rejected, err := api.Get(ctx, f.token, v.ID, other.Signal.ID)
|
||||
if err != nil || rejected.Disposition != "WAIT_SETTLED" {
|
||||
t.Fatal("competing signal not retained", err)
|
||||
}
|
||||
replay, err := api.Send(ctx, f.token, v.ID, key, in)
|
||||
if err != nil || !replay.Deduplicated || replay.Signal.State != "PENDING" || replay.Signal.Revision != 1 {
|
||||
t.Fatal("original receipt was mutated", err)
|
||||
}
|
||||
var output string
|
||||
if err := db.QueryRowContext(ctx, `SELECT output FROM weave_activations WHERE namespace_id=? AND id=(SELECT current_activation_id FROM weave_steps WHERE namespace_id=? AND workflow_id=? AND `+"`key`"+`='signal')`, f.resources.NamespaceID, f.resources.NamespaceID, v.ID).Scan(&output); err != nil || !strings.Contains(output, "9007199254740993") {
|
||||
t.Fatal("signal number changed", err)
|
||||
}
|
||||
})
|
||||
t.Run("early_signal_survives_activation_and_fresh_runtime", func(t *testing.T) {
|
||||
f, v, _ := signalStart(t, ctx, db, true)
|
||||
sendSignal(t, f, v, signalInput(v))
|
||||
if f.count("weave_waits", "1=1") != 0 {
|
||||
t.Fatal("early signal activated blocked step")
|
||||
}
|
||||
ref := retryStart(t, f)
|
||||
retryFinish(t, f, ref, "SUCCEEDED", false)
|
||||
runtimeConsume(t, f, newRuntime(t, db), ref.TaskID)
|
||||
if !runClock(t, f, clockCandidate(t, f, v.ID, "signal")) {
|
||||
t.Fatal("early signal not resumed")
|
||||
}
|
||||
out := runtimeView(t, f, v.ID, "SUCCEEDED")
|
||||
if !strings.Contains(string(out.Output), "9007199254740993") || f.count("pulse_tasks", "1=1") != 1 {
|
||||
t.Fatal("early result lost or new work created")
|
||||
}
|
||||
})
|
||||
t.Run("paused_collects_and_persists_root", func(t *testing.T) {
|
||||
f, v, _ := signalStart(t, ctx, db, false)
|
||||
f.exec(`UPDATE weave_instances SET paused=TRUE WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, v.ID)
|
||||
f.exec(`UPDATE pulse_execution_gates SET paused=TRUE WHERE namespace_id=?`, f.resources.NamespaceID)
|
||||
sendSignal(t, f, v, signalInput(v))
|
||||
runClock(t, f, clockCandidate(t, f, v.ID, "signal"))
|
||||
if f.count("weave_signals", "state='CONSUMED'") != 1 || f.count("pulse_tasks", "1=1") != 0 || f.count("weave_pending_advances", "1=1") != 1 {
|
||||
t.Fatal("pause dropped evidence or started successor")
|
||||
}
|
||||
f.exec(`UPDATE weave_instances SET paused=FALSE WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, v.ID)
|
||||
f.exec(`UPDATE pulse_execution_gates SET paused=FALSE WHERE namespace_id=?`, f.resources.NamespaceID)
|
||||
runtimeAdvance(t, f, newRuntime(t, db), v.ID)
|
||||
if f.count("pulse_tasks", "1=1") != 1 {
|
||||
t.Fatal("root did not resume")
|
||||
}
|
||||
})
|
||||
for _, timely := range []bool{false, true} {
|
||||
name := "deadline_without_signal"
|
||||
if timely {
|
||||
name = "timely_acceptance_after_consumer_deadline"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
f := newExecutionFixture(t, ctx, db)
|
||||
spec := signalSpec(f.resources.QueueID, false)
|
||||
spec.Body.Steps[0].Signal.TimeoutMS = 1000
|
||||
v := runtimeStart(t, f, spec)
|
||||
grantWorkflowSignal(f)
|
||||
if timely {
|
||||
sendSignal(t, f, v, signalInput(v))
|
||||
}
|
||||
c := clockCandidate(t, f, v.ID, "signal")
|
||||
if !timely {
|
||||
runClock(t, f, c)
|
||||
}
|
||||
var due time.Time
|
||||
if err := db.QueryRowContext(ctx, `SELECT deadline_at FROM weave_waits WHERE namespace_id=?`, f.resources.NamespaceID).Scan(&due); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for time.Now().Before(due.Add(20 * time.Millisecond)) {
|
||||
time.Sleep(20 * time.Millisecond)
|
||||
}
|
||||
if !runClock(t, f, c) {
|
||||
t.Fatal("deadline not handled")
|
||||
}
|
||||
if timely {
|
||||
runtimeView(t, f, v.ID, "RUNNING")
|
||||
if f.count("pulse_tasks", "1=1") != 1 {
|
||||
t.Fatal("timely signal lost")
|
||||
}
|
||||
} else {
|
||||
runtimeView(t, f, v.ID, "NEEDS_REVIEW")
|
||||
if f.count("weave_waits", "state='EXPIRED'") != 1 || f.count("pulse_tasks", "1=1") != 0 {
|
||||
t.Fatal("expired wait progressed")
|
||||
}
|
||||
late := sendSignal(t, f, v, signalInput(v))
|
||||
if late.Signal.State != "REJECTED" {
|
||||
t.Fatal("settled wait accepted late signal")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Run("audit_failure_rolls_back_consumption_wait_successor", func(t *testing.T) {
|
||||
f, v, _ := signalStart(t, ctx, db, false)
|
||||
sendSignal(t, f, v, signalInput(v))
|
||||
c := clockCandidate(t, f, v.ID, "signal")
|
||||
f.exec(`CREATE TRIGGER test_signal_consume_failure BEFORE INSERT ON orbit_audit_events FOR EACH ROW SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='injected signal consume failure'`)
|
||||
defer f.exec(`DROP TRIGGER IF EXISTS test_signal_consume_failure`)
|
||||
if done, err := newRuntime(t, db).process(ctx, c, false); err == nil || done {
|
||||
t.Fatal("audit failure committed")
|
||||
}
|
||||
if f.count("weave_signals", "state='PENDING'") != 1 || f.count("weave_waits", "state='WAITING' AND winner_signal_id IS NULL") != 1 || f.count("pulse_tasks", "1=1") != 0 {
|
||||
t.Fatal("partial signal transaction")
|
||||
}
|
||||
f.exec(`DROP TRIGGER test_signal_consume_failure`)
|
||||
if !runClock(t, f, c) || f.count("pulse_tasks", "1=1") != 1 {
|
||||
t.Fatal("rollback not recoverable")
|
||||
}
|
||||
})
|
||||
for _, mutation := range []string{"payload", "actor", "accepted_at"} {
|
||||
t.Run("tampered_"+mutation+"_requires_review", func(t *testing.T) {
|
||||
f, v, _ := signalStart(t, ctx, db, false)
|
||||
out := sendSignal(t, f, v, signalInput(v))
|
||||
switch mutation {
|
||||
case "payload":
|
||||
f.exec(`UPDATE weave_signals SET payload='{"n":1}' WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID)
|
||||
case "actor":
|
||||
f.exec(`UPDATE weave_signals SET actor_id='spoofed' WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID)
|
||||
default:
|
||||
f.exec(`UPDATE weave_signals SET accepted_at=DATE_SUB(accepted_at,INTERVAL 1 SECOND) WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, out.Signal.ID)
|
||||
}
|
||||
if !runClock(t, f, clockCandidate(t, f, v.ID, "signal")) {
|
||||
t.Fatal("invalid source not handled")
|
||||
}
|
||||
runtimeView(t, f, v.ID, "NEEDS_REVIEW")
|
||||
if f.count("weave_signals", "state='REJECTED'") != 1 || f.count("weave_steps", "state='NEEDS_REVIEW'") != 1 || f.count("pulse_tasks", "1=1") != 0 {
|
||||
t.Fatal("mutable signal authorized work")
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Run("early_mismatch_expires_without_changing_deadline_on_backoff", func(t *testing.T) {
|
||||
f, v, _ := signalStart(t, ctx, db, true)
|
||||
in := signalInput(v)
|
||||
in.CorrelationKey += "wrong"
|
||||
out := sendSignal(t, f, v, in)
|
||||
c := workflowCandidate{kind: "SIGNAL", ns: f.resources.NamespaceID, workflow: v.ID, id: out.Signal.ID, revision: 1}
|
||||
// 仅故障注入过期扫描条件，不把篡改过的信号用于匹配或推进。
|
||||
f.exec(`UPDATE weave_signals SET expires_at=DATE_SUB(UTC_TIMESTAMP(6),INTERVAL 1 SECOND),available_at=DATE_SUB(UTC_TIMESTAMP(6),INTERVAL 1 SECOND) WHERE namespace_id=? AND id=?`, c.ns, c.id)
|
||||
var expires time.Time
|
||||
if err := db.QueryRowContext(ctx, `SELECT expires_at FROM weave_signals WHERE namespace_id=? AND id=?`, c.ns, c.id).Scan(&expires); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := newRuntime(t, db)
|
||||
if err := r.deferCandidate(ctx, c); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var current, due time.Time
|
||||
if err := db.QueryRowContext(ctx, `SELECT expires_at,available_at FROM weave_signals WHERE namespace_id=? AND id=?`, c.ns, c.id).Scan(¤t, &due); err != nil || !current.Equal(expires) || !due.After(time.Now()) {
|
||||
t.Fatal("backoff extended deadline", err)
|
||||
}
|
||||
if done, err := r.process(ctx, c, false); err != nil || done {
|
||||
t.Fatal("ignored backoff", err)
|
||||
}
|
||||
f.exec(`UPDATE weave_signals SET available_at=UTC_TIMESTAMP(6) WHERE namespace_id=? AND id=?`, c.ns, c.id)
|
||||
if done, err := r.process(ctx, c, false); err != nil || !done {
|
||||
t.Fatal("unmatched expiry failed", err)
|
||||
}
|
||||
if f.count("weave_signals", "state='EXPIRED' AND disposition='UNMATCHED_DEADLINE'") != 1 || f.count("pulse_tasks", "1=1") != 1 {
|
||||
t.Fatal("expiry authorized work")
|
||||
}
|
||||
})
|
||||
t.Run("poll_signal_between_rounds_does_not_issue_query", func(t *testing.T) {
|
||||
f := newClockFixture(t, ctx, db)
|
||||
v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID))
|
||||
grantWorkflowSignal(f)
|
||||
ref := retryStart(t, f)
|
||||
finishPoll(t, f, ref, "PENDING")
|
||||
sendSignal(t, f, v, pollSignalInput(v))
|
||||
runClock(t, f, clockCandidate(t, f, v.ID, "query"))
|
||||
runtimeView(t, f, v.ID, "SUCCEEDED")
|
||||
if f.count("pulse_tasks", "1=1") != 1 || f.count("weave_poll_states", "used_rounds=1 AND resolved_at IS NOT NULL") != 1 || f.used() != 0 {
|
||||
t.Fatal("final signal consumed extra budget")
|
||||
}
|
||||
})
|
||||
t.Run("poll_signal_cancels_unstarted_query", func(t *testing.T) {
|
||||
f := newClockFixture(t, ctx, db)
|
||||
v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID))
|
||||
grantWorkflowSignal(f)
|
||||
sendSignal(t, f, v, pollSignalInput(v))
|
||||
runClock(t, f, clockCandidate(t, f, v.ID, "query"))
|
||||
var task core.ID
|
||||
if err := db.QueryRowContext(ctx, `SELECT id FROM pulse_tasks WHERE namespace_id=?`, f.resources.NamespaceID).Scan(&task); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.get(task).State != "CANCELLED" || f.count("weave_steps", "`key`='query' AND state='ACTIVE'") != 1 {
|
||||
t.Fatal("unstarted query not safely cancelled")
|
||||
}
|
||||
runtimeConsume(t, f, newRuntime(t, db), task)
|
||||
runtimeView(t, f, v.ID, "SUCCEEDED")
|
||||
if f.count("weave_inbox", "state='PROCESSED'") != 1 || f.count("weave_pending_advances", "1=1") != 0 {
|
||||
t.Fatal("cancel result not collected")
|
||||
}
|
||||
})
|
||||
for _, outcome := range []string{"CANCELLED", "PENDING", "DONE", "UNKNOWN", "FAILED", "CONFLICT"} {
|
||||
t.Run("poll_running_signal_then_"+outcome, func(t *testing.T) {
|
||||
f := newClockFixture(t, ctx, db)
|
||||
v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID))
|
||||
grantWorkflowSignal(f)
|
||||
ref := retryStart(t, f)
|
||||
before := f.used()
|
||||
sendSignal(t, f, v, pollSignalInput(v))
|
||||
runClock(t, f, clockCandidate(t, f, v.ID, "query"))
|
||||
task := f.get(ref.TaskID)
|
||||
if task.State != "RUNNING" || task.CancelRequestedAt == nil || f.used() != before || f.count("weave_poll_states", "current_task_id IS NOT NULL AND resolved_at IS NULL") != 1 {
|
||||
t.Fatal("signal prematurely released execution")
|
||||
}
|
||||
switch outcome {
|
||||
case "PENDING", "DONE":
|
||||
finishQueryOnly(t, f, ref, outcome)
|
||||
case "CONFLICT":
|
||||
finishQueryOnly(t, f, ref, "FAILED")
|
||||
default:
|
||||
retryFinish(t, f, ref, outcome, false)
|
||||
}
|
||||
runtimeConsume(t, f, newRuntime(t, db), ref.TaskID)
|
||||
want := "SUCCEEDED"
|
||||
if outcome == "UNKNOWN" || outcome == "CONFLICT" {
|
||||
want = "NEEDS_REVIEW"
|
||||
}
|
||||
runtimeView(t, f, v.ID, want)
|
||||
if f.used() != 0 || f.count("pulse_tasks", "1=1") != 1 || f.count("weave_poll_states", "current_task_id IS NULL AND resolved_at IS NOT NULL") != 1 {
|
||||
t.Fatal("query not settled exactly once")
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, clockFirst := range []bool{true, false} {
|
||||
name := "query_earlier_inbox_first"
|
||||
if clockFirst {
|
||||
name = "query_earlier_clock_first"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
f := newClockFixture(t, ctx, db)
|
||||
v := runtimeStart(t, f, finalSignalSpec(f.resources.QueueID))
|
||||
grantWorkflowSignal(f)
|
||||
ref := retryStart(t, f)
|
||||
finishQueryOnly(t, f, ref, "DONE")
|
||||
in := pollSignalInput(v)
|
||||
in.Payload = json.RawMessage(`"FAILED"`)
|
||||
sendSignal(t, f, v, in)
|
||||
if clockFirst {
|
||||
runClock(t, f, clockCandidate(t, f, v.ID, "query"))
|
||||
} else {
|
||||
runtimeConsume(t, f, newRuntime(t, db), ref.TaskID)
|
||||
}
|
||||
runtimeView(t, f, v.ID, "SUCCEEDED")
|
||||
if f.count("weave_signals", "state='REJECTED' AND disposition='QUERY_SETTLED'") != 1 || f.count("weave_waits", "state='SATISFIED' AND winner_signal_id IS NULL") != 1 {
|
||||
t.Fatal("scan order changed winner")
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Log("signal wait/early/duplicate/deadline/pause/rollback/proof and POLL final-signal settlement verified on real MySQL")
|
||||
}
|
||||
166
internal/storage/mysql/workflow_signal_source.go
Normal file
166
internal/storage/mysql/workflow_signal_source.go
Normal file
@@ -0,0 +1,166 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/definition"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/signal"
|
||||
)
|
||||
|
||||
type workflowSignalWait struct {
|
||||
id core.ID
|
||||
sequence uint32
|
||||
state, correlation string
|
||||
deadline, activated time.Time
|
||||
winner *core.ID
|
||||
contract *definition.Signal
|
||||
}
|
||||
|
||||
// readWorkflowSignalWait 读取已经冻结的等待身份，不能把历史NULL元数据解释成有效信号契约。
|
||||
func readWorkflowSignalWait(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep) (workflowSignalWait, error) {
|
||||
var w workflowSignalWait
|
||||
node, ok := g.program.Step(row.view.Key)
|
||||
w.contract = signalContract(node)
|
||||
if !ok || w.contract == nil || row.view.ActivationID == nil {
|
||||
return w, errors.New("signal step definition invalid")
|
||||
}
|
||||
var count int
|
||||
if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_waits WHERE namespace_id=? AND workflow_id=? AND activation_id=?`, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&count); err != nil {
|
||||
return w, err
|
||||
}
|
||||
if count != 1 {
|
||||
return w, errors.New("signal wait set invalid")
|
||||
}
|
||||
var name, version *string
|
||||
var kind string
|
||||
err := tx.QueryRowContext(ctx, `SELECT w.id,w.kind,w.state,w.signal_name,w.payload_version,w.correlation_key,w.deadline_at,w.winner_signal_id,a.sequence,a.created_at FROM weave_waits w JOIN weave_activations a ON a.namespace_id=w.namespace_id AND a.workflow_id=w.workflow_id AND a.id=w.activation_id AND a.step_id=? WHERE w.namespace_id=? AND w.workflow_id=? AND w.activation_id=? FOR UPDATE`, row.view.ID, g.view.NamespaceID, g.view.ID, row.view.ActivationID).Scan(&w.id, &kind, &w.state, &name, &version, &w.correlation, &w.deadline, &w.winner, &w.sequence, &w.activated)
|
||||
if err != nil {
|
||||
return w, err
|
||||
}
|
||||
if name == nil || version == nil {
|
||||
return w, workflowSourceError("SIGNAL_WAIT_UNVERIFIED")
|
||||
}
|
||||
if w.sequence == 0 || kind != string(row.view.Kind) || *name != w.contract.Name || *version != w.contract.PayloadVersion || !utf8.ValidString(w.correlation) || len(w.correlation) == 0 || len(w.correlation) > 512 || !w.deadline.Equal(minimumTime(g.view.DeadlineAt, w.activated.Add(time.Duration(w.contract.TimeoutMS)*time.Millisecond))) {
|
||||
return w, workflowSourceError("SIGNAL_WAIT_INVALID")
|
||||
}
|
||||
if node.Signal != nil {
|
||||
var input struct {
|
||||
Name string `json:"name"`
|
||||
Version string `json:"payload_version"`
|
||||
Correlation string `json:"correlation"`
|
||||
}
|
||||
if json.Unmarshal(row.input, &input) != nil || input.Name != *name || input.Version != *version || input.Correlation != w.correlation {
|
||||
return w, workflowSourceError("SIGNAL_ACTIVATION_INPUT_INVALID")
|
||||
}
|
||||
}
|
||||
return w, nil
|
||||
}
|
||||
|
||||
// verifyWorkflowSignal 核对不可改写受理审计及请求摘要，防止可变状态读取替换原受理参数或时间。
|
||||
func verifyWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, w workflowSignalWait, v signal.View, now time.Time) error {
|
||||
if v.NamespaceID != g.view.NamespaceID || v.WorkflowID != g.view.ID || v.TargetStepKey != row.view.Key || v.TargetActivationSequence != w.sequence || v.CorrelationKey != w.correlation || v.Name != w.contract.Name || v.PayloadVersion != w.contract.PayloadVersion || v.AcceptedAt.After(now) || !v.AcceptedAt.Before(w.deadline) || !v.ExpiresAt.Equal(g.view.DeadlineAt) || definition.ValidateData(w.contract.PayloadType, v.Payload) != nil {
|
||||
return workflowSourceError("SIGNAL_IDENTITY_INVALID")
|
||||
}
|
||||
var digest core.Digest
|
||||
if err := tx.QueryRowContext(ctx, `SELECT request_digest FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, g.view.NamespaceID, g.view.ID, v.ID).Scan(&digest); err != nil {
|
||||
return err
|
||||
}
|
||||
var count int
|
||||
if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM orbit_audit_events WHERE namespace_id=? AND resource_kind='workflow_signal' AND resource_id=? AND action='workflow.signal'`, g.view.NamespaceID, v.ID).Scan(&count); err != nil {
|
||||
return err
|
||||
}
|
||||
if count != 1 {
|
||||
return workflowSourceError("SIGNAL_ACCEPTANCE_PROOF_MISSING")
|
||||
}
|
||||
var actor, reason string
|
||||
var evidence sql.NullString
|
||||
var raw []byte
|
||||
if err := tx.QueryRowContext(ctx, "SELECT actor_id,reason,evidence_ref,`after` FROM orbit_audit_events WHERE namespace_id=? AND resource_kind='workflow_signal' AND resource_id=? AND action='workflow.signal'", g.view.NamespaceID, v.ID).Scan(&actor, &reason, &evidence, &raw); err != nil {
|
||||
return err
|
||||
}
|
||||
var meta struct {
|
||||
SignalID core.ID `json:"signal_id"`
|
||||
WorkflowID core.ID `json:"workflow_id"`
|
||||
Key string `json:"target_step_key"`
|
||||
Sequence uint32 `json:"target_activation_sequence"`
|
||||
Digest string `json:"request_digest"`
|
||||
AcceptedAt time.Time `json:"accepted_at"`
|
||||
ExpiresAt time.Time `json:"expires_at"`
|
||||
}
|
||||
if json.Unmarshal(raw, &meta) != nil || meta.SignalID != v.ID || meta.WorkflowID != g.view.ID || meta.Key != row.view.Key || meta.Sequence != w.sequence || meta.Digest != hex.EncodeToString(digest[:]) || !meta.AcceptedAt.Equal(v.AcceptedAt) || !meta.ExpiresAt.Equal(v.ExpiresAt) || actor != v.ActorID || evidence.String != v.EvidenceRef {
|
||||
return workflowSourceError("SIGNAL_ACCEPTANCE_PROOF_INVALID")
|
||||
}
|
||||
expected, err := requestDigest(signal.Input{Name: v.Name, PayloadVersion: v.PayloadVersion, TargetStepKey: v.TargetStepKey, TargetActivationSequence: v.TargetActivationSequence, CorrelationKey: v.CorrelationKey, Payload: v.Payload, Reason: reason, EvidenceRef: v.EvidenceRef})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if expected != digest {
|
||||
return workflowSourceError("SIGNAL_REQUEST_CHANGED")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// selectedWorkflowSignal 返回已胜出信号或最早按时受理的匹配信号；查询结果竞争由调用方在同一实例锁下裁决。
|
||||
func selectedWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, w workflowSignalWait, now time.Time) (*signal.View, error) {
|
||||
var id core.ID
|
||||
if w.winner != nil {
|
||||
if w.state != "SATISFIED" {
|
||||
return nil, workflowSourceError("SIGNAL_WINNER_WAIT_INVALID")
|
||||
}
|
||||
id = *w.winner
|
||||
} else {
|
||||
if w.state != "WAITING" {
|
||||
return nil, nil
|
||||
}
|
||||
err := tx.QueryRowContext(ctx, `SELECT id FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND target_step_key=? AND target_activation_sequence=? AND state='PENDING' AND type=? AND payload_version=? AND correlation_key=? AND accepted_at<? ORDER BY accepted_at,id LIMIT 1 FOR UPDATE`, g.view.NamespaceID, g.view.ID, row.view.Key, w.sequence, w.contract.Name, w.contract.PayloadVersion, []byte(w.correlation), w.deadline).Scan(&id)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
v, err := readWorkflowSignal(ctx, tx, g.view.NamespaceID, g.view.ID, id)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if w.winner != nil && (v.State != "CONSUMED" || v.ConsumedWaitID == nil || *v.ConsumedWaitID != w.id) {
|
||||
return nil, workflowSourceError("SIGNAL_WINNER_ASSOCIATION_INVALID")
|
||||
}
|
||||
if err := verifyWorkflowSignal(ctx, tx, g, row, w, v, now); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &v, nil
|
||||
}
|
||||
|
||||
func claimWorkflowSignal(ctx context.Context, tx *sql.Tx, g *workflowGraph, w *workflowSignalWait, v *signal.View, now time.Time) error {
|
||||
if w.winner != nil {
|
||||
return nil
|
||||
}
|
||||
res, err := tx.ExecContext(ctx, `UPDATE weave_signals SET state='CONSUMED',disposition='WAIT_MATCHED',consumed_wait_id=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND workflow_id=? AND revision=? AND state='PENDING'`, w.id, now, g.view.NamespaceID, v.ID, g.view.ID, v.Revision)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return err
|
||||
}
|
||||
res, err = tx.ExecContext(ctx, `UPDATE weave_waits SET state='SATISFIED',winner_signal_id=?,resolved_at=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND workflow_id=? AND state='WAITING' AND winner_signal_id IS NULL`, v.ID, now, now, g.view.NamespaceID, w.id, g.view.ID)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return err
|
||||
}
|
||||
w.winner = &v.ID
|
||||
w.state = "SATISFIED"
|
||||
v.State = "CONSUMED"
|
||||
v.Revision++
|
||||
v.Disposition = "WAIT_MATCHED"
|
||||
v.ConsumedWaitID = &w.id
|
||||
return nil
|
||||
}
|
||||
func rejectOtherWorkflowSignals(ctx context.Context, tx *sql.Tx, g *workflowGraph, row *workflowStep, sequence uint32, reason string, now time.Time) error {
|
||||
_, err := tx.ExecContext(ctx, `UPDATE weave_signals SET state='REJECTED',disposition=?,revision=revision+1,updated_at=? WHERE namespace_id=? AND workflow_id=? AND target_step_key=? AND target_activation_sequence=? AND state='PENDING'`, reason, now, g.view.NamespaceID, g.view.ID, row.view.Key, sequence)
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,158 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
driver "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
// 用真实v19结构验证信号历史身份隔离和等待时钟回填，不用当前结构模拟旧数据库。
|
||||
func TestIntegrationWorkflowSignalUpgrade(t *testing.T) {
|
||||
dsn := os.Getenv("ORBIT_TEST_MYSQL_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
|
||||
defer cancel()
|
||||
admin, err := Open(ctx, dsn, 2, 1)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer admin.Close()
|
||||
name := "orbit_signal_upgrade_" + string(executionID(t))
|
||||
if _, err := admin.ExecContext(ctx, "CREATE DATABASE `"+name+"`"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() {
|
||||
cleanup, stop := context.WithTimeout(context.Background(), 10*time.Second)
|
||||
defer stop()
|
||||
if _, err := admin.ExecContext(cleanup, "DROP DATABASE `"+name+"`"); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
cfg, err := driver.ParseDSN(dsn)
|
||||
if err != nil {
|
||||
t.Fatal("invalid test DSN")
|
||||
}
|
||||
cfg.DBName = name
|
||||
db, err := Open(ctx, cfg.FormatDSN(), 4, 2)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
exec := func(q string, args ...any) {
|
||||
t.Helper()
|
||||
if _, err := db.ExecContext(ctx, q, args...); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
exec(`CREATE TABLE orbit_schema_migrations (version BIGINT UNSIGNED PRIMARY KEY,name VARCHAR(191) NOT NULL,checksum CHAR(64) CHARACTER SET ascii COLLATE ascii_bin NOT NULL,dirty BOOLEAN NOT NULL,applied_at DATETIME(6) NULL) ENGINE=InnoDB`)
|
||||
migrations, err := Migrations()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, m := range migrations {
|
||||
if m.Version > 19 {
|
||||
break
|
||||
}
|
||||
statements, err := splitSQL(m.SQL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, q := range statements {
|
||||
exec(q)
|
||||
}
|
||||
exec(`INSERT INTO orbit_schema_migrations(version,name,checksum,dirty,applied_at) VALUES (?,?,?,FALSE,UTC_TIMESTAMP(6))`, m.Version, m.Name, m.Checksum)
|
||||
}
|
||||
var now time.Time
|
||||
if err := db.QueryRowContext(ctx, `SELECT UTC_TIMESTAMP(6)`).Scan(&now); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ns := executionID(t)
|
||||
deadline := now.Add(time.Hour)
|
||||
type fixture struct{ workflow, step, activation core.ID }
|
||||
seed := func(key string, terminal, stale bool) fixture {
|
||||
t.Helper()
|
||||
f := fixture{executionID(t), executionID(t), executionID(t)}
|
||||
state := "WAITING"
|
||||
var completed *time.Time
|
||||
if terminal {
|
||||
state = "CANCELLED"
|
||||
completed = &now
|
||||
}
|
||||
exec(`INSERT INTO weave_instances(namespace_id,id,definition_id,definition_version_id,gate_id,state,input,policy_snapshot,deadline_at,completed_at) VALUES (?,?,?,?,?,?,'{}','{}',?,?)`, ns, f.workflow, executionID(t), executionID(t), executionID(t), state, deadline, completed)
|
||||
current := f.activation
|
||||
if stale {
|
||||
current = executionID(t)
|
||||
}
|
||||
exec("INSERT INTO weave_steps(namespace_id,id,workflow_id,`key`,kind,state,current_activation_id,revision) VALUES (?,?,?,?,'SIGNAL','WAITING',?,7)", ns, f.step, f.workflow, key, current)
|
||||
exec(`INSERT INTO weave_activations(namespace_id,id,workflow_id,step_id,sequence,state,input) VALUES (?,?,?,?,1,'WAITING','null')`, ns, f.activation, f.workflow, f.step)
|
||||
exec(`INSERT INTO weave_waits(namespace_id,id,workflow_id,activation_id,kind,correlation_key,deadline_at) VALUES (?,?,?,?,'SIGNAL','historical',?)`, ns, executionID(t), f.workflow, f.activation, deadline)
|
||||
return f
|
||||
}
|
||||
active := seed("signal", false, false)
|
||||
existing := seed("existing", false, false)
|
||||
seed("terminal", true, false)
|
||||
seed("stale", false, true)
|
||||
existingID := executionID(t)
|
||||
original := `{"schema_version":1,"kind":"SIGNAL","activation_id":"` + string(existing.activation) + `"}`
|
||||
exec(`INSERT INTO weave_pending_advances(namespace_id,id,workflow_id,step_id,expected_step_revision,checkpoint,available_at,revision) VALUES (?,?,?,?,7,?,?,11)`, ns, existingID, existing.workflow, existing.step, original, now)
|
||||
pending, consumed, wait := executionID(t), executionID(t), executionID(t)
|
||||
correlation := strings.Repeat("😀", 191)
|
||||
name64 := strings.Repeat("a", 64)
|
||||
exec(`INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,target_step_key,correlation_key,payload,actor_id,accepted_at,expires_at) VALUES (?,?,?,RANDOM_BYTES(32),RANDOM_BYTES(32),?,'signal',?,'{"n":9007199254740993}','legacy',?,?)`, ns, pending, active.workflow, name64, correlation, now, deadline)
|
||||
exec(`INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,target_step_key,correlation_key,payload,actor_id,state,consumed_wait_id,accepted_at,expires_at) VALUES (?,?,?,RANDOM_BYTES(32),RANDOM_BYTES(32),'done','signal','history ','null','legacy','CONSUMED',?,?,?)`, ns, consumed, active.workflow, wait, now, deadline)
|
||||
if err := Migrate(ctx, db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := CheckSchema(ctx, db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var payload, corr, signalName, state, disposition string
|
||||
var sequence *uint32
|
||||
var version *string
|
||||
var revision uint64
|
||||
var expires, available time.Time
|
||||
if err := db.QueryRowContext(ctx, `SELECT payload,correlation_key,type,state,disposition,target_activation_sequence,payload_version,revision,expires_at,available_at FROM weave_signals WHERE namespace_id=? AND id=?`, ns, pending).Scan(&payload, &corr, &signalName, &state, &disposition, &sequence, &version, &revision, &expires, &available); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(payload, "9007199254740993") || corr != correlation || signalName != name64 || state != "REJECTED" || disposition != "LEGACY_UNVERIFIED" || sequence != nil || version != nil || revision != 2 || !expires.Equal(deadline) || !available.Equal(deadline) {
|
||||
t.Fatal("legacy signal rewritten or treated as verified")
|
||||
}
|
||||
var count int
|
||||
if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_signals WHERE namespace_id=? AND id=? AND state='CONSUMED' AND consumed_wait_id=? AND revision=1 AND payload_version IS NULL AND target_activation_sequence IS NULL AND correlation_key=?`, ns, consumed, wait, []byte("history ")).Scan(&count); err != nil || count != 1 {
|
||||
t.Fatal("historical consumed evidence changed", err)
|
||||
}
|
||||
var raw []byte
|
||||
var due time.Time
|
||||
var expected uint64
|
||||
if err := db.QueryRowContext(ctx, `SELECT checkpoint,available_at,expected_step_revision FROM weave_pending_advances WHERE namespace_id=? AND workflow_id=? AND step_id=?`, ns, active.workflow, active.step).Scan(&raw, &due, &expected); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var checkpoint workflowCheckpoint
|
||||
if err := json.Unmarshal(raw, &checkpoint); err != nil || checkpoint.Kind != "SIGNAL" || checkpoint.SchemaVersion != 1 || checkpoint.ActivationID == nil || *checkpoint.ActivationID != active.activation || !due.Equal(deadline) || expected != 7 {
|
||||
t.Fatal("signal checkpoint reset original wait deadline", err)
|
||||
}
|
||||
if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_pending_advances`).Scan(&count); err != nil || count != 2 {
|
||||
t.Fatal("stale/terminal or duplicate checkpoint", count, err)
|
||||
}
|
||||
if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM weave_pending_advances WHERE id=? AND revision=11 AND available_at=? AND checkpoint=?`, existingID, now, original).Scan(&count); err != nil || count != 1 {
|
||||
t.Fatal("existing checkpoint changed", err)
|
||||
}
|
||||
if err := db.QueryRowContext(ctx, `SELECT COUNT(*) FROM information_schema.REFERENTIAL_CONSTRAINTS WHERE CONSTRAINT_SCHEMA=DATABASE()`).Scan(&count); err != nil || count != 0 {
|
||||
t.Fatal("foreign keys introduced", err)
|
||||
}
|
||||
if err := Migrate(ctx, db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := db.QueryRowContext(ctx, `SELECT revision FROM weave_signals WHERE namespace_id=? AND id=?`, ns, pending).Scan(&revision); err != nil || revision != 2 {
|
||||
t.Fatal("migration replay modified history", err)
|
||||
}
|
||||
t.Log("real v19 upgrade preserves 764-byte legacy correlation, exact payload and consumed evidence; isolates unverifiable pending signals; backfills current SIGNAL at original deadline; zero foreign keys")
|
||||
}
|
||||
207
internal/storage/mysql/workflow_signals.go
Normal file
207
internal/storage/mysql/workflow_signals.go
Normal file
@@ -0,0 +1,207 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/orbit/internal/fault"
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/platform/auth"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/definition"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/engine"
|
||||
weave "git.yuntuo.hk/k2safe/orbit/weave/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/signal"
|
||||
)
|
||||
|
||||
type WorkflowSignalStore struct{ *TaskStore }
|
||||
|
||||
func NewWorkflowSignalStore(db *sql.DB) *WorkflowSignalStore {
|
||||
return &WorkflowSignalStore{NewTaskStore(db)}
|
||||
}
|
||||
|
||||
var _ signal.Store = (*WorkflowSignalStore)(nil)
|
||||
|
||||
func signalContract(node definition.Step) *definition.Signal {
|
||||
if node.Signal != nil {
|
||||
return node.Signal
|
||||
}
|
||||
if node.Poll != nil {
|
||||
return node.Poll.FinalSignal
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
const signalColumns = `id,namespace_id,workflow_id,revision,type,COALESCE(payload_version,''),target_step_key,COALESCE(target_activation_sequence,0),correlation_key,payload,actor_id,COALESCE(evidence_ref,''),state,COALESCE(disposition,''),accepted_at,expires_at,consumed_wait_id`
|
||||
|
||||
func readWorkflowSignal(ctx context.Context, tx *sql.Tx, ns, workflow, id core.ID) (signal.View, error) {
|
||||
var v signal.View
|
||||
var payload []byte
|
||||
err := tx.QueryRowContext(ctx, `SELECT `+signalColumns+` FROM weave_signals WHERE namespace_id=? AND workflow_id=? AND id=?`, ns, workflow, id).Scan(&v.ID, &v.NamespaceID, &v.WorkflowID, &v.Revision, &v.Name, &v.PayloadVersion, &v.TargetStepKey, &v.TargetActivationSequence, &v.CorrelationKey, &payload, &v.ActorID, &v.EvidenceRef, &v.State, &v.Disposition, &v.AcceptedAt, &v.ExpiresAt, &v.ConsumedWaitID)
|
||||
v.Payload = payload
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
err = fault.NotFound
|
||||
}
|
||||
return v, err
|
||||
}
|
||||
|
||||
func (s *WorkflowSignalStore) Send(ctx context.Context, t auth.Token, workflow core.ID, key string, in signal.Input) (signal.Result, error) {
|
||||
var out signal.Result
|
||||
err := s.transaction(ctx, t, signal.SendAction, func(tx *sql.Tx, p auth.Permissions) error {
|
||||
if _, err := lockApplicationResources(ctx, tx, t.NamespaceID, false); err != nil {
|
||||
return err
|
||||
}
|
||||
id, err := core.NewID()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
identity, cached, err := reserveIdentity(ctx, tx, t.NamespaceID, "workflow.signal", string(workflow), key, "workflow_signal", id, in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
g, _, err := lockWorkflowGraph(ctx, tx, t.NamespaceID, workflow, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, queue := range g.version.Analysis.Queues {
|
||||
if !p.AllowsQueue(queue) {
|
||||
return fault.NotFound
|
||||
}
|
||||
}
|
||||
node, ok := g.program.Step(in.TargetStepKey)
|
||||
contract := signalContract(node)
|
||||
if !ok || contract == nil || contract.Name != in.Name || contract.PayloadVersion != in.PayloadVersion || definition.ValidateData(contract.PayloadType, in.Payload) != nil {
|
||||
return fault.Invalid("signal does not match fixed workflow contract")
|
||||
}
|
||||
if node.Poll != nil {
|
||||
outcome, err := g.program.ClassifyPoll(in.TargetStepKey, in.Payload)
|
||||
if err != nil || outcome == engine.PollPending {
|
||||
return fault.Invalid("final signal requires a terminal business result")
|
||||
}
|
||||
}
|
||||
if cached != nil {
|
||||
if err := json.Unmarshal(cached, &out); err != nil {
|
||||
return err
|
||||
}
|
||||
current, err := readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, out.Signal.ID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if out.Signal.NamespaceID != t.NamespaceID || out.Signal.WorkflowID != workflow || !current.AcceptedAt.Equal(out.Signal.AcceptedAt) {
|
||||
return errors.New("workflow signal receipt mismatch")
|
||||
}
|
||||
out.Deduplicated = true
|
||||
return nil
|
||||
}
|
||||
var count int
|
||||
if err := tx.QueryRowContext(ctx, `SELECT COUNT(*) FROM (SELECT id FROM weave_signals WHERE namespace_id=? AND workflow_id=? LIMIT ?) admitted_signals`, t.NamespaceID, workflow, signal.MaxPerWorkflow).Scan(&count); err != nil {
|
||||
return err
|
||||
}
|
||||
if count >= signal.MaxPerWorkflow {
|
||||
return fault.New("WORKFLOW_SIGNAL_LIMIT", "workflow signal limit reached")
|
||||
}
|
||||
row := g.steps[in.TargetStepKey]
|
||||
sequence := uint32(1)
|
||||
if row.view.ActivationID != nil {
|
||||
if err := tx.QueryRowContext(ctx, `SELECT sequence FROM weave_activations WHERE namespace_id=? AND workflow_id=? AND step_id=? AND id=?`, t.NamespaceID, workflow, row.view.ID, row.view.ActivationID).Scan(&sequence); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if in.TargetActivationSequence > sequence {
|
||||
return fault.Conflict
|
||||
}
|
||||
now, err := dbNow(ctx, tx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
state, disposition := weave.SignalState("PENDING"), ""
|
||||
switch {
|
||||
case in.TargetActivationSequence < sequence:
|
||||
state, disposition = "REJECTED", "STALE_ACTIVATION"
|
||||
case g.view.State.Terminal():
|
||||
state, disposition = "REJECTED", "WORKFLOW_SETTLED"
|
||||
case !now.Before(g.view.DeadlineAt):
|
||||
state, disposition = "EXPIRED", "WORKFLOW_DEADLINE"
|
||||
case row.view.State == weave.StepSkipped || row.view.State == weave.StepSucceeded || row.view.State == weave.StepFailed || row.view.State == weave.StepCancelled || row.view.State == weave.StepNeedsReview:
|
||||
state, disposition = "REJECTED", "STEP_SETTLED"
|
||||
case row.view.State == weave.StepBlocked && g.view.CancelRequestedAt != nil:
|
||||
state, disposition = "REJECTED", "WORKFLOW_CANCELLING"
|
||||
}
|
||||
if disposition == "" && row.view.ActivationID != nil {
|
||||
var name, version *string
|
||||
var correlation, waitState string
|
||||
var deadline time.Time
|
||||
if err := tx.QueryRowContext(ctx, `SELECT signal_name,payload_version,correlation_key,state,deadline_at FROM weave_waits WHERE namespace_id=? AND workflow_id=? AND activation_id=? FOR UPDATE`, t.NamespaceID, workflow, row.view.ActivationID).Scan(&name, &version, &correlation, &waitState, &deadline); err != nil {
|
||||
return err
|
||||
}
|
||||
if name == nil || version == nil || *name != in.Name || *version != in.PayloadVersion {
|
||||
return errors.New("workflow signal wait contract invalid")
|
||||
}
|
||||
switch {
|
||||
case waitState != "WAITING":
|
||||
state, disposition = "REJECTED", "WAIT_SETTLED"
|
||||
case correlation != in.CorrelationKey:
|
||||
state, disposition = "REJECTED", "CORRELATION_MISMATCH"
|
||||
case !now.Before(deadline):
|
||||
state, disposition = "EXPIRED", "SIGNAL_DEADLINE"
|
||||
}
|
||||
}
|
||||
digest, err := requestDigest(in)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
keyDigest := core.Digest(sha256.Sum256([]byte(key)))
|
||||
_, err = tx.ExecContext(ctx, `INSERT INTO weave_signals(namespace_id,id,workflow_id,key_digest,request_digest,type,payload_version,target_step_key,target_activation_sequence,correlation_key,payload,actor_id,evidence_ref,state,disposition,accepted_at,expires_at,available_at,created_at,updated_at) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)`, t.NamespaceID, id, workflow, keyDigest, digest, in.Name, in.PayloadVersion, in.TargetStepKey, in.TargetActivationSequence, []byte(in.CorrelationKey), string(in.Payload), string(t.CredentialID), businessKeyValue(in.EvidenceRef), state, businessKeyValue(disposition), now, g.view.DeadlineAt, g.view.DeadlineAt, now, now)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if state == "PENDING" && row.view.ActivationID != nil {
|
||||
if err := saveWorkflowClock(ctx, tx, g, row, string(row.view.Kind), row.view.Revision, now); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
res, err := tx.ExecContext(ctx, `UPDATE weave_instances SET revision=revision+1,updated_at=? WHERE namespace_id=? AND id=? AND revision=?`, now, t.NamespaceID, workflow, g.view.Revision)
|
||||
if err := exactlyOne(res, err); err != nil {
|
||||
return err
|
||||
}
|
||||
g.view.Revision++
|
||||
meta := map[string]any{"signal_id": id, "workflow_id": workflow, "target_step_key": in.TargetStepKey, "target_activation_sequence": in.TargetActivationSequence, "state": state, "disposition": disposition, "request_digest": hex.EncodeToString(digest[:]), "accepted_at": now, "expires_at": g.view.DeadlineAt}
|
||||
if err := emitEvent(ctx, tx, t.NamespaceID, workflow, g.view.Revision, "workflow", "workflow.signal.accepted", meta); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := appendAuditWithEvidence(ctx, tx, t, identity, id, "workflow_signal", "workflow.signal", in.Reason, in.EvidenceRef, nil, meta); err != nil {
|
||||
return err
|
||||
}
|
||||
out.Signal, err = readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return saveIdentity(ctx, tx, t.NamespaceID, identity, out)
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
func (s *WorkflowSignalStore) Get(ctx context.Context, t auth.Token, workflow, id core.ID) (signal.View, error) {
|
||||
var out signal.View
|
||||
err := s.transaction(ctx, t, signal.ReadAction, func(tx *sql.Tx, p auth.Permissions) error {
|
||||
v, _, err := readWorkflowInstance(ctx, tx, t.NamespaceID, workflow, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
version, err := instanceVersion(ctx, tx, v)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, queue := range version.Analysis.Queues {
|
||||
if !p.AllowsQueue(queue) {
|
||||
return fault.NotFound
|
||||
}
|
||||
}
|
||||
out, err = readWorkflowSignal(ctx, tx, t.NamespaceID, workflow, id)
|
||||
return err
|
||||
})
|
||||
return out, err
|
||||
}
|
||||
161
internal/storage/mysql/workflow_signals_integration_test.go
Normal file
161
internal/storage/mysql/workflow_signals_integration_test.go
Normal file
@@ -0,0 +1,161 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/orbit/internal/fault"
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/definition"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/instance"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/signal"
|
||||
)
|
||||
|
||||
func signalSpec(queue core.ID, early bool) definition.Specification {
|
||||
spec := runtimeSpec(queue, 1)
|
||||
node := definition.Step{Key: "signal", Kind: "SIGNAL", Signal: &definition.Signal{Name: "external.done", PayloadVersion: "v1", Correlation: definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "workflow", Path: []string{"business_key"}}}, PayloadType: spec.InputType, TimeoutMS: 30000}}
|
||||
if early {
|
||||
spec.Body.Steps = append(spec.Body.Steps, node)
|
||||
spec.Body.Output = definition.Value{Kind: "REFERENCE", Reference: &definition.Reference{Source: "signal"}}
|
||||
} else {
|
||||
spec.Body.Steps = append([]definition.Step{node}, spec.Body.Steps...)
|
||||
}
|
||||
return spec
|
||||
}
|
||||
func grantWorkflowSignal(f *executionFixture) {
|
||||
f.exec(`UPDATE orbit_credentials SET permissions=JSON_ARRAY_APPEND(permissions,'$.actions','workflows:signal') WHERE namespace_id=? AND id=?`, f.resources.NamespaceID, f.resources.CredentialID)
|
||||
}
|
||||
func signalInput(v instance.View) signal.Input {
|
||||
return signal.Input{Name: "external.done", PayloadVersion: "v1", TargetStepKey: "signal", TargetActivationSequence: 1, CorrelationKey: v.BusinessKey, Payload: json.RawMessage(`{"n":9007199254740993}`), Reason: "verified external completion", EvidenceRef: "proof:event-1"}
|
||||
}
|
||||
func signalStart(t *testing.T, ctx context.Context, db *sql.DB, early bool) (*executionFixture, instance.View, *signal.Service) {
|
||||
t.Helper()
|
||||
f := newExecutionFixture(t, ctx, db)
|
||||
v := runtimeStart(t, f, signalSpec(f.resources.QueueID, early))
|
||||
grantWorkflowSignal(f)
|
||||
return f, v, signal.New(NewWorkflowSignalStore(db))
|
||||
}
|
||||
func TestIntegrationWorkflowSignalAdmission(t *testing.T) {
|
||||
dsn := os.Getenv("ORBIT_TEST_MYSQL_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL")
|
||||
}
|
||||
ctx, stop := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer stop()
|
||||
db, err := Open(ctx, dsn, 32, 8)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
if err := Migrate(ctx, db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Run("concurrent_identity_exact_input_and_frozen_receipt", func(t *testing.T) {
|
||||
f, v, api := signalStart(t, ctx, db, false)
|
||||
in := signalInput(v)
|
||||
key := string(executionID(t))
|
||||
var wg sync.WaitGroup
|
||||
results := make(chan signal.Result, 12)
|
||||
errs := make(chan error, 12)
|
||||
for i := 0; i < 12; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
out, err := api.Send(ctx, f.token, v.ID, key, in)
|
||||
results <- out
|
||||
errs <- err
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
close(results)
|
||||
close(errs)
|
||||
for err := range errs {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
var id core.ID
|
||||
fresh := 0
|
||||
for out := range results {
|
||||
if id == "" {
|
||||
id = out.Signal.ID
|
||||
}
|
||||
if out.Signal.ID != id || out.Signal.State != "PENDING" || out.Signal.CorrelationKey != in.CorrelationKey || string(out.Signal.Payload) != string(in.Payload) {
|
||||
t.Fatal("signal replay changed identity")
|
||||
}
|
||||
if !out.Deduplicated {
|
||||
fresh++
|
||||
}
|
||||
}
|
||||
if fresh != 1 || f.count("weave_signals", "1=1") != 1 || f.count("orbit_audit_events", "action='workflow.signal'") != 1 {
|
||||
t.Fatal("duplicate signal admission")
|
||||
}
|
||||
changed := in
|
||||
changed.Payload = json.RawMessage(`{"n":2}`)
|
||||
if _, err := api.Send(ctx, f.token, v.ID, key, changed); fault.Code(err) != "IDEMPOTENCY_CONFLICT" {
|
||||
t.Fatal("same key changed content", err)
|
||||
}
|
||||
current, err := api.Get(ctx, f.token, v.ID, id)
|
||||
if err != nil || current.ActorID != string(f.resources.CredentialID) || current.EvidenceRef != in.EvidenceRef {
|
||||
t.Fatal("signal query lost provenance", err)
|
||||
}
|
||||
})
|
||||
t.Run("early_signal_is_durable_without_premature_activation", func(t *testing.T) {
|
||||
f, v, api := signalStart(t, ctx, db, true)
|
||||
out, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), signalInput(v))
|
||||
if err != nil || out.Signal.State != "PENDING" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f.count("pulse_tasks", "1=1") != 1 || f.count("weave_steps", "kind='SIGNAL' AND state='BLOCKED' AND current_activation_id IS NULL") != 1 {
|
||||
t.Fatal("early signal activated successor")
|
||||
}
|
||||
})
|
||||
t.Run("contract_generation_and_scope_are_checked", func(t *testing.T) {
|
||||
f, v, api := signalStart(t, ctx, db, false)
|
||||
in := signalInput(v)
|
||||
for _, change := range []func(*signal.Input){func(in *signal.Input) { in.Name = "wrong" }, func(in *signal.Input) { in.PayloadVersion = "v2" }, func(in *signal.Input) { in.Payload = json.RawMessage(`"bad"`) }, func(in *signal.Input) { in.TargetActivationSequence = 2 }} {
|
||||
bad := in
|
||||
change(&bad)
|
||||
if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), bad); err == nil {
|
||||
t.Fatal("invalid signal contract or generation admitted")
|
||||
}
|
||||
}
|
||||
if f.count("weave_signals", "1=1") != 0 {
|
||||
t.Fatal("rejected request was partially stored")
|
||||
}
|
||||
wrong := in
|
||||
wrong.CorrelationKey = "different"
|
||||
out, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), wrong)
|
||||
if err != nil || out.Signal.State != "REJECTED" || out.Signal.Disposition != "CORRELATION_MISMATCH" {
|
||||
t.Fatal("mismatched correlation granted authority", err)
|
||||
}
|
||||
grantWorkflowInstance(t, f)
|
||||
if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), in); fault.Code(err) != "PERMISSION_DENIED" {
|
||||
t.Fatal("signal action not enforced", err)
|
||||
}
|
||||
grantWorkflowSignal(f)
|
||||
grantWorkflowInstance(t, f, string(executionID(t)))
|
||||
grantWorkflowSignal(f)
|
||||
if _, err := api.Send(ctx, f.token, v.ID, string(executionID(t)), in); fault.Code(err) != "NOT_FOUND" {
|
||||
t.Fatal("signal queue scope not enforced", err)
|
||||
}
|
||||
})
|
||||
t.Run("audit_failure_rolls_back_signal_identity_and_clock", func(t *testing.T) {
|
||||
f, v, api := signalStart(t, ctx, db, false)
|
||||
key := string(executionID(t))
|
||||
f.exec(`CREATE TRIGGER test_signal_admission_failure BEFORE INSERT ON orbit_audit_events FOR EACH ROW SIGNAL SQLSTATE '45000' SET MESSAGE_TEXT='injected signal audit failure'`)
|
||||
_, err := api.Send(ctx, f.token, v.ID, key, signalInput(v))
|
||||
f.exec(`DROP TRIGGER test_signal_admission_failure`)
|
||||
if err == nil || f.count("weave_signals", "1=1") != 0 || f.count("orbit_idempotency_keys", "operation='workflow.signal'") != 0 {
|
||||
t.Fatal("signal admission partially committed")
|
||||
}
|
||||
if out, err := api.Send(ctx, f.token, v.ID, key, signalInput(v)); err != nil || out.Deduplicated {
|
||||
t.Fatal("original identity did not recover", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -58,7 +58,7 @@ type Permissions struct {
|
||||
Queues []string `json:"queues"`
|
||||
}
|
||||
|
||||
var supported = map[string]bool{"workflows:start": true, "workflows:read": true, "workflows:definitions:read": true, "workflows:definitions:publish": true, "tasks:review": true, "tasks:retry": true, "tasks:audit:read": true, "events:read": true, "events:manage": true, "events:consume": true, "application:resources:read": true, "application:resources:manage": true, "tasks:submit": true, "tasks:read": true, "tasks:cancel": true, "queues:read": true, "queues:manage": true, "workers:connect": true, "workers:consume": true, "schedules:read": true, "schedules:manage": true}
|
||||
var supported = map[string]bool{"workflows:signal": true, "workflows:start": true, "workflows:read": true, "workflows:definitions:read": true, "workflows:definitions:publish": true, "tasks:review": true, "tasks:retry": true, "tasks:audit:read": true, "events:read": true, "events:manage": true, "events:consume": true, "application:resources:read": true, "application:resources:manage": true, "tasks:submit": true, "tasks:read": true, "tasks:cancel": true, "queues:read": true, "queues:manage": true, "workers:connect": true, "workers:consume": true, "schedules:read": true, "schedules:manage": true}
|
||||
|
||||
func (p Permissions) Validate() error {
|
||||
if len(p.Actions) == 0 || len(p.Actions) > 32 || len(p.Queues) == 0 || len(p.Queues) > 128 {
|
||||
|
||||
@@ -19,20 +19,30 @@ type Signal struct {
|
||||
RequestDigest core.Digest `db:"request_digest"`
|
||||
// Type 事件或信号的稳定类型标识。
|
||||
Type string `db:"type"`
|
||||
// PayloadVersion 固定信号契约版本；为空仅表示不能参与新等待匹配的历史记录。
|
||||
PayloadVersion *string `db:"payload_version"`
|
||||
// TargetStepKey 信号目标步骤键，用于早到信号的后续匹配。
|
||||
TargetStepKey string `db:"target_step_key"`
|
||||
// CorrelationKey 外部业务操作关联键，防止旧信号误用于新工作。
|
||||
// TargetActivationSequence 目标激活序号，早到首轮为1；为空仅表示身份未核实的历史信号。
|
||||
TargetActivationSequence *uint32 `db:"target_activation_sequence"`
|
||||
// CorrelationKey 精确UTF8操作关联，新信号最多512字节，历史兼容764字节，保留尾部空格。
|
||||
CorrelationKey string `db:"correlation_key"`
|
||||
// Payload 版本化输入或事件内容；仅承载通用 JSON，不解释行业规则。
|
||||
Payload core.JSON `db:"payload"`
|
||||
// ActorID 经认证的操作主体标识，不能直接信任请求自报值。
|
||||
ActorID string `db:"actor_id"`
|
||||
// EvidenceRef 可选受控证据引用，最多512个无空白字符；平台不请求该地址。
|
||||
EvidenceRef *string `db:"evidence_ref"`
|
||||
// State 当前状态；只允许定义的状态转换并配合事务校验。
|
||||
State SignalState `db:"state"`
|
||||
// Disposition 稳定消费/拒绝分类；PENDING时为空，不记录原始错误或业务参数。
|
||||
Disposition *string `db:"disposition"`
|
||||
// AcceptedAt 信号可靠受理时间，不能用稍后的消费时间替代。
|
||||
AcceptedAt time.Time `db:"accepted_at"`
|
||||
// ExpiresAt 过期时间，统一 UTC；到期后的行为由对应资源策略决定。
|
||||
ExpiresAt time.Time `db:"expires_at"`
|
||||
// AvailableAt 未匹配信号下次到期检查时间；数据库故障只推迟检查，不延长原受理截止。
|
||||
AvailableAt time.Time `db:"available_at"`
|
||||
// ConsumedWaitID 消费该信号的等待记录；尚未消费时为空。
|
||||
ConsumedWaitID *core.ID `db:"consumed_wait_id"`
|
||||
}
|
||||
|
||||
@@ -23,6 +23,8 @@ type Wait struct {
|
||||
SignalName *string `db:"signal_name"`
|
||||
// PayloadVersion 激活时冻结的信号内容契约版本；TIMER及待核实历史记录为空。
|
||||
PayloadVersion *string `db:"payload_version"`
|
||||
// WinnerSignalID 胜出的外部信号；时间或查询结果胜出时为空，须核对双向归属。
|
||||
WinnerSignalID *core.ID `db:"winner_signal_id"`
|
||||
// CorrelationKey 精确UTF8业务关联且保留尾部空格；新受理最多512字节，历史兼容764字节，TIMER为空串。
|
||||
CorrelationKey string `db:"correlation_key"`
|
||||
// DeadlineAt 等待截止时间，用可靠受理时间判断信号是否按时到达。
|
||||
|
||||
94
weave/signal/service.go
Normal file
94
weave/signal/service.go
Normal file
@@ -0,0 +1,94 @@
|
||||
// Package signal 定义可靠外部信号的受理与查询；受理成功不等于已经推进流程。
|
||||
package signal
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/orbit/internal/fault"
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/platform/auth"
|
||||
"git.yuntuo.hk/k2safe/orbit/pulse/service"
|
||||
"git.yuntuo.hk/k2safe/orbit/weave/definition"
|
||||
weave "git.yuntuo.hk/k2safe/orbit/weave/model"
|
||||
)
|
||||
|
||||
const SendAction = "workflows:signal"
|
||||
const ReadAction = "workflows:read"
|
||||
const MaxPerWorkflow = 1024 // MaxPerWorkflow 限制实例全部受理信号数，终结/拒绝记录也保留占用，原键恢复不受上限影响。
|
||||
|
||||
type Input struct {
|
||||
Name string `json:"name"` // Name 固定定义信号名称，最多128个ASCII字符。
|
||||
PayloadVersion string `json:"payload_version"` // PayloadVersion 固定内容契约版本，最多32个ASCII字符。
|
||||
TargetStepKey string `json:"target_step_key"` // TargetStepKey 固定定义的目标步骤，最多64个ASCII字符。
|
||||
TargetActivationSequence uint32 `json:"target_activation_sequence"` // TargetActivationSequence 显式目标激活代次，首次及早到信号为1，不能自动跟随人工恢复。
|
||||
CorrelationKey string `json:"correlation_key"` // CorrelationKey 1至512字节精确UTF8操作关联，保留尾部空格。
|
||||
Payload json.RawMessage `json:"payload" nullable:"true"` // Payload 固定契约内容，最多256KiB，显式null与缺失不同。
|
||||
Reason string `json:"reason"` // Reason 受理原因，1至1024个Unicode字符，进入审计。
|
||||
EvidenceRef string `json:"evidence_ref,omitempty"` // EvidenceRef 可选受控证据引用，最多512个无空白/控制字符，不携带秘密。
|
||||
}
|
||||
|
||||
type View struct {
|
||||
ID core.ID `json:"id"` // ID 信号持久身份。
|
||||
NamespaceID core.ID `json:"namespace_id"` // NamespaceID 授权和数据隔离范围。
|
||||
WorkflowID core.ID `json:"workflow_id"` // WorkflowID 固定目标实例。
|
||||
Revision uint64 `json:"revision,string"` // Revision 当前信号记录版本。
|
||||
Name string `json:"name"` // Name 固定信号类型。
|
||||
PayloadVersion string `json:"payload_version"` // PayloadVersion 固定契约版本。
|
||||
TargetStepKey string `json:"target_step_key"` // TargetStepKey 目标步骤名称。
|
||||
TargetActivationSequence uint32 `json:"target_activation_sequence"` // TargetActivationSequence 固定激活序号。
|
||||
CorrelationKey string `json:"correlation_key"` // CorrelationKey 原始操作关联。
|
||||
Payload json.RawMessage `json:"payload"` // Payload 原始精确内容。
|
||||
ActorID string `json:"actor_id"` // ActorID 平台认证主体，不接受请求自报身份。
|
||||
EvidenceRef string `json:"evidence_ref,omitempty"` // EvidenceRef 原证据引用。
|
||||
State weave.SignalState `json:"state"` // State PENDING/CONSUMED/EXPIRED/REJECTED。
|
||||
Disposition string `json:"disposition,omitempty"` // Disposition 稳定消费或拒绝原因。
|
||||
AcceptedAt time.Time `json:"accepted_at"` // AcceptedAt 数据库可靠受理时间，截止不含等号。
|
||||
ExpiresAt time.Time `json:"expires_at"` // ExpiresAt 固定实例总截止，不因重报延长。
|
||||
ConsumedWaitID *core.ID `json:"consumed_wait_id,omitempty"` // ConsumedWaitID 消费它的唯一等待身份。
|
||||
}
|
||||
type Result struct {
|
||||
Signal View `json:"signal"` // Signal 首次受理的冻结视图，当前状态通过Get查询。
|
||||
Deduplicated bool `json:"deduplicated"` // Deduplicated 原身份恢复，不重报新事件或重新计时。
|
||||
}
|
||||
type Store interface {
|
||||
Send(context.Context, auth.Token, core.ID, string, Input) (Result, error)
|
||||
Get(context.Context, auth.Token, core.ID, core.ID) (View, error)
|
||||
}
|
||||
type Service struct{ store Store }
|
||||
|
||||
func New(store Store) *Service { return &Service{store: store} }
|
||||
func (s *Service) Send(ctx context.Context, t auth.Token, workflow core.ID, key string, in Input) (Result, error) {
|
||||
if err := service.ValidateKey(key); err != nil {
|
||||
return Result{}, err
|
||||
}
|
||||
if workflow.Validate() != nil || !validName(in.Name, 128) || !validName(in.PayloadVersion, 32) || !validName(in.TargetStepKey, 64) || in.TargetActivationSequence == 0 {
|
||||
return Result{}, fault.Invalid("valid signal name, contract, step and activation sequence required")
|
||||
}
|
||||
if !utf8.ValidString(in.CorrelationKey) || len(in.CorrelationKey) == 0 || len(in.CorrelationKey) > 512 || strings.ContainsRune(in.CorrelationKey, 0) {
|
||||
return Result{}, fault.Invalid("signal correlation must be 1 to 512 UTF-8 bytes")
|
||||
}
|
||||
if err := definition.ValidateData(definition.Type{Kind: "JSON"}, in.Payload); err != nil {
|
||||
return Result{}, fault.Invalid("valid signal payload required, maximum 256 KiB")
|
||||
}
|
||||
if !utf8.ValidString(in.Reason) || strings.TrimSpace(in.Reason) == "" || utf8.RuneCountInString(in.Reason) > 1024 || strings.ContainsRune(in.Reason, 0) {
|
||||
return Result{}, fault.Invalid("signal reason required, maximum 1024 characters")
|
||||
}
|
||||
if !utf8.ValidString(in.EvidenceRef) || utf8.RuneCountInString(in.EvidenceRef) > 512 || strings.IndexFunc(in.EvidenceRef, func(r rune) bool { return unicode.IsSpace(r) || unicode.IsControl(r) }) >= 0 {
|
||||
return Result{}, fault.Invalid("invalid signal evidence reference")
|
||||
}
|
||||
return s.store.Send(ctx, t, workflow, key, in)
|
||||
}
|
||||
func validName(value string, max int) bool {
|
||||
return service.ValidName(value, max) && !strings.ContainsRune(value, ':')
|
||||
}
|
||||
func (s *Service) Get(ctx context.Context, t auth.Token, workflow, id core.ID) (View, error) {
|
||||
if workflow.Validate() != nil || id.Validate() != nil {
|
||||
return View{}, fault.Invalid("invalid workflow or signal id")
|
||||
}
|
||||
return s.store.Get(ctx, t, workflow, id)
|
||||
}
|
||||
55
weave/signal/service_test.go
Normal file
55
weave/signal/service_test.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package signal
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
core "git.yuntuo.hk/k2safe/orbit/model"
|
||||
"git.yuntuo.hk/k2safe/orbit/platform/auth"
|
||||
)
|
||||
|
||||
type captureStore struct {
|
||||
calls int
|
||||
input Input
|
||||
}
|
||||
|
||||
func (s *captureStore) Send(_ context.Context, _ auth.Token, _ core.ID, _ string, in Input) (Result, error) {
|
||||
s.calls++
|
||||
s.input = in
|
||||
return Result{}, nil
|
||||
}
|
||||
func (s *captureStore) Get(context.Context, auth.Token, core.ID, core.ID) (View, error) {
|
||||
s.calls++
|
||||
return View{}, nil
|
||||
}
|
||||
func TestSignalInputBoundary(t *testing.T) {
|
||||
store := &captureStore{}
|
||||
api := New(store)
|
||||
id := core.ID("11111111111111111111111111111111")
|
||||
base := Input{Name: "completed", PayloadVersion: "v1", TargetStepKey: "wait", TargetActivationSequence: 1, CorrelationKey: "关联 ", Payload: json.RawMessage(`{"n":9007199254740993}`), Reason: "verified source", EvidenceRef: "proof:record-1"}
|
||||
for _, raw := range []string{`{"n":9007199254740993}`, `null`} {
|
||||
in := base
|
||||
in.Payload = json.RawMessage(raw)
|
||||
if _, err := api.Send(context.Background(), auth.Token{}, id, "original", in); err != nil || string(store.input.Payload) != raw || store.input.CorrelationKey != base.CorrelationKey {
|
||||
t.Fatal("signal input changed", err)
|
||||
}
|
||||
}
|
||||
for _, change := range []func(*Input){
|
||||
func(in *Input) { in.TargetActivationSequence = 0 }, func(in *Input) { in.CorrelationKey = "" }, func(in *Input) { in.CorrelationKey = strings.Repeat("中", 171) }, func(in *Input) { in.CorrelationKey = "x\x00y" },
|
||||
func(in *Input) { in.Name = "a:b" }, func(in *Input) { in.Name = strings.Repeat("n", 129) }, func(in *Input) { in.PayloadVersion = "" }, func(in *Input) { in.TargetStepKey = "$root" },
|
||||
func(in *Input) { in.Payload = nil }, func(in *Input) { in.Payload = json.RawMessage(`{"n":1,"n":2}`) }, func(in *Input) { in.Payload = json.RawMessage(`"\ud800"`) },
|
||||
func(in *Input) { in.Reason = " " }, func(in *Input) { in.Reason = strings.Repeat("中", 1025) }, func(in *Input) { in.EvidenceRef = "proof:secret\nvalue" }, func(in *Input) { in.EvidenceRef = strings.Repeat("x", 513) },
|
||||
} {
|
||||
in := base
|
||||
change(&in)
|
||||
before := store.calls
|
||||
if _, err := api.Send(context.Background(), auth.Token{}, id, "original", in); err == nil || before != store.calls {
|
||||
t.Fatal("invalid signal reached store")
|
||||
}
|
||||
}
|
||||
if _, err := api.Send(context.Background(), auth.Token{}, id, "", base); err == nil {
|
||||
t.Fatal("missing identity accepted")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user