diff --git a/AGENTS.md b/AGENTS.md index 4aa64ca..f654bd7 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -86,3 +86,4 @@ - 进程指标、固定状态采样、格式验证、告警规则和标准回归证据见docs/reports/phase-6-observability.md及docs/reports/phase-6-business-observability.md。仍须补受理/授权/完成与流程推进延迟、持久吞吐/积压的全局聚合、重试/核查/恢复量、发送队列、实际采集告警及负载开销,不能把进程观测通过写成P6已完成。普通清理仍须保护有效幂等、未确认事件、未知结果和未终结流程。 - 凭据轮换保留credential_id及订阅/Worker授权绑定,在同一事务替换摘要、固定到期时间、幂等回执和管理审计;新令牌必须重新建立会话,认证路径每次重新核对凭据状态。撤销必须让认证失效并保留原任务、订阅、交付、会话和审计证据,允许在应用或命名空间停用后执行;重复应用同一固定操作幂等返回。管理CLI先把固定操作文件写成0600,再提交;提交不明时保留文件并用原文件重放,不能重新生成秘密。日志、审计摘要、指标和业务表不得保存秘密原文或摘要。不要把轮换同一凭据ID解释成历史令牌仍有效,也不得级联删除订阅、任务或事件。 - `sdk/go/cmd/orbit-load` 只用于隔离环境受理测量:每项使用稳定独立幂等键,固定请求形状和并发上限,按请求超时,不自动重试,不启动Worker或执行外部副作用;输出延迟百分位与有限错误分类,失败必须以非零状态返回。其结果不能替代真实Worker、流程推进、数据库拓扑或生产容量验收。 +- `sdk/go/cmd/orbit-exec-load` 独立测量真实Worker执行:每次运行固定专属处理器/业务键/序号/模拟时长,最多一次尝试且无外部副作用;任务成功须核对持久化结果、唯一尝试、时间顺序及单次本地调用。队列额度只读,提交不重试;整体截止和有界排空后输出实际分母,未完成/未知结果必须非零退出。数据库阶段耗时不得混用客户端轮询观察时间,不将隔离样本解释为生产容量。 diff --git a/README.md b/README.md index 7ef71f9..189a757 100644 --- a/README.md +++ b/README.md @@ -82,6 +82,8 @@ make protocol-check 受理层负载采样工具见 [有界受理负载工具](docs/api/load.md),只允许在隔离命名空间或一次性数据库中运行;它不自动重试、不启动 Worker,也不提供生产容量承诺。 +真实Worker与任务完成路径测量使用[`orbit-exec-load`](docs/api/execution-load.md):短/混合/中长模拟任务、持久结果和单次尝试核验、阶段延迟分布;不修改队列额度、不调用外部业务系统。 + SDK 独立发布门禁使用 `make sdk-check`,在 `GOWORK=off` 下检查完整依赖图,并运行 SDK 的竞态测试、vet 和全包构建;规则和接入边界见 [SDK 发布门禁](docs/sdk/release.md)。 修改已应用迁移时会触发校验和拒绝;新增结构使用新的前向迁移。初始结构尚未部署生产。 diff --git a/docs/08-development-plan.md b/docs/08-development-plan.md index 9160e62..7ced9ff 100644 --- a/docs/08-development-plan.md +++ b/docs/08-development-plan.md @@ -62,7 +62,7 @@ P4/P5定时等待与后续查询轮次的实现及回归记录见[时钟报告]( 2. 已实现[基础运行观测](api/observability.md)及[有界业务状态与事件采样](reports/phase-6-business-observability.md):HTTP/WS、数据库连接池、后台批次、固定任务/交付/流程/Inbox状态计数、最近事件类别、终态耗时样本、固定事件阶段耗时、最早年龄、截断/新鲜度指标和可离线验证的告警规则;独立采集监听默认关闭,固定指标目录拒绝动态标签;发布前只读结构核验由[`orbit-admin verify`](api/operations.md)提供,运维人员也可用 `orbit-admin observe snapshot` 获取同一有界单节点快照。仍需端到端外部效果延迟、全局持久吞吐、Inbox从事件产生到推进完成的精确阶段延迟、全局发送队列、多节点聚合及生产采集/通知验证。 3. 已补齐受保护管理环境中的固定凭据操作文件、轮换、撤销、幂等回执和脱敏管理审计;轮换保留授权绑定并要求新令牌重新建立会话,旧 Worker 会话被排空且立即过期,撤销允许在停用范围内执行且不改写历史记录。已用一次性 MySQL 定向验证旧令牌失效、新令牌重绑和零外键;仍需补齐提交不明故障切点、多节点安全重绑演练、凭据最小权限配置核对和生产密钥托管/审计演练。 4. 已补[`orbit-admin retention inventory`](api/retention.md)只读盘点和[阶段报告](reports/phase-6-retention-inventory.md),能区分过期幂等、确定终态、未知效果、未终结流程、已确认/未确认交付、已消费/未消费Inbox、事件和审计规模;它不执行归档或删除。后续仍需评审各类保留周期、归档格式、受保护记录核对,再执行备份恢复、兼容升级和六类容量负载。未确认交付、有效幂等身份、当前/历史执行证据和未终结流程不能被普通清理破坏。生产HA及RPO/RTO继续按实际拓扑验证,不缩减为本机多进程检查。 -5. 已补[`orbit-load`](api/load.md)有界受理负载工具:固定请求形状、并发上限、每请求超时、无自动重试、延迟百分位和有限错误分类;已完成24次冒烟及1000次扩大隔离样本,证据见[负载报告](reports/phase-6-load.md)。标准 Go 1.27.1 隔离回归仍需在工具链恢复后复跑;该工具只提供受理层测量,真实 Worker/流程/数据库拓扑容量仍需独立报告。 +5. 已补[`orbit-load`](api/load.md)有界受理负载工具及24次/1000次隔离样本,见[受理报告](reports/phase-6-load.md)。新增独立[`orbit-exec-load`](api/execution-load.md),通过真实Worker、任务结果和尝试历史核对,提供短/混合/中长任务的受理、排队、授权执行及完成耗时分布;具体环境、原始JSON和失败分母见[执行测量报告](reports/phase-6-execution-load.md)。标准Go 1.27.1全量仍待复跑,下一步补定时集中触发、Weave流程推进、长期积压和资源曲线;目标数据库拓扑容量仍单独验收。 6. 已补[隔离备份恢复演练](api/backup-restore.md):源库和恢复库均为一次性 MySQL 容器,逻辑备份恢复后核对迁移账本、核心记录数量、固定索引和零外键;证据见[恢复报告](reports/phase-6-backup-restore.md)。该演练不替代生产备份介质、复制切换、跨版本恢复或 RPO/RTO 验收。 7. 已补[A18 数据库短暂不可用恢复演练](api/db-recovery.md):readiness 在数据库停止期间返回 503,同卷恢复后返回 200,并用原权限范围继续受理任务;证据见[数据库恢复报告](reports/phase-6-db-recovery.md)。该演练不替代主从切换、复制确认、跨故障域或 RPO/RTO 验收。 diff --git a/docs/api/README.md b/docs/api/README.md index 6bd6b9c..f0f85fc 100644 --- a/docs/api/README.md +++ b/docs/api/README.md @@ -24,6 +24,7 @@ | `orbit-admin observe snapshot` 只读运行快照 | 已实现固定状态、事件窗口和阶段耗时的有界单节点快照 | [运维核验](operations.md) | | `orbit-admin retention inventory` 保留清单 | 已实现带 UTC 截止时间的只读盘点;不执行归档、删除或权限变更 | [保留清单](retention.md) | | `sdk/go/cmd/orbit-load` 有界受理负载工具 | 已实现固定并发、无自动重试、延迟百分位和错误分类;只用于隔离环境 | [负载工具](load.md) | +| `sdk/go/cmd/orbit-exec-load` 有界任务执行测量 | 已实现真实Worker、固定混合时长、最终结果/尝试核验及阶段耗时分布 | [执行测量](execution-load.md) | | `scripts/test-backup-restore.sh` 隔离备份恢复演练 | 已实现一次性 MySQL 逻辑备份、独立恢复、核心计数及零外键核验 | [备份恢复演练](backup-restore.md) | | `scripts/test-db-recovery.sh` MySQL 短暂不可用恢复演练 | 已实现 readiness 503、同卷恢复 200 及恢复后任务受理核验 | [数据库恢复演练](db-recovery.md) | | `/v1/worker/connect` | P2 已实现并通过阶段验收 | [Worker 契约](worker.md)、[Schema](worker-schema.md) | diff --git a/docs/api/execution-load.md b/docs/api/execution-load.md new file mode 100644 index 0000000..516b65d --- /dev/null +++ b/docs/api/execution-load.md @@ -0,0 +1,96 @@ +# 有界任务执行测量 + +`sdk/go/cmd/orbit-exec-load` 通过公共 Go SDK 运行 HTTP 提交 → WSS Worker 授权/执行/续租/结果确认 → HTTP 读取持久化状态/尝试历史的完整链路。它测量普通 Pulse 任务;`orbit-load` 继续只测受理,两个命令独立。 + +## 准备及运行 + +使用一次性数据库或专用测试命名空间。凭据需有 `queues:read`、`tasks:submit`、`tasks:read`、`workers:connect`、`workers:consume`,覆盖指定队列。开发测试可通过 `orbit-admin bootstrap ... --worker-access --output /private/execution.credentials.json` 初始化,完整参数见[鉴权手册](authentication.md)。 + +启动匹配版本的 Orbit 后,在SDK模块目录执行: + +```sh +cd sdk/go +export ORBIT_URL='http://127.0.0.1:8080' +export ORBIT_CREDENTIALS_FILE='/private/execution.credentials.json' + +# 空处理器的框架路径 +go run ./cmd/orbit-exec-load \ + -requests 96 -concurrency 8 -workers 2 -capacity 4 \ + -durations 0ms -run-timeout 2m + +# 按序号依次使用0ms、100ms、1s的混合任务 +go run ./cmd/orbit-exec-load \ + -requests 96 -concurrency 8 -workers 2 -capacity 4 \ + -durations 0ms,100ms,1s -run-timeout 2m + +# 超过一个租约周期的任务,观察真实续租 +go run ./cmd/orbit-exec-load \ + -requests 6 -concurrency 3 -workers 2 -capacity 4 \ + -durations 35s -run-timeout 2m +``` + +默认从凭据文件读取token和queue_id,也支持 `ORBIT_TOKEN`、`ORBIT_QUEUE_ID`。环境变量中非空token/queue覆盖文件值;`-queue` 和 `-url` 可覆盖对应环境变量。凭据不作为命令行参数或JSON结果输出。每个Worker均为负载进程中的独立WSS运行器,不是独立主机或故障域;Orbit服务进程保持独立。 + +工具读取并输出当前队列配置,不修改队列和应用额度。因此结果包含实际并发/开始速率限制的影响。运行前记录应用额度、服务连接池、数据库配置、CPU/内存限制及数据量,不能用Worker总capacity直接替代队列的有效许可上限。 + +## 参数与资源预算 + +| 参数 | 默认 | 约束 | +| --- | --- | --- | +| `-requests` | 100 | 1–10000,独立任务总数 | +| `-concurrency` | 8 | 1–128,提交和后续核验两个阶段各自的HTTP并发上限;最多等于任务数 | +| `-workers` | 2 | 1–32条独立WSS Worker连接 | +| `-capacity` | 4 | 每Worker 1–256,总计最多1024个物理处理器槽 | +| `-durations` | `0ms` | 1–16个逗号分隔时长,按序号取模;0–60秒的整毫秒 | +| `-timeout` | 5秒 | 单次HTTP请求1毫秒–1分钟,不是任务时限 | +| `-run-timeout` | 2分钟 | 整体1秒–1小时;必须比最大模拟时长至少长1秒,仍须为排队留余量 | +| `-poll-interval` | 250毫秒 | 每个活跃核验器100毫秒–10秒的查询间隔 | + +整个运行共用截止上下文,结束时所有Worker有界收尾,最多再等5秒。中断或截止停止继续提交/核验,不自动取消、删除已受理任务。未完成任务继续保留原预算和身份;报告失败后不能把它们当作全部未执行。 + +每个任务最多一次执行尝试,安全策略为 `RECONCILE_ONLY`。总时限及单次尝试时限等于 `run-timeout`,以数据库首次受理时间计时;客户端整体截止另从本次运行开始计时。提交HTTP不自动重试。Worker沿现有SDK语义重连、续租和恢复控制确认;未知执行不改成明确失败来凑满成功数。 + +## 固定身份与核验 + +随机 `run_id` 同时进入本次专属处理器名称 `load.exec.{run_id}`、业务键 `orbit-exec-{run_id}`、提交幂等键 `orbit-exec-{run_id}-{sequence六位十进制}` 及输入。序号从0开始,每项输入和预期结果都是 `{run_id, sequence, duration_ms}`。处理器只有可取消等待和回显,不调用文件、数据库或外部业务接口。 + +所有提交发起完后,有界核验器读取已确认的任务及尝试。每一项成功必须同时满足: + +- 受理回执身份唯一,队列/处理器/版本/关联键/输入/尝试预算符合本次固定请求。 +- 任务为 `SUCCEEDED`、`effect_uncertain=false`、`attempt_count=1`,结果匹配预期输入。 +- 尝试历史恰好一项、没有后续页;属于原任务,代次1、`SUCCEEDED`,授权和结束时间齐全。 +- 数据库时间满足创建、可执行、领取、授权、结束顺序,任务完成时间与尝试结束时间一致。 +- 本地处理器对应原任务且该序号恰好调用一次;重复调用及错误输入留在错误计数中。 + +成功任务通过HTTP读取持久化事实确认;SDK的 `ReportsConfirmed`、事件通知或处理器返回nil不能替代核验。 + +## 报告与错误退出 + +标准输出是一行JSON,原始样本见[执行测量报告](../reports/phase-6-execution-load.md)。`complete=true` 才返回退出码0;缺少权限、提交失败/回执未知、读取失败、任务失败/核查、截止、Worker错误或核验不一致均产生不完整报告和非零退出码。 + +| 字段 | 含义 | +| --- | --- | +| `attempted` / `not_submitted` | 实际提交调用数 / 截止前未提交数 | +| `accepted` / `admission_errors` | 已核验唯一回执数 / 明确拒绝或受理结果无法确认数;后者不等于数据库中没有任务 | +| `verified_success` / `unverified` | 已受理项中最终通过完整核验数 / 其余项数 | +| `handler_calls` / `duplicate_invocations` | 本地有效输入处理器调用数 / 同一序号多余调用数 | +| `unconfirmed_handler_calls` | 处理器执行了但没有收到有效受理回执的调用数;整个运行仍失败 | +| `worker_started` / `worker_reconnects` / `worker_renewals_confirmed` | SDK本进程辅助观测,不代表持久成功数 | +| `errors` | 有限错误分类及数量,既有每项错误,也有运行级错误,不应求和为失败任务数 | + +每个耗时分布返回 `count`、`p50_ms`、`p95_ms`、`p99_ms`、`max_ms`,采用最近秩。没有样本时百分位为null。分母分别是: + +| 耗时 | 时钟与分母 | +| --- | --- | +| `admission` | 本机单调时钟;所有实际提交的HTTP交换,包含失败 | +| `queue_wait` | 数据库 `available_at` → `authorized_at`;仅完整核验成功项,包含等待领取及开始授权 | +| `authorized_execution` | 数据库 `authorized_at` → `ended_at`;仅完整核验成功项,包含开始确认传输、模拟处理、结果回传至结束事务取时点 | +| `task_completion` | 数据库 `created_at` → `completed_at`;仅完整核验成功项 | + +后面三个分布使用同一数据库时钟,避免核验轮询顺序污染执行耗时。时间戳记录事务内取时点,不能测量该取时点之后的COMMIT落盘及确认延迟;这些行最终已持久提交由后续HTTP核验确认。它们依赖数据库墙钟;时间逆序会报告 `TIMESTAMP_MISMATCH`。不完整运行的成功子集存在幸存者偏差,不能与完整样本直接比较。整次墙钟包含预检、读取和Worker收尾,不能拿 `requests / 整次耗时` 声称纯执行吞吐。 + +## 回归与生产边界 + +`make integration` 在隔离MySQL及真实服务上执行6项混合任务的命令冒烟;`make sdk-check` 检查独立模块并执行测量计算、失败/超时/错配及重复调用测试。 + +该工具新增普通任务短/混合/中长执行路径测量能力。当前仍需要定时集中触发、Weave流程、长期积压、慢消费者/重连风暴、目标硬件资源曲线、复制切换及跨故障域测试;当前样本不能替代这些生产门槛。 diff --git a/docs/api/load.md b/docs/api/load.md index 76a1caf..c85a8a0 100644 --- a/docs/api/load.md +++ b/docs/api/load.md @@ -2,6 +2,8 @@ `orbit-load` 是独立 Go SDK module 中的受理负载工具,用于在隔离环境测量 HTTP 受理请求的延迟分布。它只提交带独立幂等键的任务,不启动 Worker、不执行业务副作用、不自动重试,也不把测量结果解释为生产吞吐或租户容量保证。 +需要测量真实Worker及任务完成路径时,使用独立的[`orbit-exec-load`](execution-load.md)。 + ## 运行 先为专用测试命名空间准备凭据,再运行: diff --git a/docs/release-checklist.md b/docs/release-checklist.md index 45d510d..eec952d 100644 --- a/docs/release-checklist.md +++ b/docs/release-checklist.md @@ -38,7 +38,7 @@ | 项目 | 必须提供的证据 | 状态 | | --- | --- | --- | -| 受理、派发、流程推进和恢复吞吐 | 固定硬件、数据量、任务/流程混合模型,p50/p95/p99和资源曲线 | 已有24次及1000次隔离受理样本;Worker/流程/积压/资源曲线和生产容量未验收 | +| 受理、派发、流程推进和恢复吞吐 | 固定硬件、数据量、任务/流程混合模型,p50/p95/p99和资源曲线 | 已有受理样本及[真实Worker短/混合/中长执行样本](reports/phase-6-execution-load.md);定时/流程/积压/资源曲线和目标生产容量未验收 | | 100万+企业规模推导 | 租户、队列、连接、事件、索引和保存周期的容量模型 | 未验收,不能由租户数直接推导 | | MySQL 复制、切换、RPO/RTO | 目标拓扑上的实际切换、复制确认、恢复点和恢复时间 | 未验收 | | 慢消费者、重连风暴、磁盘和锁等待 | 受控故障负载及积压清空证据 | 未验收 | diff --git a/docs/reports/data/phase-6-execution-load/deadline.json b/docs/reports/data/phase-6-execution-load/deadline.json new file mode 100644 index 0000000..52a4ea4 --- /dev/null +++ b/docs/reports/data/phase-6-execution-load/deadline.json @@ -0,0 +1 @@ +{"run_id":"a7b51ef05252222cba0c3a458a2a330c","go_version":"go1.25.14","goos":"linux","goarch":"arm64","started_at":"2026-09-19T18:25:48.034952468Z","admission_finished_at":"2026-09-19T18:25:48.127721385Z","finished_at":"2026-09-19T18:25:50.044512177Z","requests":24,"concurrency":4,"workers":1,"capacity_per_worker":1,"durations_ms":[1000],"run_timeout_ms":2000,"poll_interval_ms":250,"http_timeout_ms":5000,"queue":{"max_unfinished_tasks":100000,"unfinished_tasks":"0","id":"61e5494b4cbf4c7126454a074a56412d","namespace_id":"68983a8088c43ef337aa69bbd349686e","name":"execution","paused":false,"max_concurrency":4,"rate_per_second":10,"revision":"1"},"attempted":24,"not_submitted":0,"accepted":24,"admission_errors":0,"verified_success":0,"unverified":24,"handler_calls":1,"duplicate_invocations":0,"unconfirmed_handler_calls":0,"worker_started":1,"worker_reconnects":0,"worker_renewals_confirmed":6,"admission":{"count":24,"p50_ms":12.970542,"p95_ms":20.052458,"p99_ms":22.639583,"max_ms":22.639583},"queue_wait":{"count":0,"p50_ms":null,"p95_ms":null,"p99_ms":null,"max_ms":null},"authorized_execution":{"count":0,"p50_ms":null,"p95_ms":null,"p99_ms":null,"max_ms":null},"task_completion":{"count":0,"p50_ms":null,"p95_ms":null,"p99_ms":null,"max_ms":null},"errors":{"INVOCATION_MISMATCH":23,"RUN_TIMEOUT":1,"TASK_NEEDS_REVIEW":1,"VERIFY_TIMEOUT":23},"complete":false} diff --git a/docs/reports/data/phase-6-execution-load/environment.json b/docs/reports/data/phase-6-execution-load/environment.json new file mode 100644 index 0000000..291fd7a --- /dev/null +++ b/docs/reports/data/phase-6-execution-load/environment.json @@ -0,0 +1,55 @@ +{ + "date_local": "2026-09-20", + "timezone": "Asia/Shanghai", + "test_go": "go1.25.14", + "declared_go": "1.27.1", + "temporary_modfile": true, + "mysql": "8.4.10", + "innodb_flush_log_at_trx_commit": 1, + "sync_binlog": 1, + "mysql_global_transaction_isolation": "REPEATABLE-READ", + "orbit_transaction_isolation": "READ COMMITTED", + "mysql_max_connections": 151, + "innodb_buffer_pool_size_bytes": 134217728, + "orbit_db_max_open": 20, + "orbit_db_max_idle": 5, + "docker_vm_cpus": 12, + "docker_vm_memory_bytes": 12529274880, + "architecture": "arm64", + "orbit_processes": 1, + "mysql_nodes": 1, + "worker_processes": 1, + "worker_connections_normal": 2, + "worker_capacity_each": 4, + "queue_max_concurrency": 4, + "queue_rate_per_second": 10, + "metrics_enabled": false, + "cpu_memory_limits": "No dedicated limits; shared local Docker VM, other workloads possible", + "profiles_in_order": [ + "short", + "mixed", + "long", + "deadline" + ], + "prior_task_counts": [ + 0, + 96, + 192, + 198 + ], + "fresh_disposable_database": true, + "network": "Go runner shares MySQL container network namespace; loopback HTTP/WSS and MySQL", + "schema_verification": { + "schema_version": 28, + "migration_count": 28, + "foreign_key_count": 0, + "required_index_count": 6, + "complete": true + }, + "source_sha256": { + "sdk/go/cmd/orbit-exec-load/main.go": "bcaecce6be9fe8aca7a0d93b4671cbc460b1083232a77d13a04642a96ae45d81", + "sdk/go/cmd/orbit-exec-load/measure.go": "ea0a22e6598e12e175f281fd45d511db32e241e532f5bdcf40ca615392fa586e", + "sdk/go/cmd/orbit-exec-load/measure_test.go": "bda2ce1e946745e149c7471462d511ad3bcf66352a8d14fab57b2aa1ef280c3d", + "sdk/go/cmd/orbit-exec-load/report.go": "3d1da9acc1e63142bd3b4b37041d609e61e172eef56ff27467f1803dee857272" + } +} diff --git a/docs/reports/data/phase-6-execution-load/long.json b/docs/reports/data/phase-6-execution-load/long.json new file mode 100644 index 0000000..b3edc63 --- /dev/null +++ b/docs/reports/data/phase-6-execution-load/long.json @@ -0,0 +1 @@ +{"run_id":"980e0bccdca1cb9fbb1bdcc78536cc90","go_version":"go1.25.14","goos":"linux","goarch":"arm64","started_at":"2026-09-19T18:24:34.92203642Z","admission_finished_at":"2026-09-19T18:24:34.965149878Z","finished_at":"2026-09-19T18:25:48.023298384Z","requests":6,"concurrency":3,"workers":2,"capacity_per_worker":4,"durations_ms":[35000],"run_timeout_ms":120000,"poll_interval_ms":250,"http_timeout_ms":5000,"queue":{"max_unfinished_tasks":100000,"unfinished_tasks":"0","id":"61e5494b4cbf4c7126454a074a56412d","namespace_id":"68983a8088c43ef337aa69bbd349686e","name":"execution","paused":false,"max_concurrency":4,"rate_per_second":10,"revision":"1"},"attempted":6,"not_submitted":0,"accepted":6,"admission_errors":0,"verified_success":6,"unverified":0,"handler_calls":6,"duplicate_invocations":0,"unconfirmed_handler_calls":0,"worker_started":6,"worker_reconnects":0,"worker_renewals_confirmed":18,"admission":{"count":6,"p50_ms":13.193791,"p95_ms":28.028083,"p99_ms":28.028083,"max_ms":28.028083},"queue_wait":{"count":6,"p50_ms":1058.772,"p95_ms":38015.493,"p99_ms":38015.493,"max_ms":38015.493},"authorized_execution":{"count":6,"p50_ms":35013.932,"p95_ms":35024.585,"p99_ms":35024.585,"max_ms":35024.585},"task_completion":{"count":6,"p50_ms":36067.492,"p95_ms":73040.078,"p99_ms":73040.078,"max_ms":73040.078},"errors":{},"complete":true} diff --git a/docs/reports/data/phase-6-execution-load/mixed.json b/docs/reports/data/phase-6-execution-load/mixed.json new file mode 100644 index 0000000..bc8ea46 --- /dev/null +++ b/docs/reports/data/phase-6-execution-load/mixed.json @@ -0,0 +1 @@ +{"run_id":"e78c366e6d6acb6cce8159dae900e8ca","go_version":"go1.25.14","goos":"linux","goarch":"arm64","started_at":"2026-09-19T18:23:52.454385095Z","admission_finished_at":"2026-09-19T18:23:52.759812553Z","finished_at":"2026-09-19T18:24:34.90756817Z","requests":96,"concurrency":8,"workers":2,"capacity_per_worker":4,"durations_ms":[0,100,1000],"run_timeout_ms":120000,"poll_interval_ms":250,"http_timeout_ms":5000,"queue":{"max_unfinished_tasks":100000,"unfinished_tasks":"0","id":"61e5494b4cbf4c7126454a074a56412d","namespace_id":"68983a8088c43ef337aa69bbd349686e","name":"execution","paused":false,"max_concurrency":4,"rate_per_second":10,"revision":"1"},"attempted":96,"not_submitted":0,"accepted":96,"admission_errors":0,"verified_success":96,"unverified":0,"handler_calls":96,"duplicate_invocations":0,"unconfirmed_handler_calls":0,"worker_started":96,"worker_reconnects":0,"worker_renewals_confirmed":0,"admission":{"count":96,"p50_ms":23.680834,"p95_ms":36.472167,"p99_ms":40.274167,"max_ms":40.274167},"queue_wait":{"count":96,"p50_ms":20883.756,"p95_ms":38754.793,"p99_ms":41024.973,"max_ms":41024.973},"authorized_execution":{"count":96,"p50_ms":109.912,"p95_ms":1021.918,"p99_ms":1042.773,"max_ms":1042.773},"task_completion":{"count":96,"p50_ms":20898.154,"p95_ms":39775.854,"p99_ms":42047.09,"max_ms":42047.09},"errors":{},"complete":true} diff --git a/docs/reports/data/phase-6-execution-load/short.json b/docs/reports/data/phase-6-execution-load/short.json new file mode 100644 index 0000000..56ef7c3 --- /dev/null +++ b/docs/reports/data/phase-6-execution-load/short.json @@ -0,0 +1 @@ +{"run_id":"e16f23365b9accaef3ce8b41349d77ea","go_version":"go1.25.14","goos":"linux","goarch":"arm64","started_at":"2026-09-19T18:23:38.326433797Z","admission_finished_at":"2026-09-19T18:23:38.671806422Z","finished_at":"2026-09-19T18:23:52.44758097Z","requests":96,"concurrency":8,"workers":2,"capacity_per_worker":4,"durations_ms":[0],"run_timeout_ms":120000,"poll_interval_ms":250,"http_timeout_ms":5000,"queue":{"max_unfinished_tasks":100000,"unfinished_tasks":"0","id":"61e5494b4cbf4c7126454a074a56412d","namespace_id":"68983a8088c43ef337aa69bbd349686e","name":"execution","paused":false,"max_concurrency":4,"rate_per_second":10,"revision":"1"},"attempted":96,"not_submitted":0,"accepted":96,"admission_errors":0,"verified_success":96,"unverified":0,"handler_calls":96,"duplicate_invocations":0,"unconfirmed_handler_calls":0,"worker_started":96,"worker_reconnects":0,"worker_renewals_confirmed":0,"admission":{"count":96,"p50_ms":26.13175,"p95_ms":38.935083,"p99_ms":54.67775,"max_ms":54.67775},"queue_wait":{"count":96,"p50_ms":5951.689,"p95_ms":12805.411,"p99_ms":13752.092,"max_ms":13752.092},"authorized_execution":{"count":96,"p50_ms":10.405,"p95_ms":23.504,"p99_ms":28.838,"max_ms":28.838},"task_completion":{"count":96,"p50_ms":5959.781,"p95_ms":12813.065,"p99_ms":13768.327,"max_ms":13768.327},"errors":{},"complete":true} diff --git a/docs/reports/phase-6-execution-load.md b/docs/reports/phase-6-execution-load.md new file mode 100644 index 0000000..0bdb431 --- /dev/null +++ b/docs/reports/phase-6-execution-load.md @@ -0,0 +1,71 @@ +# P6:真实 Worker 任务执行测量 + +日期:2026-09-20(Asia/Shanghai,原始报告时间为UTC)。新增独立命令 `sdk/go/cmd/orbit-exec-load`,保留原 `orbit-load` 的纯受理边界。完整参数、权限、错误退出、分母和时钟定义见[执行测量手册](../api/execution-load.md)。本批交付短/混合/中长任务的可运行测量及正确性核验,不代表生产容量验收。 + +## 实现和核验 + +负载程序在业务侧独立SDK模块运行,与Orbit服务进程分离,只使用HTTP/WSS公共接口;没有数据库驱动或服务端内部依赖。每次使用独立处理器、固定序号/时长/输入、唯一提交键。HTTP提交不自动重试,任务最多一次尝试,Worker按现有协议心跳、续租、重连和收尾。 + +成功数量来自持久化任务及尝试历史的读取,逐项核验状态、输入/结果、队列/版本/业务键、单次尝试、授权/结束时间及单次处理器调用。SDK收到控制确认、处理器返回nil和本地事件都不能单独计为成功。截止或任一错配必须以 `complete=false` 和非零退出码结束;没有完成样本时百分位为null。 + +## 最终环境 + +原始配置及当前测量源码SHA-256保存在[环境清单](data/phase-6-execution-load/environment.json)。 + +| 项目 | 实际值 | +| --- | --- | +| 主机/运行环境 | macOS arm64、Docker Linux/arm64;共享VM可见12 CPU、12529274880字节内存,未独占硬件 | +| 工具链 | 实际 `go1.25.14`,仅临时modfile替换Go指令;仓库仍声明Go 1.27.1 | +| 拓扑 | 一次性MySQL 8.4.10,1个Orbit进程,1个负载进程内2条Worker连接,每条capacity=4 | +| 网络 | Go运行容器共享MySQL容器网络命名空间;HTTP/WSS和MySQL均走loopback | +| MySQL | `innodb_flush_log_at_trx_commit=1`、`sync_binlog=1`、buffer pool=128MiB、max_connections=151 | +| 隔离级别 | MySQL全局REPEATABLE-READ;Orbit写事务显式READ COMMITTED | +| Orbit | 数据库连接池max_open=20、max_idle=5;鉴权和持久事件/审计启用;metrics未启用 | +| 队列 | max_concurrency=4、rate_per_second=10、max_unfinished_tasks=100000;工具未更改配置 | +| 核验 | 每个HTTP请求5秒上限、单核验器250ms间隔,核验器最多与提交并发数相同 | +| 数据量 | 空库初始化;short/mixed/long/deadline顺序运行,开始时已有任务数分别0/96/192/198;各运行身份隔离 | + +最后 `orbit-admin verify` 返回 schema=28、migration_count=28、foreign_key_count=0、required_index_count=6、complete=true。测试容器退出后销毁一次性数据库和凭据。 + +## 原始结果 + +前三组均在2分钟整体预算内完整通过,退出0。三个阶段分布的分母分别为96、96、6;都没有重复处理器调用、无回执调用或Worker重连。 + +| 样本 | 任务/处理器时长 | 受理p99 ms | 排队p99 ms | 授权执行p99 ms | 任务完成p99 ms | 持久成功/计划数 | +| --- | --- | --- | --- | --- | --- | --- | +| [short原始JSON](data/phase-6-execution-load/short.json) | 96项,0ms | 54.678 | 13752.092 | 28.838 | 13768.327 | 96/96 | +| [mixed原始JSON](data/phase-6-execution-load/mixed.json) | 96项,0/100/1000ms循环,各32项 | 40.274 | 41024.973 | 1042.773 | 42047.090 | 96/96 | +| [long原始JSON](data/phase-6-execution-load/long.json) | 6项,每项35000ms | 28.028 | 38015.493 | 35024.585 | 73040.078 | 6/6 | + +长任务总计18次续租确认,最终仍为每项一次尝试、一次调用。该数字验证了此次实际续租路径,不表示节点/网络故障已经发生。 + +排队分布包含当前4并发/每秒10次开始的队列约束及调度扫描;Worker声明8个槽不会绕过队列限制。这组数字不能视为去掉配额后的最大吞吐。 + +`authorized_execution` 和 `task_completion` 用持久化数据库时间戳计算;终点是结束事务内部取时点,未包含其后COMMIT及确认延迟。已提交事实由后续读取核验。客户端核验顺序不计入这些分布,但查询负载会影响服务资源消耗。没有收集独占CPU/内存/磁盘/锁等待曲线,也没有多轮稳态和生产指标开启后的对照。 + +## 失败语义实测 + +[deadline原始JSON](data/phase-6-execution-load/deadline.json):24项、4并发提交、1个Worker/1个槽、每项1秒、整体预算2秒。命令按预期退出1,`accepted=24`、`verified_success=0`、`unverified=24`、`complete=false`;本地只有1次处理器调用。读取观察到1项NEEDS_REVIEW,23项未能在截止前验证,三种完成阶段的分布均为 `count=0` / 百分位null。 + +错误计数保留 `RUN_TIMEOUT`、`TASK_NEEDS_REVIEW`、`VERIFY_TIMEOUT` 和 `INVOCATION_MISMATCH`。这些计数含运行级及交叠的核验错误,不能求和当成失败任务总数。工具没有删除、强制成功或重新提交这些任务;未观察到终态不等于数据库确认失败,剩余状态留在原身份下供核查。 + +## 复现和回归 + +按[手册](../api/execution-load.md)准备一次性数据库、当前迁移、带Worker权限的凭据及真实Orbit进程,然后在 `sdk/go` 运行: + +```sh +go run ./cmd/orbit-exec-load -requests 96 -concurrency 8 -workers 2 -capacity 4 -durations 0ms -run-timeout 2m +go run ./cmd/orbit-exec-load -requests 96 -concurrency 8 -workers 2 -capacity 4 -durations 0ms,100ms,1s -run-timeout 2m +go run ./cmd/orbit-exec-load -requests 6 -concurrency 3 -workers 2 -capacity 4 -durations 35s -run-timeout 2m +# 此项应失败;必须读取JSON确认是预算不足,不应仅判断任意非零退出。 +go run ./cmd/orbit-exec-load -requests 24 -concurrency 4 -workers 1 -capacity 1 -durations 1s -run-timeout 2s +``` + +本轮在上述Docker环境使用临时modfile构建真实Linux二进制,执行了四条等价命令及下列检查: + +- SDK全模块 `go test -race ./...`、`go vet ./...`、`go build ./...` 通过;没有设置原SDK集成凭据的普通单测不冒充全量集成。 +- 新增单测覆盖:错配/缺失/空字段结果、未知效果、第二次尝试、悬空历史、时间逆序、提交超时/429不重报、整次取消、核查不算成功、读取失败、重复调用、无权限预检及错误脱敏。 +- SDK完整依赖图218项中,服务端内部/模型/模块和MySQL驱动依赖均为0。 +- `scripts/test-mysql.sh` 已加入6项真实执行命令冒烟,shell语法检查通过。本轮运行的是四个独立真实样本,未重新执行整个 `make integration`。 + +当前不把临时Go 1.25兼容回归升级为声明Go 1.27.1的正式发布结论。下一批继续定时集中触发、流程推进、长期积压、慢消费者/重连风暴及资源曲线,生产HA、复制切换与RPO/RTO仍需目标拓扑证据。 diff --git a/docs/reports/phase-7-regression-status.md b/docs/reports/phase-7-regression-status.md index 7cd4401..ad76921 100644 --- a/docs/reports/phase-7-regression-status.md +++ b/docs/reports/phase-7-regression-status.md @@ -17,6 +17,7 @@ | SDK Worker/Event/Workflow 子进程故障 | 同一共享网络命名空间;容器内构建 Orbit 二进制;SDK `TestIntegration(Worker\|Event\|Workflow)Process` | PASS,136.394 秒;派发未送达、租约接管、ACK 提交回执丢失、重试未知结果、计划/优先级/键并发、流程暂停全停接管均通过 | | SDK 应用配额子进程故障 | 独立双命名空间应用配额凭据;共享网络命名空间;`TestIntegrationWorkerProcessApplicationBackpressure` | PASS,1.069 秒;重启后共享额度、原回执和跨命名空间背压保持 | | Orbit HTTP/metrics 子进程 | 同一共享网络命名空间;`TestIntegrationMetricsProcess`;metrics 启用/禁用各一组 | PASS,3.081 秒;监听隔离、隐私、后台采样和优雅退出均通过 | +| 真实Worker执行测量 | `orbit-exec-load`;最终短/混合/中长及截止不足四组,环境及原始JSON见[测量报告](phase-6-execution-load.md) | 96+96+6项全部逐项核验,长任务18次续租;24项截止负例按预期非零退出。SDK全模块竞态/vet/build通过,生产容量未验收 | ## 当前环境仍未覆盖 diff --git a/scripts/test-mysql.sh b/scripts/test-mysql.sh index 29d315c..3239c1c 100755 --- a/scripts/test-mysql.sh +++ b/scripts/test-mysql.sh @@ -46,6 +46,7 @@ ORBIT_TEST_SERVER_BINARY="$TASK_TEST_DIR/orbit" "$TASK_GO" test -race -count=1 - "$TASK_GO" build -o "$TASK_TEST_DIR/orbit-migrate" ./cmd/orbit-migrate "$TASK_GO" build -o "$TASK_TEST_DIR/orbit-admin" ./cmd/orbit-admin (cd sdk/go && GOWORK=off "$TASK_GO" build -o "$TASK_TEST_DIR/orbit-load" ./cmd/orbit-load) +(cd sdk/go && GOWORK=off "$TASK_GO" build -o "$TASK_TEST_DIR/orbit-exec-load" ./cmd/orbit-exec-load) ORBIT_MYSQL_DSN="$ORBIT_TEST_MYSQL_DSN" "$TASK_TEST_DIR/orbit-migrate" status ORBIT_MYSQL_DSN="$ORBIT_TEST_MYSQL_DSN" "$TASK_TEST_DIR/orbit-admin" verify TASK_SNAPSHOT_JSON=$(ORBIT_MYSQL_DSN="$ORBIT_TEST_MYSQL_DSN" "$TASK_TEST_DIR/orbit-admin" observe snapshot) @@ -80,6 +81,7 @@ if [ -z "$TASK_HTTP_PORT" ]; then printf '%s\n' 'Bootstrap HTTP server did not s (cd sdk/go && ORBIT_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_CREDENTIALS_FILE="$TASK_TEST_DIR/sdk.credentials.json" "$TASK_GO" run ./examples/tasks) (cd sdk/go && ORBIT_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_CREDENTIALS_FILE="$TASK_TEST_DIR/sdk.credentials.json" "$TASK_GO" run ./examples/task-query) ORBIT_MYSQL_DSN="$ORBIT_TEST_MYSQL_DSN" "$TASK_TEST_DIR/orbit-admin" bootstrap --application sdk-worker --namespace default --environment test --queue worker-examples --worker-access --schedule-access --event-access --output "$TASK_TEST_DIR/worker.credentials.json" +(cd sdk/go && ORBIT_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_CREDENTIALS_FILE="$TASK_TEST_DIR/worker.credentials.json" "$TASK_TEST_DIR/orbit-exec-load" -requests 6 -concurrency 2 -workers 2 -capacity 2 -durations 0ms,100ms -run-timeout 30s) (cd sdk/go && ORBIT_TEST_WORKER_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_TEST_WORKER_CREDENTIALS_FILE="$TASK_TEST_DIR/worker.credentials.json" "$TASK_GO" test -race -count=1 -run 'IntegrationWorker(Connection|Runtime)' -v .) (cd sdk/go && ORBIT_TEST_WORKER_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_TEST_WORKER_CREDENTIALS_FILE="$TASK_TEST_DIR/worker.credentials.json" "$TASK_GO" test -race -count=1 -run '^TestIntegrationSchedule' -v .) (cd sdk/go && ORBIT_TEST_WORKER_URL="http://127.0.0.1:$TASK_HTTP_PORT" ORBIT_TEST_WORKER_CREDENTIALS_FILE="$TASK_TEST_DIR/worker.credentials.json" "$TASK_GO" test -race -count=1 -run '^TestIntegrationQueueResourcesSDK$' -v .) diff --git a/sdk/go/cmd/orbit-exec-load/main.go b/sdk/go/cmd/orbit-exec-load/main.go new file mode 100644 index 0000000..a6dba51 --- /dev/null +++ b/sdk/go/cmd/orbit-exec-load/main.go @@ -0,0 +1,139 @@ +// orbit-exec-load 在隔离环境测量真实 HTTP 受理、WSS Worker 和持久化完成链路。 +// 处理器只有有界等待及回显,不访问外部业务系统。 +package main + +import ( + "context" + "encoding/json" + "errors" + "flag" + "fmt" + "io" + "net/http" + "os" + "os/signal" + "strings" + "syscall" + "time" + + orbit "git.yuntuo.hk/k2safe/orbit/sdk/go" +) + +type config struct { + URL, Token, QueueID string + Requests, Concurrency, Workers, Capacity int + Timeout, RunTimeout, PollInterval time.Duration + Durations []time.Duration +} + +func main() { + if err := run(); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} + +func run() error { + cfg, err := parseConfig(os.Args[1:]) + if err != nil { + return err + } + client, err := orbit.NewClient(cfg.URL, orbit.Options{Token: cfg.Token, HTTPClient: &http.Client{Timeout: cfg.Timeout}}) + if err != nil { + return errors.New("invalid client configuration") + } + runID, err := orbit.NewWorkerMessageID() + if err != nil { + return errors.New("cannot generate run identity") + } + ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) + defer stop() + report := measure(ctx, cfg, client, runID) + if err := json.NewEncoder(os.Stdout).Encode(report); err != nil { + return err + } + if !report.Complete { + return errors.New("execution sample incomplete; inspect report and original run identities") + } + return nil +} + +func parseConfig(args []string) (config, error) { + cfg := config{URL: os.Getenv("ORBIT_URL"), Token: os.Getenv("ORBIT_TOKEN"), QueueID: os.Getenv("ORBIT_QUEUE_ID"), Requests: 100, Concurrency: 8, Workers: 2, Capacity: 4, Timeout: 5 * time.Second, RunTimeout: 2 * time.Minute, PollInterval: 250 * time.Millisecond} + flags := flag.NewFlagSet("orbit-exec-load", flag.ContinueOnError) + var credentials, durations string + flags.StringVar(&cfg.URL, "url", cfg.URL, "Orbit HTTP(S) URL") + flags.StringVar(&credentials, "credentials", os.Getenv("ORBIT_CREDENTIALS_FILE"), "private credentials file for an isolated namespace") + flags.StringVar(&cfg.QueueID, "queue", cfg.QueueID, "existing isolated queue; configuration is not changed") + flags.IntVar(&cfg.Requests, "requests", cfg.Requests, "tasks, 1..10000") + flags.IntVar(&cfg.Concurrency, "concurrency", cfg.Concurrency, "maximum concurrent submission/verification requests, 1..128") + flags.IntVar(&cfg.Workers, "workers", cfg.Workers, "independent WSS Workers in this process, 1..32") + flags.IntVar(&cfg.Capacity, "capacity", cfg.Capacity, "concurrent handlers per Worker, 1..256; total at most 1024") + flags.DurationVar(&cfg.Timeout, "timeout", cfg.Timeout, "per HTTP request timeout, 1ms..1m") + flags.DurationVar(&cfg.RunTimeout, "run-timeout", cfg.RunTimeout, "whole sample deadline, 1s..1h, plus at most 5s Worker shutdown") + flags.DurationVar(&cfg.PollInterval, "poll-interval", cfg.PollInterval, "per active verifier polling interval, 100ms..10s") + flags.StringVar(&durations, "durations", "0ms", "1..16 round-robin simulated handler durations, e.g. 0ms,100ms,2s; whole milliseconds, 0..60s") + if err := flags.Parse(args); err != nil { + return config{}, err + } + if flags.NArg() != 0 { + return config{}, errors.New("positional arguments are not supported") + } + if credentials != "" { + f, err := os.Open(credentials) + if err != nil { + return config{}, errors.New("cannot open private credentials file") + } + raw, err := io.ReadAll(io.LimitReader(f, (16<<10)+1)) + f.Close() + var saved struct { + Token string `json:"token"` + QueueID string `json:"queue_id"` + } + if err != nil || len(raw) > 16<<10 || json.Unmarshal(raw, &saved) != nil { + return config{}, errors.New("invalid credentials file (maximum 16 KiB)") + } + if cfg.Token == "" { + cfg.Token = saved.Token + } + if cfg.QueueID == "" { + cfg.QueueID = saved.QueueID + } + } + if cfg.URL == "" || cfg.Token == "" || !validID(cfg.QueueID) { + return config{}, errors.New("URL, token and a 32-character lowercase hexadecimal queue ID are required") + } + if cfg.Requests < 1 || cfg.Requests > 10000 || cfg.Concurrency < 1 || cfg.Concurrency > 128 || cfg.Workers < 1 || cfg.Workers > 32 || cfg.Capacity < 1 || cfg.Capacity > 256 || cfg.Workers*cfg.Capacity > 1024 { + return config{}, errors.New("request, concurrency or Worker count is out of bounds") + } + if cfg.Timeout < time.Millisecond || cfg.Timeout > time.Minute || cfg.RunTimeout < time.Second || cfg.RunTimeout > time.Hour || cfg.PollInterval < 100*time.Millisecond || cfg.PollInterval > 10*time.Second { + return config{}, errors.New("timeout or poll interval is out of bounds") + } + parts := strings.Split(durations, ",") + if len(parts) > 16 { + return config{}, errors.New("at most 16 handler durations are allowed") + } + for _, part := range parts { + d, err := time.ParseDuration(strings.TrimSpace(part)) + if err != nil || d < 0 || d > time.Minute || d%time.Millisecond != 0 || d+time.Second > cfg.RunTimeout { + return config{}, errors.New("handler duration must be whole milliseconds in 0..60s and allow at least 1s within run-timeout") + } + cfg.Durations = append(cfg.Durations, d) + } + if cfg.Concurrency > cfg.Requests { + cfg.Concurrency = cfg.Requests + } + return cfg, nil +} + +func validID(id string) bool { + if len(id) != 32 { + return false + } + for _, c := range id { + if !(c >= '0' && c <= '9' || c >= 'a' && c <= 'f') { + return false + } + } + return true +} diff --git a/sdk/go/cmd/orbit-exec-load/measure.go b/sdk/go/cmd/orbit-exec-load/measure.go new file mode 100644 index 0000000..ec64bad --- /dev/null +++ b/sdk/go/cmd/orbit-exec-load/measure.go @@ -0,0 +1,367 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "io" + "sync" + "time" + + orbit "git.yuntuo.hk/k2safe/orbit/sdk/go" +) + +// loadPayload 的固定内容同时是处理器的输入和预期结果;不含凭据或外部地址。 +type loadPayload struct { + RunID string `json:"run_id"` // RunID 隔离本次运行,不作为业务幂等保证。 + Sequence int `json:"sequence"` // Sequence 从0开始的请求序号,用于发现重复执行及错配。 + DurationMS int64 `json:"duration_ms"` // DurationMS 模拟等待毫秒数,按本次固定时长数组取模。 +} + +func decodePayload(raw json.RawMessage) (loadPayload, bool) { + var wire struct { + RunID *string `json:"run_id"` + Sequence *int `json:"sequence"` + DurationMS *int64 `json:"duration_ms"` + } + decoder := json.NewDecoder(bytes.NewReader(raw)) + decoder.DisallowUnknownFields() + if decoder.Decode(&wire) != nil || wire.RunID == nil || wire.Sequence == nil || wire.DurationMS == nil { + return loadPayload{}, false + } + var trailing any + if decoder.Decode(&trailing) != io.EOF { + return loadPayload{}, false + } + return loadPayload{*wire.RunID, *wire.Sequence, *wire.DurationMS}, true +} + +type admission struct { + sequence int + task orbit.Task + duration time.Duration + code string +} + +type taskAPI interface { + SubmitTask(context.Context, string, orbit.SubmitTaskRequest) (orbit.SubmitTaskResult, error) + GetTask(context.Context, string) (orbit.Task, error) + ListTaskAttempts(context.Context, string, orbit.HistoryPageRequest) (orbit.AttemptPage, error) +} + +type measurement struct { + queue, execution, completed time.Duration +} + +func expected(cfg config, runID string, sequence int) loadPayload { + return loadPayload{runID, sequence, cfg.Durations[sequence%len(cfg.Durations)].Milliseconds()} +} + +func handlerName(runID string) string { return "load.exec." + runID } +func businessKey(runID string) string { return "orbit-exec-" + runID } +func requestKey(runID string, sequence int) string { + return fmt.Sprintf("orbit-exec-%s-%06d", runID, sequence) +} + +// invocationLog 有界保存本次每个序号的实际处理器调用,发现重复时不覆盖首条证据。 +type invocationLog struct { + mu sync.Mutex + calls []int + taskIDs []string + invalid int +} + +func newInvocationLog(n int) *invocationLog { + return &invocationLog{calls: make([]int, n), taskIDs: make([]string, n)} +} + +func (log *invocationLog) handler(runCtx context.Context, cfg config, runID string) orbit.WorkerHandler { + return orbit.WorkerHandler{Capability: orbit.WorkerCapability{Handler: handlerName(runID), HandlerVersion: "v1", PayloadVersion: "v1"}, Handle: func(ctx context.Context, execution orbit.WorkerExecution) (json.RawMessage, error) { + payload, valid := decodePayload(execution.Task.Payload) + if !valid || payload.RunID != runID || payload.Sequence < 0 || payload.Sequence >= cfg.Requests || payload != expected(cfg, runID, payload.Sequence) || execution.Task.QueueID != cfg.QueueID || execution.Task.BusinessKey != businessKey(runID) || execution.Task.Handler != handlerName(runID) { + log.mu.Lock() + log.invalid++ + log.mu.Unlock() + return nil, &orbit.WorkerFailure{Code: "LOAD_INPUT_MISMATCH"} + } + log.mu.Lock() + log.calls[payload.Sequence]++ + first := log.calls[payload.Sequence] == 1 + if first { + log.taskIDs[payload.Sequence] = execution.Task.ID + } + log.mu.Unlock() + if !first { + return nil, &orbit.WorkerFailure{Code: "LOAD_DUPLICATE_EXECUTION"} + } + timer := time.NewTimer(time.Duration(payload.DurationMS) * time.Millisecond) + defer timer.Stop() + select { + case <-ctx.Done(): + return nil, ctx.Err() + case <-runCtx.Done(): + return nil, runCtx.Err() + case <-timer.C: + return json.Marshal(payload) + } + }} +} + +func measure(parent context.Context, cfg config, client *orbit.Client, runID string) executionReport { + r := newReport(cfg, runID) + ctx, cancel := context.WithTimeout(parent, cfg.RunTimeout) + defer cancel() + requestCtx, stop := context.WithTimeout(ctx, cfg.Timeout) + queue, err := client.GetQueue(requestCtx, cfg.QueueID) + stop() + if err != nil { + r.Errors["PREFLIGHT_"+classify(err)]++ + return r.finish() + } + r.Queue = &queue + if queue.ID != cfg.QueueID || queue.Paused || queue.MaxConcurrency == 0 || queue.RatePerSecond == 0 { + r.Errors["QUEUE_UNAVAILABLE"]++ + return r.finish() + } + log := newInvocationLog(cfg.Requests) + var workers []*orbit.Worker + for i := 0; i < cfg.Workers; i++ { + w, err := orbit.NewWorker(client, orbit.WorkerOptions{QueueID: cfg.QueueID, Capacity: uint32(cfg.Capacity), ShutdownTimeout: 5 * time.Second, Handlers: []orbit.WorkerHandler{log.handler(ctx, cfg, runID)}}) + if err != nil { + r.Errors["WORKER_CONFIG"]++ + return r.finish() + } + workers = append(workers, w) + } + workerCtx, stopWorkers := context.WithCancel(ctx) + done := make(chan error, len(workers)) + for _, w := range workers { + go func(w *orbit.Worker) { + err := w.Run(workerCtx) + if workerCtx.Err() == nil { + cancel() + } // 不可恢复连接错误立即停止继续提交。 + done <- err + }(w) + } + admissions := submit(ctx, cfg, client, runID) + var successful []admission + var admissionDurations []time.Duration + seen := map[string]bool{} + for _, a := range admissions { + admissionDurations = append(admissionDurations, a.duration) + if a.code != "" { + r.AdmissionErrors++ + r.Errors["ADMISSION_"+a.code]++ + continue + } + if seen[a.task.ID] { + r.AdmissionErrors++ + r.Errors["DUPLICATE_TASK_ID"]++ + continue + } + seen[a.task.ID] = true + successful = append(successful, a) + } + r.Attempted = len(admissions) + r.Accepted = len(successful) + r.NotSubmitted = cfg.Requests - r.Attempted + r.Admission = distributionOf(admissionDurations) + r.AdmissionFinishedAt = time.Now().UTC() + verified := verifyAll(ctx, cfg, client, runID, successful) + var queued, executed, completed []time.Duration + for _, v := range verified { + if v.code != "" { + r.Errors[v.code]++ + continue + } + r.VerifiedSuccess++ + queued = append(queued, v.measurement.queue) + executed = append(executed, v.measurement.execution) + completed = append(completed, v.measurement.completed) + } + if ctx.Err() != nil { + r.Errors["RUN_"+classify(ctx.Err())]++ + } + stopWorkers() + for range workers { + if err := <-done; err != nil { + r.Errors["WORKER_STOP"]++ + } + } + for _, w := range workers { + s := w.Stats() + r.WorkerStarted += s.Started + r.Reconnects += s.Reconnects + r.Renewals += s.RenewalsConfirmed + } + log.mu.Lock() + for i, calls := range log.calls { + r.HandlerCalls += calls + if calls > 1 { + r.DuplicateInvocations += calls - 1 + } + if calls > 0 && !seen[log.taskIDs[i]] { + r.UnconfirmedHandlerCalls += calls + } + } + for _, a := range successful { + if log.calls[a.sequence] != 1 || log.taskIDs[a.sequence] != a.task.ID { + r.Errors["INVOCATION_MISMATCH"]++ + } + } + r.Errors["INVALID_HANDLER_INPUT"] += log.invalid + log.mu.Unlock() + r.QueueWait = distributionOf(queued) + r.AuthorizedExecution = distributionOf(executed) + r.TaskCompletion = distributionOf(completed) + return r.finish() +} + +func submit(ctx context.Context, cfg config, api taskAPI, runID string) []admission { + jobs := make(chan int) + out := make(chan admission, cfg.Requests) + var wg sync.WaitGroup + for i := 0; i < cfg.Concurrency; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for sequence := range jobs { + if ctx.Err() != nil { + continue + } + payload, _ := json.Marshal(expected(cfg, runID, sequence)) + start := time.Now() + call, stop := context.WithTimeout(ctx, cfg.Timeout) + res, err := api.SubmitTask(call, requestKey(runID, sequence), orbit.SubmitTaskRequest{QueueID: cfg.QueueID, Handler: handlerName(runID), HandlerVersion: "v1", PayloadVersion: "v1", BusinessKey: businessKey(runID), Payload: payload, TaskTimeoutMS: uint64(cfg.RunTimeout.Milliseconds()), RecoverySafety: orbit.RecoveryReconcileOnly, Retry: orbit.RetryPolicy{MaxAttempts: 1, AttemptTimeoutMS: uint64(cfg.RunTimeout.Milliseconds())}}) + stop() + a := admission{sequence: sequence, task: res.Task, duration: time.Since(start)} + if err != nil { + a.code = classify(err) + } else if res.Deduplicated || !validID(res.Task.ID) || !matches(res.Task, cfg, runID, sequence, false) { + a.code = "INVALID_RECEIPT" + } + out <- a + } + }() + } +feed: + for i := 0; i < cfg.Requests; i++ { + select { + case <-ctx.Done(): + break feed + case jobs <- i: + } + } + close(jobs) + wg.Wait() + close(out) + results := make([]admission, 0, cfg.Requests) + for a := range out { + results = append(results, a) + } + return results +} + +type verification struct { + measurement measurement + code string +} + +func verifyAll(ctx context.Context, cfg config, api taskAPI, runID string, admissions []admission) []verification { + jobs := make(chan admission, len(admissions)) + for _, a := range admissions { + jobs <- a + } + close(jobs) + out := make(chan verification, len(admissions)) + var wg sync.WaitGroup + for i := 0; i < cfg.Concurrency; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for a := range jobs { + m, code := verify(ctx, cfg, api, runID, a) + out <- verification{m, code} + } + }() + } + wg.Wait() + close(out) + results := make([]verification, 0, len(admissions)) + for v := range out { + results = append(results, v) + } + return results +} + +func verify(ctx context.Context, cfg config, api taskAPI, runID string, a admission) (measurement, string) { + for { + if ctx.Err() != nil { + return measurement{}, "VERIFY_" + classify(ctx.Err()) + } + call, stop := context.WithTimeout(ctx, cfg.Timeout) + task, err := api.GetTask(call, a.task.ID) + stop() + if err != nil { + return measurement{}, "READ_" + classify(err) + } + if task.ID != a.task.ID || !matches(task, cfg, runID, a.sequence, false) { + return measurement{}, "TASK_MISMATCH" + } + switch task.State { + case orbit.TaskSucceeded: + call, stop = context.WithTimeout(ctx, cfg.Timeout) + history, err := api.ListTaskAttempts(call, task.ID, orbit.HistoryPageRequest{Limit: 2}) + stop() + if err != nil { + return measurement{}, "HISTORY_" + classify(err) + } + return checkCompletion(cfg, runID, a.sequence, task, history) + case orbit.TaskFailed, orbit.TaskCancelled, orbit.TaskNeedsReview: + return measurement{}, "TASK_" + string(task.State) + case orbit.TaskWaiting, orbit.TaskLeased, orbit.TaskRunning: + default: + return measurement{}, "UNKNOWN_TASK_STATE" + } + timer := time.NewTimer(cfg.PollInterval) + select { + case <-ctx.Done(): + timer.Stop() + return measurement{}, "VERIFY_" + classify(ctx.Err()) + case <-timer.C: + } + } +} + +func matches(task orbit.Task, cfg config, runID string, sequence int, result bool) bool { + payload, valid := decodePayload(task.Payload) + if task.QueueID != cfg.QueueID || task.Handler != handlerName(runID) || task.HandlerVersion != "v1" || task.PayloadVersion != "v1" || task.BusinessKey != businessKey(runID) || task.Retry.MaxAttempts != 1 || task.RecoverySafety != orbit.RecoveryReconcileOnly || !valid || payload != expected(cfg, runID, sequence) { + return false + } + if result { + output, valid := decodePayload(task.Result) + if !valid || output != expected(cfg, runID, sequence) { + return false + } + } + return true +} + +func checkCompletion(cfg config, runID string, sequence int, task orbit.Task, history orbit.AttemptPage) (measurement, string) { + if task.State != orbit.TaskSucceeded || task.EffectUncertain || task.AttemptCount != 1 || task.CompletedAt == nil || !matches(task, cfg, runID, sequence, true) { + return measurement{}, "COMPLETION_MISMATCH" + } + if len(history.Items) != 1 || history.NextCursor != "" { + return measurement{}, "ATTEMPT_COUNT_MISMATCH" + } + a := history.Items[0] + if !validID(a.ID) || a.TaskID != task.ID || a.Epoch != 1 || a.State != orbit.AttemptSucceeded || a.AuthorizedAt == nil || a.EndedAt == nil { + return measurement{}, "ATTEMPT_MISMATCH" + } + if task.CreatedAt.IsZero() || task.AvailableAt.Before(task.CreatedAt) || a.CreatedAt.Before(task.AvailableAt) || a.AuthorizedAt.Before(a.CreatedAt) || a.EndedAt.Before(*a.AuthorizedAt) || !a.EndedAt.Equal(*task.CompletedAt) { + return measurement{}, "TIMESTAMP_MISMATCH" + } + return measurement{queue: a.AuthorizedAt.Sub(task.AvailableAt), execution: a.EndedAt.Sub(*a.AuthorizedAt), completed: task.CompletedAt.Sub(task.CreatedAt)}, "" +} diff --git a/sdk/go/cmd/orbit-exec-load/measure_test.go b/sdk/go/cmd/orbit-exec-load/measure_test.go new file mode 100644 index 0000000..d26049e --- /dev/null +++ b/sdk/go/cmd/orbit-exec-load/measure_test.go @@ -0,0 +1,268 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "fmt" + "net/http" + "net/http/httptest" + "reflect" + "strings" + "sync" + "testing" + "time" + + orbit "git.yuntuo.hk/k2safe/orbit/sdk/go" +) + +const testRunID = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa" +const testQueueID = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb" + +func testConfig() config { + return config{QueueID: testQueueID, Requests: 6, Concurrency: 3, Workers: 2, Capacity: 2, Timeout: time.Second, RunTimeout: time.Minute, PollInterval: 100 * time.Millisecond, Durations: []time.Duration{0, 10 * time.Millisecond}} +} + +func completedFixture(cfg config) (orbit.Task, orbit.AttemptPage) { + t0 := time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC) + leasing, auth, end := t0.Add(time.Second), t0.Add(2*time.Second), t0.Add(3*time.Second) + payload, _ := json.Marshal(expected(cfg, testRunID, 0)) + task := orbit.Task{ID: "cccccccccccccccccccccccccccccccc", QueueID: cfg.QueueID, Handler: handlerName(testRunID), HandlerVersion: "v1", PayloadVersion: "v1", BusinessKey: businessKey(testRunID), Payload: payload, Result: payload, State: orbit.TaskSucceeded, AttemptCount: 1, CreatedAt: t0, AvailableAt: t0, CompletedAt: &end, RecoverySafety: orbit.RecoveryReconcileOnly, Retry: orbit.RetryPolicy{MaxAttempts: 1}} + history := orbit.AttemptPage{Items: []orbit.AttemptSummary{{ID: "dddddddddddddddddddddddddddddddd", TaskID: task.ID, Epoch: 1, State: orbit.AttemptSucceeded, CreatedAt: leasing, AuthorizedAt: &auth, EndedAt: &end}}} + return task, history +} + +func TestCompletionRequiresResultSingleAttemptAndOrderedServerTimestamps(t *testing.T) { + cfg := testConfig() + task, history := completedFixture(cfg) + m, code := checkCompletion(cfg, testRunID, 0, task, history) + if code != "" || m.queue != 2*time.Second || m.execution != time.Second || m.completed != 3*time.Second { + t.Fatal(m, code) + } + for _, tc := range []struct { + name string + mutate func(*orbit.Task, *orbit.AttemptPage) + }{ + {"wrong-result", func(v *orbit.Task, _ *orbit.AttemptPage) { v.Result = json.RawMessage(`{}`) }}, + {"missing-zero-valued-fields", func(v *orbit.Task, _ *orbit.AttemptPage) { + v.Result = json.RawMessage(`{"run_id":"` + testRunID + `"}`) + }}, + {"null-zero-valued-fields", func(v *orbit.Task, _ *orbit.AttemptPage) { + v.Result = json.RawMessage(`{"run_id":"` + testRunID + `","sequence":null,"duration_ms":null}`) + }}, + {"result-other-run", func(v *orbit.Task, _ *orbit.AttemptPage) { + v.Result = json.RawMessage(`{"run_id":"other","sequence":0,"duration_ms":0}`) + }}, + {"unknown-effect", func(v *orbit.Task, _ *orbit.AttemptPage) { v.EffectUncertain = true }}, + {"wrong-task", func(_ *orbit.Task, h *orbit.AttemptPage) { h.Items[0].TaskID = testRunID }}, + {"second-attempt", func(_ *orbit.Task, h *orbit.AttemptPage) { h.Items = append(h.Items, h.Items[0]) }}, + {"more-attempt-pages", func(_ *orbit.Task, h *orbit.AttemptPage) { h.NextCursor = "more" }}, + {"missing-authorization", func(_ *orbit.Task, h *orbit.AttemptPage) { h.Items[0].AuthorizedAt = nil }}, + {"reversed-time", func(v *orbit.Task, h *orbit.AttemptPage) { h.Items[0].EndedAt = &v.CreatedAt }}, + {"completion-not-settled", func(v *orbit.Task, _ *orbit.AttemptPage) { v.CompletedAt = nil }}, + {"wrong-budget", func(v *orbit.Task, _ *orbit.AttemptPage) { v.Retry.MaxAttempts = 2 }}, + } { + t.Run(tc.name, func(t *testing.T) { + v, h := completedFixture(cfg) + tc.mutate(&v, &h) + if _, code := checkCompletion(cfg, testRunID, 0, v, h); code == "" { + t.Fatal("invalid completion accepted") + } + }) + } +} + +type stubAPI struct { + submit func(context.Context, string, orbit.SubmitTaskRequest) (orbit.SubmitTaskResult, error) + get func(context.Context, string) (orbit.Task, error) + history func(context.Context, string, orbit.HistoryPageRequest) (orbit.AttemptPage, error) +} + +func (s stubAPI) SubmitTask(c context.Context, k string, v orbit.SubmitTaskRequest) (orbit.SubmitTaskResult, error) { + return s.submit(c, k, v) +} +func (s stubAPI) GetTask(c context.Context, k string) (orbit.Task, error) { return s.get(c, k) } +func (s stubAPI) ListTaskAttempts(c context.Context, k string, v orbit.HistoryPageRequest) (orbit.AttemptPage, error) { + return s.history(c, k, v) +} + +func TestSubmissionIsBoundedDoesNotRetryOrCountAmbiguousReceipts(t *testing.T) { + cfg := testConfig() + var mu sync.Mutex + keys := map[string]int{} + active, peak := 0, 0 + api := stubAPI{submit: func(ctx context.Context, key string, in orbit.SubmitTaskRequest) (orbit.SubmitTaskResult, error) { + mu.Lock() + keys[key]++ + active++ + if active > peak { + peak = active + } + mu.Unlock() + defer func() { mu.Lock(); active--; mu.Unlock() }() + var payload loadPayload + if json.Unmarshal(in.Payload, &payload) != nil { + t.Error("invalid sample payload") + } + if key != requestKey(testRunID, payload.Sequence) || in.Retry.MaxAttempts != 1 { + t.Error("identity or retry budget changed") + } + if payload.Sequence == 0 { + <-ctx.Done() + return orbit.SubmitTaskResult{}, ctx.Err() + } + if payload.Sequence == 1 { + return orbit.SubmitTaskResult{}, &orbit.APIError{StatusCode: 429, Code: "QUEUE_FULL"} + } + task := orbit.Task{ID: fmt.Sprintf("%032x", payload.Sequence), QueueID: in.QueueID, Handler: in.Handler, HandlerVersion: in.HandlerVersion, PayloadVersion: in.PayloadVersion, BusinessKey: in.BusinessKey, Payload: in.Payload, Retry: in.Retry, RecoverySafety: in.RecoverySafety} + return orbit.SubmitTaskResult{Task: task}, nil + }} + cfg.Timeout = 30 * time.Millisecond + results := submit(context.Background(), cfg, api, testRunID) + if len(results) != cfg.Requests || len(keys) != cfg.Requests || peak > cfg.Concurrency { + t.Fatal("lost requests or exceeded bound", len(results), len(keys), peak) + } + counts := map[string]int{} + for _, a := range results { + counts[a.code]++ + } + if !reflect.DeepEqual(counts, map[string]int{"TIMEOUT": 1, "BACKPRESSURE": 1, "": 4}) { + t.Fatal(counts) + } + for _, n := range keys { + if n != 1 { + t.Fatal("submission retried") + } + } + ctx, cancel := context.WithCancel(context.Background()) + cancel() + if len(submit(ctx, cfg, api, testRunID)) != 0 { + t.Fatal("submitted after cancellation") + } +} + +func TestVerificationCannotPromoteReviewTimeoutOrMissingHistory(t *testing.T) { + cfg := testConfig() + task, history := completedFixture(cfg) + for _, state := range []orbit.TaskState{orbit.TaskNeedsReview, orbit.TaskFailed, orbit.TaskCancelled} { + v := task + v.State = state + api := stubAPI{get: func(context.Context, string) (orbit.Task, error) { return v, nil }} + if _, code := verify(context.Background(), cfg, api, testRunID, admission{task: task}); code != "TASK_"+string(state) { + t.Fatal(state, code) + } + } + waiting := task + waiting.State = orbit.TaskRunning + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Millisecond) + defer cancel() + api := stubAPI{get: func(context.Context, string) (orbit.Task, error) { return waiting, nil }} + if _, code := verify(ctx, cfg, api, testRunID, admission{task: task}); code != "VERIFY_TIMEOUT" { + t.Fatal(code) + } + api.get = func(context.Context, string) (orbit.Task, error) { return task, nil } + api.history = func(context.Context, string, orbit.HistoryPageRequest) (orbit.AttemptPage, error) { + return orbit.AttemptPage{}, errors.New("private history error") + } + if _, code := verify(context.Background(), cfg, api, testRunID, admission{task: task}); code != "HISTORY_TRANSPORT" { + t.Fatal(code) + } + api.history = func(context.Context, string, orbit.HistoryPageRequest) (orbit.AttemptPage, error) { + return history, nil + } + if _, code := verify(context.Background(), cfg, api, testRunID, admission{task: task}); code != "" { + t.Fatal(code) + } +} + +func TestHandlerDetectsDuplicatesAndForeignPayloadWithoutExternalEffects(t *testing.T) { + cfg := testConfig() + task, _ := completedFixture(cfg) + log := newInvocationLog(cfg.Requests) + h := log.handler(context.Background(), cfg, testRunID) + result, err := h.Handle(context.Background(), orbit.WorkerExecution{Task: task}) + if err != nil || string(result) != string(task.Payload) { + t.Fatal(err, string(result)) + } + if _, err = h.Handle(context.Background(), orbit.WorkerExecution{Task: task}); err == nil { + t.Fatal("duplicate execution accepted") + } + task.Payload = json.RawMessage(`{"run_id":"other","sequence":0,"duration_ms":0}`) + if _, err = h.Handle(context.Background(), orbit.WorkerExecution{Task: task}); err == nil { + t.Fatal("foreign sample accepted") + } + if log.calls[0] != 2 || log.invalid != 1 { + t.Fatal("lost invocation evidence") + } +} + +func TestReportDenominatorsAndRedaction(t *testing.T) { + r := newReport(testConfig(), testRunID) + r.Accepted = 5 + r.VerifiedSuccess = 4 + r.HandlerCalls = 6 + r = r.finish() + if r.Complete || r.Unverified != 1 { + t.Fatal("incomplete sample reported complete") + } + raw, _ := json.Marshal(r) + if !strings.Contains(string(raw), `"p99_ms":null`) { + t.Fatal("empty distribution not explicit") + } + d := distributionOf([]time.Duration{3 * time.Millisecond, time.Millisecond, 2 * time.Millisecond}) + if d.Count != 3 || *d.P50 != 2 || *d.P99 != 3 { + t.Fatal(d) + } + for _, err := range []error{errors.New("secret"), &orbit.APIError{StatusCode: 503, Code: "secret", Message: "token"}} { + if strings.Contains(classify(err), "secret") { + t.Fatal("error leaked") + } + } +} + +func TestConfigBoundsAndDeterministicMix(t *testing.T) { + t.Setenv("ORBIT_URL", "http://127.0.0.1:8080") + t.Setenv("ORBIT_TOKEN", "test-token") + t.Setenv("ORBIT_QUEUE_ID", testQueueID) + t.Setenv("ORBIT_CREDENTIALS_FILE", "") + cfg, err := parseConfig([]string{"-durations", "0ms,10ms,2s", "-requests", "3", "-concurrency", "8"}) + if err != nil || cfg.Concurrency != 3 || expected(cfg, testRunID, 5).DurationMS != 2000 { + t.Fatal(cfg, err) + } + for _, args := range [][]string{{"-requests", "0"}, {"-workers", "32", "-capacity", "256"}, {"-durations", "-1s"}, {"-durations", "1us"}, {"-durations", "60s", "-run-timeout", "60s"}, {"-poll-interval", "1ms"}, {"-timeout", "2h"}, {"-run-timeout", "2h"}} { + if _, err := parseConfig(args); err == nil { + t.Fatal("invalid bounds accepted", args) + } + } +} + +func TestPreflightDenialNeverSubmitsOrStartsWorkers(t *testing.T) { + var mu sync.Mutex + var paths []string + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + mu.Lock() + paths = append(paths, r.URL.Path) + mu.Unlock() + w.Header().Set("Content-Type", "application/json") + w.WriteHeader(403) + fmt.Fprint(w, `{"request_id":"`+testRunID+`","error":{"code":"FORBIDDEN","message":"private data"}}`) + })) + defer server.Close() + client, err := orbit.NewClient(server.URL, orbit.Options{Token: "test-token"}) + if err != nil { + t.Fatal(err) + } + r := measure(context.Background(), testConfig(), client, testRunID) + if r.Complete || r.Attempted != 0 || r.NotSubmitted != 6 || r.Errors["PREFLIGHT_AUTH"] != 1 { + t.Fatal(r) + } + mu.Lock() + defer mu.Unlock() + if !reflect.DeepEqual(paths, []string{"/v1/queues/" + testQueueID}) { + t.Fatal(paths) + } + raw, _ := json.Marshal(r) + if strings.Contains(string(raw), "private data") || strings.Contains(string(raw), "test-token") { + t.Fatal("private data leaked") + } +} diff --git a/sdk/go/cmd/orbit-exec-load/report.go b/sdk/go/cmd/orbit-exec-load/report.go new file mode 100644 index 0000000..fbc63ea --- /dev/null +++ b/sdk/go/cmd/orbit-exec-load/report.go @@ -0,0 +1,121 @@ +package main + +import ( + "context" + "errors" + "math" + "runtime" + "sort" + "time" + + orbit "git.yuntuo.hk/k2safe/orbit/sdk/go" +) + +// distribution 只计算明确纳入的样本;空集合的百分位返回null,不能冒充零延迟。 +type distribution struct { + Count int `json:"count"` // Count 样本数;完成阶段仅包含通过持久化核验的成功任务。 + P50 *float64 `json:"p50_ms"` // P50 最近秩中位数,毫秒。 + P95 *float64 `json:"p95_ms"` // P95 最近秩95百分位,毫秒。 + P99 *float64 `json:"p99_ms"` // P99 最近秩99百分位,毫秒。 + Max *float64 `json:"max_ms"` // Max 样本最大耗时,毫秒。 +} + +// executionReport 是单次有界测量结果,不是容量承诺或持久化审计。 +type executionReport struct { + RunID string `json:"run_id"` // RunID 固定提交键和处理器身份的组成部分。 + GoVersion string `json:"go_version"` // GoVersion 实际编译工具链版本,不从仓库要求推断。 + GOOS string `json:"goos"` // GOOS 负载进程操作系统。 + GOARCH string `json:"goarch"` // GOARCH 负载进程架构。 + StartedAt time.Time `json:"started_at"` // StartedAt 本机开始时间,包含预检。 + AdmissionFinishedAt time.Time `json:"admission_finished_at"` // AdmissionFinishedAt 本机受理组结束时间;预检失败时为零值。 + FinishedAt time.Time `json:"finished_at"` // FinishedAt 核验及Worker收尾结束时间。 + Requests int `json:"requests"` // Requests 计划独立任务数。 + Concurrency int `json:"concurrency"` // Concurrency 提交及核验各自最大并发。 + Workers int `json:"workers"` // Workers 同一负载进程内独立WSS运行器数。 + Capacity int `json:"capacity_per_worker"` // Capacity 每个运行器物理处理器并发上限。 + DurationsMS []int64 `json:"durations_ms"` // DurationsMS 按序号循环选取的模拟等待时长。 + RunTimeoutMS int64 `json:"run_timeout_ms"` // RunTimeoutMS 整体期限,收尾最多另加5秒。 + PollIntervalMS int64 `json:"poll_interval_ms"` // PollIntervalMS 每个活跃核验器两次读取之间的间隔。 + HTTPTimeoutMS int64 `json:"http_timeout_ms"` // HTTPTimeoutMS 单次HTTP请求时限,不是任务时限。 + Queue *orbit.Queue `json:"queue,omitempty"` // Queue 预检时队列额度快照;工具不会修改配置。 + Attempted int `json:"attempted"` // Attempted 实际发起的提交调用数,不自动重试。 + NotSubmitted int `json:"not_submitted"` // NotSubmitted 截止/错误前未发起的项数。 + Accepted int `json:"accepted"` // Accepted 收到并核验唯一回执的项数。 + AdmissionErrors int `json:"admission_errors"` // AdmissionErrors 明确拒绝或提交结果无法确认的项数,不能当作全部未落库。 + VerifiedSuccess int `json:"verified_success"` // VerifiedSuccess 最终任务、尝试及结果均通过核验的任务数。 + Unverified int `json:"unverified"` // Unverified 已受理但未获核验成功的数量,包括失败/核查/仍待执行。 + HandlerCalls int `json:"handler_calls"` // HandlerCalls 本进程有效输入处理器调用数。 + DuplicateInvocations int `json:"duplicate_invocations"` // DuplicateInvocations 同一序号超过一次的调用数。 + UnconfirmedHandlerCalls int `json:"unconfirmed_handler_calls"` // UnconfirmedHandlerCalls 没有已确认受理回执的处理器调用数。 + WorkerStarted uint64 `json:"worker_started"` // WorkerStarted SDK本进程统计,不替代持久化成功数。 + Reconnects uint64 `json:"worker_reconnects"` // Reconnects SDK重连计数。 + Renewals uint64 `json:"worker_renewals_confirmed"` // Renewals SDK续租确认数。 + Admission distribution `json:"admission"` // Admission 全部提交HTTP交换的本机单调时钟耗时,含失败。 + QueueWait distribution `json:"queue_wait"` // QueueWait 数据库available_at至authorized_at,含领取及开始授权。 + AuthorizedExecution distribution `json:"authorized_execution"` // AuthorizedExecution 数据库authorized_at至ended_at,止于结束事务取时点,不包含其后提交确认。 + TaskCompletion distribution `json:"task_completion"` // TaskCompletion 数据库created_at至completed_at,不受客户端轮询顺序影响。 + Errors map[string]int `json:"errors"` // Errors 固定分类,不保存内部异常、SQL、凭据或请求原文。 + Complete bool `json:"complete"` // Complete 仅所有计划任务和本地调用逐项核验且无错误时为true。 +} + +func newReport(cfg config, runID string) executionReport { + r := executionReport{RunID: runID, GoVersion: runtime.Version(), GOOS: runtime.GOOS, GOARCH: runtime.GOARCH, StartedAt: time.Now().UTC(), Requests: cfg.Requests, NotSubmitted: cfg.Requests, Concurrency: cfg.Concurrency, Workers: cfg.Workers, Capacity: cfg.Capacity, RunTimeoutMS: cfg.RunTimeout.Milliseconds(), PollIntervalMS: cfg.PollInterval.Milliseconds(), HTTPTimeoutMS: cfg.Timeout.Milliseconds(), Errors: map[string]int{}} + for _, d := range cfg.Durations { + r.DurationsMS = append(r.DurationsMS, d.Milliseconds()) + } + return r +} +func (r executionReport) finish() executionReport { + for code, count := range r.Errors { + if count == 0 { + delete(r.Errors, code) + } + } + r.FinishedAt = time.Now().UTC() + r.Unverified = r.Accepted - r.VerifiedSuccess + r.Complete = r.Attempted == r.Requests && r.NotSubmitted == 0 && r.AdmissionErrors == 0 && r.Accepted == r.Requests && r.VerifiedSuccess == r.Requests && r.HandlerCalls == r.Requests && r.WorkerStarted == uint64(r.Requests) && r.DuplicateInvocations == 0 && r.UnconfirmedHandlerCalls == 0 && len(r.Errors) == 0 + return r +} +func distributionOf(values []time.Duration) distribution { + d := distribution{Count: len(values)} + if len(values) == 0 { + return d + } + sort.Slice(values, func(i, j int) bool { return values[i] < values[j] }) + percentile := func(q float64) *float64 { + v := float64(values[int(math.Ceil(float64(len(values))*q))-1]) / float64(time.Millisecond) + return &v + } + d.P50 = percentile(.5) + d.P95 = percentile(.95) + d.P99 = percentile(.99) + d.Max = percentile(1) + return d +} +func classify(err error) string { + if errors.Is(err, context.DeadlineExceeded) { + return "TIMEOUT" + } + if errors.Is(err, context.Canceled) { + return "CANCELLED" + } + var api *orbit.APIError + if errors.As(err, &api) { + switch api.StatusCode { + case 401, 403: + return "AUTH" + case 409: + return "CONFLICT" + case 429: + return "BACKPRESSURE" + case 400, 422: + return "INVALID_REQUEST" + } + return "HTTP" + } + var h *orbit.HTTPError + if errors.As(err, &h) { + return "HTTP" + } + return "TRANSPORT" +}