Compare commits
66 Commits
v0.5.0
...
04a1f7999d
| Author | SHA1 | Date | |
|---|---|---|---|
| 04a1f7999d | |||
| 208a9c3fbd | |||
| f91034477b | |||
| b9259e628e | |||
| fe66221a9c | |||
| 467c890363 | |||
| be620d2243 | |||
| 8e18528fc2 | |||
| 59bf04a461 | |||
| b54a303ccd | |||
| 84670f7be1 | |||
| fa2be5e016 | |||
| dfe8a1a9af | |||
| 830cb3a7ad | |||
| d94df4fc54 | |||
| 1a534c7976 | |||
| 2b93545507 | |||
| 15825f0ab8 | |||
| 1b9215114f | |||
| 73a05b8c3c | |||
| 64a9fde204 | |||
| ca0e1d6b92 | |||
| 98f17660c8 | |||
| c8660026ce | |||
| 695bdb7a00 | |||
| 7e9e8aa67c | |||
| de33eda846 | |||
| dd7d3fef0a | |||
| 46c5c9406c | |||
| 1487a96f67 | |||
| 1a20ab53ac | |||
| a072da4a31 | |||
| fcd6f48b53 | |||
| 83633f724e | |||
| a523e6ba3f | |||
| 4a8feaca36 | |||
| 95dc4568f6 | |||
| 2259606f72 | |||
| 35b7470be2 | |||
| 4f97aa0f3d | |||
| 3b4d0a3fad | |||
| a4d843b8f4 | |||
| fadd17a4d4 | |||
| ac46071dfe | |||
| 52d603c31e | |||
| 42021538b2 | |||
| ca707761fc | |||
| 30fdad44c2 | |||
| b7dc36fc99 | |||
| 8e474a24d9 | |||
| ef98d1dd8d | |||
| 5ab906098f | |||
| c5e415ca52 | |||
| 9c9c8ab59a | |||
| e6c82a01fe | |||
| 847621d218 | |||
| 4daf3b211c | |||
| 62b0f09546 | |||
| 694374fde8 | |||
| 22f1c9ebfb | |||
| b3cb9e8857 | |||
| efd3163902 | |||
| 241a568c83 | |||
| 22939fdb0b | |||
| 3d5f225fb2 | |||
| c36bb70f96 |
149
ADVANCED.md
Normal file
149
ADVANCED.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# 扩展业务模块
|
||||
|
||||
本批提供八类扩展,以及分页、审计和显式数据库迁移。模块按需导入;`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase;Go 导出标识符和 `_test.go` 保持语言约定。
|
||||
|
||||
按本次范围决定,**不集成 Orbit,不引入 Orbit SDK,也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit,可独立依赖它的 SDK;当前没有承诺只换队列适配器就能迁移其全部语义。
|
||||
|
||||
## 能力与边界
|
||||
|
||||
| 模块 | 本批实现 | 明确边界 |
|
||||
| --- | --- | --- |
|
||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||
| `storage` | Local、S3 兼容(含 R2 配置)、OSS 官方 SDK 适配、临时下载链接 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||
|
||||
## 对象存储
|
||||
|
||||
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||
|
||||
```go
|
||||
type FileService struct { Files storage.Store }
|
||||
|
||||
// Local:私有目录,流式临时文件 + 原子替换。
|
||||
files, err := local.New("/srv/app/objects", 32<<20)
|
||||
|
||||
// S3:AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
|
||||
files, err := s3.Open(s3.Config{
|
||||
Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
|
||||
Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
|
||||
Credentials: credentialProvider, Transport: transport,
|
||||
})
|
||||
|
||||
// Cloudflare R2:复用 S3 适配器,签名区域为 auto。
|
||||
cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
|
||||
cfg.Transport = transport
|
||||
files, err := s3.Open(cfg)
|
||||
|
||||
// OSS:单独适配官方 Go SDK v2,保留平台签名与凭据提供器。
|
||||
client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
|
||||
WithRegion("cn-hangzhou").
|
||||
WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
|
||||
files, err := oss.New(client, bucket, 32<<20)
|
||||
```
|
||||
|
||||
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||
|
||||
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,调用方保证与内容一致。默认单对象 32 MiB,流式读写;OSS 本适配器只做 simple upload,最大配置 5 GiB。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||
|
||||
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。首版未封装前端直传、分片续传、生命周期、ACL、CDN 自定义域名和版本管理。
|
||||
|
||||
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||
|
||||
验证区分:Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试;Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限,不只是切配置。
|
||||
|
||||
## 可靠队列
|
||||
|
||||
```go
|
||||
backend, err := queueRedis.New(redisClient, queueRedis.Config{
|
||||
Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
|
||||
MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
|
||||
})
|
||||
consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
|
||||
Workers: 4, MaxAttempts: 3,
|
||||
TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
|
||||
})
|
||||
|
||||
// ID 与内容在业务端持久化;超时后复用原 ID 和原内容。
|
||||
receipt, err := backend.Publish(ctx, queue.Message{
|
||||
ID: operationID, Name: "generateReport", Payload: payload,
|
||||
})
|
||||
|
||||
// 在服务生命周期中运行;返回错误须让监控/进程管理感知。
|
||||
err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
|
||||
return generateReportIdempotently(ctx, item.ID, item.Payload)
|
||||
})
|
||||
```
|
||||
|
||||
业务依赖 `queue.Publisher` / `queue.Consumer`,Redis 类型留在组合处。receipt 表示受理,不代表业务成功。普通 error/panic 按配置重试,`queue.Reject(code)` 保留为失败消息;超过次数同样保留。`DeadLetters(after, limit)` 是有界分页,`DiscardDead` 仅在人工核对后删除匹配的失败消息,并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略,框架不自动无限重投。
|
||||
|
||||
一个 namespace 使用一个受管 consumer group;多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner,旧执行不能确认新执行;但不能阻止旧处理器已发出的外部副作用,因此仍必须业务幂等。处理器需响应 context;Go 无法强杀不配合的函数,超过 shutdownTimeout 返回错误时,最多 Workers 个处理器可能尚未退出。
|
||||
|
||||
容量包含待处理、失败及未过期的已完成身份,满额拒绝新 ID,保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期;未完成/失败消息不自动清除,不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction;本批验证的是投递/接管协议,未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理,不能用 DELCONSUMER 丢掉尚未接管的 pending。
|
||||
|
||||
`task.Runner` 仍适用于本进程短任务和定时轮询,重启会丢失;需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
|
||||
|
||||
## 认证与授权
|
||||
|
||||
`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表;只开放 RS256 / EdDSA,必须有 exp/sub,检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表,轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
|
||||
|
||||
`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token;存储仅使用 token 的 SHA-256 键,可 `Revoke`。Redis 缓存可共享 Session,内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
|
||||
|
||||
`RequireAuth(authenticator)` 在前,`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal,业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
|
||||
|
||||
## 配置与日志热更新
|
||||
|
||||
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase,duration 可以写 `"5s"`;文件上限 1 MiB,拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线,例如 `BILLING_`。空环境变量沿用文件值,不清空已配置字段。
|
||||
|
||||
```yaml
|
||||
address: '127.0.0.1:8080'
|
||||
transport: native
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
level: info
|
||||
format: json
|
||||
```
|
||||
|
||||
先按配置创建 `logx.New(cfg.Log)`,再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`;后者阻塞,生命周期由应用负责。只有 `log.level` 允许变化,校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
|
||||
## 熔断与观测
|
||||
|
||||
为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`;通用调用用 `Do`,HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取,必须关闭 body;网络错误/5xx 计失败,冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
|
||||
|
||||
`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`;可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter,基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站,`telemetry.Transport` 采集出站并传播 W3C trace context;Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`,SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
|
||||
|
||||
创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类;自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定,避免用用户 ID 或对象 key。
|
||||
|
||||
## 分布式限流与幂等
|
||||
|
||||
`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶;满额/存储失败返回 503,额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`;Scope 必须来自认证后的租户/用户,先认证再幂等。不同请求内容复用同一身份返回 409,已完成请求回放原响应和 requestId。
|
||||
|
||||
claim 在开始业务前持久保存,完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期;返回 409 等待核查,`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理,TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace,数据丢失时不存在框架层面的重复执行保证。
|
||||
|
||||
## 迁移、审计与分页
|
||||
|
||||
迁移命名:`0001_createUsers.up.sql`,可附 `.down.sql`,但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单,不连接数据库;部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁;MySQL DDL 可能部分提交,失败保留 dirty,禁止自动 force 或回滚。当前基于 golang-migrate,数据库版本表不保存每个历史文件校验和,部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止,重试需新建 Runner。
|
||||
|
||||
审计:`audit.New(sink, "approvedField")` 建白名单,再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`;补入服务端时间、requestId/traceId,只保留明确允许的属性,Sink 失败原样返回。JSONSink 不保证与业务提交原子一致,需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
|
||||
|
||||
分页:`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20,上限 100,最大 offset=1,000,000;拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages,空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器,各限 2 CPU、2 GiB、256 pids;数据阶段无外网、无发布端口,不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准,未提供云凭据的 OSS/R2 不标为实网通过。
|
||||
|
||||
本批已通过上述 Linux 验证,源代码提交、98 个文件摘要、服务版本和边界见 [验收记录](performance/advancedAcceptance.md)。
|
||||
43
CHANGELOG.md
43
CHANGELOG.md
@@ -1,10 +1,51 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||
- 新增 `TestTLSMixedSoak` 与 `verifyHTTP2.py --scenario mixed`:8 条 TLS 连接、128 个 worker 持续混跑普通请求、慢上传和慢下载,校验 TLS/HTTP2、requestId、响应内容及传输摘要;分别统计三类请求与每条连接覆盖情况,并记录退出/GC 后的回收数据。
|
||||
- 完成 Linux amd64 30 分钟多连接慢客户端混合长测:53,191,687 次请求(含 106,971 次慢上传、5,588 次慢下载),0 失败与协议错误;采样 RSS 峰值约 116.6 MiB,连接/请求、goroutine 和 FD 回收,释放测试数据后存活堆约 450 KiB。保留完整日志、120 份采样及每条连接的分类计数。
|
||||
- 新增独立 `transport/http2` 实验 Runner:支持 h2c/TLS h2、连接上限、全局请求并发闸门、GOAWAY 排空、RST_STREAM 单流取消和生命周期统计;不改变 native HTTP/1.1 默认路径。
|
||||
- HTTP/2 实验 Runner 增加标准 1 MiB 头部上限、默认 8 MiB 请求体上限、128 路同连接多路复用、2 MiB 慢读取流控验收、协议错误、RST_STREAM 和 GOAWAY 帧计数;已声明超限请求在进入业务 handler 前返回 413。
|
||||
- 新增可配置时长的 HTTP/2 TLS soak:默认 3 秒,Linux 验收可通过 `GO_OMNI_HTTP2_SOAK_DURATION=30m` 覆盖,混合验证 128 路 stream、慢 handler、请求体、Keep-Alive 和退出回收。
|
||||
- HTTP/2 soak 每 15 秒输出请求/失败数、Go 堆、Linux RSS、goroutine、FD 和协议统计;校验响应协议、内容和独立请求 ID,不再忽略压测期限附近已发出请求的失败。
|
||||
- 补充 HTTP/2 慢响应流窗口为零时的隔离与恢复验收:同连接 204、另一连接带 body 响应、WINDOW_UPDATE 恢复发送及 RST_STREAM 释放请求槽位;补充 TLS 握手未完成时的连接上限拒绝、握手超时回收和正常请求恢复。
|
||||
- 补充连接级接收窗口耗尽测试:保持 stream 信用充足,消费完 65,535 字节连接窗口后确认大响应暂停,同连接 204 和另一条连接仍可完成,连接级 WINDOW_UPDATE 恢复后继续复用;补充零窗口阻塞写在关闭超时后解除阻塞、取消请求与连接回收的验收。
|
||||
- 新增 `scripts/verifyHTTP2.py`,固定在 Go 1.25 Linux 容器中运行 HTTP/2 TLS 长测;默认使用普通模式,`--race` 默认运行 60 秒。容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,带总时限、中断清理和日志大小限制。
|
||||
- 完成 Linux amd64 单 TLS 连接、128 个 worker 的 30 分钟 HTTP/2 长测:58,102,476 次请求,0 失败与协议错误,采样 RSS 峰值约 22.6 MiB(包含客户端和服务端),退出后连接/请求/FD 回收;保留原始日志和 120 份采样。
|
||||
- native TLS 明确限制 ALPN 为 `http/1.1`:缺省自动宣告该协议,误配置 `h2` 等未实现协议时在启动阶段报错。
|
||||
- native `Server.Stats()` 新增连接和请求过载拒绝计数;新增可信代理配置,访问日志只在 socket 对端属于配置网段时解析 `X-Forwarded-For`。
|
||||
- 新增 transport-neutral `metrics` 包,支持请求数、错误数、in-flight、累计耗时和状态分布,并可直接注册 `/metrics` 接口。
|
||||
- native 新增 `Server.Stats()`,提供 accepted connections、active connections 和 active requests 生命周期指标。
|
||||
- native 请求体改为流式读取,Content-Length 与 chunked body 不再在进入 Handler 前整体复制到内存;保留大小限制、读超时、trailer 校验和 Keep-Alive 排空。
|
||||
- 统一标准库与 native 的真实 TCP benchmark,增加固定并发档位、响应状态与读写错误检查。
|
||||
- 新增独立进程并发与长连接内存测量工具,记录成功吞吐、P95/P99、服务端分配、空闲连接堆内存和回收情况;修正并行 `ns/op` 与客户端/服务端混合分配的解释。
|
||||
- 加固 native HTTP/1.1 解析边界:校验 Host、重复 Content-Length、Connection token、Transfer-Encoding、控制字符和响应头名称;补充 HTTP/1.0 Keep-Alive、HEAD、204 以及异常请求测试。
|
||||
- native 支持带 trailers 的 chunked 请求体,并继续通过 `MaxBodyBytes`、读超时和头部上限控制解析内存与连接占用。
|
||||
- native 响应实现 `http.Flusher`:HTTP/1.1 流式响应自动使用 chunked framing,普通响应继续自动生成 `Content-Length`。
|
||||
- native 在长 Handler 运行期间监测客户端断开并取消请求 context,流式写失败同步取消;短请求延迟启动监测,管线数据保留给下一次请求。补充断开、监测退出和长连接复用验收。
|
||||
- native 增加 `ServeTLS` / `RunTLS` 和 `TLSConfig` 启动入口,App.RunWith 可以直接启用 TLS;补充真实 TLS 客户端握手、请求和优雅退出验收。
|
||||
- native 请求补齐 `Request.TLS` 连接状态;Linux/arm64 容器通过全量 race、vet、build,以及 TLS 并发、Keep-Alive、慢握手和连接上限回收验收。
|
||||
- 补充慢请求体超时、大请求体和同连接管线请求验收,确认流式 body 读取超时会取消请求并关闭连接,连续请求不会串线。
|
||||
- 补充 `MaxConnections` 满载拒绝与槽位释放验收,确认请求完成后连接容量可再次使用。
|
||||
- 补充 `MaxConcurrentRequests` 过载恢复验收,确认 503 拒绝后请求槽位释放并可继续处理新请求。
|
||||
- 增加高频建连/断连的状态池 churn 验收,覆盖连接对象回收与容量复用。
|
||||
- 新增 Linux Nginx 反向代理验收脚本,覆盖 native TLS、Keep-Alive、上游超时和优雅关闭。
|
||||
|
||||
## v0.6.0-preview
|
||||
|
||||
- native 网络层完成流式 Content-Length / chunked body、慢请求头、慢 body、慢 TLS 握手和 TLS 请求元数据验收;Handler 可在 body 未完全到达时开始读取,超时、畸形和超限 body 会被拒绝并关闭连接。
|
||||
- 将 native 连接 churn race 验收扩展为 16 个 worker、3,200 次建连/断连,并覆盖明文与 TLS 的连接上限、Keep-Alive 和槽位回收。
|
||||
- perfcheck 支持最长 30 分钟持续场景,新增 Linux 文件描述符采样;固定记录服务端堆、goroutine、连接数、accept 数和 FD 回收状态。
|
||||
- 新增 `scripts/verifyLinux.py`,在 `golang:1.25` Linux 容器内统一执行 race、vet、build 和可选 TLS / 非 TLS 压测。
|
||||
- 完成 Linux/arm64 30 秒高并发回归:128 并发流量、512 条空闲连接,TLS 与非 TLS 均为 0 错误,连接、goroutine 和 FD 回收正常;原始报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`。
|
||||
|
||||
## v0.5.0
|
||||
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端,接管 TCP 监听、连接、Keep-Alive、请求解析、响应写回和优雅关闭。
|
||||
- 新增 `App.RunWith` / `omni.Runner` 启动边界;native 通过标准 `http.Handler` 接入,现有 Handler、Service 和 Repository 无需修改。
|
||||
- native 支持 `Content-Length` 请求体、请求头 / 请求体限制、读写超时和服务关闭时的请求取消;通过对象池和 `MaxConnections` / `MaxConcurrentRequests` 控制高并发内存;chunked 请求体暂返回 501。
|
||||
- native 支持 `Content-Length` / chunked 请求体、请求头 / 请求体限制、读写超时和服务关闭时的请求取消;通过对象池和 `MaxConnections` / `MaxConcurrentRequests` 控制高并发内存。
|
||||
- `omni.New()` 默认仍使用只依赖标准库的 `transport/nethttp` 路由,native 作为显式启用的网络层预览实现。
|
||||
|
||||
## v0.4.0
|
||||
|
||||
172
MODULES.md
Normal file
172
MODULES.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# 业务基础模块
|
||||
|
||||
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||
|
||||
这批能力位于 main,尚未包含在 `v0.6.0-preview` 标签中。统一示例见
|
||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||
|
||||
## 缓存:cache 与 cache/redis
|
||||
|
||||
```go
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 4096, MaxBytes: 32 << 20})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 128, Timeout: 5 * time.Second})
|
||||
if err != nil { return err }
|
||||
value, err := loader.GetOrLoad(ctx, "tenant:42:product:7", time.Minute,
|
||||
func(ctx context.Context) ([]byte, error) { return repository.LoadProduct(ctx, "42", "7") })
|
||||
```
|
||||
|
||||
- `Store` 统一 `Get/Set/Delete`,返回字节切片;业务可用 JSON 编解码。
|
||||
- 内存缓存使用 LRU,同时限制条目数和 key/value 总字节数;字节上限不包含 Go 对象开销。
|
||||
读写复制切片,过期时懒清理,没有每个条目一个 goroutine。
|
||||
- TTL 为零表示不过期,负值拒绝;缺失返回 `cache.ErrMiss`。删除不存在的 key 成功。
|
||||
- Loader 合并**同进程、同 key**的并发回源,最多同时运行 `MaxLoads` 个不同 key 的加载。
|
||||
单个调用者取消只停止等待;共享加载有独立超时,加载函数必须响应 context。
|
||||
超限返回 `ErrBusy`,缓存故障直接返回错误,不自动当成 miss 冲击数据库。
|
||||
- 同 key 的调用必须具有相同数据含义、权限范围和 TTL;加载采用首个调用者的 context 值。
|
||||
多租户 key 必须含租户标识。进程间合并、分布式锁、缓存更新与数据库事务一致性由业务另外设计。
|
||||
|
||||
Redis 适配器借用调用方创建的 go-redis 客户端和连接池:
|
||||
|
||||
```go
|
||||
client := goredis.NewClient(&goredis.Options{
|
||||
Addr: "127.0.0.1:6379", PoolSize: 16, MaxRetries: -1,
|
||||
DialTimeout: time.Second, ReadTimeout: time.Second,
|
||||
WriteTimeout: time.Second, PoolTimeout: time.Second,
|
||||
ContextTimeoutEnabled: true,
|
||||
})
|
||||
// 启动时先使用有截止时间的 context 执行 client.Ping(ctx)。
|
||||
store, err := rediscache.New(client, "billing:") // cache/redis 的导入别名
|
||||
if err != nil { _ = client.Close(); return err }
|
||||
application.OnShutdown(client.Close)
|
||||
// store 可直接传入 cache.NewLoader。
|
||||
```
|
||||
|
||||
连接失败与 Redis 错误保持原样;只有 Redis nil 映射为 `ErrMiss`。Redis TTL 精度为毫秒,
|
||||
正的亚毫秒 TTL 提升到 1ms。适配器不负责关闭借用的客户端,Redis 服务端容量策略由部署配置。
|
||||
连接池和超时配置参考 [go-redis 官方生产使用说明](https://redis.io/docs/latest/develop/clients/go/produsage/)。
|
||||
|
||||
## 出站 HTTP:httpc
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
|
||||
MaxResponseBytes: 1 << 20,
|
||||
})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(client.Close)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
|
||||
```
|
||||
|
||||
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机
|
||||
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
|
||||
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
|
||||
|
||||
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`;
|
||||
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
|
||||
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
|
||||
|
||||
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
|
||||
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
|
||||
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
|
||||
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
|
||||
重定向交给业务处理,3xx 不会自动跟随。
|
||||
|
||||
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时,
|
||||
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
|
||||
|
||||
## 请求保护:middleware
|
||||
|
||||
```go
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40})
|
||||
if err != nil { return err }
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{
|
||||
Scope: func(c *httpx.Context) (string, error) {
|
||||
principal, _ := auth.FromContext(c)
|
||||
return principal.UserID, nil // 多租户应用应组合租户和用户 ID
|
||||
},
|
||||
})
|
||||
if err != nil { return err }
|
||||
api := application.Group("/api", middleware.Timeout(3*time.Second),
|
||||
auth.RequireAuth(authenticator), limiter.Middleware())
|
||||
api.POST("/orders", createOrder, idempotency.Middleware())
|
||||
```
|
||||
|
||||
限流为进程内令牌桶,`Rate` 为每秒额度,`Burst` 为突发容量。默认按已解析的可信客户端 IP,
|
||||
可用 `Key` 按已认证身份分桶。额度不足返回 **429 + Retry-After**;key 数超过 `MaxKeys`
|
||||
返回 **503**。满容量时不会踢掉活跃桶让攻击者绕过限流;空闲桶经过完整补充周期后才允许回收。
|
||||
|
||||
Timeout 设置协作式 deadline,不额外启动 Handler goroutine。业务及下游调用必须响应 context;
|
||||
超时后尚未提交响应时返回 **504**,已经提交的响应保持原状态。它不能强制中断任意 Go 代码,
|
||||
也不能替代网络层读写超时。不要在业务代码中异步持有 `httpx.Context`。
|
||||
|
||||
幂等保护适用于小型、非流式写接口:
|
||||
|
||||
- 必须先认证/鉴权,再执行幂等中间件;`Scope` 必须取自服务端验证过的身份。
|
||||
GET/HEAD/OPTIONS 直接通过,写请求要求 `Idempotency-Key`,长度 1..128 个非空格 ASCII 字符。
|
||||
- 同 Scope + key + 方法/URL/Content-Type/body 重放首次响应;内容改变返回 **409**。
|
||||
正在执行或结果不可重放的相同请求也返回 **409**,不并发执行第二次。
|
||||
- 默认最多 32 个处理中请求、256 条记录、16 MiB 保存的响应头/body,单次请求体 1 MiB、
|
||||
响应 body 64 KiB。请求超限 **413**,容量满 **503**,响应超限 **500**。
|
||||
- panic、5xx、超时、响应超限、Set-Cookie、Cache-Control:no-store 或保存容量不足时保留失败记录,
|
||||
阻止在 TTL 内自动重执行业务。业务可能已经提交,调用方应查询实际结果后处理。
|
||||
- 完成记录默认保留 10 分钟;在途记录不会因 TTL 到期而允许重复执行。
|
||||
重放保留原响应 body、原 `requestId` 与原 `X-Request-ID`,增加 `Idempotency-Replayed:true`;
|
||||
本次访问日志仍使用本次请求 ID。
|
||||
- 记录仅在当前进程有效,重启、TTL 到期和多副本切换后均无法保证去重。支付、库存等关键写入
|
||||
应在数据库事务内配合唯一业务键。该实现不提供跨进程 exactly-once,也不支持 SSE/流式/升级连接。
|
||||
|
||||
## 后台执行:task
|
||||
|
||||
```go
|
||||
jobs, err := task.New(task.Config{Workers: 4, QueueSize: 128, TaskTimeout: 30*time.Second})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(jobs.Close) // 在注册数据库/缓存关闭之后注册,先排空任务
|
||||
future, err := jobs.Submit(serviceContext, "refreshCatalog", refreshCatalog)
|
||||
if err != nil { return err } // 满队列立即返回 ErrQueueFull
|
||||
err = future.Wait(ctx) // 可选;不等待时,任务失败仍通过 OnError/logx 报告
|
||||
err = jobs.Every("refreshCatalog", time.Minute, refreshCatalog)
|
||||
```
|
||||
|
||||
固定 worker 和有界队列控制资源。任务保留提交 context;需要独立于 HTTP 请求执行时使用服务
|
||||
context,或明确使用 `context.WithoutCancel` 保留日志关联值。任务仍受 Runner 的超时/退出控制。
|
||||
任务函数与 `OnError` 回调应及时返回,尤其要响应取消;panic 被转为失败,worker 继续工作。
|
||||
|
||||
`Every` 使用固定间隔,第一次在一个间隔后执行;同一计划上轮尚未完成、排队或队列满时跳过。
|
||||
默认最多 128 个计划,名称不可重复。`Stats` 提供 active/queued/completed/failed/skipped。
|
||||
它是进程内任务执行器,没有任务持久化、失败自动重试、cron 表达式、跨副本选主或崩溃恢复。
|
||||
|
||||
`Shutdown(ctx)` 停止接收和调度,排空已接受任务;截止时间到后取消任务并及时返回。
|
||||
忽略取消的任务不能被 Go 强杀,可能继续运行。`Close` 默认给排空 10 秒。
|
||||
|
||||
## 增加业务模块:go-omni add
|
||||
|
||||
```sh
|
||||
go build -o /tmp/go-omni ./cmd/go-omni
|
||||
/tmp/go-omni add orderItems --dir /path/to/billing
|
||||
```
|
||||
|
||||
生成 `internal/orderItems/{module.go,model,repository,service,handler,README.md}`,包含 name/createdAt
|
||||
示例模型、内存 CRUD、输入校验、错误映射及可注入 Repository。模块名使用 lowerCamelCase。
|
||||
|
||||
在现有启动代码中导入模块后调用 `orderItems.Register(application, repository, authMiddleware)`。
|
||||
脚手架不自动修改现有路由、main、go.mod 或数据库,已有目录不会覆盖。
|
||||
内存 Repository 仅为可运行示例:重启丢数据,列表未分页;正式业务应实现自己的持久化 Repository。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --goproxy https://goproxy.cn,direct
|
||||
```
|
||||
|
||||
Linux runner 顺序运行两个容器:全量 race/vet/build/独立生成项目,随后真实 Redis 适配验证。
|
||||
每次最多一个容器,限制 2 CPU、2 GiB(不额外使用 swap)、256 pids;Redis 测试容器禁外网,
|
||||
只用内部 loopback,不发布端口。Redis 集成二进制为静态非 race,适配器单测在全量 race 中覆盖。
|
||||
结束清理本轮容器,并记录提交、源文件摘要、镜像 ID、日志摘要和原有容器是否仍运行。
|
||||
|
||||
本轮通过结果和原始证据见 [业务模块验收](performance/modulesAcceptance.md)。
|
||||
@@ -1,20 +1,89 @@
|
||||
# GoOmni 性能基线
|
||||
|
||||
这份基线用于比较未来自研网络层,不代表生产容量承诺。测试只测进程内请求分发和 GoOmni 请求链,不包含真实 TCP、TLS、数据库或业务存储延迟。
|
||||
这轮测量显示 native 与标准库处于相同吞吐量级,在本机短时测试中,native 的空闲长连接堆内存更少。当前 native 仅实现 HTTP 的部分能力,尚不能据此判断生产容量或替代完整标准库服务端。
|
||||
|
||||
运行:
|
||||
## 运行方式
|
||||
|
||||
```sh
|
||||
# race、vet、build,以及进程内和真实 TCP benchmark
|
||||
python3 scripts/verifyPerformance.py
|
||||
|
||||
# 另加独立服务进程的并发、延迟和空闲长连接内存矩阵
|
||||
python3 scripts/verifyPerformance.py --load --output /tmp/goOmniPerf.json
|
||||
|
||||
# 只运行独立进程矩阵,默认每档 2 秒、3 轮
|
||||
go run ./internal/perfcheck --duration 2s --count 3 --output /tmp/goOmniPerf.json
|
||||
|
||||
# 在 Linux Docker 容器内固定运行 race、vet、build 和 TLS 压测
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30s \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS.json
|
||||
|
||||
# 发布候选验收可把持续流量延长到 30 分钟
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30m \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS-30m.json
|
||||
|
||||
# Linux 上用 Nginx 反向代理接入 native TLS 服务,检查 Keep-Alive、超时和优雅关闭
|
||||
python3 scripts/verifyProxy.py
|
||||
```
|
||||
|
||||
本次记录环境:Go 1.25.0、darwin/arm64、Apple M2 Max;每项连续运行 3 次:
|
||||
可通过 `--concurrency 1,32,128` / `--idle 128,1024` 调整矩阵;传空字符串可跳过对应场景。工具只监听本机随机端口,每个场景启动独立服务进程,并在结束后关闭客户端、检查连接排空、关闭服务。响应状态、完整响应体、requestId 或传输错误均计入失败,不能把快速返回的错误页算成成功吞吐。
|
||||
|
||||
```text
|
||||
BenchmarkTransportRouteDispatch/nethttp 568–671 ns/op 1344 B/op 13 allocs/op
|
||||
BenchmarkTransportApplicationRequest/nethttp 2017–2208 ns/op 3091 B/op 34 allocs/op
|
||||
```
|
||||
Linux 验收脚本使用 `golang:1.25` 容器,固定执行 `go test -race ./...`、`go vet ./...`、`go build ./...`,再按参数运行 perfcheck。perfcheck 在 Linux 读取 `/proc/self/fd`,报告连接建立期间的文件描述符峰值以及回收后的数量;同时记录服务端堆、goroutine、连接数和累计 accept 数。30 秒报告用于快速回归,30 分钟命令用于发布前持续压测。
|
||||
|
||||
`BenchmarkTransportRouteDispatch` 只覆盖标准库路由、路径参数和 `http.Handler` 调用;`BenchmarkTransportApplicationRequest` 还包含 GoOmni Context、RequestID、统一 JSON 响应和默认中间件链。
|
||||
## 2026-09-21 本机测量
|
||||
|
||||
比较新网络层时,应使用相同 Go 版本、机器和 benchmark 命令,并同时运行 `-race` 全量测试。性能下降需要结合分配次数、取消传播、错误响应和业务契约结果一起判断。
|
||||
环境:Go 1.25.0、darwin/arm64、Apple M2 Max、GOMAXPROCS=12。网络实现来自 `3d5f225` 工作树(native 与 v0.6.0-preview 相同);本次新增测量工具,不修改服务端实现。原始测量值见 [tcpBaseline.json](performance/tcpBaseline.json)。报告时间为 UTC,北京时间是 9 月 21 日。
|
||||
|
||||
每个并发档位持续约 2 秒、重复 3 轮,每轮交替标准库/native 的先后顺序。下表为各指标的三轮中位数;全部流量场景共完成 **2,660,828** 次请求,错误 **0**。另有 128 / 1,024 条空闲长连接的建立、保持和回收测试。
|
||||
|
||||
| 并发请求 | 标准库成功请求/秒 | native 成功请求/秒 | 标准库 P95 | native P95 | 标准库 P99 | native P99 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | 19,731 | 21,497 | 0.074 ms | 0.070 ms | 0.139 ms | 0.119 ms |
|
||||
| 32 | 90,099 | 88,812 | 1.016 ms | 0.983 ms | 1.507 ms | 1.507 ms |
|
||||
| 128 | 112,422 | 112,105 | 3.277 ms | 3.408 ms | 4.719 ms | 4.981 ms |
|
||||
|
||||
128 并发时,两者吞吐差约 0.3%,native 的尾延迟略高。基于这组结果,暂时没有理由仅为健康接口吞吐就改成手写事件循环。
|
||||
|
||||
| 空闲 Keep-Alive 连接 | 标准库新增堆内存 | native 新增堆内存 | 标准库约每连接 | native 约每连接 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 128 | 1.275 MiB | 0.658 MiB | 10,444 B | 5,389 B |
|
||||
| 1,024 | 9.871 MiB | 5.137 MiB | 10,108 B | 5,260 B |
|
||||
|
||||
1,024 条空闲连接时,native 的新增堆内存约少 48%。这是**服务端 Go 堆内存增量**,不包括 goroutine 栈、进程其他内存和内核 socket 缓冲区,不能解释为整个进程内存减半。同档服务端 `StackInuse` 分别约为标准库 8.75 MiB、native 8.69 MiB,原始报告单独记录。
|
||||
|
||||
连接断开后,两者连接数都回到 0,goroutine 分别回到 2 / 3 的初始基线。一次 GC 后仍有对象池缓存;第二次 GC 后,总堆内存中位数回到约 2.10 / 2.08 MiB(启动基线约 1.54 MiB)。堆内存不会立刻精确回到启动数值,不把缓存保留解释为泄漏,也不把这轮短测解释为长期无泄漏证明。
|
||||
|
||||
128 并发时,**服务端单独累计分配**约为标准库 4,454 B/请求、47 次/请求,native 4,297 B/请求、55 次/请求。native 的分配字节较少、次数较多;具体来源需用分配 profile 确认,不能仅凭总量归因于 HTTP 桥接或 JSON。
|
||||
|
||||
## 统计边界
|
||||
|
||||
- 同一台机器上,压测客户端和服务端是两个独立进程。两者使用同一健康接口、相同小型 JSON 响应、固定 `requestId=perfCheck` 和 Keep-Alive;不启用访问日志、TLS 或数据库。
|
||||
- 并发模型为闭环:每个 worker 等上一个请求完成后发下一个。实际经过时间包含最后请求排空;吞吐只计算成功响应。不能据此推断开放到达流量下的排队或过载表现。
|
||||
- 延迟从客户端发起请求到完整响应校验完成,包含失败请求。直方图使用每个 2 的幂区间内 16 档,报告桶上界,舍入误差至多约 6.25%。小幅差别不应过度解释。
|
||||
- 服务端 `runtime.MemStats` 通过独立 stdin/stdout 控制通道读取,客户端内存不计入。累计分配差值包含少量采样开销。堆峰值每 200 ms 采样,可能错过瞬时峰值;这些指标不是进程 RSS。
|
||||
- Linux 文件描述符来自服务端子进程的 `/proc/self/fd`,包含标准输入输出、监听 socket 和连接;macOS 等没有该路径的平台会省略该字段。回收后的数量应回到启动附近,不能代替内核 socket 缓冲或 RSS 测量。
|
||||
- 空闲场景在所有连接完成健康请求后做 GC 再采样。新增堆内存包括连接、缓冲区及分摊的运行时开销。`after` 为客户端断开并排空后一次 GC,`afterReclaim` 为第二次 GC,用于观察对象池缓存释放。
|
||||
- native 配置上限为 8,192 连接 / 4,096 活跃请求;本轮并发低于上限,未验证过载保护容量。标准库没有同样的并发闸门;比较包括各自真实实现的开销。
|
||||
- 可用 `python3 scripts/verifyPerformance.py --load --tls --duration 2s` 运行 TLS 版本的独立进程矩阵。该模式在每个子进程生成短期自签名证书,客户端仅用于本机基准验证;本报告的历史表格仍是无 TLS 数据。
|
||||
|
||||
## benchmark 口径修正
|
||||
|
||||
## TLS 验收记录
|
||||
|
||||
macOS arm64 本机的 5 秒 TLS 矩阵见 [tlsBaseline.json](performance/tlsBaseline.json):1 / 32 / 128 并发和 128 / 1,024 条空闲连接均为 0 错误;native 在 128 并发约 114,232 req/s,1,024 条空闲 TLS 连接的服务端堆增量约 8.94 MiB,连接关闭后 goroutine 回到基线。
|
||||
|
||||
Linux arm64 容器使用 Go 1.25.14、GOMAXPROCS=4 运行 3 秒矩阵,见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json):1 / 32 / 128 并发和 128 / 512 条空闲连接均为 0 错误;native 在 128 并发约 99,876 req/s,512 条空闲 TLS 连接的服务端堆增量约 4.47 MiB,连接关闭后 goroutine 回到基线。容器验证同时通过全量 race、vet、build。
|
||||
|
||||
Linux arm64 的 30 秒高并发回归见 [linuxSoak.json](performance/linuxSoak.json) 和 [linuxSoakTLS.json](performance/linuxSoakTLS.json):两份报告均使用 128 并发流量和 512 条空闲连接,所有场景错误为 0;native 非 TLS 约 146,614 req/s、TLS 约 149,720 req/s,空闲连接堆增量分别约 2.57 MiB 和 4.47 MiB,FD 均由 7 增至 519 并回到 7,goroutine 均回到启动附近。
|
||||
|
||||
目标 Linux amd64 服务器的 30 分钟 TLS 验收使用 1 个 CPU、1 GiB 内存的隔离 Docker 容器完成:net/http 约 11,865 req/s,native 约 13,098 req/s,两个流量场景均为 0 错误;native 的 P95/P99 为 44.040/52.429 ms。512 条空闲 TLS 连接时,native 服务端堆增量约 4.50 MiB(约 9,215 B/连接),文件描述符由 7 增至 519 后回到 7,goroutine 和连接均正常回收。原始报告保存在服务器的 `performance/linuxServerSoakTLS.json`。
|
||||
|
||||
`BenchmarkTransportTCPApplicationRequest/{nethttp,native}/concurrency{1,32,128}` 使用同一测试入口、客户端、应用与响应检查。原先独立的 `BenchmarkNativeApplicationRequest` 已合并,避免两个入口配置漂移。`verifyPerformance.py` 会一并运行。
|
||||
|
||||
并行 benchmark 的 **`ns/op` 是总经过时间除以请求数,反映吞吐成本,不是单请求延迟**。`B/op` / `allocs/op` 统计同一进程中的客户端与服务端总分配。此前报告的 native 15.4–16.7 µs/op、约 7.8 KB/op、100 allocs/op 应按这个含义理解,不能作为服务端延迟或独立内存指标。
|
||||
|
||||
`BenchmarkTransportRouteDispatch` 和 `BenchmarkTransportApplicationRequest` 继续保留,分别测进程内路由分发和完整框架处理链;不包含 TCP,不能直接与上述网络延迟比较。
|
||||
|
||||
后续通过 30 分钟命令完成发布前持续压测,并在目标反向代理和部署环境继续观察内核 socket 缓冲、FD 上限与真实业务流量;再通过 profile 决定是否优化对象分配或事件状态机。
|
||||
|
||||
92
README.md
92
README.md
@@ -5,7 +5,7 @@ GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用
|
||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.5.0
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
||||
```
|
||||
|
||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||
@@ -13,7 +13,7 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.5.0
|
||||
创建新业务项目:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.5.0
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -38,16 +38,35 @@ go-omni/
|
||||
├── health/ # 存活和就绪检查
|
||||
├── middleware/ # 请求 ID 等基础中间件
|
||||
├── logx/ # 业务层结构化日志
|
||||
├── metrics/ # 请求指标与状态统计
|
||||
├── config/ # 配置管理
|
||||
├── validate/ # 参数校验
|
||||
├── auth/ # 认证和权限接口
|
||||
├── database/ # database/sql 生命周期和事务
|
||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
||||
├── task/ # 有界后台任务与间隔调度
|
||||
├── queue/ # 持久队列接口与 Redis Streams 适配
|
||||
├── storage/ # Local、S3/R2、OSS 与临时下载链接
|
||||
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
|
||||
├── telemetry/ # 可注入 OpenTelemetry 观测
|
||||
├── resilience/ # 熔断与上游并发隔离
|
||||
├── audit/ # 显式业务审计与可注入 Sink
|
||||
├── pagination/ # 有界分页和统一结果
|
||||
├── transport/nethttp/ # 标准库网络适配器
|
||||
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
||||
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
||||
├── examples/ # 示例项目
|
||||
└── cmd/ # go-omni 项目脚手架
|
||||
```
|
||||
|
||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||
可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
|
||||
|
||||
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```go
|
||||
@@ -121,7 +140,13 @@ if err := app.RunWith(ctx, native.New(nil), ":8080"); err != nil {
|
||||
}
|
||||
```
|
||||
|
||||
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 请求体、读写超时和优雅关闭;chunked 请求体暂返回 501。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
|
||||
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 和 chunked 请求体、`http.Flusher` 流式响应、可选 TLS、读写超时和优雅关闭;不支持的传输编码会被拒绝。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
|
||||
|
||||
native TLS 当前只支持并宣告 ALPN `http/1.1`;配置 `h2` 会在启动时快速失败。需要 HTTP/2 时,可由 Nginx/Cloudflare 终止后回源 HTTP/1.1,或使用默认的 `transport/nethttp` 适配器。
|
||||
|
||||
`transport/http2` 是独立的 opt-in 实验适配器,使用 `x/net/http2` 处理帧和流控,同时保留 GoOmni 的连接上限、全局请求并发闸门和生命周期边界。它不会改变 native 默认路径;实验入口、限制和验收结果见 [HTTP/2 进入条件](transport/native/HTTP2.md)。
|
||||
|
||||
自研网络层还提供 `server.Stats()`,可读取 accepted、rejected、active connections 和 active requests,与上面的业务请求 metrics 组合后能区分连接堆积、过载拒绝和 Handler 慢请求。`rejectedConnections` 表示连接闸门拒绝数,`rejectedRequests` 表示并发请求闸门返回 503 的次数。
|
||||
|
||||
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖具体网络库:
|
||||
|
||||
@@ -132,6 +157,16 @@ func (s *UserService) Create(ctx *httpx.Context) error {
|
||||
}
|
||||
```
|
||||
|
||||
需要基础运行指标时,可以注册 transport-neutral metrics middleware 和接口:
|
||||
|
||||
```go
|
||||
requestMetrics := omni.NewMetrics()
|
||||
app.Use(requestMetrics.Middleware())
|
||||
app.GET("/metrics", requestMetrics.Handler())
|
||||
```
|
||||
|
||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
||||
|
||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||
@@ -151,9 +186,40 @@ application.OnShutdown(logger.Close)
|
||||
|
||||
常用变量包括 `GO_OMNI_ENABLE_LOGGER`、`GO_OMNI_LOG_LEVEL`、
|
||||
`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_OUTPUT`、`GO_OMNI_SERVICE`、
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR` 和 `GO_OMNI_SHUTDOWN_TIMEOUT`。数据库连接仍由
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR`、`GO_OMNI_SHUTDOWN_TIMEOUT` 和
|
||||
`GO_OMNI_TRUSTED_PROXIES`。可信代理变量使用逗号分隔的 IP/CIDR,例如
|
||||
`GO_OMNI_TRUSTED_PROXIES=127.0.0.1,10.0.0.0/8`;只有来自这些地址的连接才会解析
|
||||
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
|
||||
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
|
||||
|
||||
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
|
||||
|
||||
| 配置 | 默认值 / 用途 |
|
||||
| --- | --- |
|
||||
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
|
||||
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
|
||||
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
|
||||
|
||||
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_TRANSPORT=native go run ./examples/reference
|
||||
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
|
||||
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
|
||||
go run ./examples/reference
|
||||
```
|
||||
|
||||
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
|
||||
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
|
||||
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
|
||||
|
||||
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
|
||||
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
|
||||
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
|
||||
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
|
||||
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
|
||||
|
||||
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
|
||||
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
|
||||
|
||||
@@ -200,12 +266,24 @@ userService := service.NewUsers(services)
|
||||
从远程版本生成并升级一个独立业务模块,可运行:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyBusiness.py --version v0.5.0
|
||||
python3 scripts/verifyBusiness.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.5.0`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.6.0-preview`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
|
||||
标准库路由和完整 GoOmni 请求链的可重复性能基线见 [PERFORMANCE.md](PERFORMANCE.md),运行 `python3 scripts/verifyPerformance.py` 可重新采样。
|
||||
标准库与 native 的并发、延迟和长连接内存对照见 [PERFORMANCE.md](PERFORMANCE.md)。`python3 scripts/verifyPerformance.py` 运行检查与 benchmark;增加 `--load --output /tmp/goOmniPerf.json` 可测量独立服务进程并保存报告。
|
||||
|
||||
HTTP/2 仍是显式 opt-in 实验适配器,不会改变 native 的 HTTP/1.1 默认路径。Linux TLS 长测使用 `python3 scripts/verifyHTTP2.py --duration 30m`,默认使用普通 Go 运行模式;`--race` 默认改为 60 秒,用于并发检查,其额外内存开销不能当成生产运行基线。
|
||||
|
||||
多连接混合场景使用 `python3 scripts/verifyHTTP2.py --scenario mixed --duration 30m`:8 条独立 TLS 连接,每条连接 14 个普通请求 worker、1 个慢上传 worker、1 个慢下载 worker,共 128 个 worker。上传为 256 KiB、每 4 KiB 等待 2 ms;下载为 8 MiB、每次最多读取 64 KiB 后等待 20 ms。日志分别记录三类成功数、传输字节数、每条连接的覆盖情况,以及停止并释放测试数据后两次 GC 的采样。
|
||||
|
||||
该脚本默认把测试容器限制为 2 核、2 GiB 内存(禁用 swap)、256 个进程/线程,整个命令最多运行 40 分钟,中断和超时会清理本次容器。这些限制可用 `--cpus`、`--memory-mb`、`--pids-limit`、`--wall-timeout` 调整;共享服务器应先用 `--duration 60s` 短测,再逐步延长。每 15 秒输出的堆、RSS 和 FD 统计包含同一进程内的客户端与服务端。
|
||||
|
||||
Linux amd64 已完成 30 分钟、单 TLS 连接、128 个 worker 的普通模式长测:58,102,476 次请求,0 失败、0 协议错误,采样 RSS 峰值约 22.6 MiB,连接/活动请求归零且 FD 回到基线。完整采样和原始日志见 [HTTP/2 长测报告](performance/http2SoakLinux.json) 与 [原始日志](performance/http2SoakLinux.txt),此前的 race 对照见 [短测记录](performance/http2SmokeLinux.json)。这项结果覆盖单连接复用。
|
||||
|
||||
多连接慢客户端场景也已完成 Linux amd64 的 30 分钟长测:8 条 TLS 连接、128 个 worker,53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,共 53,191,687 次,0 失败与协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端、服务端和下载缓冲);退出后连接/请求归零,goroutine 回到 2、FD 回到 6;释放测试数据并两次 GC 后存活堆约 450 KiB,RSS 不要求立即归还操作系统。完整 120 份采样见 [混合长测报告](performance/http2MixedSoakLinux.json) 和 [原始日志](performance/http2MixedSoakLinux.txt)。连接级窗口耗尽、WINDOW_UPDATE 恢复及零窗口写阻塞的关闭回收也已有独立测试;Nginx 的 HTTP/2 回源验收结果见下方,适配器保持实验状态。
|
||||
|
||||
`python3 scripts/verifyHTTP2Proxy.py --count 3` 验证真实 Nginx 双段 TLS/h2、回源连接复用、证书拒绝、超时/取消恢复与新旧实例的 GOAWAY 排空;增加 `--race` 可运行并发检查。Linux amd64、Nginx 1.30.5 的普通与 race 模式各 3 轮均已通过。编译和测试容器顺序运行,均限制为 2 核、2 GiB;测试容器禁用外部网络,全部请求在容器内完成。使用方法、报告及当前边界见 [HTTP/2 代理回源验收](transport/http2/PROXY.md)。
|
||||
|
||||
在框架根目录运行:
|
||||
|
||||
|
||||
55
ROADMAP.md
55
ROADMAP.md
@@ -35,12 +35,57 @@
|
||||
- 业务试用没有要求修改 Handler / Service;现有公共 API 足以支撑一套分层业务模块。
|
||||
- 自动验收入口:`python3 scripts/verifyBusiness.py --version v0.4.0`。
|
||||
|
||||
## 5. 自研网络层、稳定性与性能基线(进行中,v0.5.0 预览)
|
||||
## 5. 自研网络层、稳定性与性能基线(v0.6.0-preview 验收完成)
|
||||
|
||||
- 已加入标准库路由和完整 GoOmni 请求链 benchmark,输出分配次数和耗时,作为未来自研网络层的对照基线。
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端:自行管理 TCP listener、Keep-Alive、Content-Length 请求解析、响应写回、读写超时和优雅关闭;通过 `App.RunWith` 接入,业务 Handler / Service 保持不变。
|
||||
- native 当前拒绝 chunked 请求体,仍使用标准 `http.Request` / `http.ResponseWriter` 作为桥接;解析器、连接抽象和生产压测将在后续阶段继续下沉。
|
||||
- 已具备全量 race、vet、build 验收;生产压测、长连接规模和真实部署验证仍待完成。
|
||||
- 自动验收入口:`python3 scripts/verifyPerformance.py`。
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端:自行管理 TCP listener、Keep-Alive、Content-Length / chunked 请求解析、响应写回、读写超时和优雅关闭;通过 `App.RunWith` 接入,业务 Handler / Service 保持不变。
|
||||
- 标准库与 native 共用 TCP benchmark,固定 1 / 32 / 128 并发;相同应用、客户端、超时和响应检查,避免两个测试入口逐渐失去可比性。
|
||||
- 新增独立服务进程压测 `internal/perfcheck`,记录成功吞吐、P50/P95/P99、服务端分配、采样堆峰值,以及 128 / 1,024 条空闲长连接的堆内存、goroutine 和断开后的回收情况。原始报告保存在 `performance/tcpBaseline.json`。
|
||||
- native 已支持受 `MaxBodyBytes` 约束的 `Content-Length` / chunked 请求体,仍使用标准 `http.Request` / `http.ResponseWriter` 作为桥接;解析器、连接抽象和生产压测将在后续阶段继续下沉。
|
||||
- native 请求体已改为流式读取,Handler 可边到边处理,服务端只在 Handler 返回后排空剩余 body,超时、畸形和超限 body 会取消请求并关闭连接。
|
||||
- native 响应已支持标准 `http.Flusher`;HTTP/1.1 流式响应使用 chunked framing,普通响应继续走 `Content-Length`,HTTP/1.0 回退到关闭连接分帧。
|
||||
- native 长请求已补客户端断开及流式写失败取消,监测退出后再复用 Reader;已验收普通/流式断开、慢 Handler 后连续 Keep-Alive 和管线数据不被误消费。短请求延迟启动监测,已有管线数据时取消依赖后续写失败或 Handler 返回。
|
||||
- 已补充慢请求头、慢 Content-Length body、慢 chunked body、慢 TLS 握手、大请求体、HTTP/1.0 Keep-Alive、HEAD/204 和管线请求验收;Linux/arm64 容器已通过全量 race、vet、build,以及 3 秒 TLS 并发/空闲连接矩阵。原始报告见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json)。
|
||||
- 已验证 `MaxConnections` 满载时快速拒绝新连接、请求完成后释放容量;后续仍需在目标部署环境观察文件描述符、内核 socket 缓冲和反向代理联动。
|
||||
- 已验证 `MaxConcurrentRequests` 满载返回 503,活动请求结束后槽位释放,后续请求可以恢复处理。
|
||||
- native `Stats()` 已记录连接和请求过载拒绝数;访问日志已支持可信代理网段配置,默认拒绝伪造的 `X-Forwarded-For`。
|
||||
- native TLS 已明确只宣告 HTTP/1.1 ALPN;HTTP/2 暂由边缘代理终止或使用 `transport/nethttp`,已整理独立协议设计、流控和压测门槛,见 [native HTTP/2 进入条件](transport/native/HTTP2.md),完成后再接入 native。
|
||||
- 已加入独立 `transport/http2` 实验 Runner,验证 h2c/TLS h2、连接上限、全局请求并发拒绝和恢复,以及 GOAWAY 排空、RST_STREAM 单流取消和服务关闭超时;它使用 `x/net/http2` 作为临时帧/流控实现,已完成下述 Nginx HTTP/2 回源联调,尚未接入 native 默认路径。
|
||||
- HTTP/2 实验已补齐头部/请求体上限、128 路同连接多路复用、慢读取流控、协议错误、RST_STREAM 与 GOAWAY 统计,并提供 `scripts/verifyHTTP2.py --duration 30m` 的 Linux TLS 长测入口;长测默认不启用 race,短时并发检查使用 `--race`(默认 60 秒)。测试容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,并设总时限和退出清理;受限长测和 Nginx 回源联调结果见下方。
|
||||
- 新 Linux amd64 环境在上述硬上限内完成 HTTP/2 普通模式 2 分钟与 race 模式 60 秒短测,分别为 3,885,273 和 377,788 次请求,均为 0 失败;连接/请求归零、FD 回到基线。SIGTERM、SIGHUP 和总时限退出清理已实测。报告见 [HTTP/2 短测记录](performance/http2SmokeLinux.json);此前无资源上限的 race 长测因 RSS 增长而停止,不作为通过依据。
|
||||
- 已完成受限的 HTTP/2 单连接 TLS 30 分钟长测:128 个 worker、58,102,476 次请求、0 失败及协议错误;包含客户端与服务端的采样 RSS 峰值约 22.6 MiB,连接/活动请求归零,FD 由 9 回到 6。120 份定时采样见 [HTTP/2 长测报告](performance/http2SoakLinux.json),测试提交和原始日志校验值保存在报告中。Nginx HTTP/2 回源的 GOAWAY/排空结果见下方。
|
||||
- 已完成 8 条 TLS 连接、128 个 worker 的 30 分钟混合长测:53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,0 失败及协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端和服务端),前段与最后 5 分钟 RSS 中位数分别约 111.3/110.4 MiB;退出后活动连接/请求归零,goroutine 2、FD 6 均回到基线,两次 GC 后存活堆约 450 KiB。原始日志、120 份采样、测试提交和校验值见 [混合长测报告](performance/http2MixedSoakLinux.json)。
|
||||
- 已补连接级窗口耗尽时的暂停、同连接 204/跨连接隔离、WINDOW_UPDATE 恢复和连接复用;已补零窗口写阻塞在关闭超时后的解除阻塞、请求取消与回收。Linux amd64 的 HTTP/2 全包 race、vet、build 已通过,见 [边界回归报告](performance/http2FlowControlLinux.json)。
|
||||
- 新增 HTTP/2 零 stream 窗口时的慢响应隔离、WINDOW_UPDATE 恢复、RST_STREAM 取消与请求槽位回收测试;补充 TLS 半开握手占满连接上限时的及时拒绝、超时回收及正常请求恢复测试。
|
||||
- 已完成 Nginx 1.30.5 → HTTP/2 的 TLS 回源联调:普通与 race 模式各 3 轮,共 18 个子场景通过;覆盖两段证书校验、32 次 POST 复用一条回源连接、不可信 CA/主机名拒绝、504 超时、客户端取消、GOAWAY 后旧 POST 完成及 20 个新 POST 转入新实例,无重复业务执行。见 [验收说明与报告](transport/http2/PROXY.md)。
|
||||
- `scripts/verifyHTTP2Proxy.py` 顺序运行受限编译容器和禁用外部网络的测试容器;总时限、SIGTERM、SIGHUP 清理已在 Linux 实测,原有 7 个业务容器仍运行。当前结果不覆盖 Nginx master/worker 重载或 Cloudflare 实网部署。
|
||||
- 已接入 `omni.New` 的统一 transport / TLS / shutdownTimeout 配置:默认 nethttp,可显式选择 native 或实验 HTTP/2;证书在监听前加载,HTTP/2 配置必须启用 TLS。`RunWith` 继续支持自定义网络层,业务不依赖具体适配器。
|
||||
- 参考业务已增加五种真实 TCP/TLS 组合的 CRUD、认证权限、错误响应、requestId 日志与协议协商测试;三种适配器共用排空和超时退出测试。修复 native 退出时过早取消在途请求,并使关闭超时错误返回调用方。下一步仍为业务环境试用、native 多协议监听器设计与接入回归,HTTP/2 保持实验状态。
|
||||
- 统一启动已通过 Linux 全仓库 race/vet/build 及关键场景 10 次重复验证;修复了重复测试发现的 native 活动连接转为空闲时错过关闭的竞态,并保证不响应取消的 Handler 不阻塞关闭超时返回。脚手架独立项目的 native SIGINT/SIGTERM 验证通过;失败复现与最终验收证据见 [统一启动验收](performance/transportStartup.md)。
|
||||
- 已用 16 个并发 worker 完成 3,200 次建连/断连 race 验收;MaxConnections 满载拒绝、槽位释放和 TLS 连接上限均已覆盖。
|
||||
- 新增 `scripts/verifyLinux.py`,固定在 Go 1.25 Linux 容器内运行 race、vet、build 和可选 TLS/非 TLS perfcheck;perfcheck 记录堆、goroutine、连接和 Linux 文件描述符。
|
||||
- Linux/arm64 已完成 30 秒、128 并发、512 空闲连接的 TLS 与非 TLS 持续回归,报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`,均为 0 错误且连接、goroutine、FD 回收正常。
|
||||
- 目标 Linux amd64 服务器已完成 30 分钟、128 并发、512 空闲连接的 TLS 验收:net/http 约 11,865 req/s、native 约 13,098 req/s,均为 0 错误;native P95/P99 为 44.040/52.429 ms,空闲连接堆增量约 4.50 MiB,FD 由 7 增至 519 后回到 7。
|
||||
- 新增 `scripts/verifyProxy.py`,在 Linux Docker 中用 Nginx 反向代理 native TLS 服务,检查真实 TLS、Keep-Alive、上游超时和 native 优雅关闭。
|
||||
- 自动验收入口:`python3 scripts/verifyPerformance.py`;Linux 固定验收入口:`python3 scripts/verifyLinux.py --load --tls --duration 30m`;代理联调入口:`python3 scripts/verifyProxy.py`。
|
||||
|
||||
## 6. 可复用业务基础模块(第一版)
|
||||
|
||||
- `cache`:有界 LRU 内存缓存、TTL、数据复制、同进程合并回源;`cache/redis` 适配 go-redis 客户端,保留借用连接池的生命周期边界。
|
||||
- `httpc`:复用出站连接、每目标主机连接上限、覆盖响应体的总超时、保守重试、requestId 传递、JSON 响应大小限制与安全日志字段。
|
||||
- `middleware`:有容量上限的令牌桶限流;按已认证身份隔离的幂等保护,覆盖冲突、处理中记录、失败记录、容量与请求/响应大小上限;协作式请求 deadline。
|
||||
- `task`:固定 worker、有界队列、任务超时、panic 隔离、间隔调度、不重叠执行、停止接收与排空/取消。
|
||||
- `go-omni add`:向已有项目生成 Handler / Service / Repository / Model 和模块注册入口;目标存在时拒绝覆盖,默认内存 CRUD 可注入持久化 Repository。
|
||||
- `examples/modules` 将以上运行时能力串联,使用真实 TLS 在 nethttp、native、实验 HTTP/2 下验证缓存、出站 requestId、认证、幂等、后台任务与退出。
|
||||
- 使用文档:[业务基础模块](MODULES.md)。验证入口:`GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...` 和 `python3 scripts/verifyModules.py`;Linux runner 顺序运行全量检查及独立 Redis 容器,硬限制 2 CPU / 2 GiB / 256 pids。
|
||||
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
||||
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
||||
|
||||
## 7. 扩展业务模块(本批)
|
||||
|
||||
- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储;补充迁移、审计和分页。
|
||||
- 业务依赖独立接口,云平台 SDK 和 Redis 留在适配器中;按本次范围决定不集成 Orbit,不实现工作流编排。
|
||||
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
|
||||
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
|
||||
|
||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
||||
|
||||
68
app/app.go
68
app/app.go
@@ -21,16 +21,17 @@ import (
|
||||
// App is the top-level GoOmni application. Its public behavior depends on
|
||||
// httpx.Router, leaving the network implementation replaceable.
|
||||
type App struct {
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
clientIPResolver *httpx.ClientIPResolver
|
||||
}
|
||||
|
||||
// Runner is the small lifecycle boundary used by an HTTP server
|
||||
// implementation. The default App.RunContext uses net/http; a custom runner
|
||||
// implementation. App.RunContext selects a configured built-in server; a custom runner
|
||||
// can own the listener, protocol parser and connection lifecycle.
|
||||
type Runner interface {
|
||||
Run(context.Context, http.Handler, string) error
|
||||
@@ -63,12 +64,17 @@ func configure(options ...config.Option) config.Config {
|
||||
}
|
||||
|
||||
func newApp(router httpx.Router, settings config.Config) *App {
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||
if err != nil {
|
||||
panic("go-omni: " + err.Error())
|
||||
}
|
||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||
if settings.EnableLogger {
|
||||
middlewares = append(middlewares, middleware.AccessLog())
|
||||
}
|
||||
middlewares = append(middlewares, middleware.Recovery())
|
||||
return &App{router: router, config: settings, middlewares: middlewares}
|
||||
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
||||
}
|
||||
|
||||
// Router returns the transport contract used by this application.
|
||||
@@ -84,7 +90,11 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
||||
}
|
||||
|
||||
// Config returns the effective application configuration.
|
||||
func (a *App) Config() config.Config { return a.config }
|
||||
func (a *App) Config() config.Config {
|
||||
settings := a.config
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
return settings
|
||||
}
|
||||
|
||||
// Use adds middleware to routes registered after this call.
|
||||
func (a *App) Use(middleware ...httpx.Middleware) *App {
|
||||
@@ -123,7 +133,8 @@ func (a *App) Run(address ...string) error {
|
||||
return a.RunContext(ctx, address...)
|
||||
}
|
||||
|
||||
// NewServer creates a standard HTTP server around the application.
|
||||
// NewServer creates a plain standard HTTP server around the application.
|
||||
// Use RunContext or ServeContext to apply the configured transport and TLS.
|
||||
func (a *App) NewServer(address string) *http.Server {
|
||||
if address == "" {
|
||||
address = a.config.Address
|
||||
@@ -142,23 +153,21 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
}
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
server := a.NewServer(listenOn)
|
||||
listener, err := net.Listen("tcp", server.Addr)
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return a.serveContext(ctx, server, listener)
|
||||
listener, err := net.Listen("tcp", a.listenAddress(address...))
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
// RunWith starts a custom server runner while keeping App-owned cleanup in the
|
||||
// same lifecycle. The runner receives the transport-neutral App handler.
|
||||
// same lifecycle. The runner receives the transport-neutral App handler and
|
||||
// owns transport/TLS settings; built-in transport selection is bypassed.
|
||||
func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) error {
|
||||
if runner == nil {
|
||||
return errors.Join(errors.New("go-omni: nil runner"), a.Close())
|
||||
@@ -166,6 +175,13 @@ func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) err
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, a.listenAddress(address...)), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) listenAddress(address ...string) string {
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
@@ -173,12 +189,18 @@ func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) err
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, listenOn), a.Close())
|
||||
return listenOn
|
||||
}
|
||||
|
||||
func (a *App) serveContext(ctx context.Context, server *http.Server, listener net.Listener) error {
|
||||
serveErr := make(chan error, 1)
|
||||
go func() { serveErr <- server.Serve(listener) }()
|
||||
go func() {
|
||||
if server.TLSConfig != nil {
|
||||
serveErr <- server.ServeTLS(listener, "", "")
|
||||
} else {
|
||||
serveErr <- server.Serve(listener)
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-serveErr:
|
||||
|
||||
@@ -21,6 +21,9 @@ func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.
|
||||
}
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
if len(a.config.TrustedProxies) > 0 {
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
|
||||
}
|
||||
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
|
||||
if err := wrapped(requestContext); err != nil {
|
||||
a.writeError(requestContext, err)
|
||||
|
||||
79
app/transport.go
Normal file
79
app/transport.go
Normal file
@@ -0,0 +1,79 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
h2transport "git.yuntuo.hk/k2safe/go-omni/transport/http2"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
// ServeContext serves a caller-created listener using the configured transport
|
||||
// and TLS. App takes ownership of the listener, including on startup failure,
|
||||
// and runs resource cleanup after the server has stopped.
|
||||
func (a *App) ServeContext(ctx context.Context, listener net.Listener) error {
|
||||
if listener == nil {
|
||||
return errors.Join(errors.New("go-omni: nil listener"), a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
_ = listener.Close()
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
_ = listener.Close()
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) prepareTransport() (func(context.Context, net.Listener) error, error) {
|
||||
if err := a.config.ValidateTransport(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shutdownTimeout := a.config.ShutdownTimeout
|
||||
if shutdownTimeout <= 0 {
|
||||
shutdownTimeout = 10 * time.Second
|
||||
}
|
||||
var tlsConfig *tls.Config
|
||||
if strings.TrimSpace(a.config.TLS.CertFile) != "" {
|
||||
certificate, err := tls.LoadX509KeyPair(a.config.TLS.CertFile, a.config.TLS.KeyFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("go-omni: load TLS certificate: %w", err)
|
||||
}
|
||||
tlsConfig = &tls.Config{Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
switch a.config.Transport {
|
||||
case config.TransportNative:
|
||||
server := native.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
if tlsConfig != nil {
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
}
|
||||
return server.Serve, nil
|
||||
case config.TransportHTTP2:
|
||||
server := h2transport.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
default:
|
||||
server := a.NewServer("")
|
||||
server.TLSConfig = tlsConfig
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return a.serveContext(ctx, server, listener)
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
322
app/transport_test.go
Normal file
322
app/transport_test.go
Normal file
@@ -0,0 +1,322 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
func TestConfiguredTransportLifecycle(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
for _, timeout := range []bool{false, true} {
|
||||
name := transport + "/drain"
|
||||
if timeout {
|
||||
name = transport + "/timeout"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
limit := 3 * time.Second
|
||||
if timeout {
|
||||
limit = 80 * time.Millisecond
|
||||
}
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(limit))
|
||||
started, release, canceled := make(chan struct{}), make(chan struct{}), make(chan struct{})
|
||||
var once sync.Once
|
||||
unblock := func() { once.Do(func() { close(release) }) }
|
||||
defer unblock()
|
||||
var cleaned atomic.Int32
|
||||
application.OnShutdown(func() error { cleaned.Add(1); return nil })
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
select {
|
||||
case <-ctx.Context().Done():
|
||||
close(canceled)
|
||||
return ctx.Context().Err()
|
||||
case <-release:
|
||||
return ctx.OK("finished")
|
||||
}
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wrapped := &closingListener{Listener: listener, closed: make(chan struct{})}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
defer listener.Close()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, wrapped) }()
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
defer response.Body.Close()
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
err = readErr
|
||||
if response.StatusCode != 200 || !strings.Contains(string(body), "finished") {
|
||||
err = errors.New("drained response was lost")
|
||||
}
|
||||
}
|
||||
requestDone <- err
|
||||
}()
|
||||
await(t, started)
|
||||
began := time.Now()
|
||||
cancel()
|
||||
await(t, wrapped.closed)
|
||||
if !timeout {
|
||||
select {
|
||||
case <-canceled:
|
||||
t.Fatal("active request canceled before drain")
|
||||
default:
|
||||
}
|
||||
if cleaned.Load() != 0 {
|
||||
t.Fatal("cleanup preceded active request")
|
||||
}
|
||||
unblock()
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
if timeout {
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(began) < limit {
|
||||
t.Fatalf("timeout ignored: %v after %v", err, time.Since(began))
|
||||
}
|
||||
await(t, canceled)
|
||||
} else if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("server failed to stop")
|
||||
}
|
||||
select {
|
||||
case err := <-requestDone:
|
||||
if !timeout && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("client failed to finish")
|
||||
}
|
||||
_ = application.Close()
|
||||
if cleaned.Load() != 1 {
|
||||
t.Fatalf("cleanup count %d", cleaned.Load())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredStartupFailureClosesResources(t *testing.T) {
|
||||
occupied, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer occupied.Close()
|
||||
for _, test := range []struct {
|
||||
name, errorText string
|
||||
options []config.Option
|
||||
}{
|
||||
{"unknown", "transport", []config.Option{config.WithTransport("unknown")}},
|
||||
{"http2WithoutTLS", "tls", []config.Option{config.WithTransport("http2")}},
|
||||
{"partialTLS", "tls", []config.Option{config.WithTLS("missing.crt", "")}},
|
||||
{"nethttpCertificate", "load TLS certificate", []config.Option{config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"nativeCertificate", "load TLS certificate", []config.Option{config.WithTransport("native"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"http2Certificate", "load TLS certificate", []config.Option{config.WithTransport("http2"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
application := New(test.options...)
|
||||
cleanupError := errors.New("cleanup failure")
|
||||
count := 0
|
||||
application.OnShutdown(func() error { count++; return cleanupError })
|
||||
// The occupied address must not mask configuration/certificate errors.
|
||||
err := application.RunContext(context.Background(), occupied.Addr().String())
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorText) || !errors.Is(err, cleanupError) {
|
||||
t.Fatalf("startup: %v", err)
|
||||
}
|
||||
_ = application.Close()
|
||||
if count != 1 {
|
||||
t.Fatalf("cleanup count %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("unknown"))
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
err = application.ServeContext(ctx, listener)
|
||||
cancel()
|
||||
if canceled && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !canceled && err == nil {
|
||||
t.Fatal("invalid transport accepted")
|
||||
}
|
||||
if _, err := listener.Accept(); !errors.Is(err, net.ErrClosed) {
|
||||
t.Fatalf("listener leaked: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunContextConfiguredTLS(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
reserved, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
address := reserved.Addr().String()
|
||||
reserved.Close()
|
||||
settings := config.Default()
|
||||
settings.Address, settings.Transport = address, transport
|
||||
settings.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
application := New(config.WithConfig(settings))
|
||||
application.GET("/health", func(ctx *httpx.Context) error { return ctx.OK("ok") })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.RunContext(ctx) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("RunContext failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
response, err := client.Get("https://" + address + "/health")
|
||||
if err == nil {
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || !strings.Contains(string(body), `"data":"ok"`) {
|
||||
t.Fatalf("response %s: %v", body, readErr)
|
||||
}
|
||||
want := "h2"
|
||||
if transport == config.TransportNative {
|
||||
want = "http/1.1"
|
||||
}
|
||||
if response.TLS.NegotiatedProtocol != want || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatalf("TLS negotiation: %+v", response.TLS)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type runnerFunc func(context.Context, http.Handler, string) error
|
||||
|
||||
func (f runnerFunc) Run(ctx context.Context, handler http.Handler, address string) error {
|
||||
return f(ctx, handler, address)
|
||||
}
|
||||
|
||||
func TestRunWithRemainsIndependentOfBuiltInTransport(t *testing.T) {
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("custom"), config.WithTLS("missing", "missing"))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
calls, cleanups := 0, 0
|
||||
application.OnShutdown(func() error { cleanups++; return nil })
|
||||
err := application.RunWith(ctx, runnerFunc(func(_ context.Context, handler http.Handler, address string) error {
|
||||
calls++
|
||||
if handler != application || address != "127.0.0.1:1234" {
|
||||
t.Fatal("custom runner arguments changed")
|
||||
}
|
||||
return nil
|
||||
}), "127.0.0.1:1234")
|
||||
cancel()
|
||||
wantCalls := 1
|
||||
if canceled {
|
||||
wantCalls = 0
|
||||
}
|
||||
if err != nil || calls != wantCalls || cleanups != 1 {
|
||||
t.Fatalf("RunWith: %v calls=%d cleanups=%d", err, calls, cleanups)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond))
|
||||
started := make(chan context.Context, 1)
|
||||
release, finished := make(chan struct{}), make(chan struct{})
|
||||
application.GET("/blocked", func(ctx *httpx.Context) error {
|
||||
defer close(finished)
|
||||
started <- ctx.Context()
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan struct{})
|
||||
go func() {
|
||||
defer close(requestDone)
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/blocked")
|
||||
if err == nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
}()
|
||||
var requestContext context.Context
|
||||
select {
|
||||
case requestContext = <-started:
|
||||
case <-time.After(3 * time.Second):
|
||||
close(release)
|
||||
t.Fatal("handler did not start")
|
||||
}
|
||||
defer func() { close(release); await(t, finished) }()
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("handler defeated shutdown deadline")
|
||||
}
|
||||
// net/http's HTTP/2 streams observe the closed socket asynchronously.
|
||||
await(t, requestContext.Done())
|
||||
await(t, requestDone)
|
||||
})
|
||||
}
|
||||
}
|
||||
31
app_test.go
31
app_test.go
@@ -128,3 +128,34 @@ func TestAppUsesConfiguredAddressWhenNoOverrideIsProvided(t *testing.T) {
|
||||
t.Fatalf("expected configured address %q, got %q", settings.Address, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppConfigCopiesTrustedProxies(t *testing.T) {
|
||||
proxies := []string{"127.0.0.1"}
|
||||
app := New(WithTrustedProxies(proxies...))
|
||||
proxies[0] = "10.0.0.0/8"
|
||||
settings := app.Config()
|
||||
if len(settings.TrustedProxies) != 1 || settings.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatalf("trusted proxy config was not copied: %+v", settings.TrustedProxies)
|
||||
}
|
||||
settings.TrustedProxies[0] = "192.0.2.0/24"
|
||||
if app.Config().TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("App.Config should return a copy")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppTrustedProxyClientIPReachesHandlers(t *testing.T) {
|
||||
app := New(WithTrustedProxies("127.0.0.1"))
|
||||
app.GET("/client-ip", func(ctx *Context) error { return ctx.OK(ctx.ClientIP()) })
|
||||
request := httptest.NewRequest(http.MethodGet, "/client-ip", nil)
|
||||
request.RemoteAddr = "127.0.0.1:443"
|
||||
request.Header.Set("X-Forwarded-For", "198.51.100.7")
|
||||
recorder := httptest.NewRecorder()
|
||||
app.Engine().ServeHTTP(recorder, request)
|
||||
var response Response
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.Data != "198.51.100.7" {
|
||||
t.Fatalf("expected trusted proxy client IP, got %#v", response.Data)
|
||||
}
|
||||
}
|
||||
|
||||
106
audit/audit.go
Normal file
106
audit/audit.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package audit records explicitly selected business events without request bodies.
|
||||
package audit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
Time time.Time `json:"time"`
|
||||
ActorID string `json:"actorId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Action string `json:"action"`
|
||||
Resource string `json:"resource"`
|
||||
Outcome string `json:"outcome"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
Fields map[string]string `json:"fields,omitempty"`
|
||||
}
|
||||
type Sink interface {
|
||||
Write(context.Context, Event) error
|
||||
}
|
||||
type Recorder struct {
|
||||
sink Sink
|
||||
allowed map[string]bool
|
||||
}
|
||||
|
||||
func New(sink Sink, allowedFields ...string) (*Recorder, error) {
|
||||
if sink == nil || len(allowedFields) > 32 {
|
||||
return nil, errors.New("audit: sink and at most 32 fields required")
|
||||
}
|
||||
r := &Recorder{sink: sink, allowed: map[string]bool{}}
|
||||
for _, name := range allowedFields {
|
||||
if name == "" || len(name) > 64 {
|
||||
return nil, errors.New("audit: invalid field")
|
||||
}
|
||||
r.allowed[name] = true
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
func (r *Recorder) Record(ctx context.Context, event Event) error {
|
||||
if ctx == nil {
|
||||
return errors.New("audit: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
|
||||
return errors.New("audit: actor/action/resource and valid outcome required")
|
||||
}
|
||||
for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
|
||||
if len(value) > 512 {
|
||||
return errors.New("audit: event field too large")
|
||||
}
|
||||
}
|
||||
event.Time = time.Now().UTC()
|
||||
event.RequestID = logx.RequestID(ctx)
|
||||
event.TraceID = logx.TraceID(ctx)
|
||||
filtered := map[string]string{}
|
||||
for name, value := range event.Fields {
|
||||
if r.allowed[name] {
|
||||
if len(value) > 1024 {
|
||||
return errors.New("audit: attribute too large")
|
||||
}
|
||||
filtered[name] = value
|
||||
}
|
||||
}
|
||||
event.Fields = filtered
|
||||
return r.sink.Write(ctx, event)
|
||||
}
|
||||
|
||||
// JSONSink borrows its writer and serializes complete JSON lines. Write errors
|
||||
// are returned; callers choose whether a failed audit must reject an operation.
|
||||
type JSONSink struct {
|
||||
mu sync.Mutex
|
||||
writer io.Writer
|
||||
}
|
||||
|
||||
func NewJSONSink(writer io.Writer) (*JSONSink, error) {
|
||||
if writer == nil {
|
||||
return nil, errors.New("audit: writer required")
|
||||
}
|
||||
return &JSONSink{writer: writer}, nil
|
||||
}
|
||||
func (s *JSONSink) Write(ctx context.Context, event Event) error {
|
||||
data, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.writer.Write(data)
|
||||
if err == nil && n != len(data) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
return err
|
||||
}
|
||||
50
audit/audit_test.go
Normal file
50
audit/audit_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package audit
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type failedWriter struct{}
|
||||
|
||||
func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
|
||||
func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
sink, _ := NewJSONSink(&buf)
|
||||
recorder, _ := New(sink, "status")
|
||||
ctx := logx.WithRequestID(context.Background(), "r1")
|
||||
ctx = logx.WithTraceID(ctx, "t1")
|
||||
event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := recorder.Record(ctx, event); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
decoder := json.NewDecoder(&buf)
|
||||
for i := 0; i < 20; i++ {
|
||||
var got Event
|
||||
if err := decoder.Decode(&got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
bad, _ := NewJSONSink(failedWriter{})
|
||||
recorder, _ = New(bad)
|
||||
if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
105
auth/adapters_test.go
Normal file
105
auth/adapters_test.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestJWTValidationAndRotation(t *testing.T) {
|
||||
pub, private, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
|
||||
sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
|
||||
t.Helper()
|
||||
token := jwt.NewWithClaims(method, c)
|
||||
token.Header["kid"] = kid
|
||||
raw, err := token.SignedString(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
|
||||
// Constructor clones caller-owned key bytes.
|
||||
pub[0] ^= 1
|
||||
p, err := j.Authenticate(context.Background(), raw)
|
||||
if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
|
||||
t.Fatalf("%+v %v", p, err)
|
||||
}
|
||||
for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
bad := claims
|
||||
change(&bad)
|
||||
if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
|
||||
if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestSessionsRevokeAndIsolation(t *testing.T) {
|
||||
mem, _ := cache.NewMemory(cache.MemoryConfig{})
|
||||
defer mem.Close()
|
||||
s, err := NewSessions(mem, "session:", time.Minute)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, _ := s.key(token)
|
||||
if strings.Contains(key, token) {
|
||||
t.Fatal("plaintext session secret stored")
|
||||
}
|
||||
p, err := s.Authenticate(ctx, token)
|
||||
if err != nil || p.UserID != "u1" {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
p.Roles[0] = "admin"
|
||||
p, _ = s.Authenticate(ctx, token)
|
||||
if p.HasRole("admin") {
|
||||
t.Fatal("principal alias")
|
||||
}
|
||||
if err = s.Revoke(ctx, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -21,9 +21,10 @@ var (
|
||||
// Principal is the authenticated identity made available to business handlers.
|
||||
// Claims are optional provider-specific values and should not be required by core business code.
|
||||
type Principal struct {
|
||||
UserID string
|
||||
Roles []string
|
||||
Claims map[string]any
|
||||
UserID string `json:"userId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
Claims map[string]any `json:"claims,omitempty"`
|
||||
}
|
||||
|
||||
func (p Principal) HasRole(role string) bool {
|
||||
|
||||
127
auth/jwt.go
Normal file
127
auth/jwt.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rsa"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
)
|
||||
|
||||
// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
|
||||
// Keys come from server configuration, never a token's jku/x5u URL.
|
||||
type JWTKey struct {
|
||||
Algorithm string
|
||||
PublicKey any
|
||||
}
|
||||
type JWTConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
Audience string `json:"audience"`
|
||||
Leeway time.Duration `json:"leeway"`
|
||||
Keys map[string]JWTKey `json:"-"`
|
||||
}
|
||||
type JWT struct {
|
||||
config JWTConfig
|
||||
mu sync.RWMutex
|
||||
keys map[string]JWTKey
|
||||
}
|
||||
type JWTClaims struct {
|
||||
jwt.RegisteredClaims
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
}
|
||||
|
||||
func NewJWT(cfg JWTConfig) (*JWT, error) {
|
||||
if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
|
||||
return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
|
||||
}
|
||||
j := &JWT{config: cfg}
|
||||
j.config.Keys = nil
|
||||
if err := j.ReplaceKeys(cfg.Keys); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return j, nil
|
||||
}
|
||||
|
||||
// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
|
||||
// their tokens expire if a rotation should preserve existing sessions.
|
||||
func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
|
||||
if len(keys) == 0 || len(keys) > 64 {
|
||||
return errors.New("auth: require 1..64 JWT keys")
|
||||
}
|
||||
copyKeys := make(map[string]JWTKey, len(keys))
|
||||
for id, key := range keys {
|
||||
if id == "" || len(id) > 128 {
|
||||
return errors.New("auth: invalid key ID")
|
||||
}
|
||||
switch key.Algorithm {
|
||||
case "RS256":
|
||||
k, ok := key.PublicKey.(*rsa.PublicKey)
|
||||
if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
|
||||
return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
|
||||
}
|
||||
case "EdDSA":
|
||||
k, ok := key.PublicKey.(ed25519.PublicKey)
|
||||
if !ok || len(k) != ed25519.PublicKeySize {
|
||||
return errors.New("auth: EdDSA requires an Ed25519 public key")
|
||||
}
|
||||
default:
|
||||
return errors.New("auth: supported algorithms are RS256 and EdDSA")
|
||||
}
|
||||
encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
copyKeys[id] = key
|
||||
}
|
||||
j.mu.Lock()
|
||||
j.keys = copyKeys
|
||||
j.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
|
||||
if ctx == nil {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
if len(raw) == 0 || len(raw) > 8192 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
claims := new(JWTClaims)
|
||||
token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
|
||||
id, ok := t.Header["kid"].(string)
|
||||
if !ok {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
j.mu.RLock()
|
||||
key, exists := j.keys[id]
|
||||
j.mu.RUnlock()
|
||||
if !exists || t.Method.Alg() != key.Algorithm {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
return key.PublicKey, nil
|
||||
}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
|
||||
if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
for _, role := range claims.Roles {
|
||||
if role == "" || len(role) > 128 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
|
||||
}
|
||||
34
auth/policy.go
Normal file
34
auth/policy.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type Policy interface {
|
||||
Authorize(context.Context, Principal, string, string) error
|
||||
}
|
||||
type PolicyFunc func(context.Context, Principal, string, string) error
|
||||
|
||||
func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
|
||||
return f(ctx, p, action, resource)
|
||||
}
|
||||
func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
p, ok := FromContext(c)
|
||||
if !ok {
|
||||
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||
}
|
||||
if policy == nil || action == "" || resource == nil {
|
||||
return apierrors.Internal(errors.New("auth: incomplete permission policy"))
|
||||
}
|
||||
if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
|
||||
return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
87
auth/session.go
Normal file
87
auth/session.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
)
|
||||
|
||||
// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
|
||||
// The caller verifies login credentials before Issue; Store owns persistence.
|
||||
type Sessions struct {
|
||||
store cache.Store
|
||||
prefix string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
|
||||
if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
|
||||
return nil, errors.New("auth: invalid session configuration")
|
||||
}
|
||||
return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
|
||||
}
|
||||
func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
|
||||
if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
data, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(data) > 16384 {
|
||||
return "", errors.New("auth: session principal too large")
|
||||
}
|
||||
secret := make([]byte, 32)
|
||||
if _, err := rand.Read(secret); err != nil {
|
||||
return "", err
|
||||
}
|
||||
token := base64.RawURLEncoding.EncodeToString(secret)
|
||||
key, _ := s.key(token)
|
||||
if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
func (s *Sessions) key(token string) (string, error) {
|
||||
if len(token) != 43 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
b, err := base64.RawURLEncoding.DecodeString(token)
|
||||
if err != nil || len(b) != 32 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
sum := sha256.Sum256(b)
|
||||
return s.prefix + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
data, err := s.store.Get(ctx, key)
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
var p Principal
|
||||
if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
func (s *Sessions) Revoke(ctx context.Context, token string) error {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Delete(ctx, key)
|
||||
}
|
||||
135
cache/cache.go
vendored
Normal file
135
cache/cache.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
// Package cache provides replaceable byte-oriented caches and bounded local
|
||||
// request coalescing. Storage errors are never silently treated as misses.
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrMiss = errors.New("cache: key not found")
|
||||
ErrClosed = errors.New("cache: closed")
|
||||
ErrTooLarge = errors.New("cache: entry exceeds capacity")
|
||||
ErrBusy = errors.New("cache: too many concurrent loads")
|
||||
)
|
||||
|
||||
// Store implementations copy values and accept ttl=0 for no expiration.
|
||||
// Negative TTLs are invalid. Delete is successful for missing keys.
|
||||
type Store interface {
|
||||
Get(context.Context, string) ([]byte, error)
|
||||
Set(context.Context, string, []byte, time.Duration) error
|
||||
Delete(context.Context, ...string) error
|
||||
}
|
||||
|
||||
func Check(ctx context.Context, key string, ttl time.Duration) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if key == "" || len(key) > 1024 {
|
||||
return errors.New("cache: key must contain 1..1024 bytes")
|
||||
}
|
||||
if ttl < 0 {
|
||||
return errors.New("cache: negative TTL")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type LoaderConfig struct {
|
||||
MaxLoads int `json:"maxLoads"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
}
|
||||
|
||||
type flight struct {
|
||||
done chan struct{}
|
||||
value []byte
|
||||
err error
|
||||
}
|
||||
|
||||
// Loader coalesces concurrent misses for the same key within this process.
|
||||
// Caller cancellation stops waiting without canceling other callers' load.
|
||||
// Load functions must observe their context, which has a bounded deadline.
|
||||
type Loader struct {
|
||||
store Store
|
||||
config LoaderConfig
|
||||
mu sync.Mutex
|
||||
flights map[string]*flight
|
||||
}
|
||||
|
||||
func NewLoader(store Store, cfg LoaderConfig) (*Loader, error) {
|
||||
if store == nil || cfg.MaxLoads < 0 || cfg.Timeout < 0 {
|
||||
return nil, errors.New("cache: invalid loader configuration")
|
||||
}
|
||||
if cfg.MaxLoads == 0 {
|
||||
cfg.MaxLoads = 128
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
return &Loader{store: store, config: cfg, flights: make(map[string]*flight)}, nil
|
||||
}
|
||||
|
||||
func (l *Loader) GetOrLoad(ctx context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error)) ([]byte, error) {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if load == nil {
|
||||
return nil, errors.New("cache: nil loader")
|
||||
}
|
||||
value, err := l.store.Get(ctx, key)
|
||||
if !errors.Is(err, ErrMiss) {
|
||||
return value, err
|
||||
}
|
||||
l.mu.Lock()
|
||||
f := l.flights[key]
|
||||
if f == nil {
|
||||
if len(l.flights) >= l.config.MaxLoads {
|
||||
l.mu.Unlock()
|
||||
return nil, ErrBusy
|
||||
}
|
||||
f = &flight{done: make(chan struct{})}
|
||||
l.flights[key] = f
|
||||
go l.run(context.WithoutCancel(ctx), key, ttl, load, f)
|
||||
}
|
||||
l.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-f.done:
|
||||
return append([]byte(nil), f.value...), f.err
|
||||
}
|
||||
}
|
||||
|
||||
func (l *Loader) run(parent context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error), f *flight) {
|
||||
ctx, cancel := context.WithTimeout(parent, l.config.Timeout)
|
||||
defer cancel()
|
||||
defer func() {
|
||||
if value := recover(); value != nil {
|
||||
f.err = fmt.Errorf("cache: loader panic: %v", value)
|
||||
}
|
||||
l.mu.Lock()
|
||||
delete(l.flights, key)
|
||||
close(f.done)
|
||||
l.mu.Unlock()
|
||||
}()
|
||||
// Another request may have populated the key between the first Get and
|
||||
// flight acquisition. Recheck inside the coalesced load.
|
||||
f.value, f.err = l.store.Get(ctx, key)
|
||||
if !errors.Is(f.err, ErrMiss) {
|
||||
return
|
||||
}
|
||||
f.value, f.err = load(ctx)
|
||||
if f.err == nil {
|
||||
f.err = ctx.Err()
|
||||
}
|
||||
if f.err == nil {
|
||||
f.err = l.store.Set(ctx, key, f.value, ttl)
|
||||
}
|
||||
f.value = append([]byte(nil), f.value...)
|
||||
}
|
||||
118
cache/cache_test.go
vendored
Normal file
118
cache/cache_test.go
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestMemoryOwnershipCapacityExpiryAndClose(t *testing.T) {
|
||||
m, err := NewMemory(MemoryConfig{MaxEntries: 2, MaxBytes: 10})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now()
|
||||
m.now = func() time.Time { return now }
|
||||
ctx := context.Background()
|
||||
value := []byte("aa")
|
||||
if err := m.Set(ctx, "a", value, time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value[0] = 'x'
|
||||
got, _ := m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("retained caller memory")
|
||||
}
|
||||
got[0] = 'z'
|
||||
got, _ = m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("returned mutable cache memory")
|
||||
}
|
||||
m.Set(ctx, "b", []byte("bb"), 0)
|
||||
m.Get(ctx, "a")
|
||||
m.Set(ctx, "c", []byte("cc"), 0)
|
||||
if _, err := m.Get(ctx, "b"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("LRU eviction failed")
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if _, err := m.Get(ctx, "a"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("expiration boundary ignored")
|
||||
}
|
||||
if err := m.Set(ctx, "large", []byte("123456"), 0); !errors.Is(err, ErrTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := m.Get(ctx, "c"); err != nil {
|
||||
t.Fatal("oversized write evicted valid data")
|
||||
}
|
||||
canceled, cancel := context.WithCancel(ctx)
|
||||
cancel()
|
||||
if err := m.Delete(canceled, "c"); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
m.Close()
|
||||
if _, err := m.Get(ctx, "c"); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderCoalescesAndSeparatesCallerCancellation(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
loader, _ := NewLoader(m, LoaderConfig{MaxLoads: 1, Timeout: time.Second})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
load := func(ctx context.Context) ([]byte, error) {
|
||||
calls.Add(1)
|
||||
close(started)
|
||||
select {
|
||||
case <-release:
|
||||
return []byte("value"), nil
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
first := make(chan error, 1)
|
||||
go func() { _, err := loader.GetOrLoad(ctx, "key", time.Minute, load); first <- err }()
|
||||
<-started
|
||||
second := make(chan error, 1)
|
||||
go func() {
|
||||
v, err := loader.GetOrLoad(context.Background(), "key", time.Minute, load)
|
||||
if err == nil && string(v) != "value" {
|
||||
err = errors.New("wrong value")
|
||||
}
|
||||
second <- err
|
||||
}()
|
||||
if _, err := loader.GetOrLoad(context.Background(), "other", 0, load); !errors.Is(err, ErrBusy) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel()
|
||||
if err := <-first; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
close(release)
|
||||
if err := <-second; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 1 {
|
||||
t.Fatalf("loads=%d", calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderPanicDeadlineAndStoreFailure(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
l, _ := NewLoader(m, LoaderConfig{Timeout: 20 * time.Millisecond})
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(context.Context) ([]byte, error) { panic("failure") }); err == nil {
|
||||
t.Fatal("panic escaped")
|
||||
}
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(ctx context.Context) ([]byte, error) { <-ctx.Done(); return nil, ctx.Err() }); !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
if _, err := l.GetOrLoad(context.Background(), "closed", 0, func(context.Context) ([]byte, error) { t.Fatal("store outage triggered load"); return nil, nil }); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
135
cache/memory.go
vendored
Normal file
135
cache/memory.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type MemoryConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"` // Sum of key and value bytes, excluding Go object overhead.
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
key string
|
||||
value []byte
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// Memory is an LRU cache with lazy expiration and no background goroutines.
|
||||
type Memory struct {
|
||||
mu sync.Mutex
|
||||
config MemoryConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
bytes int
|
||||
closed bool
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewMemory(cfg MemoryConfig) (*Memory, error) {
|
||||
if cfg.MaxEntries < 0 || cfg.MaxBytes < 0 {
|
||||
return nil, errors.New("cache: negative capacity")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 4096
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 32 << 20
|
||||
}
|
||||
return &Memory{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (m *Memory) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
e := m.items[key]
|
||||
if e == nil {
|
||||
return nil, ErrMiss
|
||||
}
|
||||
item := e.Value.(*entry)
|
||||
if !item.expires.IsZero() && !m.now().Before(item.expires) {
|
||||
m.remove(e)
|
||||
return nil, ErrMiss
|
||||
}
|
||||
m.lru.MoveToFront(e)
|
||||
return append([]byte(nil), item.value...), nil
|
||||
}
|
||||
|
||||
func (m *Memory) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(value) > m.config.MaxBytes-len(key) {
|
||||
return ErrTooLarge
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
for len(m.items) >= m.config.MaxEntries || m.bytes+len(key)+len(value) > m.config.MaxBytes {
|
||||
m.remove(m.lru.Back())
|
||||
}
|
||||
item := &entry{key: key, value: append([]byte(nil), value...)}
|
||||
if ttl > 0 {
|
||||
item.expires = m.now().Add(ttl)
|
||||
}
|
||||
m.items[key] = m.lru.PushFront(item)
|
||||
m.bytes += len(key) + len(value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, key := range keys {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
for _, key := range keys {
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) remove(e *list.Element) {
|
||||
item := e.Value.(*entry)
|
||||
m.bytes -= len(item.key) + len(item.value)
|
||||
delete(m.items, item.key)
|
||||
m.lru.Remove(e)
|
||||
}
|
||||
|
||||
func (m *Memory) Close() error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.closed = true
|
||||
clear(m.items)
|
||||
m.lru.Init()
|
||||
m.bytes = 0
|
||||
return nil
|
||||
}
|
||||
68
cache/redis/redis.go
vendored
Normal file
68
cache/redis/redis.go
vendored
Normal file
@@ -0,0 +1,68 @@
|
||||
// Package redis adapts go-redis's pooled client to cache.Store.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
// Store borrows the client; its owner must close the client after users stop.
|
||||
type Store struct {
|
||||
client goredis.Cmdable
|
||||
prefix string
|
||||
}
|
||||
|
||||
func New(client goredis.Cmdable, prefix string) (*Store, error) {
|
||||
if client == nil || len(prefix) > 1024 || strings.ContainsAny(prefix, "\r\n") {
|
||||
return nil, errors.New("cache/redis: invalid client or prefix")
|
||||
}
|
||||
return &Store{client: client, prefix: prefix}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value, err := s.client.Get(ctx, s.prefix+key).Bytes()
|
||||
if errors.Is(err, goredis.Nil) {
|
||||
return nil, cache.ErrMiss
|
||||
}
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (s *Store) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := cache.Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
// Redis expiration precision is milliseconds; round up, never truncate a
|
||||
// positive duration to an unbounded lifetime.
|
||||
if ttl > 0 && ttl < time.Millisecond {
|
||||
ttl = time.Millisecond
|
||||
}
|
||||
return s.client.Set(ctx, s.prefix+key, value, ttl).Err()
|
||||
}
|
||||
|
||||
func (s *Store) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache/redis: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
names := make([]string, len(keys))
|
||||
for i, key := range keys {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
names[i] = s.prefix + key
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return nil
|
||||
}
|
||||
return s.client.Del(ctx, names...).Err()
|
||||
}
|
||||
125
cache/redis/redis_test.go
vendored
Normal file
125
cache/redis/redis_test.go
vendored
Normal file
@@ -0,0 +1,125 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
type fakeClient struct {
|
||||
goredis.Cmdable
|
||||
key string
|
||||
ttl time.Duration
|
||||
failure error
|
||||
}
|
||||
|
||||
func (f *fakeClient) Get(ctx context.Context, key string) *goredis.StringCmd {
|
||||
f.key = key
|
||||
return goredis.NewStringResult("", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Set(ctx context.Context, key string, value interface{}, ttl time.Duration) *goredis.StatusCmd {
|
||||
f.key, f.ttl = key, ttl
|
||||
return goredis.NewStatusResult("OK", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Del(ctx context.Context, keys ...string) *goredis.IntCmd {
|
||||
f.key = keys[0]
|
||||
return goredis.NewIntResult(1, f.failure)
|
||||
}
|
||||
|
||||
func TestAdapterPreservesErrorsAndExpiration(t *testing.T) {
|
||||
f := &fakeClient{failure: goredis.Nil}
|
||||
store, _ := New(f, "app:")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, cache.ErrMiss) || f.key != "app:key" {
|
||||
t.Fatal(err, f.key)
|
||||
}
|
||||
f.failure = errors.New("unavailable")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, f.failure) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.failure = nil
|
||||
if err := store.Set(context.Background(), "key", []byte("value"), time.Nanosecond); err != nil || f.ttl != time.Millisecond {
|
||||
t.Fatal(err, f.ttl)
|
||||
}
|
||||
if err := store.Set(context.Background(), "key", nil, -time.Second); err == nil {
|
||||
t.Fatal("negative TTL accepted")
|
||||
}
|
||||
if err := store.Delete(context.Background(), "key"); err != nil || f.key != "app:key" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("set GO_OMNI_REDIS_TEST_ADDR to an isolated test Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 2, MaxRetries: -1, ContextTimeoutEnabled: true, DialTimeout: time.Second, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.Ping(ctx).Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prefix := fmt.Sprintf("goOmniTest:%d:", time.Now().UnixNano())
|
||||
store, _ := New(client, prefix)
|
||||
defer store.Delete(context.Background(), "key", "forever", "list", "canceled")
|
||||
if err := store.Set(ctx, "key", []byte{0, 1, 255}, 80*time.Millisecond); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v, err := store.Get(ctx, "key")
|
||||
if err != nil || len(v) != 3 || v[2] != 255 {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
_, err := store.Get(ctx, "key")
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
break
|
||||
}
|
||||
if err != nil || time.Now().After(deadline) {
|
||||
t.Fatal("TTL failed", err)
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
if err := store.Set(ctx, "forever", []byte("ok"), 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ttl := client.PTTL(ctx, prefix+"forever").Val(); ttl != -1 {
|
||||
t.Fatalf("unexpected persistent TTL %v", ttl)
|
||||
}
|
||||
if err := store.Delete(ctx, "forever"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "forever"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.RPush(ctx, prefix+"list", "value").Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "list"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("Redis error treated as miss", err)
|
||||
}
|
||||
canceled, stop := context.WithCancel(ctx)
|
||||
stop()
|
||||
if err := store.Set(canceled, "canceled", []byte("value"), 0); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "canceled"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("canceled write reached Redis", err)
|
||||
}
|
||||
if err := store.Delete(ctx, "list"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "key"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("closed client treated as miss", err)
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,16 @@
|
||||
# go-omni CLI
|
||||
|
||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||
该命令尚未包含在下方历史标签中;当前源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||
|
||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.5.0
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -16,7 +23,7 @@ go run ./cmd/server
|
||||
需要选择其他已发布版本时,使用完整版本号:
|
||||
|
||||
```sh
|
||||
go-omni new --framework-version v0.5.0 --module example.com/billing ./billing
|
||||
go-omni new --framework-version v0.6.0-preview --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
`--framework-version` 禁用自动本地替换;`main`、`latest` 等浮动引用不接受为固定版本。
|
||||
@@ -48,8 +55,8 @@ MySQL 项目额外包含 `schema.sql`、MySQL Repository 和可选的数据库
|
||||
发布验收使用 Python 3 标准库脚本,不需要额外 Python 依赖:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyScaffold.py --version v0.5.0
|
||||
python3 scripts/verifyDatabase.py --version v0.5.0
|
||||
python3 scripts/verifyScaffold.py --version v0.6.0-preview
|
||||
python3 scripts/verifyDatabase.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
脚本在源码目录外安装远程 CLI、生成项目,确认固定版本且没有 `replace`,再下载依赖、
|
||||
@@ -65,3 +72,20 @@ python3 scripts/verifyDatabase.py --cli /tmp/go-omni --framework-dir .
|
||||
```
|
||||
|
||||
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
||||
|
||||
## 显式迁移(main)
|
||||
|
||||
SQL 命名为 `0001_createUsers.up.sql`。先预览版本和 SHA-256 清单:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations
|
||||
```
|
||||
|
||||
部署步骤配置 `GO_OMNI_DB_DSN`,显式执行:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations --apply
|
||||
```
|
||||
|
||||
预览不连接数据库;应用启动不自动迁移。失败保留 dirty 状态,不提供自动 down/force。
|
||||
脚手架原有 `schema.sql` 不会自动变成有版本的迁移,已有库需要先确定基线。详见 [扩展模块](../../ADVANCED.md)。
|
||||
|
||||
@@ -17,6 +17,16 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "migrate":
|
||||
if err := runMigrate(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "add":
|
||||
if err := runAdd(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "new":
|
||||
if err := runNew(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni new:", err)
|
||||
@@ -103,8 +113,38 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||
fmt.Fprintln(os.Stderr, "")
|
||||
fmt.Fprintln(os.Stderr, "Usage:")
|
||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
|
||||
}
|
||||
|
||||
func runAdd(args []string) error {
|
||||
directory, name := ".", ""
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--dir":
|
||||
if index+1 >= len(args) {
|
||||
return errors.New("--dir requires a project directory")
|
||||
}
|
||||
index++
|
||||
directory = args[index]
|
||||
case strings.HasPrefix(arg, "--dir="):
|
||||
directory = strings.TrimPrefix(arg, "--dir=")
|
||||
case strings.HasPrefix(arg, "-"):
|
||||
return fmt.Errorf("unknown option %q", arg)
|
||||
case name == "":
|
||||
name = arg
|
||||
default:
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
}
|
||||
if err := scaffold.AddModule(directory, name); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created internal/%s; see its README.md for route registration\n", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -32,6 +32,30 @@ func TestNewRejectsEmptyFrameworkVersion(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleCLI(t *testing.T) {
|
||||
for _, equals := range []bool{false, true} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := runNew([]string{"--module", "example.com/billing", project}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
args := []string{"orders", "--dir", project}
|
||||
if equals {
|
||||
args = []string{"--dir=" + project, "orders"}
|
||||
}
|
||||
if err := runAdd(args); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "internal/orders/module.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{nil, {"orders", "--dir"}, {"orders", "--dir="}, {"orders", "extra"}, {"orders", "--unknown"}} {
|
||||
if err := runAdd(args); err == nil {
|
||||
t.Fatalf("accepted invalid arguments %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewMySQLDatabaseOptions(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--database", "mysql"}, {"--database=mysql"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
|
||||
60
cmd/go-omni/migrate.go
Normal file
60
cmd/go-omni/migrate.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func runMigrate(args []string) error {
|
||||
flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
directory := flags.String("dir", "migrations", "SQL directory")
|
||||
dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
|
||||
apply := flags.Bool("apply", false, "apply pending up migrations")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("migrate: unexpected argument")
|
||||
}
|
||||
_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !*apply {
|
||||
data, err := json.MarshalIndent(manifest, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
dsn := os.Getenv(*dsnEnv)
|
||||
if dsn == "" {
|
||||
return errors.New("migrate: DSN environment variable is empty")
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
version, dirty, err := runner.Version()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
|
||||
return nil
|
||||
}
|
||||
25
cmd/go-omni/migrate_test.go
Normal file
25
cmd/go-omni/migrate_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMigrationPreviewDoesNotConnect(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
|
||||
if err := runMigrate([]string{"--dir", dir}); err != nil {
|
||||
t.Fatal("preview tried to connect", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "")
|
||||
if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
|
||||
t.Fatal("missing DSN accepted")
|
||||
}
|
||||
if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
|
||||
t.Fatal("unsafe unknown flag accepted")
|
||||
}
|
||||
}
|
||||
109
cmd/go-omni/scaffold/addModule.go
Normal file
109
cmd/go-omni/scaffold/addModule.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
)
|
||||
|
||||
var moduleName = regexp.MustCompile(`^[a-z][a-zA-Z0-9]{0,47}$`)
|
||||
|
||||
// AddModule creates a complete, injectable memory-backed CRUD module. It never
|
||||
// edits existing routes, go.mod, migrations or files in another module.
|
||||
func AddModule(directory, name string) (err error) {
|
||||
if strings.TrimSpace(directory) == "" || !moduleName.MatchString(name) || token.Lookup(name).IsKeyword() || name == "main" || name == "init" || name == "internal" {
|
||||
return errors.New("module name must be a non-keyword lowerCamelCase Go identifier (1..48 characters)")
|
||||
}
|
||||
root, err := filepath.Abs(directory)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
root, err = filepath.EvalSymlinks(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("read project go.mod: %w", err)
|
||||
}
|
||||
manifest, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || manifest.Module == nil {
|
||||
return errors.New("project go.mod must declare a valid module")
|
||||
}
|
||||
found := false
|
||||
for _, dependency := range manifest.Require {
|
||||
if dependency.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("project must already require go-omni; create it with go-omni new")
|
||||
}
|
||||
module := manifest.Module.Mod.Path
|
||||
if strings.ContainsAny(module, "\"\\\r\n") {
|
||||
return errors.New("invalid project module path")
|
||||
}
|
||||
paths := []string{"module.go", "model/model.go", "repository/repository.go", "service/service.go", "handler/handler.go", "README.md"}
|
||||
rendered := make(map[string][]byte, len(paths))
|
||||
for _, path := range paths {
|
||||
source, readErr := templateFiles.ReadFile("moduleTemplates/" + strings.ReplaceAll(path, "/", "-") + ".tmpl")
|
||||
if readErr != nil {
|
||||
return readErr
|
||||
}
|
||||
tmpl, parseErr := template.New(path).Parse(string(source))
|
||||
if parseErr != nil {
|
||||
return parseErr
|
||||
}
|
||||
var output bytes.Buffer
|
||||
if err := tmpl.Execute(&output, struct{ Module, Name string }{module, name}); err != nil {
|
||||
return err
|
||||
}
|
||||
content := output.Bytes()
|
||||
if strings.HasSuffix(path, ".go") {
|
||||
content, err = format.Source(content)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rendered[path] = content
|
||||
}
|
||||
parent := filepath.Join(root, "internal")
|
||||
if info, statErr := os.Lstat(parent); statErr == nil {
|
||||
if !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return errors.New("internal must be a real directory, not a symlink")
|
||||
}
|
||||
} else if !errors.Is(statErr, os.ErrNotExist) {
|
||||
return statErr
|
||||
} else if err := os.Mkdir(parent, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
target := filepath.Join(parent, name)
|
||||
// Exclusive creation protects existing files, including empty directories.
|
||||
if err := os.Mkdir(target, 0755); err != nil {
|
||||
return fmt.Errorf("create module (existing modules are never overwritten): %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err != nil {
|
||||
_ = os.RemoveAll(target)
|
||||
}
|
||||
}()
|
||||
for _, path := range paths {
|
||||
file := filepath.Join(target, filepath.FromSlash(path))
|
||||
if err = os.MkdirAll(filepath.Dir(file), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = os.WriteFile(file, rendered[path], 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
@@ -0,0 +1,137 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func moduleProject(t *testing.T) string {
|
||||
t.Helper()
|
||||
root := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: root, Module: "example.com/billing", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
func TestAddModulePreservesProjectAndRejectsOverwrite(t *testing.T) {
|
||||
root := moduleProject(t)
|
||||
manifest, _ := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
main, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go"))
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, path := range []string{"module.go", "model/model.go", "handler/handler.go", "service/service.go", "repository/repository.go", "README.md"} {
|
||||
if _, err := os.Stat(filepath.Join(root, "internal/orderItems", path)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
marker := filepath.Join(root, "internal/orderItems/keep.txt")
|
||||
if err := os.WriteFile(marker, []byte("keep"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orderItems"); err == nil {
|
||||
t.Fatal("overwrote existing module")
|
||||
}
|
||||
if got, _ := os.ReadFile(marker); string(got) != "keep" {
|
||||
t.Fatal("existing file changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "go.mod")); string(got) != string(manifest) {
|
||||
t.Fatal("go.mod changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go")); string(got) != string(main) {
|
||||
t.Fatal("main changed")
|
||||
}
|
||||
for _, name := range []string{"../escape", "Order", "order_items", "func", "main", "init", "internal", "", strings.Repeat("a", 49)} {
|
||||
if err := AddModule(root, name); err == nil {
|
||||
t.Fatalf("accepted %q", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleRejectsForeignProjectAndSymlink(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.com/foreign\ngo 1.22\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted foreign project")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(root, "internal")); !os.IsNotExist(err) {
|
||||
t.Fatal("created files before validation")
|
||||
}
|
||||
root = moduleProject(t)
|
||||
if err := os.Rename(filepath.Join(root, "internal"), filepath.Join(root, "original")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(filepath.Join(root, "original"), filepath.Join(root, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted symlink parent")
|
||||
}
|
||||
}
|
||||
|
||||
// Opt-in because this compiles a separate consuming module and may need module
|
||||
// downloads. The Linux feature acceptance runner enables it explicitly.
|
||||
func TestAddModuleIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-project CRUD")
|
||||
}
|
||||
root := moduleProject(t)
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "internal/orderItems/module_test.go"), []byte(generatedModuleTest), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "./...")
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated project: %v\n%s", err, output)
|
||||
}
|
||||
}
|
||||
|
||||
const generatedModuleTest = `package orderItems_test
|
||||
import (
|
||||
"context"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"example.com/billing/internal/orderItems"
|
||||
)
|
||||
func TestGeneratedCRUD(t *testing.T) {
|
||||
app:=omni.New()
|
||||
repo:=orderItems.NewMemory()
|
||||
orderItems.Register(app,repo,func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if c.Header("Authorization")!="Bearer demo" {return omni.Unauthorized("DENIED","denied")};return next(c)
|
||||
}})
|
||||
request:=func(method,path,body string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type","application/json")
|
||||
if authorized {r.Header.Set("Authorization","Bearer demo")}
|
||||
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/orderItems","{\"name\":\"first\"}",false);w.Code!=401 {t.Fatal(w.Code)}
|
||||
cases:=[]struct{method,path,body string;status int;contains string}{
|
||||
{"POST","/orderItems","{\"name\":\" first \"}",201,"createdAt"},
|
||||
{"GET","/orderItems/1","",200,"first"},
|
||||
{"PUT","/orderItems/1","{\"name\":\"changed\"}",200,"changed"},
|
||||
{"GET","/orderItems","",200,"changed"},
|
||||
{"POST","/orderItems","{\"name\":\" \"}",400,"INVALID_NAME"},
|
||||
{"DELETE","/orderItems/1","",204,""},
|
||||
{"GET","/orderItems/1","",404,"NOT_FOUND"},
|
||||
}
|
||||
for _,tc:=range cases{w:=request(tc.method,tc.path,tc.body,true);if w.Code!=tc.status||!strings.Contains(w.Body.String(),tc.contains){t.Fatalf("%s %s: %d %s",tc.method,tc.path,w.Code,w.Body.String())}}
|
||||
ctx,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if _,err:=repo.List(ctx);err!=context.Canceled {t.Fatal(err)}
|
||||
}
|
||||
`
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
# {{.Name}}
|
||||
|
||||
在业务启动文件导入并注册:
|
||||
|
||||
```go
|
||||
import "{{.Module}}/internal/{{.Name}}"
|
||||
|
||||
{{.Name}}.Register(application, {{.Name}}.NewMemory())
|
||||
```
|
||||
|
||||
提供 `POST /{{.Name}}`、`GET /{{.Name}}`、`GET /{{.Name}}/:id`、`PUT /{{.Name}}/:id`、
|
||||
`DELETE /{{.Name}}/:id`。写入 JSON 为 `{"name":"example"}`;输出字段为 id、name、createdAt。
|
||||
代码按 Handler / Service / Repository / Model 分层,`Register` 可注入自己的 Repository 和认证中间件。
|
||||
内存示例不持久化、不提供分页;列表适合小规模演示。接实际业务时定义领域字段、分页、鉴权和数据库实现。
|
||||
生成器不会自动修改主入口、已有模块或数据库结构,也不会覆盖已有文件。
|
||||
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
@@ -0,0 +1,52 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"errors"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Handler struct { service *service.Service }
|
||||
type WriteRequest struct { Name string `json:"name" binding:"required,max=200"` }
|
||||
func New(service *service.Service) *Handler {return &Handler{service:service}}
|
||||
func (h *Handler) Register(routes *omni.RouterGroup) {
|
||||
routes.POST("",h.Create)
|
||||
routes.GET("",h.List)
|
||||
routes.GET("/:id",h.Get)
|
||||
routes.PUT("/:id",h.Update)
|
||||
routes.DELETE("/:id",h.Delete)
|
||||
}
|
||||
func (h *Handler) Create(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Create(c.Context(),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.Created(item)
|
||||
}
|
||||
func (h *Handler) Get(c *omni.Context) error {
|
||||
item,err:=h.service.Get(c.Context(),c.Param("id"))
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) List(c *omni.Context) error {
|
||||
items,err:=h.service.List(c.Context())
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(items)
|
||||
}
|
||||
func (h *Handler) Update(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Update(c.Context(),c.Param("id"),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) Delete(c *omni.Context) error {
|
||||
if err:=h.service.Delete(c.Context(),c.Param("id")); err!=nil {return publicError(err)}
|
||||
return c.NoContent()
|
||||
}
|
||||
func publicError(err error) error {
|
||||
if errors.Is(err,model.ErrNotFound) {return omni.NotFound("NOT_FOUND","record not found").WithCause(err)}
|
||||
if errors.Is(err,model.ErrInvalidName) {return omni.BadRequest("INVALID_NAME","name must contain 1..200 bytes").WithCause(err)}
|
||||
return err
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
var ErrNotFound = errors.New("{{.Name}}: record not found")
|
||||
var ErrInvalidName = errors.New("{{.Name}}: name must contain 1..200 bytes")
|
||||
|
||||
type Item struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
@@ -0,0 +1,19 @@
|
||||
package {{.Name}}
|
||||
|
||||
import (
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/handler"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Repository = repository.Repository
|
||||
|
||||
func NewMemory() Repository { return repository.NewMemory() }
|
||||
|
||||
// Register wires this module with caller-owned storage. Pass auth middleware
|
||||
// here to apply it to every route before module-specific middleware.
|
||||
func Register(app *omni.App, repo Repository, middleware ...omni.Middleware) {
|
||||
if repo == nil { panic("{{.Name}}: repository is required") }
|
||||
handler.New(service.New(repo)).Register(app.Group("/{{.Name}}", middleware...))
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
)
|
||||
|
||||
type Repository interface {
|
||||
Create(context.Context,string) (model.Item,error)
|
||||
Get(context.Context,string) (model.Item,error)
|
||||
List(context.Context) ([]model.Item,error)
|
||||
Update(context.Context,string,string) (model.Item,error)
|
||||
Delete(context.Context,string) error
|
||||
}
|
||||
|
||||
// Memory is demonstration storage; data and IDs reset with the process.
|
||||
type Memory struct { mu sync.Mutex; next uint64; items map[string]model.Item }
|
||||
func NewMemory() *Memory { return &Memory{items:make(map[string]model.Item)} }
|
||||
func (m *Memory) Create(ctx context.Context, name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
m.next++
|
||||
item:=model.Item{ID:strconv.FormatUint(m.next,10),Name:name,CreatedAt:time.Now().UTC()}
|
||||
m.items[item.ID]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Get(ctx context.Context,id string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) List(ctx context.Context) ([]model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return nil,err}
|
||||
items:=make([]model.Item,0,len(m.items))
|
||||
for _,item:=range m.items {items=append(items,item)}
|
||||
sort.Slice(items,func(i,j int)bool{return items[i].ID<items[j].ID})
|
||||
return items,nil
|
||||
}
|
||||
func (m *Memory) Update(ctx context.Context,id,name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
item.Name=name; m.items[id]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Delete(ctx context.Context,id string) error {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return err}
|
||||
if _,ok:=m.items[id]; !ok {return model.ErrNotFound}
|
||||
delete(m.items,id)
|
||||
return nil
|
||||
}
|
||||
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
@@ -0,0 +1,27 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
)
|
||||
|
||||
type Service struct { repo repository.Repository }
|
||||
func New(repo repository.Repository) *Service { return &Service{repo:repo} }
|
||||
func name(value string) (string,error) {
|
||||
value=strings.TrimSpace(value)
|
||||
if len(value)==0 || len(value)>200 {return "",model.ErrInvalidName}
|
||||
return value,nil
|
||||
}
|
||||
func (s *Service) Create(ctx context.Context,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Create(ctx,value)
|
||||
}
|
||||
func (s *Service) Get(ctx context.Context,id string) (model.Item,error) { return s.repo.Get(ctx,id) }
|
||||
func (s *Service) List(ctx context.Context) ([]model.Item,error) { return s.repo.List(ctx) }
|
||||
func (s *Service) Update(ctx context.Context,id,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Update(ctx,id,value)
|
||||
}
|
||||
func (s *Service) Delete(ctx context.Context,id string) error {return s.repo.Delete(ctx,id)}
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
"text/template"
|
||||
)
|
||||
|
||||
//go:embed templates
|
||||
//go:embed templates moduleTemplates
|
||||
var templateFiles embed.FS
|
||||
|
||||
// Options controls project generation.
|
||||
|
||||
@@ -47,6 +47,11 @@ go build -o server ./cmd/server
|
||||
```
|
||||
|
||||
`GO_OMNI_SHUTDOWN_TIMEOUT` 控制等待请求完成的时间,默认 `10s`。
|
||||
|
||||
使用支持统一网络配置的框架版本时,设置 `GO_OMNI_TRANSPORT=native` 即可切换为自研 HTTP/1.1;
|
||||
默认 `nethttp`。`GO_OMNI_TLS_CERT_FILE` 和 `GO_OMNI_TLS_KEY_FILE` 配对启用 HTTPS。
|
||||
实验版 HTTP/2 使用 `GO_OMNI_TRANSPORT=http2`,必须同时配置 TLS 证书和私钥。
|
||||
业务 Handler / Service / Repository 不需要修改;三种适配器共用上述退出等待时间。
|
||||
{{if eq .Database "mysql"}}
|
||||
`/health/ready` 已注册数据库探针:连接失败返回 `503`,数据库恢复后重新返回 `200`。
|
||||
`/health/live` 始终只检查服务存活。`GO_OMNI_DB_PING_TIMEOUT` 限制初始化检查和数据库探针的等待时间。
|
||||
|
||||
@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.5.0"
|
||||
const defaultFrameworkVersion = "v0.6.0-preview"
|
||||
|
||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||
|
||||
|
||||
@@ -4,23 +4,40 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
const defaultRequestIDHeader = "X-Request-ID"
|
||||
|
||||
const (
|
||||
TransportNetHTTP = "nethttp"
|
||||
TransportNative = "native"
|
||||
TransportHTTP2 = "http2"
|
||||
)
|
||||
|
||||
// TLSConfig enables HTTPS when both certificate paths are provided.
|
||||
type TLSConfig struct {
|
||||
CertFile string `json:"certFile"`
|
||||
KeyFile string `json:"keyFile"`
|
||||
}
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Address string
|
||||
RequestIDHeader string
|
||||
EnableLogger bool
|
||||
ShutdownTimeout time.Duration
|
||||
Log logx.Config
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
return Config{
|
||||
Address: ":8080",
|
||||
Transport: TransportNetHTTP,
|
||||
RequestIDHeader: defaultRequestIDHeader,
|
||||
ShutdownTimeout: 10 * time.Second,
|
||||
Log: logx.Default(),
|
||||
@@ -38,6 +55,27 @@ func (c Config) Validate() error {
|
||||
if c.ShutdownTimeout <= 0 {
|
||||
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
|
||||
}
|
||||
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
|
||||
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
|
||||
}
|
||||
return c.ValidateTransport()
|
||||
}
|
||||
|
||||
// ValidateTransport checks built-in transport selection without opening files
|
||||
// or sockets. Empty Transport preserves the standard net/http default.
|
||||
func (c Config) ValidateTransport() error {
|
||||
switch c.Transport {
|
||||
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
|
||||
default:
|
||||
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
|
||||
}
|
||||
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
|
||||
if (cert == "") != (key == "") {
|
||||
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}
|
||||
}
|
||||
if c.Transport == TransportHTTP2 && cert == "" {
|
||||
return &ConfigError{Field: "tls", Message: "is required for the experimental http2 transport"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -52,6 +90,16 @@ func (e *ConfigError) Error() string { return "config " + e.Field + " " + e.Mess
|
||||
// Option changes an App's configuration.
|
||||
type Option func(*Config)
|
||||
|
||||
// WithTransport selects a built-in server. HTTP/2 is experimental and requires TLS.
|
||||
func WithTransport(name string) Option {
|
||||
return func(cfg *Config) { cfg.Transport = name }
|
||||
}
|
||||
|
||||
// WithTLS enables HTTPS. Certificates are loaded before Run opens its listener.
|
||||
func WithTLS(certFile, keyFile string) Option {
|
||||
return func(cfg *Config) { cfg.TLS = TLSConfig{CertFile: certFile, KeyFile: keyFile} }
|
||||
}
|
||||
|
||||
// WithRequestIDHeader changes the header used to read and return request IDs.
|
||||
func WithRequestIDHeader(header string) Option {
|
||||
return func(cfg *Config) {
|
||||
@@ -66,10 +114,22 @@ func WithLogger(enabled bool) Option {
|
||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||
}
|
||||
|
||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||
// by Config.Validate and panic when passed directly to App.New.
|
||||
func WithTrustedProxies(proxies ...string) Option {
|
||||
proxies = append([]string(nil), proxies...)
|
||||
return func(cfg *Config) { cfg.TrustedProxies = append([]string(nil), proxies...) }
|
||||
}
|
||||
|
||||
// WithConfig applies a complete configuration loaded from a file or
|
||||
// environment. Later options can still override individual fields.
|
||||
func WithConfig(settings Config) Option {
|
||||
return func(cfg *Config) { *cfg = settings }
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
return func(cfg *Config) {
|
||||
*cfg = settings
|
||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithLogConfig configures the logger used by the application runtime.
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
for _, key := range []string{
|
||||
"GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_TRUSTED_PROXIES",
|
||||
"GO_OMNI_TRANSPORT", "GO_OMNI_TLS_CERT_FILE", "GO_OMNI_TLS_KEY_FILE",
|
||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||
@@ -20,7 +22,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != ":8080" || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
if cfg.Address != ":8080" || cfg.Transport != TransportNetHTTP || cfg.TLS != (TLSConfig{}) || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.EnableLogger || cfg.Log.Level != "info" || cfg.Log.Format != "json" || cfg.Log.Output != "stderr" {
|
||||
@@ -28,9 +30,48 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTransportEnvironment(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport, cert, key, errorField string
|
||||
}{
|
||||
{"default", "", "", "", ""},
|
||||
{"native", "native", "", "", ""},
|
||||
{"nativeTLS", "native", "server.crt", "server.key", ""},
|
||||
{"http2", "http2", "server.crt", "server.key", ""},
|
||||
{"unknown", "typo", "", "", "transport"},
|
||||
{"certOnly", "nethttp", "server.crt", "", "tls"},
|
||||
{"keyOnly", "native", "", "server.key", "tls"},
|
||||
{"http2WithoutTLS", "http2", "", "", "tls"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("TRANSPORT_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_CERT_FILE", test.cert)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_KEY_FILE", test.key)
|
||||
cfg, err := LoadFromEnv("TRANSPORT_TEST_")
|
||||
if test.errorField != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorField) {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := test.transport
|
||||
if want == "" {
|
||||
want = TransportNetHTTP
|
||||
}
|
||||
if cfg.Transport != want || cfg.TLS.CertFile != test.cert || cfg.TLS.KeyFile != test.key {
|
||||
t.Fatalf("settings: %+v", cfg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:9090")
|
||||
t.Setenv("GO_OMNI_REQUEST_ID_HEADER", "X-Correlation-ID")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "127.0.0.1, 10.0.0.0/8")
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "true")
|
||||
t.Setenv("GO_OMNI_LOG_LEVEL", "debug")
|
||||
t.Setenv("GO_OMNI_LOG_FORMAT", "text")
|
||||
@@ -44,7 +85,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger {
|
||||
if cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger || len(cfg.TrustedProxies) != 2 || cfg.TrustedProxies[1] != "10.0.0.0/8" {
|
||||
t.Fatalf("unexpected application settings: %+v", cfg)
|
||||
}
|
||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||
@@ -65,6 +106,11 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
}
|
||||
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
|
||||
t.Fatalf("expected invalid trusted proxy error, got %v", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
|
||||
cfg := Default()
|
||||
cfg.Address = ""
|
||||
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "address") {
|
||||
@@ -80,11 +126,16 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
settings := Default()
|
||||
settings.Log.Service = "orders"
|
||||
settings.TrustedProxies = []string{"127.0.0.1"}
|
||||
cfg := Default()
|
||||
WithConfig(settings)(&cfg)
|
||||
if cfg.Log.Service != "orders" {
|
||||
if cfg.Log.Service != "orders" || len(cfg.TrustedProxies) != 1 {
|
||||
t.Fatalf("complete config was not applied: %+v", cfg)
|
||||
}
|
||||
settings.TrustedProxies[0] = "10.0.0.0/8"
|
||||
if cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("WithConfig should copy trusted proxy settings")
|
||||
}
|
||||
|
||||
WithLogConfig(Default().Log)(&cfg)
|
||||
if cfg.Log.Service != "" {
|
||||
|
||||
@@ -13,13 +13,25 @@ const defaultEnvPrefix = "GO_OMNI_"
|
||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||
// database settings remain owned by database.ConfigFromEnv.
|
||||
func LoadFromEnv(prefix string) (Config, error) {
|
||||
return applyEnv(Default(), prefix)
|
||||
}
|
||||
|
||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
cfg := Default()
|
||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||
cfg.TrustedProxies = strings.Split(value, ",")
|
||||
for i := range cfg.TrustedProxies {
|
||||
cfg.TrustedProxies[i] = strings.TrimSpace(cfg.TrustedProxies[i])
|
||||
}
|
||||
}
|
||||
cfg.Log.Level = envString(prefix+"LOG_LEVEL", cfg.Log.Level)
|
||||
cfg.Log.Format = envString(prefix+"LOG_FORMAT", cfg.Log.Format)
|
||||
cfg.Log.Output = envString(prefix+"LOG_OUTPUT", cfg.Log.Output)
|
||||
|
||||
203
config/file.go
Normal file
203
config/file.go
Normal file
@@ -0,0 +1,203 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gopkg.in/yaml.v3"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if len(data) > 1<<20 {
|
||||
return zero, errors.New("config: file exceeds 1 MiB")
|
||||
}
|
||||
var value any
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".yaml", ".yml":
|
||||
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one YAML document required")
|
||||
}
|
||||
case ".json":
|
||||
dec := json.NewDecoder(bytes.NewReader(data))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one JSON document required")
|
||||
}
|
||||
default:
|
||||
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||
}
|
||||
if _, ok := value.(map[string]any); !ok {
|
||||
return zero, errors.New("config: root must be an object")
|
||||
}
|
||||
value, err = durations(value, reflect.TypeOf(defaults))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||
original, err := json.Marshal(defaults)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var result T
|
||||
if err = json.Unmarshal(original, &result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func durations(v any, t reflect.Type) (any, error) {
|
||||
if t == nil {
|
||||
return v, nil
|
||||
}
|
||||
for t.Kind() == reflect.Pointer {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
if s, ok := v.(string); ok {
|
||||
d, err := time.ParseDuration(s)
|
||||
return int64(d), err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
if t.Kind() == reflect.Struct {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
field := t.Field(n)
|
||||
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = field.Name
|
||||
}
|
||||
if item, exists := m[name]; exists {
|
||||
converted, err := durations(item, field.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m[name] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
converted, err := durations(item, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items[n] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func LoadFile(path, prefix string) (Config, error) {
|
||||
cfg, err := DecodeFile(path, Default())
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return applyEnv(cfg, prefix)
|
||||
}
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||
type Reloader struct {
|
||||
mu sync.Mutex
|
||||
path, prefix string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
}
|
||||
|
||||
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path, prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||
}
|
||||
func (r *Reloader) Reload() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
next, err := LoadFile(r.path, r.prefix)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
comparable := next
|
||||
comparable.Log.Level = r.current.Log.Level
|
||||
if !reflect.DeepEqual(comparable, r.current) {
|
||||
return errors.New("config: static fields changed; restart required")
|
||||
}
|
||||
if err = r.logger.SetLevel(next.Log.Level); err != nil {
|
||||
return err
|
||||
}
|
||||
r.current = next
|
||||
return nil
|
||||
}
|
||||
func (r *Reloader) Snapshot() Config {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
cfg := r.current
|
||||
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||
return cfg
|
||||
}
|
||||
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||
if ctx == nil || interval < 100*time.Millisecond {
|
||||
return errors.New("config: context and interval >=100ms required")
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
if err := r.Reload(); err != nil && onError != nil {
|
||||
onError(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
59
config/file_test.go
Normal file
59
config/file_test.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "service.yaml")
|
||||
write := func(raw string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
lc := cfg.Log
|
||||
lc.Writer = &output
|
||||
l, err := logx.New(lc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
child := l.With("scope", "test")
|
||||
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("hidden")
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: debug\n")
|
||||
if err = r.Reload(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("visible")
|
||||
if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
|
||||
t.Fatal(output.String())
|
||||
}
|
||||
write("shutdownTimeout: 8s\nlog:\n level: error\n")
|
||||
if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
|
||||
t.Fatal("static reload changed live config", err)
|
||||
}
|
||||
for _, raw := range []string{"unknownOption: true", "log:\n level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
|
||||
write(raw)
|
||||
if err = r.Reload(); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
}
|
||||
245
database/migrate/migrate.go
Normal file
245
database/migrate/migrate.go
Normal file
@@ -0,0 +1,245 @@
|
||||
// Package migrate provides explicit MySQL schema migration jobs. Application
|
||||
// startup never runs migrations automatically; use a separate deployment step.
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
mysqlDriver "github.com/go-sql-driver/mysql"
|
||||
migration "github.com/golang-migrate/migrate/v4"
|
||||
migrateMySQL "github.com/golang-migrate/migrate/v4/database/mysql"
|
||||
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||
"io"
|
||||
"io/fs"
|
||||
"path"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type File struct {
|
||||
Version uint64 `json:"version"`
|
||||
Name string `json:"name"`
|
||||
Direction string `json:"direction"`
|
||||
SHA256 string `json:"sha256"`
|
||||
}
|
||||
|
||||
var filename = regexp.MustCompile(`^([0-9]+)_([A-Za-z][A-Za-z0-9]*)\.(up|down)\.sql$`)
|
||||
|
||||
// Snapshot returns immutable migration bytes and a reviewable digest manifest.
|
||||
// Follow lowerCamelCase names, e.g. 0001_createUsers.up.sql.
|
||||
func Snapshot(source fs.FS, directory string) (fs.FS, []File, error) {
|
||||
if source == nil || !fs.ValidPath(directory) {
|
||||
return nil, nil, errors.New("migrate: invalid source directory")
|
||||
}
|
||||
entries, err := fs.ReadDir(source, directory)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var archive bytes.Buffer
|
||||
writer := zip.NewWriter(&archive)
|
||||
manifest := []File{}
|
||||
seen := map[string]bool{}
|
||||
total := 0
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
parts := filename.FindStringSubmatch(entry.Name())
|
||||
if parts == nil {
|
||||
if strings.HasSuffix(entry.Name(), ".sql") {
|
||||
return nil, nil, fmt.Errorf("migrate: invalid migration filename %s", entry.Name())
|
||||
}
|
||||
continue
|
||||
}
|
||||
if len(manifest) >= 1000 {
|
||||
return nil, nil, errors.New("migrate: too many files")
|
||||
}
|
||||
version, err := strconv.ParseUint(parts[1], 10, 32)
|
||||
if err != nil || version == 0 {
|
||||
return nil, nil, errors.New("migrate: version must be 1..4294967295")
|
||||
}
|
||||
identity := fmt.Sprint(version) + parts[3]
|
||||
if seen[identity] {
|
||||
return nil, nil, errors.New("migrate: duplicate migration version/direction")
|
||||
}
|
||||
seen[identity] = true
|
||||
file, err := source.Open(path.Join(directory, entry.Name()))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||
closeErr := file.Close()
|
||||
if err == nil {
|
||||
err = closeErr
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
total += len(data)
|
||||
if len(data) == 0 || len(data) > 1<<20 || total > 16<<20 {
|
||||
return nil, nil, errors.New("migrate: files must be 1..1MiB and <=16MiB total")
|
||||
}
|
||||
sum := sha256.Sum256(data)
|
||||
manifest = append(manifest, File{Version: version, Name: parts[2], Direction: parts[3], SHA256: hex.EncodeToString(sum[:])})
|
||||
fileWriter, err := writer.CreateHeader(&zip.FileHeader{Name: entry.Name(), Method: zip.Store})
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if _, err = fileWriter.Write(data); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
}
|
||||
if len(manifest) == 0 {
|
||||
return nil, nil, errors.New("migrate: no SQL migrations")
|
||||
}
|
||||
for _, file := range manifest {
|
||||
if file.Direction == "down" && !seen[fmt.Sprint(file.Version)+"up"] {
|
||||
return nil, nil, errors.New("migrate: down file lacks up migration")
|
||||
}
|
||||
}
|
||||
sort.Slice(manifest, func(i, j int) bool {
|
||||
if manifest[i].Version == manifest[j].Version {
|
||||
return manifest[i].Direction < manifest[j].Direction
|
||||
}
|
||||
return manifest[i].Version < manifest[j].Version
|
||||
})
|
||||
if err = writer.Close(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
files, err := zip.NewReader(bytes.NewReader(archive.Bytes()), int64(archive.Len()))
|
||||
return files, manifest, err
|
||||
}
|
||||
|
||||
type Runner struct {
|
||||
mu sync.Mutex
|
||||
engine *migration.Migrate
|
||||
db *sql.DB
|
||||
closed bool
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func NewMySQL(ctx context.Context, dsn string, source fs.FS, directory string) (runner *Runner, err error) {
|
||||
if ctx == nil {
|
||||
return nil, errors.New("migrate: context required")
|
||||
}
|
||||
snapshot, _, err := Snapshot(source, directory)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cfg, err := mysqlDriver.ParseDSN(dsn)
|
||||
if err != nil || cfg.DBName == "" {
|
||||
return nil, errors.New("migrate: valid MySQL DSN with database required")
|
||||
}
|
||||
cfg.MultiStatements = true
|
||||
if cfg.Timeout == 0 || cfg.Timeout > 5*time.Second {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
if cfg.ReadTimeout == 0 || cfg.ReadTimeout > 30*time.Second {
|
||||
cfg.ReadTimeout = 30 * time.Second
|
||||
}
|
||||
if cfg.WriteTimeout == 0 || cfg.WriteTimeout > 30*time.Second {
|
||||
cfg.WriteTimeout = 30 * time.Second
|
||||
}
|
||||
connector, err := mysqlDriver.NewConnector(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db := sql.OpenDB(connector)
|
||||
db.SetMaxOpenConns(1)
|
||||
db.SetMaxIdleConns(1)
|
||||
defer func() {
|
||||
if err != nil {
|
||||
db.Close()
|
||||
}
|
||||
}()
|
||||
conn, err := db.Conn(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
driver, err := migrateMySQL.WithConnection(ctx, conn, &migrateMySQL.Config{DatabaseName: cfg.DBName, StatementTimeout: 30 * time.Second})
|
||||
if err != nil {
|
||||
conn.Close()
|
||||
return nil, err
|
||||
}
|
||||
src, err := iofs.New(snapshot, ".")
|
||||
if err != nil {
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine, err := migration.NewWithInstance("iofs", src, cfg.DBName, driver)
|
||||
if err != nil {
|
||||
src.Close()
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine.LockTimeout = 10 * time.Second
|
||||
return &Runner{engine: engine, db: db}, nil
|
||||
}
|
||||
|
||||
// Up applies pending up scripts. MySQL DDL can partially commit; a failed
|
||||
// migration remains dirty and requires operator investigation. No automatic
|
||||
// down/force/dirty reset is provided. Cancellation stops after the current SQL
|
||||
// operation; statements/read I/O are bounded to 30 seconds.
|
||||
func (r *Runner) Up(ctx context.Context) error {
|
||||
if ctx == nil {
|
||||
return errors.New("migrate: context required")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed || r.stopped {
|
||||
return errors.New("migrate: closed")
|
||||
}
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- r.engine.Up() }()
|
||||
var err error
|
||||
select {
|
||||
case err = <-done:
|
||||
case <-ctx.Done():
|
||||
r.stopped = true // golang-migrate retains its graceful-stop state; create a fresh runner to retry.
|
||||
select {
|
||||
case r.engine.GracefulStop <- true:
|
||||
default:
|
||||
}
|
||||
err = <-done
|
||||
return errors.Join(ctx.Err(), err)
|
||||
}
|
||||
if errors.Is(err, migration.ErrNoChange) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
func (r *Runner) Version() (uint, bool, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return 0, false, errors.New("migrate: closed")
|
||||
}
|
||||
version, dirty, err := r.engine.Version()
|
||||
if errors.Is(err, migration.ErrNilVersion) {
|
||||
return 0, false, nil
|
||||
}
|
||||
return version, dirty, err
|
||||
}
|
||||
func (r *Runner) Close() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return nil
|
||||
}
|
||||
r.closed = true
|
||||
a, b := r.engine.Close()
|
||||
return errors.Join(a, b, r.db.Close())
|
||||
}
|
||||
99
database/migrate/migrate_test.go
Normal file
99
database/migrate/migrate_test.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"go.opentelemetry.io/otel/sdk/trace/tracetest"
|
||||
"io/fs"
|
||||
"os"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSnapshotBoundsAndIdentity(t *testing.T) {
|
||||
source := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("SELECT 1;")}}
|
||||
f, manifest, err := Snapshot(source, ".")
|
||||
if err != nil || len(manifest) != 1 {
|
||||
t.Fatal(manifest, err)
|
||||
}
|
||||
source["0001_createItems.up.sql"].Data[0] = 'x'
|
||||
raw, err := fs.ReadFile(f, "0001_createItems.up.sql")
|
||||
if err != nil || string(raw) != "SELECT 1;" {
|
||||
t.Fatal(string(raw), err)
|
||||
}
|
||||
for _, files := range []fstest.MapFS{{"0001_bad_name.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: []byte("x")}, "1_b.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.down.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: make([]byte, (1<<20)+1)}}} {
|
||||
if _, _, err = Snapshot(files, "."); err == nil {
|
||||
t.Fatal("invalid migration accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestMySQLIntegration(t *testing.T) {
|
||||
dsn := os.Getenv("GO_OMNI_MIGRATE_TEST_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL database")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
files := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("CREATE TABLE testItems (id BIGINT NOT NULL PRIMARY KEY, value VARCHAR(64) NOT NULL);")}, "0002_seedItems.up.sql": &fstest.MapFile{Data: []byte("INSERT INTO testItems (id,value) VALUES (1,'original');")}}
|
||||
runner, err := NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal("repeat", err)
|
||||
}
|
||||
v, dirty, err := runner.Version()
|
||||
if err != nil || v != 2 || dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
runner.Close()
|
||||
db, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
exporter := tracetest.NewInMemoryExporter()
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "dbTest", SampleRatio: 1, Exporter: exporter})
|
||||
defer tr.Close()
|
||||
instrumented := telemetry.SQL{DB: db, Runtime: tr}
|
||||
if _, err = instrumented.ExecContext(ctx, "UPDATE testItems SET value=? WHERE id=1", "secret parameter"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rows, err := instrumented.QueryContext(ctx, "SELECT value FROM testItems WHERE id=1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !rows.Next() {
|
||||
t.Fatal("missing row")
|
||||
}
|
||||
var value string
|
||||
if err = rows.Scan(&value); err != nil || value != "secret parameter" {
|
||||
t.Fatal(value, err)
|
||||
}
|
||||
rows.Close()
|
||||
tr.Flush(ctx)
|
||||
if len(exporter.GetSpans()) != 2 {
|
||||
t.Fatal("SQL spans missing")
|
||||
}
|
||||
files["0003_failMigration.up.sql"] = &fstest.MapFile{Data: []byte("INSERT INTO missingTable VALUES (1);")}
|
||||
runner, err = NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("bad migration passed")
|
||||
}
|
||||
v, dirty, err = runner.Version()
|
||||
if err != nil || v != 3 || !dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("dirty migration auto-cleared")
|
||||
}
|
||||
}
|
||||
33
examples/modules/README.md
Normal file
33
examples/modules/README.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# 基础模块组合示例
|
||||
|
||||
在仓库根目录执行:
|
||||
|
||||
```sh
|
||||
go run ./examples/modules
|
||||
```
|
||||
|
||||
默认监听 `127.0.0.1:8080`,使用仅限开发的 `Bearer dev-token`。
|
||||
配置 `GO_OMNI_TRANSPORT=native` 可直接切换自研网络层,TLS 配置与主框架一致。
|
||||
|
||||
```sh
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/catalog
|
||||
curl -H 'Authorization: Bearer dev-token' -H 'Content-Type: application/json' \
|
||||
-H 'Idempotency-Key: report-1' -d '{"name":"report"}' http://127.0.0.1:8080/api/jobs
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/jobs
|
||||
```
|
||||
|
||||
catalog 缓存 30 秒;可通过启动环境变量 `GO_OMNI_DEMO_UPSTREAM` 指定可信 JSON 上游,
|
||||
演示 HTTP 连接池、请求 ID 传递与合并回源。URL 来自服务配置,不接受请求参数传入。
|
||||
|
||||
jobs 展示固定 worker、队列、结构化日志与状态查询;相同 Idempotency-Key 和请求内容重放
|
||||
第一次 202 响应,内容不同返回 409。每分钟的统计任务演示间隔调度。退出先排空任务,再关闭
|
||||
HTTP 客户端和缓存。这个示例的任务、缓存和幂等记录均保存在进程内。
|
||||
|
||||
组合测试通过真实 TLS 连接覆盖 nethttp、native 和实验 HTTP/2,检查缓存回源次数、
|
||||
幂等重放、认证不可被重放绕过、任务只提交一次与关闭流程:
|
||||
|
||||
```sh
|
||||
go test -race ./examples/modules
|
||||
```
|
||||
|
||||
完整 API、默认上限和适用范围见 [业务基础模块](../../MODULES.md)。
|
||||
133
examples/modules/main.go
Normal file
133
examples/modules/main.go
Normal file
@@ -0,0 +1,133 @@
|
||||
// modules combines caching, outbound calls, middleware and background jobs.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
"git.yuntuo.hk/k2safe/go-omni/task"
|
||||
)
|
||||
|
||||
func newApp(settings omni.Config, token, upstream string) (application *omni.App, err error) {
|
||||
application = omni.New(omni.WithConfig(settings))
|
||||
defer func(app *omni.App) {
|
||||
if err != nil {
|
||||
_ = app.Close()
|
||||
}
|
||||
}(application)
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 128, MaxBytes: 1 << 20})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 8, Timeout: time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
client, err := httpc.New(httpc.Config{Timeout: time.Second, MaxRetries: 1, MaxConnsPerHost: 8})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
application.OnShutdown(client.Close)
|
||||
jobs, err := task.New(task.Config{Workers: 2, QueueSize: 16, TaskTimeout: 2 * time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
// Reverse cleanup order: drain jobs before closing their dependencies.
|
||||
application.OnShutdown(jobs.Close)
|
||||
if err = jobs.Every("reportStats", time.Minute, func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "jobs.stats", "stats", jobs.Stats())
|
||||
return nil
|
||||
}); err != nil {
|
||||
return application, err
|
||||
}
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40, MaxKeys: 128, Key: func(c *omni.Context) string { p, _ := auth.FromContext(c); return p.UserID }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Scope: func(c *omni.Context) (string, error) { p, _ := auth.FromContext(c); return p.UserID, nil }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
api := application.Group("/api", middleware.Timeout(2*time.Second), auth.RequireAuth(auth.StaticAuthenticator{token: {UserID: "demo-user"}}), limiter.Middleware())
|
||||
api.GET("/catalog", func(c *omni.Context) error {
|
||||
value, err := loader.GetOrLoad(c.Context(), "publicCatalog:v1", 30*time.Second, func(ctx context.Context) ([]byte, error) {
|
||||
data := map[string]any{"name": "demo catalog"}
|
||||
if upstream != "" {
|
||||
if err := client.JSON(ctx, "GET", upstream, nil, &data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return json.Marshal(data)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.OK(json.RawMessage(value))
|
||||
})
|
||||
var nextID atomic.Uint64
|
||||
api.POST("/jobs", func(c *omni.Context) error {
|
||||
var input struct {
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
}
|
||||
if err := c.BindJSON(&input); err != nil {
|
||||
return err
|
||||
}
|
||||
id := nextID.Add(1)
|
||||
// This job deliberately outlives the HTTP request. Runner still applies
|
||||
// its own deadline and shutdown cancellation; log correlation survives.
|
||||
_, err := jobs.Submit(context.WithoutCancel(c.Context()), "demoJob", func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "job.finished", "jobId", id, "name", input.Name)
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, task.ErrQueueFull) {
|
||||
return omni.NewError(503, "QUEUE_FULL", "job queue is full")
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Accepted(map[string]uint64{"jobId": id})
|
||||
}, idempotency.Middleware())
|
||||
api.GET("/jobs", func(c *omni.Context) error { return c.OK(jobs.Stats()) })
|
||||
return application, nil
|
||||
}
|
||||
|
||||
func run(ctx context.Context) error {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEV_TOKEN")
|
||||
if token == "" {
|
||||
token = "dev-token"
|
||||
}
|
||||
application, err := newApp(settings, token, os.Getenv("GO_OMNI_DEMO_UPSTREAM"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func main() {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
126
examples/modules/main_test.go
Normal file
126
examples/modules/main_test.go
Normal file
@@ -0,0 +1,126 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// Exercise the composed feature chain through real sockets and all runners.
|
||||
func TestModulesConfiguredTransports(t *testing.T) {
|
||||
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
var loads atomic.Int32
|
||||
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
loads.Add(1)
|
||||
if r.Header.Get("X-Request-ID") == "" {
|
||||
t.Error("missing outbound correlation")
|
||||
}
|
||||
io.WriteString(w, `{"name":"upstream catalog"}`)
|
||||
}))
|
||||
defer upstream.Close()
|
||||
settings, err := omni.LoadConfig("MODULES_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
settings.Transport = transport
|
||||
settings.TLS.CertFile = cert
|
||||
settings.TLS.KeyFile = key
|
||||
application, err := newApp(settings, "demo", upstream.URL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("server failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, key, requestID, token string) (*http.Response, string) {
|
||||
r, err := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("Idempotency-Key", key)
|
||||
r.Header.Set("X-Request-ID", requestID)
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
data, err := io.ReadAll(response.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("unverified TLS")
|
||||
}
|
||||
if transport == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal(response.Proto)
|
||||
}
|
||||
return response, string(data)
|
||||
}
|
||||
for n := 0; n < 2; n++ {
|
||||
response, body := request("GET", "/api/catalog", "", "", "catalogRequest", "demo")
|
||||
if response.StatusCode != 200 || !strings.Contains(body, "upstream catalog") {
|
||||
t.Fatal(response.StatusCode, body)
|
||||
}
|
||||
}
|
||||
if loads.Load() != 1 {
|
||||
t.Fatal("cache missed", loads.Load())
|
||||
}
|
||||
first, original := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "firstRequest", "demo")
|
||||
second, replayed := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "secondRequest", "demo")
|
||||
if first.StatusCode != 202 || second.StatusCode != 202 || original != replayed || second.Header.Get("Idempotency-Replayed") != "true" || second.Header.Get("X-Request-ID") != "firstRequest" {
|
||||
t.Fatal(first.StatusCode, second.StatusCode, original, replayed)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"other"}`, "job1", "conflictRequest", "demo"); response.StatusCode != 409 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "unauthorized", "wrong"); response.StatusCode != 401 {
|
||||
t.Fatal("replay bypassed authentication", response.StatusCode)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
response, body := request("GET", "/api/jobs", "", "", "statsRequest", "demo")
|
||||
if response.StatusCode != 200 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if strings.Contains(body, `"completed":1`) {
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("job did not finish once", body)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
14
examples/platform/README.md
Normal file
14
examples/platform/README.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# 扩展模块组合示例
|
||||
|
||||
业务通过 `storage.Store` 写文件,身份接口和 Policy 控制权限,OpenAPI 与路由一起注册,链路追踪关联 HTTP 与文件操作。`newApp` 不依赖 Local/S3/OSS 类型;启动处本例选择 Local。可替换成 S3/R2/OSS 构造结果,业务路由不变。
|
||||
|
||||
```sh
|
||||
export GO_OMNI_DEMO_TOKEN='use-a-private-development-token'
|
||||
go run ./examples/platform --root /tmp/goOmniDemoFiles
|
||||
```
|
||||
|
||||
接口:`POST /api/files`,JSON 为 `{"name":"hello.txt","content":"hello"}`,请求加 `Authorization: Bearer <GO_OMNI_DEMO_TOKEN>`;返回 201 和存储信息。`GET /openapi.json` 同样要求身份。
|
||||
|
||||
也可传 `--config service.yaml`,配置规则见 [扩展模块](../../ADVANCED.md)。本例的静态身份只用于演示,实际应用注入 `auth.JWT` 或 `auth.Sessions`;trace provider 默认不外发,接入 collector 时注入 exporter/reader。
|
||||
|
||||
`TestPlatformTLSAdapters` 通过三个 Runner 的真实 TLS 请求验证 401/403/201、文件内容、非法 key、OpenAPI 和关闭。文件根目录必须是应用私有目录。示例不执行自动数据迁移,也不启动 Orbit。
|
||||
132
examples/platform/main.go
Normal file
132
examples/platform/main.go
Normal file
@@ -0,0 +1,132 @@
|
||||
// platform composes storage, authentication, permission policy, tracing and
|
||||
// OpenAPI using provider-neutral business dependencies.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
)
|
||||
|
||||
type dependencies struct {
|
||||
Files storage.Store
|
||||
Auth auth.Authenticator
|
||||
Tracing *telemetry.Runtime
|
||||
}
|
||||
type upload struct {
|
||||
Name string `json:"name" binding:"required,max=128"`
|
||||
Content string `json:"content" binding:"max=4096"`
|
||||
}
|
||||
|
||||
// Dependencies are borrowed; the composition root owns their shutdown.
|
||||
func newApp(cfg omni.Config, d dependencies) (*omni.App, error) {
|
||||
if d.Files == nil || d.Auth == nil || d.Tracing == nil {
|
||||
return nil, errors.New("platform: dependencies required")
|
||||
}
|
||||
app := omni.New(omni.WithConfig(cfg))
|
||||
app.Use(d.Tracing.Middleware())
|
||||
api, err := openapi.New("File service", "1")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input, err := openapi.SchemaOf[upload]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
output, err := openapi.SchemaOf[storage.Info]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
policy := auth.PolicyFunc(func(_ context.Context, p auth.Principal, action, resource string) error {
|
||||
if p.TenantID != "" && p.HasRole("writer") && action == "create" && resource == "files" {
|
||||
return nil
|
||||
}
|
||||
return errors.New("denied")
|
||||
})
|
||||
err = api.Handle(app, openapi.Operation{Method: "POST", Path: "/api/files", OperationID: "createFile", Summary: "Store a small text file", Request: input, BearerAuth: true, Responses: map[int]openapi.Response{201: {Description: "Created", Schema: openapi.Envelope(output)}}}, func(c *omni.Context) error {
|
||||
var in upload
|
||||
if err := c.BindJSON(&in); err != nil {
|
||||
return err
|
||||
}
|
||||
p, _ := auth.FromContext(c)
|
||||
if strings.ContainsAny(in.Name, "/\\") {
|
||||
return omni.BadRequest("INVALID_NAME", "name must be one path segment")
|
||||
}
|
||||
key := p.TenantID + "/" + in.Name
|
||||
if err := storage.Check(c.Context(), key); err != nil {
|
||||
return omni.BadRequest("INVALID_KEY", "invalid storage key")
|
||||
}
|
||||
var info storage.Info
|
||||
err := d.Tracing.Do(c.Context(), "files.put", func(ctx context.Context) error {
|
||||
var err error
|
||||
info, err = d.Files.Put(ctx, key, strings.NewReader(in.Content), int64(len(in.Content)), "text/plain; charset=utf-8")
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Created(info)
|
||||
}, auth.RequireAuth(d.Auth), auth.RequirePermission(policy, "create", func(*omni.Context) string { return "files" }))
|
||||
if err != nil {
|
||||
app.Close()
|
||||
return nil, err
|
||||
}
|
||||
app.GET("/openapi.json", api.Handler(), auth.RequireAuth(d.Auth))
|
||||
return app, nil
|
||||
}
|
||||
func run(ctx context.Context, configFile, root string) error {
|
||||
var cfg config.Config
|
||||
var err error
|
||||
if configFile != "" {
|
||||
cfg, err = config.LoadFile(configFile, "")
|
||||
} else {
|
||||
cfg, err = config.LoadFromEnv("")
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
cfg.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEMO_TOKEN")
|
||||
if token == "" {
|
||||
return errors.New("set GO_OMNI_DEMO_TOKEN for the demonstration credential")
|
||||
}
|
||||
files, err := local.New(root, 64<<10)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tracing, err := telemetry.New(telemetry.Config{Service: "fileService", SampleRatio: 1})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tracing.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: files, Auth: auth.StaticAuthenticator{token: {UserID: "demoUser", TenantID: "demoTenant", Roles: []string{"writer"}}}, Tracing: tracing})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return app.RunContext(ctx)
|
||||
}
|
||||
func main() {
|
||||
file := flag.String("config", "", "optional JSON/YAML config")
|
||||
root := flag.String("root", "./fileData", "private local object directory")
|
||||
flag.Parse()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx, *file, *root); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
102
examples/platform/main_test.go
Normal file
102
examples/platform/main_test.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestPlatformTLSAdapters(t *testing.T) {
|
||||
for _, name := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.Transport = name
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
cfg.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
store, _ := local.New(t.TempDir(), 64<<10)
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "platformTest", SampleRatio: 1})
|
||||
defer tr.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: store, Auth: auth.StaticAuthenticator{"writer": {UserID: "u", TenantID: "t", Roles: []string{"writer"}}, "reader": {UserID: "r", TenantID: "t"}}, Tracing: tr})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- app.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("shutdown timeout")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, token string) (int, string) {
|
||||
t.Helper()
|
||||
r, _ := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, err := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("TLS unverified")
|
||||
}
|
||||
if name == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal("protocol")
|
||||
}
|
||||
return response.StatusCode, string(raw)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
token string
|
||||
status int
|
||||
}{{"", 401}, {"reader", 403}, {"writer", 201}} {
|
||||
status, body := request("POST", "/api/files", `{"name":"hello.txt","content":"hello"}`, test.token)
|
||||
if status != test.status {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
}
|
||||
object, err := store.Get(context.Background(), "t/hello.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := io.ReadAll(object.Body)
|
||||
object.Body.Close()
|
||||
if string(raw) != "hello" {
|
||||
t.Fatal(string(raw))
|
||||
}
|
||||
if status, _ := request("POST", "/api/files", `{"name":"../escape","content":"x"}`, "writer"); status != 400 {
|
||||
t.Fatal(status)
|
||||
}
|
||||
if status, body := request("GET", "/openapi.json", "", "writer"); status != 200 || !strings.Contains(body, `"/api/files"`) {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
@@ -41,28 +42,52 @@ func registerRoutes(application *omni.App, users service.UserRepository, logger
|
||||
}
|
||||
|
||||
func main() {
|
||||
logger, err := logx.New(logx.Config{
|
||||
Service: "reference-api",
|
||||
Environment: "local",
|
||||
Format: "json",
|
||||
Writer: os.Stdout,
|
||||
})
|
||||
if err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
defer logger.Close()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := runReference(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func referenceConfig() (omni.Config, error) {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return settings, err
|
||||
}
|
||||
// Preserve the example's loopback default and legacy address variable.
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_HTTP_ADDR")) == "" {
|
||||
settings.Address = strings.TrimSpace(os.Getenv("GO_OMNI_ADDR"))
|
||||
if settings.Address == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if settings.Log.Service == "" {
|
||||
settings.Log.Service = "reference-api"
|
||||
}
|
||||
if settings.Log.Environment == "" {
|
||||
settings.Log.Environment = "local"
|
||||
}
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")) == "" {
|
||||
settings.Log.Output = "stdout"
|
||||
}
|
||||
return settings, nil
|
||||
}
|
||||
|
||||
func runReference(ctx context.Context) error {
|
||||
settings, err := referenceConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
logger, err := logx.New(settings.Log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer logger.Close()
|
||||
logx.SetDefault(logger)
|
||||
users, closeStorage, storage, err := openUsers(ctx, logger)
|
||||
if err != nil {
|
||||
logger.Error("storage initialization failed", "error", err)
|
||||
os.Exit(1)
|
||||
return err
|
||||
}
|
||||
defer func() {
|
||||
if err := closeStorage(); err != nil {
|
||||
logger.Error("storage close failed", "error", err)
|
||||
}
|
||||
}()
|
||||
devToken := strings.TrimSpace(os.Getenv("GO_OMNI_DEV_TOKEN"))
|
||||
if devToken == "" {
|
||||
devToken = "dev-token"
|
||||
@@ -70,16 +95,11 @@ func main() {
|
||||
authenticator := auth.StaticAuthenticator{
|
||||
devToken: {UserID: "dev-user", Roles: []string{"admin"}},
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator)
|
||||
address := os.Getenv("GO_OMNI_ADDR")
|
||||
if address == "" {
|
||||
address = "127.0.0.1:8080"
|
||||
}
|
||||
logger.Info("starting reference API", "address", address, "storage", storage)
|
||||
if err := application.RunContext(ctx, address); err != nil {
|
||||
logger.Error("server stopped", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(closeStorage)
|
||||
logger.Info("starting reference API", "address", settings.Address, "transport", settings.Transport, "tls", settings.TLS.CertFile != "", "storage", storage)
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func openUsers(ctx context.Context, logger *logx.Logger) (service.UserRepository, func() error, string, error) {
|
||||
|
||||
@@ -118,7 +118,7 @@ func TestUserCRUD(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func checkUserCRUD(t *testing.T, app *omni.App) {
|
||||
func checkUserCRUD(t *testing.T, app http.Handler) {
|
||||
t.Helper()
|
||||
health := dataAs[map[string]string](t, request(t, app, "GET", "/health", "", 200, "OK"))
|
||||
if health["status"] != "ok" {
|
||||
|
||||
182
examples/reference/transport_test.go
Normal file
182
examples/reference/transport_test.go
Normal file
@@ -0,0 +1,182 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/examples/reference/repository"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// networkHandler runs the same business assertions through a real TCP client.
|
||||
// The application handler is only called by its configured server transport.
|
||||
type networkHandler struct {
|
||||
t *testing.T
|
||||
client *http.Client
|
||||
address, scheme, token, alpn string
|
||||
proto int
|
||||
requests *int
|
||||
}
|
||||
|
||||
func (h networkHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
h.t.Helper()
|
||||
req := r.Clone(r.Context())
|
||||
req.RequestURI = ""
|
||||
req.URL.Scheme, req.URL.Host, req.Host = h.scheme, h.address, h.address
|
||||
if h.token != "" && req.Header.Get("Authorization") == "" {
|
||||
req.Header.Set("Authorization", "Bearer "+h.token)
|
||||
}
|
||||
response, err := h.client.Do(req)
|
||||
if err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if strings.HasPrefix(req.URL.Path, "/api/v1/") {
|
||||
*h.requests++
|
||||
}
|
||||
if response.ProtoMajor != h.proto {
|
||||
h.t.Fatalf("protocol %s, want HTTP/%d", response.Proto, h.proto)
|
||||
}
|
||||
if h.scheme == "https" && (response.TLS == nil || len(response.TLS.VerifiedChains) == 0 || response.TLS.NegotiatedProtocol != h.alpn) {
|
||||
h.t.Fatalf("TLS/ALPN verification failed: %+v", response.TLS)
|
||||
}
|
||||
for key, values := range response.Header {
|
||||
w.Header()[key] = append([]string(nil), values...)
|
||||
}
|
||||
w.WriteHeader(response.StatusCode)
|
||||
if _, err := io.Copy(w, response.Body); err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
type lockedLog struct {
|
||||
sync.Mutex
|
||||
bytes.Buffer
|
||||
}
|
||||
|
||||
func (b *lockedLog) Write(p []byte) (int, error) {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
|
||||
func (b *lockedLog) snapshot() string {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.String()
|
||||
}
|
||||
|
||||
func TestBusinessConfiguredTransports(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport string
|
||||
tls bool
|
||||
proto int
|
||||
alpn string
|
||||
}{
|
||||
{"nethttp", omni.TransportNetHTTP, false, 1, ""},
|
||||
{"native", omni.TransportNative, false, 1, ""},
|
||||
{"nethttpTLS", omni.TransportNetHTTP, true, 2, "h2"},
|
||||
{"nativeTLS", omni.TransportNative, true, 1, "http/1.1"},
|
||||
{"http2TLS", omni.TransportHTTP2, true, 2, "h2"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
t.Setenv("REFERENCE_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("REFERENCE_TEST_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", "")
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", "")
|
||||
if test.tls {
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", cert)
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", key)
|
||||
}
|
||||
settings, err := omni.LoadConfig("REFERENCE_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var logs lockedLog
|
||||
logger := testLogger(&logs)
|
||||
application := newProtectedApp(repository.NewMemoryUsers(), logger, auth.StaticAuthenticator{
|
||||
"admin": {UserID: "admin", Roles: []string{"admin"}},
|
||||
"user": {UserID: "user", Roles: []string{"user"}},
|
||||
}, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
t.Cleanup(func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("business server did not stop")
|
||||
listener.Close()
|
||||
}
|
||||
})
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
scheme := "http"
|
||||
if test.tls {
|
||||
scheme = "https"
|
||||
}
|
||||
requests := 0
|
||||
network := networkHandler{t: t, client: client, address: listener.Addr().String(), scheme: scheme, proto: test.proto, alpn: test.alpn, requests: &requests}
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":"Alice","email":"alice@example.com"}`, 401, "UNAUTHENTICATED")
|
||||
requestWithAuthorization(t, network, "DELETE", "/api/v1/users/1", "", "Bearer user", 403, "FORBIDDEN")
|
||||
network.token = "admin"
|
||||
checkUserCRUD(t, network)
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":`, 400, "INVALID_BODY")
|
||||
count := 0
|
||||
for _, line := range strings.Split(strings.TrimSpace(logs.snapshot()), "\n") {
|
||||
var record map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &record); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record["requestId"] == "reference-test" {
|
||||
count++
|
||||
}
|
||||
}
|
||||
if count < requests {
|
||||
t.Fatalf("missing correlated business logs: %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReferenceAddressCompatibility(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_TRANSPORT", "native")
|
||||
t.Setenv("GO_OMNI_TLS_CERT_FILE", "")
|
||||
t.Setenv("GO_OMNI_TLS_KEY_FILE", "")
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "")
|
||||
t.Setenv("GO_OMNI_ADDR", "")
|
||||
settings, err := referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8080" || settings.Transport != "native" {
|
||||
t.Fatalf("default: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_ADDR", "127.0.0.1:8081")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8081" {
|
||||
t.Fatalf("legacy: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:8082")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8082" {
|
||||
t.Fatalf("standard: %+v %v", settings, err)
|
||||
}
|
||||
}
|
||||
39
go.mod
39
go.mod
@@ -1,21 +1,48 @@
|
||||
module git.yuntuo.hk/k2safe/go-omni
|
||||
|
||||
go 1.22
|
||||
go 1.22.0
|
||||
|
||||
require (
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0
|
||||
github.com/go-playground/validator/v10 v10.20.0
|
||||
github.com/go-sql-driver/mysql v1.9.3
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1
|
||||
github.com/minio/minio-go/v7 v7.0.83
|
||||
github.com/redis/go-redis/v9 v9.17.3
|
||||
go.opentelemetry.io/otel v1.31.0
|
||||
go.opentelemetry.io/otel/metric v1.31.0
|
||||
go.opentelemetry.io/otel/sdk v1.31.0
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0
|
||||
go.opentelemetry.io/otel/trace v1.31.0
|
||||
golang.org/x/mod v0.21.0
|
||||
golang.org/x/net v0.33.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
require (
|
||||
filippo.io/edwards25519 v1.1.0 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||
github.com/go-ini/ini v1.67.0 // indirect
|
||||
github.com/go-logr/logr v1.4.2 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-playground/locales v0.14.1 // indirect
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/goccy/go-json v0.10.4 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/klauspost/compress v1.17.11 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/stretchr/testify v1.9.0 // indirect
|
||||
golang.org/x/crypto v0.23.0 // indirect
|
||||
golang.org/x/net v0.25.0 // indirect
|
||||
golang.org/x/sys v0.20.0 // indirect
|
||||
golang.org/x/text v0.15.0 // indirect
|
||||
github.com/minio/md5-simd v1.1.2 // indirect
|
||||
github.com/rs/xid v1.6.0 // indirect
|
||||
go.uber.org/atomic v1.7.0 // indirect
|
||||
golang.org/x/crypto v0.31.0 // indirect
|
||||
golang.org/x/sys v0.28.0 // indirect
|
||||
golang.org/x/text v0.21.0 // indirect
|
||||
golang.org/x/time v0.5.0 // indirect
|
||||
)
|
||||
|
||||
118
go.sum
118
go.sum
@@ -1,9 +1,45 @@
|
||||
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
|
||||
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0 h1:uWzn3io54f9L9mvwsQQSv1KpkkFA06hBxI++RvIyvpI=
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
|
||||
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
|
||||
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
|
||||
github.com/dhui/dktest v0.4.3 h1:wquqUxAFdcUgabAVLvSCOKOlag5cIZuaOjYIBOWdsR0=
|
||||
github.com/dhui/dktest v0.4.3/go.mod h1:zNK8IwktWzQRm6I/l2Wjp7MakiyaFWv4G1hjmodmMTs=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
|
||||
github.com/docker/docker v27.2.0+incompatible h1:Rk9nIVdfH3+Vz4cyI/uhbINhEZ/oLmc+CBXmH6fbNk4=
|
||||
github.com/docker/docker v27.2.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
|
||||
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
|
||||
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
|
||||
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
|
||||
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||
@@ -14,19 +50,85 @@ github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBEx
|
||||
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
|
||||
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||
github.com/goccy/go-json v0.10.4 h1:JSwxQzIqKfmFX1swYPpUThQZp/Ka4wzJdK0LWVytLPM=
|
||||
github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1 h1:JML/k+t4tpHCpQTCAD62Nu43NUFzHY4CV3uAuvHGC+Y=
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1/go.mod h1:HAX6m3sQgcdO81tdjn5exv20+3Kb13cmGli1hrD6hks=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
|
||||
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
|
||||
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
|
||||
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
|
||||
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 h1:66ze0taIn2H33fBvCkXuv9BmCwDfafmiIVpKV9kKGuY=
|
||||
github.com/klauspost/cpuid/v2 v2.2.9/go.mod h1:rqkxqrZ1EhYM9G+hXH7YdowN5R5RGN6NK4QwQ3WMXF8=
|
||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
|
||||
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
|
||||
github.com/minio/minio-go/v7 v7.0.83 h1:W4Kokksvlz3OKf3OqIlzDNKd4MERlC2oN8YptwJ0+GA=
|
||||
github.com/minio/minio-go/v7 v7.0.83/go.mod h1:57YXpvc5l3rjPdhqNrDsvVlY0qPI6UTk1bflAe+9doY=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
|
||||
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
|
||||
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
|
||||
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
|
||||
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
|
||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/redis/go-redis/v9 v9.17.3 h1:fN29NdNrE17KttK5Ndf20buqfDZwGNgoUr9qjl1DQx4=
|
||||
github.com/redis/go-redis/v9 v9.17.3/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
|
||||
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
|
||||
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI=
|
||||
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
|
||||
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
|
||||
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
|
||||
golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
|
||||
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
|
||||
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 h1:TT4fX+nBOA/+LUkobKGW1ydGcn+G3vRw9+g5HwCphpk=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0/go.mod h1:L7UH0GbB0p47T4Rri3uHjbpCFYrVrwc1I25QhNPiGK8=
|
||||
go.opentelemetry.io/otel v1.31.0 h1:NsJcKPIW0D0H3NgzPDHmo0WW6SptzPdqg/L1zsIm2hY=
|
||||
go.opentelemetry.io/otel v1.31.0/go.mod h1:O0C14Yl9FgkjqcCZAsE053C13OaddMYr/hz6clDkEJE=
|
||||
go.opentelemetry.io/otel/metric v1.31.0 h1:FSErL0ATQAmYHUIzSezZibnyVlft1ybhy4ozRPcF2fE=
|
||||
go.opentelemetry.io/otel/metric v1.31.0/go.mod h1:C3dEloVbLuYoX41KpmAhOqNriGbA+qqH6PQ5E5mUfnY=
|
||||
go.opentelemetry.io/otel/sdk v1.31.0 h1:xLY3abVHYZ5HSfOg3l2E5LUj2Cwva5Y7yGxnSW9H5Gk=
|
||||
go.opentelemetry.io/otel/sdk v1.31.0/go.mod h1:TfRbMdhvxIIr/B2N2LQW2S5v9m3gOQ/08KsbbO5BPT0=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0 h1:i9hxxLJF/9kkvfHppyLL55aW7iIJz4JjxTeYusH7zMc=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0/go.mod h1:CRInTMVvNhUKgSAMbKyTMxqOBC0zgyxzW55lZzX43Y8=
|
||||
go.opentelemetry.io/otel/trace v1.31.0 h1:ffjsj1aRouKewfr85U2aGagJ46+MvodynlQ1HYdmJys=
|
||||
go.opentelemetry.io/otel/trace v1.31.0/go.mod h1:TXZkRk7SM2ZQLtR6eoAWQFIHPvzQ06FJAsO1tJg480A=
|
||||
go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
|
||||
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
|
||||
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||
golang.org/x/mod v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
|
||||
golang.org/x/mod v0.21.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
|
||||
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
|
||||
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
|
||||
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
||||
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
|
||||
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
|
||||
255
httpc/client.go
Normal file
255
httpc/client.go
Normal file
@@ -0,0 +1,255 @@
|
||||
// Package httpc provides a pooled HTTP client with an end-to-end deadline and
|
||||
// conservative, bounded retries. Redirects are returned to the caller.
|
||||
package httpc
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
var ErrClosed = errors.New("httpc: closed")
|
||||
var ErrResponseTooLarge = errors.New("httpc: response exceeds limit")
|
||||
|
||||
type Config struct {
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
MaxRetries int `json:"maxRetries"`
|
||||
Backoff time.Duration `json:"backoff"`
|
||||
MaxBackoff time.Duration `json:"maxBackoff"`
|
||||
MaxConnsPerHost int `json:"maxConnsPerHost"`
|
||||
MaxResponseBytes int64 `json:"maxResponseBytes"`
|
||||
DisableRequestID bool `json:"disableRequestId"`
|
||||
DisableLogging bool `json:"disableLogging"`
|
||||
Transport http.RoundTripper `json:"-"` // Borrowed. The caller owns its lifecycle.
|
||||
}
|
||||
|
||||
type Client struct {
|
||||
config Config
|
||||
client *http.Client
|
||||
owned *http.Transport
|
||||
closed atomic.Bool
|
||||
}
|
||||
|
||||
func New(cfg Config) (*Client, error) {
|
||||
if cfg.Timeout < 0 || cfg.Backoff < 0 || cfg.MaxBackoff < 0 || cfg.MaxRetries < 0 || cfg.MaxRetries > 5 || cfg.MaxConnsPerHost < 0 || cfg.MaxResponseBytes < 0 {
|
||||
return nil, errors.New("httpc: invalid configuration")
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 10 * time.Second
|
||||
}
|
||||
if cfg.Backoff == 0 {
|
||||
cfg.Backoff = 100 * time.Millisecond
|
||||
}
|
||||
if cfg.MaxBackoff == 0 {
|
||||
cfg.MaxBackoff = 2 * time.Second
|
||||
}
|
||||
if cfg.Backoff > cfg.MaxBackoff {
|
||||
return nil, errors.New("httpc: backoff exceeds maxBackoff")
|
||||
}
|
||||
if cfg.MaxConnsPerHost == 0 {
|
||||
cfg.MaxConnsPerHost = 64
|
||||
}
|
||||
if cfg.MaxResponseBytes == 0 {
|
||||
cfg.MaxResponseBytes = 1 << 20
|
||||
}
|
||||
if cfg.MaxResponseBytes > 1<<30 || cfg.MaxConnsPerHost > 65536 {
|
||||
return nil, errors.New("httpc: response or connection limit exceeds supported capacity")
|
||||
}
|
||||
c := &Client{config: cfg}
|
||||
transport := cfg.Transport
|
||||
if transport == nil {
|
||||
// Own a private pool; never mutate the process-wide DefaultTransport.
|
||||
base, ok := http.DefaultTransport.(*http.Transport)
|
||||
if !ok {
|
||||
return nil, errors.New("httpc: provide Transport when DefaultTransport is replaced")
|
||||
}
|
||||
c.owned = base.Clone()
|
||||
c.owned.MaxConnsPerHost = cfg.MaxConnsPerHost
|
||||
c.owned.MaxIdleConnsPerHost = cfg.MaxConnsPerHost
|
||||
c.owned.MaxIdleConns = cfg.MaxConnsPerHost * 2
|
||||
transport = c.owned
|
||||
}
|
||||
c.client = &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// Do keeps the total deadline active until the response body is closed or
|
||||
// consumed. Callers own response.Body. Only GET, HEAD and OPTIONS are retried,
|
||||
// and only when their bodies can be replayed. MaxRetries=0 disables retries.
|
||||
func (c *Client) Do(request *http.Request) (*http.Response, error) {
|
||||
var requestError error
|
||||
if c.closed.Load() {
|
||||
requestError = ErrClosed
|
||||
} else if request == nil || request.URL == nil {
|
||||
requestError = errors.New("httpc: nil request or URL")
|
||||
}
|
||||
if requestError != nil {
|
||||
// Match http.Client.Do's ownership rule even when rejection happens
|
||||
// before the underlying transport takes ownership of the body.
|
||||
if request != nil && request.Body != nil {
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
return nil, requestError
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(request.Context(), c.config.Timeout)
|
||||
base := request.Clone(ctx)
|
||||
if base.Header == nil {
|
||||
base.Header = make(http.Header)
|
||||
}
|
||||
if !c.config.DisableRequestID && base.Header.Get("X-Request-ID") == "" {
|
||||
if id := logx.RequestID(ctx); id != "" {
|
||||
base.Header.Set("X-Request-ID", id)
|
||||
}
|
||||
}
|
||||
replayable := (base.Method == "" || base.Method == "GET" || base.Method == "HEAD" || base.Method == "OPTIONS") && (base.Body == nil || base.Body == http.NoBody || base.GetBody != nil)
|
||||
for attempt := 0; ; attempt++ {
|
||||
req := base.Clone(ctx)
|
||||
if attempt > 0 && base.Body != nil && base.Body != http.NoBody {
|
||||
body, err := base.GetBody()
|
||||
if err != nil {
|
||||
cancel()
|
||||
return nil, err
|
||||
}
|
||||
req.Body = body
|
||||
}
|
||||
started := time.Now()
|
||||
response, err := c.client.Do(req)
|
||||
status := 0
|
||||
if response != nil {
|
||||
status = response.StatusCode
|
||||
}
|
||||
if !c.config.DisableLogging {
|
||||
logx.InfoContext(ctx, "http.client", "method", req.Method, "host", req.URL.Hostname(), "status", status, "attempt", attempt+1, "durationMs", time.Since(started).Seconds()*1000, "failed", err != nil)
|
||||
}
|
||||
retry := replayable && attempt < c.config.MaxRetries && ctx.Err() == nil && (err != nil || status == 429 || status == 502 || status == 503 || status == 504)
|
||||
delay := c.config.Backoff
|
||||
for i := 0; i < attempt && delay < c.config.MaxBackoff; i++ {
|
||||
if delay > c.config.MaxBackoff/2 {
|
||||
delay = c.config.MaxBackoff
|
||||
} else {
|
||||
delay *= 2
|
||||
}
|
||||
}
|
||||
if response != nil && response.Header.Get("Retry-After") != "" {
|
||||
if value, ok := retryAfter(response.Header.Get("Retry-After")); ok {
|
||||
delay = value
|
||||
}
|
||||
} else if delay > 1 {
|
||||
delay = delay/2 + time.Duration(rand.Int63n(int64(delay/2)+1))
|
||||
}
|
||||
if delay > c.config.MaxBackoff {
|
||||
retry = false
|
||||
}
|
||||
if !retry {
|
||||
if err != nil {
|
||||
cancel()
|
||||
return response, err
|
||||
}
|
||||
response.Body = &responseBody{ReadCloser: response.Body, cancel: cancel}
|
||||
return response, nil
|
||||
}
|
||||
if response != nil {
|
||||
if response.ContentLength >= 0 && response.ContentLength <= 32<<10 {
|
||||
_, _ = io.Copy(io.Discard, response.Body)
|
||||
}
|
||||
_ = response.Body.Close()
|
||||
}
|
||||
timer := time.NewTimer(delay)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
cancel()
|
||||
return nil, ctx.Err()
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type responseBody struct {
|
||||
io.ReadCloser
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func (b *responseBody) Read(data []byte) (int, error) {
|
||||
n, err := b.ReadCloser.Read(data)
|
||||
if err != nil {
|
||||
b.cancel()
|
||||
}
|
||||
return n, err
|
||||
}
|
||||
func (b *responseBody) Close() error { defer b.cancel(); return b.ReadCloser.Close() }
|
||||
|
||||
func retryAfter(value string) (time.Duration, bool) {
|
||||
if n, err := strconv.ParseInt(strings.TrimSpace(value), 10, 32); err == nil && n >= 0 {
|
||||
return time.Duration(n) * time.Second, true
|
||||
}
|
||||
if at, err := http.ParseTime(value); err == nil {
|
||||
return max(time.Until(at), 0), true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
type StatusError struct{ StatusCode int }
|
||||
|
||||
func (e *StatusError) Error() string { return fmt.Sprintf("httpc: HTTP status %d", e.StatusCode) }
|
||||
|
||||
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
|
||||
// Error bodies, URLs, credentials and query strings are not included in logs.
|
||||
func (c *Client) JSON(ctx context.Context, method, url string, input, output any) error {
|
||||
if ctx == nil {
|
||||
return errors.New("httpc: nil context")
|
||||
}
|
||||
var body io.Reader
|
||||
if input != nil {
|
||||
data, err := json.Marshal(input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body = bytes.NewReader(data)
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, method, url, body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if input != nil {
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
response, err := c.Do(request)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return &StatusError{StatusCode: response.StatusCode}
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(response.Body, c.config.MaxResponseBytes+1))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if int64(len(data)) > c.config.MaxResponseBytes {
|
||||
return ErrResponseTooLarge
|
||||
}
|
||||
if output == nil || len(bytes.TrimSpace(data)) == 0 {
|
||||
return nil
|
||||
}
|
||||
return json.Unmarshal(data, output)
|
||||
}
|
||||
|
||||
func (c *Client) Close() error {
|
||||
c.closed.Store(true)
|
||||
if c.owned != nil {
|
||||
c.owned.CloseIdleConnections()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
156
httpc/client_test.go
Normal file
156
httpc/client_test.go
Normal file
@@ -0,0 +1,156 @@
|
||||
package httpc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type trackedBody struct {
|
||||
io.Reader
|
||||
closed bool
|
||||
}
|
||||
|
||||
func (b *trackedBody) Close() error { b.closed = true; return nil }
|
||||
|
||||
func TestRejectedRequestClosesBody(t *testing.T) {
|
||||
for _, closed := range []bool{false, true} {
|
||||
client, _ := New(Config{DisableLogging: true})
|
||||
body := &trackedBody{Reader: strings.NewReader("body")}
|
||||
request, _ := http.NewRequest("POST", "http://unused.invalid", body)
|
||||
if closed {
|
||||
client.Close()
|
||||
} else {
|
||||
request.URL = nil
|
||||
}
|
||||
_, err := client.Do(request)
|
||||
client.Close()
|
||||
if err == nil || !body.closed {
|
||||
t.Fatalf("error=%v bodyClosed=%v", err, body.closed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetryCorrelationPoolAndWriteSafety(t *testing.T) {
|
||||
var calls, connections atomic.Int32
|
||||
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
n := calls.Add(1)
|
||||
if r.Header.Get("X-Request-ID") != "request-1" {
|
||||
t.Error("missing correlation")
|
||||
}
|
||||
if n == 1 || r.Method == "POST" {
|
||||
w.WriteHeader(503)
|
||||
io.WriteString(w, "busy")
|
||||
return
|
||||
}
|
||||
io.WriteString(w, `{"ok":true}`)
|
||||
}))
|
||||
server.Config.ConnState = func(_ net.Conn, s http.ConnState) {
|
||||
if s == http.StateNew {
|
||||
connections.Add(1)
|
||||
}
|
||||
}
|
||||
server.Start()
|
||||
defer server.Close()
|
||||
client, err := New(Config{MaxRetries: 2, Backoff: time.Millisecond, DisableLogging: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
ctx := logx.WithRequestID(context.Background(), "request-1")
|
||||
var output struct {
|
||||
OK bool `json:"ok"`
|
||||
}
|
||||
if err := client.JSON(ctx, "GET", server.URL, nil, &output); err != nil || !output.OK {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 2 || connections.Load() != 1 {
|
||||
t.Fatalf("calls=%d connections=%d", calls.Load(), connections.Load())
|
||||
}
|
||||
if err := client.JSON(ctx, "POST", server.URL, map[string]string{"name": "value"}, nil); err == nil {
|
||||
t.Fatal("expected 503")
|
||||
}
|
||||
if calls.Load() != 3 {
|
||||
t.Fatal("POST was retried")
|
||||
}
|
||||
client.Close()
|
||||
if err := client.JSON(ctx, "GET", server.URL, nil, nil); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTotalDeadlineIncludesBackoffAndBody(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/retry" {
|
||||
w.Header().Set("Retry-After", "1")
|
||||
w.WriteHeader(503)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
w.(http.Flusher).Flush()
|
||||
<-r.Context().Done()
|
||||
}))
|
||||
defer server.Close()
|
||||
client, _ := New(Config{Timeout: 40 * time.Millisecond, MaxRetries: 3, DisableLogging: true})
|
||||
defer client.Close()
|
||||
started := time.Now()
|
||||
err := client.JSON(context.Background(), "GET", server.URL+"/retry", nil, nil)
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(started) > time.Second {
|
||||
t.Fatal(err, time.Since(started))
|
||||
}
|
||||
req, _ := http.NewRequest("GET", server.URL+"/body", nil)
|
||||
response, err := client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if _, err := io.ReadAll(response.Body); err == nil {
|
||||
t.Fatal("body deadline was canceled too early")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLimitsRedirectAndNonReplayableBody(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
calls.Add(1)
|
||||
switch r.URL.Path {
|
||||
case "/redirect":
|
||||
http.Redirect(w, r, "/large", 302)
|
||||
case "/large":
|
||||
io.WriteString(w, strings.Repeat("x", 17))
|
||||
default:
|
||||
w.WriteHeader(503)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
client, _ := New(Config{MaxRetries: 2, MaxResponseBytes: 16, DisableLogging: true})
|
||||
defer client.Close()
|
||||
if err := client.JSON(context.Background(), "GET", server.URL+"/large", nil, nil); !errors.Is(err, ErrResponseTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var status *StatusError
|
||||
if err := client.JSON(context.Background(), "GET", server.URL+"/redirect", nil, nil); !errors.As(err, &status) || status.StatusCode != 302 {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 2 {
|
||||
t.Fatal("followed redirect")
|
||||
}
|
||||
req, _ := http.NewRequest("GET", server.URL, io.NopCloser(strings.NewReader("body")))
|
||||
res, err := client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res.Body.Close()
|
||||
if calls.Load() != 3 {
|
||||
t.Fatal("non-replayable body was retried")
|
||||
}
|
||||
}
|
||||
105
httpx/clientIP.go
Normal file
105
httpx/clientIP.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ClientIPResolver trusts X-Forwarded-For only from explicitly configured
|
||||
// proxy addresses. It is immutable and safe to share between requests.
|
||||
// The zero value (and a nil resolver) trusts no proxy.
|
||||
type ClientIPResolver struct {
|
||||
trusted []netip.Prefix
|
||||
}
|
||||
|
||||
// NewClientIPResolver accepts IP addresses or CIDR networks, never hostnames.
|
||||
func NewClientIPResolver(proxies ...string) (*ClientIPResolver, error) {
|
||||
r := &ClientIPResolver{}
|
||||
for _, value := range proxies {
|
||||
value = strings.TrimSpace(value)
|
||||
if address, err := netip.ParseAddr(value); err == nil && address.Zone() == "" {
|
||||
address = address.Unmap()
|
||||
r.trusted = append(r.trusted, netip.PrefixFrom(address, address.BitLen()))
|
||||
continue
|
||||
}
|
||||
prefix, err := netip.ParsePrefix(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid trusted proxy %q: expected an IP address or CIDR", value)
|
||||
}
|
||||
if prefix.Addr().Is4In6() {
|
||||
if prefix.Bits() < 96 {
|
||||
return nil, fmt.Errorf("invalid trusted proxy %q: mapped IPv4 prefix must be at least /96", value)
|
||||
}
|
||||
prefix = netip.PrefixFrom(prefix.Addr().Unmap(), prefix.Bits()-96)
|
||||
}
|
||||
r.trusted = append(r.trusted, prefix.Masked())
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
func (r *ClientIPResolver) trusts(address netip.Addr) bool {
|
||||
if r != nil {
|
||||
for _, prefix := range r.trusted {
|
||||
if prefix.Contains(address) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ClientIP walks X-Forwarded-For from right to left, stopping at the first
|
||||
// untrusted address. Missing or malformed forwarding data falls back to the
|
||||
// socket peer. X-Real-IP, Forwarded and provider-specific headers are ignored.
|
||||
func (r *ClientIPResolver) ClientIP(request *http.Request) string {
|
||||
if request == nil {
|
||||
return ""
|
||||
}
|
||||
host := request.RemoteAddr
|
||||
if value, _, err := net.SplitHostPort(host); err == nil {
|
||||
host = value
|
||||
}
|
||||
peer, err := netip.ParseAddr(host)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
peer = peer.WithZone("").Unmap()
|
||||
if !r.trusts(peer) {
|
||||
return peer.String()
|
||||
}
|
||||
values := request.Header.Values("X-Forwarded-For")
|
||||
var chain []netip.Addr
|
||||
for _, value := range values {
|
||||
for _, part := range strings.Split(value, ",") {
|
||||
address, err := netip.ParseAddr(strings.TrimSpace(part))
|
||||
if err != nil || address.Zone() != "" {
|
||||
return peer.String()
|
||||
}
|
||||
chain = append(chain, address.Unmap())
|
||||
}
|
||||
}
|
||||
current := peer
|
||||
for i := len(chain) - 1; i >= 0 && r.trusts(current); i-- {
|
||||
current = chain[i]
|
||||
}
|
||||
return current.String()
|
||||
}
|
||||
|
||||
type clientIPResolverKey struct{}
|
||||
|
||||
// WithClientIPResolver configures ClientIP for Context values derived from ctx.
|
||||
// Adapters should set this before running application middleware.
|
||||
func WithClientIPResolver(ctx context.Context, resolver *ClientIPResolver) context.Context {
|
||||
return context.WithValue(ctx, clientIPResolverKey{}, resolver)
|
||||
}
|
||||
|
||||
// ClientIP returns the verified client address without a port. By default it
|
||||
// returns the socket peer; App's trusted proxy configuration enables XFF.
|
||||
func (c *Context) ClientIP() string {
|
||||
resolver, _ := c.Context().Value(clientIPResolverKey{}).(*ClientIPResolver)
|
||||
return resolver.ClientIP(c.request)
|
||||
}
|
||||
76
httpx/clientIP_test.go
Normal file
76
httpx/clientIP_test.go
Normal file
@@ -0,0 +1,76 @@
|
||||
package httpx_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
func TestClientIPResolverUsesSocketPeerByDefault(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("192.0.2.10:443", "198.51.100.7")
|
||||
if got := resolver.ClientIP(request); got != "192.0.2.10" {
|
||||
t.Fatalf("untrusted forwarding header changed client IP: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIPResolverWalksTrustedForwardingChain(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("10.0.0.0/8")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("10.0.0.5:443", "198.51.100.7, 10.0.0.4")
|
||||
if got := resolver.ClientIP(request); got != "198.51.100.7" {
|
||||
t.Fatalf("expected original client IP, got %q", got)
|
||||
}
|
||||
|
||||
request = newClientIPRequest("10.0.0.5:443", "198.51.100.7, 203.0.113.9, 10.0.0.4")
|
||||
if got := resolver.ClientIP(request); got != "203.0.113.9" {
|
||||
t.Fatalf("expected first untrusted hop, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIPResolverRejectsMalformedForwardingData(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("127.0.0.1:443", "198.51.100.7, not-an-ip")
|
||||
if got := resolver.ClientIP(request); got != "127.0.0.1" {
|
||||
t.Fatalf("malformed forwarding header should fall back to peer, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewClientIPResolverRejectsHostnames(t *testing.T) {
|
||||
if _, err := httpx.NewClientIPResolver("proxy.example.com"); err == nil || !strings.Contains(err.Error(), "invalid trusted proxy") {
|
||||
t.Fatalf("expected hostname validation error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContextClientIPUsesResolverFromRequestContext(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("127.0.0.1:443", "198.51.100.7")
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), resolver))
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
if got := ctx.ClientIP(); got != "198.51.100.7" {
|
||||
t.Fatalf("expected context client IP, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func newClientIPRequest(remote, forwarded string) *http.Request {
|
||||
request := httptest.NewRequest(http.MethodGet, "http://example.test/", nil)
|
||||
request.RemoteAddr = remote
|
||||
if forwarded != "" {
|
||||
request.Header.Set("X-Forwarded-For", forwarded)
|
||||
}
|
||||
return request
|
||||
}
|
||||
@@ -117,6 +117,37 @@ func (c *Context) Context() context.Context {
|
||||
// Prefer the transport-neutral methods on Context for portable handlers.
|
||||
func (c *Context) Raw() http.ResponseWriter { return c.writer }
|
||||
|
||||
// Clone gives a buffering middleware its own response state and values map.
|
||||
// Stored values themselves are shared; middleware must not mutate them concurrently.
|
||||
func (c *Context) Clone(ctx context.Context, writer http.ResponseWriter) *Context {
|
||||
clone := NewContext(c.request.Clone(ctx), writer, c.params)
|
||||
for key, value := range c.values {
|
||||
clone.values[key] = value
|
||||
}
|
||||
return clone
|
||||
}
|
||||
|
||||
// WriteResponse commits a buffered response through GoOmni's response tracking.
|
||||
func (c *Context) WriteResponse(status int, headers http.Header, body []byte) error {
|
||||
if c.responded {
|
||||
return nil
|
||||
}
|
||||
if c.writer == nil {
|
||||
return apierrors.Internal(nil)
|
||||
}
|
||||
for key, values := range headers {
|
||||
c.writer.Header()[key] = append([]string(nil), values...)
|
||||
}
|
||||
c.responded, c.status = true, status
|
||||
c.writer.WriteHeader(status)
|
||||
if len(body) > 0 {
|
||||
if _, err := c.writer.Write(body); err != nil {
|
||||
return apierrors.Internal(err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Context) Param(name string) string { return c.params[name] }
|
||||
|
||||
func (c *Context) Query(name string) string {
|
||||
|
||||
696
internal/perfcheck/main.go
Normal file
696
internal/perfcheck/main.go
Normal file
@@ -0,0 +1,696 @@
|
||||
// perfcheck compares real TCP servers in separate processes. It is a local
|
||||
// measurement tool, not part of the framework's public CLI or server API.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/big"
|
||||
"math/bits"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
const expectedBody = `{"code":"OK","message":"success","data":{"status":"ok"},"requestId":"perfCheck"}`
|
||||
|
||||
type snapshot struct {
|
||||
Address string `json:"address,omitempty"`
|
||||
HeapAlloc uint64 `json:"heapAlloc"`
|
||||
HeapInuse uint64 `json:"heapInuse"`
|
||||
StackInuse uint64 `json:"stackInuse"`
|
||||
TotalAlloc uint64 `json:"totalAlloc"`
|
||||
Mallocs uint64 `json:"mallocs"`
|
||||
NumGC uint32 `json:"numGc"`
|
||||
Goroutines int `json:"goroutines"`
|
||||
Connections int64 `json:"connections"`
|
||||
Accepted int64 `json:"accepted"`
|
||||
ActiveRequests int64 `json:"activeRequests,omitempty"`
|
||||
FileDescriptors int `json:"fileDescriptors,omitempty"`
|
||||
}
|
||||
|
||||
type result struct {
|
||||
Adapter string `json:"adapter"`
|
||||
Scenario string `json:"scenario"`
|
||||
Concurrency int `json:"concurrency"`
|
||||
Run int `json:"run"`
|
||||
Requests uint64 `json:"requests,omitempty"`
|
||||
Errors uint64 `json:"errors"`
|
||||
FirstError string `json:"firstError,omitempty"`
|
||||
ElapsedSeconds float64 `json:"elapsedSeconds,omitempty"`
|
||||
RequestsPerSecond float64 `json:"requestsPerSecond,omitempty"`
|
||||
P50Millis float64 `json:"p50Millis,omitempty"`
|
||||
P95Millis float64 `json:"p95Millis,omitempty"`
|
||||
P99Millis float64 `json:"p99Millis,omitempty"`
|
||||
ServerBytesPerRequest float64 `json:"serverBytesPerRequest,omitempty"`
|
||||
ServerAllocsPerRequest float64 `json:"serverAllocsPerRequest,omitempty"`
|
||||
SampledPeakHeapAlloc uint64 `json:"sampledPeakHeapAlloc,omitempty"`
|
||||
SampledPeakFileDescriptors int `json:"sampledPeakFileDescriptors,omitempty"`
|
||||
IdleHeapDelta int64 `json:"idleHeapDelta,omitempty"`
|
||||
IdleHeapPerConnection float64 `json:"idleHeapPerConnection,omitempty"`
|
||||
Before snapshot `json:"before"`
|
||||
During snapshot `json:"during"`
|
||||
After snapshot `json:"after"`
|
||||
AfterReclaim snapshot `json:"afterReclaim"`
|
||||
}
|
||||
|
||||
type report struct {
|
||||
CreatedAt string `json:"createdAt"`
|
||||
GoVersion string `json:"goVersion"`
|
||||
Platform string `json:"platform"`
|
||||
GOMAXPROCS int `json:"gomaxprocs"`
|
||||
Duration string `json:"duration"`
|
||||
TLS bool `json:"tls"`
|
||||
Notes []string `json:"notes"`
|
||||
Results []result `json:"results"`
|
||||
}
|
||||
|
||||
type trackedListener struct {
|
||||
net.Listener
|
||||
active, accepted atomic.Int64
|
||||
}
|
||||
|
||||
type trackedConn struct {
|
||||
net.Conn
|
||||
once sync.Once
|
||||
active *atomic.Int64
|
||||
}
|
||||
|
||||
func (l *trackedListener) Accept() (net.Conn, error) {
|
||||
c, err := l.Listener.Accept()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
l.active.Add(1)
|
||||
l.accepted.Add(1)
|
||||
return &trackedConn{Conn: c, active: &l.active}, nil
|
||||
}
|
||||
|
||||
func (c *trackedConn) Close() error {
|
||||
err := c.Conn.Close()
|
||||
c.once.Do(func() { c.active.Add(-1) })
|
||||
return err
|
||||
}
|
||||
|
||||
func testTLSConfig() (*tls.Config, error) {
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 120))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
now := time.Now()
|
||||
template := &x509.Certificate{
|
||||
SerialNumber: serial,
|
||||
Subject: pkix.Name{CommonName: "localhost"},
|
||||
NotBefore: now.Add(-time.Minute),
|
||||
NotAfter: now.Add(time.Hour),
|
||||
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
||||
DNSNames: []string{"localhost"},
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &tls.Config{
|
||||
Certificates: []tls.Certificate{{Certificate: [][]byte{der}, PrivateKey: key}},
|
||||
MinVersion: tls.VersionTLS12,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
serverMode := flag.String("server", "", "internal server subprocess: nethttp or native")
|
||||
tlsEnabled := flag.Bool("tls", false, "run the comparison over TLS")
|
||||
duration := flag.Duration("duration", 2*time.Second, "measured duration per traffic case")
|
||||
concurrencyText := flag.String("concurrency", "1,32,128", "traffic concurrency levels")
|
||||
idleText := flag.String("idle", "128,1024", "established idle Keep-Alive connection levels")
|
||||
runs := flag.Int("count", 3, "repetitions; adapter order alternates each repetition")
|
||||
output := flag.String("output", "", "optional JSON report path")
|
||||
flag.Parse()
|
||||
var err error
|
||||
if *serverMode != "" {
|
||||
err = serve(*serverMode, *tlsEnabled)
|
||||
} else {
|
||||
err = compare(*duration, *concurrencyText, *idleText, *runs, *output, *tlsEnabled)
|
||||
}
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func serve(adapter string, tlsEnabled bool) error {
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tracked := &trackedListener{Listener: listener}
|
||||
var serveListener net.Listener = tracked
|
||||
if tlsEnabled {
|
||||
config, configErr := testTLSConfig()
|
||||
if configErr != nil {
|
||||
listener.Close()
|
||||
return configErr
|
||||
}
|
||||
serveListener = tls.NewListener(tracked, config)
|
||||
}
|
||||
application := omni.New()
|
||||
application.GET("/health", func(c *omni.Context) error {
|
||||
return c.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
application.GET("/slow", func(c *omni.Context) error {
|
||||
timer := time.NewTimer(2 * time.Second)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-timer.C:
|
||||
return c.OK(map[string]string{"status": "slow"})
|
||||
case <-c.Context().Done():
|
||||
return c.Context().Err()
|
||||
}
|
||||
})
|
||||
done := make(chan error, 1)
|
||||
var shutdown func(context.Context) error
|
||||
var stats func() native.Stats
|
||||
switch adapter {
|
||||
case "native":
|
||||
server := native.New(application)
|
||||
server.MaxConnections = 8192
|
||||
server.MaxConcurrentRequests = 4096
|
||||
shutdown = server.Shutdown
|
||||
stats = server.Stats
|
||||
go func() { done <- server.Serve(context.Background(), serveListener) }()
|
||||
case "nethttp":
|
||||
server := &http.Server{
|
||||
Handler: application, ReadHeaderTimeout: 5 * time.Second,
|
||||
ReadTimeout: 30 * time.Second, WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: time.Minute, MaxHeaderBytes: 1 << 20,
|
||||
}
|
||||
shutdown = server.Shutdown
|
||||
go func() {
|
||||
err := server.Serve(serveListener)
|
||||
if errors.Is(err, http.ErrServerClosed) {
|
||||
err = nil
|
||||
}
|
||||
done <- err
|
||||
}()
|
||||
default:
|
||||
listener.Close()
|
||||
return fmt.Errorf("unknown server %q", adapter)
|
||||
}
|
||||
stop := func() error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := shutdown(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
return err
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
encoder := json.NewEncoder(os.Stdout)
|
||||
if err := encoder.Encode(snapshot{Address: listener.Addr().String()}); err != nil {
|
||||
return err
|
||||
}
|
||||
decoder := json.NewDecoder(os.Stdin)
|
||||
for {
|
||||
var command string
|
||||
if err := decoder.Decode(&command); err != nil || command == "stop" {
|
||||
return stop()
|
||||
}
|
||||
if command == "gc" {
|
||||
runtime.GC()
|
||||
}
|
||||
var memory runtime.MemStats
|
||||
runtime.ReadMemStats(&memory)
|
||||
connections, accepted, activeRequests := tracked.active.Load(), tracked.accepted.Load(), int64(0)
|
||||
if stats != nil {
|
||||
value := stats()
|
||||
connections, accepted, activeRequests = value.ActiveConnections, int64(value.AcceptedConnections), value.ActiveRequests
|
||||
}
|
||||
if err := encoder.Encode(snapshot{
|
||||
HeapAlloc: memory.HeapAlloc, HeapInuse: memory.HeapInuse,
|
||||
StackInuse: memory.StackInuse, TotalAlloc: memory.TotalAlloc,
|
||||
Mallocs: memory.Mallocs, NumGC: memory.NumGC,
|
||||
Goroutines: runtime.NumGoroutine(), Connections: connections, Accepted: accepted, ActiveRequests: activeRequests,
|
||||
FileDescriptors: openFileDescriptorCount(),
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func openFileDescriptorCount() int {
|
||||
entries, err := os.ReadDir("/proc/self/fd")
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return len(entries)
|
||||
}
|
||||
|
||||
type child struct {
|
||||
cmd *exec.Cmd
|
||||
input io.WriteCloser
|
||||
encoder *json.Encoder
|
||||
decoder *json.Decoder
|
||||
address string
|
||||
tls bool
|
||||
cancel context.CancelFunc
|
||||
closed bool
|
||||
}
|
||||
|
||||
func startChild(adapter string) (*child, error) {
|
||||
return startChildMode(adapter, false)
|
||||
}
|
||||
|
||||
func startChildMode(adapter string, tlsEnabled bool) (*child, error) {
|
||||
executable, err := os.Executable()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// This bounds even a broken server or protocol read; it is not a load
|
||||
// duration. Keep it above the supported 30-minute soak plus drain time.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 35*time.Minute)
|
||||
args := []string{"--server", adapter}
|
||||
if tlsEnabled {
|
||||
args = append(args, "--tls")
|
||||
}
|
||||
c := &child{cmd: exec.CommandContext(ctx, executable, args...), cancel: cancel, tls: tlsEnabled}
|
||||
c.cmd.Stderr = os.Stderr
|
||||
if c.input, err = c.cmd.StdinPipe(); err != nil {
|
||||
cancel()
|
||||
return nil, err
|
||||
}
|
||||
output, err := c.cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
cancel()
|
||||
c.input.Close()
|
||||
return nil, err
|
||||
}
|
||||
if err := c.cmd.Start(); err != nil {
|
||||
cancel()
|
||||
c.input.Close()
|
||||
output.Close()
|
||||
return nil, err
|
||||
}
|
||||
c.encoder, c.decoder = json.NewEncoder(c.input), json.NewDecoder(output)
|
||||
var ready snapshot
|
||||
if err := c.decoder.Decode(&ready); err != nil {
|
||||
cancel()
|
||||
c.cmd.Wait()
|
||||
return nil, err
|
||||
}
|
||||
c.address = ready.Address
|
||||
return c, nil
|
||||
}
|
||||
|
||||
func (c *child) snapshot(gc bool) (snapshot, error) {
|
||||
command := "snapshot"
|
||||
if gc {
|
||||
command = "gc"
|
||||
}
|
||||
var value snapshot
|
||||
if err := c.encoder.Encode(command); err != nil {
|
||||
return value, err
|
||||
}
|
||||
err := c.decoder.Decode(&value)
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (c *child) close() error {
|
||||
if c.closed {
|
||||
return nil
|
||||
}
|
||||
c.closed = true
|
||||
defer c.cancel()
|
||||
err := c.encoder.Encode("stop")
|
||||
c.input.Close()
|
||||
return errors.Join(err, c.cmd.Wait())
|
||||
}
|
||||
|
||||
// Histograms use 16 subdivisions per power of two. Percentiles are bucket
|
||||
// upper bounds (at most about 6.25% rounding), with bounded client memory.
|
||||
type histogram [1024]uint64
|
||||
|
||||
func (h *histogram) add(d time.Duration) {
|
||||
n := uint64(max(d.Nanoseconds(), 1))
|
||||
exponent := bits.Len64(n) - 1
|
||||
base := uint64(1) << exponent
|
||||
sub := (n - base) * 16 / base
|
||||
h[exponent*16+int(sub)]++
|
||||
}
|
||||
|
||||
func (h *histogram) percentile(count uint64, percent uint64) float64 {
|
||||
target := (count*percent + 99) / 100
|
||||
if target == 0 {
|
||||
return 0
|
||||
}
|
||||
var seen uint64
|
||||
for bucket, n := range h {
|
||||
seen += n
|
||||
if seen >= target {
|
||||
base := uint64(1) << (bucket / 16)
|
||||
return float64(base+uint64(bucket%16+1)*base/16) / float64(time.Millisecond)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func exchange(client *http.Client, address string) error {
|
||||
return exchangeMode(client, address, false)
|
||||
}
|
||||
|
||||
func exchangeMode(client *http.Client, address string, tlsEnabled bool) error {
|
||||
scheme := "http"
|
||||
if tlsEnabled {
|
||||
scheme = "https"
|
||||
}
|
||||
req, err := http.NewRequest(http.MethodGet, scheme+"://"+address+"/health", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("X-Request-ID", "perfCheck")
|
||||
response, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body, readErr := io.ReadAll(io.LimitReader(response.Body, 4096))
|
||||
closeErr := response.Body.Close()
|
||||
if response.StatusCode != 200 || response.Header.Get("X-Request-ID") != "perfCheck" || strings.TrimSpace(string(body)) != expectedBody {
|
||||
return fmt.Errorf("invalid response status=%d body=%q", response.StatusCode, body)
|
||||
}
|
||||
return errors.Join(readErr, closeErr)
|
||||
}
|
||||
|
||||
func traffic(c *child, concurrency int, duration time.Duration) (result, error) {
|
||||
r := result{Scenario: "traffic", Concurrency: concurrency}
|
||||
transport := &http.Transport{
|
||||
MaxIdleConns: concurrency, MaxIdleConnsPerHost: concurrency,
|
||||
MaxConnsPerHost: concurrency, IdleConnTimeout: time.Minute,
|
||||
}
|
||||
if c.tls {
|
||||
transport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true} // test certificate is self-signed
|
||||
}
|
||||
client := &http.Client{Transport: transport, Timeout: 5 * time.Second}
|
||||
defer client.CloseIdleConnections()
|
||||
// Warm every worker before forcing a baseline GC in the server.
|
||||
warmErrors := make(chan error, concurrency)
|
||||
for i := 0; i < concurrency; i++ {
|
||||
go func() { warmErrors <- exchangeMode(client, c.address, c.tls) }()
|
||||
}
|
||||
for i := 0; i < concurrency; i++ {
|
||||
if err := <-warmErrors; err != nil {
|
||||
return r, err
|
||||
}
|
||||
}
|
||||
var err error
|
||||
if r.Before, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
type workerResult struct {
|
||||
latencies histogram
|
||||
count, errors uint64
|
||||
firstError string
|
||||
}
|
||||
workers := make([]workerResult, concurrency)
|
||||
start := make(chan struct{})
|
||||
var group sync.WaitGroup
|
||||
var deadline time.Time
|
||||
for i := range workers {
|
||||
group.Add(1)
|
||||
go func(w *workerResult) {
|
||||
defer group.Done()
|
||||
<-start
|
||||
for time.Now().Before(deadline) {
|
||||
began := time.Now()
|
||||
err := exchangeMode(client, c.address, c.tls)
|
||||
w.latencies.add(time.Since(began))
|
||||
w.count++
|
||||
if err != nil {
|
||||
w.errors++
|
||||
if w.firstError == "" {
|
||||
w.firstError = err.Error()
|
||||
}
|
||||
}
|
||||
}
|
||||
}(&workers[i])
|
||||
}
|
||||
done := make(chan struct{})
|
||||
go func() { group.Wait(); close(done) }()
|
||||
begin := time.Now()
|
||||
deadline = begin.Add(duration)
|
||||
close(start)
|
||||
ticker := time.NewTicker(200 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
var sampleErr error
|
||||
loading := true
|
||||
for loading {
|
||||
select {
|
||||
case <-done:
|
||||
loading = false
|
||||
case <-ticker.C:
|
||||
s, err := c.snapshot(false)
|
||||
if err != nil {
|
||||
sampleErr = err
|
||||
} else if s.HeapAlloc > r.SampledPeakHeapAlloc {
|
||||
r.SampledPeakHeapAlloc = s.HeapAlloc
|
||||
}
|
||||
if s.FileDescriptors > r.SampledPeakFileDescriptors {
|
||||
r.SampledPeakFileDescriptors = s.FileDescriptors
|
||||
}
|
||||
}
|
||||
}
|
||||
r.ElapsedSeconds = time.Since(begin).Seconds()
|
||||
if sampleErr != nil {
|
||||
return r, sampleErr
|
||||
}
|
||||
var latencies histogram
|
||||
for _, w := range workers {
|
||||
r.Requests += w.count
|
||||
r.Errors += w.errors
|
||||
if r.FirstError == "" {
|
||||
r.FirstError = w.firstError
|
||||
}
|
||||
for i, n := range w.latencies {
|
||||
latencies[i] += n
|
||||
}
|
||||
}
|
||||
if r.During, err = c.snapshot(false); err != nil {
|
||||
return r, err
|
||||
}
|
||||
r.SampledPeakHeapAlloc = max(r.SampledPeakHeapAlloc, r.During.HeapAlloc)
|
||||
r.SampledPeakFileDescriptors = max(r.SampledPeakFileDescriptors, r.During.FileDescriptors)
|
||||
if r.Requests == 0 {
|
||||
return r, errors.New("no requests completed")
|
||||
}
|
||||
r.RequestsPerSecond = float64(r.Requests-r.Errors) / r.ElapsedSeconds
|
||||
r.P50Millis = latencies.percentile(r.Requests, 50)
|
||||
r.P95Millis = latencies.percentile(r.Requests, 95)
|
||||
r.P99Millis = latencies.percentile(r.Requests, 99)
|
||||
r.ServerBytesPerRequest = float64(r.During.TotalAlloc-r.Before.TotalAlloc) / float64(r.Requests)
|
||||
r.ServerAllocsPerRequest = float64(r.During.Mallocs-r.Before.Mallocs) / float64(r.Requests)
|
||||
client.CloseIdleConnections()
|
||||
r.After, err = drained(c)
|
||||
if err == nil {
|
||||
r.AfterReclaim, err = c.snapshot(true)
|
||||
}
|
||||
return r, err
|
||||
}
|
||||
|
||||
func idle(c *child, count int) (result, error) {
|
||||
r := result{Scenario: "idle", Concurrency: count}
|
||||
var err error
|
||||
if r.Before, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
connections := make([]net.Conn, 0, count)
|
||||
defer func() {
|
||||
for _, conn := range connections {
|
||||
conn.Close()
|
||||
}
|
||||
}()
|
||||
for i := 0; i < count; i++ {
|
||||
conn, err := net.DialTimeout("tcp", c.address, 5*time.Second)
|
||||
if err != nil {
|
||||
return r, err
|
||||
}
|
||||
if c.tls {
|
||||
secure := tls.Client(conn, &tls.Config{InsecureSkipVerify: true, ServerName: "localhost"}) // test certificate is self-signed
|
||||
conn = secure
|
||||
}
|
||||
connections = append(connections, conn)
|
||||
conn.SetDeadline(time.Now().Add(5 * time.Second))
|
||||
if _, err := io.WriteString(conn, "GET /health HTTP/1.1\r\nHost: localhost\r\nX-Request-ID: perfCheck\r\n\r\n"); err != nil {
|
||||
return r, err
|
||||
}
|
||||
response, err := http.ReadResponse(bufio.NewReader(conn), &http.Request{Method: http.MethodGet})
|
||||
if err != nil {
|
||||
return r, err
|
||||
}
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || strings.TrimSpace(string(body)) != expectedBody || response.Close {
|
||||
return r, fmt.Errorf("invalid Keep-Alive response: status=%d error=%v", response.StatusCode, readErr)
|
||||
}
|
||||
conn.SetDeadline(time.Time{})
|
||||
}
|
||||
if r.During, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
if r.During.Connections != int64(count) {
|
||||
return r, fmt.Errorf("idle connections: want %d, have %d", count, r.During.Connections)
|
||||
}
|
||||
if r.During.FileDescriptors > 0 && r.Before.FileDescriptors > 0 && r.During.FileDescriptors < r.Before.FileDescriptors+count {
|
||||
return r, fmt.Errorf("idle file descriptors: want at least %d, have %d", r.Before.FileDescriptors+count, r.During.FileDescriptors)
|
||||
}
|
||||
r.IdleHeapDelta = int64(r.During.HeapAlloc) - int64(r.Before.HeapAlloc)
|
||||
r.IdleHeapPerConnection = float64(r.IdleHeapDelta) / float64(count)
|
||||
for _, conn := range connections {
|
||||
conn.Close()
|
||||
}
|
||||
connections = nil
|
||||
r.After, err = drained(c)
|
||||
if err == nil {
|
||||
r.AfterReclaim, err = c.snapshot(true)
|
||||
}
|
||||
return r, err
|
||||
}
|
||||
|
||||
func drained(c *child) (snapshot, error) {
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
s, err := c.snapshot(false)
|
||||
if err != nil {
|
||||
return s, err
|
||||
}
|
||||
if s.Connections == 0 {
|
||||
return c.snapshot(true)
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
return s, fmt.Errorf("%d connections remained after clients closed", s.Connections)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func levels(text string) ([]int, error) {
|
||||
var values []int
|
||||
if text == "" {
|
||||
return values, nil
|
||||
}
|
||||
for _, item := range strings.Split(text, ",") {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(item))
|
||||
if err != nil || n < 1 || n > 4096 {
|
||||
return nil, fmt.Errorf("invalid level %q (expected 1..4096)", item)
|
||||
}
|
||||
values = append(values, n)
|
||||
}
|
||||
return values, nil
|
||||
}
|
||||
|
||||
func compare(duration time.Duration, concurrencyText, idleText string, count int, output string, tlsEnabled bool) error {
|
||||
concurrency, err := levels(concurrencyText)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
idleLevels, err := levels(idleText)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if count < 1 || duration < 200*time.Millisecond || duration > 30*time.Minute || len(concurrency)+len(idleLevels) == 0 {
|
||||
return errors.New("count must be positive, duration 200ms..30m, and at least one scenario selected")
|
||||
}
|
||||
report := report{
|
||||
CreatedAt: time.Now().UTC().Format(time.RFC3339), GoVersion: runtime.Version(),
|
||||
Platform: runtime.GOOS + "/" + runtime.GOARCH, GOMAXPROCS: runtime.GOMAXPROCS(0), Duration: duration.String(), TLS: tlsEnabled,
|
||||
Notes: []string{
|
||||
"Loopback HTTP/1.1 health response, fixed requestId, optional TLS, no access logs or database.",
|
||||
"Each case starts a fresh server process; load client runs in the parent process on the same machine.",
|
||||
"Closed-loop concurrency; throughput is successful responses / actual elapsed time, including drain.",
|
||||
"Latency percentiles include failures and are histogram upper bounds with <=6.25% rounding.",
|
||||
"Memory is server-only runtime.MemStats (not RSS or kernel socket memory); sampling interval 200ms.",
|
||||
"FileDescriptors is read from /proc/self/fd on Linux and omitted on other platforms.",
|
||||
"Idle memory measured after a forced GC; delta includes heap growth amortized across connections, excluding stacks.",
|
||||
"After is after closing clients and one forced GC; afterReclaim is a second GC to observe pooled-cache release.",
|
||||
"Native limits: 8192 connections / 4096 active requests. These cases do not benchmark overload rejection.",
|
||||
},
|
||||
}
|
||||
var caseErrors []error
|
||||
for run := 1; run <= count; run++ {
|
||||
adapters := []string{"nethttp", "native"}
|
||||
if run%2 == 0 {
|
||||
adapters[0], adapters[1] = adapters[1], adapters[0]
|
||||
}
|
||||
for _, scenario := range []struct {
|
||||
name string
|
||||
levels []int
|
||||
}{{"traffic", concurrency}, {"idle", idleLevels}} {
|
||||
for _, level := range scenario.levels {
|
||||
for _, adapter := range adapters {
|
||||
c, err := startChildMode(adapter, tlsEnabled)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var value result
|
||||
if scenario.name == "traffic" {
|
||||
value, err = traffic(c, level, duration)
|
||||
} else {
|
||||
value, err = idle(c, level)
|
||||
}
|
||||
err = errors.Join(err, c.close())
|
||||
value.Adapter, value.Run = adapter, run
|
||||
if err != nil {
|
||||
value.Errors++
|
||||
value.FirstError = err.Error()
|
||||
}
|
||||
report.Results = append(report.Results, value)
|
||||
if value.Errors > 0 {
|
||||
caseErrors = append(caseErrors, fmt.Errorf("%s %s/%d run %d: %s", adapter, scenario.name, level, run, value.FirstError))
|
||||
}
|
||||
if scenario.name == "traffic" {
|
||||
fmt.Printf("%s concurrency=%d run=%d: %.0f req/s p95=%.3fms p99=%.3fms server=%.0f B/req errors=%d\n", adapter, level, run, value.RequestsPerSecond, value.P95Millis, value.P99Millis, value.ServerBytesPerRequest, value.Errors)
|
||||
} else {
|
||||
fmt.Printf("%s idle=%d run=%d: server heap delta=%.2f MiB (%.0f B/conn), goroutines=%d->%d->%d, fds=%d->%d->%d errors=%d\n", adapter, level, run, float64(value.IdleHeapDelta)/(1<<20), value.IdleHeapPerConnection, value.Before.Goroutines, value.During.Goroutines, value.After.Goroutines, value.Before.FileDescriptors, value.During.FileDescriptors, value.After.FileDescriptors, value.Errors)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if output != "" {
|
||||
data, err := json.MarshalIndent(report, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(output, append(data, '\n'), 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("Report: %s\n", output)
|
||||
}
|
||||
return errors.Join(caseErrors...)
|
||||
}
|
||||
75
internal/perfcheck/main_test.go
Normal file
75
internal/perfcheck/main_test.go
Normal file
@@ -0,0 +1,75 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A load generator must never report a fast failure page as good throughput.
|
||||
func TestExchangeRejectsInvalidResponses(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
status int
|
||||
body string
|
||||
requestID string
|
||||
truncated bool
|
||||
wantOK bool
|
||||
}{
|
||||
{"valid", 200, expectedBody, "perfCheck", false, true},
|
||||
{"overload", 503, "server busy", "perfCheck", false, false},
|
||||
{"wrongPayload", 200, `{"data":"wrong"}`, "perfCheck", false, false},
|
||||
{"missingRequestId", 200, expectedBody, "", false, false},
|
||||
{"truncated", 200, expectedBody, "perfCheck", true, false},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("X-Request-ID") != "perfCheck" {
|
||||
t.Error("client did not send fixed requestId")
|
||||
}
|
||||
w.Header().Set("X-Request-ID", tt.requestID)
|
||||
if tt.truncated {
|
||||
w.Header().Set("Content-Length", "4096")
|
||||
}
|
||||
w.WriteHeader(tt.status)
|
||||
_, _ = w.Write([]byte(tt.body))
|
||||
}))
|
||||
defer server.Close()
|
||||
err := exchange(server.Client(), strings.TrimPrefix(server.URL, "http://"))
|
||||
if (err == nil) != tt.wantOK {
|
||||
t.Fatalf("valid=%v, error=%v", tt.wantOK, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExchangeModeTLS(t *testing.T) {
|
||||
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Request-ID", "perfCheck")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(expectedBody))
|
||||
}))
|
||||
defer server.Close()
|
||||
if err := exchangeMode(server.Client(), strings.TrimPrefix(server.URL, "https://"), true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPercentileBounds(t *testing.T) {
|
||||
var h histogram
|
||||
// Known ranks, including a tail much slower than the median.
|
||||
for i := 1; i <= 100; i++ {
|
||||
h.add(time.Duration(i) * time.Millisecond)
|
||||
}
|
||||
for _, p := range []uint64{50, 95, 99} {
|
||||
value := h.percentile(100, p)
|
||||
if value < float64(p) || value > float64(p)*1.0625 {
|
||||
t.Fatalf("p%d=%gms outside advertised bucket rounding", p, value)
|
||||
}
|
||||
}
|
||||
if value := h.percentile(0, 95); value != 0 {
|
||||
t.Fatalf("empty histogram returned %g", value)
|
||||
}
|
||||
}
|
||||
63
internal/storagefixture/server.go
Normal file
63
internal/storagefixture/server.go
Normal file
@@ -0,0 +1,63 @@
|
||||
// Package storagefixture supplies a bounded protocol fixture, not a cloud service.
|
||||
package storagefixture
|
||||
|
||||
import (
|
||||
"crypto/md5"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type value struct {
|
||||
data []byte
|
||||
contentType string
|
||||
etag string
|
||||
}
|
||||
|
||||
func Handler() http.Handler {
|
||||
var mu sync.Mutex
|
||||
values := map[string]value{}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if r.Header.Get("Authorization") == "" && r.URL.Query().Get("X-Amz-Signature") == "" && r.URL.Query().Get("x-oss-signature") == "" {
|
||||
http.Error(w, "signature required", 403)
|
||||
return
|
||||
}
|
||||
key := r.URL.Path
|
||||
switch r.Method {
|
||||
case "PUT":
|
||||
data, err := io.ReadAll(io.LimitReader(r.Body, 1025))
|
||||
if err != nil || len(data) > 1024 {
|
||||
http.Error(w, "too large", 413)
|
||||
return
|
||||
}
|
||||
v := value{data: data, contentType: r.Header.Get("Content-Type"), etag: fmt.Sprintf("%x", md5.Sum(data))}
|
||||
values[key] = v
|
||||
w.Header().Set("ETag", `"`+v.etag+`"`)
|
||||
w.WriteHeader(200)
|
||||
case "DELETE":
|
||||
delete(values, key)
|
||||
w.WriteHeader(204)
|
||||
case "GET", "HEAD":
|
||||
v, ok := values[key]
|
||||
if !ok {
|
||||
w.Header().Set("Content-Type", "application/xml")
|
||||
w.WriteHeader(404)
|
||||
fmt.Fprint(w, `<Error><Code>NoSuchKey</Code><Message>not found</Message></Error>`)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Length", fmt.Sprint(len(v.data)))
|
||||
w.Header().Set("Content-Type", v.contentType)
|
||||
w.Header().Set("ETag", `"`+v.etag+`"`)
|
||||
w.Header().Set("Last-Modified", time.Now().UTC().Format(http.TimeFormat))
|
||||
if r.Method == "GET" {
|
||||
w.Write(v.data)
|
||||
}
|
||||
default:
|
||||
http.Error(w, "unsupported", 405)
|
||||
}
|
||||
})
|
||||
}
|
||||
61
internal/testutil/tls.go
Normal file
61
internal/testutil/tls.go
Normal file
@@ -0,0 +1,61 @@
|
||||
// Package testutil contains fixtures shared by transport integration tests.
|
||||
package testutil
|
||||
|
||||
import (
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"encoding/pem"
|
||||
"math/big"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// CertificateFiles creates a temporary localhost certificate and a client
|
||||
// configuration that verifies it. No insecure TLS bypass is needed in tests.
|
||||
func CertificateFiles(t testing.TB) (certFile, keyFile string, client *tls.Config) {
|
||||
t.Helper()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 120))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
certificate := &x509.Certificate{
|
||||
SerialNumber: serial,
|
||||
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
|
||||
DNSNames: []string{"localhost"}, IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
|
||||
KeyUsage: x509.KeyUsageDigitalSignature,
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, certificate, certificate, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keyDER, err := x509.MarshalPKCS8PrivateKey(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
directory := t.TempDir()
|
||||
certFile, keyFile = filepath.Join(directory, "server.crt"), filepath.Join(directory, "server.key")
|
||||
if err := os.WriteFile(certFile, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(keyFile, pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyDER}), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
leaf, err := x509.ParseCertificate(der)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
roots := x509.NewCertPool()
|
||||
roots.AddCert(leaf)
|
||||
return certFile, keyFile, &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
40
logx/logx.go
40
logx/logx.go
@@ -21,14 +21,14 @@ import (
|
||||
// standard stream. File takes precedence over Output and appends to the named
|
||||
// file.
|
||||
type Config struct {
|
||||
Level string
|
||||
Format string
|
||||
Output string
|
||||
File string
|
||||
Service string
|
||||
Environment string
|
||||
AddSource bool
|
||||
Writer io.Writer
|
||||
Level string `json:"level"`
|
||||
Format string `json:"format"`
|
||||
Output string `json:"output"`
|
||||
File string `json:"file"`
|
||||
Service string `json:"service"`
|
||||
Environment string `json:"environment"`
|
||||
AddSource bool `json:"addSource"`
|
||||
Writer io.Writer `json:"-"`
|
||||
}
|
||||
|
||||
// Default returns the default production-friendly logger configuration.
|
||||
@@ -45,6 +45,7 @@ func Default() Config {
|
||||
type Logger struct {
|
||||
logger *slog.Logger
|
||||
closer *closeState
|
||||
level *slog.LevelVar
|
||||
}
|
||||
|
||||
type closeState struct {
|
||||
@@ -78,7 +79,9 @@ func New(cfg Config) (*Logger, error) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
opts := &slog.HandlerOptions{Level: level, AddSource: cfg.AddSource}
|
||||
dynamicLevel := new(slog.LevelVar)
|
||||
dynamicLevel.Set(level)
|
||||
opts := &slog.HandlerOptions{Level: dynamicLevel, AddSource: cfg.AddSource}
|
||||
var handler slog.Handler
|
||||
if cfg.Format == "text" {
|
||||
handler = slog.NewTextHandler(writer, opts)
|
||||
@@ -97,7 +100,7 @@ func New(cfg Config) (*Logger, error) {
|
||||
if closeWriter != nil {
|
||||
closer = &closeState{fn: closeWriter}
|
||||
}
|
||||
return &Logger{logger: logger, closer: closer}, nil
|
||||
return &Logger{logger: logger, closer: closer, level: dynamicLevel}, nil
|
||||
}
|
||||
|
||||
// Setup replaces the process-wide logger used by the package-level helpers.
|
||||
@@ -152,14 +155,27 @@ func (l *Logger) With(args ...any) *Logger {
|
||||
if l == nil || l.logger == nil {
|
||||
return defaultLogger.Load().With(args...)
|
||||
}
|
||||
return &Logger{logger: l.logger.With(args...), closer: l.closer}
|
||||
return &Logger{logger: l.logger.With(args...), closer: l.closer, level: l.level}
|
||||
}
|
||||
|
||||
func (l *Logger) WithGroup(name string) *Logger {
|
||||
if l == nil || l.logger == nil {
|
||||
return defaultLogger.Load().WithGroup(name)
|
||||
}
|
||||
return &Logger{logger: l.logger.WithGroup(name), closer: l.closer}
|
||||
return &Logger{logger: l.logger.WithGroup(name), closer: l.closer, level: l.level}
|
||||
}
|
||||
|
||||
// SetLevel atomically changes the threshold shared by this logger and children.
|
||||
func (l *Logger) SetLevel(value string) error {
|
||||
if l == nil || l.level == nil {
|
||||
return errors.New("logx: logger does not support dynamic levels")
|
||||
}
|
||||
level, err := parseLevel(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
l.level.Set(level)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Close releases resources owned by the root logger. It is safe to call more
|
||||
|
||||
106
metrics/metrics.go
Normal file
106
metrics/metrics.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package metrics provides a small transport-neutral request metrics collector.
|
||||
// It is intentionally in-memory and dependency-free; applications can expose
|
||||
// Snapshot through any route or export it to their monitoring system.
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
// Snapshot is a point-in-time view of request metrics collected by Metrics.
|
||||
type Snapshot struct {
|
||||
Requests uint64 `json:"requests"`
|
||||
Errors uint64 `json:"errors"`
|
||||
InFlight int64 `json:"inFlight"`
|
||||
TotalDurationMs float64 `json:"totalDurationMs"`
|
||||
StatusCounts map[string]uint64 `json:"statusCounts"`
|
||||
}
|
||||
|
||||
// Metrics collects request counts, status counts, in-flight requests and
|
||||
// cumulative handler time. A zero Metrics value is ready for use.
|
||||
type Metrics struct {
|
||||
requests atomic.Uint64
|
||||
errors atomic.Uint64
|
||||
inFlight atomic.Int64
|
||||
durationNs atomic.Int64
|
||||
statuses [600]atomic.Uint64
|
||||
}
|
||||
|
||||
// New creates an empty metrics collector.
|
||||
func New() *Metrics { return &Metrics{} }
|
||||
|
||||
// Middleware records every request that reaches the middleware chain. Errors
|
||||
// include returned handler errors and responses with a status of 400 or greater.
|
||||
func (m *Metrics) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(ctx *httpx.Context) error {
|
||||
if m == nil {
|
||||
return next(ctx)
|
||||
}
|
||||
m.inFlight.Add(1)
|
||||
started := time.Now()
|
||||
err := next(ctx)
|
||||
status := ctx.Status()
|
||||
if err != nil && !ctx.Responded() {
|
||||
status = apierrors.From(err).Status
|
||||
}
|
||||
m.requests.Add(1)
|
||||
if err != nil || status >= 400 {
|
||||
m.errors.Add(1)
|
||||
}
|
||||
if status >= 0 && status < len(m.statuses) {
|
||||
m.statuses[status].Add(1)
|
||||
}
|
||||
m.durationNs.Add(time.Since(started).Nanoseconds())
|
||||
m.inFlight.Add(-1)
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handler exposes the current snapshot through the standard GoOmni response
|
||||
// envelope. Register it explicitly, for example app.GET("/metrics", m.Handler()).
|
||||
func (m *Metrics) Handler() httpx.Handler {
|
||||
return func(ctx *httpx.Context) error {
|
||||
return ctx.OK(m.Snapshot())
|
||||
}
|
||||
}
|
||||
|
||||
// Snapshot returns a consistent-enough point-in-time view for operational
|
||||
// reporting. Counters may advance while the snapshot is being assembled.
|
||||
func (m *Metrics) Snapshot() Snapshot {
|
||||
if m == nil {
|
||||
return Snapshot{StatusCounts: map[string]uint64{}}
|
||||
}
|
||||
statuses := make(map[string]uint64)
|
||||
for status := 100; status < len(m.statuses); status++ {
|
||||
if count := m.statuses[status].Load(); count > 0 {
|
||||
statuses[itoa(status)] = count
|
||||
}
|
||||
}
|
||||
return Snapshot{
|
||||
Requests: m.requests.Load(),
|
||||
Errors: m.errors.Load(),
|
||||
InFlight: m.inFlight.Load(),
|
||||
TotalDurationMs: float64(m.durationNs.Load()) / float64(time.Millisecond),
|
||||
StatusCounts: statuses,
|
||||
}
|
||||
}
|
||||
|
||||
func itoa(value int) string {
|
||||
if value == 0 {
|
||||
return "0"
|
||||
}
|
||||
var digits [3]byte
|
||||
index := len(digits)
|
||||
for value > 0 {
|
||||
index--
|
||||
digits[index] = byte('0' + value%10)
|
||||
value /= 10
|
||||
}
|
||||
return string(digits[index:])
|
||||
}
|
||||
67
metrics/metrics_test.go
Normal file
67
metrics/metrics_test.go
Normal file
@@ -0,0 +1,67 @@
|
||||
package metrics_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/metrics"
|
||||
)
|
||||
|
||||
func TestMiddlewareCollectsStatusesAndErrors(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/health", nil)
|
||||
first := httptest.NewRecorder()
|
||||
ctx := httpx.NewContext(request, first, nil)
|
||||
if err := collector.Middleware()(func(c *httpx.Context) error { return c.OK("ok") })(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
second := httptest.NewRecorder()
|
||||
ctx = httpx.NewContext(request, second, nil)
|
||||
handled := collector.Middleware()(func(*httpx.Context) error {
|
||||
return apierrors.NotFound("MISSING", "missing")
|
||||
})
|
||||
if err := handled(ctx); err == nil {
|
||||
t.Fatal("expected handler error")
|
||||
}
|
||||
|
||||
snapshot := collector.Snapshot()
|
||||
if snapshot.Requests != 2 || snapshot.Errors != 1 || snapshot.InFlight != 0 || snapshot.StatusCounts["200"] != 1 || snapshot.StatusCounts["404"] != 1 {
|
||||
t.Fatalf("unexpected snapshot: %+v", snapshot)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerReturnsJSONSnapshot(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/metrics", nil)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx := httpx.NewContext(request, recorder, nil)
|
||||
if err := collector.Handler()(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var envelope struct {
|
||||
Code string `json:"code"`
|
||||
Data metrics.Snapshot `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if envelope.Code != "OK" || envelope.Data.StatusCounts == nil {
|
||||
t.Fatalf("unexpected metrics response: %s", recorder.Body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMiddlewarePreservesApplicationErrors(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/", nil)
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
want := errors.New("database unavailable")
|
||||
got := collector.Middleware()(func(*httpx.Context) error { return want })(ctx)
|
||||
if !errors.Is(got, want) {
|
||||
t.Fatalf("middleware replaced error: %v", got)
|
||||
}
|
||||
}
|
||||
242
middleware/idempotency.go
Normal file
242
middleware/idempotency.go
Normal file
@@ -0,0 +1,242 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type IdempotencyConfig struct {
|
||||
Store IdempotencyStore `json:"-"` // Optional shared store; nil keeps local behavior.
|
||||
TTL time.Duration `json:"ttl"`
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"`
|
||||
MaxRequestBytes int64 `json:"maxRequestBytes"`
|
||||
MaxResponseBytes int `json:"maxResponseBytes"`
|
||||
MaxInFlight int `json:"maxInFlight"`
|
||||
// Scope must return a server-verified tenant/user identity. Register auth
|
||||
// before this middleware. Client-provided identity is not authorization.
|
||||
Scope func(*httpx.Context) (string, error) `json:"-"`
|
||||
}
|
||||
|
||||
type replay struct {
|
||||
fingerprint [32]byte
|
||||
status int
|
||||
headers http.Header
|
||||
body []byte
|
||||
expires time.Time
|
||||
running, failed bool
|
||||
size int
|
||||
}
|
||||
type Idempotency struct {
|
||||
mu sync.Mutex
|
||||
config IdempotencyConfig
|
||||
items map[[32]byte]*replay
|
||||
bytes int
|
||||
slots chan struct{}
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewIdempotency(cfg IdempotencyConfig) (*Idempotency, error) {
|
||||
if cfg.Scope == nil || cfg.TTL < 0 || cfg.MaxEntries < 0 || cfg.MaxBytes < 0 || cfg.MaxRequestBytes < 0 || cfg.MaxResponseBytes < 0 || cfg.MaxInFlight < 0 {
|
||||
return nil, errors.New("middleware: invalid idempotency configuration or missing scope")
|
||||
}
|
||||
if cfg.TTL == 0 {
|
||||
cfg.TTL = 10 * time.Minute
|
||||
}
|
||||
if cfg.Store != nil && cfg.TTL > 30*24*time.Hour {
|
||||
return nil, errors.New("middleware: shared idempotency TTL exceeds 30d")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 256
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 16 << 20
|
||||
}
|
||||
if cfg.MaxRequestBytes == 0 {
|
||||
cfg.MaxRequestBytes = 1 << 20
|
||||
}
|
||||
if cfg.MaxResponseBytes == 0 {
|
||||
cfg.MaxResponseBytes = 64 << 10
|
||||
}
|
||||
if cfg.MaxInFlight == 0 {
|
||||
cfg.MaxInFlight = 32
|
||||
}
|
||||
if cfg.MaxRequestBytes > 1<<30 {
|
||||
return nil, errors.New("middleware: maxRequestBytes exceeds 1 GiB")
|
||||
}
|
||||
return &Idempotency{config: cfg, items: make(map[[32]byte]*replay), slots: make(chan struct{}, cfg.MaxInFlight), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (i *Idempotency) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
r := c.Request()
|
||||
if r.Method == "GET" || r.Method == "HEAD" || r.Method == "OPTIONS" {
|
||||
return next(c)
|
||||
}
|
||||
select {
|
||||
case i.slots <- struct{}{}:
|
||||
defer func() { <-i.slots }()
|
||||
default:
|
||||
return apierrors.New(503, "IDEMPOTENCY_BUSY", "idempotency capacity reached")
|
||||
}
|
||||
key := r.Header.Get("Idempotency-Key")
|
||||
if len(r.Header.Values("Idempotency-Key")) != 1 || len(key) < 1 || len(key) > 128 || strings.IndexFunc(key, func(r rune) bool { return r < 33 || r > 126 }) >= 0 {
|
||||
return apierrors.BadRequest("INVALID_IDEMPOTENCY_KEY", "Idempotency-Key must contain 1..128 printable ASCII characters")
|
||||
}
|
||||
scope, err := i.config.Scope(c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if scope == "" || len(scope) > 1024 {
|
||||
return apierrors.Unauthorized("IDEMPOTENCY_SCOPE_REQUIRED", "authenticated scope is required")
|
||||
}
|
||||
if r.ContentLength > i.config.MaxRequestBytes {
|
||||
return apierrors.New(413, "BODY_TOO_LARGE", "request body too large")
|
||||
}
|
||||
var body []byte
|
||||
if r.Body != nil {
|
||||
body, err = io.ReadAll(io.LimitReader(r.Body, i.config.MaxRequestBytes+1))
|
||||
if err != nil {
|
||||
return apierrors.BadRequest("INVALID_BODY", "unable to read request body").WithCause(err)
|
||||
}
|
||||
if int64(len(body)) > i.config.MaxRequestBytes {
|
||||
return apierrors.New(413, "BODY_TOO_LARGE", "request body too large")
|
||||
}
|
||||
}
|
||||
if err := c.Context().Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
originalBody := r.Body
|
||||
r.Body = io.NopCloser(bytes.NewReader(body))
|
||||
defer func() { r.Body = originalBody }()
|
||||
parts, _ := json.Marshal([]string{r.Method, r.URL.RequestURI(), r.Header.Get("Content-Type")})
|
||||
hash := sha256.New()
|
||||
hash.Write(parts)
|
||||
hash.Write(body)
|
||||
var fingerprint [32]byte
|
||||
copy(fingerprint[:], hash.Sum(nil))
|
||||
namespace, _ := json.Marshal([]string{scope, key})
|
||||
id := sha256.Sum256(namespace)
|
||||
if i.config.Store != nil {
|
||||
return i.shared(c, next, id, fingerprint)
|
||||
}
|
||||
i.mu.Lock()
|
||||
now := i.now()
|
||||
for k, v := range i.items {
|
||||
if !v.running && !now.Before(v.expires) {
|
||||
delete(i.items, k)
|
||||
i.bytes -= v.size
|
||||
}
|
||||
}
|
||||
if saved := i.items[id]; saved != nil {
|
||||
if saved.fingerprint != fingerprint {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(409, "IDEMPOTENCY_CONFLICT", "key was used with a different request")
|
||||
}
|
||||
if saved.running || saved.failed {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(409, "IDEMPOTENCY_UNAVAILABLE", "request is in progress or its outcome cannot be replayed")
|
||||
}
|
||||
status, headers, data := saved.status, saved.headers.Clone(), append([]byte(nil), saved.body...)
|
||||
i.mu.Unlock()
|
||||
headers.Set("Idempotency-Replayed", "true")
|
||||
return c.WriteResponse(status, headers, data)
|
||||
}
|
||||
if len(i.items) >= i.config.MaxEntries {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(503, "IDEMPOTENCY_BUSY", "idempotency capacity reached")
|
||||
}
|
||||
saved := &replay{fingerprint: fingerprint, running: true}
|
||||
i.items[id] = saved
|
||||
i.mu.Unlock()
|
||||
buffer := &boundedResponse{headers: c.Raw().Header().Clone(), limit: i.config.MaxResponseBytes}
|
||||
// Always retain an outcome tombstone on panic/error/overflow. Deleting
|
||||
// the key could execute a partially committed business operation twice.
|
||||
defer func() {
|
||||
i.mu.Lock()
|
||||
defer i.mu.Unlock()
|
||||
saved.running = false
|
||||
saved.failed = true
|
||||
saved.expires = i.now().Add(i.config.TTL)
|
||||
size := buffer.body.Len()
|
||||
for k, values := range buffer.headers {
|
||||
size += len(k)
|
||||
for _, v := range values {
|
||||
size += len(v)
|
||||
}
|
||||
}
|
||||
if buffer.complete && buffer.err == nil && buffer.status < 500 && buffer.headers.Get("Set-Cookie") == "" && !strings.Contains(strings.ToLower(buffer.headers.Get("Cache-Control")), "no-store") && size <= i.config.MaxBytes-i.bytes {
|
||||
saved.failed = false
|
||||
saved.status = buffer.status
|
||||
saved.headers = buffer.headers.Clone()
|
||||
saved.body = append([]byte(nil), buffer.body.Bytes()...)
|
||||
saved.size = size
|
||||
i.bytes += size
|
||||
}
|
||||
}()
|
||||
child := c.Clone(c.Context(), buffer)
|
||||
err = next(child)
|
||||
// Keep an ambiguous outcome tombstone, but let an outer Timeout
|
||||
// middleware turn deadline expiry into 504 before committing bytes.
|
||||
if ctxErr := child.Context().Err(); ctxErr != nil {
|
||||
return ctxErr
|
||||
}
|
||||
if err != nil {
|
||||
child.WriteError(err)
|
||||
}
|
||||
if buffer.err != nil {
|
||||
return apierrors.Internal(buffer.err)
|
||||
}
|
||||
if buffer.status == 0 {
|
||||
buffer.status = 200
|
||||
}
|
||||
buffer.complete = true
|
||||
return c.WriteResponse(buffer.status, buffer.headers, buffer.body.Bytes())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Intentionally lacks Flush/Hijack: only buffered, bounded endpoints are valid.
|
||||
type boundedResponse struct {
|
||||
headers http.Header
|
||||
status, limit int
|
||||
body bytes.Buffer
|
||||
err error
|
||||
complete bool
|
||||
}
|
||||
|
||||
func (b *boundedResponse) Header() http.Header { return b.headers }
|
||||
func (b *boundedResponse) WriteHeader(status int) {
|
||||
if status < 200 {
|
||||
b.err = errors.New("idempotency: informational/upgrade responses are unsupported")
|
||||
return
|
||||
}
|
||||
if b.status == 0 {
|
||||
b.status = status
|
||||
}
|
||||
}
|
||||
func (b *boundedResponse) Write(data []byte) (int, error) {
|
||||
if b.status == 0 {
|
||||
b.status = 200
|
||||
}
|
||||
if b.err != nil {
|
||||
return 0, b.err
|
||||
}
|
||||
if len(data) > b.limit-b.body.Len() {
|
||||
b.err = fmt.Errorf("idempotency: response exceeds %d bytes", b.limit)
|
||||
return 0, b.err
|
||||
}
|
||||
return b.body.Write(data)
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package middleware
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
@@ -64,19 +63,9 @@ func AccessLog() httpx.Middleware {
|
||||
"route", httpx.RoutePattern(ctx.Context()),
|
||||
"status", status,
|
||||
"durationMs", time.Since(started).Seconds()*1000,
|
||||
"clientIp", clientIP(ctx.Request()),
|
||||
"clientIp", ctx.ClientIP(),
|
||||
)
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func clientIP(request *http.Request) string {
|
||||
if request == nil {
|
||||
return ""
|
||||
}
|
||||
if value := request.Header.Get("X-Forwarded-For"); value != "" {
|
||||
return value
|
||||
}
|
||||
return request.RemoteAddr
|
||||
}
|
||||
|
||||
@@ -55,3 +55,34 @@ func TestAccessLogReportsCommittedStatus(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccessLogUsesVerifiedClientIP(t *testing.T) {
|
||||
loggerOutput := bytes.Buffer{}
|
||||
logger, err := logx.New(logx.Config{Format: "json", Writer: &loggerOutput})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
logx.SetDefault(logger)
|
||||
defer logx.SetDefault(nil)
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := httptest.NewRequest("GET", "/health", nil)
|
||||
request.RemoteAddr = "127.0.0.1:443"
|
||||
request.Header.Set("X-Forwarded-For", "198.51.100.7")
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), resolver))
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
if err := middleware.AccessLog()(func(c *httpx.Context) error { return c.OK("ok") })(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var entry struct {
|
||||
ClientIP string `json:"clientIp"`
|
||||
}
|
||||
if err := json.Unmarshal(loggerOutput.Bytes(), &entry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entry.ClientIP != "198.51.100.7" {
|
||||
t.Fatalf("expected verified client IP, got %q", entry.ClientIP)
|
||||
}
|
||||
}
|
||||
|
||||
227
middleware/protection_test.go
Normal file
227
middleware/protection_test.go
Normal file
@@ -0,0 +1,227 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
func invoke(h httpx.Handler, method, body, scope, key string) *httptest.ResponseRecorder {
|
||||
r := httptest.NewRequest(method, "/records?version=1", strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
if key != "" {
|
||||
r.Header.Set("Idempotency-Key", key)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
c := httpx.NewContext(r, w, nil)
|
||||
c.Set("verifiedIdentity", scope)
|
||||
if err := h(c); err != nil {
|
||||
c.WriteError(err)
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
func identity(c *httpx.Context) (string, error) {
|
||||
v, _ := c.Get("verifiedIdentity")
|
||||
s, _ := v.(string)
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func TestRateLimitRefillCapacityAndIdleExpiry(t *testing.T) {
|
||||
l, err := NewRateLimiter(RateLimitConfig{Rate: 2, Burst: 2, MaxKeys: 1, IdleTTL: time.Second, Key: func(c *httpx.Context) string { s, _ := identity(c); return s }})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Unix(100, 0)
|
||||
l.now = func() time.Time { return now }
|
||||
h := l.Middleware()(func(c *httpx.Context) error { return c.NoContent() })
|
||||
for _, status := range []int{204, 204, 429} {
|
||||
w := invoke(h, "GET", "", "a", "")
|
||||
if w.Code != status {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if status == 429 && w.Header().Get("Retry-After") != "1" {
|
||||
t.Fatal(w.Header())
|
||||
}
|
||||
}
|
||||
if w := invoke(h, "GET", "", "b", ""); w.Code != 503 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(500 * time.Millisecond)
|
||||
if w := invoke(h, "GET", "", "a", ""); w.Code != 204 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if w := invoke(h, "GET", "", "b", ""); w.Code != 204 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if _, err := NewRateLimiter(RateLimitConfig{Rate: 1, Burst: 2, IdleTTL: time.Second}); err == nil {
|
||||
t.Fatal("early bucket expiry accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimitConcurrentBurst(t *testing.T) {
|
||||
l, _ := NewRateLimiter(RateLimitConfig{Rate: 1, Burst: 7})
|
||||
l.now = func() time.Time { return time.Unix(100, 0) }
|
||||
var allowed atomic.Int32
|
||||
var wg sync.WaitGroup
|
||||
for n := 0; n < 100; n++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
delay, full := l.allow("same")
|
||||
if delay == 0 && !full {
|
||||
allowed.Add(1)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if allowed.Load() != 7 {
|
||||
t.Fatal(allowed.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeoutCooperativeAndCommittedResponse(t *testing.T) {
|
||||
for _, committed := range []bool{false, true} {
|
||||
h := Timeout(5 * time.Millisecond)(func(c *httpx.Context) error {
|
||||
if committed {
|
||||
_ = c.Accepted("accepted")
|
||||
}
|
||||
<-c.Context().Done()
|
||||
return c.Context().Err()
|
||||
})
|
||||
w := invoke(h, "GET", "", "", "")
|
||||
want := 504
|
||||
if committed {
|
||||
want = 202
|
||||
}
|
||||
if w.Code != want {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyReplayScopeConflictAndExpiry(t *testing.T) {
|
||||
i, err := NewIdempotency(IdempotencyConfig{Scope: identity, TTL: time.Minute})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Unix(100, 0)
|
||||
i.now = func() time.Time { return now }
|
||||
calls := 0
|
||||
h := i.Middleware()(func(c *httpx.Context) error { calls++; return c.Created(map[string]int{"calls": calls}) })
|
||||
first := invoke(h, "POST", `{"name":"one"}`, "alice", "key1")
|
||||
replay := invoke(h, "POST", `{"name":"one"}`, "alice", "key1")
|
||||
if first.Code != 201 || replay.Code != 201 || first.Body.String() != replay.Body.String() || replay.Header().Get("Idempotency-Replayed") != "true" || calls != 1 {
|
||||
t.Fatal(first, replay, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", `{"name":"two"}`, "alice", "key1"); w.Code != 409 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", `{"name":"one"}`, "bob", "key1"); w.Code != 201 || calls != 2 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", `{}`, "", "key1"); w.Code != 401 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", `{}`, "alice", ""); w.Code != 400 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(time.Minute)
|
||||
if w := invoke(h, "POST", `{"name":"one"}`, "alice", "key1"); w.Code != 201 || calls != 3 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyActiveClaimNeverExpires(t *testing.T) {
|
||||
i, _ := NewIdempotency(IdempotencyConfig{Scope: identity, TTL: time.Nanosecond, MaxInFlight: 2})
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
h := i.Middleware()(func(c *httpx.Context) error { calls.Add(1); close(started); <-release; return c.Created("saved") })
|
||||
done := make(chan *httptest.ResponseRecorder, 1)
|
||||
go func() { done <- invoke(h, "POST", "{}", "alice", "key") }()
|
||||
<-started
|
||||
for n := 0; n < 30; n++ {
|
||||
if w := invoke(h, "POST", "{}", "alice", "key"); w.Code != 409 {
|
||||
t.Errorf("duplicate status %d", w.Code)
|
||||
}
|
||||
}
|
||||
close(release)
|
||||
if w := <-done; w.Code != 201 || calls.Load() != 1 {
|
||||
t.Fatal(w.Code, calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyBoundsAndAmbiguousOutcomes(t *testing.T) {
|
||||
for _, mode := range []string{"panic", "error", "overflow", "cookie", "noStore", "memory"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
cfg := IdempotencyConfig{Scope: identity, MaxRequestBytes: 8, MaxResponseBytes: 128, MaxEntries: 1}
|
||||
if mode == "memory" {
|
||||
cfg.MaxBytes = 1
|
||||
}
|
||||
i, _ := NewIdempotency(cfg)
|
||||
calls := 0
|
||||
h := Recovery()(i.Middleware()(func(c *httpx.Context) error {
|
||||
calls++
|
||||
switch mode {
|
||||
case "panic":
|
||||
panic("failure")
|
||||
case "error":
|
||||
return errors.New("failed")
|
||||
case "overflow":
|
||||
return c.OK(strings.Repeat("x", 200))
|
||||
case "cookie":
|
||||
c.SetResponseHeader("Set-Cookie", "session=secret")
|
||||
case "noStore":
|
||||
c.SetResponseHeader("Cache-Control", "no-store")
|
||||
}
|
||||
return c.Created("ok")
|
||||
}))
|
||||
if w := invoke(h, "POST", "123456789", "alice", "key"); w.Code != 413 || calls != 0 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
first := invoke(h, "POST", "{}", "alice", "key")
|
||||
if mode == "panic" || mode == "error" || mode == "overflow" {
|
||||
if first.Code != 500 {
|
||||
t.Fatal(first.Code)
|
||||
}
|
||||
} else if first.Code != 201 {
|
||||
t.Fatal(first.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", "{}", "alice", "key"); w.Code != 409 || calls != 1 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", "{}", "alice", "newKey"); w.Code != 503 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyInFlightLimitAndTimeoutComposition(t *testing.T) {
|
||||
i, _ := NewIdempotency(IdempotencyConfig{Scope: identity, MaxInFlight: 1})
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
h := i.Middleware()(func(c *httpx.Context) error { close(started); <-release; return c.NoContent() })
|
||||
done := make(chan struct{})
|
||||
go func() { defer close(done); invoke(h, "POST", "{}", "alice", "one") }()
|
||||
<-started
|
||||
if w := invoke(h, "POST", "{}", "alice", "two"); w.Code != 503 {
|
||||
t.Error(w.Code)
|
||||
}
|
||||
close(release)
|
||||
<-done
|
||||
timed := Timeout(5 * time.Millisecond)(i.Middleware()(func(c *httpx.Context) error { <-c.Context().Done(); return context.DeadlineExceeded }))
|
||||
if w := invoke(timed, "POST", "{}", "alice", "timeout"); w.Code != 504 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if w := invoke(timed, "POST", "{}", "alice", "timeout"); w.Code != 409 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
}
|
||||
120
middleware/rateLimit.go
Normal file
120
middleware/rateLimit.go
Normal file
@@ -0,0 +1,120 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"errors"
|
||||
"math"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type RateLimitConfig struct {
|
||||
Backend RateLimitBackend `json:"-"`
|
||||
Rate float64 `json:"rate"` // Tokens per second, per key.
|
||||
Burst int `json:"burst"`
|
||||
MaxKeys int `json:"maxKeys"`
|
||||
IdleTTL time.Duration `json:"idleTtl"`
|
||||
Key func(*httpx.Context) string `json:"-"` // Default: trusted ClientIP.
|
||||
}
|
||||
|
||||
type bucket struct {
|
||||
key string
|
||||
tokens float64
|
||||
updated time.Time
|
||||
}
|
||||
type RateLimiter struct {
|
||||
mu sync.Mutex
|
||||
config RateLimitConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewRateLimiter(cfg RateLimitConfig) (*RateLimiter, error) {
|
||||
if math.IsNaN(cfg.Rate) || math.IsInf(cfg.Rate, 0) || cfg.Rate <= 0 || cfg.Burst < 1 || cfg.MaxKeys < 0 || cfg.IdleTTL < 0 {
|
||||
return nil, errors.New("middleware: invalid rate limit configuration")
|
||||
}
|
||||
refill := float64(cfg.Burst) / cfg.Rate
|
||||
if refill > (24 * time.Hour).Seconds() {
|
||||
return nil, errors.New("middleware: bucket refill time must not exceed 24h")
|
||||
}
|
||||
minimum := time.Duration(math.Ceil(refill * float64(time.Second)))
|
||||
if cfg.IdleTTL == 0 {
|
||||
cfg.IdleTTL = max(5*time.Minute, minimum)
|
||||
}
|
||||
if cfg.IdleTTL < minimum {
|
||||
return nil, errors.New("middleware: idleTtl must allow a full bucket refill")
|
||||
}
|
||||
if cfg.MaxKeys == 0 {
|
||||
cfg.MaxKeys = 10000
|
||||
}
|
||||
if cfg.Key == nil {
|
||||
cfg.Key = func(c *httpx.Context) string { return c.ClientIP() }
|
||||
}
|
||||
return &RateLimiter{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (l *RateLimiter) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
key := l.config.Key(c)
|
||||
if key == "" || len(key) > 1024 {
|
||||
return apierrors.BadRequest("INVALID_RATE_KEY", "invalid rate limit key")
|
||||
}
|
||||
var delay time.Duration
|
||||
var full bool
|
||||
if l.config.Backend != nil {
|
||||
var err error
|
||||
delay, err = l.config.Backend.Take(c.Context(), key)
|
||||
if err != nil {
|
||||
return apierrors.New(503, "RATE_LIMIT_UNAVAILABLE", "rate limiter unavailable").WithCause(err)
|
||||
}
|
||||
} else {
|
||||
delay, full = l.allow(key)
|
||||
}
|
||||
if full {
|
||||
return apierrors.New(503, "RATE_LIMIT_CAPACITY", "rate limiter capacity reached")
|
||||
}
|
||||
if delay > 0 {
|
||||
c.SetResponseHeader("Retry-After", strconv.Itoa(max(1, int(math.Ceil(delay.Seconds())))))
|
||||
return apierrors.New(429, "RATE_LIMITED", "too many requests")
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (l *RateLimiter) allow(key string) (time.Duration, bool) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
now := l.now()
|
||||
e := l.items[key]
|
||||
if e == nil {
|
||||
for l.lru.Len() > 0 {
|
||||
old := l.lru.Back()
|
||||
if now.Sub(old.Value.(*bucket).updated) < l.config.IdleTTL {
|
||||
break
|
||||
}
|
||||
delete(l.items, old.Value.(*bucket).key)
|
||||
l.lru.Remove(old)
|
||||
}
|
||||
if len(l.items) >= l.config.MaxKeys {
|
||||
return 0, true
|
||||
}
|
||||
e = l.lru.PushFront(&bucket{key: key, tokens: float64(l.config.Burst), updated: now})
|
||||
l.items[key] = e
|
||||
}
|
||||
b := e.Value.(*bucket)
|
||||
b.tokens = min(float64(l.config.Burst), b.tokens+max(0, now.Sub(b.updated).Seconds())*l.config.Rate)
|
||||
b.updated = now
|
||||
l.lru.MoveToFront(e)
|
||||
if b.tokens >= 1 {
|
||||
b.tokens--
|
||||
return 0, false
|
||||
}
|
||||
return time.Duration(math.Ceil((1 - b.tokens) / l.config.Rate * float64(time.Second))), false
|
||||
}
|
||||
178
middleware/redis/store.go
Normal file
178
middleware/redis/store.go
Normal file
@@ -0,0 +1,178 @@
|
||||
// Package redis supplies atomic shared middleware state in one Redis hash slot.
|
||||
// Use a dedicated namespace and a noeviction Redis deployment for correctness.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type IdempotencyConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxRecordBytes int `json:"maxRecordBytes"`
|
||||
}
|
||||
type IdempotencyStore struct {
|
||||
client goredis.Scripter
|
||||
keys []string
|
||||
cfg IdempotencyConfig
|
||||
signature string
|
||||
}
|
||||
|
||||
func namespace(prefix, kind string) (string, error) {
|
||||
if prefix == "" || len(prefix) > 256 || strings.ContainsAny(prefix, "{}\r\n") {
|
||||
return "", errors.New("middleware/redis: nonempty bounded namespace without braces required")
|
||||
}
|
||||
sum := sha256.Sum256([]byte(prefix))
|
||||
return "goOmni:{" + hex.EncodeToString(sum[:]) + "}:" + kind, nil
|
||||
}
|
||||
func NewIdempotencyStore(client goredis.Scripter, prefix string, cfg IdempotencyConfig) (*IdempotencyStore, error) {
|
||||
if client == nil || cfg.MaxEntries < 0 || cfg.MaxRecordBytes < 0 {
|
||||
return nil, errors.New("middleware/redis: invalid configuration")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 128
|
||||
}
|
||||
if cfg.MaxRecordBytes == 0 {
|
||||
cfg.MaxRecordBytes = 128 << 10
|
||||
}
|
||||
if cfg.MaxEntries > 100000 || cfg.MaxRecordBytes < 512 || cfg.MaxRecordBytes > 1<<20 {
|
||||
return nil, errors.New("middleware/redis: capacity outside supported bounds")
|
||||
}
|
||||
base, err := namespace(prefix, "idempotency")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &IdempotencyStore{client: client, keys: []string{base + ":records", base + ":expiry", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%d:%d", cfg.MaxEntries, cfg.MaxRecordBytes)}, nil
|
||||
}
|
||||
|
||||
var claim = goredis.NewScript(`
|
||||
local config=redis.call('GET',KEYS[3])
|
||||
if config and config~=ARGV[5] then return redis.error_reply('configuration mismatch') end
|
||||
if not config then redis.call('SET',KEYS[3],ARGV[5]) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now,'LIMIT',0,128)
|
||||
for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
|
||||
local expiry=redis.call('ZSCORE',KEYS[2],ARGV[1]);if expiry and tonumber(expiry)<=now then redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]) end
|
||||
local value=redis.call('HGET',KEYS[1],ARGV[1]);if value then return {0,value} end
|
||||
if redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return redis.error_reply('idempotency capacity reached') end
|
||||
value=cjson.encode({fingerprint=ARGV[2],owner=ARGV[3],state='running'})
|
||||
redis.call('HSET',KEYS[1],ARGV[1],value);return {1,value}
|
||||
`)
|
||||
|
||||
func (s *IdempotencyStore) Claim(ctx context.Context, key, fp, owner string) (middleware.IdempotencyRecord, bool, error) {
|
||||
if len(key) != 64 || len(fp) != 64 || len(owner) != 32 {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim")
|
||||
}
|
||||
result, err := claim.Run(ctx, s.client, s.keys, key, fp, owner, s.cfg.MaxEntries, s.signature).Slice()
|
||||
if err != nil {
|
||||
return middleware.IdempotencyRecord{}, false, err
|
||||
}
|
||||
if len(result) != 2 {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim result")
|
||||
}
|
||||
value, ok := result[1].(string)
|
||||
if !ok || len(value) > s.cfg.MaxRecordBytes {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid record")
|
||||
}
|
||||
var record middleware.IdempotencyRecord
|
||||
err = json.Unmarshal([]byte(value), &record)
|
||||
return record, result[0] == int64(1), err
|
||||
}
|
||||
|
||||
var complete = goredis.NewScript(`
|
||||
local current=redis.call('HGET',KEYS[1],ARGV[1]);if not current then return redis.error_reply('claim missing') end
|
||||
local value=cjson.decode(current)
|
||||
if value.owner~=ARGV[2] or value.state~='running' then return redis.error_reply('claim owner/state changed') end
|
||||
local response=ARGV[3]
|
||||
if response~='' then value.state='completed';value.response=cjson.decode(response) else value.state='failed' end
|
||||
local encoded=cjson.encode(value)
|
||||
if string.len(encoded)>tonumber(ARGV[5]) then value.state='failed';value.response=nil;encoded=cjson.encode(value) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
redis.call('HSET',KEYS[1],ARGV[1],encoded);redis.call('ZADD',KEYS[2],now+tonumber(ARGV[4]),ARGV[1]);return 1
|
||||
`)
|
||||
|
||||
func (s *IdempotencyStore) Complete(ctx context.Context, key, owner string, response *middleware.StoredResponse, ttl time.Duration) error {
|
||||
if ttl <= 0 || ttl > 30*24*time.Hour {
|
||||
return errors.New("middleware/redis: TTL must be positive and <=30d")
|
||||
}
|
||||
var encoded []byte
|
||||
var err error
|
||||
if response != nil {
|
||||
encoded, err = json.Marshal(response)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(encoded) > s.cfg.MaxRecordBytes {
|
||||
encoded = nil
|
||||
}
|
||||
}
|
||||
return complete.Run(ctx, s.client, s.keys, key, owner, string(encoded), max(1, ttl.Milliseconds()), s.cfg.MaxRecordBytes).Err()
|
||||
}
|
||||
|
||||
// Resolve removes only a matching non-replayable claim after the operator has
|
||||
// reconciled the business outcome. Never invoke automatically on a timeout.
|
||||
func (s *IdempotencyStore) Resolve(ctx context.Context, key, owner string) error {
|
||||
return goredis.NewScript(`local v=redis.call('HGET',KEYS[1],ARGV[1]);if not v then return 0 end;local r=cjson.decode(v);if r.owner~=ARGV[2] or r.state=='completed' then return redis.error_reply('claim cannot be resolved') end;redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]);return 1`).Run(ctx, s.client, s.keys, key, owner).Err()
|
||||
}
|
||||
|
||||
type limiter struct {
|
||||
client goredis.Scripter
|
||||
keys []string
|
||||
cfg middleware.RateLimitConfig
|
||||
signature string
|
||||
}
|
||||
|
||||
func NewRateLimiter(client goredis.Scripter, prefix string, cfg middleware.RateLimitConfig) (*middleware.RateLimiter, error) {
|
||||
if client == nil {
|
||||
return nil, errors.New("middleware/redis: client required")
|
||||
}
|
||||
if cfg.MaxKeys == 0 {
|
||||
cfg.MaxKeys = 10000
|
||||
}
|
||||
if cfg.IdleTTL == 0 {
|
||||
cfg.IdleTTL = 24 * time.Hour
|
||||
}
|
||||
if _, err := middleware.NewRateLimiter(cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
base, err := namespace(prefix, "rate")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
backend := &limiter{client: client, keys: []string{base + ":buckets", base + ":seen", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%g:%d:%d:%d", cfg.Rate, cfg.Burst, cfg.MaxKeys, cfg.IdleTTL.Milliseconds())}
|
||||
cfg.Backend = backend
|
||||
return middleware.NewRateLimiter(cfg)
|
||||
}
|
||||
|
||||
var take = goredis.NewScript(`
|
||||
local config=redis.call('GET',KEYS[3]);if config and config~=ARGV[6] then return redis.error_reply('configuration mismatch') end
|
||||
if not config then redis.call('SET',KEYS[3],ARGV[6]) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now-tonumber(ARGV[5]),'LIMIT',0,128)
|
||||
for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
|
||||
local value=redis.call('HGET',KEYS[1],ARGV[1]);local tokens=tonumber(ARGV[3])
|
||||
if value then local b=cjson.decode(value);tokens=math.min(tokens,b.tokens+math.max(0,now-b.time)*tonumber(ARGV[2])/1000)
|
||||
elseif redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return -1 end
|
||||
local delay=0;if tokens>=1 then tokens=tokens-1 else delay=math.ceil((1-tokens)*1000/tonumber(ARGV[2])) end
|
||||
redis.call('HSET',KEYS[1],ARGV[1],cjson.encode({tokens=tokens,time=now}));redis.call('ZADD',KEYS[2],now,ARGV[1]);return delay
|
||||
`)
|
||||
|
||||
func (l *limiter) Take(ctx context.Context, key string) (time.Duration, error) {
|
||||
sum := sha256.Sum256([]byte(key))
|
||||
delay, err := take.Run(ctx, l.client, l.keys, hex.EncodeToString(sum[:]), l.cfg.Rate, l.cfg.Burst, l.cfg.MaxKeys, l.cfg.IdleTTL.Milliseconds(), l.signature).Int64()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if delay < 0 {
|
||||
return 0, middleware.ErrLimitCapacity
|
||||
}
|
||||
return time.Duration(delay) * time.Millisecond, nil
|
||||
}
|
||||
145
middleware/redis/store_test.go
Normal file
145
middleware/redis/store_test.go
Normal file
@@ -0,0 +1,145 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSharedRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("requires isolated Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 8, MaxRetries: -1, ContextTimeoutEnabled: true, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
prefix := fmt.Sprint("shared:", time.Now().UnixNano())
|
||||
s, err := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Del(context.Background(), s.keys...)
|
||||
key, fp := strings.Repeat("a", 64), strings.Repeat("b", 64)
|
||||
var claims atomic.Int32
|
||||
var wg sync.WaitGroup
|
||||
owners := make(chan string, 32)
|
||||
for i := 0; i < 32; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
owner := fmt.Sprintf("%032x", i)
|
||||
_, ok, err := s.Claim(ctx, key, fp, owner)
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
if ok {
|
||||
claims.Add(1)
|
||||
owners <- owner
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
if claims.Load() != 1 {
|
||||
t.Fatal(claims.Load())
|
||||
}
|
||||
owner := <-owners
|
||||
if err = s.Complete(ctx, key, "wrong", nil, time.Minute); err == nil {
|
||||
t.Fatal("foreign completion accepted")
|
||||
}
|
||||
if err = s.Complete(ctx, key, owner, &middleware.StoredResponse{Status: 201, Headers: http.Header{}, Body: []byte{0, 1, 255}}, time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
record, ok, err := s.Claim(ctx, key, fp, strings.Repeat("c", 32))
|
||||
if err != nil || ok || record.State != "completed" || len(record.Response.Body) != 3 {
|
||||
t.Fatal(record, ok, err)
|
||||
}
|
||||
// Two independently constructed apps use the same backend, so only one runs.
|
||||
var executions atomic.Int32
|
||||
makeApp := func() *omni.App {
|
||||
backend, _ := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
|
||||
idem, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Store: backend, Scope: func(*httpx.Context) (string, error) { return "verifiedTenant", nil }})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a := omni.New(omni.WithLogger(false))
|
||||
a.POST("/submit", func(c *httpx.Context) error { executions.Add(1); return c.OK("saved") }, idem.Middleware())
|
||||
return a
|
||||
}
|
||||
a, b := makeApp(), makeApp()
|
||||
request := func(a *omni.App, body string) *httptest.ResponseRecorder {
|
||||
r := httptest.NewRequest("POST", "/submit", strings.NewReader(body))
|
||||
r.Header.Set("Idempotency-Key", "stableKey")
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, r)
|
||||
return w
|
||||
}
|
||||
first, second := request(a, `{}`), request(b, `{}`)
|
||||
if first.Code != 200 || second.Code != 200 || first.Body.String() != second.Body.String() || second.Header().Get("Idempotency-Replayed") != "true" || executions.Load() != 1 {
|
||||
t.Fatal(first.Code, second.Code, second.Body.String(), executions.Load())
|
||||
}
|
||||
if request(b, `{"changed":true}`).Code != 409 {
|
||||
t.Fatal("conflict accepted")
|
||||
}
|
||||
// Shared limits do not multiply when a second replica is added.
|
||||
limitPrefix := prefix + "limit"
|
||||
cfg := middleware.RateLimitConfig{Rate: .01, Burst: 2, Key: func(*httpx.Context) string { return "tenant" }}
|
||||
r1, _ := NewRateLimiter(client, limitPrefix, cfg)
|
||||
r2, _ := NewRateLimiter(client, limitPrefix, cfg)
|
||||
base, _ := namespace(limitPrefix, "rate")
|
||||
defer client.Del(context.Background(), base+":buckets", base+":seen", base+":config")
|
||||
limited := func(l *middleware.RateLimiter) int {
|
||||
a := omni.New(omni.WithLogger(false))
|
||||
a.GET("/", func(c *httpx.Context) error { return c.OK("ok") }, l.Middleware())
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
|
||||
return w.Code
|
||||
}
|
||||
if limited(r1) != 200 || limited(r2) != 200 || limited(r1) != 429 {
|
||||
t.Fatal("limits multiplied across instances")
|
||||
}
|
||||
// Redis failure must fail closed before executing a new request.
|
||||
client.Close()
|
||||
if status := limited(r2); status != 503 {
|
||||
t.Fatal(status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpiredTargetBeyondCleanupBatch(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("requires isolated Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address})
|
||||
defer client.Close()
|
||||
s, _ := NewIdempotencyStore(client, fmt.Sprint("expiry:", time.Now().UnixNano()), IdempotencyConfig{MaxEntries: 256})
|
||||
defer client.Del(context.Background(), s.keys...)
|
||||
ctx := context.Background()
|
||||
owner := strings.Repeat("1", 32)
|
||||
fp := strings.Repeat("2", 64)
|
||||
for i := 0; i < 140; i++ {
|
||||
key := fmt.Sprintf("%064x", i)
|
||||
if _, ok, err := s.Claim(ctx, key, fp, owner); err != nil || !ok {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.Complete(ctx, key, owner, nil, time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
client.ZAdd(ctx, s.keys[1], goredis.Z{Score: 0, Member: key})
|
||||
}
|
||||
if _, ok, err := s.Claim(ctx, fmt.Sprintf("%064x", 139), fp, owner); err != nil || !ok {
|
||||
t.Fatal("expired target retained beyond TTL", ok, err)
|
||||
}
|
||||
}
|
||||
102
middleware/sharedIdempotency.go
Normal file
102
middleware/sharedIdempotency.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type StoredResponse struct {
|
||||
Status int `json:"status"`
|
||||
Headers http.Header `json:"headers"`
|
||||
Body []byte `json:"body"`
|
||||
}
|
||||
type IdempotencyRecord struct {
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
Owner string `json:"owner"`
|
||||
State string `json:"state"`
|
||||
Response *StoredResponse `json:"response,omitempty"`
|
||||
}
|
||||
|
||||
// IdempotencyStore must atomically claim absent keys, compare owners when
|
||||
// finishing and retain in-flight claims until explicit resolution. Only completed
|
||||
// or failed records expire; an uncertain outcome must never become a new claim.
|
||||
type IdempotencyStore interface {
|
||||
Claim(context.Context, string, string, string) (IdempotencyRecord, bool, error)
|
||||
Complete(context.Context, string, string, *StoredResponse, time.Duration) error
|
||||
}
|
||||
|
||||
func (i *Idempotency) shared(c *httpx.Context, next httpx.Handler, id, fingerprint [32]byte) error {
|
||||
key, fp := hex.EncodeToString(id[:]), hex.EncodeToString(fingerprint[:])
|
||||
token := make([]byte, 16)
|
||||
if _, err := rand.Read(token); err != nil {
|
||||
return err
|
||||
}
|
||||
owner := hex.EncodeToString(token)
|
||||
record, claimed, err := i.config.Store.Claim(c.Context(), key, fp, owner)
|
||||
if err != nil {
|
||||
return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "idempotency store unavailable").WithCause(err)
|
||||
}
|
||||
if !claimed {
|
||||
if record.Fingerprint != fp {
|
||||
return apierrors.New(409, "IDEMPOTENCY_CONFLICT", "key was used with a different request")
|
||||
}
|
||||
if record.State != "completed" || record.Response == nil {
|
||||
return apierrors.New(409, "IDEMPOTENCY_UNAVAILABLE", "request is in progress or outcome requires reconciliation")
|
||||
}
|
||||
response := record.Response
|
||||
headers := response.Headers.Clone()
|
||||
headers.Set("Idempotency-Replayed", "true")
|
||||
return c.WriteResponse(response.Status, headers, response.Body)
|
||||
}
|
||||
completed := false
|
||||
finish := func(response *StoredResponse) error {
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(c.Context()), time.Second)
|
||||
defer cancel()
|
||||
return i.config.Store.Complete(ctx, key, owner, response, i.config.TTL)
|
||||
}
|
||||
defer func() {
|
||||
if !completed {
|
||||
if err := finish(nil); err != nil {
|
||||
logx.ErrorContext(c.Context(), "idempotency.persistFailed", "error", err)
|
||||
}
|
||||
}
|
||||
}()
|
||||
buffer := &boundedResponse{headers: c.Raw().Header().Clone(), limit: i.config.MaxResponseBytes}
|
||||
child := c.Clone(c.Context(), buffer)
|
||||
err = next(child)
|
||||
if ctxErr := child.Context().Err(); ctxErr != nil {
|
||||
return ctxErr
|
||||
}
|
||||
if err != nil {
|
||||
child.WriteError(err)
|
||||
}
|
||||
if buffer.err != nil {
|
||||
return apierrors.Internal(buffer.err)
|
||||
}
|
||||
if buffer.status == 0 {
|
||||
buffer.status = 200
|
||||
}
|
||||
var response *StoredResponse
|
||||
if buffer.status < 500 && buffer.headers.Get("Set-Cookie") == "" && !strings.Contains(strings.ToLower(buffer.headers.Get("Cache-Control")), "no-store") {
|
||||
response = &StoredResponse{Status: buffer.status, Headers: buffer.headers.Clone(), Body: append([]byte(nil), buffer.body.Bytes()...)}
|
||||
}
|
||||
if err = finish(response); err != nil {
|
||||
return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "outcome requires reconciliation").WithCause(err)
|
||||
}
|
||||
completed = true
|
||||
return c.WriteResponse(buffer.status, buffer.headers, buffer.body.Bytes())
|
||||
}
|
||||
|
||||
var ErrLimitCapacity = errors.New("middleware: shared rate limiter capacity")
|
||||
|
||||
type RateLimitBackend interface {
|
||||
Take(context.Context, string) (time.Duration, error)
|
||||
}
|
||||
35
middleware/timeout.go
Normal file
35
middleware/timeout.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
// Timeout applies a cooperative deadline to handlers and downstream calls.
|
||||
// It does not spawn a goroutine or race a handler's response. Business code must
|
||||
// observe Context().Done(); an already committed response cannot be replaced.
|
||||
func Timeout(duration time.Duration) httpx.Middleware {
|
||||
if duration <= 0 {
|
||||
panic("middleware: timeout must be positive")
|
||||
}
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
parent := c.Context()
|
||||
ctx, cancel := context.WithTimeout(parent, duration)
|
||||
defer cancel()
|
||||
c.WithContext(ctx)
|
||||
defer c.WithContext(parent)
|
||||
if ctx.Err() != nil {
|
||||
return apierrors.New(504, "REQUEST_TIMEOUT", "request deadline exceeded")
|
||||
}
|
||||
err := next(c)
|
||||
if ctx.Err() == context.DeadlineExceeded && !c.Responded() {
|
||||
return apierrors.New(504, "REQUEST_TIMEOUT", "request deadline exceeded").WithCause(err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
17
omni.go
17
omni.go
@@ -9,6 +9,7 @@ import (
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/metrics"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
@@ -24,10 +25,18 @@ type Router = httpx.Router
|
||||
type HTTPRouterGroup = httpx.RouterGroup
|
||||
type APIError = apierrors.APIError
|
||||
type Config = config.Config
|
||||
type TLSConfig = config.TLSConfig
|
||||
type Option = config.Option
|
||||
type Response = response.Response
|
||||
type LogConfig = logx.Config
|
||||
type Logger = logx.Logger
|
||||
type Metrics = metrics.Metrics
|
||||
|
||||
const (
|
||||
TransportNetHTTP = config.TransportNetHTTP
|
||||
TransportNative = config.TransportNative
|
||||
TransportHTTP2 = config.TransportHTTP2
|
||||
)
|
||||
|
||||
func New(options ...Option) *App { return app.New(options...) }
|
||||
|
||||
@@ -39,8 +48,14 @@ func WithRequestIDHeader(header string) Option { return config.WithRequestIDHead
|
||||
|
||||
func WithLogger(enabled bool) Option { return config.WithLogger(enabled) }
|
||||
|
||||
func WithTrustedProxies(proxies ...string) Option { return config.WithTrustedProxies(proxies...) }
|
||||
|
||||
func WithConfig(settings Config) Option { return config.WithConfig(settings) }
|
||||
|
||||
func WithTransport(name string) Option { return config.WithTransport(name) }
|
||||
|
||||
func WithTLS(certFile, keyFile string) Option { return config.WithTLS(certFile, keyFile) }
|
||||
|
||||
func WithLogConfig(settings LogConfig) Option { return config.WithLogConfig(settings) }
|
||||
|
||||
func WithShutdownTimeout(timeout time.Duration) Option { return config.WithShutdownTimeout(timeout) }
|
||||
@@ -49,6 +64,8 @@ func LoadConfig(prefix string) (Config, error) { return config.LoadFromEnv(prefi
|
||||
|
||||
func SetupLogger(cfg LogConfig) (func() error, error) { return logx.Setup(cfg) }
|
||||
|
||||
func NewMetrics() *Metrics { return metrics.New() }
|
||||
|
||||
func NewError(status int, code, message string) *APIError {
|
||||
return apierrors.New(status, code, message)
|
||||
}
|
||||
|
||||
267
openapi/openapi.go
Normal file
267
openapi/openapi.go
Normal file
@@ -0,0 +1,267 @@
|
||||
// Package openapi registers routes and their OpenAPI 3.0.3 contracts together.
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Schema struct {
|
||||
Type string `json:"type,omitempty"`
|
||||
Format string `json:"format,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Properties map[string]*Schema `json:"properties,omitempty"`
|
||||
Required []string `json:"required,omitempty"`
|
||||
Items *Schema `json:"items,omitempty"`
|
||||
AdditionalProperties *Schema `json:"additionalProperties,omitempty"`
|
||||
Nullable bool `json:"nullable,omitempty"`
|
||||
}
|
||||
type Parameter struct {
|
||||
Name string `json:"name"`
|
||||
In string `json:"in"`
|
||||
Required bool `json:"required"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Schema *Schema `json:"schema"`
|
||||
}
|
||||
type Response struct {
|
||||
Description string
|
||||
Schema *Schema
|
||||
}
|
||||
type Operation struct {
|
||||
Method, Path, OperationID, Summary string
|
||||
Parameters []Parameter
|
||||
Request *Schema
|
||||
Responses map[int]Response
|
||||
BearerAuth bool
|
||||
}
|
||||
type Registrar interface {
|
||||
Handle(string, string, httpx.Handler, ...httpx.Middleware)
|
||||
}
|
||||
type API struct {
|
||||
mu sync.RWMutex
|
||||
document map[string]any
|
||||
paths map[string]map[string]any
|
||||
ids map[string]bool
|
||||
}
|
||||
|
||||
var identifier = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
|
||||
|
||||
func New(title, version string) (*API, error) {
|
||||
if title == "" || version == "" {
|
||||
return nil, errors.New("openapi: title and version required")
|
||||
}
|
||||
paths := make(map[string]map[string]any)
|
||||
return &API{paths: paths, ids: make(map[string]bool), document: map[string]any{"openapi": "3.0.3", "info": map[string]string{"title": title, "version": version}, "paths": paths, "components": map[string]any{"securitySchemes": map[string]any{"bearerAuth": map[string]string{"type": "http", "scheme": "bearer"}}}}}, nil
|
||||
}
|
||||
|
||||
// Handle registers an absolute application route. To document a group, include
|
||||
// its full prefix here and pass middleware explicitly. Schemas describe wire
|
||||
// responses; Envelope wraps a data schema in GoOmni's standard JSON envelope.
|
||||
func (a *API) Handle(router Registrar, op Operation, handler httpx.Handler, middleware ...httpx.Middleware) error {
|
||||
if router == nil || handler == nil || !identifier.MatchString(op.OperationID) || !strings.HasPrefix(op.Path, "/") || strings.ContainsAny(op.Path, "*{}?# ") {
|
||||
return errors.New("openapi: invalid route")
|
||||
}
|
||||
method := strings.ToLower(op.Method)
|
||||
switch method {
|
||||
case "get", "post", "put", "patch", "delete", "head", "options":
|
||||
default:
|
||||
return errors.New("openapi: unsupported HTTP method")
|
||||
}
|
||||
parts := strings.Split(op.Path, "/")
|
||||
params := make(map[string]bool)
|
||||
for n, part := range parts {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
name := part[1:]
|
||||
if !identifier.MatchString(name) || params[name] {
|
||||
return errors.New("openapi: invalid path parameter")
|
||||
}
|
||||
parts[n] = "{" + name + "}"
|
||||
params[name] = true
|
||||
}
|
||||
}
|
||||
path := strings.Join(parts, "/")
|
||||
seen := make(map[string]bool)
|
||||
for _, p := range op.Parameters {
|
||||
if p.Name == "" || p.Schema == nil || seen[p.In+":"+p.Name] {
|
||||
return errors.New("openapi: invalid or duplicate parameter")
|
||||
}
|
||||
seen[p.In+":"+p.Name] = true
|
||||
switch p.In {
|
||||
case "path":
|
||||
if !params[p.Name] || !p.Required {
|
||||
return errors.New("openapi: path parameter must match and be required")
|
||||
}
|
||||
delete(params, p.Name)
|
||||
case "query", "header", "cookie":
|
||||
default:
|
||||
return errors.New("openapi: invalid parameter location")
|
||||
}
|
||||
}
|
||||
if len(params) != 0 || len(op.Responses) == 0 {
|
||||
return errors.New("openapi: declare all path parameters and at least one response")
|
||||
}
|
||||
responses := map[string]any{}
|
||||
for code, response := range op.Responses {
|
||||
if code < 100 || code > 599 || response.Description == "" {
|
||||
return errors.New("openapi: invalid response")
|
||||
}
|
||||
item := map[string]any{"description": response.Description}
|
||||
if response.Schema != nil {
|
||||
item["content"] = map[string]any{"application/json": map[string]any{"schema": response.Schema}}
|
||||
}
|
||||
responses[fmt.Sprint(code)] = item
|
||||
}
|
||||
operation := map[string]any{"operationId": op.OperationID, "summary": op.Summary, "responses": responses}
|
||||
if len(op.Parameters) > 0 {
|
||||
operation["parameters"] = op.Parameters
|
||||
}
|
||||
if op.Request != nil {
|
||||
operation["requestBody"] = map[string]any{"required": true, "content": map[string]any{"application/json": map[string]any{"schema": op.Request}}}
|
||||
}
|
||||
if op.BearerAuth {
|
||||
operation["security"] = []map[string][]string{{"bearerAuth": {}}}
|
||||
}
|
||||
// Detach caller-owned maps/slices before concurrent document reads.
|
||||
encoded, err := json.Marshal(operation)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var frozen any
|
||||
if err = json.Unmarshal(encoded, &frozen); err != nil {
|
||||
return err
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
if a.ids[op.OperationID] || a.paths[path][method] != nil {
|
||||
return errors.New("openapi: duplicate operation ID or route")
|
||||
}
|
||||
router.Handle(strings.ToUpper(method), op.Path, handler, middleware...)
|
||||
if a.paths[path] == nil {
|
||||
a.paths[path] = make(map[string]any)
|
||||
}
|
||||
a.paths[path][method] = frozen
|
||||
a.ids[op.OperationID] = true
|
||||
return nil
|
||||
}
|
||||
func (a *API) JSON() ([]byte, error) {
|
||||
a.mu.RLock()
|
||||
defer a.mu.RUnlock()
|
||||
return json.MarshalIndent(a.document, "", " ")
|
||||
}
|
||||
func (a *API) Handler() httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
data, err := a.JSON()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.WriteResponse(200, http.Header{"Content-Type": {"application/json; charset=utf-8"}}, data)
|
||||
}
|
||||
}
|
||||
func Envelope(data *Schema) *Schema {
|
||||
return &Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}, "data": data}}
|
||||
}
|
||||
func SchemaOf[T any]() (*Schema, error) {
|
||||
return schema(reflect.TypeOf((*T)(nil)).Elem(), map[reflect.Type]bool{}, 0)
|
||||
}
|
||||
func schema(t reflect.Type, stack map[reflect.Type]bool, depth int) (*Schema, error) {
|
||||
if depth > 32 || stack[t] {
|
||||
return nil, errors.New("openapi: recursive DTO requires an explicit schema")
|
||||
}
|
||||
if t.Kind() == reflect.Pointer {
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
if err == nil {
|
||||
s.Nullable = true
|
||||
}
|
||||
return s, err
|
||||
}
|
||||
if t == reflect.TypeOf(time.Time{}) {
|
||||
return &Schema{Type: "string", Format: "date-time"}, nil
|
||||
}
|
||||
if t == reflect.TypeOf(json.RawMessage{}) {
|
||||
return &Schema{}, nil
|
||||
}
|
||||
if t == reflect.TypeOf(json.Number("")) {
|
||||
return &Schema{Type: "number"}, nil
|
||||
}
|
||||
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) {
|
||||
return nil, errors.New("openapi: custom JSON marshaler requires an explicit schema")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.String:
|
||||
return &Schema{Type: "string"}, nil
|
||||
case reflect.Bool:
|
||||
return &Schema{Type: "boolean"}, nil
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
return &Schema{Type: "integer"}, nil
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return &Schema{Type: "number"}, nil
|
||||
case reflect.Slice, reflect.Array:
|
||||
if t.Elem().Kind() == reflect.Uint8 && t.Kind() == reflect.Slice {
|
||||
return &Schema{Type: "string", Format: "byte"}, nil
|
||||
}
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
return &Schema{Type: "array", Items: s}, err
|
||||
case reflect.Map:
|
||||
if t.Key().Kind() != reflect.String {
|
||||
return nil, errors.New("openapi: map keys must be strings")
|
||||
}
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
return &Schema{Type: "object", AdditionalProperties: s}, err
|
||||
case reflect.Interface:
|
||||
return &Schema{}, nil
|
||||
case reflect.Struct:
|
||||
stack[t] = true
|
||||
defer delete(stack, t)
|
||||
s := &Schema{Type: "object", Properties: map[string]*Schema{}}
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
f := t.Field(n)
|
||||
if f.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
tag := strings.Split(f.Tag.Get("json"), ",")
|
||||
if tag[0] == "-" {
|
||||
continue
|
||||
}
|
||||
if f.Anonymous && tag[0] == "" {
|
||||
return nil, errors.New("openapi: embedded fields require an explicit JSON name")
|
||||
}
|
||||
name := tag[0]
|
||||
if name == "" {
|
||||
name = f.Name
|
||||
}
|
||||
child, err := schema(f.Type, stack, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if s.Properties[name] != nil {
|
||||
return nil, errors.New("openapi: duplicate JSON field")
|
||||
}
|
||||
s.Properties[name] = child
|
||||
optional := false
|
||||
for _, option := range tag[1:] {
|
||||
if option == "string" {
|
||||
child.Type = "string"
|
||||
child.Format = ""
|
||||
}
|
||||
if option == "omitempty" || option == "omitzero" {
|
||||
optional = true
|
||||
}
|
||||
}
|
||||
if !optional {
|
||||
s.Required = append(s.Required, name)
|
||||
}
|
||||
}
|
||||
return s, nil
|
||||
default:
|
||||
return nil, fmt.Errorf("openapi: unsupported DTO type %s", t)
|
||||
}
|
||||
}
|
||||
54
openapi/openapi_test.go
Normal file
54
openapi/openapi_test.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package openapi_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRouteAndContract(t *testing.T) {
|
||||
type item struct {
|
||||
ID int64 `json:"id,string"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
Payload json.RawMessage `json:"payload"`
|
||||
}
|
||||
schema, err := openapi.SchemaOf[item]()
|
||||
if err != nil || schema.Properties["id"].Type != "string" || schema.Properties["payload"].Type != "" {
|
||||
t.Fatal(schema, err)
|
||||
}
|
||||
api, _ := openapi.New("Test API", "1")
|
||||
app := omni.New(omni.WithLogger(false))
|
||||
op := openapi.Operation{Method: "GET", Path: "/items/:id", OperationID: "getItem", Parameters: []openapi.Parameter{{Name: "id", In: "path", Required: true, Schema: &openapi.Schema{Type: "string"}}}, Responses: map[int]openapi.Response{200: {Description: "OK", Schema: openapi.Envelope(schema)}}}
|
||||
if err = api.Handle(app, op, func(c *httpx.Context) error { return c.OK(item{ID: 1, Payload: json.RawMessage(`{}`)}) }); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Caller edits must not mutate the published contract.
|
||||
schema.Properties["id"].Type = "boolean"
|
||||
if err = api.Handle(app, op, func(*httpx.Context) error { return nil }); err == nil {
|
||||
t.Fatal("duplicate accepted")
|
||||
}
|
||||
app.GET("/openapi.json", api.Handler())
|
||||
r := httptest.NewRecorder()
|
||||
app.ServeHTTP(r, httptest.NewRequest("GET", "/openapi.json", nil))
|
||||
var doc struct {
|
||||
OpenAPI string `json:"openapi"`
|
||||
Paths map[string]json.RawMessage `json:"paths"`
|
||||
}
|
||||
if err = json.Unmarshal(r.Body.Bytes(), &doc); err != nil || doc.OpenAPI != "3.0.3" || doc.Paths["/items/{id}"] == nil {
|
||||
t.Fatal(r.Body.String(), err)
|
||||
}
|
||||
r = httptest.NewRecorder()
|
||||
app.ServeHTTP(r, httptest.NewRequest("GET", "/items/1", nil))
|
||||
if r.Code != 200 {
|
||||
t.Fatal(r.Code)
|
||||
}
|
||||
type recursive struct {
|
||||
Next *recursive `json:"next"`
|
||||
}
|
||||
if _, err = openapi.SchemaOf[recursive](); err == nil {
|
||||
t.Fatal("recursive DTO silently truncated")
|
||||
}
|
||||
}
|
||||
73
pagination/pagination.go
Normal file
73
pagination/pagination.go
Normal file
@@ -0,0 +1,73 @@
|
||||
// Package pagination defines bounded offset pagination with camelCase fields.
|
||||
package pagination
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/url"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
DefaultSize int `json:"defaultSize"`
|
||||
MaxSize int `json:"maxSize"`
|
||||
MaxOffset int64 `json:"maxOffset"`
|
||||
}
|
||||
type Request struct {
|
||||
Page int64 `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
|
||||
func (r Request) Offset() int64 { return (r.Page - 1) * int64(r.PageSize) }
|
||||
func Parse(query url.Values, cfg Config) (Request, error) {
|
||||
if cfg.DefaultSize == 0 {
|
||||
cfg.DefaultSize = 20
|
||||
}
|
||||
if cfg.MaxSize == 0 {
|
||||
cfg.MaxSize = 100
|
||||
}
|
||||
if cfg.MaxOffset == 0 {
|
||||
cfg.MaxOffset = 1000000
|
||||
}
|
||||
if cfg.DefaultSize < 1 || cfg.MaxSize < cfg.DefaultSize || cfg.MaxSize > 10000 || cfg.MaxOffset < 0 {
|
||||
return Request{}, errors.New("pagination: invalid configuration")
|
||||
}
|
||||
page, size := int64(1), int64(cfg.DefaultSize)
|
||||
for key, target := range map[string]*int64{"page": &page, "pageSize": &size} {
|
||||
if v, ok := query[key]; ok {
|
||||
if len(v) != 1 {
|
||||
return Request{}, errors.New("pagination: duplicate parameter")
|
||||
}
|
||||
n, err := strconv.ParseInt(v[0], 10, 64)
|
||||
if err != nil || n < 1 {
|
||||
return Request{}, errors.New("pagination: positive integer required")
|
||||
}
|
||||
*target = n
|
||||
}
|
||||
}
|
||||
if size > int64(cfg.MaxSize) || (page-1) > cfg.MaxOffset/size {
|
||||
return Request{}, errors.New("pagination: size or offset exceeds limit")
|
||||
}
|
||||
return Request{Page: page, PageSize: int(size)}, nil
|
||||
}
|
||||
|
||||
type Result[T any] struct {
|
||||
Items []T `json:"items"`
|
||||
Page int64 `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
Total int64 `json:"total"`
|
||||
TotalPages int64 `json:"totalPages"`
|
||||
}
|
||||
|
||||
func NewResult[T any](r Request, items []T, total int64) (Result[T], error) {
|
||||
if r.Page < 1 || r.PageSize < 1 || total < 0 || len(items) > r.PageSize || int64(len(items)) > total {
|
||||
return Result[T]{}, errors.New("pagination: invalid result")
|
||||
}
|
||||
if items == nil {
|
||||
items = []T{}
|
||||
}
|
||||
pages := total / int64(r.PageSize)
|
||||
if total%int64(r.PageSize) != 0 {
|
||||
pages++
|
||||
}
|
||||
return Result[T]{Items: items, Page: r.Page, PageSize: r.PageSize, Total: total, TotalPages: pages}, nil
|
||||
}
|
||||
30
pagination/pagination_test.go
Normal file
30
pagination/pagination_test.go
Normal file
@@ -0,0 +1,30 @@
|
||||
package pagination
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"math"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestBoundsAndEnvelope(t *testing.T) {
|
||||
for _, raw := range []string{"page=0", "page=1&page=2", "pageSize=101", "page=9223372036854775807&pageSize=100", "pageSize=-1"} {
|
||||
q, _ := url.ParseQuery(raw)
|
||||
if _, err := Parse(q, Config{}); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
r, err := Parse(url.Values{"page": {"2"}, "pageSize": {"30"}}, Config{})
|
||||
if err != nil || r.Offset() != 30 {
|
||||
t.Fatal(r, err)
|
||||
}
|
||||
result, err := NewResult(r, []string(nil), math.MaxInt64)
|
||||
if err != nil || result.TotalPages != math.MaxInt64/30+1 {
|
||||
t.Fatal(result, err)
|
||||
}
|
||||
raw, _ := json.Marshal(result)
|
||||
if !strings.Contains(string(raw), `"items":[]`) || !strings.Contains(string(raw), `"pageSize":30`) {
|
||||
t.Fatal(string(raw))
|
||||
}
|
||||
}
|
||||
44
performance/advancedAcceptance.md
Normal file
44
performance/advancedAcceptance.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# 扩展模块验收
|
||||
|
||||
代码提交:`208a9c3fbdb7ccf2ed409ba380050842e89829d8`。本报告与随后文档提交没有改变被测运行代码。
|
||||
|
||||
2026-09-23 在独立 Linux amd64 验证目录运行,开始 06:51:31 UTC,完成 06:55:56 UTC。工作树在启动时干净。原始 [JSON](advancedLinux.json) 记录 98 个源码/模板/脚本摘要、镜像 ID、命令和容器清理;[TXT](advancedLinux.txt) 保存实际输出。回传本机后已逐个核对源码摘要及日志 SHA-256。
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyModules.py --advanced \
|
||||
--goproxy https://goproxy.cn,direct \
|
||||
--output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
## 环境与隔离
|
||||
|
||||
- Go 1.25.14,Linux amd64。实际依赖版本以提交内 go.mod/go.sum 为准;新增依赖后已覆盖原有网络适配器回归。
|
||||
- Redis 7.4.11,MinIO RELEASE.2025-09-07T16-13-09Z,MySQL 8.4.11。
|
||||
- 编译、Redis、MinIO、MySQL 顺序运行,最多一个测试容器;每个硬限制 2 CPU / 2 GiB 内存与内存加 swap / 256 pids。
|
||||
- 数据服务阶段 `--network none`,仅容器回环/Unix socket,无发布端口,数据位于本次临时目录/tmpfs。没有连接现有业务数据库。
|
||||
- 四个测试容器已清理,原有 7 个业务容器均仍运行。
|
||||
|
||||
## 通过范围
|
||||
|
||||
| 项目 | 实际证据 |
|
||||
| --- | --- |
|
||||
| 全仓库检查 | `go test -race -count=1 -timeout=240s ./...`、`go vet ./...`、`go build ./...`;脚手架集成开关已开启,包含独立生成项目 |
|
||||
| 认证与配置 | JWT 过期/缺 exp/错误发行者或受众/算法混用/轮换、Session 撤销与状态隔离、文件环境覆盖、非法/静态配置拒绝、子日志级别更新 |
|
||||
| 契约与观测 | OpenAPI 路径参数与 DTO、重复注册拒绝、W3C 入站/出站关联、外层访问日志 traceId、HTTP 指标、Redis miss 错误保持、SQL spans |
|
||||
| 熔断与通用模块 | 冷却探针、迟到结果、响应体期间槽位、panic 释放、分页溢出与重复参数、审计白名单与写入失败 |
|
||||
| 组合示例 | nethttp/native/实验 HTTP2 的真实 TLS 401/403/201、对象内容、非法路径、OpenAPI 和关闭;原有 modules/reference 回归同批通过 |
|
||||
| 共享 Redis | 并发 claim 唯一赢家、错误 owner 拒绝、二进制响应回放、两个应用实例仅执行一次、冲突 409、共享限流 429、存储失败 503、过期目标不受批次清理顺序影响 |
|
||||
| Redis 队列 | 重报回执、内容冲突、容量拒绝、旧 owner 无法 ACK、保留未确认消息后自动接管、两个 Consumer 竞争、有限重试/永久拒绝、失败消息分页/受控删除、完成身份 TTL |
|
||||
| 对象存储 | Local 契约及失败上传保留原对象、拒绝符号链接、签名篡改;OSS SDK 协议 fixture;S3 SDK fixture 和 R2 endpoint/region/签名参数;真实 MinIO 上传/覆盖/读取/删除与错误签名 403 |
|
||||
| MySQL 迁移 | 全新测试库 up、重复 up 不重执行、版本读取、SQL 查询/更新观测、失败迁移保留 dirty、后续执行被阻断 |
|
||||
|
||||
真实 Redis/MinIO/MySQL 阶段使用 Go 编译的静态测试程序;该阶段没有启用 race。全仓库 race 阶段不连接这些外部数据服务。两类验证各自的范围不混为一项。
|
||||
|
||||
## 保留边界
|
||||
|
||||
- 本批按用户决定不集成 Orbit,未修改 Orbit 或业务项目,也没有用通用队列模拟其流程编排协议。
|
||||
- OSS/R2 没有真实云账号、bucket 权限、临时凭据轮换、地域/公网故障验收;MinIO 通过不能替代这些验证。
|
||||
- 队列接管通过真实 Redis 中的未确认 delivery 验证,没有本批 Worker 进程强杀、Redis AOF 重启/复制切换、长期负载或 exactly-once 证明。测试 Redis 关闭持久化以隔离协议检查,生产需另行配置持久化与故障验收。
|
||||
- 没有在生产业务上自动执行迁移;没有证明任意 DDL 可事务回滚。MySQL 失败保留 dirty,由部署流程核查。
|
||||
- OpenTelemetry 的采集/传播/exporter 接口通过本批测试,未连接实际 OTLP collector 或告警系统。
|
||||
- 这是扩展模块的首版功能验收,不是生产规模容量或完整运维验收。
|
||||
303
performance/advancedLinux.json
Normal file
303
performance/advancedLinux.json
Normal file
@@ -0,0 +1,303 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T06:51:31.468977+00:00",
|
||||
"testedCommit": "208a9c3fbdb7ccf2ed409ba380050842e89829d8",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"audit/audit.go": "48da5d6ecd498230d22db04413d379650b9810b3faee5fb5d52415ab6f681eea",
|
||||
"audit/audit_test.go": "7ecb96dcc8294c905465bb026030a2b0d2d5064359a9cb6b95199d5d15491c49",
|
||||
"auth/adapters_test.go": "a45dd935fd91b91d16435c97c3b6b6b82615a55bc16463eb94448704a2433fe1",
|
||||
"auth/auth.go": "067b75edf2d73338ceeff3b70698e6f5426606c5d510a7d0249bcf6fd7796eb8",
|
||||
"auth/auth_test.go": "158626ac86c504a337e37faad65b2bad77d68f54ce6b6ac2cda9d287aeefa963",
|
||||
"auth/jwt.go": "cfd4ead3712e921cb08d65b9ea81a06d1b6de420bc94cf240540a95959a2f241",
|
||||
"auth/policy.go": "f02cf32593033908921172190a65debaf9f8c0af616ceb95982ea94649170b62",
|
||||
"auth/session.go": "544c71e00c6af913722c1cca510e1fb2938925234c246e0ec52de22284a81b18",
|
||||
"cache/cache.go": "4aad0a644c0c9ce6353e61e70059715bea5402f0d7132b50003ee126c3836cad",
|
||||
"cache/cache_test.go": "c850cd8a90c1e6a8d701a415e283d4147fbd66d12d054002014cbf348863aea2",
|
||||
"cache/memory.go": "3ac2a710a0d4677ad58dd6817005033ef7cc939be7eb3e40476212ce7f18ad42",
|
||||
"cache/redis/redis.go": "3f522258da8a0fc6627645573221e9f5ccbdfea0fe8a1450985cfd084e2c18ce",
|
||||
"cache/redis/redis_test.go": "4948647b2d31803252af1ee4e4b96372b53f9fac46f03ab064b40abe2d5a0aa1",
|
||||
"cmd/go-omni/main.go": "811cd23a5288cd0a7efc132782d9509307f736afbcbbb2d001c59aad9d8633a9",
|
||||
"cmd/go-omni/main_test.go": "312e0dea1d46cbbd436acdb4fcac0704d4781746d9e8433f1203f67ef9601065",
|
||||
"cmd/go-omni/migrate.go": "f1ce89fbe419205eb82172da2b90e8888905d2b85d1a9db2a72b568bcf849a70",
|
||||
"cmd/go-omni/migrate_test.go": "15d9a4aaebf9f899e523e908f1fa3c2282eb1e6c8cadbefae6b68f1ae4e00e6e",
|
||||
"cmd/go-omni/scaffold/addModule.go": "8f43bd68c2b3820cf1d036c4f2c5f9d06d1c7c8e5114f5081597d905f8d8cdcc",
|
||||
"cmd/go-omni/scaffold/addModule_test.go": "667c45312f628f0fb5b8206ddef7ccc02a75f33b72f73fd6a17b03da0cb62e91",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl": "9d3bfc46cac5618bcff7d8a6b2cc9f123e818aeb3dee9bcdfe0f91db584994bd",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl": "729cd0f6faba53d5d2c1492935e12ac2adcefde4490748e25c18be7ccb3f7139",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl": "a005611f6b455d4d8469e316bc26fa571043b2da1834db2b3dd14a96d66221d7",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl": "fb0be7fd571ba2c0edb01d70d68cdbbfe53561caab98346bf1ac7cd8df07205a",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/repository-repository.go.tmpl": "902da3d9f12395fb226178c4428ff547e1c2fce3261882dba98d1400d8c90747",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl": "49d26044a69a43e983aa5ba4b5b2399b26b77898b23e230751ad1372c2af3178",
|
||||
"cmd/go-omni/scaffold/scaffold.go": "15baca1b5c4607c8f96fb6f6d4b96a661a86899012f93a6c380bc6d894b18803",
|
||||
"cmd/go-omni/scaffold/scaffold_test.go": "58aab9251b01e394bbc90aeb538ff035dd32c73c0d4815132ac6cbc086527cda",
|
||||
"cmd/go-omni/scaffold/templates/README.md.tmpl": "dc5ce58c49b4c0b50a87fdca794513cad38f428975bdfe772e89944e39614bd4",
|
||||
"cmd/go-omni/scaffold/templates/cmd-server-main.go.tmpl": "07c4f81109ece43fcf15bb807ada5ae6dc7e732babc7293269514a21cd389955",
|
||||
"cmd/go-omni/scaffold/templates/databaseIntegration_test.go.tmpl": "81c4f7fbc919d74b738d3a7338248cbd83cfa0803b2058e8ee184e3ce51e7e9b",
|
||||
"cmd/go-omni/scaffold/templates/go.mod.tmpl": "7d3bb15d2292e36939cc1905258d2800aeaf0e37d7d041b2ff8cfdb6fc84c57a",
|
||||
"cmd/go-omni/scaffold/templates/internal-handler-user.go.tmpl": "7f8dbcd2f9b0129a90160d6059e47d19d4758274060a1a3e5160055d0592d34f",
|
||||
"cmd/go-omni/scaffold/templates/internal-model-errors.go.tmpl": "f123eace52952321b0665a92a2923514c52c8c3bf13df1065710e9675750f837",
|
||||
"cmd/go-omni/scaffold/templates/internal-model-user.go.tmpl": "b391143e7386e169b53e7fed25de1146e8c52f3fa287b12728e8d22b33f386de",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-memory.go.tmpl": "148abc273c1fab3f28536706792762ae41e8687dfe9986c374372f162600feac",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-mysql.go.tmpl": "34bd333e29211a0025ff6ebaea18c7f142f18f1cc451d692f9d360e8c8f6f635",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-users.go.tmpl": "6d7dd80f34a9c215daeb4a86bfbe11e845e7ff7e9864c8af43e1c2ec3a8b28ec",
|
||||
"cmd/go-omni/scaffold/templates/internal-service-user.go.tmpl": "3fde559ac634a6c85d322dbc3487691a192644268e85aa3911dd2561a4116196",
|
||||
"cmd/go-omni/scaffold/templates/internal-servicecontext-context.go.tmpl": "2504d9750194a543121c9e93328f5f2a535f1db9f8dbd361edf3681cdfa21c39",
|
||||
"cmd/go-omni/scaffold/templates/schema.sql.tmpl": "7c5108f18fc7d8ff75c97afd96fcaa9f8ed026c79c71c5612f55572d73354abb",
|
||||
"cmd/go-omni/scaffold/version.go": "4d2d1b0d6d0ea716ff4cb7ff08cc25e6f91e06b1a4585176e3128a22e2f2f4ad",
|
||||
"config/config.go": "5cbdf774afa62001910bac3d6523399dc5c9dfdafc6111ab86bd5ce1502d7811",
|
||||
"config/config_test.go": "53b72d5bbe2b5a38c0e3c17bd426f1bb5b43beb2014e7391aef9c438d9e521ac",
|
||||
"config/env.go": "2c05cc2d97ae778158a52d390bcec4226d4196833e1e051a76a2eeb425997c0b",
|
||||
"config/file.go": "2e9f0f9ae30f995994ba0841c27bd27ede7f0652b2c822d71140f439e023d95e",
|
||||
"config/file_test.go": "1479d625aad26a1f773bc4a08c146347f163bbf7b4f11385a32f687119df5817",
|
||||
"database/migrate/migrate.go": "fd6cd27d2dce5b35ffe80eb5442c47c657b93ffc6545bbcc1ee4651a70a4f80b",
|
||||
"database/migrate/migrate_test.go": "d7ba91eed8cfd97e4ef0741d582f493dd411b65ad3192f05b167f75d9b9dcf90",
|
||||
"examples/modules/main.go": "6b44b981c0f654e241664c94612cd95ba8a394155b589add0814d81397156a26",
|
||||
"examples/modules/main_test.go": "4ceb869a7e1ab3fb037af5411ec3dec47af66738adb66c4db01498d446aab7fb",
|
||||
"examples/platform/main.go": "f4fea7a6cdb37413ccab847de68fcbcf8608142858005e1c75754692e2c676d9",
|
||||
"examples/platform/main_test.go": "fd120cfcda2dcaeef5038bdee0ffaaff88bec4450e285ecb4a4de458230d7a12",
|
||||
"go.mod": "f025c81671bfe1399028ce52b934c3b9c264a788c563dfe63a8eb3bc1c458093",
|
||||
"go.sum": "8db5712ded87123d25b155ae67c5c5b168e8dc39d84e00a256766c5ef199cb76",
|
||||
"httpc/client.go": "bbec35b6234a554a4aefd0fef97ace3850288642faab2e382dcbe19a6cb95cd8",
|
||||
"httpc/client_test.go": "972ebd2efab618629e394484c288afe4e675fcfe0c50121d9fa9071680d6e763",
|
||||
"httpx/context.go": "104434499f074eedf2c54e4d0e90db587feb230ca8bb76803b87f2e42d9a8118",
|
||||
"internal/storagefixture/server.go": "defacc905246a643faf5371e437f9ce458f05461e0a922766034d1f9ffdaf003",
|
||||
"logx/logx.go": "1307adf3758967651dd802b814fc0f5804280163a4dc463478bf3667af641740",
|
||||
"logx/logx_test.go": "0f99eff445decdd81a55b3d5374c22d2e038d2a4748adb2c9367d6d03e481671",
|
||||
"middleware/idempotency.go": "a67e0040cbb15cd0140379154017e8f1549b06fc1f307c2549dad515d977e05f",
|
||||
"middleware/middleware.go": "a1e6cfae40d3e1701238a680009d66121f1bbaed786e81b3f6133eb1198a1cf6",
|
||||
"middleware/middleware_test.go": "9249aac16f251858bfe7937f768dc628bf4cc392af75f90223ccacd0f13d2cb9",
|
||||
"middleware/protection_test.go": "4347bb6074078c4c7bd9f65f5fa13a0ac2bc9a8615b297588cc3358aede69a7e",
|
||||
"middleware/rateLimit.go": "fa78a39495a432b3c3349915421a60bed8a76b0921d1f11d7962171fcb1e25ae",
|
||||
"middleware/redis/store.go": "168cac5588ed56e6ef33f2a5de6f8e91599ddf0583f509dd46f5210940fb148a",
|
||||
"middleware/redis/store_test.go": "6659dd4f486da08f4bffb27c698f18c67c3f8c3b59d9f3ebc68e7cd79b9dae15",
|
||||
"middleware/request_id.go": "d52bf17bf80245bafe878026b044857442f0af8df689bcaed4d967d006cbd947",
|
||||
"middleware/sharedIdempotency.go": "2fcc5dec3564680ee8039a8d43451948165553524c9c85248ec7218a5fa68ced",
|
||||
"middleware/timeout.go": "e2f446d992c9f0c986deb75fa71297f19c78f8e43593f174cd19fe93e385e440",
|
||||
"openapi/openapi.go": "13cc2a4fcaa4a50e6f38a38a3d168b6240dcdb49b2ebb5a2782f744f2635857d",
|
||||
"openapi/openapi_test.go": "c92f4a2f548f0304708ac157402fb8641273473e04c524226f2429c2b64b99ab",
|
||||
"pagination/pagination.go": "5c3ca00d27769dcef18d234ac14d5ccc635759ab5be77ef0c765aeba66d39532",
|
||||
"pagination/pagination_test.go": "f78fa86928d4d34a763e96423b29148bc74403aaf58c04d58fe29e225f6598eb",
|
||||
"queue/queue.go": "99a0a8d577dc401d7e69d01602c1889d93d370514bc333111706602044de158a",
|
||||
"queue/redis/consumer.go": "fdb9163db4917f27e4d716e5b42c64dc9694809c460e53b0bff034ebe2859be6",
|
||||
"queue/redis/consumer_test.go": "ea10f5e4804a3350bcee8bdc4bd0debe42820bd3451e28e36f6afc6aeadc0a46",
|
||||
"queue/redis/store.go": "ab5320bcdc7e80c7f3fdc77d583d895055d8970e0a22dbd78bac9b2801cfc634",
|
||||
"resilience/breaker.go": "ca1052185b7dd92e0c289c50aab7f37fb0e6aec6c4c214ae02dff7b8eb91b137",
|
||||
"resilience/breaker_test.go": "6f5abc9534d2bf2506afa3f79fcbb2efb18fa9d488cb010e870c42aa1c8c20d1",
|
||||
"resilience/http.go": "b84a64cef10943af51035c4fc19569b06cde0872046adf1bd1569b6b38170df4",
|
||||
"scripts/verifyModules.py": "ce68eb1f095bbaf721f924ec67c6fa48e0669303f3cdd83be56df06f97e18277",
|
||||
"storage/local/local.go": "ce5fac2e54680b96db3e7dfd9a8d065888f246c5607882607d79c302ee3d231b",
|
||||
"storage/local/local_test.go": "3a2241840968df8e37fd67169c3ef708228449e81f65aff908c8b60d49434dc0",
|
||||
"storage/oss/oss.go": "4b189138172ff42fb8c7549750eea59286c52bcc336f2e4da9d3831cdf3b41b9",
|
||||
"storage/oss/oss_test.go": "555a954010b1218804dc7951fc75e42eae24eba9a99c0871f5ba787f3fc87d45",
|
||||
"storage/s3/config.go": "626ec6a2c0b06c95ee7eb7aa0832b190cfcdeb8a127b6b89e99204040bc3b041",
|
||||
"storage/s3/s3.go": "a9f92c4c59930dd6bf9ec630bcb128ec46bef335f97704ca47647a9b73e15a58",
|
||||
"storage/s3/s3_test.go": "da670fb98a31ff1498d02682cdcd41e3adbab22a44b37f88907b1ea93ab2977f",
|
||||
"storage/signed.go": "72f0255c6ddd275298ba1b9fd50eb48c1858cc8102c43f6764e46f209fffb634",
|
||||
"storage/signed_test.go": "eaffd40894018814e976477f96cf0fbaecfa1987caad78830d9fc8e96995882f",
|
||||
"storage/storage.go": "cb670872f01cf1d7b4ebf21cec9fc6f52ad93bfdf523b2f28bbb8d9c7fe39a6c",
|
||||
"storage/storagetest/contract.go": "e024c5cd3c8a6c2ed656bad370a707f6960f4e717bc5d7dfd5624f6076c00605",
|
||||
"task/task.go": "ef671a9b7d84e7f53cdb2dcb787881e47b00ef0afb29fb6e4698fa8765a7bfe7",
|
||||
"task/task_test.go": "6e33ecd81345ece6102f48e60050a0d990e1bf75c3840e262d5da2962c743681",
|
||||
"telemetry/adapters.go": "a892ef0c2b268caf266a3ea0f833ecf7e9228ff59aeaeecae6fe4a2170ba51e2",
|
||||
"telemetry/telemetry.go": "5567b9d68f52ca54e34568c5f0c9a5b985d90c0c4b5224e57622d6f60adebe69",
|
||||
"telemetry/telemetry_test.go": "7040df811a5cbf9108af3d5e0a98639d1e31efdc8ce4eee6ac0332f3a58691de"
|
||||
},
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 900,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"redis:7.4-alpine": "sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499",
|
||||
"quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z": "sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e",
|
||||
"m.daocloud.io/docker.io/library/mysql:8.4": "sha256:b3b90af2a6552ae30c266fdb7d5dd55f3afb72404bb78d37fe8a23eb857fd3fb"
|
||||
},
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GO_OMNI_SCAFFOLD_INTEGRATION=1",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go test -race -count=1 -timeout=240s ./... && go vet ./... && go build ./... && CGO_ENABLED=0 go test -c -o /output/redis.test ./cache/redis && CGO_ENABLED=0 go test -c -o /output/shared.test ./middleware/redis && CGO_ENABLED=0 go test -c -o /output/queue.test ./queue/redis && CGO_ENABLED=0 go test -c -o /output/s3.test ./storage/s3 && CGO_ENABLED=0 go test -c -o /output/mysql.test ./database/migrate"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-redis",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"--user",
|
||||
"999:999",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=64m,mode=1777",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GO_OMNI_REDIS_TEST_ADDR=127.0.0.1:6379",
|
||||
"redis:7.4-alpine",
|
||||
"-c",
|
||||
"set -eu\nredis-server --version\nredis-server --bind 127.0.0.1 --port 6379 --save '' --appendonly no --maxmemory 32mb --daemonize yes --pidfile /tmp/redis.pid --logfile /tmp/redis.log --dir /tmp\ntrap 'redis-cli -h 127.0.0.1 shutdown nosave >/dev/null 2>&1 || true' EXIT\nattempt=0\nuntil redis-cli -h 127.0.0.1 ping >/dev/null 2>&1; do\n attempt=$((attempt + 1))\n if [ \"$attempt\" -gt 50 ]; then cat /tmp/redis.log; exit 1; fi\n sleep 0.1\ndone\n/test/redis.test -test.v -test.run '^TestRedisIntegration$' -test.count=1 -test.timeout=30s\n/test/shared.test -test.v -test.count=1 -test.timeout=45s\n/test/queue.test -test.v -test.count=1 -test.timeout=45s\n"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-s3",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"--user",
|
||||
"1000:1000",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=64m,mode=1777",
|
||||
"--tmpfs",
|
||||
"/data:rw,nosuid,noexec,size=128m,mode=1777",
|
||||
"-e",
|
||||
"MINIO_ROOT_USER=goOmniTest",
|
||||
"-e",
|
||||
"MINIO_ROOT_PASSWORD=goOmniTestSecret",
|
||||
"-e",
|
||||
"GO_OMNI_S3_TEST_ADDR=127.0.0.1:9000",
|
||||
"quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z",
|
||||
"-c",
|
||||
"set -eu\nminio --version\nminio server /data --address 127.0.0.1:9000 --console-address 127.0.0.1:9001 >/tmp/minio.log 2>&1 &\nserver=$!\ntrap 'kill \"$server\" 2>/dev/null || true; wait \"$server\" 2>/dev/null || true' EXIT\n/test/s3.test -test.v -test.run '^TestS3Integration$' -test.count=1 -test.timeout=45s || { cat /tmp/minio.log; exit 1; }\n"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-mysql",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"--user",
|
||||
"999:999",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=512m,mode=1777",
|
||||
"--tmpfs",
|
||||
"/var/lib/mysql:rw,nosuid,noexec,size=1m,mode=1777",
|
||||
"-e",
|
||||
"GO_OMNI_MIGRATE_TEST_DSN=root@unix(/tmp/mysql.sock)/omni_test?parseTime=true&timeout=2s&readTimeout=5s&writeTimeout=5s",
|
||||
"m.daocloud.io/docker.io/library/mysql:8.4",
|
||||
"-c",
|
||||
"set -eu\nmysqld --version\nmkdir /tmp/mysql\nmysqld --no-defaults --initialize-insecure --datadir=/tmp/mysql --innodb-redo-log-capacity=67108864 >/tmp/init.log 2>&1 || { cat /tmp/init.log; exit 1; }\nmysqld --no-defaults --datadir=/tmp/mysql --socket=/tmp/mysql.sock --pid-file=/tmp/mysql.pid --bind-address=127.0.0.1 --port=3306 --mysqlx=OFF --skip-log-bin --skip-name-resolve --innodb-buffer-pool-size=134217728 --innodb-redo-log-capacity=67108864 --max-connections=8 --log-error=/tmp/mysql.log &\nserver=$!\ntrap 'kill \"$server\" 2>/dev/null || true; wait \"$server\" 2>/dev/null || true' EXIT\nattempt=0\nuntil mysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'SELECT 1' >/dev/null 2>&1; do\n attempt=$((attempt + 1)); if [ \"$attempt\" -gt 200 ]; then cat /tmp/mysql.log; exit 1; fi; sleep 0.1\ndone\nmysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'CREATE DATABASE omni_test'\n/test/mysql.test -test.v -test.run '^TestMySQLIntegration$' -test.count=1 -test.timeout=60s\n"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-modules-7387653de963-build",
|
||||
"go-omni-modules-7387653de963-redis",
|
||||
"go-omni-modules-7387653de963-s3",
|
||||
"go-omni-modules-7387653de963-mysql"
|
||||
],
|
||||
"notes": [
|
||||
"Full repository race/vet/build plus independent generated-module CRUD in the compiler container.",
|
||||
"Real Redis uses a statically linked non-race test binary, loopback only, network=none and no published ports.",
|
||||
"At most one validation container; no pre-existing service or database is accessed."
|
||||
],
|
||||
"logSha256": "e76d05ff859f96330f19e4cb2a96241cf5d63cf118abd80db16fe67ff3a1c51f",
|
||||
"finishedAt": "2026-09-23T06:55:56.890295+00:00",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
119
performance/advancedLinux.txt
Normal file
119
performance/advancedLinux.txt
Normal file
@@ -0,0 +1,119 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
go: downloading github.com/redis/go-redis/v9 v9.17.3
|
||||
go: downloading golang.org/x/mod v0.21.0
|
||||
go: downloading github.com/go-sql-driver/mysql v1.9.3
|
||||
go: downloading github.com/golang-migrate/migrate/v4 v4.18.1
|
||||
go: downloading gopkg.in/yaml.v3 v3.0.1
|
||||
go: downloading go.opentelemetry.io/otel/sdk v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel v1.31.0
|
||||
go: downloading github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0
|
||||
go: downloading github.com/minio/minio-go/v7 v7.0.83
|
||||
go: downloading golang.org/x/net v0.33.0
|
||||
go: downloading github.com/go-playground/validator/v10 v10.20.0
|
||||
go: downloading go.opentelemetry.io/otel/metric v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel/sdk/metric v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel/trace v1.31.0
|
||||
go: downloading filippo.io/edwards25519 v1.1.0
|
||||
go: downloading github.com/cespare/xxhash/v2 v2.3.0
|
||||
go: downloading github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
|
||||
go: downloading github.com/hashicorp/go-multierror v1.1.1
|
||||
go: downloading go.uber.org/atomic v1.7.0
|
||||
go: downloading golang.org/x/time v0.5.0
|
||||
go: downloading github.com/goccy/go-json v0.10.4
|
||||
go: downloading github.com/google/uuid v1.6.0
|
||||
go: downloading github.com/klauspost/compress v1.17.11
|
||||
go: downloading github.com/minio/md5-simd v1.1.2
|
||||
go: downloading github.com/go-ini/ini v1.67.0
|
||||
go: downloading github.com/gabriel-vasile/mimetype v1.4.3
|
||||
go: downloading github.com/go-playground/universal-translator v0.18.1
|
||||
go: downloading github.com/leodido/go-urn v1.4.0
|
||||
go: downloading golang.org/x/crypto v0.31.0
|
||||
go: downloading golang.org/x/text v0.21.0
|
||||
go: downloading golang.org/x/sys v0.28.0
|
||||
go: downloading github.com/hashicorp/errwrap v1.1.0
|
||||
go: downloading github.com/klauspost/cpuid/v2 v2.2.9
|
||||
go: downloading github.com/dustin/go-humanize v1.0.1
|
||||
go: downloading github.com/rs/xid v1.6.0
|
||||
go: downloading github.com/go-playground/locales v0.14.1
|
||||
go: downloading github.com/go-logr/logr v1.4.2
|
||||
go: downloading github.com/go-logr/stdr v1.2.2
|
||||
ok git.yuntuo.hk/k2safe/go-omni 1.098s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/app 1.652s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/audit 1.016s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/auth 1.048s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cache 1.034s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cache/redis 1.014s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni 1.194s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold 4.092s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/config 1.042s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/database 1.013s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/database/migrate 1.053s
|
||||
? git.yuntuo.hk/k2safe/go-omni/errors [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/basic [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/modules 1.072s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/platform 1.105s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/reference 1.156s
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/handler [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/model [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/reference/repository 1.016s
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/service [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/health 1.040s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/httpc 1.102s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/httpx 1.033s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/internal/perfcheck 1.050s
|
||||
? git.yuntuo.hk/k2safe/go-omni/internal/storagefixture [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/internal/testutil [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/logx 1.013s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/metrics 1.054s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/middleware 1.059s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/middleware/redis 1.036s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/openapi 1.036s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/pagination 1.013s
|
||||
? git.yuntuo.hk/k2safe/go-omni/queue [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/queue/redis 1.014s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/resilience 1.014s
|
||||
? git.yuntuo.hk/k2safe/go-omni/response [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage 1.020s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/local 1.021s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/oss 1.029s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/s3 1.035s
|
||||
? git.yuntuo.hk/k2safe/go-omni/storage/storagetest [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/task 1.092s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/telemetry 1.038s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport 1.053s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 10.647s
|
||||
? git.yuntuo.hk/k2safe/go-omni/transport/internal/routing [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/native 2.513s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/nethttp 1.071s
|
||||
? git.yuntuo.hk/k2safe/go-omni/transport/transporttest [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/validate [no test files]
|
||||
Redis server v=7.4.11 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=40ff01a501d8e4b6
|
||||
=== RUN TestRedisIntegration
|
||||
--- PASS: TestRedisIntegration (0.09s)
|
||||
PASS
|
||||
=== RUN TestSharedRedisIntegration
|
||||
--- PASS: TestSharedRedisIntegration (0.01s)
|
||||
=== RUN TestExpiredTargetBeyondCleanupBatch
|
||||
--- PASS: TestExpiredTargetBeyondCleanupBatch (0.03s)
|
||||
PASS
|
||||
=== RUN TestQueueRedisIntegration
|
||||
=== RUN TestQueueRedisIntegration/publicationAndCapacity
|
||||
=== RUN TestQueueRedisIntegration/staleAcknowledgement
|
||||
=== RUN TestQueueRedisIntegration/crashReclaimRetryAndDeadLetters
|
||||
--- PASS: TestQueueRedisIntegration (1.11s)
|
||||
--- PASS: TestQueueRedisIntegration/publicationAndCapacity (0.00s)
|
||||
--- PASS: TestQueueRedisIntegration/staleAcknowledgement (0.00s)
|
||||
--- PASS: TestQueueRedisIntegration/crashReclaimRetryAndDeadLetters (1.11s)
|
||||
PASS
|
||||
minio version RELEASE.2025-09-07T16-13-09Z (commit-id=07c3a429bfed433e49018cb0f78a52145d4bedeb)
|
||||
Runtime: go1.24.6 linux/amd64
|
||||
License: GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html
|
||||
Copyright: 2015-2025 MinIO, Inc.
|
||||
=== RUN TestS3Integration
|
||||
--- PASS: TestS3Integration (0.33s)
|
||||
PASS
|
||||
/usr/sbin/mysqld Ver 8.4.11 for Linux on x86_64 (MySQL Community Server - GPL)
|
||||
=== RUN TestMySQLIntegration
|
||||
--- PASS: TestMySQLIntegration (0.03s)
|
||||
PASS
|
||||
80
performance/http2FlowControlLinux.json
Normal file
80
performance/http2FlowControlLinux.json
Normal file
@@ -0,0 +1,80 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"testedCommit": "fa2be5e0166f1fa581a2b0c7ee2fc29c44ac2676",
|
||||
"startedAt": "2026-09-23T02:50:32.502669+00:00",
|
||||
"imageId": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"command": [
|
||||
"docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-boundary-363c306dd46d",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go test -race -count=1 -timeout 2m -v ./transport/http2 && go vet ./transport/http2 && go build ./transport/http2"
|
||||
],
|
||||
"wallTimeoutSeconds": 300,
|
||||
"finishedAt": "2026-09-23T02:51:28.854252+00:00",
|
||||
"exitCode": 0,
|
||||
"logSha256": "52c96133d72ac15fa13b5784ba045a732d5ed0cf728f111481f0046407efff81",
|
||||
"platform": "linux/amd64",
|
||||
"goVersion": "go1.25.14",
|
||||
"package": "./transport/http2",
|
||||
"checks": [
|
||||
"race",
|
||||
"vet",
|
||||
"build"
|
||||
],
|
||||
"passedTests": [
|
||||
"TestTLSConnectionLimitRecoversAfterHandshakeTimeout",
|
||||
"TestSlowResponseStreamIsolationAndRecovery",
|
||||
"TestConnectionWindowIsolationAndRecovery",
|
||||
"TestShutdownReleasesFlowControlledWriter",
|
||||
"TestShutdownSendsGoAwayAndDrains",
|
||||
"TestRSTStreamCancelsOnlyOneRequest",
|
||||
"TestCanceledServeContextWaitsForShutdownDeadline",
|
||||
"TestTLSMixedSoak",
|
||||
"TestServerHandlesH2CRequest",
|
||||
"TestServerEnforcesGlobalRequestLimitAndRecovers",
|
||||
"TestServerRejectsBodyAboveLimitAndRecovers",
|
||||
"TestServerRejectsHeadersAboveLimitAndRecovers",
|
||||
"TestServerMultiplexes128Streams",
|
||||
"TestServerStreamsLargeBodyWithFlowControl",
|
||||
"TestServerTLSUsesH2ALPN",
|
||||
"TestTLSSoak"
|
||||
],
|
||||
"notes": [
|
||||
"Sequentially run after the 30-minute mixed soak; limited to 2 CPUs, 2048 MiB RAM with no swap, and 256 tasks.",
|
||||
"The working tree was clean at testedCommit and mounted read-only. The supervisor enforced a 300-second wall timeout and removed only its own container.",
|
||||
"Malformed-frame and header-limit tests deliberately trigger HTTP/2 error logs; each corresponding test passed.",
|
||||
"Race-detector memory in this regression log is not a production memory baseline. Ordinary-mode mixed-soak measurements are recorded separately."
|
||||
],
|
||||
"postRunCheck": {
|
||||
"checkedAt": "2026-09-23T02:53:33.308077+00:00",
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingHTTP2ValidationContainers": []
|
||||
}
|
||||
}
|
||||
53
performance/http2FlowControlLinux.txt
Normal file
53
performance/http2FlowControlLinux.txt
Normal file
@@ -0,0 +1,53 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestTLSConnectionLimitRecoversAfterHandshakeTimeout
|
||||
--- PASS: TestTLSConnectionLimitRecoversAfterHandshakeTimeout (1.12s)
|
||||
=== RUN TestSlowResponseStreamIsolationAndRecovery
|
||||
--- PASS: TestSlowResponseStreamIsolationAndRecovery (0.00s)
|
||||
=== RUN TestConnectionWindowIsolationAndRecovery
|
||||
--- PASS: TestConnectionWindowIsolationAndRecovery (0.00s)
|
||||
=== RUN TestShutdownReleasesFlowControlledWriter
|
||||
--- PASS: TestShutdownReleasesFlowControlledWriter (0.10s)
|
||||
=== RUN TestShutdownSendsGoAwayAndDrains
|
||||
--- PASS: TestShutdownSendsGoAwayAndDrains (0.00s)
|
||||
=== RUN TestRSTStreamCancelsOnlyOneRequest
|
||||
--- PASS: TestRSTStreamCancelsOnlyOneRequest (0.00s)
|
||||
=== RUN TestCanceledServeContextWaitsForShutdownDeadline
|
||||
--- PASS: TestCanceledServeContextWaitsForShutdownDeadline (0.10s)
|
||||
=== RUN TestTLSMixedSoak
|
||||
mixedSoak_test.go:37: phase=before elapsed=1ms requests=0 failures=0 heapBytes=2409792 rssBytes=37199872 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:38: scenario=mixed connections=8 workersPerConnection=16 uploadBytes=262144 uploadChunkBytes=4096 uploadDelay=2ms downloadBytes=8388608 downloadChunkBytes=65536 downloadDelay=20ms
|
||||
mixedSoak_test.go:190: phase=after elapsed=5.655s requests=10489 failures=0 heapBytes=62134360 rssBytes=470847488 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:4 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:124: mixedTotals phase=after health=10356 slowUploads=117 slowDownloads=16 uploadedBytes=30670848 downloadedBytes=134217728 serverFailures=0
|
||||
mixedSoak_test.go:196: connection=0 health=1318 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=1 health=1228 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=2 health=1194 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=3 health=1322 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=4 health=1331 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=5 health=1342 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=6 health=1321 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=7 health=1300 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:209: phase=afterReclaim elapsed=5.661s requests=10489 failures=0 heapBytes=440880 rssBytes=468951040 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:4 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSMixedSoak (5.66s)
|
||||
=== RUN TestServerHandlesH2CRequest
|
||||
--- PASS: TestServerHandlesH2CRequest (0.00s)
|
||||
=== RUN TestServerEnforcesGlobalRequestLimitAndRecovers
|
||||
--- PASS: TestServerEnforcesGlobalRequestLimitAndRecovers (0.00s)
|
||||
=== RUN TestServerRejectsBodyAboveLimitAndRecovers
|
||||
--- PASS: TestServerRejectsBodyAboveLimitAndRecovers (0.00s)
|
||||
=== RUN TestServerRejectsHeadersAboveLimitAndRecovers
|
||||
2026/09/23 02:51:04 http2: server connection error from 127.0.0.1:36276: connection error: COMPRESSION_ERROR
|
||||
--- PASS: TestServerRejectsHeadersAboveLimitAndRecovers (1.00s)
|
||||
=== RUN TestServerMultiplexes128Streams
|
||||
--- PASS: TestServerMultiplexes128Streams (0.02s)
|
||||
=== RUN TestServerStreamsLargeBodyWithFlowControl
|
||||
--- PASS: TestServerStreamsLargeBodyWithFlowControl (0.16s)
|
||||
=== RUN TestServerTLSUsesH2ALPN
|
||||
--- PASS: TestServerTLSUsesH2ALPN (0.14s)
|
||||
=== RUN TestTLSSoak
|
||||
soak_test.go:26: phase=before elapsed=0s requests=0 failures=0 heapBytes=1497944 rssBytes=391569408 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
soak_test.go:159: phase=after elapsed=3.122s requests=18182 failures=0 heapBytes=4739552 rssBytes=238292992 goroutines=3 fileDescriptors=6 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSSoak (3.12s)
|
||||
PASS
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 12.478s
|
||||
3681
performance/http2MixedSoakLinux.json
Normal file
3681
performance/http2MixedSoakLinux.json
Normal file
File diff suppressed because it is too large
Load Diff
265
performance/http2MixedSoakLinux.txt
Normal file
265
performance/http2MixedSoakLinux.txt
Normal file
@@ -0,0 +1,265 @@
|
||||
HTTP/2 Linux soak image: golang:1.25
|
||||
HTTP/2 TLS soak duration: 30m
|
||||
HTTP/2 TLS soak mode: regular
|
||||
HTTP/2 TLS soak scenario: mixed
|
||||
HTTP/2 soak container: go-omni-http2-soak-fa422c57dfa1; limits: 2 CPUs, 2048 MiB, 256 tasks
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestTLSMixedSoak
|
||||
mixedSoak_test.go:37: phase=before elapsed=0s requests=0 failures=0 heapBytes=170136 rssBytes=8613888 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:38: scenario=mixed connections=8 workersPerConnection=16 uploadBytes=262144 uploadChunkBytes=4096 uploadDelay=2ms downloadBytes=8388608 downloadChunkBytes=65536 downloadDelay=20ms
|
||||
mixedSoak_test.go:135: phase=running elapsed=15.143s requests=444111 failures=0 heapBytes=41226328 rssBytes=100597760 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=443187 slowUploads=884 slowDownloads=40 uploadedBytes=231735296 downloadedBytes=335544320 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=30.143s requests=886889 failures=0 heapBytes=76939344 rssBytes=119885824 goroutines=351 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=885025 slowUploads=1776 slowDownloads=88 uploadedBytes=465567744 downloadedBytes=738197504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=45.143s requests=1327236 failures=0 heapBytes=98442464 rssBytes=119926784 goroutines=348 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=1324437 slowUploads=2663 slowDownloads=136 uploadedBytes=698089472 downloadedBytes=1140850688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m0.143s requests=1768857 failures=0 heapBytes=99328992 rssBytes=119402496 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=1765116 slowUploads=3557 slowDownloads=184 uploadedBytes=932446208 downloadedBytes=1543503872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m15.143s requests=2211456 failures=0 heapBytes=60732216 rssBytes=90337280 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=2206781 slowUploads=4445 slowDownloads=232 uploadedBytes=1165230080 downloadedBytes=1946157056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m30.143s requests=2652108 failures=0 heapBytes=53063064 rssBytes=85688320 goroutines=350 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=2646501 slowUploads=5335 slowDownloads=272 uploadedBytes=1398538240 downloadedBytes=2281701376 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m45.143s requests=3094082 failures=0 heapBytes=81458456 rssBytes=117563392 goroutines=343 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3087534 slowUploads=6228 slowDownloads=320 uploadedBytes=1632632832 downloadedBytes=2684354560 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m0.143s requests=3532884 failures=0 heapBytes=86878776 rssBytes=119468032 goroutines=292 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3525398 slowUploads=7118 slowDownloads=368 uploadedBytes=1865940992 downloadedBytes=3087007744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m15.143s requests=3974010 failures=0 heapBytes=98205016 rssBytes=119029760 goroutines=313 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3965593 slowUploads=8008 slowDownloads=416 uploadedBytes=2099249152 downloadedBytes=3489660928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m30.143s requests=4415820 failures=0 heapBytes=90842632 rssBytes=109375488 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=4406456 slowUploads=8900 slowDownloads=464 uploadedBytes=2333081600 downloadedBytes=3892314112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m45.143s requests=4859795 failures=0 heapBytes=46212528 rssBytes=75120640 goroutines=382 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=4849501 slowUploads=9790 slowDownloads=504 uploadedBytes=2566389760 downloadedBytes=4227858432 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m0.143s requests=5302557 failures=0 heapBytes=59378560 rssBytes=103960576 goroutines=323 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=5291324 slowUploads=10681 slowDownloads=552 uploadedBytes=2799960064 downloadedBytes=4630511616 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m15.143s requests=5745355 failures=0 heapBytes=65581312 rssBytes=120012800 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=5733185 slowUploads=11571 slowDownloads=600 uploadedBytes=3033268224 downloadedBytes=5033164800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m30.143s requests=6184725 failures=0 heapBytes=83765136 rssBytes=119152640 goroutines=365 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=6171616 slowUploads=12461 slowDownloads=648 uploadedBytes=3266576384 downloadedBytes=5435817984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m45.143s requests=6627613 failures=0 heapBytes=83719272 rssBytes=119508992 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=6613566 slowUploads=13352 slowDownloads=696 uploadedBytes=3500146688 downloadedBytes=5838471168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m0.143s requests=7072007 failures=0 heapBytes=40007456 rssBytes=83099648 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7057021 slowUploads=14246 slowDownloads=740 uploadedBytes=3734503424 downloadedBytes=6207569920 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m15.143s requests=7515397 failures=0 heapBytes=52621088 rssBytes=88940544 goroutines=320 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7499478 slowUploads=15136 slowDownloads=784 uploadedBytes=3967811584 downloadedBytes=6576668672 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m30.143s requests=7957026 failures=0 heapBytes=49206800 rssBytes=115929088 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7940168 slowUploads=16026 slowDownloads=832 uploadedBytes=4201119744 downloadedBytes=6979321856 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m45.143s requests=8395015 failures=0 heapBytes=57830768 rssBytes=120049664 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=8377223 slowUploads=16918 slowDownloads=880 uploadedBytes=4434952192 downloadedBytes=7381975040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m0.143s requests=8838797 failures=0 heapBytes=66531296 rssBytes=120057856 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=8820061 slowUploads=17808 slowDownloads=928 uploadedBytes=4668260352 downloadedBytes=7784628224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m15.143s requests=9279778 failures=0 heapBytes=60693440 rssBytes=103329792 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=9260102 slowUploads=18700 slowDownloads=976 uploadedBytes=4902092800 downloadedBytes=8187281408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m30.143s requests=9726070 failures=0 heapBytes=29946072 rssBytes=75563008 goroutines=288 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=9705463 slowUploads=19591 slowDownloads=1016 uploadedBytes=5135663104 downloadedBytes=8522825728 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m45.144s requests=10170774 failures=0 heapBytes=41152368 rssBytes=106692608 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=10149227 slowUploads=20483 slowDownloads=1064 uploadedBytes=5369495552 downloadedBytes=8925478912 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m0.143s requests=10614632 failures=0 heapBytes=80360752 rssBytes=120602624 goroutines=325 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=10592151 slowUploads=21373 slowDownloads=1112 uploadedBytes=5602803712 downloadedBytes=9328132096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m15.143s requests=11059452 failures=0 heapBytes=87305144 rssBytes=120602624 goroutines=324 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11036026 slowUploads=22266 slowDownloads=1160 uploadedBytes=5836898304 downloadedBytes=9730785280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m30.143s requests=11499123 failures=0 heapBytes=95978912 rssBytes=120606720 goroutines=365 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11474760 slowUploads=23155 slowDownloads=1208 uploadedBytes=6069944320 downloadedBytes=10133438464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m45.143s requests=11944358 failures=0 heapBytes=56042736 rssBytes=75292672 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11919058 slowUploads=24048 slowDownloads=1252 uploadedBytes=6304038912 downloadedBytes=10502537216 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m0.143s requests=12386374 failures=0 heapBytes=40072440 rssBytes=94666752 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=12360140 slowUploads=24938 slowDownloads=1296 uploadedBytes=6537347072 downloadedBytes=10871635968 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m15.143s requests=12830700 failures=0 heapBytes=62440144 rssBytes=120365056 goroutines=328 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=12803533 slowUploads=25829 slowDownloads=1344 uploadedBytes=6770917376 downloadedBytes=11274289152 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m30.143s requests=13273492 failures=0 heapBytes=67101096 rssBytes=121159680 goroutines=334 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=13245381 slowUploads=26719 slowDownloads=1392 uploadedBytes=7004225536 downloadedBytes=11676942336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m45.143s requests=13715931 failures=0 heapBytes=78929632 rssBytes=120643584 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=13686880 slowUploads=27611 slowDownloads=1440 uploadedBytes=7238057984 downloadedBytes=12079595520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m0.143s requests=14158538 failures=0 heapBytes=83472520 rssBytes=102039552 goroutines=342 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=14128556 slowUploads=28500 slowDownloads=1488 uploadedBytes=7471104000 downloadedBytes=12482248704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m15.143s requests=14606106 failures=0 heapBytes=36808416 rssBytes=77897728 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=14575184 slowUploads=29394 slowDownloads=1528 uploadedBytes=7705460736 downloadedBytes=12817793024 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m30.143s requests=15050580 failures=0 heapBytes=51099992 rssBytes=106217472 goroutines=315 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15018720 slowUploads=30284 slowDownloads=1576 uploadedBytes=7938768896 downloadedBytes=13220446208 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m45.143s requests=15492752 failures=0 heapBytes=99120736 rssBytes=121679872 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15459953 slowUploads=31175 slowDownloads=1624 uploadedBytes=8172339200 downloadedBytes=13623099392 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m0.143s requests=15937860 failures=0 heapBytes=95959552 rssBytes=121716736 goroutines=344 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15904122 slowUploads=32066 slowDownloads=1672 uploadedBytes=8405909504 downloadedBytes=14025752576 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m15.143s requests=16379592 failures=0 heapBytes=54420112 rssBytes=121249792 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=16344914 slowUploads=32958 slowDownloads=1720 uploadedBytes=8639741952 downloadedBytes=14428405760 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m30.143s requests=16825611 failures=0 heapBytes=62855872 rssBytes=81920000 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=16790004 slowUploads=33848 slowDownloads=1764 uploadedBytes=8873050112 downloadedBytes=14797504512 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m45.144s requests=17273037 failures=0 heapBytes=56223560 rssBytes=98967552 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=17236495 slowUploads=34739 slowDownloads=1808 uploadedBytes=9106620416 downloadedBytes=15166603264 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m0.143s requests=17718289 failures=0 heapBytes=90363368 rssBytes=121196544 goroutines=312 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=17680804 slowUploads=35632 slowDownloads=1856 uploadedBytes=9340715008 downloadedBytes=15569256448 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m15.143s requests=18164393 failures=0 heapBytes=91620912 rssBytes=121708544 goroutines=328 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=18125968 slowUploads=36521 slowDownloads=1904 uploadedBytes=9573761024 downloadedBytes=15971909632 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m30.143s requests=18604920 failures=0 heapBytes=85189216 rssBytes=119672832 goroutines=397 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=18565557 slowUploads=37412 slowDownloads=1952 uploadedBytes=9807331328 downloadedBytes=16374562816 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m45.143s requests=19049168 failures=0 heapBytes=71446040 rssBytes=97730560 goroutines=371 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19008872 slowUploads=38304 slowDownloads=2000 uploadedBytes=10041163776 downloadedBytes=16777216000 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m0.143s requests=19496908 failures=0 heapBytes=35314648 rssBytes=81412096 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:6}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19455674 slowUploads=39195 slowDownloads=2040 uploadedBytes=10274734080 downloadedBytes=17112760320 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m15.143s requests=19941947 failures=0 heapBytes=52063200 rssBytes=112910336 goroutines=353 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19899771 slowUploads=40088 slowDownloads=2088 uploadedBytes=10508828672 downloadedBytes=17515413504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m30.143s requests=20387835 failures=0 heapBytes=91179064 rssBytes=121262080 goroutines=337 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=20344721 slowUploads=40978 slowDownloads=2136 uploadedBytes=10742136832 downloadedBytes=17918066688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m45.143s requests=20834104 failures=0 heapBytes=96727232 rssBytes=121204736 goroutines=373 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=20790049 slowUploads=41871 slowDownloads=2184 uploadedBytes=10976231424 downloadedBytes=18320719872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m0.143s requests=21274032 failures=0 heapBytes=73363000 rssBytes=121729024 goroutines=395 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=21229039 slowUploads=42761 slowDownloads=2232 uploadedBytes=11209539584 downloadedBytes=18723373056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m15.143s requests=21717175 failures=0 heapBytes=55163016 rssBytes=80625664 goroutines=345 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=21671245 slowUploads=43654 slowDownloads=2276 uploadedBytes=11443634176 downloadedBytes=19092471808 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m30.143s requests=22165567 failures=0 heapBytes=58932064 rssBytes=100704256 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22118701 slowUploads=44546 slowDownloads=2320 uploadedBytes=11677466624 downloadedBytes=19461570560 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m45.143s requests=22605128 failures=0 heapBytes=65821152 rssBytes=121774080 goroutines=295 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22557331 slowUploads=45436 slowDownloads=2368 uploadedBytes=11910774784 downloadedBytes=19864223744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m0.143s requests=23048491 failures=0 heapBytes=71758864 rssBytes=121741312 goroutines=346 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22999748 slowUploads=46327 slowDownloads=2416 uploadedBytes=12144345088 downloadedBytes=20266876928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m15.143s requests=23490751 failures=0 heapBytes=93734424 rssBytes=121221120 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=23441069 slowUploads=47219 slowDownloads=2464 uploadedBytes=12378177536 downloadedBytes=20669530112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m30.143s requests=23937415 failures=0 heapBytes=67599608 rssBytes=86278144 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=23886795 slowUploads=48109 slowDownloads=2511 uploadedBytes=12611485696 downloadedBytes=21063794688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m45.143s requests=24380375 failures=0 heapBytes=39698288 rssBytes=80867328 goroutines=299 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=24328828 slowUploads=49001 slowDownloads=2553 uploadedBytes=12845318144 downloadedBytes=21416116224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m0.143s requests=24821991 failures=0 heapBytes=56792672 rssBytes=110620672 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=24769501 slowUploads=49890 slowDownloads=2600 uploadedBytes=13078364160 downloadedBytes=21810380800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m15.143s requests=25267435 failures=0 heapBytes=59737600 rssBytes=121171968 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=25214005 slowUploads=50784 slowDownloads=2648 uploadedBytes=13312720896 downloadedBytes=22213033984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m30.143s requests=25712957 failures=0 heapBytes=91392208 rssBytes=119664640 goroutines=341 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=25658586 slowUploads=51675 slowDownloads=2696 uploadedBytes=13546291200 downloadedBytes=22615687168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m45.143s requests=26155555 failures=0 heapBytes=57260256 rssBytes=109522944 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26100248 slowUploads=52568 slowDownloads=2744 uploadedBytes=13780385792 downloadedBytes=23018340352 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m0.143s requests=26597733 failures=0 heapBytes=40190080 rssBytes=81768448 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26541487 slowUploads=53458 slowDownloads=2788 uploadedBytes=14013693952 downloadedBytes=23387439104 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m15.143s requests=27037735 failures=0 heapBytes=66516128 rssBytes=100077568 goroutines=362 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26980556 slowUploads=54350 slowDownloads=2832 uploadedBytes=14247526400 downloadedBytes=23756537856 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m30.143s requests=27476076 failures=0 heapBytes=60893368 rssBytes=121782272 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=27417961 slowUploads=55241 slowDownloads=2880 uploadedBytes=14481096704 downloadedBytes=24159191040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m45.143s requests=27919500 failures=0 heapBytes=88112872 rssBytes=121765888 goroutines=349 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=27860441 slowUploads=56131 slowDownloads=2928 uploadedBytes=14714404864 downloadedBytes=24561844224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m0.143s requests=28363283 failures=0 heapBytes=76293576 rssBytes=121774080 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=28303284 slowUploads=57023 slowDownloads=2976 uploadedBytes=14948237312 downloadedBytes=24964497408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m15.143s requests=28803143 failures=0 heapBytes=50553176 rssBytes=94076928 goroutines=335 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=28742207 slowUploads=57913 slowDownloads=3023 uploadedBytes=15181545472 downloadedBytes=25358761984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m30.143s requests=29248721 failures=0 heapBytes=50037296 rssBytes=88997888 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=29186854 slowUploads=58806 slowDownloads=3064 uploadedBytes=15415640064 downloadedBytes=25702694912 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m45.143s requests=29691598 failures=0 heapBytes=56490576 rssBytes=112680960 goroutines=339 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=29628794 slowUploads=59696 slowDownloads=3112 uploadedBytes=15648948224 downloadedBytes=26105348096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m0.143s requests=30135271 failures=0 heapBytes=101057680 rssBytes=121397248 goroutines=350 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30071522 slowUploads=60589 slowDownloads=3160 uploadedBytes=15883042816 downloadedBytes=26508001280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m15.143s requests=30578566 failures=0 heapBytes=102154656 rssBytes=121921536 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:27}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30513878 slowUploads=61480 slowDownloads=3208 uploadedBytes=16116613120 downloadedBytes=26910654464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m30.143s requests=31020857 failures=0 heapBytes=86815776 rssBytes=105476096 goroutines=323 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30955229 slowUploads=62372 slowDownloads=3256 uploadedBytes=16350445568 downloadedBytes=27313307648 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m45.143s requests=31459944 failures=0 heapBytes=51699872 rssBytes=79634432 goroutines=310 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=31393379 slowUploads=63265 slowDownloads=3300 uploadedBytes=16584540160 downloadedBytes=27682406400 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m0.143s requests=31903673 failures=0 heapBytes=77125768 rssBytes=103821312 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=31836174 slowUploads=64155 slowDownloads=3344 uploadedBytes=16817848320 downloadedBytes=28051505152 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m15.143s requests=32347088 failures=0 heapBytes=87810928 rssBytes=121810944 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=32278650 slowUploads=65047 slowDownloads=3392 uploadedBytes=17051680768 downloadedBytes=28454158336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m30.143s requests=32791958 failures=0 heapBytes=83378912 rssBytes=121843712 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=32722582 slowUploads=65940 slowDownloads=3440 uploadedBytes=17285775360 downloadedBytes=28856811520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m45.143s requests=33238456 failures=0 heapBytes=73365344 rssBytes=121778176 goroutines=310 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=33168137 slowUploads=66831 slowDownloads=3488 uploadedBytes=17519345664 downloadedBytes=29259464704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m0.143s requests=33678524 failures=0 heapBytes=73470384 rssBytes=96964608 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=33607268 slowUploads=67723 slowDownloads=3533 uploadedBytes=17753178112 downloadedBytes=29636952064 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m15.143s requests=34124154 failures=0 heapBytes=44371688 rssBytes=86138880 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34051968 slowUploads=68615 slowDownloads=3576 uploadedBytes=17987010560 downloadedBytes=29997662208 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m30.143s requests=34567776 failures=0 heapBytes=69339968 rssBytes=116219904 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34494646 slowUploads=69506 slowDownloads=3624 uploadedBytes=18220580864 downloadedBytes=30400315392 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m45.143s requests=35012899 failures=0 heapBytes=77088312 rssBytes=122314752 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34938829 slowUploads=70399 slowDownloads=3672 uploadedBytes=18454675456 downloadedBytes=30802968576 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m0.143s requests=35459295 failures=0 heapBytes=71491968 rssBytes=121794560 goroutines=346 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=35384287 slowUploads=71292 slowDownloads=3720 uploadedBytes=18688770048 downloadedBytes=31205621760 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m15.143s requests=35904737 failures=0 heapBytes=71675952 rssBytes=106598400 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=35828787 slowUploads=72182 slowDownloads=3768 uploadedBytes=18922078208 downloadedBytes=31608274944 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m30.143s requests=36350214 failures=0 heapBytes=50574208 rssBytes=84537344 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:10}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=36273329 slowUploads=73074 slowDownloads=3811 uploadedBytes=19155910656 downloadedBytes=31968985088 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m45.143s requests=36794194 failures=0 heapBytes=52755624 rssBytes=104304640 goroutines=347 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=36716374 slowUploads=73964 slowDownloads=3856 uploadedBytes=19389218816 downloadedBytes=32346472448 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m0.143s requests=37237267 failures=0 heapBytes=90759440 rssBytes=121802752 goroutines=285 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=37158506 slowUploads=74857 slowDownloads=3904 uploadedBytes=19623313408 downloadedBytes=32749125632 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m15.143s requests=37682135 failures=0 heapBytes=75071160 rssBytes=121802752 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=37602436 slowUploads=75747 slowDownloads=3952 uploadedBytes=19856621568 downloadedBytes=33151778816 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m30.143s requests=38125279 failures=0 heapBytes=84651016 rssBytes=117342208 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38044640 slowUploads=76639 slowDownloads=4000 uploadedBytes=20090454016 downloadedBytes=33554432000 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m45.143s requests=38569220 failures=0 heapBytes=70593904 rssBytes=97050624 goroutines=373 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38487645 slowUploads=77531 slowDownloads=4044 uploadedBytes=20324286464 downloadedBytes=33923530752 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m0.143s requests=39016925 failures=0 heapBytes=59654016 rssBytes=93556736 goroutines=325 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38934413 slowUploads=78424 slowDownloads=4088 uploadedBytes=20558381056 downloadedBytes=34292629504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m15.143s requests=39463910 failures=0 heapBytes=53261856 rssBytes=114139136 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=39380459 slowUploads=79315 slowDownloads=4136 uploadedBytes=20791951360 downloadedBytes=34695282688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m30.143s requests=39910237 failures=0 heapBytes=62860672 rssBytes=121815040 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=39825849 slowUploads=80209 slowDownloads=4184 uploadedBytes=21026308096 downloadedBytes=35097935872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m45.143s requests=40360417 failures=0 heapBytes=70322976 rssBytes=121286656 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=40275086 slowUploads=81099 slowDownloads=4232 uploadedBytes=21259616256 downloadedBytes=35500589056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m0.143s requests=40801788 failures=0 heapBytes=76912600 rssBytes=97243136 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=40715518 slowUploads=81990 slowDownloads=4280 uploadedBytes=21493186560 downloadedBytes=35903242240 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m15.143s requests=41246645 failures=0 heapBytes=54950248 rssBytes=84979712 goroutines=337 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=41159438 slowUploads=82883 slowDownloads=4324 uploadedBytes=21727281152 downloadedBytes=36272340992 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m30.143s requests=41687236 failures=0 heapBytes=43818576 rssBytes=101183488 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=41599097 slowUploads=83774 slowDownloads=4368 uploadedBytes=21960851456 downloadedBytes=36641439744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m45.143s requests=42128546 failures=0 heapBytes=54495128 rssBytes=121827328 goroutines=288 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42039470 slowUploads=84667 slowDownloads=4416 uploadedBytes=22194946048 downloadedBytes=37044092928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m0.143s requests=42569443 failures=0 heapBytes=98238024 rssBytes=119263232 goroutines=362 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42479426 slowUploads=85556 slowDownloads=4464 uploadedBytes=22427992064 downloadedBytes=37446746112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m15.143s requests=43006434 failures=0 heapBytes=61451176 rssBytes=115818496 goroutines=330 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42915474 slowUploads=86448 slowDownloads=4512 uploadedBytes=22661824512 downloadedBytes=37849399296 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m30.143s requests=43446211 failures=0 heapBytes=53221152 rssBytes=96935936 goroutines=294 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=43354315 slowUploads=87339 slowDownloads=4557 uploadedBytes=22895394816 downloadedBytes=38226886656 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m45.143s requests=43888201 failures=0 heapBytes=41094952 rssBytes=88412160 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=43795377 slowUploads=88230 slowDownloads=4600 uploadedBytes=23128965120 downloadedBytes=38587596800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m0.143s requests=44331683 failures=0 heapBytes=74520696 rssBytes=117968896 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=44237912 slowUploads=89123 slowDownloads=4648 uploadedBytes=23363059712 downloadedBytes=38990249984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m15.143s requests=44771023 failures=0 heapBytes=61522184 rssBytes=121311232 goroutines=314 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=44676311 slowUploads=90016 slowDownloads=4696 uploadedBytes=23597154304 downloadedBytes=39392903168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m30.143s requests=45210979 failures=0 heapBytes=71823520 rssBytes=121315328 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=45115328 slowUploads=90908 slowDownloads=4744 uploadedBytes=23830986752 downloadedBytes=39795556352 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m45.143s requests=45655488 failures=0 heapBytes=73294312 rssBytes=98951168 goroutines=352 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=45558899 slowUploads=91798 slowDownloads=4791 uploadedBytes=24064294912 downloadedBytes=40189820928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m0.143s requests=46098111 failures=0 heapBytes=61891384 rssBytes=85794816 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46000586 slowUploads=92690 slowDownloads=4835 uploadedBytes=24298127360 downloadedBytes=40558919680 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m15.143s requests=46542135 failures=0 heapBytes=60185208 rssBytes=103088128 goroutines=303 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46443672 slowUploads=93583 slowDownloads=4880 uploadedBytes=24532221952 downloadedBytes=40936407040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m30.143s requests=46985865 failures=0 heapBytes=82859896 rssBytes=121839616 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46886464 slowUploads=94473 slowDownloads=4928 uploadedBytes=24765530112 downloadedBytes=41339060224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m45.143s requests=47426140 failures=0 heapBytes=76512768 rssBytes=121827328 goroutines=319 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=47325797 slowUploads=95367 slowDownloads=4976 uploadedBytes=24999886848 downloadedBytes=41741713408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m0.143s requests=47864835 failures=0 heapBytes=96303144 rssBytes=115740672 goroutines=380 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=47763553 slowUploads=96258 slowDownloads=5024 uploadedBytes=25233457152 downloadedBytes=42144366592 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m15.143s requests=48307600 failures=0 heapBytes=74108016 rssBytes=93282304 goroutines=358 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=48205383 slowUploads=97149 slowDownloads=5068 uploadedBytes=25467027456 downloadedBytes=42513465344 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m30.143s requests=48750920 failures=0 heapBytes=48575472 rssBytes=90746880 goroutines=376 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=48647765 slowUploads=98043 slowDownloads=5112 uploadedBytes=25701384192 downloadedBytes=42882564096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m45.143s requests=49191794 failures=0 heapBytes=89907488 rssBytes=120475648 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49087700 slowUploads=98934 slowDownloads=5160 uploadedBytes=25934954496 downloadedBytes=43285217280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m0.143s requests=49637025 failures=0 heapBytes=71241912 rssBytes=120827904 goroutines=338 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49531996 slowUploads=99826 slowDownloads=5208 uploadedBytes=26168786944 downloadedBytes=43687870464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m15.143s requests=50077201 failures=0 heapBytes=71823760 rssBytes=121327616 goroutines=342 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49971226 slowUploads=100719 slowDownloads=5256 uploadedBytes=26402881536 downloadedBytes=44090523648 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m30.143s requests=50524644 failures=0 heapBytes=88321680 rssBytes=107315200 goroutines=315 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=50417733 slowUploads=101610 slowDownloads=5303 uploadedBytes=26636451840 downloadedBytes=44484788224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m45.143s requests=50971603 failures=0 heapBytes=49721352 rssBytes=87646208 goroutines=335 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=50863761 slowUploads=102501 slowDownloads=5346 uploadedBytes=26870022144 downloadedBytes=44845498368 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m0.143s requests=51417802 failures=0 heapBytes=55859872 rssBytes=104423424 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=51309017 slowUploads=103394 slowDownloads=5392 uploadedBytes=27104116736 downloadedBytes=45231374336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m15.143s requests=51862535 failures=0 heapBytes=63482984 rssBytes=121856000 goroutines=349 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:10}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=51752810 slowUploads=104285 slowDownloads=5440 uploadedBytes=27337687040 downloadedBytes=45634027520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m30.143s requests=52305456 failures=0 heapBytes=85512792 rssBytes=118308864 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=52194795 slowUploads=105177 slowDownloads=5488 uploadedBytes=27571519488 downloadedBytes=46036680704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m45.143s requests=52747376 failures=0 heapBytes=74143584 rssBytes=114286592 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=52635773 slowUploads=106069 slowDownloads=5536 uploadedBytes=27805351936 downloadedBytes=46439333888 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=30m0.143s requests=53191576 failures=0 heapBytes=43650296 rssBytes=97910784 goroutines=272 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=53079033 slowUploads=106963 slowDownloads=5580 uploadedBytes=28039708672 downloadedBytes=46808432640 serverFailures=0
|
||||
mixedSoak_test.go:190: phase=after elapsed=30m2.635s requests=53191687 failures=0 heapBytes=44586352 rssBytes=97927168 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:5 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:124: mixedTotals phase=after health=53079128 slowUploads=106971 slowDownloads=5588 uploadedBytes=28041805824 downloadedBytes=46875541504 serverFailures=0
|
||||
mixedSoak_test.go:196: connection=0 health=6632549 slowUploads=13372 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=1 health=6629530 slowUploads=13371 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=2 health=6636544 slowUploads=13376 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=3 health=6634170 slowUploads=13369 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=4 health=6636236 slowUploads=13368 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=5 health=6640096 slowUploads=13371 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=6 health=6634043 slowUploads=13372 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=7 health=6635960 slowUploads=13372 slowDownloads=699
|
||||
mixedSoak_test.go:209: phase=afterReclaim elapsed=30m2.639s requests=53191687 failures=0 heapBytes=461176 rssBytes=95506432 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:5 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSMixedSoak (1802.64s)
|
||||
PASS
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 1802.651s
|
||||
PASS HTTP/2 TLS soak
|
||||
42
performance/http2ProxyCleanupLinux.json
Normal file
42
performance/http2ProxyCleanupLinux.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"runnerSha256": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"startedAt": "2026-09-23T03:13:01.977797+00:00",
|
||||
"cases": [
|
||||
{
|
||||
"case": "wallTimeout",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "Command '['/usr/bin/docker', 'run', '--rm', '--name', 'go-omni-http2-proxy-ba2aaeecdd00-build', '--cpus', '2', '--memory', '2048m', '--memory-swap', '2048m', '--pids-limit', '256', '--log-opt', 'max-size=10m', '--log-opt', 'max-file=1', '-v', '/root/go-omni-validation:/workspace:ro', '-v', '/tmp/goOmniHTTP2Proxyi0b0o9y6:/output', '-w', '/workspace', '-e', 'GOWORK=off', '-e', 'GOMAXPROCS=2', '-e', 'CGO_ENABLED=0', '-e', 'GOPROXY=https://goproxy.cn,direct', 'golang:1.25', 'sh', '-c', 'go version && go vet ./transport/http2 && go test -c -o /output/http2.test ./transport/http2']' timed out after 1.8251966070383787 seconds",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-ba2aaeecdd00-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
},
|
||||
{
|
||||
"case": "sigterm",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "KeyboardInterrupt",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-6307142e05ee-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
},
|
||||
{
|
||||
"case": "sighup",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "KeyboardInterrupt",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-dcf3bd54fbe4-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
}
|
||||
],
|
||||
"status": "passed",
|
||||
"finishedAt": "2026-09-23T03:13:06.934079+00:00"
|
||||
}
|
||||
142
performance/http2ProxyLinux.json
Normal file
142
performance/http2ProxyLinux.json
Normal file
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T03:10:31.408631+00:00",
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"go.mod": "28d68eb2c0a019cceea884e88ab2335f44ad930e1890e28871638f4b073b4f86",
|
||||
"go.sum": "d9c2a5d0de94f08bd18a14a9626f9dda559c245d89bf8a689e32acbd2136fb05",
|
||||
"scripts/verifyHTTP2Proxy.py": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"transport/http2/admission_test.go": "342a7d02bba508100a207b77246ca92e0fa04dd5f8e4a27091a01866b6750d39",
|
||||
"transport/http2/flowControl_test.go": "595b2e937cef2142ff6b6386116e338d874d0fe49e63d80f6de2e1268378dba2",
|
||||
"transport/http2/frameObserver.go": "fcdf431f8269ad9cfd65dd14102ce5c42ac8c4b8b833ed42c4ae15f2a7c7f14c",
|
||||
"transport/http2/lifecycle_test.go": "370a16de8b20248bb78021ab338b9f62ec51f733f270c0fa92f48b1a3ccb9eae",
|
||||
"transport/http2/mixedSoak_test.go": "2111737fe0405bc809bf06325573f7356160ae7fcaebcce06dcd94b2d9e7dccd",
|
||||
"transport/http2/proxy_test.go": "808d0fa7e6c3b0237b1adf4a71a4f30ae2dbc7d1a9d53fab164cda33b15c9d69",
|
||||
"transport/http2/server.go": "aeecdc3aec42219862d70ec788bef5df332189895555bc5d390849a51bd0fbfb",
|
||||
"transport/http2/server_test.go": "c4613e77fa885b6362c06c393937c7dcf9a7ffb4a9a32f85e21fc2f741de81a6",
|
||||
"transport/http2/soak_test.go": "52baa37e37dba497c579b6b55d723bda866ac57676254dc878c511345acc5af2"
|
||||
},
|
||||
"repetitions": 3,
|
||||
"race": false,
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 600,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"nginx:1.30.5-alpine": "sha256:bf3201ab56f23e5954646379c775d511fc466e9f11376d9725361064ad07ed35"
|
||||
},
|
||||
"notes": [
|
||||
"Compiler and integration stages run sequentially; at most one 2-CPU/2-GiB validation container is active.",
|
||||
"The integration container has network=none, no published ports and a read-only root filesystem. Client, Nginx and origins communicate over loopback inside that container.",
|
||||
"Both TLS legs validate certificates; invalid upstream CA and hostname must fail before reaching the handler.",
|
||||
"HTTP/2 is verified independently on both legs. This is a functional integration test, not a throughput benchmark or a live Cloudflare deployment.",
|
||||
"Single-process Nginx fixture with generated test certificates. Rolling drain switches between old and backup origins; this does not test Nginx master/worker configuration reloads or live Cloudflare routing."
|
||||
],
|
||||
"proxyVersions": [
|
||||
"nginx version: nginx/1.30.5"
|
||||
],
|
||||
"results": [
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}"
|
||||
],
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/amd64",
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-52da1679e6df-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxyk3cc3h1_:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"CGO_ENABLED=0",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go vet ./transport/http2 && go test -c -o /output/http2.test ./transport/http2"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-52da1679e6df-check",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=128m",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxyk3cc3h1_:/test:ro",
|
||||
"--entrypoint",
|
||||
"/test/http2.test",
|
||||
"-e",
|
||||
"GO_OMNI_HTTP2_PROXY_TEST=1",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"nginx:1.30.5-alpine",
|
||||
"-test.v",
|
||||
"-test.run",
|
||||
"^TestNginxHTTP2Proxy$",
|
||||
"-test.timeout",
|
||||
"120s",
|
||||
"-test.count",
|
||||
"3"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-52da1679e6df-build",
|
||||
"go-omni-http2-proxy-52da1679e6df-check"
|
||||
],
|
||||
"finishedAt": "2026-09-23T03:11:00.363119+00:00",
|
||||
"logSha256": "1263ceb8ce7b65f542898e17031ba56127672d363defd0ee6520199fa1927472",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
40
performance/http2ProxyLinux.txt
Normal file
40
performance/http2ProxyLinux.txt
Normal file
@@ -0,0 +1,40 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (1.03s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.39s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.42s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.22s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.81s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.17s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.45s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.20s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.89s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.21s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.44s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.23s)
|
||||
PASS
|
||||
147
performance/http2ProxyRaceLinux.json
Normal file
147
performance/http2ProxyRaceLinux.json
Normal file
@@ -0,0 +1,147 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T03:11:00.470197+00:00",
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"go.mod": "28d68eb2c0a019cceea884e88ab2335f44ad930e1890e28871638f4b073b4f86",
|
||||
"go.sum": "d9c2a5d0de94f08bd18a14a9626f9dda559c245d89bf8a689e32acbd2136fb05",
|
||||
"scripts/verifyHTTP2Proxy.py": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"transport/http2/admission_test.go": "342a7d02bba508100a207b77246ca92e0fa04dd5f8e4a27091a01866b6750d39",
|
||||
"transport/http2/flowControl_test.go": "595b2e937cef2142ff6b6386116e338d874d0fe49e63d80f6de2e1268378dba2",
|
||||
"transport/http2/frameObserver.go": "fcdf431f8269ad9cfd65dd14102ce5c42ac8c4b8b833ed42c4ae15f2a7c7f14c",
|
||||
"transport/http2/lifecycle_test.go": "370a16de8b20248bb78021ab338b9f62ec51f733f270c0fa92f48b1a3ccb9eae",
|
||||
"transport/http2/mixedSoak_test.go": "2111737fe0405bc809bf06325573f7356160ae7fcaebcce06dcd94b2d9e7dccd",
|
||||
"transport/http2/proxy_test.go": "808d0fa7e6c3b0237b1adf4a71a4f30ae2dbc7d1a9d53fab164cda33b15c9d69",
|
||||
"transport/http2/server.go": "aeecdc3aec42219862d70ec788bef5df332189895555bc5d390849a51bd0fbfb",
|
||||
"transport/http2/server_test.go": "c4613e77fa885b6362c06c393937c7dcf9a7ffb4a9a32f85e21fc2f741de81a6",
|
||||
"transport/http2/soak_test.go": "52baa37e37dba497c579b6b55d723bda866ac57676254dc878c511345acc5af2"
|
||||
},
|
||||
"repetitions": 3,
|
||||
"race": true,
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 600,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"nginx:1.30.5": "sha256:0aa2d81d65bc0cac0407e738b8f07d312c8685a84225fcb4db7bcbdd8c9bdf11"
|
||||
},
|
||||
"notes": [
|
||||
"Compiler and integration stages run sequentially; at most one 2-CPU/2-GiB validation container is active.",
|
||||
"The integration container has network=none, no published ports and a read-only root filesystem. Client, Nginx and origins communicate over loopback inside that container.",
|
||||
"Both TLS legs validate certificates; invalid upstream CA and hostname must fail before reaching the handler.",
|
||||
"HTTP/2 is verified independently on both legs. This is a functional integration test, not a throughput benchmark or a live Cloudflare deployment.",
|
||||
"Single-process Nginx fixture with generated test certificates. Rolling drain switches between old and backup origins; this does not test Nginx master/worker configuration reloads or live Cloudflare routing."
|
||||
],
|
||||
"proxyVersions": [
|
||||
"nginx version: nginx/1.30.5"
|
||||
],
|
||||
"results": [
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}"
|
||||
],
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/amd64",
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"pull",
|
||||
"nginx:1.30.5"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-ca37da979ea6-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxy8ll9pvvs:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"CGO_ENABLED=1",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go vet ./transport/http2 && go test -race -c -o /output/http2.test ./transport/http2"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-ca37da979ea6-check",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=128m",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxy8ll9pvvs:/test:ro",
|
||||
"--entrypoint",
|
||||
"/test/http2.test",
|
||||
"-e",
|
||||
"GO_OMNI_HTTP2_PROXY_TEST=1",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"nginx:1.30.5",
|
||||
"-test.v",
|
||||
"-test.run",
|
||||
"^TestNginxHTTP2Proxy$",
|
||||
"-test.timeout",
|
||||
"120s",
|
||||
"-test.count",
|
||||
"3"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-ca37da979ea6-build",
|
||||
"go-omni-http2-proxy-ca37da979ea6-check"
|
||||
],
|
||||
"finishedAt": "2026-09-23T03:12:51.684028+00:00",
|
||||
"logSha256": "aaae7678dd4f882c51c008eaeb78afd897ccf2f7ba31c926a40839de755bb595",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
67
performance/http2ProxyRaceLinux.txt
Normal file
67
performance/http2ProxyRaceLinux.txt
Normal file
@@ -0,0 +1,67 @@
|
||||
1.30.5: Pulling from library/nginx
|
||||
757049f14b94: Pulling fs layer
|
||||
af529266d394: Pulling fs layer
|
||||
3a55ec62cf16: Pulling fs layer
|
||||
2aa35770407f: Pulling fs layer
|
||||
f3cb4f951034: Pulling fs layer
|
||||
6b37362b3da7: Pulling fs layer
|
||||
b9614c6dbd86: Pulling fs layer
|
||||
2aa35770407f: Download complete
|
||||
d6363fd2ac35: Download complete
|
||||
3a55ec62cf16: Download complete
|
||||
02ac47b2a7a6: Download complete
|
||||
af529266d394: Download complete
|
||||
757049f14b94: Download complete
|
||||
f3cb4f951034: Download complete
|
||||
6b37362b3da7: Download complete
|
||||
6b37362b3da7: Pull complete
|
||||
b9614c6dbd86: Download complete
|
||||
3a55ec62cf16: Pull complete
|
||||
f3cb4f951034: Pull complete
|
||||
b9614c6dbd86: Pull complete
|
||||
2aa35770407f: Pull complete
|
||||
af529266d394: Pull complete
|
||||
757049f14b94: Pull complete
|
||||
Digest: sha256:0aa2d81d65bc0cac0407e738b8f07d312c8685a84225fcb4db7bcbdd8c9bdf11
|
||||
Status: Downloaded newer image for nginx:1.30.5
|
||||
docker.io/library/nginx:1.30.5
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.92s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.31s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.41s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.19s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (1.14s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.28s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.56s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.30s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.92s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.31s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.44s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.16s)
|
||||
PASS
|
||||
84
performance/http2SmokeLinux.json
Normal file
84
performance/http2SmokeLinux.json
Normal file
@@ -0,0 +1,84 @@
|
||||
{
|
||||
"createdAt": "2026-09-22T17:22:44.674797+00:00",
|
||||
"platform": "linux/amd64",
|
||||
"image": "golang:1.25",
|
||||
"imageId": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"sourceSha256": {
|
||||
"scripts/verifyHTTP2.py": "2c9f4ca095e16e9b30b80b5d812b2644577222274e5a8041e0fb178e305197a4",
|
||||
"transport/http2/soak_test.go": "9964a1670c8ffe30b27e96375e0ae72d00996b59beede748756013e8fd52eacf"
|
||||
},
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pids": 256
|
||||
},
|
||||
"gomaxprocs": 2,
|
||||
"workers": 128,
|
||||
"notes": [
|
||||
"Bounded smoke validation, not completion of the 30-minute HTTP/2 acceptance gate.",
|
||||
"TLS HTTP/2 over container loopback, with the client and server in one process; memory is not server-only.",
|
||||
"One TCP connection multiplexes the 128 workers; each response checks HTTP/2, status, body and a unique request ID.",
|
||||
"MiB uses 1024*1024 bytes. RSS is /proc/self/statm; heap is runtime.ReadMemStats. Samples every 15 seconds.",
|
||||
"The after sample includes the one monitoring goroutine; it is stopped and joined before the test returns.",
|
||||
"All seven pre-existing containers remained running, and no soak containers remained after cleanup.",
|
||||
"The prior unbounded 30-minute race run was stopped after RSS growth; its result is not counted as a pass.",
|
||||
"The short A/B comparison does not establish the complete cause of the old server incident."
|
||||
],
|
||||
"results": [
|
||||
{
|
||||
"mode": "regular",
|
||||
"duration": "2m",
|
||||
"elapsedSeconds": 120.04,
|
||||
"requests": 3885273,
|
||||
"failures": 0,
|
||||
"protocolErrors": 0,
|
||||
"sampledPeakRssBytes": 22437888,
|
||||
"before": {
|
||||
"heapBytes": 168040,
|
||||
"rssBytes": 8491008,
|
||||
"goroutines": 2,
|
||||
"fileDescriptors": 6
|
||||
},
|
||||
"after": {
|
||||
"heapBytes": 3925848,
|
||||
"rssBytes": 21364736,
|
||||
"goroutines": 3,
|
||||
"fileDescriptors": 6,
|
||||
"activeConnections": 0,
|
||||
"activeRequests": 0
|
||||
},
|
||||
"exitCode": 0
|
||||
},
|
||||
{
|
||||
"mode": "race",
|
||||
"duration": "60s",
|
||||
"elapsedSeconds": 60.127,
|
||||
"requests": 377788,
|
||||
"failures": 0,
|
||||
"protocolErrors": 0,
|
||||
"sampledPeakRssBytes": 492707840,
|
||||
"before": {
|
||||
"heapBytes": 169256,
|
||||
"rssBytes": 23732224,
|
||||
"goroutines": 2,
|
||||
"fileDescriptors": 6
|
||||
},
|
||||
"after": {
|
||||
"heapBytes": 2722632,
|
||||
"rssBytes": 487239680,
|
||||
"goroutines": 3,
|
||||
"fileDescriptors": 6,
|
||||
"activeConnections": 0,
|
||||
"activeRequests": 0
|
||||
},
|
||||
"exitCode": 0
|
||||
}
|
||||
],
|
||||
"cleanupValidation": {
|
||||
"sigterm": "passed",
|
||||
"sighup": "passed",
|
||||
"wallTimeout": "passed"
|
||||
},
|
||||
"thirtyMinuteAcceptance": "pending"
|
||||
}
|
||||
2624
performance/http2SoakLinux.json
Normal file
2624
performance/http2SoakLinux.json
Normal file
File diff suppressed because it is too large
Load Diff
133
performance/http2SoakLinux.txt
Normal file
133
performance/http2SoakLinux.txt
Normal file
@@ -0,0 +1,133 @@
|
||||
HTTP/2 Linux soak image: golang:1.25
|
||||
HTTP/2 TLS soak duration: 30m
|
||||
HTTP/2 TLS soak mode: regular
|
||||
HTTP/2 soak container: go-omni-http2-soak-b8d45477120a; limits: 2 CPUs, 2048 MiB, 256 tasks
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestTLSSoak
|
||||
soak_test.go:33: phase=before elapsed=0s requests=0 failures=0 heapBytes=168680 rssBytes=8441856 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=15.06s requests=483494 failures=0 heapBytes=6913536 rssBytes=22216704 goroutines=240 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=30.06s requests=961350 failures=0 heapBytes=6818840 rssBytes=22302720 goroutines=327 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=45.06s requests=1434755 failures=0 heapBytes=5234048 rssBytes=21180416 goroutines=267 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=1m0.06s requests=1917020 failures=0 heapBytes=3959208 rssBytes=21983232 goroutines=256 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=1m15.06s requests=2405587 failures=0 heapBytes=6696432 rssBytes=22294528 goroutines=355 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=1m30.06s requests=2893800 failures=0 heapBytes=5880824 rssBytes=21860352 goroutines=335 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=1m45.06s requests=3381213 failures=0 heapBytes=4176008 rssBytes=22257664 goroutines=331 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=2m0.06s requests=3867826 failures=0 heapBytes=6923296 rssBytes=22417408 goroutines=299 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=2m15.06s requests=4355136 failures=0 heapBytes=4863216 rssBytes=22257664 goroutines=331 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=2m30.06s requests=4844005 failures=0 heapBytes=4875168 rssBytes=22208512 goroutines=352 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=2m45.06s requests=5331034 failures=0 heapBytes=4053832 rssBytes=22085632 goroutines=354 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=3m0.061s requests=5824954 failures=0 heapBytes=7276088 rssBytes=22253568 goroutines=276 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=3m15.06s requests=6316211 failures=0 heapBytes=6082672 rssBytes=22413312 goroutines=340 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=3m30.06s requests=6795314 failures=0 heapBytes=6157488 rssBytes=22106112 goroutines=266 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=3m45.06s requests=7279828 failures=0 heapBytes=6538536 rssBytes=22454272 goroutines=296 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=4m0.06s requests=7763620 failures=0 heapBytes=5209696 rssBytes=22003712 goroutines=371 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=4m15.06s requests=8245520 failures=0 heapBytes=4119296 rssBytes=22532096 goroutines=325 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=4m30.06s requests=8728841 failures=0 heapBytes=6352704 rssBytes=22863872 goroutines=336 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=4m45.06s requests=9206490 failures=0 heapBytes=5696104 rssBytes=21614592 goroutines=227 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=5m0.06s requests=9692044 failures=0 heapBytes=6982152 rssBytes=22798336 goroutines=266 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=5m15.06s requests=10181610 failures=0 heapBytes=4842816 rssBytes=23273472 goroutines=233 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=5m30.06s requests=10674122 failures=0 heapBytes=4987832 rssBytes=21663744 goroutines=390 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=5m45.06s requests=11158495 failures=0 heapBytes=4861520 rssBytes=21594112 goroutines=255 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=6m0.06s requests=11643125 failures=0 heapBytes=6995288 rssBytes=22351872 goroutines=370 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=6m15.06s requests=12134189 failures=0 heapBytes=5791656 rssBytes=22269952 goroutines=322 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=6m30.06s requests=12617974 failures=0 heapBytes=4615832 rssBytes=22204416 goroutines=293 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=6m45.06s requests=13109107 failures=0 heapBytes=6241656 rssBytes=22806528 goroutines=331 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=7m0.06s requests=13597755 failures=0 heapBytes=4717608 rssBytes=22777856 goroutines=355 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=7m15.06s requests=14082999 failures=0 heapBytes=7078376 rssBytes=23724032 goroutines=352 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=7m30.06s requests=14573956 failures=0 heapBytes=4402216 rssBytes=21753856 goroutines=364 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=7m45.06s requests=15062222 failures=0 heapBytes=5805904 rssBytes=22765568 goroutines=366 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=8m0.06s requests=15550159 failures=0 heapBytes=6706008 rssBytes=22376448 goroutines=332 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=8m15.06s requests=16032745 failures=0 heapBytes=5471120 rssBytes=22126592 goroutines=318 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=8m30.061s requests=16516193 failures=0 heapBytes=7097144 rssBytes=22880256 goroutines=365 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=8m45.061s requests=16995862 failures=0 heapBytes=5984144 rssBytes=22609920 goroutines=288 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=9m0.06s requests=17480917 failures=0 heapBytes=5254728 rssBytes=22745088 goroutines=284 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=9m15.06s requests=17969665 failures=0 heapBytes=5609544 rssBytes=22839296 goroutines=264 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=9m30.06s requests=18456021 failures=0 heapBytes=4073608 rssBytes=22032384 goroutines=291 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=9m45.06s requests=18943865 failures=0 heapBytes=5703584 rssBytes=22515712 goroutines=250 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=10m0.06s requests=19427994 failures=0 heapBytes=4484904 rssBytes=23121920 goroutines=286 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=10m15.06s requests=19911036 failures=0 heapBytes=5654336 rssBytes=22589440 goroutines=330 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=10m30.06s requests=20398330 failures=0 heapBytes=7287736 rssBytes=22564864 goroutines=262 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=10m45.06s requests=20885304 failures=0 heapBytes=4866728 rssBytes=22773760 goroutines=340 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=11m0.06s requests=21377882 failures=0 heapBytes=4500144 rssBytes=22474752 goroutines=197 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=11m15.06s requests=21859381 failures=0 heapBytes=4837744 rssBytes=22278144 goroutines=346 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=11m30.06s requests=22340499 failures=0 heapBytes=4255808 rssBytes=22675456 goroutines=265 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=11m45.06s requests=22831603 failures=0 heapBytes=7122880 rssBytes=22138880 goroutines=322 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=12m0.06s requests=23322670 failures=0 heapBytes=4139848 rssBytes=22515712 goroutines=325 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=12m15.06s requests=23810489 failures=0 heapBytes=6155784 rssBytes=22671360 goroutines=281 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=12m30.06s requests=24298600 failures=0 heapBytes=6864792 rssBytes=22568960 goroutines=196 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=12m45.06s requests=24784321 failures=0 heapBytes=3708712 rssBytes=22278144 goroutines=295 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=13m0.06s requests=25270364 failures=0 heapBytes=5901032 rssBytes=22634496 goroutines=337 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=13m15.06s requests=25748374 failures=0 heapBytes=5453384 rssBytes=22368256 goroutines=313 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=13m30.06s requests=26231067 failures=0 heapBytes=7024176 rssBytes=22790144 goroutines=259 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=13m45.06s requests=26712235 failures=0 heapBytes=6159920 rssBytes=23277568 goroutines=332 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=14m0.06s requests=27192429 failures=0 heapBytes=6571248 rssBytes=22429696 goroutines=281 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=14m15.06s requests=27674241 failures=0 heapBytes=5498824 rssBytes=22700032 goroutines=307 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=14m30.06s requests=28161405 failures=0 heapBytes=7281776 rssBytes=22188032 goroutines=365 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=14m45.06s requests=28643479 failures=0 heapBytes=5348280 rssBytes=22425600 goroutines=257 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=15m0.06s requests=29126808 failures=0 heapBytes=7028080 rssBytes=23273472 goroutines=264 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=15m15.06s requests=29606183 failures=0 heapBytes=5075368 rssBytes=21872640 goroutines=363 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=15m30.06s requests=30083605 failures=0 heapBytes=5377456 rssBytes=22351872 goroutines=343 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=15m45.06s requests=30567760 failures=0 heapBytes=4157600 rssBytes=22609920 goroutines=266 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=16m0.06s requests=31051568 failures=0 heapBytes=4455056 rssBytes=22544384 goroutines=369 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=16m15.06s requests=31534567 failures=0 heapBytes=6660704 rssBytes=22876160 goroutines=271 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=16m30.06s requests=32010137 failures=0 heapBytes=6978528 rssBytes=22081536 goroutines=359 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=16m45.06s requests=32497005 failures=0 heapBytes=6214880 rssBytes=21352448 goroutines=252 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=17m0.06s requests=32977986 failures=0 heapBytes=4009360 rssBytes=21745664 goroutines=265 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=17m15.06s requests=33456000 failures=0 heapBytes=6263808 rssBytes=23236608 goroutines=344 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=17m30.06s requests=33943796 failures=0 heapBytes=4312032 rssBytes=22126592 goroutines=316 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=17m45.06s requests=34422999 failures=0 heapBytes=4311080 rssBytes=21950464 goroutines=237 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=18m0.06s requests=34905892 failures=0 heapBytes=4459192 rssBytes=22847488 goroutines=378 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=18m15.06s requests=35387063 failures=0 heapBytes=4965536 rssBytes=22679552 goroutines=272 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=18m30.06s requests=35872523 failures=0 heapBytes=7317240 rssBytes=22810624 goroutines=356 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=18m45.06s requests=36357886 failures=0 heapBytes=7180144 rssBytes=23207936 goroutines=379 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=19m0.06s requests=36847661 failures=0 heapBytes=6535800 rssBytes=22257664 goroutines=260 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=19m15.06s requests=37333781 failures=0 heapBytes=4758968 rssBytes=21053440 goroutines=261 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=19m30.06s requests=37819005 failures=0 heapBytes=5854656 rssBytes=22798336 goroutines=266 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=19m45.06s requests=38302079 failures=0 heapBytes=6477368 rssBytes=23130112 goroutines=231 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=20m0.06s requests=38781438 failures=0 heapBytes=4912296 rssBytes=22179840 goroutines=291 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=20m15.06s requests=39266527 failures=0 heapBytes=6723304 rssBytes=22417408 goroutines=267 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=20m30.06s requests=39747087 failures=0 heapBytes=4455048 rssBytes=22949888 goroutines=282 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=20m45.06s requests=40229194 failures=0 heapBytes=7650552 rssBytes=22667264 goroutines=314 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=21m0.06s requests=40719856 failures=0 heapBytes=5024096 rssBytes=21934080 goroutines=327 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=21m15.06s requests=41201734 failures=0 heapBytes=4340216 rssBytes=22867968 goroutines=372 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=21m30.06s requests=41687427 failures=0 heapBytes=6020200 rssBytes=22179840 goroutines=356 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=21m45.06s requests=42169416 failures=0 heapBytes=5102400 rssBytes=21901312 goroutines=284 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=22m0.06s requests=42649854 failures=0 heapBytes=5587144 rssBytes=23244800 goroutines=257 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=22m15.061s requests=43134689 failures=0 heapBytes=7256936 rssBytes=22966272 goroutines=247 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=22m30.06s requests=43615393 failures=0 heapBytes=7194832 rssBytes=23044096 goroutines=370 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=22m45.06s requests=44094551 failures=0 heapBytes=4162944 rssBytes=22044672 goroutines=298 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=23m0.06s requests=44581507 failures=0 heapBytes=5801200 rssBytes=21823488 goroutines=266 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=23m15.06s requests=45066149 failures=0 heapBytes=4212824 rssBytes=22265856 goroutines=359 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=23m30.06s requests=45549300 failures=0 heapBytes=4478232 rssBytes=21921792 goroutines=340 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=23m45.06s requests=46035904 failures=0 heapBytes=6875824 rssBytes=22740992 goroutines=249 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=24m0.06s requests=46519641 failures=0 heapBytes=6247888 rssBytes=21938176 goroutines=291 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=24m15.06s requests=46997264 failures=0 heapBytes=7072560 rssBytes=23109632 goroutines=377 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=24m30.06s requests=47482230 failures=0 heapBytes=5093112 rssBytes=23351296 goroutines=392 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=24m45.06s requests=47955814 failures=0 heapBytes=5715440 rssBytes=22499328 goroutines=350 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=25m0.06s requests=48434219 failures=0 heapBytes=5736984 rssBytes=22720512 goroutines=336 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=25m15.06s requests=48911328 failures=0 heapBytes=7415632 rssBytes=22880256 goroutines=256 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=25m30.06s requests=49400802 failures=0 heapBytes=3811240 rssBytes=21762048 goroutines=347 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=25m45.06s requests=49885035 failures=0 heapBytes=4371400 rssBytes=22343680 goroutines=374 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:3}
|
||||
soak_test.go:96: phase=running elapsed=26m0.06s requests=50365248 failures=0 heapBytes=4201696 rssBytes=23511040 goroutines=324 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=26m15.06s requests=50845851 failures=0 heapBytes=4646712 rssBytes=22171648 goroutines=274 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=26m30.06s requests=51332514 failures=0 heapBytes=4957128 rssBytes=22810624 goroutines=264 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
soak_test.go:96: phase=running elapsed=26m45.06s requests=51818521 failures=0 heapBytes=5534424 rssBytes=22429696 goroutines=361 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=27m0.06s requests=52303713 failures=0 heapBytes=5460112 rssBytes=22978560 goroutines=264 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=27m15.06s requests=52785633 failures=0 heapBytes=6919824 rssBytes=22196224 goroutines=293 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=27m30.06s requests=53270255 failures=0 heapBytes=6976928 rssBytes=22519808 goroutines=273 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=27m45.06s requests=53754584 failures=0 heapBytes=6265560 rssBytes=23269376 goroutines=300 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=28m0.06s requests=54243260 failures=0 heapBytes=6019184 rssBytes=22319104 goroutines=304 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=28m15.06s requests=54727233 failures=0 heapBytes=6646888 rssBytes=22794240 goroutines=289 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=28m30.06s requests=55211460 failures=0 heapBytes=7514304 rssBytes=22654976 goroutines=345 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=28m45.06s requests=55691054 failures=0 heapBytes=4597656 rssBytes=21585920 goroutines=337 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:26}
|
||||
soak_test.go:96: phase=running elapsed=29m0.06s requests=56175794 failures=0 heapBytes=5366888 rssBytes=22618112 goroutines=240 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=29m15.06s requests=56657414 failures=0 heapBytes=5976448 rssBytes=22413312 goroutines=393 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=29m30.06s requests=57140848 failures=0 heapBytes=4945864 rssBytes=23572480 goroutines=276 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:2}
|
||||
soak_test.go:96: phase=running elapsed=29m45.06s requests=57624110 failures=0 heapBytes=4265224 rssBytes=22007808 goroutines=164 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:1}
|
||||
soak_test.go:96: phase=running elapsed=30m0.06s requests=58102348 failures=0 heapBytes=5912816 rssBytes=21807104 goroutines=333 fileDescriptors=9 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:13}
|
||||
soak_test.go:166: phase=after elapsed=30m0.067s requests=58102476 failures=0 heapBytes=4154504 rssBytes=21807104 goroutines=3 fileDescriptors=6 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSSoak (1800.07s)
|
||||
PASS
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 1800.073s
|
||||
PASS HTTP/2 TLS soak
|
||||
265
performance/linuxSoak.json
Normal file
265
performance/linuxSoak.json
Normal file
@@ -0,0 +1,265 @@
|
||||
{
|
||||
"createdAt": "2026-09-21T17:29:24Z",
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/arm64",
|
||||
"gomaxprocs": 4,
|
||||
"duration": "30s",
|
||||
"tls": false,
|
||||
"notes": [
|
||||
"Loopback HTTP/1.1 health response, fixed requestId, optional TLS, no access logs or database.",
|
||||
"Each case starts a fresh server process; load client runs in the parent process on the same machine.",
|
||||
"Closed-loop concurrency; throughput is successful responses / actual elapsed time, including drain.",
|
||||
"Latency percentiles include failures and are histogram upper bounds with \u003c=6.25% rounding.",
|
||||
"Memory is server-only runtime.MemStats (not RSS or kernel socket memory); sampling interval 200ms.",
|
||||
"FileDescriptors is read from /proc/self/fd on Linux and omitted on other platforms.",
|
||||
"Idle memory measured after a forced GC; delta includes heap growth amortized across connections, excluding stacks.",
|
||||
"After is after closing clients and one forced GC; afterReclaim is a second GC to observe pooled-cache release.",
|
||||
"Native limits: 8192 connections / 4096 active requests. These cases do not benchmark overload rejection."
|
||||
],
|
||||
"results": [
|
||||
{
|
||||
"adapter": "nethttp",
|
||||
"scenario": "traffic",
|
||||
"concurrency": 128,
|
||||
"run": 1,
|
||||
"requests": 4456189,
|
||||
"errors": 0,
|
||||
"elapsedSeconds": 30.001393348,
|
||||
"requestsPerSecond": 148532.7347403705,
|
||||
"p50Millis": 0.589824,
|
||||
"p95Millis": 2.62144,
|
||||
"p99Millis": 3.538944,
|
||||
"serverBytesPerRequest": 4427.911252417705,
|
||||
"serverAllocsPerRequest": 47.07614668049313,
|
||||
"sampledPeakHeapAlloc": 6219064,
|
||||
"sampledPeakFileDescriptors": 135,
|
||||
"before": {
|
||||
"heapAlloc": 2105600,
|
||||
"heapInuse": 4202496,
|
||||
"stackInuse": 1015808,
|
||||
"totalAlloc": 4172144,
|
||||
"mallocs": 30855,
|
||||
"numGc": 2,
|
||||
"goroutines": 58,
|
||||
"connections": 56,
|
||||
"accepted": 56,
|
||||
"fileDescriptors": 63
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 3500920,
|
||||
"heapInuse": 5947392,
|
||||
"stackInuse": 2097152,
|
||||
"totalAlloc": 19735781560,
|
||||
"mallocs": 209811062,
|
||||
"numGc": 6570,
|
||||
"goroutines": 130,
|
||||
"connections": 128,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 2828640,
|
||||
"heapInuse": 5054464,
|
||||
"stackInuse": 1015808,
|
||||
"totalAlloc": 19735811920,
|
||||
"mallocs": 209811396,
|
||||
"numGc": 6571,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1722584,
|
||||
"heapInuse": 3530752,
|
||||
"stackInuse": 1015808,
|
||||
"totalAlloc": 19735822392,
|
||||
"mallocs": 209811421,
|
||||
"numGc": 6572,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "native",
|
||||
"scenario": "traffic",
|
||||
"concurrency": 128,
|
||||
"run": 1,
|
||||
"requests": 4398474,
|
||||
"errors": 0,
|
||||
"elapsedSeconds": 30.000375472,
|
||||
"requestsPerSecond": 146613.96501871088,
|
||||
"p50Millis": 0.622592,
|
||||
"p95Millis": 2.62144,
|
||||
"p99Millis": 3.670016,
|
||||
"serverBytesPerRequest": 4591.297385411395,
|
||||
"serverAllocsPerRequest": 59.0589788640333,
|
||||
"sampledPeakHeapAlloc": 4980920,
|
||||
"sampledPeakFileDescriptors": 135,
|
||||
"before": {
|
||||
"heapAlloc": 2202712,
|
||||
"heapInuse": 4120576,
|
||||
"stackInuse": 1671168,
|
||||
"totalAlloc": 4300920,
|
||||
"mallocs": 32647,
|
||||
"numGc": 2,
|
||||
"goroutines": 131,
|
||||
"connections": 128,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 4382752,
|
||||
"heapInuse": 6021120,
|
||||
"stackInuse": 1638400,
|
||||
"totalAlloc": 20199003096,
|
||||
"mallocs": 259802030,
|
||||
"numGc": 9032,
|
||||
"goroutines": 131,
|
||||
"connections": 128,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 2260152,
|
||||
"heapInuse": 4382720,
|
||||
"stackInuse": 557056,
|
||||
"totalAlloc": 20199025048,
|
||||
"mallocs": 259802362,
|
||||
"numGc": 9033,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1645880,
|
||||
"heapInuse": 3489792,
|
||||
"stackInuse": 557056,
|
||||
"totalAlloc": 20199035520,
|
||||
"mallocs": 259802387,
|
||||
"numGc": 9034,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "nethttp",
|
||||
"scenario": "idle",
|
||||
"concurrency": 512,
|
||||
"run": 1,
|
||||
"errors": 0,
|
||||
"idleHeapDelta": 5162344,
|
||||
"idleHeapPerConnection": 10082.703125,
|
||||
"before": {
|
||||
"heapAlloc": 1516408,
|
||||
"heapInuse": 3301376,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 3006544,
|
||||
"mallocs": 23238,
|
||||
"numGc": 1,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 0,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 6678752,
|
||||
"heapInuse": 8675328,
|
||||
"stackInuse": 4718592,
|
||||
"totalAlloc": 10467200,
|
||||
"mallocs": 59319,
|
||||
"numGc": 4,
|
||||
"goroutines": 514,
|
||||
"connections": 512,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 519
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 6108264,
|
||||
"heapInuse": 8069120,
|
||||
"stackInuse": 491520,
|
||||
"totalAlloc": 10563168,
|
||||
"mallocs": 60468,
|
||||
"numGc": 5,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1806072,
|
||||
"heapInuse": 3629056,
|
||||
"stackInuse": 491520,
|
||||
"totalAlloc": 10573944,
|
||||
"mallocs": 60497,
|
||||
"numGc": 6,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "native",
|
||||
"scenario": "idle",
|
||||
"concurrency": 512,
|
||||
"run": 1,
|
||||
"errors": 0,
|
||||
"idleHeapDelta": 2692536,
|
||||
"idleHeapPerConnection": 5258.859375,
|
||||
"before": {
|
||||
"heapAlloc": 1511104,
|
||||
"heapInuse": 3309568,
|
||||
"stackInuse": 425984,
|
||||
"totalAlloc": 3001496,
|
||||
"mallocs": 23227,
|
||||
"numGc": 1,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 0,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 4203640,
|
||||
"heapInuse": 6283264,
|
||||
"stackInuse": 4718592,
|
||||
"totalAlloc": 8087336,
|
||||
"mallocs": 58784,
|
||||
"numGc": 4,
|
||||
"goroutines": 515,
|
||||
"connections": 512,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 519
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 4233984,
|
||||
"heapInuse": 6316032,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 8182368,
|
||||
"mallocs": 60013,
|
||||
"numGc": 5,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1804784,
|
||||
"heapInuse": 3686400,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 8193144,
|
||||
"mallocs": 60042,
|
||||
"numGc": 6,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
265
performance/linuxSoakTLS.json
Normal file
265
performance/linuxSoakTLS.json
Normal file
@@ -0,0 +1,265 @@
|
||||
{
|
||||
"createdAt": "2026-09-21T17:27:47Z",
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/arm64",
|
||||
"gomaxprocs": 4,
|
||||
"duration": "30s",
|
||||
"tls": true,
|
||||
"notes": [
|
||||
"Loopback HTTP/1.1 health response, fixed requestId, optional TLS, no access logs or database.",
|
||||
"Each case starts a fresh server process; load client runs in the parent process on the same machine.",
|
||||
"Closed-loop concurrency; throughput is successful responses / actual elapsed time, including drain.",
|
||||
"Latency percentiles include failures and are histogram upper bounds with \u003c=6.25% rounding.",
|
||||
"Memory is server-only runtime.MemStats (not RSS or kernel socket memory); sampling interval 200ms.",
|
||||
"FileDescriptors is read from /proc/self/fd on Linux and omitted on other platforms.",
|
||||
"Idle memory measured after a forced GC; delta includes heap growth amortized across connections, excluding stacks.",
|
||||
"After is after closing clients and one forced GC; afterReclaim is a second GC to observe pooled-cache release.",
|
||||
"Native limits: 8192 connections / 4096 active requests. These cases do not benchmark overload rejection."
|
||||
],
|
||||
"results": [
|
||||
{
|
||||
"adapter": "nethttp",
|
||||
"scenario": "traffic",
|
||||
"concurrency": 128,
|
||||
"run": 1,
|
||||
"requests": 4442303,
|
||||
"errors": 0,
|
||||
"elapsedSeconds": 30.001232972,
|
||||
"requestsPerSecond": 148070.68109987277,
|
||||
"p50Millis": 0.589824,
|
||||
"p95Millis": 2.62144,
|
||||
"p99Millis": 3.670016,
|
||||
"serverBytesPerRequest": 4475.325703807237,
|
||||
"serverAllocsPerRequest": 49.07248492504901,
|
||||
"sampledPeakHeapAlloc": 7609472,
|
||||
"sampledPeakFileDescriptors": 135,
|
||||
"before": {
|
||||
"heapAlloc": 3340488,
|
||||
"heapInuse": 5988352,
|
||||
"stackInuse": 2195456,
|
||||
"totalAlloc": 9430136,
|
||||
"mallocs": 83639,
|
||||
"numGc": 5,
|
||||
"goroutines": 124,
|
||||
"connections": 122,
|
||||
"accepted": 122,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 6485192,
|
||||
"heapInuse": 8716288,
|
||||
"stackInuse": 1998848,
|
||||
"totalAlloc": 19890182936,
|
||||
"mallocs": 218078486,
|
||||
"numGc": 5603,
|
||||
"goroutines": 130,
|
||||
"connections": 128,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 2853976,
|
||||
"heapInuse": 5251072,
|
||||
"stackInuse": 884736,
|
||||
"totalAlloc": 19890213312,
|
||||
"mallocs": 218078820,
|
||||
"numGc": 5604,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1745496,
|
||||
"heapInuse": 3686400,
|
||||
"stackInuse": 884736,
|
||||
"totalAlloc": 19890215312,
|
||||
"mallocs": 218078842,
|
||||
"numGc": 5605,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "native",
|
||||
"scenario": "traffic",
|
||||
"concurrency": 128,
|
||||
"run": 1,
|
||||
"requests": 4491731,
|
||||
"errors": 0,
|
||||
"elapsedSeconds": 30.000919805,
|
||||
"requestsPerSecond": 149719.77623337405,
|
||||
"p50Millis": 0.622592,
|
||||
"p95Millis": 2.62144,
|
||||
"p99Millis": 3.538944,
|
||||
"serverBytesPerRequest": 4807.806184297323,
|
||||
"serverAllocsPerRequest": 61.06011045630293,
|
||||
"sampledPeakHeapAlloc": 5859960,
|
||||
"sampledPeakFileDescriptors": 135,
|
||||
"before": {
|
||||
"heapAlloc": 2685024,
|
||||
"heapInuse": 5136384,
|
||||
"stackInuse": 2621440,
|
||||
"totalAlloc": 8854080,
|
||||
"mallocs": 83570,
|
||||
"numGc": 5,
|
||||
"goroutines": 125,
|
||||
"connections": 122,
|
||||
"accepted": 122,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 3345480,
|
||||
"heapInuse": 5980160,
|
||||
"stackInuse": 1605632,
|
||||
"totalAlloc": 21604226160,
|
||||
"mallocs": 274349161,
|
||||
"numGc": 7802,
|
||||
"goroutines": 131,
|
||||
"connections": 128,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 135
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 2764288,
|
||||
"heapInuse": 5341184,
|
||||
"stackInuse": 1605632,
|
||||
"totalAlloc": 21604248208,
|
||||
"mallocs": 274349494,
|
||||
"numGc": 7803,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1654056,
|
||||
"heapInuse": 3506176,
|
||||
"stackInuse": 1605632,
|
||||
"totalAlloc": 21604258680,
|
||||
"mallocs": 274349519,
|
||||
"numGc": 7804,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 128,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "nethttp",
|
||||
"scenario": "idle",
|
||||
"concurrency": 512,
|
||||
"run": 1,
|
||||
"errors": 0,
|
||||
"idleHeapDelta": 7274360,
|
||||
"idleHeapPerConnection": 14207.734375,
|
||||
"before": {
|
||||
"heapAlloc": 1515704,
|
||||
"heapInuse": 3301376,
|
||||
"stackInuse": 425984,
|
||||
"totalAlloc": 3023896,
|
||||
"mallocs": 23663,
|
||||
"numGc": 1,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 0,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 8790064,
|
||||
"heapInuse": 12066816,
|
||||
"stackInuse": 4718592,
|
||||
"totalAlloc": 29570320,
|
||||
"mallocs": 272972,
|
||||
"numGc": 9,
|
||||
"goroutines": 514,
|
||||
"connections": 512,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 519
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 6138608,
|
||||
"heapInuse": 8765440,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 29675376,
|
||||
"mallocs": 274141,
|
||||
"numGc": 10,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1820824,
|
||||
"heapInuse": 3825664,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 29677376,
|
||||
"mallocs": 274163,
|
||||
"numGc": 11,
|
||||
"goroutines": 2,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
},
|
||||
{
|
||||
"adapter": "native",
|
||||
"scenario": "idle",
|
||||
"concurrency": 512,
|
||||
"run": 1,
|
||||
"errors": 0,
|
||||
"idleHeapDelta": 4689576,
|
||||
"idleHeapPerConnection": 9159.328125,
|
||||
"before": {
|
||||
"heapAlloc": 1520864,
|
||||
"heapInuse": 3284992,
|
||||
"stackInuse": 458752,
|
||||
"totalAlloc": 3029160,
|
||||
"mallocs": 23661,
|
||||
"numGc": 1,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 0,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"during": {
|
||||
"heapAlloc": 6210440,
|
||||
"heapInuse": 9314304,
|
||||
"stackInuse": 8880128,
|
||||
"totalAlloc": 27150344,
|
||||
"mallocs": 271905,
|
||||
"numGc": 10,
|
||||
"goroutines": 515,
|
||||
"connections": 512,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 519
|
||||
},
|
||||
"after": {
|
||||
"heapAlloc": 6230688,
|
||||
"heapInuse": 9330688,
|
||||
"stackInuse": 491520,
|
||||
"totalAlloc": 27245184,
|
||||
"mallocs": 273078,
|
||||
"numGc": 11,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
},
|
||||
"afterReclaim": {
|
||||
"heapAlloc": 1817296,
|
||||
"heapInuse": 3702784,
|
||||
"stackInuse": 491520,
|
||||
"totalAlloc": 27255960,
|
||||
"mallocs": 273107,
|
||||
"numGc": 12,
|
||||
"goroutines": 3,
|
||||
"connections": 0,
|
||||
"accepted": 512,
|
||||
"fileDescriptors": 7
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user