Compare commits
68 Commits
v0.3.1
...
04a1f7999d
| Author | SHA1 | Date | |
|---|---|---|---|
| 04a1f7999d | |||
| 208a9c3fbd | |||
| f91034477b | |||
| b9259e628e | |||
| fe66221a9c | |||
| 467c890363 | |||
| be620d2243 | |||
| 8e18528fc2 | |||
| 59bf04a461 | |||
| b54a303ccd | |||
| 84670f7be1 | |||
| fa2be5e016 | |||
| dfe8a1a9af | |||
| 830cb3a7ad | |||
| d94df4fc54 | |||
| 1a534c7976 | |||
| 2b93545507 | |||
| 15825f0ab8 | |||
| 1b9215114f | |||
| 73a05b8c3c | |||
| 64a9fde204 | |||
| ca0e1d6b92 | |||
| 98f17660c8 | |||
| c8660026ce | |||
| 695bdb7a00 | |||
| 7e9e8aa67c | |||
| de33eda846 | |||
| dd7d3fef0a | |||
| 46c5c9406c | |||
| 1487a96f67 | |||
| 1a20ab53ac | |||
| a072da4a31 | |||
| fcd6f48b53 | |||
| 83633f724e | |||
| a523e6ba3f | |||
| 4a8feaca36 | |||
| 95dc4568f6 | |||
| 2259606f72 | |||
| 35b7470be2 | |||
| 4f97aa0f3d | |||
| 3b4d0a3fad | |||
| a4d843b8f4 | |||
| fadd17a4d4 | |||
| ac46071dfe | |||
| 52d603c31e | |||
| 42021538b2 | |||
| ca707761fc | |||
| 30fdad44c2 | |||
| b7dc36fc99 | |||
| 8e474a24d9 | |||
| ef98d1dd8d | |||
| 5ab906098f | |||
| c5e415ca52 | |||
| 9c9c8ab59a | |||
| e6c82a01fe | |||
| 847621d218 | |||
| 4daf3b211c | |||
| 62b0f09546 | |||
| 694374fde8 | |||
| 22f1c9ebfb | |||
| b3cb9e8857 | |||
| efd3163902 | |||
| 241a568c83 | |||
| 22939fdb0b | |||
| 3d5f225fb2 | |||
| c36bb70f96 | |||
| 05b4088a25 | |||
| 3bbf4627a1 |
149
ADVANCED.md
Normal file
149
ADVANCED.md
Normal file
@@ -0,0 +1,149 @@
|
||||
# 扩展业务模块
|
||||
|
||||
本批提供八类扩展,以及分页、审计和显式数据库迁移。模块按需导入;`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase;Go 导出标识符和 `_test.go` 保持语言约定。
|
||||
|
||||
按本次范围决定,**不集成 Orbit,不引入 Orbit SDK,也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit,可独立依赖它的 SDK;当前没有承诺只换队列适配器就能迁移其全部语义。
|
||||
|
||||
## 能力与边界
|
||||
|
||||
| 模块 | 本批实现 | 明确边界 |
|
||||
| --- | --- | --- |
|
||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||
| `storage` | Local、S3 兼容(含 R2 配置)、OSS 官方 SDK 适配、临时下载链接 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||
|
||||
## 对象存储
|
||||
|
||||
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||
|
||||
```go
|
||||
type FileService struct { Files storage.Store }
|
||||
|
||||
// Local:私有目录,流式临时文件 + 原子替换。
|
||||
files, err := local.New("/srv/app/objects", 32<<20)
|
||||
|
||||
// S3:AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
|
||||
files, err := s3.Open(s3.Config{
|
||||
Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
|
||||
Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
|
||||
Credentials: credentialProvider, Transport: transport,
|
||||
})
|
||||
|
||||
// Cloudflare R2:复用 S3 适配器,签名区域为 auto。
|
||||
cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
|
||||
cfg.Transport = transport
|
||||
files, err := s3.Open(cfg)
|
||||
|
||||
// OSS:单独适配官方 Go SDK v2,保留平台签名与凭据提供器。
|
||||
client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
|
||||
WithRegion("cn-hangzhou").
|
||||
WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
|
||||
files, err := oss.New(client, bucket, 32<<20)
|
||||
```
|
||||
|
||||
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||
|
||||
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,调用方保证与内容一致。默认单对象 32 MiB,流式读写;OSS 本适配器只做 simple upload,最大配置 5 GiB。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||
|
||||
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。首版未封装前端直传、分片续传、生命周期、ACL、CDN 自定义域名和版本管理。
|
||||
|
||||
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||
|
||||
验证区分:Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试;Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限,不只是切配置。
|
||||
|
||||
## 可靠队列
|
||||
|
||||
```go
|
||||
backend, err := queueRedis.New(redisClient, queueRedis.Config{
|
||||
Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
|
||||
MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
|
||||
})
|
||||
consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
|
||||
Workers: 4, MaxAttempts: 3,
|
||||
TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
|
||||
})
|
||||
|
||||
// ID 与内容在业务端持久化;超时后复用原 ID 和原内容。
|
||||
receipt, err := backend.Publish(ctx, queue.Message{
|
||||
ID: operationID, Name: "generateReport", Payload: payload,
|
||||
})
|
||||
|
||||
// 在服务生命周期中运行;返回错误须让监控/进程管理感知。
|
||||
err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
|
||||
return generateReportIdempotently(ctx, item.ID, item.Payload)
|
||||
})
|
||||
```
|
||||
|
||||
业务依赖 `queue.Publisher` / `queue.Consumer`,Redis 类型留在组合处。receipt 表示受理,不代表业务成功。普通 error/panic 按配置重试,`queue.Reject(code)` 保留为失败消息;超过次数同样保留。`DeadLetters(after, limit)` 是有界分页,`DiscardDead` 仅在人工核对后删除匹配的失败消息,并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略,框架不自动无限重投。
|
||||
|
||||
一个 namespace 使用一个受管 consumer group;多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner,旧执行不能确认新执行;但不能阻止旧处理器已发出的外部副作用,因此仍必须业务幂等。处理器需响应 context;Go 无法强杀不配合的函数,超过 shutdownTimeout 返回错误时,最多 Workers 个处理器可能尚未退出。
|
||||
|
||||
容量包含待处理、失败及未过期的已完成身份,满额拒绝新 ID,保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期;未完成/失败消息不自动清除,不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction;本批验证的是投递/接管协议,未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理,不能用 DELCONSUMER 丢掉尚未接管的 pending。
|
||||
|
||||
`task.Runner` 仍适用于本进程短任务和定时轮询,重启会丢失;需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
|
||||
|
||||
## 认证与授权
|
||||
|
||||
`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表;只开放 RS256 / EdDSA,必须有 exp/sub,检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表,轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
|
||||
|
||||
`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token;存储仅使用 token 的 SHA-256 键,可 `Revoke`。Redis 缓存可共享 Session,内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
|
||||
|
||||
`RequireAuth(authenticator)` 在前,`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal,业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
|
||||
|
||||
## 配置与日志热更新
|
||||
|
||||
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase,duration 可以写 `"5s"`;文件上限 1 MiB,拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线,例如 `BILLING_`。空环境变量沿用文件值,不清空已配置字段。
|
||||
|
||||
```yaml
|
||||
address: '127.0.0.1:8080'
|
||||
transport: native
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
level: info
|
||||
format: json
|
||||
```
|
||||
|
||||
先按配置创建 `logx.New(cfg.Log)`,再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`;后者阻塞,生命周期由应用负责。只有 `log.level` 允许变化,校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
|
||||
## 熔断与观测
|
||||
|
||||
为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`;通用调用用 `Do`,HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取,必须关闭 body;网络错误/5xx 计失败,冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
|
||||
|
||||
`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`;可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter,基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站,`telemetry.Transport` 采集出站并传播 W3C trace context;Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`,SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
|
||||
|
||||
创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类;自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定,避免用用户 ID 或对象 key。
|
||||
|
||||
## 分布式限流与幂等
|
||||
|
||||
`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶;满额/存储失败返回 503,额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`;Scope 必须来自认证后的租户/用户,先认证再幂等。不同请求内容复用同一身份返回 409,已完成请求回放原响应和 requestId。
|
||||
|
||||
claim 在开始业务前持久保存,完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期;返回 409 等待核查,`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理,TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace,数据丢失时不存在框架层面的重复执行保证。
|
||||
|
||||
## 迁移、审计与分页
|
||||
|
||||
迁移命名:`0001_createUsers.up.sql`,可附 `.down.sql`,但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单,不连接数据库;部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁;MySQL DDL 可能部分提交,失败保留 dirty,禁止自动 force 或回滚。当前基于 golang-migrate,数据库版本表不保存每个历史文件校验和,部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止,重试需新建 Runner。
|
||||
|
||||
审计:`audit.New(sink, "approvedField")` 建白名单,再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`;补入服务端时间、requestId/traceId,只保留明确允许的属性,Sink 失败原样返回。JSONSink 不保证与业务提交原子一致,需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
|
||||
|
||||
分页:`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20,上限 100,最大 offset=1,000,000;拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages,空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器,各限 2 CPU、2 GiB、256 pids;数据阶段无外网、无发布端口,不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准,未提供云凭据的 OSS/R2 不标为实网通过。
|
||||
|
||||
本批已通过上述 Linux 验证,源代码提交、98 个文件摘要、服务版本和边界见 [验收记录](performance/advancedAcceptance.md)。
|
||||
56
CHANGELOG.md
56
CHANGELOG.md
@@ -1,5 +1,61 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||
- 新增 `TestTLSMixedSoak` 与 `verifyHTTP2.py --scenario mixed`:8 条 TLS 连接、128 个 worker 持续混跑普通请求、慢上传和慢下载,校验 TLS/HTTP2、requestId、响应内容及传输摘要;分别统计三类请求与每条连接覆盖情况,并记录退出/GC 后的回收数据。
|
||||
- 完成 Linux amd64 30 分钟多连接慢客户端混合长测:53,191,687 次请求(含 106,971 次慢上传、5,588 次慢下载),0 失败与协议错误;采样 RSS 峰值约 116.6 MiB,连接/请求、goroutine 和 FD 回收,释放测试数据后存活堆约 450 KiB。保留完整日志、120 份采样及每条连接的分类计数。
|
||||
- 新增独立 `transport/http2` 实验 Runner:支持 h2c/TLS h2、连接上限、全局请求并发闸门、GOAWAY 排空、RST_STREAM 单流取消和生命周期统计;不改变 native HTTP/1.1 默认路径。
|
||||
- HTTP/2 实验 Runner 增加标准 1 MiB 头部上限、默认 8 MiB 请求体上限、128 路同连接多路复用、2 MiB 慢读取流控验收、协议错误、RST_STREAM 和 GOAWAY 帧计数;已声明超限请求在进入业务 handler 前返回 413。
|
||||
- 新增可配置时长的 HTTP/2 TLS soak:默认 3 秒,Linux 验收可通过 `GO_OMNI_HTTP2_SOAK_DURATION=30m` 覆盖,混合验证 128 路 stream、慢 handler、请求体、Keep-Alive 和退出回收。
|
||||
- HTTP/2 soak 每 15 秒输出请求/失败数、Go 堆、Linux RSS、goroutine、FD 和协议统计;校验响应协议、内容和独立请求 ID,不再忽略压测期限附近已发出请求的失败。
|
||||
- 补充 HTTP/2 慢响应流窗口为零时的隔离与恢复验收:同连接 204、另一连接带 body 响应、WINDOW_UPDATE 恢复发送及 RST_STREAM 释放请求槽位;补充 TLS 握手未完成时的连接上限拒绝、握手超时回收和正常请求恢复。
|
||||
- 补充连接级接收窗口耗尽测试:保持 stream 信用充足,消费完 65,535 字节连接窗口后确认大响应暂停,同连接 204 和另一条连接仍可完成,连接级 WINDOW_UPDATE 恢复后继续复用;补充零窗口阻塞写在关闭超时后解除阻塞、取消请求与连接回收的验收。
|
||||
- 新增 `scripts/verifyHTTP2.py`,固定在 Go 1.25 Linux 容器中运行 HTTP/2 TLS 长测;默认使用普通模式,`--race` 默认运行 60 秒。容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,带总时限、中断清理和日志大小限制。
|
||||
- 完成 Linux amd64 单 TLS 连接、128 个 worker 的 30 分钟 HTTP/2 长测:58,102,476 次请求,0 失败与协议错误,采样 RSS 峰值约 22.6 MiB(包含客户端和服务端),退出后连接/请求/FD 回收;保留原始日志和 120 份采样。
|
||||
- native TLS 明确限制 ALPN 为 `http/1.1`:缺省自动宣告该协议,误配置 `h2` 等未实现协议时在启动阶段报错。
|
||||
- native `Server.Stats()` 新增连接和请求过载拒绝计数;新增可信代理配置,访问日志只在 socket 对端属于配置网段时解析 `X-Forwarded-For`。
|
||||
- 新增 transport-neutral `metrics` 包,支持请求数、错误数、in-flight、累计耗时和状态分布,并可直接注册 `/metrics` 接口。
|
||||
- native 新增 `Server.Stats()`,提供 accepted connections、active connections 和 active requests 生命周期指标。
|
||||
- native 请求体改为流式读取,Content-Length 与 chunked body 不再在进入 Handler 前整体复制到内存;保留大小限制、读超时、trailer 校验和 Keep-Alive 排空。
|
||||
- 统一标准库与 native 的真实 TCP benchmark,增加固定并发档位、响应状态与读写错误检查。
|
||||
- 新增独立进程并发与长连接内存测量工具,记录成功吞吐、P95/P99、服务端分配、空闲连接堆内存和回收情况;修正并行 `ns/op` 与客户端/服务端混合分配的解释。
|
||||
- 加固 native HTTP/1.1 解析边界:校验 Host、重复 Content-Length、Connection token、Transfer-Encoding、控制字符和响应头名称;补充 HTTP/1.0 Keep-Alive、HEAD、204 以及异常请求测试。
|
||||
- native 支持带 trailers 的 chunked 请求体,并继续通过 `MaxBodyBytes`、读超时和头部上限控制解析内存与连接占用。
|
||||
- native 响应实现 `http.Flusher`:HTTP/1.1 流式响应自动使用 chunked framing,普通响应继续自动生成 `Content-Length`。
|
||||
- native 在长 Handler 运行期间监测客户端断开并取消请求 context,流式写失败同步取消;短请求延迟启动监测,管线数据保留给下一次请求。补充断开、监测退出和长连接复用验收。
|
||||
- native 增加 `ServeTLS` / `RunTLS` 和 `TLSConfig` 启动入口,App.RunWith 可以直接启用 TLS;补充真实 TLS 客户端握手、请求和优雅退出验收。
|
||||
- native 请求补齐 `Request.TLS` 连接状态;Linux/arm64 容器通过全量 race、vet、build,以及 TLS 并发、Keep-Alive、慢握手和连接上限回收验收。
|
||||
- 补充慢请求体超时、大请求体和同连接管线请求验收,确认流式 body 读取超时会取消请求并关闭连接,连续请求不会串线。
|
||||
- 补充 `MaxConnections` 满载拒绝与槽位释放验收,确认请求完成后连接容量可再次使用。
|
||||
- 补充 `MaxConcurrentRequests` 过载恢复验收,确认 503 拒绝后请求槽位释放并可继续处理新请求。
|
||||
- 增加高频建连/断连的状态池 churn 验收,覆盖连接对象回收与容量复用。
|
||||
- 新增 Linux Nginx 反向代理验收脚本,覆盖 native TLS、Keep-Alive、上游超时和优雅关闭。
|
||||
|
||||
## v0.6.0-preview
|
||||
|
||||
- native 网络层完成流式 Content-Length / chunked body、慢请求头、慢 body、慢 TLS 握手和 TLS 请求元数据验收;Handler 可在 body 未完全到达时开始读取,超时、畸形和超限 body 会被拒绝并关闭连接。
|
||||
- 将 native 连接 churn race 验收扩展为 16 个 worker、3,200 次建连/断连,并覆盖明文与 TLS 的连接上限、Keep-Alive 和槽位回收。
|
||||
- perfcheck 支持最长 30 分钟持续场景,新增 Linux 文件描述符采样;固定记录服务端堆、goroutine、连接数、accept 数和 FD 回收状态。
|
||||
- 新增 `scripts/verifyLinux.py`,在 `golang:1.25` Linux 容器内统一执行 race、vet、build 和可选 TLS / 非 TLS 压测。
|
||||
- 完成 Linux/arm64 30 秒高并发回归:128 并发流量、512 条空闲连接,TLS 与非 TLS 均为 0 错误,连接、goroutine 和 FD 回收正常;原始报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`。
|
||||
|
||||
## v0.5.0
|
||||
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端,接管 TCP 监听、连接、Keep-Alive、请求解析、响应写回和优雅关闭。
|
||||
- 新增 `App.RunWith` / `omni.Runner` 启动边界;native 通过标准 `http.Handler` 接入,现有 Handler、Service 和 Repository 无需修改。
|
||||
- native 支持 `Content-Length` / chunked 请求体、请求头 / 请求体限制、读写超时和服务关闭时的请求取消;通过对象池和 `MaxConnections` / `MaxConcurrentRequests` 控制高并发内存。
|
||||
- `omni.New()` 默认仍使用只依赖标准库的 `transport/nethttp` 路由,native 作为显式启用的网络层预览实现。
|
||||
|
||||
## v0.4.0
|
||||
|
||||
- **破坏性变更**:移除 Gin 适配器及其全部依赖,`omni.New()` 默认使用只依赖标准库的 `transport/nethttp`。
|
||||
- 移除 Gin 专属的 `Config.Mode` / `WithMode` API;网络层运行模式由具体适配器或启动层自行配置。
|
||||
- 公共 `httpx.Router`、`transport/transporttest.Run` 和 `omni.NewWithRouter` 保留,自研网络层无需修改业务 Handler / Service 即可接入。
|
||||
- benchmark 改为标准库路由和完整 GoOmni 请求链,新增 `scripts/verifyPerformance.py` 作为 race、vet、build 和性能基线入口。
|
||||
- 远程业务试用、脚手架、适配器契约和 MySQL 验收全部迁移到 `v0.4.0`。
|
||||
|
||||
## v0.3.1
|
||||
|
||||
- 新增独立业务模块验收 `scripts/verifyBusiness.py`:从远程 `v0.2.0` 升级到 `v0.3.1`,验证无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
|
||||
172
MODULES.md
Normal file
172
MODULES.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# 业务基础模块
|
||||
|
||||
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||
|
||||
这批能力位于 main,尚未包含在 `v0.6.0-preview` 标签中。统一示例见
|
||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||
|
||||
## 缓存:cache 与 cache/redis
|
||||
|
||||
```go
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 4096, MaxBytes: 32 << 20})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 128, Timeout: 5 * time.Second})
|
||||
if err != nil { return err }
|
||||
value, err := loader.GetOrLoad(ctx, "tenant:42:product:7", time.Minute,
|
||||
func(ctx context.Context) ([]byte, error) { return repository.LoadProduct(ctx, "42", "7") })
|
||||
```
|
||||
|
||||
- `Store` 统一 `Get/Set/Delete`,返回字节切片;业务可用 JSON 编解码。
|
||||
- 内存缓存使用 LRU,同时限制条目数和 key/value 总字节数;字节上限不包含 Go 对象开销。
|
||||
读写复制切片,过期时懒清理,没有每个条目一个 goroutine。
|
||||
- TTL 为零表示不过期,负值拒绝;缺失返回 `cache.ErrMiss`。删除不存在的 key 成功。
|
||||
- Loader 合并**同进程、同 key**的并发回源,最多同时运行 `MaxLoads` 个不同 key 的加载。
|
||||
单个调用者取消只停止等待;共享加载有独立超时,加载函数必须响应 context。
|
||||
超限返回 `ErrBusy`,缓存故障直接返回错误,不自动当成 miss 冲击数据库。
|
||||
- 同 key 的调用必须具有相同数据含义、权限范围和 TTL;加载采用首个调用者的 context 值。
|
||||
多租户 key 必须含租户标识。进程间合并、分布式锁、缓存更新与数据库事务一致性由业务另外设计。
|
||||
|
||||
Redis 适配器借用调用方创建的 go-redis 客户端和连接池:
|
||||
|
||||
```go
|
||||
client := goredis.NewClient(&goredis.Options{
|
||||
Addr: "127.0.0.1:6379", PoolSize: 16, MaxRetries: -1,
|
||||
DialTimeout: time.Second, ReadTimeout: time.Second,
|
||||
WriteTimeout: time.Second, PoolTimeout: time.Second,
|
||||
ContextTimeoutEnabled: true,
|
||||
})
|
||||
// 启动时先使用有截止时间的 context 执行 client.Ping(ctx)。
|
||||
store, err := rediscache.New(client, "billing:") // cache/redis 的导入别名
|
||||
if err != nil { _ = client.Close(); return err }
|
||||
application.OnShutdown(client.Close)
|
||||
// store 可直接传入 cache.NewLoader。
|
||||
```
|
||||
|
||||
连接失败与 Redis 错误保持原样;只有 Redis nil 映射为 `ErrMiss`。Redis TTL 精度为毫秒,
|
||||
正的亚毫秒 TTL 提升到 1ms。适配器不负责关闭借用的客户端,Redis 服务端容量策略由部署配置。
|
||||
连接池和超时配置参考 [go-redis 官方生产使用说明](https://redis.io/docs/latest/develop/clients/go/produsage/)。
|
||||
|
||||
## 出站 HTTP:httpc
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
|
||||
MaxResponseBytes: 1 << 20,
|
||||
})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(client.Close)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
|
||||
```
|
||||
|
||||
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机
|
||||
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
|
||||
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
|
||||
|
||||
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`;
|
||||
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
|
||||
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
|
||||
|
||||
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
|
||||
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
|
||||
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
|
||||
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
|
||||
重定向交给业务处理,3xx 不会自动跟随。
|
||||
|
||||
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时,
|
||||
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
|
||||
|
||||
## 请求保护:middleware
|
||||
|
||||
```go
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40})
|
||||
if err != nil { return err }
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{
|
||||
Scope: func(c *httpx.Context) (string, error) {
|
||||
principal, _ := auth.FromContext(c)
|
||||
return principal.UserID, nil // 多租户应用应组合租户和用户 ID
|
||||
},
|
||||
})
|
||||
if err != nil { return err }
|
||||
api := application.Group("/api", middleware.Timeout(3*time.Second),
|
||||
auth.RequireAuth(authenticator), limiter.Middleware())
|
||||
api.POST("/orders", createOrder, idempotency.Middleware())
|
||||
```
|
||||
|
||||
限流为进程内令牌桶,`Rate` 为每秒额度,`Burst` 为突发容量。默认按已解析的可信客户端 IP,
|
||||
可用 `Key` 按已认证身份分桶。额度不足返回 **429 + Retry-After**;key 数超过 `MaxKeys`
|
||||
返回 **503**。满容量时不会踢掉活跃桶让攻击者绕过限流;空闲桶经过完整补充周期后才允许回收。
|
||||
|
||||
Timeout 设置协作式 deadline,不额外启动 Handler goroutine。业务及下游调用必须响应 context;
|
||||
超时后尚未提交响应时返回 **504**,已经提交的响应保持原状态。它不能强制中断任意 Go 代码,
|
||||
也不能替代网络层读写超时。不要在业务代码中异步持有 `httpx.Context`。
|
||||
|
||||
幂等保护适用于小型、非流式写接口:
|
||||
|
||||
- 必须先认证/鉴权,再执行幂等中间件;`Scope` 必须取自服务端验证过的身份。
|
||||
GET/HEAD/OPTIONS 直接通过,写请求要求 `Idempotency-Key`,长度 1..128 个非空格 ASCII 字符。
|
||||
- 同 Scope + key + 方法/URL/Content-Type/body 重放首次响应;内容改变返回 **409**。
|
||||
正在执行或结果不可重放的相同请求也返回 **409**,不并发执行第二次。
|
||||
- 默认最多 32 个处理中请求、256 条记录、16 MiB 保存的响应头/body,单次请求体 1 MiB、
|
||||
响应 body 64 KiB。请求超限 **413**,容量满 **503**,响应超限 **500**。
|
||||
- panic、5xx、超时、响应超限、Set-Cookie、Cache-Control:no-store 或保存容量不足时保留失败记录,
|
||||
阻止在 TTL 内自动重执行业务。业务可能已经提交,调用方应查询实际结果后处理。
|
||||
- 完成记录默认保留 10 分钟;在途记录不会因 TTL 到期而允许重复执行。
|
||||
重放保留原响应 body、原 `requestId` 与原 `X-Request-ID`,增加 `Idempotency-Replayed:true`;
|
||||
本次访问日志仍使用本次请求 ID。
|
||||
- 记录仅在当前进程有效,重启、TTL 到期和多副本切换后均无法保证去重。支付、库存等关键写入
|
||||
应在数据库事务内配合唯一业务键。该实现不提供跨进程 exactly-once,也不支持 SSE/流式/升级连接。
|
||||
|
||||
## 后台执行:task
|
||||
|
||||
```go
|
||||
jobs, err := task.New(task.Config{Workers: 4, QueueSize: 128, TaskTimeout: 30*time.Second})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(jobs.Close) // 在注册数据库/缓存关闭之后注册,先排空任务
|
||||
future, err := jobs.Submit(serviceContext, "refreshCatalog", refreshCatalog)
|
||||
if err != nil { return err } // 满队列立即返回 ErrQueueFull
|
||||
err = future.Wait(ctx) // 可选;不等待时,任务失败仍通过 OnError/logx 报告
|
||||
err = jobs.Every("refreshCatalog", time.Minute, refreshCatalog)
|
||||
```
|
||||
|
||||
固定 worker 和有界队列控制资源。任务保留提交 context;需要独立于 HTTP 请求执行时使用服务
|
||||
context,或明确使用 `context.WithoutCancel` 保留日志关联值。任务仍受 Runner 的超时/退出控制。
|
||||
任务函数与 `OnError` 回调应及时返回,尤其要响应取消;panic 被转为失败,worker 继续工作。
|
||||
|
||||
`Every` 使用固定间隔,第一次在一个间隔后执行;同一计划上轮尚未完成、排队或队列满时跳过。
|
||||
默认最多 128 个计划,名称不可重复。`Stats` 提供 active/queued/completed/failed/skipped。
|
||||
它是进程内任务执行器,没有任务持久化、失败自动重试、cron 表达式、跨副本选主或崩溃恢复。
|
||||
|
||||
`Shutdown(ctx)` 停止接收和调度,排空已接受任务;截止时间到后取消任务并及时返回。
|
||||
忽略取消的任务不能被 Go 强杀,可能继续运行。`Close` 默认给排空 10 秒。
|
||||
|
||||
## 增加业务模块:go-omni add
|
||||
|
||||
```sh
|
||||
go build -o /tmp/go-omni ./cmd/go-omni
|
||||
/tmp/go-omni add orderItems --dir /path/to/billing
|
||||
```
|
||||
|
||||
生成 `internal/orderItems/{module.go,model,repository,service,handler,README.md}`,包含 name/createdAt
|
||||
示例模型、内存 CRUD、输入校验、错误映射及可注入 Repository。模块名使用 lowerCamelCase。
|
||||
|
||||
在现有启动代码中导入模块后调用 `orderItems.Register(application, repository, authMiddleware)`。
|
||||
脚手架不自动修改现有路由、main、go.mod 或数据库,已有目录不会覆盖。
|
||||
内存 Repository 仅为可运行示例:重启丢数据,列表未分页;正式业务应实现自己的持久化 Repository。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --goproxy https://goproxy.cn,direct
|
||||
```
|
||||
|
||||
Linux runner 顺序运行两个容器:全量 race/vet/build/独立生成项目,随后真实 Redis 适配验证。
|
||||
每次最多一个容器,限制 2 CPU、2 GiB(不额外使用 swap)、256 pids;Redis 测试容器禁外网,
|
||||
只用内部 loopback,不发布端口。Redis 集成二进制为静态非 race,适配器单测在全量 race 中覆盖。
|
||||
结束清理本轮容器,并记录提交、源文件摘要、镜像 ID、日志摘要和原有容器是否仍运行。
|
||||
|
||||
本轮通过结果和原始证据见 [业务模块验收](performance/modulesAcceptance.md)。
|
||||
89
PERFORMANCE.md
Normal file
89
PERFORMANCE.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# GoOmni 性能基线
|
||||
|
||||
这轮测量显示 native 与标准库处于相同吞吐量级,在本机短时测试中,native 的空闲长连接堆内存更少。当前 native 仅实现 HTTP 的部分能力,尚不能据此判断生产容量或替代完整标准库服务端。
|
||||
|
||||
## 运行方式
|
||||
|
||||
```sh
|
||||
# race、vet、build,以及进程内和真实 TCP benchmark
|
||||
python3 scripts/verifyPerformance.py
|
||||
|
||||
# 另加独立服务进程的并发、延迟和空闲长连接内存矩阵
|
||||
python3 scripts/verifyPerformance.py --load --output /tmp/goOmniPerf.json
|
||||
|
||||
# 只运行独立进程矩阵,默认每档 2 秒、3 轮
|
||||
go run ./internal/perfcheck --duration 2s --count 3 --output /tmp/goOmniPerf.json
|
||||
|
||||
# 在 Linux Docker 容器内固定运行 race、vet、build 和 TLS 压测
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30s \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS.json
|
||||
|
||||
# 发布候选验收可把持续流量延长到 30 分钟
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30m \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS-30m.json
|
||||
|
||||
# Linux 上用 Nginx 反向代理接入 native TLS 服务,检查 Keep-Alive、超时和优雅关闭
|
||||
python3 scripts/verifyProxy.py
|
||||
```
|
||||
|
||||
可通过 `--concurrency 1,32,128` / `--idle 128,1024` 调整矩阵;传空字符串可跳过对应场景。工具只监听本机随机端口,每个场景启动独立服务进程,并在结束后关闭客户端、检查连接排空、关闭服务。响应状态、完整响应体、requestId 或传输错误均计入失败,不能把快速返回的错误页算成成功吞吐。
|
||||
|
||||
Linux 验收脚本使用 `golang:1.25` 容器,固定执行 `go test -race ./...`、`go vet ./...`、`go build ./...`,再按参数运行 perfcheck。perfcheck 在 Linux 读取 `/proc/self/fd`,报告连接建立期间的文件描述符峰值以及回收后的数量;同时记录服务端堆、goroutine、连接数和累计 accept 数。30 秒报告用于快速回归,30 分钟命令用于发布前持续压测。
|
||||
|
||||
## 2026-09-21 本机测量
|
||||
|
||||
环境:Go 1.25.0、darwin/arm64、Apple M2 Max、GOMAXPROCS=12。网络实现来自 `3d5f225` 工作树(native 与 v0.6.0-preview 相同);本次新增测量工具,不修改服务端实现。原始测量值见 [tcpBaseline.json](performance/tcpBaseline.json)。报告时间为 UTC,北京时间是 9 月 21 日。
|
||||
|
||||
每个并发档位持续约 2 秒、重复 3 轮,每轮交替标准库/native 的先后顺序。下表为各指标的三轮中位数;全部流量场景共完成 **2,660,828** 次请求,错误 **0**。另有 128 / 1,024 条空闲长连接的建立、保持和回收测试。
|
||||
|
||||
| 并发请求 | 标准库成功请求/秒 | native 成功请求/秒 | 标准库 P95 | native P95 | 标准库 P99 | native P99 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | 19,731 | 21,497 | 0.074 ms | 0.070 ms | 0.139 ms | 0.119 ms |
|
||||
| 32 | 90,099 | 88,812 | 1.016 ms | 0.983 ms | 1.507 ms | 1.507 ms |
|
||||
| 128 | 112,422 | 112,105 | 3.277 ms | 3.408 ms | 4.719 ms | 4.981 ms |
|
||||
|
||||
128 并发时,两者吞吐差约 0.3%,native 的尾延迟略高。基于这组结果,暂时没有理由仅为健康接口吞吐就改成手写事件循环。
|
||||
|
||||
| 空闲 Keep-Alive 连接 | 标准库新增堆内存 | native 新增堆内存 | 标准库约每连接 | native 约每连接 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 128 | 1.275 MiB | 0.658 MiB | 10,444 B | 5,389 B |
|
||||
| 1,024 | 9.871 MiB | 5.137 MiB | 10,108 B | 5,260 B |
|
||||
|
||||
1,024 条空闲连接时,native 的新增堆内存约少 48%。这是**服务端 Go 堆内存增量**,不包括 goroutine 栈、进程其他内存和内核 socket 缓冲区,不能解释为整个进程内存减半。同档服务端 `StackInuse` 分别约为标准库 8.75 MiB、native 8.69 MiB,原始报告单独记录。
|
||||
|
||||
连接断开后,两者连接数都回到 0,goroutine 分别回到 2 / 3 的初始基线。一次 GC 后仍有对象池缓存;第二次 GC 后,总堆内存中位数回到约 2.10 / 2.08 MiB(启动基线约 1.54 MiB)。堆内存不会立刻精确回到启动数值,不把缓存保留解释为泄漏,也不把这轮短测解释为长期无泄漏证明。
|
||||
|
||||
128 并发时,**服务端单独累计分配**约为标准库 4,454 B/请求、47 次/请求,native 4,297 B/请求、55 次/请求。native 的分配字节较少、次数较多;具体来源需用分配 profile 确认,不能仅凭总量归因于 HTTP 桥接或 JSON。
|
||||
|
||||
## 统计边界
|
||||
|
||||
- 同一台机器上,压测客户端和服务端是两个独立进程。两者使用同一健康接口、相同小型 JSON 响应、固定 `requestId=perfCheck` 和 Keep-Alive;不启用访问日志、TLS 或数据库。
|
||||
- 并发模型为闭环:每个 worker 等上一个请求完成后发下一个。实际经过时间包含最后请求排空;吞吐只计算成功响应。不能据此推断开放到达流量下的排队或过载表现。
|
||||
- 延迟从客户端发起请求到完整响应校验完成,包含失败请求。直方图使用每个 2 的幂区间内 16 档,报告桶上界,舍入误差至多约 6.25%。小幅差别不应过度解释。
|
||||
- 服务端 `runtime.MemStats` 通过独立 stdin/stdout 控制通道读取,客户端内存不计入。累计分配差值包含少量采样开销。堆峰值每 200 ms 采样,可能错过瞬时峰值;这些指标不是进程 RSS。
|
||||
- Linux 文件描述符来自服务端子进程的 `/proc/self/fd`,包含标准输入输出、监听 socket 和连接;macOS 等没有该路径的平台会省略该字段。回收后的数量应回到启动附近,不能代替内核 socket 缓冲或 RSS 测量。
|
||||
- 空闲场景在所有连接完成健康请求后做 GC 再采样。新增堆内存包括连接、缓冲区及分摊的运行时开销。`after` 为客户端断开并排空后一次 GC,`afterReclaim` 为第二次 GC,用于观察对象池缓存释放。
|
||||
- native 配置上限为 8,192 连接 / 4,096 活跃请求;本轮并发低于上限,未验证过载保护容量。标准库没有同样的并发闸门;比较包括各自真实实现的开销。
|
||||
- 可用 `python3 scripts/verifyPerformance.py --load --tls --duration 2s` 运行 TLS 版本的独立进程矩阵。该模式在每个子进程生成短期自签名证书,客户端仅用于本机基准验证;本报告的历史表格仍是无 TLS 数据。
|
||||
|
||||
## benchmark 口径修正
|
||||
|
||||
## TLS 验收记录
|
||||
|
||||
macOS arm64 本机的 5 秒 TLS 矩阵见 [tlsBaseline.json](performance/tlsBaseline.json):1 / 32 / 128 并发和 128 / 1,024 条空闲连接均为 0 错误;native 在 128 并发约 114,232 req/s,1,024 条空闲 TLS 连接的服务端堆增量约 8.94 MiB,连接关闭后 goroutine 回到基线。
|
||||
|
||||
Linux arm64 容器使用 Go 1.25.14、GOMAXPROCS=4 运行 3 秒矩阵,见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json):1 / 32 / 128 并发和 128 / 512 条空闲连接均为 0 错误;native 在 128 并发约 99,876 req/s,512 条空闲 TLS 连接的服务端堆增量约 4.47 MiB,连接关闭后 goroutine 回到基线。容器验证同时通过全量 race、vet、build。
|
||||
|
||||
Linux arm64 的 30 秒高并发回归见 [linuxSoak.json](performance/linuxSoak.json) 和 [linuxSoakTLS.json](performance/linuxSoakTLS.json):两份报告均使用 128 并发流量和 512 条空闲连接,所有场景错误为 0;native 非 TLS 约 146,614 req/s、TLS 约 149,720 req/s,空闲连接堆增量分别约 2.57 MiB 和 4.47 MiB,FD 均由 7 增至 519 并回到 7,goroutine 均回到启动附近。
|
||||
|
||||
目标 Linux amd64 服务器的 30 分钟 TLS 验收使用 1 个 CPU、1 GiB 内存的隔离 Docker 容器完成:net/http 约 11,865 req/s,native 约 13,098 req/s,两个流量场景均为 0 错误;native 的 P95/P99 为 44.040/52.429 ms。512 条空闲 TLS 连接时,native 服务端堆增量约 4.50 MiB(约 9,215 B/连接),文件描述符由 7 增至 519 后回到 7,goroutine 和连接均正常回收。原始报告保存在服务器的 `performance/linuxServerSoakTLS.json`。
|
||||
|
||||
`BenchmarkTransportTCPApplicationRequest/{nethttp,native}/concurrency{1,32,128}` 使用同一测试入口、客户端、应用与响应检查。原先独立的 `BenchmarkNativeApplicationRequest` 已合并,避免两个入口配置漂移。`verifyPerformance.py` 会一并运行。
|
||||
|
||||
并行 benchmark 的 **`ns/op` 是总经过时间除以请求数,反映吞吐成本,不是单请求延迟**。`B/op` / `allocs/op` 统计同一进程中的客户端与服务端总分配。此前报告的 native 15.4–16.7 µs/op、约 7.8 KB/op、100 allocs/op 应按这个含义理解,不能作为服务端延迟或独立内存指标。
|
||||
|
||||
`BenchmarkTransportRouteDispatch` 和 `BenchmarkTransportApplicationRequest` 继续保留,分别测进程内路由分发和完整框架处理链;不包含 TCP,不能直接与上述网络延迟比较。
|
||||
|
||||
后续通过 30 分钟命令完成发布前持续压测,并在目标反向代理和部署环境继续观察内核 socket 缓冲、FD 上限与真实业务流量;再通过 profile 决定是否优化对象分配或事件状态机。
|
||||
135
README.md
135
README.md
@@ -1,17 +1,19 @@
|
||||
# GoOmni
|
||||
|
||||
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认用 Gin 作为网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
|
||||
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用只依赖标准库的网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
|
||||
|
||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.3.1
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
||||
```
|
||||
|
||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||
|
||||
创建新业务项目:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.3.1
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -36,15 +38,35 @@ go-omni/
|
||||
├── health/ # 存活和就绪检查
|
||||
├── middleware/ # 请求 ID 等基础中间件
|
||||
├── logx/ # 业务层结构化日志
|
||||
├── metrics/ # 请求指标与状态统计
|
||||
├── config/ # 配置管理
|
||||
├── validate/ # 参数校验
|
||||
├── auth/ # 认证和权限接口
|
||||
├── database/ # database/sql 生命周期和事务
|
||||
├── transport/gin/ # Gin 网络适配器,可替换
|
||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
||||
├── task/ # 有界后台任务与间隔调度
|
||||
├── queue/ # 持久队列接口与 Redis Streams 适配
|
||||
├── storage/ # Local、S3/R2、OSS 与临时下载链接
|
||||
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
|
||||
├── telemetry/ # 可注入 OpenTelemetry 观测
|
||||
├── resilience/ # 熔断与上游并发隔离
|
||||
├── audit/ # 显式业务审计与可注入 Sink
|
||||
├── pagination/ # 有界分页和统一结果
|
||||
├── transport/nethttp/ # 标准库网络适配器
|
||||
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
||||
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
||||
├── examples/ # 示例项目
|
||||
└── cmd/ # go-omni 项目脚手架
|
||||
```
|
||||
|
||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||
可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
|
||||
|
||||
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```go
|
||||
@@ -80,7 +102,7 @@ func main() {
|
||||
return omni.NotFound("USER_NOT_FOUND", "user was not found")
|
||||
```
|
||||
|
||||
默认 Gin 适配器位于 `transport/gin`。业务层不需要导入 Gin;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
|
||||
默认网络适配器位于 `transport/nethttp`。业务层不需要导入具体网络库;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
|
||||
|
||||
```go
|
||||
application := omni.NewWithRouter(myRouter)
|
||||
@@ -88,11 +110,45 @@ application := omni.NewWithRouter(myRouter)
|
||||
|
||||
仓库还提供了使用标准库路由的 `transport/nethttp` 参考实现。两种适配器已通过同一套路由、业务处理链和取消传播测试,完整用户 CRUD 复用同一份 Handler / Service。
|
||||
|
||||
自研适配器可以直接运行 `transport/transporttest.Run` 验收。具体规则、v0.3.1 的斜杠 / Any / 405 行为调整,以及自研网络内核与当前 HTTP 接口的边界,见 [适配层契约](transport/README.md)。
|
||||
自研适配器可以直接运行 `transport/transporttest.Run` 验收。具体规则、v0.4.0 的斜杠 / Any / 405 行为调整,以及自研网络内核与当前 HTTP 接口的边界,见 [适配层契约](transport/README.md)。
|
||||
|
||||
`app.Engine()` 现在返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,避免业务 API 锁定 Gin 类型。
|
||||
`app.Engine()` 返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,业务 API 不锁定具体网络实现。
|
||||
|
||||
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖 Gin:
|
||||
需要试用第一版自研网络层时,业务注册代码无需改变,只替换启动器:
|
||||
|
||||
```go
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
app := omni.New()
|
||||
app.GET("/health", func(ctx *omni.Context) error {
|
||||
return ctx.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
if err := app.RunWith(ctx, native.New(nil), ":8080"); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
```
|
||||
|
||||
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 和 chunked 请求体、`http.Flusher` 流式响应、可选 TLS、读写超时和优雅关闭;不支持的传输编码会被拒绝。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
|
||||
|
||||
native TLS 当前只支持并宣告 ALPN `http/1.1`;配置 `h2` 会在启动时快速失败。需要 HTTP/2 时,可由 Nginx/Cloudflare 终止后回源 HTTP/1.1,或使用默认的 `transport/nethttp` 适配器。
|
||||
|
||||
`transport/http2` 是独立的 opt-in 实验适配器,使用 `x/net/http2` 处理帧和流控,同时保留 GoOmni 的连接上限、全局请求并发闸门和生命周期边界。它不会改变 native 默认路径;实验入口、限制和验收结果见 [HTTP/2 进入条件](transport/native/HTTP2.md)。
|
||||
|
||||
自研网络层还提供 `server.Stats()`,可读取 accepted、rejected、active connections 和 active requests,与上面的业务请求 metrics 组合后能区分连接堆积、过载拒绝和 Handler 慢请求。`rejectedConnections` 表示连接闸门拒绝数,`rejectedRequests` 表示并发请求闸门返回 503 的次数。
|
||||
|
||||
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖具体网络库:
|
||||
|
||||
```go
|
||||
func (s *UserService) Create(ctx *httpx.Context) error {
|
||||
@@ -101,6 +157,16 @@ func (s *UserService) Create(ctx *httpx.Context) error {
|
||||
}
|
||||
```
|
||||
|
||||
需要基础运行指标时,可以注册 transport-neutral metrics middleware 和接口:
|
||||
|
||||
```go
|
||||
requestMetrics := omni.NewMetrics()
|
||||
app.Use(requestMetrics.Middleware())
|
||||
app.GET("/metrics", requestMetrics.Handler())
|
||||
```
|
||||
|
||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
||||
|
||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||
@@ -120,9 +186,40 @@ application.OnShutdown(logger.Close)
|
||||
|
||||
常用变量包括 `GO_OMNI_ENABLE_LOGGER`、`GO_OMNI_LOG_LEVEL`、
|
||||
`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_OUTPUT`、`GO_OMNI_SERVICE`、
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR` 和 `GO_OMNI_SHUTDOWN_TIMEOUT`。数据库连接仍由
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR`、`GO_OMNI_SHUTDOWN_TIMEOUT` 和
|
||||
`GO_OMNI_TRUSTED_PROXIES`。可信代理变量使用逗号分隔的 IP/CIDR,例如
|
||||
`GO_OMNI_TRUSTED_PROXIES=127.0.0.1,10.0.0.0/8`;只有来自这些地址的连接才会解析
|
||||
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
|
||||
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
|
||||
|
||||
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
|
||||
|
||||
| 配置 | 默认值 / 用途 |
|
||||
| --- | --- |
|
||||
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
|
||||
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
|
||||
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
|
||||
|
||||
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_TRANSPORT=native go run ./examples/reference
|
||||
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
|
||||
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
|
||||
go run ./examples/reference
|
||||
```
|
||||
|
||||
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
|
||||
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
|
||||
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
|
||||
|
||||
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
|
||||
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
|
||||
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
|
||||
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
|
||||
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
|
||||
|
||||
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
|
||||
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
|
||||
|
||||
@@ -169,10 +266,24 @@ userService := service.NewUsers(services)
|
||||
从远程版本生成并升级一个独立业务模块,可运行:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyBusiness.py --version v0.3.1
|
||||
python3 scripts/verifyBusiness.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.3.1`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.6.0-preview`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
|
||||
标准库与 native 的并发、延迟和长连接内存对照见 [PERFORMANCE.md](PERFORMANCE.md)。`python3 scripts/verifyPerformance.py` 运行检查与 benchmark;增加 `--load --output /tmp/goOmniPerf.json` 可测量独立服务进程并保存报告。
|
||||
|
||||
HTTP/2 仍是显式 opt-in 实验适配器,不会改变 native 的 HTTP/1.1 默认路径。Linux TLS 长测使用 `python3 scripts/verifyHTTP2.py --duration 30m`,默认使用普通 Go 运行模式;`--race` 默认改为 60 秒,用于并发检查,其额外内存开销不能当成生产运行基线。
|
||||
|
||||
多连接混合场景使用 `python3 scripts/verifyHTTP2.py --scenario mixed --duration 30m`:8 条独立 TLS 连接,每条连接 14 个普通请求 worker、1 个慢上传 worker、1 个慢下载 worker,共 128 个 worker。上传为 256 KiB、每 4 KiB 等待 2 ms;下载为 8 MiB、每次最多读取 64 KiB 后等待 20 ms。日志分别记录三类成功数、传输字节数、每条连接的覆盖情况,以及停止并释放测试数据后两次 GC 的采样。
|
||||
|
||||
该脚本默认把测试容器限制为 2 核、2 GiB 内存(禁用 swap)、256 个进程/线程,整个命令最多运行 40 分钟,中断和超时会清理本次容器。这些限制可用 `--cpus`、`--memory-mb`、`--pids-limit`、`--wall-timeout` 调整;共享服务器应先用 `--duration 60s` 短测,再逐步延长。每 15 秒输出的堆、RSS 和 FD 统计包含同一进程内的客户端与服务端。
|
||||
|
||||
Linux amd64 已完成 30 分钟、单 TLS 连接、128 个 worker 的普通模式长测:58,102,476 次请求,0 失败、0 协议错误,采样 RSS 峰值约 22.6 MiB,连接/活动请求归零且 FD 回到基线。完整采样和原始日志见 [HTTP/2 长测报告](performance/http2SoakLinux.json) 与 [原始日志](performance/http2SoakLinux.txt),此前的 race 对照见 [短测记录](performance/http2SmokeLinux.json)。这项结果覆盖单连接复用。
|
||||
|
||||
多连接慢客户端场景也已完成 Linux amd64 的 30 分钟长测:8 条 TLS 连接、128 个 worker,53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,共 53,191,687 次,0 失败与协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端、服务端和下载缓冲);退出后连接/请求归零,goroutine 回到 2、FD 回到 6;释放测试数据并两次 GC 后存活堆约 450 KiB,RSS 不要求立即归还操作系统。完整 120 份采样见 [混合长测报告](performance/http2MixedSoakLinux.json) 和 [原始日志](performance/http2MixedSoakLinux.txt)。连接级窗口耗尽、WINDOW_UPDATE 恢复及零窗口写阻塞的关闭回收也已有独立测试;Nginx 的 HTTP/2 回源验收结果见下方,适配器保持实验状态。
|
||||
|
||||
`python3 scripts/verifyHTTP2Proxy.py --count 3` 验证真实 Nginx 双段 TLS/h2、回源连接复用、证书拒绝、超时/取消恢复与新旧实例的 GOAWAY 排空;增加 `--race` 可运行并发检查。Linux amd64、Nginx 1.30.5 的普通与 race 模式各 3 轮均已通过。编译和测试容器顺序运行,均限制为 2 核、2 GiB;测试容器禁用外部网络,全部请求在容器内完成。使用方法、报告及当前边界见 [HTTP/2 代理回源验收](transport/http2/PROXY.md)。
|
||||
|
||||
在框架根目录运行:
|
||||
|
||||
@@ -182,7 +293,7 @@ go run ./examples/reference
|
||||
|
||||
默认地址为 `http://127.0.0.1:8080`。示例文档提供完整的 curl 操作流程。数据只保存在内存中,重启后清空。
|
||||
|
||||
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入 Gin。
|
||||
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入网络库。
|
||||
|
||||
## 当前边界
|
||||
|
||||
|
||||
71
ROADMAP.md
71
ROADMAP.md
@@ -18,21 +18,74 @@
|
||||
- 已验证初始化失败、缺少表结构、正常退出时关闭连接池,以及数据库停止后的 readiness 503、恢复后的 200。
|
||||
- 自动验收入口:`python3 scripts/verifyDatabase.py --version v0.2.0`;结束后清理本次测试容器、匿名卷和临时项目。
|
||||
|
||||
## 3. 网络适配一致性(已完成,v0.3.0)
|
||||
## 3. 网络适配一致性(已完成,v0.4.0)
|
||||
|
||||
- Gin 与标准库适配器运行同一套契约测试。
|
||||
- 标准库适配器运行公共契约测试,并为自研网络层保留相同的接入边界。
|
||||
- 覆盖路由参数、分组、中间件、错误响应和取消传播。
|
||||
- 验收:切换适配器后,业务 Handler 和 Service 无需修改,约定的 HTTP 行为一致。
|
||||
- 新增公共 `transport/transporttest.Run`,自研适配器可直接复用;覆盖静态 / 参数 / 通配路由、分组、405 Allow、鉴权短路、错误与 panic、响应提交、requestId、并发隔离、截止时间及真实客户端取消。
|
||||
- 两种适配器共享明确的路径与方法约定,修复尾部斜杠、通配参数、Any 和 Gin 405 Allow 差异。
|
||||
- 完整用户 CRUD 已用相同业务注册代码在两种适配器上验证;全量 race 测试通过。
|
||||
- 自动验收入口:`python3 scripts/verifyTransports.py --version v0.3.0`,从源码目录外消费远程版本与公共契约测试包。
|
||||
- 标准库路由实现明确的路径、方法、尾部斜杠、通配参数、Any 和 404/405 约定;不再引入 Gin 依赖。
|
||||
- 完整用户 CRUD 已用标准库适配器验证;自研适配器可直接复用同一公共契约测试包。
|
||||
- 自动验收入口:`python3 scripts/verifyTransports.py --version v0.4.0`,从源码目录外消费远程版本与公共契约测试包。
|
||||
|
||||
## 4. 实际业务试用(已完成,v0.3.1)
|
||||
## 4. 实际业务试用(已完成,v0.4.0)
|
||||
|
||||
- 在独立业务目录生成 billing service,先固定远程 `v0.2.0`,再升级到 `v0.3.1`,全程无本地 replace。
|
||||
- 在独立业务目录生成 billing service,先固定远程 `v0.2.0`,再升级到 `v0.4.0`,全程无本地 replace。
|
||||
- 验证环境配置、结构化访问日志、requestId 关联、参数校验、业务错误映射、CRUD、优雅退出和升级后的 race 测试。
|
||||
- 业务试用没有要求修改 Handler / Service;现有公共 API 足以支撑一套分层业务模块。
|
||||
- 自动验收入口:`python3 scripts/verifyBusiness.py --version v0.3.1`。
|
||||
- 自动验收入口:`python3 scripts/verifyBusiness.py --version v0.4.0`。
|
||||
|
||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,生产压测与自研网络内核尚未完成。后续进入稳定性与性能验证,再决定稳定版范围。
|
||||
## 5. 自研网络层、稳定性与性能基线(v0.6.0-preview 验收完成)
|
||||
|
||||
- 已加入标准库路由和完整 GoOmni 请求链 benchmark,输出分配次数和耗时,作为未来自研网络层的对照基线。
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端:自行管理 TCP listener、Keep-Alive、Content-Length / chunked 请求解析、响应写回、读写超时和优雅关闭;通过 `App.RunWith` 接入,业务 Handler / Service 保持不变。
|
||||
- 标准库与 native 共用 TCP benchmark,固定 1 / 32 / 128 并发;相同应用、客户端、超时和响应检查,避免两个测试入口逐渐失去可比性。
|
||||
- 新增独立服务进程压测 `internal/perfcheck`,记录成功吞吐、P50/P95/P99、服务端分配、采样堆峰值,以及 128 / 1,024 条空闲长连接的堆内存、goroutine 和断开后的回收情况。原始报告保存在 `performance/tcpBaseline.json`。
|
||||
- native 已支持受 `MaxBodyBytes` 约束的 `Content-Length` / chunked 请求体,仍使用标准 `http.Request` / `http.ResponseWriter` 作为桥接;解析器、连接抽象和生产压测将在后续阶段继续下沉。
|
||||
- native 请求体已改为流式读取,Handler 可边到边处理,服务端只在 Handler 返回后排空剩余 body,超时、畸形和超限 body 会取消请求并关闭连接。
|
||||
- native 响应已支持标准 `http.Flusher`;HTTP/1.1 流式响应使用 chunked framing,普通响应继续走 `Content-Length`,HTTP/1.0 回退到关闭连接分帧。
|
||||
- native 长请求已补客户端断开及流式写失败取消,监测退出后再复用 Reader;已验收普通/流式断开、慢 Handler 后连续 Keep-Alive 和管线数据不被误消费。短请求延迟启动监测,已有管线数据时取消依赖后续写失败或 Handler 返回。
|
||||
- 已补充慢请求头、慢 Content-Length body、慢 chunked body、慢 TLS 握手、大请求体、HTTP/1.0 Keep-Alive、HEAD/204 和管线请求验收;Linux/arm64 容器已通过全量 race、vet、build,以及 3 秒 TLS 并发/空闲连接矩阵。原始报告见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json)。
|
||||
- 已验证 `MaxConnections` 满载时快速拒绝新连接、请求完成后释放容量;后续仍需在目标部署环境观察文件描述符、内核 socket 缓冲和反向代理联动。
|
||||
- 已验证 `MaxConcurrentRequests` 满载返回 503,活动请求结束后槽位释放,后续请求可以恢复处理。
|
||||
- native `Stats()` 已记录连接和请求过载拒绝数;访问日志已支持可信代理网段配置,默认拒绝伪造的 `X-Forwarded-For`。
|
||||
- native TLS 已明确只宣告 HTTP/1.1 ALPN;HTTP/2 暂由边缘代理终止或使用 `transport/nethttp`,已整理独立协议设计、流控和压测门槛,见 [native HTTP/2 进入条件](transport/native/HTTP2.md),完成后再接入 native。
|
||||
- 已加入独立 `transport/http2` 实验 Runner,验证 h2c/TLS h2、连接上限、全局请求并发拒绝和恢复,以及 GOAWAY 排空、RST_STREAM 单流取消和服务关闭超时;它使用 `x/net/http2` 作为临时帧/流控实现,已完成下述 Nginx HTTP/2 回源联调,尚未接入 native 默认路径。
|
||||
- HTTP/2 实验已补齐头部/请求体上限、128 路同连接多路复用、慢读取流控、协议错误、RST_STREAM 与 GOAWAY 统计,并提供 `scripts/verifyHTTP2.py --duration 30m` 的 Linux TLS 长测入口;长测默认不启用 race,短时并发检查使用 `--race`(默认 60 秒)。测试容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,并设总时限和退出清理;受限长测和 Nginx 回源联调结果见下方。
|
||||
- 新 Linux amd64 环境在上述硬上限内完成 HTTP/2 普通模式 2 分钟与 race 模式 60 秒短测,分别为 3,885,273 和 377,788 次请求,均为 0 失败;连接/请求归零、FD 回到基线。SIGTERM、SIGHUP 和总时限退出清理已实测。报告见 [HTTP/2 短测记录](performance/http2SmokeLinux.json);此前无资源上限的 race 长测因 RSS 增长而停止,不作为通过依据。
|
||||
- 已完成受限的 HTTP/2 单连接 TLS 30 分钟长测:128 个 worker、58,102,476 次请求、0 失败及协议错误;包含客户端与服务端的采样 RSS 峰值约 22.6 MiB,连接/活动请求归零,FD 由 9 回到 6。120 份定时采样见 [HTTP/2 长测报告](performance/http2SoakLinux.json),测试提交和原始日志校验值保存在报告中。Nginx HTTP/2 回源的 GOAWAY/排空结果见下方。
|
||||
- 已完成 8 条 TLS 连接、128 个 worker 的 30 分钟混合长测:53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,0 失败及协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端和服务端),前段与最后 5 分钟 RSS 中位数分别约 111.3/110.4 MiB;退出后活动连接/请求归零,goroutine 2、FD 6 均回到基线,两次 GC 后存活堆约 450 KiB。原始日志、120 份采样、测试提交和校验值见 [混合长测报告](performance/http2MixedSoakLinux.json)。
|
||||
- 已补连接级窗口耗尽时的暂停、同连接 204/跨连接隔离、WINDOW_UPDATE 恢复和连接复用;已补零窗口写阻塞在关闭超时后的解除阻塞、请求取消与回收。Linux amd64 的 HTTP/2 全包 race、vet、build 已通过,见 [边界回归报告](performance/http2FlowControlLinux.json)。
|
||||
- 新增 HTTP/2 零 stream 窗口时的慢响应隔离、WINDOW_UPDATE 恢复、RST_STREAM 取消与请求槽位回收测试;补充 TLS 半开握手占满连接上限时的及时拒绝、超时回收及正常请求恢复测试。
|
||||
- 已完成 Nginx 1.30.5 → HTTP/2 的 TLS 回源联调:普通与 race 模式各 3 轮,共 18 个子场景通过;覆盖两段证书校验、32 次 POST 复用一条回源连接、不可信 CA/主机名拒绝、504 超时、客户端取消、GOAWAY 后旧 POST 完成及 20 个新 POST 转入新实例,无重复业务执行。见 [验收说明与报告](transport/http2/PROXY.md)。
|
||||
- `scripts/verifyHTTP2Proxy.py` 顺序运行受限编译容器和禁用外部网络的测试容器;总时限、SIGTERM、SIGHUP 清理已在 Linux 实测,原有 7 个业务容器仍运行。当前结果不覆盖 Nginx master/worker 重载或 Cloudflare 实网部署。
|
||||
- 已接入 `omni.New` 的统一 transport / TLS / shutdownTimeout 配置:默认 nethttp,可显式选择 native 或实验 HTTP/2;证书在监听前加载,HTTP/2 配置必须启用 TLS。`RunWith` 继续支持自定义网络层,业务不依赖具体适配器。
|
||||
- 参考业务已增加五种真实 TCP/TLS 组合的 CRUD、认证权限、错误响应、requestId 日志与协议协商测试;三种适配器共用排空和超时退出测试。修复 native 退出时过早取消在途请求,并使关闭超时错误返回调用方。下一步仍为业务环境试用、native 多协议监听器设计与接入回归,HTTP/2 保持实验状态。
|
||||
- 统一启动已通过 Linux 全仓库 race/vet/build 及关键场景 10 次重复验证;修复了重复测试发现的 native 活动连接转为空闲时错过关闭的竞态,并保证不响应取消的 Handler 不阻塞关闭超时返回。脚手架独立项目的 native SIGINT/SIGTERM 验证通过;失败复现与最终验收证据见 [统一启动验收](performance/transportStartup.md)。
|
||||
- 已用 16 个并发 worker 完成 3,200 次建连/断连 race 验收;MaxConnections 满载拒绝、槽位释放和 TLS 连接上限均已覆盖。
|
||||
- 新增 `scripts/verifyLinux.py`,固定在 Go 1.25 Linux 容器内运行 race、vet、build 和可选 TLS/非 TLS perfcheck;perfcheck 记录堆、goroutine、连接和 Linux 文件描述符。
|
||||
- Linux/arm64 已完成 30 秒、128 并发、512 空闲连接的 TLS 与非 TLS 持续回归,报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`,均为 0 错误且连接、goroutine、FD 回收正常。
|
||||
- 目标 Linux amd64 服务器已完成 30 分钟、128 并发、512 空闲连接的 TLS 验收:net/http 约 11,865 req/s、native 约 13,098 req/s,均为 0 错误;native P95/P99 为 44.040/52.429 ms,空闲连接堆增量约 4.50 MiB,FD 由 7 增至 519 后回到 7。
|
||||
- 新增 `scripts/verifyProxy.py`,在 Linux Docker 中用 Nginx 反向代理 native TLS 服务,检查真实 TLS、Keep-Alive、上游超时和 native 优雅关闭。
|
||||
- 自动验收入口:`python3 scripts/verifyPerformance.py`;Linux 固定验收入口:`python3 scripts/verifyLinux.py --load --tls --duration 30m`;代理联调入口:`python3 scripts/verifyProxy.py`。
|
||||
|
||||
## 6. 可复用业务基础模块(第一版)
|
||||
|
||||
- `cache`:有界 LRU 内存缓存、TTL、数据复制、同进程合并回源;`cache/redis` 适配 go-redis 客户端,保留借用连接池的生命周期边界。
|
||||
- `httpc`:复用出站连接、每目标主机连接上限、覆盖响应体的总超时、保守重试、requestId 传递、JSON 响应大小限制与安全日志字段。
|
||||
- `middleware`:有容量上限的令牌桶限流;按已认证身份隔离的幂等保护,覆盖冲突、处理中记录、失败记录、容量与请求/响应大小上限;协作式请求 deadline。
|
||||
- `task`:固定 worker、有界队列、任务超时、panic 隔离、间隔调度、不重叠执行、停止接收与排空/取消。
|
||||
- `go-omni add`:向已有项目生成 Handler / Service / Repository / Model 和模块注册入口;目标存在时拒绝覆盖,默认内存 CRUD 可注入持久化 Repository。
|
||||
- `examples/modules` 将以上运行时能力串联,使用真实 TLS 在 nethttp、native、实验 HTTP/2 下验证缓存、出站 requestId、认证、幂等、后台任务与退出。
|
||||
- 使用文档:[业务基础模块](MODULES.md)。验证入口:`GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...` 和 `python3 scripts/verifyModules.py`;Linux runner 顺序运行全量检查及独立 Redis 容器,硬限制 2 CPU / 2 GiB / 256 pids。
|
||||
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
||||
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
||||
|
||||
## 7. 扩展业务模块(本批)
|
||||
|
||||
- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储;补充迁移、审计和分页。
|
||||
- 业务依赖独立接口,云平台 SDK 和 Redis 留在适配器中;按本次范围决定不集成 Orbit,不实现工作流编排。
|
||||
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
|
||||
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
|
||||
|
||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
||||
|
||||
88
app/app.go
88
app/app.go
@@ -15,24 +15,32 @@ import (
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
ginrouter "git.yuntuo.hk/k2safe/go-omni/transport/gin"
|
||||
netrouter "git.yuntuo.hk/k2safe/go-omni/transport/nethttp"
|
||||
)
|
||||
|
||||
// App is the top-level GoOmni application. Its public behavior depends on
|
||||
// httpx.Router, leaving the network implementation replaceable.
|
||||
type App struct {
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
clientIPResolver *httpx.ClientIPResolver
|
||||
}
|
||||
|
||||
// New creates an application using the default Gin adapter.
|
||||
// Runner is the small lifecycle boundary used by an HTTP server
|
||||
// implementation. App.RunContext selects a configured built-in server; a custom runner
|
||||
// can own the listener, protocol parser and connection lifecycle.
|
||||
type Runner interface {
|
||||
Run(context.Context, http.Handler, string) error
|
||||
}
|
||||
|
||||
// New creates an application using GoOmni's standard-library router.
|
||||
func New(options ...config.Option) *App {
|
||||
settings := configure(options...)
|
||||
return newApp(ginrouter.New(settings.Mode), settings)
|
||||
return newApp(netrouter.New(), settings)
|
||||
}
|
||||
|
||||
// NewWithRouter creates an application with a custom network layer. A custom
|
||||
@@ -56,19 +64,24 @@ func configure(options ...config.Option) config.Config {
|
||||
}
|
||||
|
||||
func newApp(router httpx.Router, settings config.Config) *App {
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||
if err != nil {
|
||||
panic("go-omni: " + err.Error())
|
||||
}
|
||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||
if settings.EnableLogger {
|
||||
middlewares = append(middlewares, middleware.AccessLog())
|
||||
}
|
||||
middlewares = append(middlewares, middleware.Recovery())
|
||||
return &App{router: router, config: settings, middlewares: middlewares}
|
||||
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
||||
}
|
||||
|
||||
// Router returns the transport contract used by this application.
|
||||
func (a *App) Router() httpx.Router { return a.router }
|
||||
|
||||
// Engine is kept as a compatibility name for the old API. It returns a
|
||||
// standard http.Handler, so callers no longer need a Gin type.
|
||||
// standard http.Handler, so callers do not depend on a router type.
|
||||
func (a *App) Engine() http.Handler { return a.router }
|
||||
|
||||
// ServeHTTP implements http.Handler for standard servers and HTTP tests.
|
||||
@@ -77,7 +90,11 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
||||
}
|
||||
|
||||
// Config returns the effective application configuration.
|
||||
func (a *App) Config() config.Config { return a.config }
|
||||
func (a *App) Config() config.Config {
|
||||
settings := a.config
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
return settings
|
||||
}
|
||||
|
||||
// Use adds middleware to routes registered after this call.
|
||||
func (a *App) Use(middleware ...httpx.Middleware) *App {
|
||||
@@ -116,7 +133,8 @@ func (a *App) Run(address ...string) error {
|
||||
return a.RunContext(ctx, address...)
|
||||
}
|
||||
|
||||
// NewServer creates a standard HTTP server around the application.
|
||||
// NewServer creates a plain standard HTTP server around the application.
|
||||
// Use RunContext or ServeContext to apply the configured transport and TLS.
|
||||
func (a *App) NewServer(address string) *http.Server {
|
||||
if address == "" {
|
||||
address = a.config.Address
|
||||
@@ -135,6 +153,35 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
listener, err := net.Listen("tcp", a.listenAddress(address...))
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
// RunWith starts a custom server runner while keeping App-owned cleanup in the
|
||||
// same lifecycle. The runner receives the transport-neutral App handler and
|
||||
// owns transport/TLS settings; built-in transport selection is bypassed.
|
||||
func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) error {
|
||||
if runner == nil {
|
||||
return errors.Join(errors.New("go-omni: nil runner"), a.Close())
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, a.listenAddress(address...)), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) listenAddress(address ...string) string {
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
@@ -142,17 +189,18 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
server := a.NewServer(listenOn)
|
||||
listener, err := net.Listen("tcp", server.Addr)
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return a.serveContext(ctx, server, listener)
|
||||
return listenOn
|
||||
}
|
||||
|
||||
func (a *App) serveContext(ctx context.Context, server *http.Server, listener net.Listener) error {
|
||||
serveErr := make(chan error, 1)
|
||||
go func() { serveErr <- server.Serve(listener) }()
|
||||
go func() {
|
||||
if server.TLSConfig != nil {
|
||||
serveErr <- server.ServeTLS(listener, "", "")
|
||||
} else {
|
||||
serveErr <- server.Serve(listener)
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-serveErr:
|
||||
|
||||
@@ -21,6 +21,9 @@ func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.
|
||||
}
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
if len(a.config.TrustedProxies) > 0 {
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
|
||||
}
|
||||
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
|
||||
if err := wrapped(requestContext); err != nil {
|
||||
a.writeError(requestContext, err)
|
||||
|
||||
79
app/transport.go
Normal file
79
app/transport.go
Normal file
@@ -0,0 +1,79 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
h2transport "git.yuntuo.hk/k2safe/go-omni/transport/http2"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
// ServeContext serves a caller-created listener using the configured transport
|
||||
// and TLS. App takes ownership of the listener, including on startup failure,
|
||||
// and runs resource cleanup after the server has stopped.
|
||||
func (a *App) ServeContext(ctx context.Context, listener net.Listener) error {
|
||||
if listener == nil {
|
||||
return errors.Join(errors.New("go-omni: nil listener"), a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
_ = listener.Close()
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
_ = listener.Close()
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) prepareTransport() (func(context.Context, net.Listener) error, error) {
|
||||
if err := a.config.ValidateTransport(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shutdownTimeout := a.config.ShutdownTimeout
|
||||
if shutdownTimeout <= 0 {
|
||||
shutdownTimeout = 10 * time.Second
|
||||
}
|
||||
var tlsConfig *tls.Config
|
||||
if strings.TrimSpace(a.config.TLS.CertFile) != "" {
|
||||
certificate, err := tls.LoadX509KeyPair(a.config.TLS.CertFile, a.config.TLS.KeyFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("go-omni: load TLS certificate: %w", err)
|
||||
}
|
||||
tlsConfig = &tls.Config{Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
switch a.config.Transport {
|
||||
case config.TransportNative:
|
||||
server := native.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
if tlsConfig != nil {
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
}
|
||||
return server.Serve, nil
|
||||
case config.TransportHTTP2:
|
||||
server := h2transport.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
default:
|
||||
server := a.NewServer("")
|
||||
server.TLSConfig = tlsConfig
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return a.serveContext(ctx, server, listener)
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
322
app/transport_test.go
Normal file
322
app/transport_test.go
Normal file
@@ -0,0 +1,322 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
func TestConfiguredTransportLifecycle(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
for _, timeout := range []bool{false, true} {
|
||||
name := transport + "/drain"
|
||||
if timeout {
|
||||
name = transport + "/timeout"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
limit := 3 * time.Second
|
||||
if timeout {
|
||||
limit = 80 * time.Millisecond
|
||||
}
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(limit))
|
||||
started, release, canceled := make(chan struct{}), make(chan struct{}), make(chan struct{})
|
||||
var once sync.Once
|
||||
unblock := func() { once.Do(func() { close(release) }) }
|
||||
defer unblock()
|
||||
var cleaned atomic.Int32
|
||||
application.OnShutdown(func() error { cleaned.Add(1); return nil })
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
select {
|
||||
case <-ctx.Context().Done():
|
||||
close(canceled)
|
||||
return ctx.Context().Err()
|
||||
case <-release:
|
||||
return ctx.OK("finished")
|
||||
}
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wrapped := &closingListener{Listener: listener, closed: make(chan struct{})}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
defer listener.Close()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, wrapped) }()
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
defer response.Body.Close()
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
err = readErr
|
||||
if response.StatusCode != 200 || !strings.Contains(string(body), "finished") {
|
||||
err = errors.New("drained response was lost")
|
||||
}
|
||||
}
|
||||
requestDone <- err
|
||||
}()
|
||||
await(t, started)
|
||||
began := time.Now()
|
||||
cancel()
|
||||
await(t, wrapped.closed)
|
||||
if !timeout {
|
||||
select {
|
||||
case <-canceled:
|
||||
t.Fatal("active request canceled before drain")
|
||||
default:
|
||||
}
|
||||
if cleaned.Load() != 0 {
|
||||
t.Fatal("cleanup preceded active request")
|
||||
}
|
||||
unblock()
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
if timeout {
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(began) < limit {
|
||||
t.Fatalf("timeout ignored: %v after %v", err, time.Since(began))
|
||||
}
|
||||
await(t, canceled)
|
||||
} else if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("server failed to stop")
|
||||
}
|
||||
select {
|
||||
case err := <-requestDone:
|
||||
if !timeout && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("client failed to finish")
|
||||
}
|
||||
_ = application.Close()
|
||||
if cleaned.Load() != 1 {
|
||||
t.Fatalf("cleanup count %d", cleaned.Load())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredStartupFailureClosesResources(t *testing.T) {
|
||||
occupied, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer occupied.Close()
|
||||
for _, test := range []struct {
|
||||
name, errorText string
|
||||
options []config.Option
|
||||
}{
|
||||
{"unknown", "transport", []config.Option{config.WithTransport("unknown")}},
|
||||
{"http2WithoutTLS", "tls", []config.Option{config.WithTransport("http2")}},
|
||||
{"partialTLS", "tls", []config.Option{config.WithTLS("missing.crt", "")}},
|
||||
{"nethttpCertificate", "load TLS certificate", []config.Option{config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"nativeCertificate", "load TLS certificate", []config.Option{config.WithTransport("native"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"http2Certificate", "load TLS certificate", []config.Option{config.WithTransport("http2"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
application := New(test.options...)
|
||||
cleanupError := errors.New("cleanup failure")
|
||||
count := 0
|
||||
application.OnShutdown(func() error { count++; return cleanupError })
|
||||
// The occupied address must not mask configuration/certificate errors.
|
||||
err := application.RunContext(context.Background(), occupied.Addr().String())
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorText) || !errors.Is(err, cleanupError) {
|
||||
t.Fatalf("startup: %v", err)
|
||||
}
|
||||
_ = application.Close()
|
||||
if count != 1 {
|
||||
t.Fatalf("cleanup count %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("unknown"))
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
err = application.ServeContext(ctx, listener)
|
||||
cancel()
|
||||
if canceled && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !canceled && err == nil {
|
||||
t.Fatal("invalid transport accepted")
|
||||
}
|
||||
if _, err := listener.Accept(); !errors.Is(err, net.ErrClosed) {
|
||||
t.Fatalf("listener leaked: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunContextConfiguredTLS(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
reserved, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
address := reserved.Addr().String()
|
||||
reserved.Close()
|
||||
settings := config.Default()
|
||||
settings.Address, settings.Transport = address, transport
|
||||
settings.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
application := New(config.WithConfig(settings))
|
||||
application.GET("/health", func(ctx *httpx.Context) error { return ctx.OK("ok") })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.RunContext(ctx) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("RunContext failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
response, err := client.Get("https://" + address + "/health")
|
||||
if err == nil {
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || !strings.Contains(string(body), `"data":"ok"`) {
|
||||
t.Fatalf("response %s: %v", body, readErr)
|
||||
}
|
||||
want := "h2"
|
||||
if transport == config.TransportNative {
|
||||
want = "http/1.1"
|
||||
}
|
||||
if response.TLS.NegotiatedProtocol != want || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatalf("TLS negotiation: %+v", response.TLS)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type runnerFunc func(context.Context, http.Handler, string) error
|
||||
|
||||
func (f runnerFunc) Run(ctx context.Context, handler http.Handler, address string) error {
|
||||
return f(ctx, handler, address)
|
||||
}
|
||||
|
||||
func TestRunWithRemainsIndependentOfBuiltInTransport(t *testing.T) {
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("custom"), config.WithTLS("missing", "missing"))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
calls, cleanups := 0, 0
|
||||
application.OnShutdown(func() error { cleanups++; return nil })
|
||||
err := application.RunWith(ctx, runnerFunc(func(_ context.Context, handler http.Handler, address string) error {
|
||||
calls++
|
||||
if handler != application || address != "127.0.0.1:1234" {
|
||||
t.Fatal("custom runner arguments changed")
|
||||
}
|
||||
return nil
|
||||
}), "127.0.0.1:1234")
|
||||
cancel()
|
||||
wantCalls := 1
|
||||
if canceled {
|
||||
wantCalls = 0
|
||||
}
|
||||
if err != nil || calls != wantCalls || cleanups != 1 {
|
||||
t.Fatalf("RunWith: %v calls=%d cleanups=%d", err, calls, cleanups)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond))
|
||||
started := make(chan context.Context, 1)
|
||||
release, finished := make(chan struct{}), make(chan struct{})
|
||||
application.GET("/blocked", func(ctx *httpx.Context) error {
|
||||
defer close(finished)
|
||||
started <- ctx.Context()
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan struct{})
|
||||
go func() {
|
||||
defer close(requestDone)
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/blocked")
|
||||
if err == nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
}()
|
||||
var requestContext context.Context
|
||||
select {
|
||||
case requestContext = <-started:
|
||||
case <-time.After(3 * time.Second):
|
||||
close(release)
|
||||
t.Fatal("handler did not start")
|
||||
}
|
||||
defer func() { close(release); await(t, finished) }()
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("handler defeated shutdown deadline")
|
||||
}
|
||||
// net/http's HTTP/2 streams observe the closed socket asynchronously.
|
||||
await(t, requestContext.Done())
|
||||
await(t, requestDone)
|
||||
})
|
||||
}
|
||||
}
|
||||
43
app_test.go
43
app_test.go
@@ -8,12 +8,10 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func TestAppReturnsStandardResponse(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/health", func(ctx *Context) error {
|
||||
return ctx.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
@@ -41,7 +39,7 @@ func TestAppReturnsStandardResponse(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppReturnsAPIError(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/users/:id", func(ctx *Context) error {
|
||||
return NotFound("USER_NOT_FOUND", "user was not found")
|
||||
})
|
||||
@@ -63,7 +61,7 @@ func TestAppReturnsAPIError(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppRecoversFromPanic(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/panic", func(ctx *Context) error {
|
||||
panic("boom")
|
||||
})
|
||||
@@ -85,7 +83,7 @@ func TestAppRecoversFromPanic(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppRunContextShutsDownGracefully(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode), WithShutdownTimeout(time.Second))
|
||||
app := New(WithShutdownTimeout(time.Second))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- app.RunContext(ctx, "127.0.0.1:0") }()
|
||||
@@ -102,7 +100,7 @@ func TestAppRunContextShutsDownGracefully(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppClosesResourcesInReverseOrder(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
var closed []string
|
||||
app.OnShutdown(func() error {
|
||||
closed = append(closed, "database")
|
||||
@@ -130,3 +128,34 @@ func TestAppUsesConfiguredAddressWhenNoOverrideIsProvided(t *testing.T) {
|
||||
t.Fatalf("expected configured address %q, got %q", settings.Address, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppConfigCopiesTrustedProxies(t *testing.T) {
|
||||
proxies := []string{"127.0.0.1"}
|
||||
app := New(WithTrustedProxies(proxies...))
|
||||
proxies[0] = "10.0.0.0/8"
|
||||
settings := app.Config()
|
||||
if len(settings.TrustedProxies) != 1 || settings.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatalf("trusted proxy config was not copied: %+v", settings.TrustedProxies)
|
||||
}
|
||||
settings.TrustedProxies[0] = "192.0.2.0/24"
|
||||
if app.Config().TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("App.Config should return a copy")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppTrustedProxyClientIPReachesHandlers(t *testing.T) {
|
||||
app := New(WithTrustedProxies("127.0.0.1"))
|
||||
app.GET("/client-ip", func(ctx *Context) error { return ctx.OK(ctx.ClientIP()) })
|
||||
request := httptest.NewRequest(http.MethodGet, "/client-ip", nil)
|
||||
request.RemoteAddr = "127.0.0.1:443"
|
||||
request.Header.Set("X-Forwarded-For", "198.51.100.7")
|
||||
recorder := httptest.NewRecorder()
|
||||
app.Engine().ServeHTTP(recorder, request)
|
||||
var response Response
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.Data != "198.51.100.7" {
|
||||
t.Fatalf("expected trusted proxy client IP, got %#v", response.Data)
|
||||
}
|
||||
}
|
||||
|
||||
106
audit/audit.go
Normal file
106
audit/audit.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package audit records explicitly selected business events without request bodies.
|
||||
package audit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
Time time.Time `json:"time"`
|
||||
ActorID string `json:"actorId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Action string `json:"action"`
|
||||
Resource string `json:"resource"`
|
||||
Outcome string `json:"outcome"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
Fields map[string]string `json:"fields,omitempty"`
|
||||
}
|
||||
type Sink interface {
|
||||
Write(context.Context, Event) error
|
||||
}
|
||||
type Recorder struct {
|
||||
sink Sink
|
||||
allowed map[string]bool
|
||||
}
|
||||
|
||||
func New(sink Sink, allowedFields ...string) (*Recorder, error) {
|
||||
if sink == nil || len(allowedFields) > 32 {
|
||||
return nil, errors.New("audit: sink and at most 32 fields required")
|
||||
}
|
||||
r := &Recorder{sink: sink, allowed: map[string]bool{}}
|
||||
for _, name := range allowedFields {
|
||||
if name == "" || len(name) > 64 {
|
||||
return nil, errors.New("audit: invalid field")
|
||||
}
|
||||
r.allowed[name] = true
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
func (r *Recorder) Record(ctx context.Context, event Event) error {
|
||||
if ctx == nil {
|
||||
return errors.New("audit: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
|
||||
return errors.New("audit: actor/action/resource and valid outcome required")
|
||||
}
|
||||
for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
|
||||
if len(value) > 512 {
|
||||
return errors.New("audit: event field too large")
|
||||
}
|
||||
}
|
||||
event.Time = time.Now().UTC()
|
||||
event.RequestID = logx.RequestID(ctx)
|
||||
event.TraceID = logx.TraceID(ctx)
|
||||
filtered := map[string]string{}
|
||||
for name, value := range event.Fields {
|
||||
if r.allowed[name] {
|
||||
if len(value) > 1024 {
|
||||
return errors.New("audit: attribute too large")
|
||||
}
|
||||
filtered[name] = value
|
||||
}
|
||||
}
|
||||
event.Fields = filtered
|
||||
return r.sink.Write(ctx, event)
|
||||
}
|
||||
|
||||
// JSONSink borrows its writer and serializes complete JSON lines. Write errors
|
||||
// are returned; callers choose whether a failed audit must reject an operation.
|
||||
type JSONSink struct {
|
||||
mu sync.Mutex
|
||||
writer io.Writer
|
||||
}
|
||||
|
||||
func NewJSONSink(writer io.Writer) (*JSONSink, error) {
|
||||
if writer == nil {
|
||||
return nil, errors.New("audit: writer required")
|
||||
}
|
||||
return &JSONSink{writer: writer}, nil
|
||||
}
|
||||
func (s *JSONSink) Write(ctx context.Context, event Event) error {
|
||||
data, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.writer.Write(data)
|
||||
if err == nil && n != len(data) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
return err
|
||||
}
|
||||
50
audit/audit_test.go
Normal file
50
audit/audit_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package audit
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type failedWriter struct{}
|
||||
|
||||
func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
|
||||
func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
sink, _ := NewJSONSink(&buf)
|
||||
recorder, _ := New(sink, "status")
|
||||
ctx := logx.WithRequestID(context.Background(), "r1")
|
||||
ctx = logx.WithTraceID(ctx, "t1")
|
||||
event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := recorder.Record(ctx, event); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
decoder := json.NewDecoder(&buf)
|
||||
for i := 0; i < 20; i++ {
|
||||
var got Event
|
||||
if err := decoder.Decode(&got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
bad, _ := NewJSONSink(failedWriter{})
|
||||
recorder, _ = New(bad)
|
||||
if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
105
auth/adapters_test.go
Normal file
105
auth/adapters_test.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestJWTValidationAndRotation(t *testing.T) {
|
||||
pub, private, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
|
||||
sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
|
||||
t.Helper()
|
||||
token := jwt.NewWithClaims(method, c)
|
||||
token.Header["kid"] = kid
|
||||
raw, err := token.SignedString(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
|
||||
// Constructor clones caller-owned key bytes.
|
||||
pub[0] ^= 1
|
||||
p, err := j.Authenticate(context.Background(), raw)
|
||||
if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
|
||||
t.Fatalf("%+v %v", p, err)
|
||||
}
|
||||
for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
bad := claims
|
||||
change(&bad)
|
||||
if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
|
||||
if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestSessionsRevokeAndIsolation(t *testing.T) {
|
||||
mem, _ := cache.NewMemory(cache.MemoryConfig{})
|
||||
defer mem.Close()
|
||||
s, err := NewSessions(mem, "session:", time.Minute)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, _ := s.key(token)
|
||||
if strings.Contains(key, token) {
|
||||
t.Fatal("plaintext session secret stored")
|
||||
}
|
||||
p, err := s.Authenticate(ctx, token)
|
||||
if err != nil || p.UserID != "u1" {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
p.Roles[0] = "admin"
|
||||
p, _ = s.Authenticate(ctx, token)
|
||||
if p.HasRole("admin") {
|
||||
t.Fatal("principal alias")
|
||||
}
|
||||
if err = s.Revoke(ctx, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -21,9 +21,10 @@ var (
|
||||
// Principal is the authenticated identity made available to business handlers.
|
||||
// Claims are optional provider-specific values and should not be required by core business code.
|
||||
type Principal struct {
|
||||
UserID string
|
||||
Roles []string
|
||||
Claims map[string]any
|
||||
UserID string `json:"userId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
Claims map[string]any `json:"claims,omitempty"`
|
||||
}
|
||||
|
||||
func (p Principal) HasRole(role string) bool {
|
||||
|
||||
@@ -9,11 +9,10 @@ import (
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func TestRequireAuthAndRole(t *testing.T) {
|
||||
app := omni.New(omni.WithMode(gin.TestMode))
|
||||
app := omni.New()
|
||||
authenticator := auth.StaticAuthenticator{"good-token": {UserID: "user-1", Roles: []string{"user"}}}
|
||||
private := app.Group("/private", auth.RequireAuth(authenticator))
|
||||
private.GET("/me", func(ctx *omni.Context) error {
|
||||
@@ -62,7 +61,7 @@ func TestRequireAuthAndRole(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestOptionalAuth(t *testing.T) {
|
||||
app := omni.New(omni.WithMode(gin.TestMode))
|
||||
app := omni.New()
|
||||
app.GET("/public", func(ctx *omni.Context) error {
|
||||
principal, authenticated := auth.FromContext(ctx)
|
||||
return ctx.OK(map[string]any{"authenticated": authenticated, "userId": principal.UserID})
|
||||
|
||||
127
auth/jwt.go
Normal file
127
auth/jwt.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rsa"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
)
|
||||
|
||||
// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
|
||||
// Keys come from server configuration, never a token's jku/x5u URL.
|
||||
type JWTKey struct {
|
||||
Algorithm string
|
||||
PublicKey any
|
||||
}
|
||||
type JWTConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
Audience string `json:"audience"`
|
||||
Leeway time.Duration `json:"leeway"`
|
||||
Keys map[string]JWTKey `json:"-"`
|
||||
}
|
||||
type JWT struct {
|
||||
config JWTConfig
|
||||
mu sync.RWMutex
|
||||
keys map[string]JWTKey
|
||||
}
|
||||
type JWTClaims struct {
|
||||
jwt.RegisteredClaims
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
}
|
||||
|
||||
func NewJWT(cfg JWTConfig) (*JWT, error) {
|
||||
if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
|
||||
return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
|
||||
}
|
||||
j := &JWT{config: cfg}
|
||||
j.config.Keys = nil
|
||||
if err := j.ReplaceKeys(cfg.Keys); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return j, nil
|
||||
}
|
||||
|
||||
// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
|
||||
// their tokens expire if a rotation should preserve existing sessions.
|
||||
func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
|
||||
if len(keys) == 0 || len(keys) > 64 {
|
||||
return errors.New("auth: require 1..64 JWT keys")
|
||||
}
|
||||
copyKeys := make(map[string]JWTKey, len(keys))
|
||||
for id, key := range keys {
|
||||
if id == "" || len(id) > 128 {
|
||||
return errors.New("auth: invalid key ID")
|
||||
}
|
||||
switch key.Algorithm {
|
||||
case "RS256":
|
||||
k, ok := key.PublicKey.(*rsa.PublicKey)
|
||||
if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
|
||||
return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
|
||||
}
|
||||
case "EdDSA":
|
||||
k, ok := key.PublicKey.(ed25519.PublicKey)
|
||||
if !ok || len(k) != ed25519.PublicKeySize {
|
||||
return errors.New("auth: EdDSA requires an Ed25519 public key")
|
||||
}
|
||||
default:
|
||||
return errors.New("auth: supported algorithms are RS256 and EdDSA")
|
||||
}
|
||||
encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
copyKeys[id] = key
|
||||
}
|
||||
j.mu.Lock()
|
||||
j.keys = copyKeys
|
||||
j.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
|
||||
if ctx == nil {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
if len(raw) == 0 || len(raw) > 8192 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
claims := new(JWTClaims)
|
||||
token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
|
||||
id, ok := t.Header["kid"].(string)
|
||||
if !ok {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
j.mu.RLock()
|
||||
key, exists := j.keys[id]
|
||||
j.mu.RUnlock()
|
||||
if !exists || t.Method.Alg() != key.Algorithm {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
return key.PublicKey, nil
|
||||
}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
|
||||
if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
for _, role := range claims.Roles {
|
||||
if role == "" || len(role) > 128 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
|
||||
}
|
||||
34
auth/policy.go
Normal file
34
auth/policy.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type Policy interface {
|
||||
Authorize(context.Context, Principal, string, string) error
|
||||
}
|
||||
type PolicyFunc func(context.Context, Principal, string, string) error
|
||||
|
||||
func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
|
||||
return f(ctx, p, action, resource)
|
||||
}
|
||||
func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
p, ok := FromContext(c)
|
||||
if !ok {
|
||||
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||
}
|
||||
if policy == nil || action == "" || resource == nil {
|
||||
return apierrors.Internal(errors.New("auth: incomplete permission policy"))
|
||||
}
|
||||
if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
|
||||
return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
87
auth/session.go
Normal file
87
auth/session.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
)
|
||||
|
||||
// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
|
||||
// The caller verifies login credentials before Issue; Store owns persistence.
|
||||
type Sessions struct {
|
||||
store cache.Store
|
||||
prefix string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
|
||||
if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
|
||||
return nil, errors.New("auth: invalid session configuration")
|
||||
}
|
||||
return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
|
||||
}
|
||||
func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
|
||||
if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
data, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(data) > 16384 {
|
||||
return "", errors.New("auth: session principal too large")
|
||||
}
|
||||
secret := make([]byte, 32)
|
||||
if _, err := rand.Read(secret); err != nil {
|
||||
return "", err
|
||||
}
|
||||
token := base64.RawURLEncoding.EncodeToString(secret)
|
||||
key, _ := s.key(token)
|
||||
if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
func (s *Sessions) key(token string) (string, error) {
|
||||
if len(token) != 43 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
b, err := base64.RawURLEncoding.DecodeString(token)
|
||||
if err != nil || len(b) != 32 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
sum := sha256.Sum256(b)
|
||||
return s.prefix + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
data, err := s.store.Get(ctx, key)
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
var p Principal
|
||||
if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
func (s *Sessions) Revoke(ctx context.Context, token string) error {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Delete(ctx, key)
|
||||
}
|
||||
135
cache/cache.go
vendored
Normal file
135
cache/cache.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
// Package cache provides replaceable byte-oriented caches and bounded local
|
||||
// request coalescing. Storage errors are never silently treated as misses.
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrMiss = errors.New("cache: key not found")
|
||||
ErrClosed = errors.New("cache: closed")
|
||||
ErrTooLarge = errors.New("cache: entry exceeds capacity")
|
||||
ErrBusy = errors.New("cache: too many concurrent loads")
|
||||
)
|
||||
|
||||
// Store implementations copy values and accept ttl=0 for no expiration.
|
||||
// Negative TTLs are invalid. Delete is successful for missing keys.
|
||||
type Store interface {
|
||||
Get(context.Context, string) ([]byte, error)
|
||||
Set(context.Context, string, []byte, time.Duration) error
|
||||
Delete(context.Context, ...string) error
|
||||
}
|
||||
|
||||
func Check(ctx context.Context, key string, ttl time.Duration) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if key == "" || len(key) > 1024 {
|
||||
return errors.New("cache: key must contain 1..1024 bytes")
|
||||
}
|
||||
if ttl < 0 {
|
||||
return errors.New("cache: negative TTL")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type LoaderConfig struct {
|
||||
MaxLoads int `json:"maxLoads"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
}
|
||||
|
||||
type flight struct {
|
||||
done chan struct{}
|
||||
value []byte
|
||||
err error
|
||||
}
|
||||
|
||||
// Loader coalesces concurrent misses for the same key within this process.
|
||||
// Caller cancellation stops waiting without canceling other callers' load.
|
||||
// Load functions must observe their context, which has a bounded deadline.
|
||||
type Loader struct {
|
||||
store Store
|
||||
config LoaderConfig
|
||||
mu sync.Mutex
|
||||
flights map[string]*flight
|
||||
}
|
||||
|
||||
func NewLoader(store Store, cfg LoaderConfig) (*Loader, error) {
|
||||
if store == nil || cfg.MaxLoads < 0 || cfg.Timeout < 0 {
|
||||
return nil, errors.New("cache: invalid loader configuration")
|
||||
}
|
||||
if cfg.MaxLoads == 0 {
|
||||
cfg.MaxLoads = 128
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
return &Loader{store: store, config: cfg, flights: make(map[string]*flight)}, nil
|
||||
}
|
||||
|
||||
func (l *Loader) GetOrLoad(ctx context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error)) ([]byte, error) {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if load == nil {
|
||||
return nil, errors.New("cache: nil loader")
|
||||
}
|
||||
value, err := l.store.Get(ctx, key)
|
||||
if !errors.Is(err, ErrMiss) {
|
||||
return value, err
|
||||
}
|
||||
l.mu.Lock()
|
||||
f := l.flights[key]
|
||||
if f == nil {
|
||||
if len(l.flights) >= l.config.MaxLoads {
|
||||
l.mu.Unlock()
|
||||
return nil, ErrBusy
|
||||
}
|
||||
f = &flight{done: make(chan struct{})}
|
||||
l.flights[key] = f
|
||||
go l.run(context.WithoutCancel(ctx), key, ttl, load, f)
|
||||
}
|
||||
l.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-f.done:
|
||||
return append([]byte(nil), f.value...), f.err
|
||||
}
|
||||
}
|
||||
|
||||
func (l *Loader) run(parent context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error), f *flight) {
|
||||
ctx, cancel := context.WithTimeout(parent, l.config.Timeout)
|
||||
defer cancel()
|
||||
defer func() {
|
||||
if value := recover(); value != nil {
|
||||
f.err = fmt.Errorf("cache: loader panic: %v", value)
|
||||
}
|
||||
l.mu.Lock()
|
||||
delete(l.flights, key)
|
||||
close(f.done)
|
||||
l.mu.Unlock()
|
||||
}()
|
||||
// Another request may have populated the key between the first Get and
|
||||
// flight acquisition. Recheck inside the coalesced load.
|
||||
f.value, f.err = l.store.Get(ctx, key)
|
||||
if !errors.Is(f.err, ErrMiss) {
|
||||
return
|
||||
}
|
||||
f.value, f.err = load(ctx)
|
||||
if f.err == nil {
|
||||
f.err = ctx.Err()
|
||||
}
|
||||
if f.err == nil {
|
||||
f.err = l.store.Set(ctx, key, f.value, ttl)
|
||||
}
|
||||
f.value = append([]byte(nil), f.value...)
|
||||
}
|
||||
118
cache/cache_test.go
vendored
Normal file
118
cache/cache_test.go
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestMemoryOwnershipCapacityExpiryAndClose(t *testing.T) {
|
||||
m, err := NewMemory(MemoryConfig{MaxEntries: 2, MaxBytes: 10})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now()
|
||||
m.now = func() time.Time { return now }
|
||||
ctx := context.Background()
|
||||
value := []byte("aa")
|
||||
if err := m.Set(ctx, "a", value, time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value[0] = 'x'
|
||||
got, _ := m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("retained caller memory")
|
||||
}
|
||||
got[0] = 'z'
|
||||
got, _ = m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("returned mutable cache memory")
|
||||
}
|
||||
m.Set(ctx, "b", []byte("bb"), 0)
|
||||
m.Get(ctx, "a")
|
||||
m.Set(ctx, "c", []byte("cc"), 0)
|
||||
if _, err := m.Get(ctx, "b"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("LRU eviction failed")
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if _, err := m.Get(ctx, "a"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("expiration boundary ignored")
|
||||
}
|
||||
if err := m.Set(ctx, "large", []byte("123456"), 0); !errors.Is(err, ErrTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := m.Get(ctx, "c"); err != nil {
|
||||
t.Fatal("oversized write evicted valid data")
|
||||
}
|
||||
canceled, cancel := context.WithCancel(ctx)
|
||||
cancel()
|
||||
if err := m.Delete(canceled, "c"); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
m.Close()
|
||||
if _, err := m.Get(ctx, "c"); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderCoalescesAndSeparatesCallerCancellation(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
loader, _ := NewLoader(m, LoaderConfig{MaxLoads: 1, Timeout: time.Second})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
load := func(ctx context.Context) ([]byte, error) {
|
||||
calls.Add(1)
|
||||
close(started)
|
||||
select {
|
||||
case <-release:
|
||||
return []byte("value"), nil
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
first := make(chan error, 1)
|
||||
go func() { _, err := loader.GetOrLoad(ctx, "key", time.Minute, load); first <- err }()
|
||||
<-started
|
||||
second := make(chan error, 1)
|
||||
go func() {
|
||||
v, err := loader.GetOrLoad(context.Background(), "key", time.Minute, load)
|
||||
if err == nil && string(v) != "value" {
|
||||
err = errors.New("wrong value")
|
||||
}
|
||||
second <- err
|
||||
}()
|
||||
if _, err := loader.GetOrLoad(context.Background(), "other", 0, load); !errors.Is(err, ErrBusy) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel()
|
||||
if err := <-first; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
close(release)
|
||||
if err := <-second; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 1 {
|
||||
t.Fatalf("loads=%d", calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderPanicDeadlineAndStoreFailure(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
l, _ := NewLoader(m, LoaderConfig{Timeout: 20 * time.Millisecond})
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(context.Context) ([]byte, error) { panic("failure") }); err == nil {
|
||||
t.Fatal("panic escaped")
|
||||
}
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(ctx context.Context) ([]byte, error) { <-ctx.Done(); return nil, ctx.Err() }); !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
if _, err := l.GetOrLoad(context.Background(), "closed", 0, func(context.Context) ([]byte, error) { t.Fatal("store outage triggered load"); return nil, nil }); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
135
cache/memory.go
vendored
Normal file
135
cache/memory.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type MemoryConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"` // Sum of key and value bytes, excluding Go object overhead.
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
key string
|
||||
value []byte
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// Memory is an LRU cache with lazy expiration and no background goroutines.
|
||||
type Memory struct {
|
||||
mu sync.Mutex
|
||||
config MemoryConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
bytes int
|
||||
closed bool
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewMemory(cfg MemoryConfig) (*Memory, error) {
|
||||
if cfg.MaxEntries < 0 || cfg.MaxBytes < 0 {
|
||||
return nil, errors.New("cache: negative capacity")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 4096
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 32 << 20
|
||||
}
|
||||
return &Memory{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (m *Memory) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
e := m.items[key]
|
||||
if e == nil {
|
||||
return nil, ErrMiss
|
||||
}
|
||||
item := e.Value.(*entry)
|
||||
if !item.expires.IsZero() && !m.now().Before(item.expires) {
|
||||
m.remove(e)
|
||||
return nil, ErrMiss
|
||||
}
|
||||
m.lru.MoveToFront(e)
|
||||
return append([]byte(nil), item.value...), nil
|
||||
}
|
||||
|
||||
func (m *Memory) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(value) > m.config.MaxBytes-len(key) {
|
||||
return ErrTooLarge
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
for len(m.items) >= m.config.MaxEntries || m.bytes+len(key)+len(value) > m.config.MaxBytes {
|
||||
m.remove(m.lru.Back())
|
||||
}
|
||||
item := &entry{key: key, value: append([]byte(nil), value...)}
|
||||
if ttl > 0 {
|
||||
item.expires = m.now().Add(ttl)
|
||||
}
|
||||
m.items[key] = m.lru.PushFront(item)
|
||||
m.bytes += len(key) + len(value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, key := range keys {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
for _, key := range keys {
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) remove(e *list.Element) {
|
||||
item := e.Value.(*entry)
|
||||
m.bytes -= len(item.key) + len(item.value)
|
||||
delete(m.items, item.key)
|
||||
m.lru.Remove(e)
|
||||
}
|
||||
|
||||
func (m *Memory) Close() error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.closed = true
|
||||
clear(m.items)
|
||||
m.lru.Init()
|
||||
m.bytes = 0
|
||||
return nil
|
||||
}
|
||||
68
cache/redis/redis.go
vendored
Normal file
68
cache/redis/redis.go
vendored
Normal file
@@ -0,0 +1,68 @@
|
||||
// Package redis adapts go-redis's pooled client to cache.Store.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
// Store borrows the client; its owner must close the client after users stop.
|
||||
type Store struct {
|
||||
client goredis.Cmdable
|
||||
prefix string
|
||||
}
|
||||
|
||||
func New(client goredis.Cmdable, prefix string) (*Store, error) {
|
||||
if client == nil || len(prefix) > 1024 || strings.ContainsAny(prefix, "\r\n") {
|
||||
return nil, errors.New("cache/redis: invalid client or prefix")
|
||||
}
|
||||
return &Store{client: client, prefix: prefix}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value, err := s.client.Get(ctx, s.prefix+key).Bytes()
|
||||
if errors.Is(err, goredis.Nil) {
|
||||
return nil, cache.ErrMiss
|
||||
}
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (s *Store) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := cache.Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
// Redis expiration precision is milliseconds; round up, never truncate a
|
||||
// positive duration to an unbounded lifetime.
|
||||
if ttl > 0 && ttl < time.Millisecond {
|
||||
ttl = time.Millisecond
|
||||
}
|
||||
return s.client.Set(ctx, s.prefix+key, value, ttl).Err()
|
||||
}
|
||||
|
||||
func (s *Store) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache/redis: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
names := make([]string, len(keys))
|
||||
for i, key := range keys {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
names[i] = s.prefix + key
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return nil
|
||||
}
|
||||
return s.client.Del(ctx, names...).Err()
|
||||
}
|
||||
125
cache/redis/redis_test.go
vendored
Normal file
125
cache/redis/redis_test.go
vendored
Normal file
@@ -0,0 +1,125 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
type fakeClient struct {
|
||||
goredis.Cmdable
|
||||
key string
|
||||
ttl time.Duration
|
||||
failure error
|
||||
}
|
||||
|
||||
func (f *fakeClient) Get(ctx context.Context, key string) *goredis.StringCmd {
|
||||
f.key = key
|
||||
return goredis.NewStringResult("", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Set(ctx context.Context, key string, value interface{}, ttl time.Duration) *goredis.StatusCmd {
|
||||
f.key, f.ttl = key, ttl
|
||||
return goredis.NewStatusResult("OK", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Del(ctx context.Context, keys ...string) *goredis.IntCmd {
|
||||
f.key = keys[0]
|
||||
return goredis.NewIntResult(1, f.failure)
|
||||
}
|
||||
|
||||
func TestAdapterPreservesErrorsAndExpiration(t *testing.T) {
|
||||
f := &fakeClient{failure: goredis.Nil}
|
||||
store, _ := New(f, "app:")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, cache.ErrMiss) || f.key != "app:key" {
|
||||
t.Fatal(err, f.key)
|
||||
}
|
||||
f.failure = errors.New("unavailable")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, f.failure) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.failure = nil
|
||||
if err := store.Set(context.Background(), "key", []byte("value"), time.Nanosecond); err != nil || f.ttl != time.Millisecond {
|
||||
t.Fatal(err, f.ttl)
|
||||
}
|
||||
if err := store.Set(context.Background(), "key", nil, -time.Second); err == nil {
|
||||
t.Fatal("negative TTL accepted")
|
||||
}
|
||||
if err := store.Delete(context.Background(), "key"); err != nil || f.key != "app:key" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("set GO_OMNI_REDIS_TEST_ADDR to an isolated test Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 2, MaxRetries: -1, ContextTimeoutEnabled: true, DialTimeout: time.Second, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.Ping(ctx).Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prefix := fmt.Sprintf("goOmniTest:%d:", time.Now().UnixNano())
|
||||
store, _ := New(client, prefix)
|
||||
defer store.Delete(context.Background(), "key", "forever", "list", "canceled")
|
||||
if err := store.Set(ctx, "key", []byte{0, 1, 255}, 80*time.Millisecond); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v, err := store.Get(ctx, "key")
|
||||
if err != nil || len(v) != 3 || v[2] != 255 {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
_, err := store.Get(ctx, "key")
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
break
|
||||
}
|
||||
if err != nil || time.Now().After(deadline) {
|
||||
t.Fatal("TTL failed", err)
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
if err := store.Set(ctx, "forever", []byte("ok"), 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ttl := client.PTTL(ctx, prefix+"forever").Val(); ttl != -1 {
|
||||
t.Fatalf("unexpected persistent TTL %v", ttl)
|
||||
}
|
||||
if err := store.Delete(ctx, "forever"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "forever"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.RPush(ctx, prefix+"list", "value").Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "list"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("Redis error treated as miss", err)
|
||||
}
|
||||
canceled, stop := context.WithCancel(ctx)
|
||||
stop()
|
||||
if err := store.Set(canceled, "canceled", []byte("value"), 0); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "canceled"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("canceled write reached Redis", err)
|
||||
}
|
||||
if err := store.Delete(ctx, "list"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "key"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("closed client treated as miss", err)
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,16 @@
|
||||
# go-omni CLI
|
||||
|
||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||
该命令尚未包含在下方历史标签中;当前源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||
|
||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.3.1
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -16,7 +23,7 @@ go run ./cmd/server
|
||||
需要选择其他已发布版本时,使用完整版本号:
|
||||
|
||||
```sh
|
||||
go-omni new --framework-version v0.3.1 --module example.com/billing ./billing
|
||||
go-omni new --framework-version v0.6.0-preview --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
`--framework-version` 禁用自动本地替换;`main`、`latest` 等浮动引用不接受为固定版本。
|
||||
@@ -48,8 +55,8 @@ MySQL 项目额外包含 `schema.sql`、MySQL Repository 和可选的数据库
|
||||
发布验收使用 Python 3 标准库脚本,不需要额外 Python 依赖:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyScaffold.py --version v0.3.1
|
||||
python3 scripts/verifyDatabase.py --version v0.3.1
|
||||
python3 scripts/verifyScaffold.py --version v0.6.0-preview
|
||||
python3 scripts/verifyDatabase.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
脚本在源码目录外安装远程 CLI、生成项目,确认固定版本且没有 `replace`,再下载依赖、
|
||||
@@ -65,3 +72,20 @@ python3 scripts/verifyDatabase.py --cli /tmp/go-omni --framework-dir .
|
||||
```
|
||||
|
||||
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
||||
|
||||
## 显式迁移(main)
|
||||
|
||||
SQL 命名为 `0001_createUsers.up.sql`。先预览版本和 SHA-256 清单:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations
|
||||
```
|
||||
|
||||
部署步骤配置 `GO_OMNI_DB_DSN`,显式执行:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations --apply
|
||||
```
|
||||
|
||||
预览不连接数据库;应用启动不自动迁移。失败保留 dirty 状态,不提供自动 down/force。
|
||||
脚手架原有 `schema.sql` 不会自动变成有版本的迁移,已有库需要先确定基线。详见 [扩展模块](../../ADVANCED.md)。
|
||||
|
||||
@@ -17,6 +17,16 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "migrate":
|
||||
if err := runMigrate(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "add":
|
||||
if err := runAdd(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "new":
|
||||
if err := runNew(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni new:", err)
|
||||
@@ -103,8 +113,38 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||
fmt.Fprintln(os.Stderr, "")
|
||||
fmt.Fprintln(os.Stderr, "Usage:")
|
||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
|
||||
}
|
||||
|
||||
func runAdd(args []string) error {
|
||||
directory, name := ".", ""
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--dir":
|
||||
if index+1 >= len(args) {
|
||||
return errors.New("--dir requires a project directory")
|
||||
}
|
||||
index++
|
||||
directory = args[index]
|
||||
case strings.HasPrefix(arg, "--dir="):
|
||||
directory = strings.TrimPrefix(arg, "--dir=")
|
||||
case strings.HasPrefix(arg, "-"):
|
||||
return fmt.Errorf("unknown option %q", arg)
|
||||
case name == "":
|
||||
name = arg
|
||||
default:
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
}
|
||||
if err := scaffold.AddModule(directory, name); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created internal/%s; see its README.md for route registration\n", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -32,6 +32,30 @@ func TestNewRejectsEmptyFrameworkVersion(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleCLI(t *testing.T) {
|
||||
for _, equals := range []bool{false, true} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := runNew([]string{"--module", "example.com/billing", project}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
args := []string{"orders", "--dir", project}
|
||||
if equals {
|
||||
args = []string{"--dir=" + project, "orders"}
|
||||
}
|
||||
if err := runAdd(args); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "internal/orders/module.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{nil, {"orders", "--dir"}, {"orders", "--dir="}, {"orders", "extra"}, {"orders", "--unknown"}} {
|
||||
if err := runAdd(args); err == nil {
|
||||
t.Fatalf("accepted invalid arguments %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewMySQLDatabaseOptions(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--database", "mysql"}, {"--database=mysql"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
|
||||
60
cmd/go-omni/migrate.go
Normal file
60
cmd/go-omni/migrate.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func runMigrate(args []string) error {
|
||||
flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
directory := flags.String("dir", "migrations", "SQL directory")
|
||||
dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
|
||||
apply := flags.Bool("apply", false, "apply pending up migrations")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("migrate: unexpected argument")
|
||||
}
|
||||
_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !*apply {
|
||||
data, err := json.MarshalIndent(manifest, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
dsn := os.Getenv(*dsnEnv)
|
||||
if dsn == "" {
|
||||
return errors.New("migrate: DSN environment variable is empty")
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
version, dirty, err := runner.Version()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
|
||||
return nil
|
||||
}
|
||||
25
cmd/go-omni/migrate_test.go
Normal file
25
cmd/go-omni/migrate_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMigrationPreviewDoesNotConnect(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
|
||||
if err := runMigrate([]string{"--dir", dir}); err != nil {
|
||||
t.Fatal("preview tried to connect", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "")
|
||||
if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
|
||||
t.Fatal("missing DSN accepted")
|
||||
}
|
||||
if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
|
||||
t.Fatal("unsafe unknown flag accepted")
|
||||
}
|
||||
}
|
||||
109
cmd/go-omni/scaffold/addModule.go
Normal file
109
cmd/go-omni/scaffold/addModule.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
)
|
||||
|
||||
var moduleName = regexp.MustCompile(`^[a-z][a-zA-Z0-9]{0,47}$`)
|
||||
|
||||
// AddModule creates a complete, injectable memory-backed CRUD module. It never
|
||||
// edits existing routes, go.mod, migrations or files in another module.
|
||||
func AddModule(directory, name string) (err error) {
|
||||
if strings.TrimSpace(directory) == "" || !moduleName.MatchString(name) || token.Lookup(name).IsKeyword() || name == "main" || name == "init" || name == "internal" {
|
||||
return errors.New("module name must be a non-keyword lowerCamelCase Go identifier (1..48 characters)")
|
||||
}
|
||||
root, err := filepath.Abs(directory)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
root, err = filepath.EvalSymlinks(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("read project go.mod: %w", err)
|
||||
}
|
||||
manifest, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || manifest.Module == nil {
|
||||
return errors.New("project go.mod must declare a valid module")
|
||||
}
|
||||
found := false
|
||||
for _, dependency := range manifest.Require {
|
||||
if dependency.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("project must already require go-omni; create it with go-omni new")
|
||||
}
|
||||
module := manifest.Module.Mod.Path
|
||||
if strings.ContainsAny(module, "\"\\\r\n") {
|
||||
return errors.New("invalid project module path")
|
||||
}
|
||||
paths := []string{"module.go", "model/model.go", "repository/repository.go", "service/service.go", "handler/handler.go", "README.md"}
|
||||
rendered := make(map[string][]byte, len(paths))
|
||||
for _, path := range paths {
|
||||
source, readErr := templateFiles.ReadFile("moduleTemplates/" + strings.ReplaceAll(path, "/", "-") + ".tmpl")
|
||||
if readErr != nil {
|
||||
return readErr
|
||||
}
|
||||
tmpl, parseErr := template.New(path).Parse(string(source))
|
||||
if parseErr != nil {
|
||||
return parseErr
|
||||
}
|
||||
var output bytes.Buffer
|
||||
if err := tmpl.Execute(&output, struct{ Module, Name string }{module, name}); err != nil {
|
||||
return err
|
||||
}
|
||||
content := output.Bytes()
|
||||
if strings.HasSuffix(path, ".go") {
|
||||
content, err = format.Source(content)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rendered[path] = content
|
||||
}
|
||||
parent := filepath.Join(root, "internal")
|
||||
if info, statErr := os.Lstat(parent); statErr == nil {
|
||||
if !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return errors.New("internal must be a real directory, not a symlink")
|
||||
}
|
||||
} else if !errors.Is(statErr, os.ErrNotExist) {
|
||||
return statErr
|
||||
} else if err := os.Mkdir(parent, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
target := filepath.Join(parent, name)
|
||||
// Exclusive creation protects existing files, including empty directories.
|
||||
if err := os.Mkdir(target, 0755); err != nil {
|
||||
return fmt.Errorf("create module (existing modules are never overwritten): %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err != nil {
|
||||
_ = os.RemoveAll(target)
|
||||
}
|
||||
}()
|
||||
for _, path := range paths {
|
||||
file := filepath.Join(target, filepath.FromSlash(path))
|
||||
if err = os.MkdirAll(filepath.Dir(file), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = os.WriteFile(file, rendered[path], 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
@@ -0,0 +1,137 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func moduleProject(t *testing.T) string {
|
||||
t.Helper()
|
||||
root := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: root, Module: "example.com/billing", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
func TestAddModulePreservesProjectAndRejectsOverwrite(t *testing.T) {
|
||||
root := moduleProject(t)
|
||||
manifest, _ := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
main, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go"))
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, path := range []string{"module.go", "model/model.go", "handler/handler.go", "service/service.go", "repository/repository.go", "README.md"} {
|
||||
if _, err := os.Stat(filepath.Join(root, "internal/orderItems", path)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
marker := filepath.Join(root, "internal/orderItems/keep.txt")
|
||||
if err := os.WriteFile(marker, []byte("keep"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orderItems"); err == nil {
|
||||
t.Fatal("overwrote existing module")
|
||||
}
|
||||
if got, _ := os.ReadFile(marker); string(got) != "keep" {
|
||||
t.Fatal("existing file changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "go.mod")); string(got) != string(manifest) {
|
||||
t.Fatal("go.mod changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go")); string(got) != string(main) {
|
||||
t.Fatal("main changed")
|
||||
}
|
||||
for _, name := range []string{"../escape", "Order", "order_items", "func", "main", "init", "internal", "", strings.Repeat("a", 49)} {
|
||||
if err := AddModule(root, name); err == nil {
|
||||
t.Fatalf("accepted %q", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleRejectsForeignProjectAndSymlink(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.com/foreign\ngo 1.22\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted foreign project")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(root, "internal")); !os.IsNotExist(err) {
|
||||
t.Fatal("created files before validation")
|
||||
}
|
||||
root = moduleProject(t)
|
||||
if err := os.Rename(filepath.Join(root, "internal"), filepath.Join(root, "original")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(filepath.Join(root, "original"), filepath.Join(root, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted symlink parent")
|
||||
}
|
||||
}
|
||||
|
||||
// Opt-in because this compiles a separate consuming module and may need module
|
||||
// downloads. The Linux feature acceptance runner enables it explicitly.
|
||||
func TestAddModuleIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-project CRUD")
|
||||
}
|
||||
root := moduleProject(t)
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "internal/orderItems/module_test.go"), []byte(generatedModuleTest), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "./...")
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated project: %v\n%s", err, output)
|
||||
}
|
||||
}
|
||||
|
||||
const generatedModuleTest = `package orderItems_test
|
||||
import (
|
||||
"context"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"example.com/billing/internal/orderItems"
|
||||
)
|
||||
func TestGeneratedCRUD(t *testing.T) {
|
||||
app:=omni.New()
|
||||
repo:=orderItems.NewMemory()
|
||||
orderItems.Register(app,repo,func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if c.Header("Authorization")!="Bearer demo" {return omni.Unauthorized("DENIED","denied")};return next(c)
|
||||
}})
|
||||
request:=func(method,path,body string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type","application/json")
|
||||
if authorized {r.Header.Set("Authorization","Bearer demo")}
|
||||
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/orderItems","{\"name\":\"first\"}",false);w.Code!=401 {t.Fatal(w.Code)}
|
||||
cases:=[]struct{method,path,body string;status int;contains string}{
|
||||
{"POST","/orderItems","{\"name\":\" first \"}",201,"createdAt"},
|
||||
{"GET","/orderItems/1","",200,"first"},
|
||||
{"PUT","/orderItems/1","{\"name\":\"changed\"}",200,"changed"},
|
||||
{"GET","/orderItems","",200,"changed"},
|
||||
{"POST","/orderItems","{\"name\":\" \"}",400,"INVALID_NAME"},
|
||||
{"DELETE","/orderItems/1","",204,""},
|
||||
{"GET","/orderItems/1","",404,"NOT_FOUND"},
|
||||
}
|
||||
for _,tc:=range cases{w:=request(tc.method,tc.path,tc.body,true);if w.Code!=tc.status||!strings.Contains(w.Body.String(),tc.contains){t.Fatalf("%s %s: %d %s",tc.method,tc.path,w.Code,w.Body.String())}}
|
||||
ctx,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if _,err:=repo.List(ctx);err!=context.Canceled {t.Fatal(err)}
|
||||
}
|
||||
`
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
# {{.Name}}
|
||||
|
||||
在业务启动文件导入并注册:
|
||||
|
||||
```go
|
||||
import "{{.Module}}/internal/{{.Name}}"
|
||||
|
||||
{{.Name}}.Register(application, {{.Name}}.NewMemory())
|
||||
```
|
||||
|
||||
提供 `POST /{{.Name}}`、`GET /{{.Name}}`、`GET /{{.Name}}/:id`、`PUT /{{.Name}}/:id`、
|
||||
`DELETE /{{.Name}}/:id`。写入 JSON 为 `{"name":"example"}`;输出字段为 id、name、createdAt。
|
||||
代码按 Handler / Service / Repository / Model 分层,`Register` 可注入自己的 Repository 和认证中间件。
|
||||
内存示例不持久化、不提供分页;列表适合小规模演示。接实际业务时定义领域字段、分页、鉴权和数据库实现。
|
||||
生成器不会自动修改主入口、已有模块或数据库结构,也不会覆盖已有文件。
|
||||
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
@@ -0,0 +1,52 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"errors"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Handler struct { service *service.Service }
|
||||
type WriteRequest struct { Name string `json:"name" binding:"required,max=200"` }
|
||||
func New(service *service.Service) *Handler {return &Handler{service:service}}
|
||||
func (h *Handler) Register(routes *omni.RouterGroup) {
|
||||
routes.POST("",h.Create)
|
||||
routes.GET("",h.List)
|
||||
routes.GET("/:id",h.Get)
|
||||
routes.PUT("/:id",h.Update)
|
||||
routes.DELETE("/:id",h.Delete)
|
||||
}
|
||||
func (h *Handler) Create(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Create(c.Context(),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.Created(item)
|
||||
}
|
||||
func (h *Handler) Get(c *omni.Context) error {
|
||||
item,err:=h.service.Get(c.Context(),c.Param("id"))
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) List(c *omni.Context) error {
|
||||
items,err:=h.service.List(c.Context())
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(items)
|
||||
}
|
||||
func (h *Handler) Update(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Update(c.Context(),c.Param("id"),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) Delete(c *omni.Context) error {
|
||||
if err:=h.service.Delete(c.Context(),c.Param("id")); err!=nil {return publicError(err)}
|
||||
return c.NoContent()
|
||||
}
|
||||
func publicError(err error) error {
|
||||
if errors.Is(err,model.ErrNotFound) {return omni.NotFound("NOT_FOUND","record not found").WithCause(err)}
|
||||
if errors.Is(err,model.ErrInvalidName) {return omni.BadRequest("INVALID_NAME","name must contain 1..200 bytes").WithCause(err)}
|
||||
return err
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
var ErrNotFound = errors.New("{{.Name}}: record not found")
|
||||
var ErrInvalidName = errors.New("{{.Name}}: name must contain 1..200 bytes")
|
||||
|
||||
type Item struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
@@ -0,0 +1,19 @@
|
||||
package {{.Name}}
|
||||
|
||||
import (
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/handler"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Repository = repository.Repository
|
||||
|
||||
func NewMemory() Repository { return repository.NewMemory() }
|
||||
|
||||
// Register wires this module with caller-owned storage. Pass auth middleware
|
||||
// here to apply it to every route before module-specific middleware.
|
||||
func Register(app *omni.App, repo Repository, middleware ...omni.Middleware) {
|
||||
if repo == nil { panic("{{.Name}}: repository is required") }
|
||||
handler.New(service.New(repo)).Register(app.Group("/{{.Name}}", middleware...))
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
)
|
||||
|
||||
type Repository interface {
|
||||
Create(context.Context,string) (model.Item,error)
|
||||
Get(context.Context,string) (model.Item,error)
|
||||
List(context.Context) ([]model.Item,error)
|
||||
Update(context.Context,string,string) (model.Item,error)
|
||||
Delete(context.Context,string) error
|
||||
}
|
||||
|
||||
// Memory is demonstration storage; data and IDs reset with the process.
|
||||
type Memory struct { mu sync.Mutex; next uint64; items map[string]model.Item }
|
||||
func NewMemory() *Memory { return &Memory{items:make(map[string]model.Item)} }
|
||||
func (m *Memory) Create(ctx context.Context, name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
m.next++
|
||||
item:=model.Item{ID:strconv.FormatUint(m.next,10),Name:name,CreatedAt:time.Now().UTC()}
|
||||
m.items[item.ID]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Get(ctx context.Context,id string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) List(ctx context.Context) ([]model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return nil,err}
|
||||
items:=make([]model.Item,0,len(m.items))
|
||||
for _,item:=range m.items {items=append(items,item)}
|
||||
sort.Slice(items,func(i,j int)bool{return items[i].ID<items[j].ID})
|
||||
return items,nil
|
||||
}
|
||||
func (m *Memory) Update(ctx context.Context,id,name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
item.Name=name; m.items[id]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Delete(ctx context.Context,id string) error {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return err}
|
||||
if _,ok:=m.items[id]; !ok {return model.ErrNotFound}
|
||||
delete(m.items,id)
|
||||
return nil
|
||||
}
|
||||
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
@@ -0,0 +1,27 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
)
|
||||
|
||||
type Service struct { repo repository.Repository }
|
||||
func New(repo repository.Repository) *Service { return &Service{repo:repo} }
|
||||
func name(value string) (string,error) {
|
||||
value=strings.TrimSpace(value)
|
||||
if len(value)==0 || len(value)>200 {return "",model.ErrInvalidName}
|
||||
return value,nil
|
||||
}
|
||||
func (s *Service) Create(ctx context.Context,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Create(ctx,value)
|
||||
}
|
||||
func (s *Service) Get(ctx context.Context,id string) (model.Item,error) { return s.repo.Get(ctx,id) }
|
||||
func (s *Service) List(ctx context.Context) ([]model.Item,error) { return s.repo.List(ctx) }
|
||||
func (s *Service) Update(ctx context.Context,id,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Update(ctx,id,value)
|
||||
}
|
||||
func (s *Service) Delete(ctx context.Context,id string) error {return s.repo.Delete(ctx,id)}
|
||||
@@ -14,7 +14,7 @@ import (
|
||||
"text/template"
|
||||
)
|
||||
|
||||
//go:embed templates
|
||||
//go:embed templates moduleTemplates
|
||||
var templateFiles embed.FS
|
||||
|
||||
// Options controls project generation.
|
||||
|
||||
@@ -47,6 +47,11 @@ go build -o server ./cmd/server
|
||||
```
|
||||
|
||||
`GO_OMNI_SHUTDOWN_TIMEOUT` 控制等待请求完成的时间,默认 `10s`。
|
||||
|
||||
使用支持统一网络配置的框架版本时,设置 `GO_OMNI_TRANSPORT=native` 即可切换为自研 HTTP/1.1;
|
||||
默认 `nethttp`。`GO_OMNI_TLS_CERT_FILE` 和 `GO_OMNI_TLS_KEY_FILE` 配对启用 HTTPS。
|
||||
实验版 HTTP/2 使用 `GO_OMNI_TRANSPORT=http2`,必须同时配置 TLS 证书和私钥。
|
||||
业务 Handler / Service / Repository 不需要修改;三种适配器共用上述退出等待时间。
|
||||
{{if eq .Database "mysql"}}
|
||||
`/health/ready` 已注册数据库探针:连接失败返回 `503`,数据库恢复后重新返回 `200`。
|
||||
`/health/live` 始终只检查服务存活。`GO_OMNI_DB_PING_TIMEOUT` 限制初始化检查和数据库探针的等待时间。
|
||||
|
||||
@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.3.1"
|
||||
const defaultFrameworkVersion = "v0.6.0-preview"
|
||||
|
||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||
|
||||
|
||||
@@ -4,24 +4,40 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
const defaultRequestIDHeader = "X-Request-ID"
|
||||
|
||||
const (
|
||||
TransportNetHTTP = "nethttp"
|
||||
TransportNative = "native"
|
||||
TransportHTTP2 = "http2"
|
||||
)
|
||||
|
||||
// TLSConfig enables HTTPS when both certificate paths are provided.
|
||||
type TLSConfig struct {
|
||||
CertFile string `json:"certFile"`
|
||||
KeyFile string `json:"keyFile"`
|
||||
}
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Mode string
|
||||
Address string
|
||||
RequestIDHeader string
|
||||
EnableLogger bool
|
||||
ShutdownTimeout time.Duration
|
||||
Log logx.Config
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
return Config{
|
||||
Address: ":8080",
|
||||
Transport: TransportNetHTTP,
|
||||
RequestIDHeader: defaultRequestIDHeader,
|
||||
ShutdownTimeout: 10 * time.Second,
|
||||
Log: logx.Default(),
|
||||
@@ -39,6 +55,27 @@ func (c Config) Validate() error {
|
||||
if c.ShutdownTimeout <= 0 {
|
||||
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
|
||||
}
|
||||
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
|
||||
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
|
||||
}
|
||||
return c.ValidateTransport()
|
||||
}
|
||||
|
||||
// ValidateTransport checks built-in transport selection without opening files
|
||||
// or sockets. Empty Transport preserves the standard net/http default.
|
||||
func (c Config) ValidateTransport() error {
|
||||
switch c.Transport {
|
||||
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
|
||||
default:
|
||||
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
|
||||
}
|
||||
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
|
||||
if (cert == "") != (key == "") {
|
||||
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}
|
||||
}
|
||||
if c.Transport == TransportHTTP2 && cert == "" {
|
||||
return &ConfigError{Field: "tls", Message: "is required for the experimental http2 transport"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -53,9 +90,14 @@ func (e *ConfigError) Error() string { return "config " + e.Field + " " + e.Mess
|
||||
// Option changes an App's configuration.
|
||||
type Option func(*Config)
|
||||
|
||||
// WithMode sets the default transport's running mode.
|
||||
func WithMode(mode string) Option {
|
||||
return func(cfg *Config) { cfg.Mode = mode }
|
||||
// WithTransport selects a built-in server. HTTP/2 is experimental and requires TLS.
|
||||
func WithTransport(name string) Option {
|
||||
return func(cfg *Config) { cfg.Transport = name }
|
||||
}
|
||||
|
||||
// WithTLS enables HTTPS. Certificates are loaded before Run opens its listener.
|
||||
func WithTLS(certFile, keyFile string) Option {
|
||||
return func(cfg *Config) { cfg.TLS = TLSConfig{CertFile: certFile, KeyFile: keyFile} }
|
||||
}
|
||||
|
||||
// WithRequestIDHeader changes the header used to read and return request IDs.
|
||||
@@ -72,10 +114,22 @@ func WithLogger(enabled bool) Option {
|
||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||
}
|
||||
|
||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||
// by Config.Validate and panic when passed directly to App.New.
|
||||
func WithTrustedProxies(proxies ...string) Option {
|
||||
proxies = append([]string(nil), proxies...)
|
||||
return func(cfg *Config) { cfg.TrustedProxies = append([]string(nil), proxies...) }
|
||||
}
|
||||
|
||||
// WithConfig applies a complete configuration loaded from a file or
|
||||
// environment. Later options can still override individual fields.
|
||||
func WithConfig(settings Config) Option {
|
||||
return func(cfg *Config) { *cfg = settings }
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
return func(cfg *Config) {
|
||||
*cfg = settings
|
||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithLogConfig configures the logger used by the application runtime.
|
||||
|
||||
@@ -8,7 +8,9 @@ import (
|
||||
|
||||
func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
for _, key := range []string{
|
||||
"GO_OMNI_MODE", "GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_TRUSTED_PROXIES",
|
||||
"GO_OMNI_TRANSPORT", "GO_OMNI_TLS_CERT_FILE", "GO_OMNI_TLS_KEY_FILE",
|
||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||
@@ -20,7 +22,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != ":8080" || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
if cfg.Address != ":8080" || cfg.Transport != TransportNetHTTP || cfg.TLS != (TLSConfig{}) || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.EnableLogger || cfg.Log.Level != "info" || cfg.Log.Format != "json" || cfg.Log.Output != "stderr" {
|
||||
@@ -28,10 +30,48 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTransportEnvironment(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport, cert, key, errorField string
|
||||
}{
|
||||
{"default", "", "", "", ""},
|
||||
{"native", "native", "", "", ""},
|
||||
{"nativeTLS", "native", "server.crt", "server.key", ""},
|
||||
{"http2", "http2", "server.crt", "server.key", ""},
|
||||
{"unknown", "typo", "", "", "transport"},
|
||||
{"certOnly", "nethttp", "server.crt", "", "tls"},
|
||||
{"keyOnly", "native", "", "server.key", "tls"},
|
||||
{"http2WithoutTLS", "http2", "", "", "tls"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("TRANSPORT_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_CERT_FILE", test.cert)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_KEY_FILE", test.key)
|
||||
cfg, err := LoadFromEnv("TRANSPORT_TEST_")
|
||||
if test.errorField != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorField) {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := test.transport
|
||||
if want == "" {
|
||||
want = TransportNetHTTP
|
||||
}
|
||||
if cfg.Transport != want || cfg.TLS.CertFile != test.cert || cfg.TLS.KeyFile != test.key {
|
||||
t.Fatalf("settings: %+v", cfg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_MODE", "release")
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:9090")
|
||||
t.Setenv("GO_OMNI_REQUEST_ID_HEADER", "X-Correlation-ID")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "127.0.0.1, 10.0.0.0/8")
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "true")
|
||||
t.Setenv("GO_OMNI_LOG_LEVEL", "debug")
|
||||
t.Setenv("GO_OMNI_LOG_FORMAT", "text")
|
||||
@@ -45,7 +85,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Mode != "release" || cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger {
|
||||
if cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger || len(cfg.TrustedProxies) != 2 || cfg.TrustedProxies[1] != "10.0.0.0/8" {
|
||||
t.Fatalf("unexpected application settings: %+v", cfg)
|
||||
}
|
||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||
@@ -66,6 +106,11 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
}
|
||||
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
|
||||
t.Fatalf("expected invalid trusted proxy error, got %v", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
|
||||
cfg := Default()
|
||||
cfg.Address = ""
|
||||
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "address") {
|
||||
@@ -80,13 +125,17 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
|
||||
func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
settings := Default()
|
||||
settings.Mode = "release"
|
||||
settings.Log.Service = "orders"
|
||||
settings.TrustedProxies = []string{"127.0.0.1"}
|
||||
cfg := Default()
|
||||
WithConfig(settings)(&cfg)
|
||||
if cfg.Mode != "release" || cfg.Log.Service != "orders" {
|
||||
if cfg.Log.Service != "orders" || len(cfg.TrustedProxies) != 1 {
|
||||
t.Fatalf("complete config was not applied: %+v", cfg)
|
||||
}
|
||||
settings.TrustedProxies[0] = "10.0.0.0/8"
|
||||
if cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("WithConfig should copy trusted proxy settings")
|
||||
}
|
||||
|
||||
WithLogConfig(Default().Log)(&cfg)
|
||||
if cfg.Log.Service != "" {
|
||||
|
||||
@@ -13,14 +13,25 @@ const defaultEnvPrefix = "GO_OMNI_"
|
||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||
// database settings remain owned by database.ConfigFromEnv.
|
||||
func LoadFromEnv(prefix string) (Config, error) {
|
||||
return applyEnv(Default(), prefix)
|
||||
}
|
||||
|
||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
cfg := Default()
|
||||
cfg.Mode = envString(prefix+"MODE", cfg.Mode)
|
||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||
cfg.TrustedProxies = strings.Split(value, ",")
|
||||
for i := range cfg.TrustedProxies {
|
||||
cfg.TrustedProxies[i] = strings.TrimSpace(cfg.TrustedProxies[i])
|
||||
}
|
||||
}
|
||||
cfg.Log.Level = envString(prefix+"LOG_LEVEL", cfg.Log.Level)
|
||||
cfg.Log.Format = envString(prefix+"LOG_FORMAT", cfg.Log.Format)
|
||||
cfg.Log.Output = envString(prefix+"LOG_OUTPUT", cfg.Log.Output)
|
||||
|
||||
203
config/file.go
Normal file
203
config/file.go
Normal file
@@ -0,0 +1,203 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gopkg.in/yaml.v3"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if len(data) > 1<<20 {
|
||||
return zero, errors.New("config: file exceeds 1 MiB")
|
||||
}
|
||||
var value any
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".yaml", ".yml":
|
||||
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one YAML document required")
|
||||
}
|
||||
case ".json":
|
||||
dec := json.NewDecoder(bytes.NewReader(data))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one JSON document required")
|
||||
}
|
||||
default:
|
||||
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||
}
|
||||
if _, ok := value.(map[string]any); !ok {
|
||||
return zero, errors.New("config: root must be an object")
|
||||
}
|
||||
value, err = durations(value, reflect.TypeOf(defaults))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||
original, err := json.Marshal(defaults)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var result T
|
||||
if err = json.Unmarshal(original, &result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func durations(v any, t reflect.Type) (any, error) {
|
||||
if t == nil {
|
||||
return v, nil
|
||||
}
|
||||
for t.Kind() == reflect.Pointer {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
if s, ok := v.(string); ok {
|
||||
d, err := time.ParseDuration(s)
|
||||
return int64(d), err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
if t.Kind() == reflect.Struct {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
field := t.Field(n)
|
||||
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = field.Name
|
||||
}
|
||||
if item, exists := m[name]; exists {
|
||||
converted, err := durations(item, field.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m[name] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
converted, err := durations(item, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items[n] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func LoadFile(path, prefix string) (Config, error) {
|
||||
cfg, err := DecodeFile(path, Default())
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return applyEnv(cfg, prefix)
|
||||
}
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||
type Reloader struct {
|
||||
mu sync.Mutex
|
||||
path, prefix string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
}
|
||||
|
||||
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path, prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||
}
|
||||
func (r *Reloader) Reload() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
next, err := LoadFile(r.path, r.prefix)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
comparable := next
|
||||
comparable.Log.Level = r.current.Log.Level
|
||||
if !reflect.DeepEqual(comparable, r.current) {
|
||||
return errors.New("config: static fields changed; restart required")
|
||||
}
|
||||
if err = r.logger.SetLevel(next.Log.Level); err != nil {
|
||||
return err
|
||||
}
|
||||
r.current = next
|
||||
return nil
|
||||
}
|
||||
func (r *Reloader) Snapshot() Config {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
cfg := r.current
|
||||
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||
return cfg
|
||||
}
|
||||
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||
if ctx == nil || interval < 100*time.Millisecond {
|
||||
return errors.New("config: context and interval >=100ms required")
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
if err := r.Reload(); err != nil && onError != nil {
|
||||
onError(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
59
config/file_test.go
Normal file
59
config/file_test.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "service.yaml")
|
||||
write := func(raw string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
lc := cfg.Log
|
||||
lc.Writer = &output
|
||||
l, err := logx.New(lc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
child := l.With("scope", "test")
|
||||
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("hidden")
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: debug\n")
|
||||
if err = r.Reload(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("visible")
|
||||
if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
|
||||
t.Fatal(output.String())
|
||||
}
|
||||
write("shutdownTimeout: 8s\nlog:\n level: error\n")
|
||||
if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
|
||||
t.Fatal("static reload changed live config", err)
|
||||
}
|
||||
for _, raw := range []string{"unknownOption: true", "log:\n level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
|
||||
write(raw)
|
||||
if err = r.Reload(); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
}
|
||||
245
database/migrate/migrate.go
Normal file
245
database/migrate/migrate.go
Normal file
@@ -0,0 +1,245 @@
|
||||
// Package migrate provides explicit MySQL schema migration jobs. Application
|
||||
// startup never runs migrations automatically; use a separate deployment step.
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
mysqlDriver "github.com/go-sql-driver/mysql"
|
||||
migration "github.com/golang-migrate/migrate/v4"
|
||||
migrateMySQL "github.com/golang-migrate/migrate/v4/database/mysql"
|
||||
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||
"io"
|
||||
"io/fs"
|
||||
"path"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type File struct {
|
||||
Version uint64 `json:"version"`
|
||||
Name string `json:"name"`
|
||||
Direction string `json:"direction"`
|
||||
SHA256 string `json:"sha256"`
|
||||
}
|
||||
|
||||
var filename = regexp.MustCompile(`^([0-9]+)_([A-Za-z][A-Za-z0-9]*)\.(up|down)\.sql$`)
|
||||
|
||||
// Snapshot returns immutable migration bytes and a reviewable digest manifest.
|
||||
// Follow lowerCamelCase names, e.g. 0001_createUsers.up.sql.
|
||||
func Snapshot(source fs.FS, directory string) (fs.FS, []File, error) {
|
||||
if source == nil || !fs.ValidPath(directory) {
|
||||
return nil, nil, errors.New("migrate: invalid source directory")
|
||||
}
|
||||
entries, err := fs.ReadDir(source, directory)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var archive bytes.Buffer
|
||||
writer := zip.NewWriter(&archive)
|
||||
manifest := []File{}
|
||||
seen := map[string]bool{}
|
||||
total := 0
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
parts := filename.FindStringSubmatch(entry.Name())
|
||||
if parts == nil {
|
||||
if strings.HasSuffix(entry.Name(), ".sql") {
|
||||
return nil, nil, fmt.Errorf("migrate: invalid migration filename %s", entry.Name())
|
||||
}
|
||||
continue
|
||||
}
|
||||
if len(manifest) >= 1000 {
|
||||
return nil, nil, errors.New("migrate: too many files")
|
||||
}
|
||||
version, err := strconv.ParseUint(parts[1], 10, 32)
|
||||
if err != nil || version == 0 {
|
||||
return nil, nil, errors.New("migrate: version must be 1..4294967295")
|
||||
}
|
||||
identity := fmt.Sprint(version) + parts[3]
|
||||
if seen[identity] {
|
||||
return nil, nil, errors.New("migrate: duplicate migration version/direction")
|
||||
}
|
||||
seen[identity] = true
|
||||
file, err := source.Open(path.Join(directory, entry.Name()))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||
closeErr := file.Close()
|
||||
if err == nil {
|
||||
err = closeErr
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
total += len(data)
|
||||
if len(data) == 0 || len(data) > 1<<20 || total > 16<<20 {
|
||||
return nil, nil, errors.New("migrate: files must be 1..1MiB and <=16MiB total")
|
||||
}
|
||||
sum := sha256.Sum256(data)
|
||||
manifest = append(manifest, File{Version: version, Name: parts[2], Direction: parts[3], SHA256: hex.EncodeToString(sum[:])})
|
||||
fileWriter, err := writer.CreateHeader(&zip.FileHeader{Name: entry.Name(), Method: zip.Store})
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if _, err = fileWriter.Write(data); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
}
|
||||
if len(manifest) == 0 {
|
||||
return nil, nil, errors.New("migrate: no SQL migrations")
|
||||
}
|
||||
for _, file := range manifest {
|
||||
if file.Direction == "down" && !seen[fmt.Sprint(file.Version)+"up"] {
|
||||
return nil, nil, errors.New("migrate: down file lacks up migration")
|
||||
}
|
||||
}
|
||||
sort.Slice(manifest, func(i, j int) bool {
|
||||
if manifest[i].Version == manifest[j].Version {
|
||||
return manifest[i].Direction < manifest[j].Direction
|
||||
}
|
||||
return manifest[i].Version < manifest[j].Version
|
||||
})
|
||||
if err = writer.Close(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
files, err := zip.NewReader(bytes.NewReader(archive.Bytes()), int64(archive.Len()))
|
||||
return files, manifest, err
|
||||
}
|
||||
|
||||
type Runner struct {
|
||||
mu sync.Mutex
|
||||
engine *migration.Migrate
|
||||
db *sql.DB
|
||||
closed bool
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func NewMySQL(ctx context.Context, dsn string, source fs.FS, directory string) (runner *Runner, err error) {
|
||||
if ctx == nil {
|
||||
return nil, errors.New("migrate: context required")
|
||||
}
|
||||
snapshot, _, err := Snapshot(source, directory)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cfg, err := mysqlDriver.ParseDSN(dsn)
|
||||
if err != nil || cfg.DBName == "" {
|
||||
return nil, errors.New("migrate: valid MySQL DSN with database required")
|
||||
}
|
||||
cfg.MultiStatements = true
|
||||
if cfg.Timeout == 0 || cfg.Timeout > 5*time.Second {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
if cfg.ReadTimeout == 0 || cfg.ReadTimeout > 30*time.Second {
|
||||
cfg.ReadTimeout = 30 * time.Second
|
||||
}
|
||||
if cfg.WriteTimeout == 0 || cfg.WriteTimeout > 30*time.Second {
|
||||
cfg.WriteTimeout = 30 * time.Second
|
||||
}
|
||||
connector, err := mysqlDriver.NewConnector(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db := sql.OpenDB(connector)
|
||||
db.SetMaxOpenConns(1)
|
||||
db.SetMaxIdleConns(1)
|
||||
defer func() {
|
||||
if err != nil {
|
||||
db.Close()
|
||||
}
|
||||
}()
|
||||
conn, err := db.Conn(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
driver, err := migrateMySQL.WithConnection(ctx, conn, &migrateMySQL.Config{DatabaseName: cfg.DBName, StatementTimeout: 30 * time.Second})
|
||||
if err != nil {
|
||||
conn.Close()
|
||||
return nil, err
|
||||
}
|
||||
src, err := iofs.New(snapshot, ".")
|
||||
if err != nil {
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine, err := migration.NewWithInstance("iofs", src, cfg.DBName, driver)
|
||||
if err != nil {
|
||||
src.Close()
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine.LockTimeout = 10 * time.Second
|
||||
return &Runner{engine: engine, db: db}, nil
|
||||
}
|
||||
|
||||
// Up applies pending up scripts. MySQL DDL can partially commit; a failed
|
||||
// migration remains dirty and requires operator investigation. No automatic
|
||||
// down/force/dirty reset is provided. Cancellation stops after the current SQL
|
||||
// operation; statements/read I/O are bounded to 30 seconds.
|
||||
func (r *Runner) Up(ctx context.Context) error {
|
||||
if ctx == nil {
|
||||
return errors.New("migrate: context required")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed || r.stopped {
|
||||
return errors.New("migrate: closed")
|
||||
}
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- r.engine.Up() }()
|
||||
var err error
|
||||
select {
|
||||
case err = <-done:
|
||||
case <-ctx.Done():
|
||||
r.stopped = true // golang-migrate retains its graceful-stop state; create a fresh runner to retry.
|
||||
select {
|
||||
case r.engine.GracefulStop <- true:
|
||||
default:
|
||||
}
|
||||
err = <-done
|
||||
return errors.Join(ctx.Err(), err)
|
||||
}
|
||||
if errors.Is(err, migration.ErrNoChange) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
func (r *Runner) Version() (uint, bool, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return 0, false, errors.New("migrate: closed")
|
||||
}
|
||||
version, dirty, err := r.engine.Version()
|
||||
if errors.Is(err, migration.ErrNilVersion) {
|
||||
return 0, false, nil
|
||||
}
|
||||
return version, dirty, err
|
||||
}
|
||||
func (r *Runner) Close() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return nil
|
||||
}
|
||||
r.closed = true
|
||||
a, b := r.engine.Close()
|
||||
return errors.Join(a, b, r.db.Close())
|
||||
}
|
||||
99
database/migrate/migrate_test.go
Normal file
99
database/migrate/migrate_test.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"go.opentelemetry.io/otel/sdk/trace/tracetest"
|
||||
"io/fs"
|
||||
"os"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSnapshotBoundsAndIdentity(t *testing.T) {
|
||||
source := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("SELECT 1;")}}
|
||||
f, manifest, err := Snapshot(source, ".")
|
||||
if err != nil || len(manifest) != 1 {
|
||||
t.Fatal(manifest, err)
|
||||
}
|
||||
source["0001_createItems.up.sql"].Data[0] = 'x'
|
||||
raw, err := fs.ReadFile(f, "0001_createItems.up.sql")
|
||||
if err != nil || string(raw) != "SELECT 1;" {
|
||||
t.Fatal(string(raw), err)
|
||||
}
|
||||
for _, files := range []fstest.MapFS{{"0001_bad_name.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: []byte("x")}, "1_b.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.down.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: make([]byte, (1<<20)+1)}}} {
|
||||
if _, _, err = Snapshot(files, "."); err == nil {
|
||||
t.Fatal("invalid migration accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestMySQLIntegration(t *testing.T) {
|
||||
dsn := os.Getenv("GO_OMNI_MIGRATE_TEST_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL database")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
files := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("CREATE TABLE testItems (id BIGINT NOT NULL PRIMARY KEY, value VARCHAR(64) NOT NULL);")}, "0002_seedItems.up.sql": &fstest.MapFile{Data: []byte("INSERT INTO testItems (id,value) VALUES (1,'original');")}}
|
||||
runner, err := NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal("repeat", err)
|
||||
}
|
||||
v, dirty, err := runner.Version()
|
||||
if err != nil || v != 2 || dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
runner.Close()
|
||||
db, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
exporter := tracetest.NewInMemoryExporter()
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "dbTest", SampleRatio: 1, Exporter: exporter})
|
||||
defer tr.Close()
|
||||
instrumented := telemetry.SQL{DB: db, Runtime: tr}
|
||||
if _, err = instrumented.ExecContext(ctx, "UPDATE testItems SET value=? WHERE id=1", "secret parameter"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rows, err := instrumented.QueryContext(ctx, "SELECT value FROM testItems WHERE id=1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !rows.Next() {
|
||||
t.Fatal("missing row")
|
||||
}
|
||||
var value string
|
||||
if err = rows.Scan(&value); err != nil || value != "secret parameter" {
|
||||
t.Fatal(value, err)
|
||||
}
|
||||
rows.Close()
|
||||
tr.Flush(ctx)
|
||||
if len(exporter.GetSpans()) != 2 {
|
||||
t.Fatal("SQL spans missing")
|
||||
}
|
||||
files["0003_failMigration.up.sql"] = &fstest.MapFile{Data: []byte("INSERT INTO missingTable VALUES (1);")}
|
||||
runner, err = NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("bad migration passed")
|
||||
}
|
||||
v, dirty, err = runner.Version()
|
||||
if err != nil || v != 3 || !dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("dirty migration auto-cleared")
|
||||
}
|
||||
}
|
||||
33
examples/modules/README.md
Normal file
33
examples/modules/README.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# 基础模块组合示例
|
||||
|
||||
在仓库根目录执行:
|
||||
|
||||
```sh
|
||||
go run ./examples/modules
|
||||
```
|
||||
|
||||
默认监听 `127.0.0.1:8080`,使用仅限开发的 `Bearer dev-token`。
|
||||
配置 `GO_OMNI_TRANSPORT=native` 可直接切换自研网络层,TLS 配置与主框架一致。
|
||||
|
||||
```sh
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/catalog
|
||||
curl -H 'Authorization: Bearer dev-token' -H 'Content-Type: application/json' \
|
||||
-H 'Idempotency-Key: report-1' -d '{"name":"report"}' http://127.0.0.1:8080/api/jobs
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/jobs
|
||||
```
|
||||
|
||||
catalog 缓存 30 秒;可通过启动环境变量 `GO_OMNI_DEMO_UPSTREAM` 指定可信 JSON 上游,
|
||||
演示 HTTP 连接池、请求 ID 传递与合并回源。URL 来自服务配置,不接受请求参数传入。
|
||||
|
||||
jobs 展示固定 worker、队列、结构化日志与状态查询;相同 Idempotency-Key 和请求内容重放
|
||||
第一次 202 响应,内容不同返回 409。每分钟的统计任务演示间隔调度。退出先排空任务,再关闭
|
||||
HTTP 客户端和缓存。这个示例的任务、缓存和幂等记录均保存在进程内。
|
||||
|
||||
组合测试通过真实 TLS 连接覆盖 nethttp、native 和实验 HTTP/2,检查缓存回源次数、
|
||||
幂等重放、认证不可被重放绕过、任务只提交一次与关闭流程:
|
||||
|
||||
```sh
|
||||
go test -race ./examples/modules
|
||||
```
|
||||
|
||||
完整 API、默认上限和适用范围见 [业务基础模块](../../MODULES.md)。
|
||||
133
examples/modules/main.go
Normal file
133
examples/modules/main.go
Normal file
@@ -0,0 +1,133 @@
|
||||
// modules combines caching, outbound calls, middleware and background jobs.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
"git.yuntuo.hk/k2safe/go-omni/task"
|
||||
)
|
||||
|
||||
func newApp(settings omni.Config, token, upstream string) (application *omni.App, err error) {
|
||||
application = omni.New(omni.WithConfig(settings))
|
||||
defer func(app *omni.App) {
|
||||
if err != nil {
|
||||
_ = app.Close()
|
||||
}
|
||||
}(application)
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 128, MaxBytes: 1 << 20})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 8, Timeout: time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
client, err := httpc.New(httpc.Config{Timeout: time.Second, MaxRetries: 1, MaxConnsPerHost: 8})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
application.OnShutdown(client.Close)
|
||||
jobs, err := task.New(task.Config{Workers: 2, QueueSize: 16, TaskTimeout: 2 * time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
// Reverse cleanup order: drain jobs before closing their dependencies.
|
||||
application.OnShutdown(jobs.Close)
|
||||
if err = jobs.Every("reportStats", time.Minute, func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "jobs.stats", "stats", jobs.Stats())
|
||||
return nil
|
||||
}); err != nil {
|
||||
return application, err
|
||||
}
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40, MaxKeys: 128, Key: func(c *omni.Context) string { p, _ := auth.FromContext(c); return p.UserID }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Scope: func(c *omni.Context) (string, error) { p, _ := auth.FromContext(c); return p.UserID, nil }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
api := application.Group("/api", middleware.Timeout(2*time.Second), auth.RequireAuth(auth.StaticAuthenticator{token: {UserID: "demo-user"}}), limiter.Middleware())
|
||||
api.GET("/catalog", func(c *omni.Context) error {
|
||||
value, err := loader.GetOrLoad(c.Context(), "publicCatalog:v1", 30*time.Second, func(ctx context.Context) ([]byte, error) {
|
||||
data := map[string]any{"name": "demo catalog"}
|
||||
if upstream != "" {
|
||||
if err := client.JSON(ctx, "GET", upstream, nil, &data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return json.Marshal(data)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.OK(json.RawMessage(value))
|
||||
})
|
||||
var nextID atomic.Uint64
|
||||
api.POST("/jobs", func(c *omni.Context) error {
|
||||
var input struct {
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
}
|
||||
if err := c.BindJSON(&input); err != nil {
|
||||
return err
|
||||
}
|
||||
id := nextID.Add(1)
|
||||
// This job deliberately outlives the HTTP request. Runner still applies
|
||||
// its own deadline and shutdown cancellation; log correlation survives.
|
||||
_, err := jobs.Submit(context.WithoutCancel(c.Context()), "demoJob", func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "job.finished", "jobId", id, "name", input.Name)
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, task.ErrQueueFull) {
|
||||
return omni.NewError(503, "QUEUE_FULL", "job queue is full")
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Accepted(map[string]uint64{"jobId": id})
|
||||
}, idempotency.Middleware())
|
||||
api.GET("/jobs", func(c *omni.Context) error { return c.OK(jobs.Stats()) })
|
||||
return application, nil
|
||||
}
|
||||
|
||||
func run(ctx context.Context) error {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEV_TOKEN")
|
||||
if token == "" {
|
||||
token = "dev-token"
|
||||
}
|
||||
application, err := newApp(settings, token, os.Getenv("GO_OMNI_DEMO_UPSTREAM"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func main() {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
126
examples/modules/main_test.go
Normal file
126
examples/modules/main_test.go
Normal file
@@ -0,0 +1,126 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// Exercise the composed feature chain through real sockets and all runners.
|
||||
func TestModulesConfiguredTransports(t *testing.T) {
|
||||
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
var loads atomic.Int32
|
||||
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
loads.Add(1)
|
||||
if r.Header.Get("X-Request-ID") == "" {
|
||||
t.Error("missing outbound correlation")
|
||||
}
|
||||
io.WriteString(w, `{"name":"upstream catalog"}`)
|
||||
}))
|
||||
defer upstream.Close()
|
||||
settings, err := omni.LoadConfig("MODULES_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
settings.Transport = transport
|
||||
settings.TLS.CertFile = cert
|
||||
settings.TLS.KeyFile = key
|
||||
application, err := newApp(settings, "demo", upstream.URL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("server failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, key, requestID, token string) (*http.Response, string) {
|
||||
r, err := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("Idempotency-Key", key)
|
||||
r.Header.Set("X-Request-ID", requestID)
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
data, err := io.ReadAll(response.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("unverified TLS")
|
||||
}
|
||||
if transport == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal(response.Proto)
|
||||
}
|
||||
return response, string(data)
|
||||
}
|
||||
for n := 0; n < 2; n++ {
|
||||
response, body := request("GET", "/api/catalog", "", "", "catalogRequest", "demo")
|
||||
if response.StatusCode != 200 || !strings.Contains(body, "upstream catalog") {
|
||||
t.Fatal(response.StatusCode, body)
|
||||
}
|
||||
}
|
||||
if loads.Load() != 1 {
|
||||
t.Fatal("cache missed", loads.Load())
|
||||
}
|
||||
first, original := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "firstRequest", "demo")
|
||||
second, replayed := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "secondRequest", "demo")
|
||||
if first.StatusCode != 202 || second.StatusCode != 202 || original != replayed || second.Header.Get("Idempotency-Replayed") != "true" || second.Header.Get("X-Request-ID") != "firstRequest" {
|
||||
t.Fatal(first.StatusCode, second.StatusCode, original, replayed)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"other"}`, "job1", "conflictRequest", "demo"); response.StatusCode != 409 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "unauthorized", "wrong"); response.StatusCode != 401 {
|
||||
t.Fatal("replay bypassed authentication", response.StatusCode)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
response, body := request("GET", "/api/jobs", "", "", "statsRequest", "demo")
|
||||
if response.StatusCode != 200 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if strings.Contains(body, `"completed":1`) {
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("job did not finish once", body)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
14
examples/platform/README.md
Normal file
14
examples/platform/README.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# 扩展模块组合示例
|
||||
|
||||
业务通过 `storage.Store` 写文件,身份接口和 Policy 控制权限,OpenAPI 与路由一起注册,链路追踪关联 HTTP 与文件操作。`newApp` 不依赖 Local/S3/OSS 类型;启动处本例选择 Local。可替换成 S3/R2/OSS 构造结果,业务路由不变。
|
||||
|
||||
```sh
|
||||
export GO_OMNI_DEMO_TOKEN='use-a-private-development-token'
|
||||
go run ./examples/platform --root /tmp/goOmniDemoFiles
|
||||
```
|
||||
|
||||
接口:`POST /api/files`,JSON 为 `{"name":"hello.txt","content":"hello"}`,请求加 `Authorization: Bearer <GO_OMNI_DEMO_TOKEN>`;返回 201 和存储信息。`GET /openapi.json` 同样要求身份。
|
||||
|
||||
也可传 `--config service.yaml`,配置规则见 [扩展模块](../../ADVANCED.md)。本例的静态身份只用于演示,实际应用注入 `auth.JWT` 或 `auth.Sessions`;trace provider 默认不外发,接入 collector 时注入 exporter/reader。
|
||||
|
||||
`TestPlatformTLSAdapters` 通过三个 Runner 的真实 TLS 请求验证 401/403/201、文件内容、非法 key、OpenAPI 和关闭。文件根目录必须是应用私有目录。示例不执行自动数据迁移,也不启动 Orbit。
|
||||
132
examples/platform/main.go
Normal file
132
examples/platform/main.go
Normal file
@@ -0,0 +1,132 @@
|
||||
// platform composes storage, authentication, permission policy, tracing and
|
||||
// OpenAPI using provider-neutral business dependencies.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
)
|
||||
|
||||
type dependencies struct {
|
||||
Files storage.Store
|
||||
Auth auth.Authenticator
|
||||
Tracing *telemetry.Runtime
|
||||
}
|
||||
type upload struct {
|
||||
Name string `json:"name" binding:"required,max=128"`
|
||||
Content string `json:"content" binding:"max=4096"`
|
||||
}
|
||||
|
||||
// Dependencies are borrowed; the composition root owns their shutdown.
|
||||
func newApp(cfg omni.Config, d dependencies) (*omni.App, error) {
|
||||
if d.Files == nil || d.Auth == nil || d.Tracing == nil {
|
||||
return nil, errors.New("platform: dependencies required")
|
||||
}
|
||||
app := omni.New(omni.WithConfig(cfg))
|
||||
app.Use(d.Tracing.Middleware())
|
||||
api, err := openapi.New("File service", "1")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input, err := openapi.SchemaOf[upload]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
output, err := openapi.SchemaOf[storage.Info]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
policy := auth.PolicyFunc(func(_ context.Context, p auth.Principal, action, resource string) error {
|
||||
if p.TenantID != "" && p.HasRole("writer") && action == "create" && resource == "files" {
|
||||
return nil
|
||||
}
|
||||
return errors.New("denied")
|
||||
})
|
||||
err = api.Handle(app, openapi.Operation{Method: "POST", Path: "/api/files", OperationID: "createFile", Summary: "Store a small text file", Request: input, BearerAuth: true, Responses: map[int]openapi.Response{201: {Description: "Created", Schema: openapi.Envelope(output)}}}, func(c *omni.Context) error {
|
||||
var in upload
|
||||
if err := c.BindJSON(&in); err != nil {
|
||||
return err
|
||||
}
|
||||
p, _ := auth.FromContext(c)
|
||||
if strings.ContainsAny(in.Name, "/\\") {
|
||||
return omni.BadRequest("INVALID_NAME", "name must be one path segment")
|
||||
}
|
||||
key := p.TenantID + "/" + in.Name
|
||||
if err := storage.Check(c.Context(), key); err != nil {
|
||||
return omni.BadRequest("INVALID_KEY", "invalid storage key")
|
||||
}
|
||||
var info storage.Info
|
||||
err := d.Tracing.Do(c.Context(), "files.put", func(ctx context.Context) error {
|
||||
var err error
|
||||
info, err = d.Files.Put(ctx, key, strings.NewReader(in.Content), int64(len(in.Content)), "text/plain; charset=utf-8")
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Created(info)
|
||||
}, auth.RequireAuth(d.Auth), auth.RequirePermission(policy, "create", func(*omni.Context) string { return "files" }))
|
||||
if err != nil {
|
||||
app.Close()
|
||||
return nil, err
|
||||
}
|
||||
app.GET("/openapi.json", api.Handler(), auth.RequireAuth(d.Auth))
|
||||
return app, nil
|
||||
}
|
||||
func run(ctx context.Context, configFile, root string) error {
|
||||
var cfg config.Config
|
||||
var err error
|
||||
if configFile != "" {
|
||||
cfg, err = config.LoadFile(configFile, "")
|
||||
} else {
|
||||
cfg, err = config.LoadFromEnv("")
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
cfg.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEMO_TOKEN")
|
||||
if token == "" {
|
||||
return errors.New("set GO_OMNI_DEMO_TOKEN for the demonstration credential")
|
||||
}
|
||||
files, err := local.New(root, 64<<10)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tracing, err := telemetry.New(telemetry.Config{Service: "fileService", SampleRatio: 1})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tracing.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: files, Auth: auth.StaticAuthenticator{token: {UserID: "demoUser", TenantID: "demoTenant", Roles: []string{"writer"}}}, Tracing: tracing})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return app.RunContext(ctx)
|
||||
}
|
||||
func main() {
|
||||
file := flag.String("config", "", "optional JSON/YAML config")
|
||||
root := flag.String("root", "./fileData", "private local object directory")
|
||||
flag.Parse()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx, *file, *root); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
102
examples/platform/main_test.go
Normal file
102
examples/platform/main_test.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestPlatformTLSAdapters(t *testing.T) {
|
||||
for _, name := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.Transport = name
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
cfg.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
store, _ := local.New(t.TempDir(), 64<<10)
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "platformTest", SampleRatio: 1})
|
||||
defer tr.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: store, Auth: auth.StaticAuthenticator{"writer": {UserID: "u", TenantID: "t", Roles: []string{"writer"}}, "reader": {UserID: "r", TenantID: "t"}}, Tracing: tr})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- app.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("shutdown timeout")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, token string) (int, string) {
|
||||
t.Helper()
|
||||
r, _ := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, err := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("TLS unverified")
|
||||
}
|
||||
if name == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal("protocol")
|
||||
}
|
||||
return response.StatusCode, string(raw)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
token string
|
||||
status int
|
||||
}{{"", 401}, {"reader", 403}, {"writer", 201}} {
|
||||
status, body := request("POST", "/api/files", `{"name":"hello.txt","content":"hello"}`, test.token)
|
||||
if status != test.status {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
}
|
||||
object, err := store.Get(context.Background(), "t/hello.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := io.ReadAll(object.Body)
|
||||
object.Body.Close()
|
||||
if string(raw) != "hello" {
|
||||
t.Fatal(string(raw))
|
||||
}
|
||||
if status, _ := request("POST", "/api/files", `{"name":"../escape","content":"x"}`, "writer"); status != 400 {
|
||||
t.Fatal(status)
|
||||
}
|
||||
if status, body := request("GET", "/openapi.json", "", "writer"); status != 200 || !strings.Contains(body, `"/api/files"`) {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -41,8 +41,8 @@ examples/reference/
|
||||
|
||||
调用路径:HTTP → Handler → Service → Repository。
|
||||
|
||||
- Handler 依赖 GoOmni,示例代码不导入 Gin,也不调用 `Engine()` / `Raw()`。
|
||||
- Service、Repository、Model 不依赖 GoOmni 或 Gin,通过标准 `context.Context` 传递请求取消信号。
|
||||
- Handler 依赖 GoOmni,示例代码不导入具体网络库,也不调用 `Engine()` / `Raw()`。
|
||||
- Service、Repository、Model 不依赖 GoOmni 或网络库,通过标准 `context.Context` 传递请求取消信号。
|
||||
- JSON 校验标签留在 Handler 的请求结构上;业务模型没有 HTTP 绑定标签。
|
||||
- Service 返回业务错误,由 Handler 映射为 HTTP 错误;未知错误交给框架转换成通用 500 响应。
|
||||
- 通过 `application.Group("/api/v1", handler.RequestLog(logger))` 挂载自定义中间件,再注册 `/users` 子路由。日志记录请求 ID、耗时和返回错误;panic 由框架恢复中间件处理。
|
||||
@@ -156,6 +156,6 @@ go vet ./...
|
||||
go build ./...
|
||||
```
|
||||
|
||||
`TestUserCRUD` 将同一份业务注册代码分别挂载到 Gin 与 nethttp,验证替换适配器后 Handler、Service、Repository 均无需修改。
|
||||
`TestUserCRUD` 使用标准库适配器运行完整业务注册代码;同一套 Handler、Service、Repository 也可通过 `omni.NewWithRouter` 接入自研适配器。
|
||||
|
||||
测试覆盖完整 CRUD、认证和角色权限、校验失败、分页边界、邮箱冲突及失败更新、删除后重用邮箱、请求 ID、日志关联、内部错误隐藏、并发唯一性、取消请求不修改数据、数据库配置解析和 HTTP 优雅退出。MySQL 测试不自动连接外部数据库。
|
||||
|
||||
@@ -2,6 +2,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
@@ -41,28 +42,52 @@ func registerRoutes(application *omni.App, users service.UserRepository, logger
|
||||
}
|
||||
|
||||
func main() {
|
||||
logger, err := logx.New(logx.Config{
|
||||
Service: "reference-api",
|
||||
Environment: "local",
|
||||
Format: "json",
|
||||
Writer: os.Stdout,
|
||||
})
|
||||
if err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
defer logger.Close()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := runReference(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func referenceConfig() (omni.Config, error) {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return settings, err
|
||||
}
|
||||
// Preserve the example's loopback default and legacy address variable.
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_HTTP_ADDR")) == "" {
|
||||
settings.Address = strings.TrimSpace(os.Getenv("GO_OMNI_ADDR"))
|
||||
if settings.Address == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if settings.Log.Service == "" {
|
||||
settings.Log.Service = "reference-api"
|
||||
}
|
||||
if settings.Log.Environment == "" {
|
||||
settings.Log.Environment = "local"
|
||||
}
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")) == "" {
|
||||
settings.Log.Output = "stdout"
|
||||
}
|
||||
return settings, nil
|
||||
}
|
||||
|
||||
func runReference(ctx context.Context) error {
|
||||
settings, err := referenceConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
logger, err := logx.New(settings.Log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer logger.Close()
|
||||
logx.SetDefault(logger)
|
||||
users, closeStorage, storage, err := openUsers(ctx, logger)
|
||||
if err != nil {
|
||||
logger.Error("storage initialization failed", "error", err)
|
||||
os.Exit(1)
|
||||
return err
|
||||
}
|
||||
defer func() {
|
||||
if err := closeStorage(); err != nil {
|
||||
logger.Error("storage close failed", "error", err)
|
||||
}
|
||||
}()
|
||||
devToken := strings.TrimSpace(os.Getenv("GO_OMNI_DEV_TOKEN"))
|
||||
if devToken == "" {
|
||||
devToken = "dev-token"
|
||||
@@ -70,16 +95,11 @@ func main() {
|
||||
authenticator := auth.StaticAuthenticator{
|
||||
devToken: {UserID: "dev-user", Roles: []string{"admin"}},
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator)
|
||||
address := os.Getenv("GO_OMNI_ADDR")
|
||||
if address == "" {
|
||||
address = "127.0.0.1:8080"
|
||||
}
|
||||
logger.Info("starting reference API", "address", address, "storage", storage)
|
||||
if err := application.RunContext(ctx, address); err != nil {
|
||||
logger.Error("server stopped", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(closeStorage)
|
||||
logger.Info("starting reference API", "address", settings.Address, "transport", settings.Transport, "tls", settings.TLS.CertFile != "", "storage", storage)
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func openUsers(ctx context.Context, logger *logx.Logger) (service.UserRepository, func() error, string, error) {
|
||||
|
||||
@@ -20,7 +20,6 @@ import (
|
||||
"git.yuntuo.hk/k2safe/go-omni/examples/reference/repository"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
ginrouter "git.yuntuo.hk/k2safe/go-omni/transport/gin"
|
||||
netrouter "git.yuntuo.hk/k2safe/go-omni/transport/nethttp"
|
||||
)
|
||||
|
||||
@@ -32,7 +31,7 @@ type envelope struct {
|
||||
}
|
||||
|
||||
func testApp() *omni.App {
|
||||
return newApp(repository.NewMemoryUsers(), testLogger(io.Discard), omni.WithMode("test"))
|
||||
return newApp(repository.NewMemoryUsers(), testLogger(io.Discard))
|
||||
}
|
||||
|
||||
func testLogger(writer io.Writer) *logx.Logger {
|
||||
@@ -87,7 +86,6 @@ func TestProtectedWriteRoutes(t *testing.T) {
|
||||
repository.NewMemoryUsers(),
|
||||
testLogger(io.Discard),
|
||||
auth.StaticAuthenticator{"user-token": {UserID: "user-1", Roles: []string{"user"}}},
|
||||
omni.WithMode("test"),
|
||||
)
|
||||
request(t, app, "POST", "/api/v1/users", `{"name":"Alice","email":"alice@example.com"}`, 401, "UNAUTHENTICATED")
|
||||
created := dataAs[model.User](t, requestWithAuthorization(t, app, "POST", "/api/v1/users", `{"name":"Alice","email":"alice@example.com"}`, "Bearer user-token", 201, "CREATED"))
|
||||
@@ -110,7 +108,6 @@ func TestUserCRUD(t *testing.T) {
|
||||
name string
|
||||
newRouter func() httpx.Router
|
||||
}{
|
||||
{"gin", func() httpx.Router { return ginrouter.New("test") }},
|
||||
{"nethttp", func() httpx.Router { return netrouter.New() }},
|
||||
} {
|
||||
t.Run(adapter.name, func(t *testing.T) {
|
||||
@@ -121,7 +118,7 @@ func TestUserCRUD(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func checkUserCRUD(t *testing.T, app *omni.App) {
|
||||
func checkUserCRUD(t *testing.T, app http.Handler) {
|
||||
t.Helper()
|
||||
health := dataAs[map[string]string](t, request(t, app, "GET", "/health", "", 200, "OK"))
|
||||
if health["status"] != "ok" {
|
||||
@@ -226,7 +223,7 @@ func TestErrorTranslationAndRequestLogging(t *testing.T) {
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
var logs bytes.Buffer
|
||||
app := newApp(failingUsers{repository.NewMemoryUsers(), test.err}, testLogger(&logs), omni.WithMode("test"))
|
||||
app := newApp(failingUsers{repository.NewMemoryUsers(), test.err}, testLogger(&logs))
|
||||
result := request(t, app, "GET", "/api/v1/users/1", "", test.status, test.code)
|
||||
if strings.Contains(result.Message, test.err.Error()) || len(result.Data) != 0 {
|
||||
t.Fatalf("internal error details leaked: %+v", result)
|
||||
|
||||
182
examples/reference/transport_test.go
Normal file
182
examples/reference/transport_test.go
Normal file
@@ -0,0 +1,182 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/examples/reference/repository"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// networkHandler runs the same business assertions through a real TCP client.
|
||||
// The application handler is only called by its configured server transport.
|
||||
type networkHandler struct {
|
||||
t *testing.T
|
||||
client *http.Client
|
||||
address, scheme, token, alpn string
|
||||
proto int
|
||||
requests *int
|
||||
}
|
||||
|
||||
func (h networkHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
h.t.Helper()
|
||||
req := r.Clone(r.Context())
|
||||
req.RequestURI = ""
|
||||
req.URL.Scheme, req.URL.Host, req.Host = h.scheme, h.address, h.address
|
||||
if h.token != "" && req.Header.Get("Authorization") == "" {
|
||||
req.Header.Set("Authorization", "Bearer "+h.token)
|
||||
}
|
||||
response, err := h.client.Do(req)
|
||||
if err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if strings.HasPrefix(req.URL.Path, "/api/v1/") {
|
||||
*h.requests++
|
||||
}
|
||||
if response.ProtoMajor != h.proto {
|
||||
h.t.Fatalf("protocol %s, want HTTP/%d", response.Proto, h.proto)
|
||||
}
|
||||
if h.scheme == "https" && (response.TLS == nil || len(response.TLS.VerifiedChains) == 0 || response.TLS.NegotiatedProtocol != h.alpn) {
|
||||
h.t.Fatalf("TLS/ALPN verification failed: %+v", response.TLS)
|
||||
}
|
||||
for key, values := range response.Header {
|
||||
w.Header()[key] = append([]string(nil), values...)
|
||||
}
|
||||
w.WriteHeader(response.StatusCode)
|
||||
if _, err := io.Copy(w, response.Body); err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
type lockedLog struct {
|
||||
sync.Mutex
|
||||
bytes.Buffer
|
||||
}
|
||||
|
||||
func (b *lockedLog) Write(p []byte) (int, error) {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
|
||||
func (b *lockedLog) snapshot() string {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.String()
|
||||
}
|
||||
|
||||
func TestBusinessConfiguredTransports(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport string
|
||||
tls bool
|
||||
proto int
|
||||
alpn string
|
||||
}{
|
||||
{"nethttp", omni.TransportNetHTTP, false, 1, ""},
|
||||
{"native", omni.TransportNative, false, 1, ""},
|
||||
{"nethttpTLS", omni.TransportNetHTTP, true, 2, "h2"},
|
||||
{"nativeTLS", omni.TransportNative, true, 1, "http/1.1"},
|
||||
{"http2TLS", omni.TransportHTTP2, true, 2, "h2"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
t.Setenv("REFERENCE_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("REFERENCE_TEST_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", "")
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", "")
|
||||
if test.tls {
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", cert)
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", key)
|
||||
}
|
||||
settings, err := omni.LoadConfig("REFERENCE_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var logs lockedLog
|
||||
logger := testLogger(&logs)
|
||||
application := newProtectedApp(repository.NewMemoryUsers(), logger, auth.StaticAuthenticator{
|
||||
"admin": {UserID: "admin", Roles: []string{"admin"}},
|
||||
"user": {UserID: "user", Roles: []string{"user"}},
|
||||
}, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
t.Cleanup(func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("business server did not stop")
|
||||
listener.Close()
|
||||
}
|
||||
})
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
scheme := "http"
|
||||
if test.tls {
|
||||
scheme = "https"
|
||||
}
|
||||
requests := 0
|
||||
network := networkHandler{t: t, client: client, address: listener.Addr().String(), scheme: scheme, proto: test.proto, alpn: test.alpn, requests: &requests}
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":"Alice","email":"alice@example.com"}`, 401, "UNAUTHENTICATED")
|
||||
requestWithAuthorization(t, network, "DELETE", "/api/v1/users/1", "", "Bearer user", 403, "FORBIDDEN")
|
||||
network.token = "admin"
|
||||
checkUserCRUD(t, network)
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":`, 400, "INVALID_BODY")
|
||||
count := 0
|
||||
for _, line := range strings.Split(strings.TrimSpace(logs.snapshot()), "\n") {
|
||||
var record map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &record); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record["requestId"] == "reference-test" {
|
||||
count++
|
||||
}
|
||||
}
|
||||
if count < requests {
|
||||
t.Fatalf("missing correlated business logs: %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReferenceAddressCompatibility(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_TRANSPORT", "native")
|
||||
t.Setenv("GO_OMNI_TLS_CERT_FILE", "")
|
||||
t.Setenv("GO_OMNI_TLS_KEY_FILE", "")
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "")
|
||||
t.Setenv("GO_OMNI_ADDR", "")
|
||||
settings, err := referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8080" || settings.Transport != "native" {
|
||||
t.Fatalf("default: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_ADDR", "127.0.0.1:8081")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8081" {
|
||||
t.Fatalf("legacy: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:8082")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8082" {
|
||||
t.Fatalf("standard: %+v %v", settings, err)
|
||||
}
|
||||
}
|
||||
56
go.mod
56
go.mod
@@ -1,38 +1,48 @@
|
||||
module git.yuntuo.hk/k2safe/go-omni
|
||||
|
||||
go 1.22
|
||||
go 1.22.0
|
||||
|
||||
require (
|
||||
github.com/gin-gonic/gin v1.10.0
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0
|
||||
github.com/go-playground/validator/v10 v10.20.0
|
||||
github.com/go-sql-driver/mysql v1.9.3
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1
|
||||
github.com/minio/minio-go/v7 v7.0.83
|
||||
github.com/redis/go-redis/v9 v9.17.3
|
||||
go.opentelemetry.io/otel v1.31.0
|
||||
go.opentelemetry.io/otel/metric v1.31.0
|
||||
go.opentelemetry.io/otel/sdk v1.31.0
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0
|
||||
go.opentelemetry.io/otel/trace v1.31.0
|
||||
golang.org/x/mod v0.21.0
|
||||
golang.org/x/net v0.33.0
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
)
|
||||
|
||||
require (
|
||||
filippo.io/edwards25519 v1.1.0 // indirect
|
||||
github.com/bytedance/sonic v1.11.6 // indirect
|
||||
github.com/bytedance/sonic/loader v0.1.1 // indirect
|
||||
github.com/cloudwego/base64x v0.1.4 // indirect
|
||||
github.com/cloudwego/iasm v0.2.0 // indirect
|
||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
|
||||
github.com/dustin/go-humanize v1.0.1 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||
github.com/gin-contrib/sse v0.1.0 // indirect
|
||||
github.com/go-ini/ini v1.67.0 // indirect
|
||||
github.com/go-logr/logr v1.4.2 // indirect
|
||||
github.com/go-logr/stdr v1.2.2 // indirect
|
||||
github.com/go-playground/locales v0.14.1 // indirect
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/goccy/go-json v0.10.2 // indirect
|
||||
github.com/json-iterator/go v1.1.12 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.7 // indirect
|
||||
github.com/goccy/go-json v0.10.4 // indirect
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/klauspost/compress v1.17.11 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
github.com/mattn/go-isatty v0.0.20 // indirect
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
|
||||
github.com/modern-go/reflect2 v1.0.2 // indirect
|
||||
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
|
||||
github.com/ugorji/go/codec v1.2.12 // indirect
|
||||
golang.org/x/arch v0.8.0 // indirect
|
||||
golang.org/x/crypto v0.23.0 // indirect
|
||||
golang.org/x/net v0.25.0 // indirect
|
||||
golang.org/x/sys v0.20.0 // indirect
|
||||
golang.org/x/text v0.15.0 // indirect
|
||||
google.golang.org/protobuf v1.34.1 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
github.com/minio/md5-simd v1.1.2 // indirect
|
||||
github.com/rs/xid v1.6.0 // indirect
|
||||
go.uber.org/atomic v1.7.0 // indirect
|
||||
golang.org/x/crypto v0.31.0 // indirect
|
||||
golang.org/x/sys v0.28.0 // indirect
|
||||
golang.org/x/text v0.21.0 // indirect
|
||||
golang.org/x/time v0.5.0 // indirect
|
||||
)
|
||||
|
||||
169
go.sum
169
go.sum
@@ -1,22 +1,45 @@
|
||||
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||
github.com/bytedance/sonic v1.11.6 h1:oUp34TzMlL+OY1OUWxHqsdkgC/Zfc85zGqw9siXjrc0=
|
||||
github.com/bytedance/sonic v1.11.6/go.mod h1:LysEHSvpvDySVdC2f87zGWf6CIKJcAvqab1ZaiQtds4=
|
||||
github.com/bytedance/sonic/loader v0.1.1 h1:c+e5Pt1k/cy5wMveRDyk2X4B9hF4g7an8N3zCYjJFNM=
|
||||
github.com/bytedance/sonic/loader v0.1.1/go.mod h1:ncP89zfokxS5LZrJxl5z0UJcsk4M4yY2JpfqGeCtNLU=
|
||||
github.com/cloudwego/base64x v0.1.4 h1:jwCgWpFanWmN8xoIUHa2rtzmkd5J2plF/dnLS6Xd/0Y=
|
||||
github.com/cloudwego/base64x v0.1.4/go.mod h1:0zlkT4Wn5C6NdauXdJRhSKRlJvmclQ1hhJgA0rcu/8w=
|
||||
github.com/cloudwego/iasm v0.2.0 h1:1KNIy1I1H9hNNFEEH3DVnI4UujN+1zjpuk6gwHLTssg=
|
||||
github.com/cloudwego/iasm v0.2.0/go.mod h1:8rXZaNYT2n95jn+zTI1sDr+IgcD2GVs0nlbbQPiEFhY=
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
|
||||
github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
|
||||
github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
|
||||
github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0 h1:uWzn3io54f9L9mvwsQQSv1KpkkFA06hBxI++RvIyvpI=
|
||||
github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
|
||||
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
|
||||
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
|
||||
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
|
||||
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
|
||||
github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
|
||||
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
|
||||
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
|
||||
github.com/dhui/dktest v0.4.3 h1:wquqUxAFdcUgabAVLvSCOKOlag5cIZuaOjYIBOWdsR0=
|
||||
github.com/dhui/dktest v0.4.3/go.mod h1:zNK8IwktWzQRm6I/l2Wjp7MakiyaFWv4G1hjmodmMTs=
|
||||
github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
|
||||
github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
|
||||
github.com/docker/docker v27.2.0+incompatible h1:Rk9nIVdfH3+Vz4cyI/uhbINhEZ/oLmc+CBXmH6fbNk4=
|
||||
github.com/docker/docker v27.2.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
|
||||
github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
|
||||
github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
|
||||
github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
|
||||
github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
|
||||
github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
|
||||
github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
|
||||
github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
|
||||
github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE=
|
||||
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI=
|
||||
github.com/gin-gonic/gin v1.10.0 h1:nTuyha1TYqgedzytsKYqna+DfLos46nTv2ygFy86HFU=
|
||||
github.com/gin-gonic/gin v1.10.0/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
|
||||
github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
|
||||
github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
|
||||
github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
|
||||
github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
|
||||
github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
|
||||
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
|
||||
github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
|
||||
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||
@@ -27,67 +50,85 @@ github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBEx
|
||||
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
|
||||
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU=
|
||||
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I=
|
||||
github.com/google/go-cmp v0.5.5 h1:Khx7svrCpmxxtHBq5j2mp/xVjsi8hQMfNLvJFAlrGgU=
|
||||
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE=
|
||||
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
|
||||
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
|
||||
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
|
||||
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
github.com/klauspost/cpuid/v2 v2.2.7 h1:ZWSB3igEs+d0qvnxR/ZBzXVmxkgt8DdzP6m9pfuVLDM=
|
||||
github.com/klauspost/cpuid/v2 v2.2.7/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws=
|
||||
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M=
|
||||
github.com/goccy/go-json v0.10.4 h1:JSwxQzIqKfmFX1swYPpUThQZp/Ka4wzJdK0LWVytLPM=
|
||||
github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||
github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
|
||||
github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1 h1:JML/k+t4tpHCpQTCAD62Nu43NUFzHY4CV3uAuvHGC+Y=
|
||||
github.com/golang-migrate/migrate/v4 v4.18.1/go.mod h1:HAX6m3sQgcdO81tdjn5exv20+3Kb13cmGli1hrD6hks=
|
||||
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
|
||||
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
|
||||
github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
|
||||
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
|
||||
github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
|
||||
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
|
||||
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
|
||||
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
|
||||
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 h1:66ze0taIn2H33fBvCkXuv9BmCwDfafmiIVpKV9kKGuY=
|
||||
github.com/klauspost/cpuid/v2 v2.2.9/go.mod h1:rqkxqrZ1EhYM9G+hXH7YdowN5R5RGN6NK4QwQ3WMXF8=
|
||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
|
||||
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
|
||||
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
|
||||
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
|
||||
github.com/pelletier/go-toml/v2 v2.2.2 h1:aYUidT7k73Pcl9nb2gScu7NSrKCSHIDE89b3+6Wq+LM=
|
||||
github.com/pelletier/go-toml/v2 v2.2.2/go.mod h1:1t835xjRzz80PqgE6HHgN2JOsmgYu/h4qDAS4n929Rs=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
|
||||
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
|
||||
github.com/minio/minio-go/v7 v7.0.83 h1:W4Kokksvlz3OKf3OqIlzDNKd4MERlC2oN8YptwJ0+GA=
|
||||
github.com/minio/minio-go/v7 v7.0.83/go.mod h1:57YXpvc5l3rjPdhqNrDsvVlY0qPI6UTk1bflAe+9doY=
|
||||
github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
|
||||
github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
|
||||
github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
|
||||
github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
|
||||
github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
|
||||
github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
|
||||
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
|
||||
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
|
||||
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
|
||||
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
|
||||
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
|
||||
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/redis/go-redis/v9 v9.17.3 h1:fN29NdNrE17KttK5Ndf20buqfDZwGNgoUr9qjl1DQx4=
|
||||
github.com/redis/go-redis/v9 v9.17.3/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
|
||||
github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
|
||||
github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
|
||||
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
|
||||
github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSSt89Yw=
|
||||
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
|
||||
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
|
||||
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
|
||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
|
||||
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
|
||||
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
|
||||
github.com/ugorji/go/codec v1.2.12 h1:9LC83zGrHhuUA9l16C9AHXAqEV/2wBQ4nkvumAE65EE=
|
||||
github.com/ugorji/go/codec v1.2.12/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
|
||||
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8=
|
||||
golang.org/x/arch v0.8.0 h1:3wRIsP3pM4yUptoR96otTUOXI367OS0+c9eeRi9doIc=
|
||||
golang.org/x/arch v0.8.0/go.mod h1:FEVrYAQjsQXMVJ1nsMoVVXPZg6p2JE2mx8psSWTDQys=
|
||||
golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI=
|
||||
golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
|
||||
golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
|
||||
golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
|
||||
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||
golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
|
||||
golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
|
||||
golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
|
||||
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
|
||||
google.golang.org/protobuf v1.34.1 h1:9ddQBjfCyZPOHPUiPxpYESBLc+T8P3E+Vo4IbKZgFWg=
|
||||
google.golang.org/protobuf v1.34.1/go.mod h1:c6P6GXX6sHbq/GpV6MGZEdwhWPcYBgnhAHhKbcUYpos=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 h1:TT4fX+nBOA/+LUkobKGW1ydGcn+G3vRw9+g5HwCphpk=
|
||||
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0/go.mod h1:L7UH0GbB0p47T4Rri3uHjbpCFYrVrwc1I25QhNPiGK8=
|
||||
go.opentelemetry.io/otel v1.31.0 h1:NsJcKPIW0D0H3NgzPDHmo0WW6SptzPdqg/L1zsIm2hY=
|
||||
go.opentelemetry.io/otel v1.31.0/go.mod h1:O0C14Yl9FgkjqcCZAsE053C13OaddMYr/hz6clDkEJE=
|
||||
go.opentelemetry.io/otel/metric v1.31.0 h1:FSErL0ATQAmYHUIzSezZibnyVlft1ybhy4ozRPcF2fE=
|
||||
go.opentelemetry.io/otel/metric v1.31.0/go.mod h1:C3dEloVbLuYoX41KpmAhOqNriGbA+qqH6PQ5E5mUfnY=
|
||||
go.opentelemetry.io/otel/sdk v1.31.0 h1:xLY3abVHYZ5HSfOg3l2E5LUj2Cwva5Y7yGxnSW9H5Gk=
|
||||
go.opentelemetry.io/otel/sdk v1.31.0/go.mod h1:TfRbMdhvxIIr/B2N2LQW2S5v9m3gOQ/08KsbbO5BPT0=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0 h1:i9hxxLJF/9kkvfHppyLL55aW7iIJz4JjxTeYusH7zMc=
|
||||
go.opentelemetry.io/otel/sdk/metric v1.31.0/go.mod h1:CRInTMVvNhUKgSAMbKyTMxqOBC0zgyxzW55lZzX43Y8=
|
||||
go.opentelemetry.io/otel/trace v1.31.0 h1:ffjsj1aRouKewfr85U2aGagJ46+MvodynlQ1HYdmJys=
|
||||
go.opentelemetry.io/otel/trace v1.31.0/go.mod h1:TXZkRk7SM2ZQLtR6eoAWQFIHPvzQ06FJAsO1tJg480A=
|
||||
go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
|
||||
go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
|
||||
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||
golang.org/x/mod v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
|
||||
golang.org/x/mod v0.21.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
|
||||
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
|
||||
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
|
||||
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
||||
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||
golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
|
||||
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50=
|
||||
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=
|
||||
|
||||
255
httpc/client.go
Normal file
255
httpc/client.go
Normal file
@@ -0,0 +1,255 @@
|
||||
// Package httpc provides a pooled HTTP client with an end-to-end deadline and
|
||||
// conservative, bounded retries. Redirects are returned to the caller.
|
||||
package httpc
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/rand"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
var ErrClosed = errors.New("httpc: closed")
|
||||
var ErrResponseTooLarge = errors.New("httpc: response exceeds limit")
|
||||
|
||||
type Config struct {
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
MaxRetries int `json:"maxRetries"`
|
||||
Backoff time.Duration `json:"backoff"`
|
||||
MaxBackoff time.Duration `json:"maxBackoff"`
|
||||
MaxConnsPerHost int `json:"maxConnsPerHost"`
|
||||
MaxResponseBytes int64 `json:"maxResponseBytes"`
|
||||
DisableRequestID bool `json:"disableRequestId"`
|
||||
DisableLogging bool `json:"disableLogging"`
|
||||
Transport http.RoundTripper `json:"-"` // Borrowed. The caller owns its lifecycle.
|
||||
}
|
||||
|
||||
type Client struct {
|
||||
config Config
|
||||
client *http.Client
|
||||
owned *http.Transport
|
||||
closed atomic.Bool
|
||||
}
|
||||
|
||||
func New(cfg Config) (*Client, error) {
|
||||
if cfg.Timeout < 0 || cfg.Backoff < 0 || cfg.MaxBackoff < 0 || cfg.MaxRetries < 0 || cfg.MaxRetries > 5 || cfg.MaxConnsPerHost < 0 || cfg.MaxResponseBytes < 0 {
|
||||
return nil, errors.New("httpc: invalid configuration")
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 10 * time.Second
|
||||
}
|
||||
if cfg.Backoff == 0 {
|
||||
cfg.Backoff = 100 * time.Millisecond
|
||||
}
|
||||
if cfg.MaxBackoff == 0 {
|
||||
cfg.MaxBackoff = 2 * time.Second
|
||||
}
|
||||
if cfg.Backoff > cfg.MaxBackoff {
|
||||
return nil, errors.New("httpc: backoff exceeds maxBackoff")
|
||||
}
|
||||
if cfg.MaxConnsPerHost == 0 {
|
||||
cfg.MaxConnsPerHost = 64
|
||||
}
|
||||
if cfg.MaxResponseBytes == 0 {
|
||||
cfg.MaxResponseBytes = 1 << 20
|
||||
}
|
||||
if cfg.MaxResponseBytes > 1<<30 || cfg.MaxConnsPerHost > 65536 {
|
||||
return nil, errors.New("httpc: response or connection limit exceeds supported capacity")
|
||||
}
|
||||
c := &Client{config: cfg}
|
||||
transport := cfg.Transport
|
||||
if transport == nil {
|
||||
// Own a private pool; never mutate the process-wide DefaultTransport.
|
||||
base, ok := http.DefaultTransport.(*http.Transport)
|
||||
if !ok {
|
||||
return nil, errors.New("httpc: provide Transport when DefaultTransport is replaced")
|
||||
}
|
||||
c.owned = base.Clone()
|
||||
c.owned.MaxConnsPerHost = cfg.MaxConnsPerHost
|
||||
c.owned.MaxIdleConnsPerHost = cfg.MaxConnsPerHost
|
||||
c.owned.MaxIdleConns = cfg.MaxConnsPerHost * 2
|
||||
transport = c.owned
|
||||
}
|
||||
c.client = &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
// Do keeps the total deadline active until the response body is closed or
|
||||
// consumed. Callers own response.Body. Only GET, HEAD and OPTIONS are retried,
|
||||
// and only when their bodies can be replayed. MaxRetries=0 disables retries.
|
||||
func (c *Client) Do(request *http.Request) (*http.Response, error) {
|
||||
var requestError error
|
||||
if c.closed.Load() {
|
||||
requestError = ErrClosed
|
||||
} else if request == nil || request.URL == nil {
|
||||
requestError = errors.New("httpc: nil request or URL")
|
||||
}
|
||||
if requestError != nil {
|
||||
// Match http.Client.Do's ownership rule even when rejection happens
|
||||
// before the underlying transport takes ownership of the body.
|
||||
if request != nil && request.Body != nil {
|
||||
_ = request.Body.Close()
|
||||
}
|
||||
return nil, requestError
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(request.Context(), c.config.Timeout)
|
||||
base := request.Clone(ctx)
|
||||
if base.Header == nil {
|
||||
base.Header = make(http.Header)
|
||||
}
|
||||
if !c.config.DisableRequestID && base.Header.Get("X-Request-ID") == "" {
|
||||
if id := logx.RequestID(ctx); id != "" {
|
||||
base.Header.Set("X-Request-ID", id)
|
||||
}
|
||||
}
|
||||
replayable := (base.Method == "" || base.Method == "GET" || base.Method == "HEAD" || base.Method == "OPTIONS") && (base.Body == nil || base.Body == http.NoBody || base.GetBody != nil)
|
||||
for attempt := 0; ; attempt++ {
|
||||
req := base.Clone(ctx)
|
||||
if attempt > 0 && base.Body != nil && base.Body != http.NoBody {
|
||||
body, err := base.GetBody()
|
||||
if err != nil {
|
||||
cancel()
|
||||
return nil, err
|
||||
}
|
||||
req.Body = body
|
||||
}
|
||||
started := time.Now()
|
||||
response, err := c.client.Do(req)
|
||||
status := 0
|
||||
if response != nil {
|
||||
status = response.StatusCode
|
||||
}
|
||||
if !c.config.DisableLogging {
|
||||
logx.InfoContext(ctx, "http.client", "method", req.Method, "host", req.URL.Hostname(), "status", status, "attempt", attempt+1, "durationMs", time.Since(started).Seconds()*1000, "failed", err != nil)
|
||||
}
|
||||
retry := replayable && attempt < c.config.MaxRetries && ctx.Err() == nil && (err != nil || status == 429 || status == 502 || status == 503 || status == 504)
|
||||
delay := c.config.Backoff
|
||||
for i := 0; i < attempt && delay < c.config.MaxBackoff; i++ {
|
||||
if delay > c.config.MaxBackoff/2 {
|
||||
delay = c.config.MaxBackoff
|
||||
} else {
|
||||
delay *= 2
|
||||
}
|
||||
}
|
||||
if response != nil && response.Header.Get("Retry-After") != "" {
|
||||
if value, ok := retryAfter(response.Header.Get("Retry-After")); ok {
|
||||
delay = value
|
||||
}
|
||||
} else if delay > 1 {
|
||||
delay = delay/2 + time.Duration(rand.Int63n(int64(delay/2)+1))
|
||||
}
|
||||
if delay > c.config.MaxBackoff {
|
||||
retry = false
|
||||
}
|
||||
if !retry {
|
||||
if err != nil {
|
||||
cancel()
|
||||
return response, err
|
||||
}
|
||||
response.Body = &responseBody{ReadCloser: response.Body, cancel: cancel}
|
||||
return response, nil
|
||||
}
|
||||
if response != nil {
|
||||
if response.ContentLength >= 0 && response.ContentLength <= 32<<10 {
|
||||
_, _ = io.Copy(io.Discard, response.Body)
|
||||
}
|
||||
_ = response.Body.Close()
|
||||
}
|
||||
timer := time.NewTimer(delay)
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
timer.Stop()
|
||||
cancel()
|
||||
return nil, ctx.Err()
|
||||
case <-timer.C:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type responseBody struct {
|
||||
io.ReadCloser
|
||||
cancel context.CancelFunc
|
||||
}
|
||||
|
||||
func (b *responseBody) Read(data []byte) (int, error) {
|
||||
n, err := b.ReadCloser.Read(data)
|
||||
if err != nil {
|
||||
b.cancel()
|
||||
}
|
||||
return n, err
|
||||
}
|
||||
func (b *responseBody) Close() error { defer b.cancel(); return b.ReadCloser.Close() }
|
||||
|
||||
func retryAfter(value string) (time.Duration, bool) {
|
||||
if n, err := strconv.ParseInt(strings.TrimSpace(value), 10, 32); err == nil && n >= 0 {
|
||||
return time.Duration(n) * time.Second, true
|
||||
}
|
||||
if at, err := http.ParseTime(value); err == nil {
|
||||
return max(time.Until(at), 0), true
|
||||
}
|
||||
return 0, false
|
||||
}
|
||||
|
||||
type StatusError struct{ StatusCode int }
|
||||
|
||||
func (e *StatusError) Error() string { return fmt.Sprintf("httpc: HTTP status %d", e.StatusCode) }
|
||||
|
||||
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
|
||||
// Error bodies, URLs, credentials and query strings are not included in logs.
|
||||
func (c *Client) JSON(ctx context.Context, method, url string, input, output any) error {
|
||||
if ctx == nil {
|
||||
return errors.New("httpc: nil context")
|
||||
}
|
||||
var body io.Reader
|
||||
if input != nil {
|
||||
data, err := json.Marshal(input)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body = bytes.NewReader(data)
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, method, url, body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if input != nil {
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
}
|
||||
response, err := c.Do(request)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return &StatusError{StatusCode: response.StatusCode}
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(response.Body, c.config.MaxResponseBytes+1))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if int64(len(data)) > c.config.MaxResponseBytes {
|
||||
return ErrResponseTooLarge
|
||||
}
|
||||
if output == nil || len(bytes.TrimSpace(data)) == 0 {
|
||||
return nil
|
||||
}
|
||||
return json.Unmarshal(data, output)
|
||||
}
|
||||
|
||||
func (c *Client) Close() error {
|
||||
c.closed.Store(true)
|
||||
if c.owned != nil {
|
||||
c.owned.CloseIdleConnections()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
156
httpc/client_test.go
Normal file
156
httpc/client_test.go
Normal file
@@ -0,0 +1,156 @@
|
||||
package httpc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type trackedBody struct {
|
||||
io.Reader
|
||||
closed bool
|
||||
}
|
||||
|
||||
func (b *trackedBody) Close() error { b.closed = true; return nil }
|
||||
|
||||
func TestRejectedRequestClosesBody(t *testing.T) {
|
||||
for _, closed := range []bool{false, true} {
|
||||
client, _ := New(Config{DisableLogging: true})
|
||||
body := &trackedBody{Reader: strings.NewReader("body")}
|
||||
request, _ := http.NewRequest("POST", "http://unused.invalid", body)
|
||||
if closed {
|
||||
client.Close()
|
||||
} else {
|
||||
request.URL = nil
|
||||
}
|
||||
_, err := client.Do(request)
|
||||
client.Close()
|
||||
if err == nil || !body.closed {
|
||||
t.Fatalf("error=%v bodyClosed=%v", err, body.closed)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRetryCorrelationPoolAndWriteSafety(t *testing.T) {
|
||||
var calls, connections atomic.Int32
|
||||
server := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
n := calls.Add(1)
|
||||
if r.Header.Get("X-Request-ID") != "request-1" {
|
||||
t.Error("missing correlation")
|
||||
}
|
||||
if n == 1 || r.Method == "POST" {
|
||||
w.WriteHeader(503)
|
||||
io.WriteString(w, "busy")
|
||||
return
|
||||
}
|
||||
io.WriteString(w, `{"ok":true}`)
|
||||
}))
|
||||
server.Config.ConnState = func(_ net.Conn, s http.ConnState) {
|
||||
if s == http.StateNew {
|
||||
connections.Add(1)
|
||||
}
|
||||
}
|
||||
server.Start()
|
||||
defer server.Close()
|
||||
client, err := New(Config{MaxRetries: 2, Backoff: time.Millisecond, DisableLogging: true})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Close()
|
||||
ctx := logx.WithRequestID(context.Background(), "request-1")
|
||||
var output struct {
|
||||
OK bool `json:"ok"`
|
||||
}
|
||||
if err := client.JSON(ctx, "GET", server.URL, nil, &output); err != nil || !output.OK {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 2 || connections.Load() != 1 {
|
||||
t.Fatalf("calls=%d connections=%d", calls.Load(), connections.Load())
|
||||
}
|
||||
if err := client.JSON(ctx, "POST", server.URL, map[string]string{"name": "value"}, nil); err == nil {
|
||||
t.Fatal("expected 503")
|
||||
}
|
||||
if calls.Load() != 3 {
|
||||
t.Fatal("POST was retried")
|
||||
}
|
||||
client.Close()
|
||||
if err := client.JSON(ctx, "GET", server.URL, nil, nil); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTotalDeadlineIncludesBackoffAndBody(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/retry" {
|
||||
w.Header().Set("Retry-After", "1")
|
||||
w.WriteHeader(503)
|
||||
return
|
||||
}
|
||||
w.WriteHeader(200)
|
||||
w.(http.Flusher).Flush()
|
||||
<-r.Context().Done()
|
||||
}))
|
||||
defer server.Close()
|
||||
client, _ := New(Config{Timeout: 40 * time.Millisecond, MaxRetries: 3, DisableLogging: true})
|
||||
defer client.Close()
|
||||
started := time.Now()
|
||||
err := client.JSON(context.Background(), "GET", server.URL+"/retry", nil, nil)
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(started) > time.Second {
|
||||
t.Fatal(err, time.Since(started))
|
||||
}
|
||||
req, _ := http.NewRequest("GET", server.URL+"/body", nil)
|
||||
response, err := client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if _, err := io.ReadAll(response.Body); err == nil {
|
||||
t.Fatal("body deadline was canceled too early")
|
||||
}
|
||||
}
|
||||
|
||||
func TestLimitsRedirectAndNonReplayableBody(t *testing.T) {
|
||||
var calls atomic.Int32
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
calls.Add(1)
|
||||
switch r.URL.Path {
|
||||
case "/redirect":
|
||||
http.Redirect(w, r, "/large", 302)
|
||||
case "/large":
|
||||
io.WriteString(w, strings.Repeat("x", 17))
|
||||
default:
|
||||
w.WriteHeader(503)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
client, _ := New(Config{MaxRetries: 2, MaxResponseBytes: 16, DisableLogging: true})
|
||||
defer client.Close()
|
||||
if err := client.JSON(context.Background(), "GET", server.URL+"/large", nil, nil); !errors.Is(err, ErrResponseTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var status *StatusError
|
||||
if err := client.JSON(context.Background(), "GET", server.URL+"/redirect", nil, nil); !errors.As(err, &status) || status.StatusCode != 302 {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 2 {
|
||||
t.Fatal("followed redirect")
|
||||
}
|
||||
req, _ := http.NewRequest("GET", server.URL, io.NopCloser(strings.NewReader("body")))
|
||||
res, err := client.Do(req)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
res.Body.Close()
|
||||
if calls.Load() != 3 {
|
||||
t.Fatal("non-replayable body was retried")
|
||||
}
|
||||
}
|
||||
105
httpx/clientIP.go
Normal file
105
httpx/clientIP.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package httpx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ClientIPResolver trusts X-Forwarded-For only from explicitly configured
|
||||
// proxy addresses. It is immutable and safe to share between requests.
|
||||
// The zero value (and a nil resolver) trusts no proxy.
|
||||
type ClientIPResolver struct {
|
||||
trusted []netip.Prefix
|
||||
}
|
||||
|
||||
// NewClientIPResolver accepts IP addresses or CIDR networks, never hostnames.
|
||||
func NewClientIPResolver(proxies ...string) (*ClientIPResolver, error) {
|
||||
r := &ClientIPResolver{}
|
||||
for _, value := range proxies {
|
||||
value = strings.TrimSpace(value)
|
||||
if address, err := netip.ParseAddr(value); err == nil && address.Zone() == "" {
|
||||
address = address.Unmap()
|
||||
r.trusted = append(r.trusted, netip.PrefixFrom(address, address.BitLen()))
|
||||
continue
|
||||
}
|
||||
prefix, err := netip.ParsePrefix(value)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("invalid trusted proxy %q: expected an IP address or CIDR", value)
|
||||
}
|
||||
if prefix.Addr().Is4In6() {
|
||||
if prefix.Bits() < 96 {
|
||||
return nil, fmt.Errorf("invalid trusted proxy %q: mapped IPv4 prefix must be at least /96", value)
|
||||
}
|
||||
prefix = netip.PrefixFrom(prefix.Addr().Unmap(), prefix.Bits()-96)
|
||||
}
|
||||
r.trusted = append(r.trusted, prefix.Masked())
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
|
||||
func (r *ClientIPResolver) trusts(address netip.Addr) bool {
|
||||
if r != nil {
|
||||
for _, prefix := range r.trusted {
|
||||
if prefix.Contains(address) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// ClientIP walks X-Forwarded-For from right to left, stopping at the first
|
||||
// untrusted address. Missing or malformed forwarding data falls back to the
|
||||
// socket peer. X-Real-IP, Forwarded and provider-specific headers are ignored.
|
||||
func (r *ClientIPResolver) ClientIP(request *http.Request) string {
|
||||
if request == nil {
|
||||
return ""
|
||||
}
|
||||
host := request.RemoteAddr
|
||||
if value, _, err := net.SplitHostPort(host); err == nil {
|
||||
host = value
|
||||
}
|
||||
peer, err := netip.ParseAddr(host)
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
peer = peer.WithZone("").Unmap()
|
||||
if !r.trusts(peer) {
|
||||
return peer.String()
|
||||
}
|
||||
values := request.Header.Values("X-Forwarded-For")
|
||||
var chain []netip.Addr
|
||||
for _, value := range values {
|
||||
for _, part := range strings.Split(value, ",") {
|
||||
address, err := netip.ParseAddr(strings.TrimSpace(part))
|
||||
if err != nil || address.Zone() != "" {
|
||||
return peer.String()
|
||||
}
|
||||
chain = append(chain, address.Unmap())
|
||||
}
|
||||
}
|
||||
current := peer
|
||||
for i := len(chain) - 1; i >= 0 && r.trusts(current); i-- {
|
||||
current = chain[i]
|
||||
}
|
||||
return current.String()
|
||||
}
|
||||
|
||||
type clientIPResolverKey struct{}
|
||||
|
||||
// WithClientIPResolver configures ClientIP for Context values derived from ctx.
|
||||
// Adapters should set this before running application middleware.
|
||||
func WithClientIPResolver(ctx context.Context, resolver *ClientIPResolver) context.Context {
|
||||
return context.WithValue(ctx, clientIPResolverKey{}, resolver)
|
||||
}
|
||||
|
||||
// ClientIP returns the verified client address without a port. By default it
|
||||
// returns the socket peer; App's trusted proxy configuration enables XFF.
|
||||
func (c *Context) ClientIP() string {
|
||||
resolver, _ := c.Context().Value(clientIPResolverKey{}).(*ClientIPResolver)
|
||||
return resolver.ClientIP(c.request)
|
||||
}
|
||||
76
httpx/clientIP_test.go
Normal file
76
httpx/clientIP_test.go
Normal file
@@ -0,0 +1,76 @@
|
||||
package httpx_test
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
func TestClientIPResolverUsesSocketPeerByDefault(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("192.0.2.10:443", "198.51.100.7")
|
||||
if got := resolver.ClientIP(request); got != "192.0.2.10" {
|
||||
t.Fatalf("untrusted forwarding header changed client IP: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIPResolverWalksTrustedForwardingChain(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("10.0.0.0/8")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("10.0.0.5:443", "198.51.100.7, 10.0.0.4")
|
||||
if got := resolver.ClientIP(request); got != "198.51.100.7" {
|
||||
t.Fatalf("expected original client IP, got %q", got)
|
||||
}
|
||||
|
||||
request = newClientIPRequest("10.0.0.5:443", "198.51.100.7, 203.0.113.9, 10.0.0.4")
|
||||
if got := resolver.ClientIP(request); got != "203.0.113.9" {
|
||||
t.Fatalf("expected first untrusted hop, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestClientIPResolverRejectsMalformedForwardingData(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("127.0.0.1:443", "198.51.100.7, not-an-ip")
|
||||
if got := resolver.ClientIP(request); got != "127.0.0.1" {
|
||||
t.Fatalf("malformed forwarding header should fall back to peer, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewClientIPResolverRejectsHostnames(t *testing.T) {
|
||||
if _, err := httpx.NewClientIPResolver("proxy.example.com"); err == nil || !strings.Contains(err.Error(), "invalid trusted proxy") {
|
||||
t.Fatalf("expected hostname validation error, got %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestContextClientIPUsesResolverFromRequestContext(t *testing.T) {
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := newClientIPRequest("127.0.0.1:443", "198.51.100.7")
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), resolver))
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
if got := ctx.ClientIP(); got != "198.51.100.7" {
|
||||
t.Fatalf("expected context client IP, got %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func newClientIPRequest(remote, forwarded string) *http.Request {
|
||||
request := httptest.NewRequest(http.MethodGet, "http://example.test/", nil)
|
||||
request.RemoteAddr = remote
|
||||
if forwarded != "" {
|
||||
request.Header.Set("X-Forwarded-For", forwarded)
|
||||
}
|
||||
return request
|
||||
}
|
||||
@@ -18,9 +18,9 @@ type Handler func(*Context) error
|
||||
type Middleware func(Handler) Handler
|
||||
|
||||
// Router is the small contract the application needs from a network layer.
|
||||
// The default implementation is backed by Gin, but another router or a
|
||||
// future GoOmni transport can implement this interface without changing
|
||||
// business handlers. Register routes before serving requests. Implementations
|
||||
// The default implementation is backed by the standard library, but another
|
||||
// router or a future GoOmni transport can implement this interface without
|
||||
// changing business handlers. Register routes before serving requests. Implementations
|
||||
// should pass transport/transporttest.Run; see transport/README.md for the
|
||||
// portable path, method, fallback and context-propagation rules.
|
||||
type Router interface {
|
||||
@@ -78,7 +78,8 @@ func RoutePattern(ctx context.Context) string {
|
||||
}
|
||||
|
||||
// Context provides the business-facing request and response API. It contains
|
||||
// only standard-library HTTP types, so it remains usable when Gin is replaced.
|
||||
// only standard-library HTTP types, so it remains usable when the network
|
||||
// implementation is replaced.
|
||||
type Context struct {
|
||||
request *http.Request
|
||||
writer http.ResponseWriter
|
||||
@@ -116,6 +117,37 @@ func (c *Context) Context() context.Context {
|
||||
// Prefer the transport-neutral methods on Context for portable handlers.
|
||||
func (c *Context) Raw() http.ResponseWriter { return c.writer }
|
||||
|
||||
// Clone gives a buffering middleware its own response state and values map.
|
||||
// Stored values themselves are shared; middleware must not mutate them concurrently.
|
||||
func (c *Context) Clone(ctx context.Context, writer http.ResponseWriter) *Context {
|
||||
clone := NewContext(c.request.Clone(ctx), writer, c.params)
|
||||
for key, value := range c.values {
|
||||
clone.values[key] = value
|
||||
}
|
||||
return clone
|
||||
}
|
||||
|
||||
// WriteResponse commits a buffered response through GoOmni's response tracking.
|
||||
func (c *Context) WriteResponse(status int, headers http.Header, body []byte) error {
|
||||
if c.responded {
|
||||
return nil
|
||||
}
|
||||
if c.writer == nil {
|
||||
return apierrors.Internal(nil)
|
||||
}
|
||||
for key, values := range headers {
|
||||
c.writer.Header()[key] = append([]string(nil), values...)
|
||||
}
|
||||
c.responded, c.status = true, status
|
||||
c.writer.WriteHeader(status)
|
||||
if len(body) > 0 {
|
||||
if _, err := c.writer.Write(body); err != nil {
|
||||
return apierrors.Internal(err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c *Context) Param(name string) string { return c.params[name] }
|
||||
|
||||
func (c *Context) Query(name string) string {
|
||||
|
||||
696
internal/perfcheck/main.go
Normal file
696
internal/perfcheck/main.go
Normal file
@@ -0,0 +1,696 @@
|
||||
// perfcheck compares real TCP servers in separate processes. It is a local
|
||||
// measurement tool, not part of the framework's public CLI or server API.
|
||||
package main
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"crypto/x509/pkix"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"math/big"
|
||||
"math/bits"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"runtime"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
const expectedBody = `{"code":"OK","message":"success","data":{"status":"ok"},"requestId":"perfCheck"}`
|
||||
|
||||
type snapshot struct {
|
||||
Address string `json:"address,omitempty"`
|
||||
HeapAlloc uint64 `json:"heapAlloc"`
|
||||
HeapInuse uint64 `json:"heapInuse"`
|
||||
StackInuse uint64 `json:"stackInuse"`
|
||||
TotalAlloc uint64 `json:"totalAlloc"`
|
||||
Mallocs uint64 `json:"mallocs"`
|
||||
NumGC uint32 `json:"numGc"`
|
||||
Goroutines int `json:"goroutines"`
|
||||
Connections int64 `json:"connections"`
|
||||
Accepted int64 `json:"accepted"`
|
||||
ActiveRequests int64 `json:"activeRequests,omitempty"`
|
||||
FileDescriptors int `json:"fileDescriptors,omitempty"`
|
||||
}
|
||||
|
||||
type result struct {
|
||||
Adapter string `json:"adapter"`
|
||||
Scenario string `json:"scenario"`
|
||||
Concurrency int `json:"concurrency"`
|
||||
Run int `json:"run"`
|
||||
Requests uint64 `json:"requests,omitempty"`
|
||||
Errors uint64 `json:"errors"`
|
||||
FirstError string `json:"firstError,omitempty"`
|
||||
ElapsedSeconds float64 `json:"elapsedSeconds,omitempty"`
|
||||
RequestsPerSecond float64 `json:"requestsPerSecond,omitempty"`
|
||||
P50Millis float64 `json:"p50Millis,omitempty"`
|
||||
P95Millis float64 `json:"p95Millis,omitempty"`
|
||||
P99Millis float64 `json:"p99Millis,omitempty"`
|
||||
ServerBytesPerRequest float64 `json:"serverBytesPerRequest,omitempty"`
|
||||
ServerAllocsPerRequest float64 `json:"serverAllocsPerRequest,omitempty"`
|
||||
SampledPeakHeapAlloc uint64 `json:"sampledPeakHeapAlloc,omitempty"`
|
||||
SampledPeakFileDescriptors int `json:"sampledPeakFileDescriptors,omitempty"`
|
||||
IdleHeapDelta int64 `json:"idleHeapDelta,omitempty"`
|
||||
IdleHeapPerConnection float64 `json:"idleHeapPerConnection,omitempty"`
|
||||
Before snapshot `json:"before"`
|
||||
During snapshot `json:"during"`
|
||||
After snapshot `json:"after"`
|
||||
AfterReclaim snapshot `json:"afterReclaim"`
|
||||
}
|
||||
|
||||
type report struct {
|
||||
CreatedAt string `json:"createdAt"`
|
||||
GoVersion string `json:"goVersion"`
|
||||
Platform string `json:"platform"`
|
||||
GOMAXPROCS int `json:"gomaxprocs"`
|
||||
Duration string `json:"duration"`
|
||||
TLS bool `json:"tls"`
|
||||
Notes []string `json:"notes"`
|
||||
Results []result `json:"results"`
|
||||
}
|
||||
|
||||
type trackedListener struct {
|
||||
net.Listener
|
||||
active, accepted atomic.Int64
|
||||
}
|
||||
|
||||
type trackedConn struct {
|
||||
net.Conn
|
||||
once sync.Once
|
||||
active *atomic.Int64
|
||||
}
|
||||
|
||||
func (l *trackedListener) Accept() (net.Conn, error) {
|
||||
c, err := l.Listener.Accept()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
l.active.Add(1)
|
||||
l.accepted.Add(1)
|
||||
return &trackedConn{Conn: c, active: &l.active}, nil
|
||||
}
|
||||
|
||||
func (c *trackedConn) Close() error {
|
||||
err := c.Conn.Close()
|
||||
c.once.Do(func() { c.active.Add(-1) })
|
||||
return err
|
||||
}
|
||||
|
||||
func testTLSConfig() (*tls.Config, error) {
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 120))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
now := time.Now()
|
||||
template := &x509.Certificate{
|
||||
SerialNumber: serial,
|
||||
Subject: pkix.Name{CommonName: "localhost"},
|
||||
NotBefore: now.Add(-time.Minute),
|
||||
NotAfter: now.Add(time.Hour),
|
||||
KeyUsage: x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature,
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
||||
DNSNames: []string{"localhost"},
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &tls.Config{
|
||||
Certificates: []tls.Certificate{{Certificate: [][]byte{der}, PrivateKey: key}},
|
||||
MinVersion: tls.VersionTLS12,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func main() {
|
||||
serverMode := flag.String("server", "", "internal server subprocess: nethttp or native")
|
||||
tlsEnabled := flag.Bool("tls", false, "run the comparison over TLS")
|
||||
duration := flag.Duration("duration", 2*time.Second, "measured duration per traffic case")
|
||||
concurrencyText := flag.String("concurrency", "1,32,128", "traffic concurrency levels")
|
||||
idleText := flag.String("idle", "128,1024", "established idle Keep-Alive connection levels")
|
||||
runs := flag.Int("count", 3, "repetitions; adapter order alternates each repetition")
|
||||
output := flag.String("output", "", "optional JSON report path")
|
||||
flag.Parse()
|
||||
var err error
|
||||
if *serverMode != "" {
|
||||
err = serve(*serverMode, *tlsEnabled)
|
||||
} else {
|
||||
err = compare(*duration, *concurrencyText, *idleText, *runs, *output, *tlsEnabled)
|
||||
}
|
||||
if err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func serve(adapter string, tlsEnabled bool) error {
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tracked := &trackedListener{Listener: listener}
|
||||
var serveListener net.Listener = tracked
|
||||
if tlsEnabled {
|
||||
config, configErr := testTLSConfig()
|
||||
if configErr != nil {
|
||||
listener.Close()
|
||||
return configErr
|
||||
}
|
||||
serveListener = tls.NewListener(tracked, config)
|
||||
}
|
||||
application := omni.New()
|
||||
application.GET("/health", func(c *omni.Context) error {
|
||||
return c.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
application.GET("/slow", func(c *omni.Context) error {
|
||||
timer := time.NewTimer(2 * time.Second)
|
||||
defer timer.Stop()
|
||||
select {
|
||||
case <-timer.C:
|
||||
return c.OK(map[string]string{"status": "slow"})
|
||||
case <-c.Context().Done():
|
||||
return c.Context().Err()
|
||||
}
|
||||
})
|
||||
done := make(chan error, 1)
|
||||
var shutdown func(context.Context) error
|
||||
var stats func() native.Stats
|
||||
switch adapter {
|
||||
case "native":
|
||||
server := native.New(application)
|
||||
server.MaxConnections = 8192
|
||||
server.MaxConcurrentRequests = 4096
|
||||
shutdown = server.Shutdown
|
||||
stats = server.Stats
|
||||
go func() { done <- server.Serve(context.Background(), serveListener) }()
|
||||
case "nethttp":
|
||||
server := &http.Server{
|
||||
Handler: application, ReadHeaderTimeout: 5 * time.Second,
|
||||
ReadTimeout: 30 * time.Second, WriteTimeout: 30 * time.Second,
|
||||
IdleTimeout: time.Minute, MaxHeaderBytes: 1 << 20,
|
||||
}
|
||||
shutdown = server.Shutdown
|
||||
go func() {
|
||||
err := server.Serve(serveListener)
|
||||
if errors.Is(err, http.ErrServerClosed) {
|
||||
err = nil
|
||||
}
|
||||
done <- err
|
||||
}()
|
||||
default:
|
||||
listener.Close()
|
||||
return fmt.Errorf("unknown server %q", adapter)
|
||||
}
|
||||
stop := func() error {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := shutdown(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
return err
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
}
|
||||
}
|
||||
encoder := json.NewEncoder(os.Stdout)
|
||||
if err := encoder.Encode(snapshot{Address: listener.Addr().String()}); err != nil {
|
||||
return err
|
||||
}
|
||||
decoder := json.NewDecoder(os.Stdin)
|
||||
for {
|
||||
var command string
|
||||
if err := decoder.Decode(&command); err != nil || command == "stop" {
|
||||
return stop()
|
||||
}
|
||||
if command == "gc" {
|
||||
runtime.GC()
|
||||
}
|
||||
var memory runtime.MemStats
|
||||
runtime.ReadMemStats(&memory)
|
||||
connections, accepted, activeRequests := tracked.active.Load(), tracked.accepted.Load(), int64(0)
|
||||
if stats != nil {
|
||||
value := stats()
|
||||
connections, accepted, activeRequests = value.ActiveConnections, int64(value.AcceptedConnections), value.ActiveRequests
|
||||
}
|
||||
if err := encoder.Encode(snapshot{
|
||||
HeapAlloc: memory.HeapAlloc, HeapInuse: memory.HeapInuse,
|
||||
StackInuse: memory.StackInuse, TotalAlloc: memory.TotalAlloc,
|
||||
Mallocs: memory.Mallocs, NumGC: memory.NumGC,
|
||||
Goroutines: runtime.NumGoroutine(), Connections: connections, Accepted: accepted, ActiveRequests: activeRequests,
|
||||
FileDescriptors: openFileDescriptorCount(),
|
||||
}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func openFileDescriptorCount() int {
|
||||
entries, err := os.ReadDir("/proc/self/fd")
|
||||
if err != nil {
|
||||
return 0
|
||||
}
|
||||
return len(entries)
|
||||
}
|
||||
|
||||
type child struct {
|
||||
cmd *exec.Cmd
|
||||
input io.WriteCloser
|
||||
encoder *json.Encoder
|
||||
decoder *json.Decoder
|
||||
address string
|
||||
tls bool
|
||||
cancel context.CancelFunc
|
||||
closed bool
|
||||
}
|
||||
|
||||
func startChild(adapter string) (*child, error) {
|
||||
return startChildMode(adapter, false)
|
||||
}
|
||||
|
||||
func startChildMode(adapter string, tlsEnabled bool) (*child, error) {
|
||||
executable, err := os.Executable()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// This bounds even a broken server or protocol read; it is not a load
|
||||
// duration. Keep it above the supported 30-minute soak plus drain time.
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 35*time.Minute)
|
||||
args := []string{"--server", adapter}
|
||||
if tlsEnabled {
|
||||
args = append(args, "--tls")
|
||||
}
|
||||
c := &child{cmd: exec.CommandContext(ctx, executable, args...), cancel: cancel, tls: tlsEnabled}
|
||||
c.cmd.Stderr = os.Stderr
|
||||
if c.input, err = c.cmd.StdinPipe(); err != nil {
|
||||
cancel()
|
||||
return nil, err
|
||||
}
|
||||
output, err := c.cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
cancel()
|
||||
c.input.Close()
|
||||
return nil, err
|
||||
}
|
||||
if err := c.cmd.Start(); err != nil {
|
||||
cancel()
|
||||
c.input.Close()
|
||||
output.Close()
|
||||
return nil, err
|
||||
}
|
||||
c.encoder, c.decoder = json.NewEncoder(c.input), json.NewDecoder(output)
|
||||
var ready snapshot
|
||||
if err := c.decoder.Decode(&ready); err != nil {
|
||||
cancel()
|
||||
c.cmd.Wait()
|
||||
return nil, err
|
||||
}
|
||||
c.address = ready.Address
|
||||
return c, nil
|
||||
}
|
||||
|
||||
func (c *child) snapshot(gc bool) (snapshot, error) {
|
||||
command := "snapshot"
|
||||
if gc {
|
||||
command = "gc"
|
||||
}
|
||||
var value snapshot
|
||||
if err := c.encoder.Encode(command); err != nil {
|
||||
return value, err
|
||||
}
|
||||
err := c.decoder.Decode(&value)
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (c *child) close() error {
|
||||
if c.closed {
|
||||
return nil
|
||||
}
|
||||
c.closed = true
|
||||
defer c.cancel()
|
||||
err := c.encoder.Encode("stop")
|
||||
c.input.Close()
|
||||
return errors.Join(err, c.cmd.Wait())
|
||||
}
|
||||
|
||||
// Histograms use 16 subdivisions per power of two. Percentiles are bucket
|
||||
// upper bounds (at most about 6.25% rounding), with bounded client memory.
|
||||
type histogram [1024]uint64
|
||||
|
||||
func (h *histogram) add(d time.Duration) {
|
||||
n := uint64(max(d.Nanoseconds(), 1))
|
||||
exponent := bits.Len64(n) - 1
|
||||
base := uint64(1) << exponent
|
||||
sub := (n - base) * 16 / base
|
||||
h[exponent*16+int(sub)]++
|
||||
}
|
||||
|
||||
func (h *histogram) percentile(count uint64, percent uint64) float64 {
|
||||
target := (count*percent + 99) / 100
|
||||
if target == 0 {
|
||||
return 0
|
||||
}
|
||||
var seen uint64
|
||||
for bucket, n := range h {
|
||||
seen += n
|
||||
if seen >= target {
|
||||
base := uint64(1) << (bucket / 16)
|
||||
return float64(base+uint64(bucket%16+1)*base/16) / float64(time.Millisecond)
|
||||
}
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
func exchange(client *http.Client, address string) error {
|
||||
return exchangeMode(client, address, false)
|
||||
}
|
||||
|
||||
func exchangeMode(client *http.Client, address string, tlsEnabled bool) error {
|
||||
scheme := "http"
|
||||
if tlsEnabled {
|
||||
scheme = "https"
|
||||
}
|
||||
req, err := http.NewRequest(http.MethodGet, scheme+"://"+address+"/health", nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("X-Request-ID", "perfCheck")
|
||||
response, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body, readErr := io.ReadAll(io.LimitReader(response.Body, 4096))
|
||||
closeErr := response.Body.Close()
|
||||
if response.StatusCode != 200 || response.Header.Get("X-Request-ID") != "perfCheck" || strings.TrimSpace(string(body)) != expectedBody {
|
||||
return fmt.Errorf("invalid response status=%d body=%q", response.StatusCode, body)
|
||||
}
|
||||
return errors.Join(readErr, closeErr)
|
||||
}
|
||||
|
||||
func traffic(c *child, concurrency int, duration time.Duration) (result, error) {
|
||||
r := result{Scenario: "traffic", Concurrency: concurrency}
|
||||
transport := &http.Transport{
|
||||
MaxIdleConns: concurrency, MaxIdleConnsPerHost: concurrency,
|
||||
MaxConnsPerHost: concurrency, IdleConnTimeout: time.Minute,
|
||||
}
|
||||
if c.tls {
|
||||
transport.TLSClientConfig = &tls.Config{InsecureSkipVerify: true} // test certificate is self-signed
|
||||
}
|
||||
client := &http.Client{Transport: transport, Timeout: 5 * time.Second}
|
||||
defer client.CloseIdleConnections()
|
||||
// Warm every worker before forcing a baseline GC in the server.
|
||||
warmErrors := make(chan error, concurrency)
|
||||
for i := 0; i < concurrency; i++ {
|
||||
go func() { warmErrors <- exchangeMode(client, c.address, c.tls) }()
|
||||
}
|
||||
for i := 0; i < concurrency; i++ {
|
||||
if err := <-warmErrors; err != nil {
|
||||
return r, err
|
||||
}
|
||||
}
|
||||
var err error
|
||||
if r.Before, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
type workerResult struct {
|
||||
latencies histogram
|
||||
count, errors uint64
|
||||
firstError string
|
||||
}
|
||||
workers := make([]workerResult, concurrency)
|
||||
start := make(chan struct{})
|
||||
var group sync.WaitGroup
|
||||
var deadline time.Time
|
||||
for i := range workers {
|
||||
group.Add(1)
|
||||
go func(w *workerResult) {
|
||||
defer group.Done()
|
||||
<-start
|
||||
for time.Now().Before(deadline) {
|
||||
began := time.Now()
|
||||
err := exchangeMode(client, c.address, c.tls)
|
||||
w.latencies.add(time.Since(began))
|
||||
w.count++
|
||||
if err != nil {
|
||||
w.errors++
|
||||
if w.firstError == "" {
|
||||
w.firstError = err.Error()
|
||||
}
|
||||
}
|
||||
}
|
||||
}(&workers[i])
|
||||
}
|
||||
done := make(chan struct{})
|
||||
go func() { group.Wait(); close(done) }()
|
||||
begin := time.Now()
|
||||
deadline = begin.Add(duration)
|
||||
close(start)
|
||||
ticker := time.NewTicker(200 * time.Millisecond)
|
||||
defer ticker.Stop()
|
||||
var sampleErr error
|
||||
loading := true
|
||||
for loading {
|
||||
select {
|
||||
case <-done:
|
||||
loading = false
|
||||
case <-ticker.C:
|
||||
s, err := c.snapshot(false)
|
||||
if err != nil {
|
||||
sampleErr = err
|
||||
} else if s.HeapAlloc > r.SampledPeakHeapAlloc {
|
||||
r.SampledPeakHeapAlloc = s.HeapAlloc
|
||||
}
|
||||
if s.FileDescriptors > r.SampledPeakFileDescriptors {
|
||||
r.SampledPeakFileDescriptors = s.FileDescriptors
|
||||
}
|
||||
}
|
||||
}
|
||||
r.ElapsedSeconds = time.Since(begin).Seconds()
|
||||
if sampleErr != nil {
|
||||
return r, sampleErr
|
||||
}
|
||||
var latencies histogram
|
||||
for _, w := range workers {
|
||||
r.Requests += w.count
|
||||
r.Errors += w.errors
|
||||
if r.FirstError == "" {
|
||||
r.FirstError = w.firstError
|
||||
}
|
||||
for i, n := range w.latencies {
|
||||
latencies[i] += n
|
||||
}
|
||||
}
|
||||
if r.During, err = c.snapshot(false); err != nil {
|
||||
return r, err
|
||||
}
|
||||
r.SampledPeakHeapAlloc = max(r.SampledPeakHeapAlloc, r.During.HeapAlloc)
|
||||
r.SampledPeakFileDescriptors = max(r.SampledPeakFileDescriptors, r.During.FileDescriptors)
|
||||
if r.Requests == 0 {
|
||||
return r, errors.New("no requests completed")
|
||||
}
|
||||
r.RequestsPerSecond = float64(r.Requests-r.Errors) / r.ElapsedSeconds
|
||||
r.P50Millis = latencies.percentile(r.Requests, 50)
|
||||
r.P95Millis = latencies.percentile(r.Requests, 95)
|
||||
r.P99Millis = latencies.percentile(r.Requests, 99)
|
||||
r.ServerBytesPerRequest = float64(r.During.TotalAlloc-r.Before.TotalAlloc) / float64(r.Requests)
|
||||
r.ServerAllocsPerRequest = float64(r.During.Mallocs-r.Before.Mallocs) / float64(r.Requests)
|
||||
client.CloseIdleConnections()
|
||||
r.After, err = drained(c)
|
||||
if err == nil {
|
||||
r.AfterReclaim, err = c.snapshot(true)
|
||||
}
|
||||
return r, err
|
||||
}
|
||||
|
||||
func idle(c *child, count int) (result, error) {
|
||||
r := result{Scenario: "idle", Concurrency: count}
|
||||
var err error
|
||||
if r.Before, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
connections := make([]net.Conn, 0, count)
|
||||
defer func() {
|
||||
for _, conn := range connections {
|
||||
conn.Close()
|
||||
}
|
||||
}()
|
||||
for i := 0; i < count; i++ {
|
||||
conn, err := net.DialTimeout("tcp", c.address, 5*time.Second)
|
||||
if err != nil {
|
||||
return r, err
|
||||
}
|
||||
if c.tls {
|
||||
secure := tls.Client(conn, &tls.Config{InsecureSkipVerify: true, ServerName: "localhost"}) // test certificate is self-signed
|
||||
conn = secure
|
||||
}
|
||||
connections = append(connections, conn)
|
||||
conn.SetDeadline(time.Now().Add(5 * time.Second))
|
||||
if _, err := io.WriteString(conn, "GET /health HTTP/1.1\r\nHost: localhost\r\nX-Request-ID: perfCheck\r\n\r\n"); err != nil {
|
||||
return r, err
|
||||
}
|
||||
response, err := http.ReadResponse(bufio.NewReader(conn), &http.Request{Method: http.MethodGet})
|
||||
if err != nil {
|
||||
return r, err
|
||||
}
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || strings.TrimSpace(string(body)) != expectedBody || response.Close {
|
||||
return r, fmt.Errorf("invalid Keep-Alive response: status=%d error=%v", response.StatusCode, readErr)
|
||||
}
|
||||
conn.SetDeadline(time.Time{})
|
||||
}
|
||||
if r.During, err = c.snapshot(true); err != nil {
|
||||
return r, err
|
||||
}
|
||||
if r.During.Connections != int64(count) {
|
||||
return r, fmt.Errorf("idle connections: want %d, have %d", count, r.During.Connections)
|
||||
}
|
||||
if r.During.FileDescriptors > 0 && r.Before.FileDescriptors > 0 && r.During.FileDescriptors < r.Before.FileDescriptors+count {
|
||||
return r, fmt.Errorf("idle file descriptors: want at least %d, have %d", r.Before.FileDescriptors+count, r.During.FileDescriptors)
|
||||
}
|
||||
r.IdleHeapDelta = int64(r.During.HeapAlloc) - int64(r.Before.HeapAlloc)
|
||||
r.IdleHeapPerConnection = float64(r.IdleHeapDelta) / float64(count)
|
||||
for _, conn := range connections {
|
||||
conn.Close()
|
||||
}
|
||||
connections = nil
|
||||
r.After, err = drained(c)
|
||||
if err == nil {
|
||||
r.AfterReclaim, err = c.snapshot(true)
|
||||
}
|
||||
return r, err
|
||||
}
|
||||
|
||||
func drained(c *child) (snapshot, error) {
|
||||
deadline := time.Now().Add(5 * time.Second)
|
||||
for {
|
||||
s, err := c.snapshot(false)
|
||||
if err != nil {
|
||||
return s, err
|
||||
}
|
||||
if s.Connections == 0 {
|
||||
return c.snapshot(true)
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
return s, fmt.Errorf("%d connections remained after clients closed", s.Connections)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
}
|
||||
|
||||
func levels(text string) ([]int, error) {
|
||||
var values []int
|
||||
if text == "" {
|
||||
return values, nil
|
||||
}
|
||||
for _, item := range strings.Split(text, ",") {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(item))
|
||||
if err != nil || n < 1 || n > 4096 {
|
||||
return nil, fmt.Errorf("invalid level %q (expected 1..4096)", item)
|
||||
}
|
||||
values = append(values, n)
|
||||
}
|
||||
return values, nil
|
||||
}
|
||||
|
||||
func compare(duration time.Duration, concurrencyText, idleText string, count int, output string, tlsEnabled bool) error {
|
||||
concurrency, err := levels(concurrencyText)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
idleLevels, err := levels(idleText)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if count < 1 || duration < 200*time.Millisecond || duration > 30*time.Minute || len(concurrency)+len(idleLevels) == 0 {
|
||||
return errors.New("count must be positive, duration 200ms..30m, and at least one scenario selected")
|
||||
}
|
||||
report := report{
|
||||
CreatedAt: time.Now().UTC().Format(time.RFC3339), GoVersion: runtime.Version(),
|
||||
Platform: runtime.GOOS + "/" + runtime.GOARCH, GOMAXPROCS: runtime.GOMAXPROCS(0), Duration: duration.String(), TLS: tlsEnabled,
|
||||
Notes: []string{
|
||||
"Loopback HTTP/1.1 health response, fixed requestId, optional TLS, no access logs or database.",
|
||||
"Each case starts a fresh server process; load client runs in the parent process on the same machine.",
|
||||
"Closed-loop concurrency; throughput is successful responses / actual elapsed time, including drain.",
|
||||
"Latency percentiles include failures and are histogram upper bounds with <=6.25% rounding.",
|
||||
"Memory is server-only runtime.MemStats (not RSS or kernel socket memory); sampling interval 200ms.",
|
||||
"FileDescriptors is read from /proc/self/fd on Linux and omitted on other platforms.",
|
||||
"Idle memory measured after a forced GC; delta includes heap growth amortized across connections, excluding stacks.",
|
||||
"After is after closing clients and one forced GC; afterReclaim is a second GC to observe pooled-cache release.",
|
||||
"Native limits: 8192 connections / 4096 active requests. These cases do not benchmark overload rejection.",
|
||||
},
|
||||
}
|
||||
var caseErrors []error
|
||||
for run := 1; run <= count; run++ {
|
||||
adapters := []string{"nethttp", "native"}
|
||||
if run%2 == 0 {
|
||||
adapters[0], adapters[1] = adapters[1], adapters[0]
|
||||
}
|
||||
for _, scenario := range []struct {
|
||||
name string
|
||||
levels []int
|
||||
}{{"traffic", concurrency}, {"idle", idleLevels}} {
|
||||
for _, level := range scenario.levels {
|
||||
for _, adapter := range adapters {
|
||||
c, err := startChildMode(adapter, tlsEnabled)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var value result
|
||||
if scenario.name == "traffic" {
|
||||
value, err = traffic(c, level, duration)
|
||||
} else {
|
||||
value, err = idle(c, level)
|
||||
}
|
||||
err = errors.Join(err, c.close())
|
||||
value.Adapter, value.Run = adapter, run
|
||||
if err != nil {
|
||||
value.Errors++
|
||||
value.FirstError = err.Error()
|
||||
}
|
||||
report.Results = append(report.Results, value)
|
||||
if value.Errors > 0 {
|
||||
caseErrors = append(caseErrors, fmt.Errorf("%s %s/%d run %d: %s", adapter, scenario.name, level, run, value.FirstError))
|
||||
}
|
||||
if scenario.name == "traffic" {
|
||||
fmt.Printf("%s concurrency=%d run=%d: %.0f req/s p95=%.3fms p99=%.3fms server=%.0f B/req errors=%d\n", adapter, level, run, value.RequestsPerSecond, value.P95Millis, value.P99Millis, value.ServerBytesPerRequest, value.Errors)
|
||||
} else {
|
||||
fmt.Printf("%s idle=%d run=%d: server heap delta=%.2f MiB (%.0f B/conn), goroutines=%d->%d->%d, fds=%d->%d->%d errors=%d\n", adapter, level, run, float64(value.IdleHeapDelta)/(1<<20), value.IdleHeapPerConnection, value.Before.Goroutines, value.During.Goroutines, value.After.Goroutines, value.Before.FileDescriptors, value.During.FileDescriptors, value.After.FileDescriptors, value.Errors)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if output != "" {
|
||||
data, err := json.MarshalIndent(report, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.WriteFile(output, append(data, '\n'), 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("Report: %s\n", output)
|
||||
}
|
||||
return errors.Join(caseErrors...)
|
||||
}
|
||||
75
internal/perfcheck/main_test.go
Normal file
75
internal/perfcheck/main_test.go
Normal file
@@ -0,0 +1,75 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// A load generator must never report a fast failure page as good throughput.
|
||||
func TestExchangeRejectsInvalidResponses(t *testing.T) {
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
status int
|
||||
body string
|
||||
requestID string
|
||||
truncated bool
|
||||
wantOK bool
|
||||
}{
|
||||
{"valid", 200, expectedBody, "perfCheck", false, true},
|
||||
{"overload", 503, "server busy", "perfCheck", false, false},
|
||||
{"wrongPayload", 200, `{"data":"wrong"}`, "perfCheck", false, false},
|
||||
{"missingRequestId", 200, expectedBody, "", false, false},
|
||||
{"truncated", 200, expectedBody, "perfCheck", true, false},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get("X-Request-ID") != "perfCheck" {
|
||||
t.Error("client did not send fixed requestId")
|
||||
}
|
||||
w.Header().Set("X-Request-ID", tt.requestID)
|
||||
if tt.truncated {
|
||||
w.Header().Set("Content-Length", "4096")
|
||||
}
|
||||
w.WriteHeader(tt.status)
|
||||
_, _ = w.Write([]byte(tt.body))
|
||||
}))
|
||||
defer server.Close()
|
||||
err := exchange(server.Client(), strings.TrimPrefix(server.URL, "http://"))
|
||||
if (err == nil) != tt.wantOK {
|
||||
t.Fatalf("valid=%v, error=%v", tt.wantOK, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestExchangeModeTLS(t *testing.T) {
|
||||
server := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set("X-Request-ID", "perfCheck")
|
||||
w.WriteHeader(http.StatusOK)
|
||||
_, _ = w.Write([]byte(expectedBody))
|
||||
}))
|
||||
defer server.Close()
|
||||
if err := exchangeMode(server.Client(), strings.TrimPrefix(server.URL, "https://"), true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPercentileBounds(t *testing.T) {
|
||||
var h histogram
|
||||
// Known ranks, including a tail much slower than the median.
|
||||
for i := 1; i <= 100; i++ {
|
||||
h.add(time.Duration(i) * time.Millisecond)
|
||||
}
|
||||
for _, p := range []uint64{50, 95, 99} {
|
||||
value := h.percentile(100, p)
|
||||
if value < float64(p) || value > float64(p)*1.0625 {
|
||||
t.Fatalf("p%d=%gms outside advertised bucket rounding", p, value)
|
||||
}
|
||||
}
|
||||
if value := h.percentile(0, 95); value != 0 {
|
||||
t.Fatalf("empty histogram returned %g", value)
|
||||
}
|
||||
}
|
||||
63
internal/storagefixture/server.go
Normal file
63
internal/storagefixture/server.go
Normal file
@@ -0,0 +1,63 @@
|
||||
// Package storagefixture supplies a bounded protocol fixture, not a cloud service.
|
||||
package storagefixture
|
||||
|
||||
import (
|
||||
"crypto/md5"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type value struct {
|
||||
data []byte
|
||||
contentType string
|
||||
etag string
|
||||
}
|
||||
|
||||
func Handler() http.Handler {
|
||||
var mu sync.Mutex
|
||||
values := map[string]value{}
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
if r.Header.Get("Authorization") == "" && r.URL.Query().Get("X-Amz-Signature") == "" && r.URL.Query().Get("x-oss-signature") == "" {
|
||||
http.Error(w, "signature required", 403)
|
||||
return
|
||||
}
|
||||
key := r.URL.Path
|
||||
switch r.Method {
|
||||
case "PUT":
|
||||
data, err := io.ReadAll(io.LimitReader(r.Body, 1025))
|
||||
if err != nil || len(data) > 1024 {
|
||||
http.Error(w, "too large", 413)
|
||||
return
|
||||
}
|
||||
v := value{data: data, contentType: r.Header.Get("Content-Type"), etag: fmt.Sprintf("%x", md5.Sum(data))}
|
||||
values[key] = v
|
||||
w.Header().Set("ETag", `"`+v.etag+`"`)
|
||||
w.WriteHeader(200)
|
||||
case "DELETE":
|
||||
delete(values, key)
|
||||
w.WriteHeader(204)
|
||||
case "GET", "HEAD":
|
||||
v, ok := values[key]
|
||||
if !ok {
|
||||
w.Header().Set("Content-Type", "application/xml")
|
||||
w.WriteHeader(404)
|
||||
fmt.Fprint(w, `<Error><Code>NoSuchKey</Code><Message>not found</Message></Error>`)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Content-Length", fmt.Sprint(len(v.data)))
|
||||
w.Header().Set("Content-Type", v.contentType)
|
||||
w.Header().Set("ETag", `"`+v.etag+`"`)
|
||||
w.Header().Set("Last-Modified", time.Now().UTC().Format(http.TimeFormat))
|
||||
if r.Method == "GET" {
|
||||
w.Write(v.data)
|
||||
}
|
||||
default:
|
||||
http.Error(w, "unsupported", 405)
|
||||
}
|
||||
})
|
||||
}
|
||||
61
internal/testutil/tls.go
Normal file
61
internal/testutil/tls.go
Normal file
@@ -0,0 +1,61 @@
|
||||
// Package testutil contains fixtures shared by transport integration tests.
|
||||
package testutil
|
||||
|
||||
import (
|
||||
"crypto/ecdsa"
|
||||
"crypto/elliptic"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"encoding/pem"
|
||||
"math/big"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// CertificateFiles creates a temporary localhost certificate and a client
|
||||
// configuration that verifies it. No insecure TLS bypass is needed in tests.
|
||||
func CertificateFiles(t testing.TB) (certFile, keyFile string, client *tls.Config) {
|
||||
t.Helper()
|
||||
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
serial, err := rand.Int(rand.Reader, new(big.Int).Lsh(big.NewInt(1), 120))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
certificate := &x509.Certificate{
|
||||
SerialNumber: serial,
|
||||
NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour),
|
||||
DNSNames: []string{"localhost"}, IPAddresses: []net.IP{net.ParseIP("127.0.0.1")},
|
||||
KeyUsage: x509.KeyUsageDigitalSignature,
|
||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
||||
}
|
||||
der, err := x509.CreateCertificate(rand.Reader, certificate, certificate, &key.PublicKey, key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
keyDER, err := x509.MarshalPKCS8PrivateKey(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
directory := t.TempDir()
|
||||
certFile, keyFile = filepath.Join(directory, "server.crt"), filepath.Join(directory, "server.key")
|
||||
if err := os.WriteFile(certFile, pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: der}), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(keyFile, pem.EncodeToMemory(&pem.Block{Type: "PRIVATE KEY", Bytes: keyDER}), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
leaf, err := x509.ParseCertificate(der)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
roots := x509.NewCertPool()
|
||||
roots.AddCert(leaf)
|
||||
return certFile, keyFile, &tls.Config{RootCAs: roots, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
# logx
|
||||
|
||||
`logx` 是 GoOmni 的业务层结构化日志入口。业务代码不需要依赖 Gin:
|
||||
`logx` 是 GoOmni 的业务层结构化日志入口。业务代码不需要依赖具体网络库:
|
||||
|
||||
```go
|
||||
import "git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
|
||||
42
logx/logx.go
42
logx/logx.go
@@ -1,5 +1,5 @@
|
||||
// Package logx provides the application logger used by GoOmni and business
|
||||
// packages. Its public API is deliberately independent of Gin so the same
|
||||
// packages. Its public API is deliberately independent of the network layer so the same
|
||||
// logger can be used from HTTP handlers, jobs and command line programs.
|
||||
package logx
|
||||
|
||||
@@ -21,14 +21,14 @@ import (
|
||||
// standard stream. File takes precedence over Output and appends to the named
|
||||
// file.
|
||||
type Config struct {
|
||||
Level string
|
||||
Format string
|
||||
Output string
|
||||
File string
|
||||
Service string
|
||||
Environment string
|
||||
AddSource bool
|
||||
Writer io.Writer
|
||||
Level string `json:"level"`
|
||||
Format string `json:"format"`
|
||||
Output string `json:"output"`
|
||||
File string `json:"file"`
|
||||
Service string `json:"service"`
|
||||
Environment string `json:"environment"`
|
||||
AddSource bool `json:"addSource"`
|
||||
Writer io.Writer `json:"-"`
|
||||
}
|
||||
|
||||
// Default returns the default production-friendly logger configuration.
|
||||
@@ -45,6 +45,7 @@ func Default() Config {
|
||||
type Logger struct {
|
||||
logger *slog.Logger
|
||||
closer *closeState
|
||||
level *slog.LevelVar
|
||||
}
|
||||
|
||||
type closeState struct {
|
||||
@@ -78,7 +79,9 @@ func New(cfg Config) (*Logger, error) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
opts := &slog.HandlerOptions{Level: level, AddSource: cfg.AddSource}
|
||||
dynamicLevel := new(slog.LevelVar)
|
||||
dynamicLevel.Set(level)
|
||||
opts := &slog.HandlerOptions{Level: dynamicLevel, AddSource: cfg.AddSource}
|
||||
var handler slog.Handler
|
||||
if cfg.Format == "text" {
|
||||
handler = slog.NewTextHandler(writer, opts)
|
||||
@@ -97,7 +100,7 @@ func New(cfg Config) (*Logger, error) {
|
||||
if closeWriter != nil {
|
||||
closer = &closeState{fn: closeWriter}
|
||||
}
|
||||
return &Logger{logger: logger, closer: closer}, nil
|
||||
return &Logger{logger: logger, closer: closer, level: dynamicLevel}, nil
|
||||
}
|
||||
|
||||
// Setup replaces the process-wide logger used by the package-level helpers.
|
||||
@@ -152,14 +155,27 @@ func (l *Logger) With(args ...any) *Logger {
|
||||
if l == nil || l.logger == nil {
|
||||
return defaultLogger.Load().With(args...)
|
||||
}
|
||||
return &Logger{logger: l.logger.With(args...), closer: l.closer}
|
||||
return &Logger{logger: l.logger.With(args...), closer: l.closer, level: l.level}
|
||||
}
|
||||
|
||||
func (l *Logger) WithGroup(name string) *Logger {
|
||||
if l == nil || l.logger == nil {
|
||||
return defaultLogger.Load().WithGroup(name)
|
||||
}
|
||||
return &Logger{logger: l.logger.WithGroup(name), closer: l.closer}
|
||||
return &Logger{logger: l.logger.WithGroup(name), closer: l.closer, level: l.level}
|
||||
}
|
||||
|
||||
// SetLevel atomically changes the threshold shared by this logger and children.
|
||||
func (l *Logger) SetLevel(value string) error {
|
||||
if l == nil || l.level == nil {
|
||||
return errors.New("logx: logger does not support dynamic levels")
|
||||
}
|
||||
level, err := parseLevel(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
l.level.Set(level)
|
||||
return nil
|
||||
}
|
||||
|
||||
// Close releases resources owned by the root logger. It is safe to call more
|
||||
|
||||
106
metrics/metrics.go
Normal file
106
metrics/metrics.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package metrics provides a small transport-neutral request metrics collector.
|
||||
// It is intentionally in-memory and dependency-free; applications can expose
|
||||
// Snapshot through any route or export it to their monitoring system.
|
||||
package metrics
|
||||
|
||||
import (
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
// Snapshot is a point-in-time view of request metrics collected by Metrics.
|
||||
type Snapshot struct {
|
||||
Requests uint64 `json:"requests"`
|
||||
Errors uint64 `json:"errors"`
|
||||
InFlight int64 `json:"inFlight"`
|
||||
TotalDurationMs float64 `json:"totalDurationMs"`
|
||||
StatusCounts map[string]uint64 `json:"statusCounts"`
|
||||
}
|
||||
|
||||
// Metrics collects request counts, status counts, in-flight requests and
|
||||
// cumulative handler time. A zero Metrics value is ready for use.
|
||||
type Metrics struct {
|
||||
requests atomic.Uint64
|
||||
errors atomic.Uint64
|
||||
inFlight atomic.Int64
|
||||
durationNs atomic.Int64
|
||||
statuses [600]atomic.Uint64
|
||||
}
|
||||
|
||||
// New creates an empty metrics collector.
|
||||
func New() *Metrics { return &Metrics{} }
|
||||
|
||||
// Middleware records every request that reaches the middleware chain. Errors
|
||||
// include returned handler errors and responses with a status of 400 or greater.
|
||||
func (m *Metrics) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(ctx *httpx.Context) error {
|
||||
if m == nil {
|
||||
return next(ctx)
|
||||
}
|
||||
m.inFlight.Add(1)
|
||||
started := time.Now()
|
||||
err := next(ctx)
|
||||
status := ctx.Status()
|
||||
if err != nil && !ctx.Responded() {
|
||||
status = apierrors.From(err).Status
|
||||
}
|
||||
m.requests.Add(1)
|
||||
if err != nil || status >= 400 {
|
||||
m.errors.Add(1)
|
||||
}
|
||||
if status >= 0 && status < len(m.statuses) {
|
||||
m.statuses[status].Add(1)
|
||||
}
|
||||
m.durationNs.Add(time.Since(started).Nanoseconds())
|
||||
m.inFlight.Add(-1)
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Handler exposes the current snapshot through the standard GoOmni response
|
||||
// envelope. Register it explicitly, for example app.GET("/metrics", m.Handler()).
|
||||
func (m *Metrics) Handler() httpx.Handler {
|
||||
return func(ctx *httpx.Context) error {
|
||||
return ctx.OK(m.Snapshot())
|
||||
}
|
||||
}
|
||||
|
||||
// Snapshot returns a consistent-enough point-in-time view for operational
|
||||
// reporting. Counters may advance while the snapshot is being assembled.
|
||||
func (m *Metrics) Snapshot() Snapshot {
|
||||
if m == nil {
|
||||
return Snapshot{StatusCounts: map[string]uint64{}}
|
||||
}
|
||||
statuses := make(map[string]uint64)
|
||||
for status := 100; status < len(m.statuses); status++ {
|
||||
if count := m.statuses[status].Load(); count > 0 {
|
||||
statuses[itoa(status)] = count
|
||||
}
|
||||
}
|
||||
return Snapshot{
|
||||
Requests: m.requests.Load(),
|
||||
Errors: m.errors.Load(),
|
||||
InFlight: m.inFlight.Load(),
|
||||
TotalDurationMs: float64(m.durationNs.Load()) / float64(time.Millisecond),
|
||||
StatusCounts: statuses,
|
||||
}
|
||||
}
|
||||
|
||||
func itoa(value int) string {
|
||||
if value == 0 {
|
||||
return "0"
|
||||
}
|
||||
var digits [3]byte
|
||||
index := len(digits)
|
||||
for value > 0 {
|
||||
index--
|
||||
digits[index] = byte('0' + value%10)
|
||||
value /= 10
|
||||
}
|
||||
return string(digits[index:])
|
||||
}
|
||||
67
metrics/metrics_test.go
Normal file
67
metrics/metrics_test.go
Normal file
@@ -0,0 +1,67 @@
|
||||
package metrics_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/metrics"
|
||||
)
|
||||
|
||||
func TestMiddlewareCollectsStatusesAndErrors(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/health", nil)
|
||||
first := httptest.NewRecorder()
|
||||
ctx := httpx.NewContext(request, first, nil)
|
||||
if err := collector.Middleware()(func(c *httpx.Context) error { return c.OK("ok") })(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
second := httptest.NewRecorder()
|
||||
ctx = httpx.NewContext(request, second, nil)
|
||||
handled := collector.Middleware()(func(*httpx.Context) error {
|
||||
return apierrors.NotFound("MISSING", "missing")
|
||||
})
|
||||
if err := handled(ctx); err == nil {
|
||||
t.Fatal("expected handler error")
|
||||
}
|
||||
|
||||
snapshot := collector.Snapshot()
|
||||
if snapshot.Requests != 2 || snapshot.Errors != 1 || snapshot.InFlight != 0 || snapshot.StatusCounts["200"] != 1 || snapshot.StatusCounts["404"] != 1 {
|
||||
t.Fatalf("unexpected snapshot: %+v", snapshot)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHandlerReturnsJSONSnapshot(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/metrics", nil)
|
||||
recorder := httptest.NewRecorder()
|
||||
ctx := httpx.NewContext(request, recorder, nil)
|
||||
if err := collector.Handler()(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var envelope struct {
|
||||
Code string `json:"code"`
|
||||
Data metrics.Snapshot `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &envelope); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if envelope.Code != "OK" || envelope.Data.StatusCounts == nil {
|
||||
t.Fatalf("unexpected metrics response: %s", recorder.Body)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMiddlewarePreservesApplicationErrors(t *testing.T) {
|
||||
collector := metrics.New()
|
||||
request := httptest.NewRequest("GET", "/", nil)
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
want := errors.New("database unavailable")
|
||||
got := collector.Middleware()(func(*httpx.Context) error { return want })(ctx)
|
||||
if !errors.Is(got, want) {
|
||||
t.Fatalf("middleware replaced error: %v", got)
|
||||
}
|
||||
}
|
||||
242
middleware/idempotency.go
Normal file
242
middleware/idempotency.go
Normal file
@@ -0,0 +1,242 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type IdempotencyConfig struct {
|
||||
Store IdempotencyStore `json:"-"` // Optional shared store; nil keeps local behavior.
|
||||
TTL time.Duration `json:"ttl"`
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"`
|
||||
MaxRequestBytes int64 `json:"maxRequestBytes"`
|
||||
MaxResponseBytes int `json:"maxResponseBytes"`
|
||||
MaxInFlight int `json:"maxInFlight"`
|
||||
// Scope must return a server-verified tenant/user identity. Register auth
|
||||
// before this middleware. Client-provided identity is not authorization.
|
||||
Scope func(*httpx.Context) (string, error) `json:"-"`
|
||||
}
|
||||
|
||||
type replay struct {
|
||||
fingerprint [32]byte
|
||||
status int
|
||||
headers http.Header
|
||||
body []byte
|
||||
expires time.Time
|
||||
running, failed bool
|
||||
size int
|
||||
}
|
||||
type Idempotency struct {
|
||||
mu sync.Mutex
|
||||
config IdempotencyConfig
|
||||
items map[[32]byte]*replay
|
||||
bytes int
|
||||
slots chan struct{}
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewIdempotency(cfg IdempotencyConfig) (*Idempotency, error) {
|
||||
if cfg.Scope == nil || cfg.TTL < 0 || cfg.MaxEntries < 0 || cfg.MaxBytes < 0 || cfg.MaxRequestBytes < 0 || cfg.MaxResponseBytes < 0 || cfg.MaxInFlight < 0 {
|
||||
return nil, errors.New("middleware: invalid idempotency configuration or missing scope")
|
||||
}
|
||||
if cfg.TTL == 0 {
|
||||
cfg.TTL = 10 * time.Minute
|
||||
}
|
||||
if cfg.Store != nil && cfg.TTL > 30*24*time.Hour {
|
||||
return nil, errors.New("middleware: shared idempotency TTL exceeds 30d")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 256
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 16 << 20
|
||||
}
|
||||
if cfg.MaxRequestBytes == 0 {
|
||||
cfg.MaxRequestBytes = 1 << 20
|
||||
}
|
||||
if cfg.MaxResponseBytes == 0 {
|
||||
cfg.MaxResponseBytes = 64 << 10
|
||||
}
|
||||
if cfg.MaxInFlight == 0 {
|
||||
cfg.MaxInFlight = 32
|
||||
}
|
||||
if cfg.MaxRequestBytes > 1<<30 {
|
||||
return nil, errors.New("middleware: maxRequestBytes exceeds 1 GiB")
|
||||
}
|
||||
return &Idempotency{config: cfg, items: make(map[[32]byte]*replay), slots: make(chan struct{}, cfg.MaxInFlight), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (i *Idempotency) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
r := c.Request()
|
||||
if r.Method == "GET" || r.Method == "HEAD" || r.Method == "OPTIONS" {
|
||||
return next(c)
|
||||
}
|
||||
select {
|
||||
case i.slots <- struct{}{}:
|
||||
defer func() { <-i.slots }()
|
||||
default:
|
||||
return apierrors.New(503, "IDEMPOTENCY_BUSY", "idempotency capacity reached")
|
||||
}
|
||||
key := r.Header.Get("Idempotency-Key")
|
||||
if len(r.Header.Values("Idempotency-Key")) != 1 || len(key) < 1 || len(key) > 128 || strings.IndexFunc(key, func(r rune) bool { return r < 33 || r > 126 }) >= 0 {
|
||||
return apierrors.BadRequest("INVALID_IDEMPOTENCY_KEY", "Idempotency-Key must contain 1..128 printable ASCII characters")
|
||||
}
|
||||
scope, err := i.config.Scope(c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if scope == "" || len(scope) > 1024 {
|
||||
return apierrors.Unauthorized("IDEMPOTENCY_SCOPE_REQUIRED", "authenticated scope is required")
|
||||
}
|
||||
if r.ContentLength > i.config.MaxRequestBytes {
|
||||
return apierrors.New(413, "BODY_TOO_LARGE", "request body too large")
|
||||
}
|
||||
var body []byte
|
||||
if r.Body != nil {
|
||||
body, err = io.ReadAll(io.LimitReader(r.Body, i.config.MaxRequestBytes+1))
|
||||
if err != nil {
|
||||
return apierrors.BadRequest("INVALID_BODY", "unable to read request body").WithCause(err)
|
||||
}
|
||||
if int64(len(body)) > i.config.MaxRequestBytes {
|
||||
return apierrors.New(413, "BODY_TOO_LARGE", "request body too large")
|
||||
}
|
||||
}
|
||||
if err := c.Context().Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
originalBody := r.Body
|
||||
r.Body = io.NopCloser(bytes.NewReader(body))
|
||||
defer func() { r.Body = originalBody }()
|
||||
parts, _ := json.Marshal([]string{r.Method, r.URL.RequestURI(), r.Header.Get("Content-Type")})
|
||||
hash := sha256.New()
|
||||
hash.Write(parts)
|
||||
hash.Write(body)
|
||||
var fingerprint [32]byte
|
||||
copy(fingerprint[:], hash.Sum(nil))
|
||||
namespace, _ := json.Marshal([]string{scope, key})
|
||||
id := sha256.Sum256(namespace)
|
||||
if i.config.Store != nil {
|
||||
return i.shared(c, next, id, fingerprint)
|
||||
}
|
||||
i.mu.Lock()
|
||||
now := i.now()
|
||||
for k, v := range i.items {
|
||||
if !v.running && !now.Before(v.expires) {
|
||||
delete(i.items, k)
|
||||
i.bytes -= v.size
|
||||
}
|
||||
}
|
||||
if saved := i.items[id]; saved != nil {
|
||||
if saved.fingerprint != fingerprint {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(409, "IDEMPOTENCY_CONFLICT", "key was used with a different request")
|
||||
}
|
||||
if saved.running || saved.failed {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(409, "IDEMPOTENCY_UNAVAILABLE", "request is in progress or its outcome cannot be replayed")
|
||||
}
|
||||
status, headers, data := saved.status, saved.headers.Clone(), append([]byte(nil), saved.body...)
|
||||
i.mu.Unlock()
|
||||
headers.Set("Idempotency-Replayed", "true")
|
||||
return c.WriteResponse(status, headers, data)
|
||||
}
|
||||
if len(i.items) >= i.config.MaxEntries {
|
||||
i.mu.Unlock()
|
||||
return apierrors.New(503, "IDEMPOTENCY_BUSY", "idempotency capacity reached")
|
||||
}
|
||||
saved := &replay{fingerprint: fingerprint, running: true}
|
||||
i.items[id] = saved
|
||||
i.mu.Unlock()
|
||||
buffer := &boundedResponse{headers: c.Raw().Header().Clone(), limit: i.config.MaxResponseBytes}
|
||||
// Always retain an outcome tombstone on panic/error/overflow. Deleting
|
||||
// the key could execute a partially committed business operation twice.
|
||||
defer func() {
|
||||
i.mu.Lock()
|
||||
defer i.mu.Unlock()
|
||||
saved.running = false
|
||||
saved.failed = true
|
||||
saved.expires = i.now().Add(i.config.TTL)
|
||||
size := buffer.body.Len()
|
||||
for k, values := range buffer.headers {
|
||||
size += len(k)
|
||||
for _, v := range values {
|
||||
size += len(v)
|
||||
}
|
||||
}
|
||||
if buffer.complete && buffer.err == nil && buffer.status < 500 && buffer.headers.Get("Set-Cookie") == "" && !strings.Contains(strings.ToLower(buffer.headers.Get("Cache-Control")), "no-store") && size <= i.config.MaxBytes-i.bytes {
|
||||
saved.failed = false
|
||||
saved.status = buffer.status
|
||||
saved.headers = buffer.headers.Clone()
|
||||
saved.body = append([]byte(nil), buffer.body.Bytes()...)
|
||||
saved.size = size
|
||||
i.bytes += size
|
||||
}
|
||||
}()
|
||||
child := c.Clone(c.Context(), buffer)
|
||||
err = next(child)
|
||||
// Keep an ambiguous outcome tombstone, but let an outer Timeout
|
||||
// middleware turn deadline expiry into 504 before committing bytes.
|
||||
if ctxErr := child.Context().Err(); ctxErr != nil {
|
||||
return ctxErr
|
||||
}
|
||||
if err != nil {
|
||||
child.WriteError(err)
|
||||
}
|
||||
if buffer.err != nil {
|
||||
return apierrors.Internal(buffer.err)
|
||||
}
|
||||
if buffer.status == 0 {
|
||||
buffer.status = 200
|
||||
}
|
||||
buffer.complete = true
|
||||
return c.WriteResponse(buffer.status, buffer.headers, buffer.body.Bytes())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Intentionally lacks Flush/Hijack: only buffered, bounded endpoints are valid.
|
||||
type boundedResponse struct {
|
||||
headers http.Header
|
||||
status, limit int
|
||||
body bytes.Buffer
|
||||
err error
|
||||
complete bool
|
||||
}
|
||||
|
||||
func (b *boundedResponse) Header() http.Header { return b.headers }
|
||||
func (b *boundedResponse) WriteHeader(status int) {
|
||||
if status < 200 {
|
||||
b.err = errors.New("idempotency: informational/upgrade responses are unsupported")
|
||||
return
|
||||
}
|
||||
if b.status == 0 {
|
||||
b.status = status
|
||||
}
|
||||
}
|
||||
func (b *boundedResponse) Write(data []byte) (int, error) {
|
||||
if b.status == 0 {
|
||||
b.status = 200
|
||||
}
|
||||
if b.err != nil {
|
||||
return 0, b.err
|
||||
}
|
||||
if len(data) > b.limit-b.body.Len() {
|
||||
b.err = fmt.Errorf("idempotency: response exceeds %d bytes", b.limit)
|
||||
return 0, b.err
|
||||
}
|
||||
return b.body.Write(data)
|
||||
}
|
||||
@@ -2,7 +2,6 @@ package middleware
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
@@ -31,8 +30,8 @@ func RequestID(header string) httpx.Middleware {
|
||||
}
|
||||
|
||||
// Recovery converts panics from handlers and middleware into the standard
|
||||
// internal error. The network adapter does not need a Gin-specific recovery
|
||||
// hook for this behavior.
|
||||
// internal error. Recovery belongs to GoOmni's transport-neutral middleware
|
||||
// rather than a router-specific hook.
|
||||
func Recovery() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(ctx *httpx.Context) (err error) {
|
||||
@@ -64,19 +63,9 @@ func AccessLog() httpx.Middleware {
|
||||
"route", httpx.RoutePattern(ctx.Context()),
|
||||
"status", status,
|
||||
"durationMs", time.Since(started).Seconds()*1000,
|
||||
"clientIp", clientIP(ctx.Request()),
|
||||
"clientIp", ctx.ClientIP(),
|
||||
)
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func clientIP(request *http.Request) string {
|
||||
if request == nil {
|
||||
return ""
|
||||
}
|
||||
if value := request.Header.Get("X-Forwarded-For"); value != "" {
|
||||
return value
|
||||
}
|
||||
return request.RemoteAddr
|
||||
}
|
||||
|
||||
@@ -55,3 +55,34 @@ func TestAccessLogReportsCommittedStatus(t *testing.T) {
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAccessLogUsesVerifiedClientIP(t *testing.T) {
|
||||
loggerOutput := bytes.Buffer{}
|
||||
logger, err := logx.New(logx.Config{Format: "json", Writer: &loggerOutput})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
logx.SetDefault(logger)
|
||||
defer logx.SetDefault(nil)
|
||||
resolver, err := httpx.NewClientIPResolver("127.0.0.1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
request := httptest.NewRequest("GET", "/health", nil)
|
||||
request.RemoteAddr = "127.0.0.1:443"
|
||||
request.Header.Set("X-Forwarded-For", "198.51.100.7")
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), resolver))
|
||||
ctx := httpx.NewContext(request, httptest.NewRecorder(), nil)
|
||||
if err := middleware.AccessLog()(func(c *httpx.Context) error { return c.OK("ok") })(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var entry struct {
|
||||
ClientIP string `json:"clientIp"`
|
||||
}
|
||||
if err := json.Unmarshal(loggerOutput.Bytes(), &entry); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if entry.ClientIP != "198.51.100.7" {
|
||||
t.Fatalf("expected verified client IP, got %q", entry.ClientIP)
|
||||
}
|
||||
}
|
||||
|
||||
227
middleware/protection_test.go
Normal file
227
middleware/protection_test.go
Normal file
@@ -0,0 +1,227 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
func invoke(h httpx.Handler, method, body, scope, key string) *httptest.ResponseRecorder {
|
||||
r := httptest.NewRequest(method, "/records?version=1", strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
if key != "" {
|
||||
r.Header.Set("Idempotency-Key", key)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
c := httpx.NewContext(r, w, nil)
|
||||
c.Set("verifiedIdentity", scope)
|
||||
if err := h(c); err != nil {
|
||||
c.WriteError(err)
|
||||
}
|
||||
return w
|
||||
}
|
||||
|
||||
func identity(c *httpx.Context) (string, error) {
|
||||
v, _ := c.Get("verifiedIdentity")
|
||||
s, _ := v.(string)
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func TestRateLimitRefillCapacityAndIdleExpiry(t *testing.T) {
|
||||
l, err := NewRateLimiter(RateLimitConfig{Rate: 2, Burst: 2, MaxKeys: 1, IdleTTL: time.Second, Key: func(c *httpx.Context) string { s, _ := identity(c); return s }})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Unix(100, 0)
|
||||
l.now = func() time.Time { return now }
|
||||
h := l.Middleware()(func(c *httpx.Context) error { return c.NoContent() })
|
||||
for _, status := range []int{204, 204, 429} {
|
||||
w := invoke(h, "GET", "", "a", "")
|
||||
if w.Code != status {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if status == 429 && w.Header().Get("Retry-After") != "1" {
|
||||
t.Fatal(w.Header())
|
||||
}
|
||||
}
|
||||
if w := invoke(h, "GET", "", "b", ""); w.Code != 503 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(500 * time.Millisecond)
|
||||
if w := invoke(h, "GET", "", "a", ""); w.Code != 204 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if w := invoke(h, "GET", "", "b", ""); w.Code != 204 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if _, err := NewRateLimiter(RateLimitConfig{Rate: 1, Burst: 2, IdleTTL: time.Second}); err == nil {
|
||||
t.Fatal("early bucket expiry accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRateLimitConcurrentBurst(t *testing.T) {
|
||||
l, _ := NewRateLimiter(RateLimitConfig{Rate: 1, Burst: 7})
|
||||
l.now = func() time.Time { return time.Unix(100, 0) }
|
||||
var allowed atomic.Int32
|
||||
var wg sync.WaitGroup
|
||||
for n := 0; n < 100; n++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
delay, full := l.allow("same")
|
||||
if delay == 0 && !full {
|
||||
allowed.Add(1)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if allowed.Load() != 7 {
|
||||
t.Fatal(allowed.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeoutCooperativeAndCommittedResponse(t *testing.T) {
|
||||
for _, committed := range []bool{false, true} {
|
||||
h := Timeout(5 * time.Millisecond)(func(c *httpx.Context) error {
|
||||
if committed {
|
||||
_ = c.Accepted("accepted")
|
||||
}
|
||||
<-c.Context().Done()
|
||||
return c.Context().Err()
|
||||
})
|
||||
w := invoke(h, "GET", "", "", "")
|
||||
want := 504
|
||||
if committed {
|
||||
want = 202
|
||||
}
|
||||
if w.Code != want {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyReplayScopeConflictAndExpiry(t *testing.T) {
|
||||
i, err := NewIdempotency(IdempotencyConfig{Scope: identity, TTL: time.Minute})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Unix(100, 0)
|
||||
i.now = func() time.Time { return now }
|
||||
calls := 0
|
||||
h := i.Middleware()(func(c *httpx.Context) error { calls++; return c.Created(map[string]int{"calls": calls}) })
|
||||
first := invoke(h, "POST", `{"name":"one"}`, "alice", "key1")
|
||||
replay := invoke(h, "POST", `{"name":"one"}`, "alice", "key1")
|
||||
if first.Code != 201 || replay.Code != 201 || first.Body.String() != replay.Body.String() || replay.Header().Get("Idempotency-Replayed") != "true" || calls != 1 {
|
||||
t.Fatal(first, replay, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", `{"name":"two"}`, "alice", "key1"); w.Code != 409 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", `{"name":"one"}`, "bob", "key1"); w.Code != 201 || calls != 2 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", `{}`, "", "key1"); w.Code != 401 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", `{}`, "alice", ""); w.Code != 400 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
now = now.Add(time.Minute)
|
||||
if w := invoke(h, "POST", `{"name":"one"}`, "alice", "key1"); w.Code != 201 || calls != 3 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyActiveClaimNeverExpires(t *testing.T) {
|
||||
i, _ := NewIdempotency(IdempotencyConfig{Scope: identity, TTL: time.Nanosecond, MaxInFlight: 2})
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
h := i.Middleware()(func(c *httpx.Context) error { calls.Add(1); close(started); <-release; return c.Created("saved") })
|
||||
done := make(chan *httptest.ResponseRecorder, 1)
|
||||
go func() { done <- invoke(h, "POST", "{}", "alice", "key") }()
|
||||
<-started
|
||||
for n := 0; n < 30; n++ {
|
||||
if w := invoke(h, "POST", "{}", "alice", "key"); w.Code != 409 {
|
||||
t.Errorf("duplicate status %d", w.Code)
|
||||
}
|
||||
}
|
||||
close(release)
|
||||
if w := <-done; w.Code != 201 || calls.Load() != 1 {
|
||||
t.Fatal(w.Code, calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyBoundsAndAmbiguousOutcomes(t *testing.T) {
|
||||
for _, mode := range []string{"panic", "error", "overflow", "cookie", "noStore", "memory"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
cfg := IdempotencyConfig{Scope: identity, MaxRequestBytes: 8, MaxResponseBytes: 128, MaxEntries: 1}
|
||||
if mode == "memory" {
|
||||
cfg.MaxBytes = 1
|
||||
}
|
||||
i, _ := NewIdempotency(cfg)
|
||||
calls := 0
|
||||
h := Recovery()(i.Middleware()(func(c *httpx.Context) error {
|
||||
calls++
|
||||
switch mode {
|
||||
case "panic":
|
||||
panic("failure")
|
||||
case "error":
|
||||
return errors.New("failed")
|
||||
case "overflow":
|
||||
return c.OK(strings.Repeat("x", 200))
|
||||
case "cookie":
|
||||
c.SetResponseHeader("Set-Cookie", "session=secret")
|
||||
case "noStore":
|
||||
c.SetResponseHeader("Cache-Control", "no-store")
|
||||
}
|
||||
return c.Created("ok")
|
||||
}))
|
||||
if w := invoke(h, "POST", "123456789", "alice", "key"); w.Code != 413 || calls != 0 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
first := invoke(h, "POST", "{}", "alice", "key")
|
||||
if mode == "panic" || mode == "error" || mode == "overflow" {
|
||||
if first.Code != 500 {
|
||||
t.Fatal(first.Code)
|
||||
}
|
||||
} else if first.Code != 201 {
|
||||
t.Fatal(first.Code)
|
||||
}
|
||||
if w := invoke(h, "POST", "{}", "alice", "key"); w.Code != 409 || calls != 1 {
|
||||
t.Fatal(w.Code, calls)
|
||||
}
|
||||
if w := invoke(h, "POST", "{}", "alice", "newKey"); w.Code != 503 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestIdempotencyInFlightLimitAndTimeoutComposition(t *testing.T) {
|
||||
i, _ := NewIdempotency(IdempotencyConfig{Scope: identity, MaxInFlight: 1})
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
h := i.Middleware()(func(c *httpx.Context) error { close(started); <-release; return c.NoContent() })
|
||||
done := make(chan struct{})
|
||||
go func() { defer close(done); invoke(h, "POST", "{}", "alice", "one") }()
|
||||
<-started
|
||||
if w := invoke(h, "POST", "{}", "alice", "two"); w.Code != 503 {
|
||||
t.Error(w.Code)
|
||||
}
|
||||
close(release)
|
||||
<-done
|
||||
timed := Timeout(5 * time.Millisecond)(i.Middleware()(func(c *httpx.Context) error { <-c.Context().Done(); return context.DeadlineExceeded }))
|
||||
if w := invoke(timed, "POST", "{}", "alice", "timeout"); w.Code != 504 {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
if w := invoke(timed, "POST", "{}", "alice", "timeout"); w.Code != 409 {
|
||||
t.Fatal(w.Code)
|
||||
}
|
||||
}
|
||||
120
middleware/rateLimit.go
Normal file
120
middleware/rateLimit.go
Normal file
@@ -0,0 +1,120 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"errors"
|
||||
"math"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type RateLimitConfig struct {
|
||||
Backend RateLimitBackend `json:"-"`
|
||||
Rate float64 `json:"rate"` // Tokens per second, per key.
|
||||
Burst int `json:"burst"`
|
||||
MaxKeys int `json:"maxKeys"`
|
||||
IdleTTL time.Duration `json:"idleTtl"`
|
||||
Key func(*httpx.Context) string `json:"-"` // Default: trusted ClientIP.
|
||||
}
|
||||
|
||||
type bucket struct {
|
||||
key string
|
||||
tokens float64
|
||||
updated time.Time
|
||||
}
|
||||
type RateLimiter struct {
|
||||
mu sync.Mutex
|
||||
config RateLimitConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewRateLimiter(cfg RateLimitConfig) (*RateLimiter, error) {
|
||||
if math.IsNaN(cfg.Rate) || math.IsInf(cfg.Rate, 0) || cfg.Rate <= 0 || cfg.Burst < 1 || cfg.MaxKeys < 0 || cfg.IdleTTL < 0 {
|
||||
return nil, errors.New("middleware: invalid rate limit configuration")
|
||||
}
|
||||
refill := float64(cfg.Burst) / cfg.Rate
|
||||
if refill > (24 * time.Hour).Seconds() {
|
||||
return nil, errors.New("middleware: bucket refill time must not exceed 24h")
|
||||
}
|
||||
minimum := time.Duration(math.Ceil(refill * float64(time.Second)))
|
||||
if cfg.IdleTTL == 0 {
|
||||
cfg.IdleTTL = max(5*time.Minute, minimum)
|
||||
}
|
||||
if cfg.IdleTTL < minimum {
|
||||
return nil, errors.New("middleware: idleTtl must allow a full bucket refill")
|
||||
}
|
||||
if cfg.MaxKeys == 0 {
|
||||
cfg.MaxKeys = 10000
|
||||
}
|
||||
if cfg.Key == nil {
|
||||
cfg.Key = func(c *httpx.Context) string { return c.ClientIP() }
|
||||
}
|
||||
return &RateLimiter{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (l *RateLimiter) Middleware() httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
key := l.config.Key(c)
|
||||
if key == "" || len(key) > 1024 {
|
||||
return apierrors.BadRequest("INVALID_RATE_KEY", "invalid rate limit key")
|
||||
}
|
||||
var delay time.Duration
|
||||
var full bool
|
||||
if l.config.Backend != nil {
|
||||
var err error
|
||||
delay, err = l.config.Backend.Take(c.Context(), key)
|
||||
if err != nil {
|
||||
return apierrors.New(503, "RATE_LIMIT_UNAVAILABLE", "rate limiter unavailable").WithCause(err)
|
||||
}
|
||||
} else {
|
||||
delay, full = l.allow(key)
|
||||
}
|
||||
if full {
|
||||
return apierrors.New(503, "RATE_LIMIT_CAPACITY", "rate limiter capacity reached")
|
||||
}
|
||||
if delay > 0 {
|
||||
c.SetResponseHeader("Retry-After", strconv.Itoa(max(1, int(math.Ceil(delay.Seconds())))))
|
||||
return apierrors.New(429, "RATE_LIMITED", "too many requests")
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (l *RateLimiter) allow(key string) (time.Duration, bool) {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
now := l.now()
|
||||
e := l.items[key]
|
||||
if e == nil {
|
||||
for l.lru.Len() > 0 {
|
||||
old := l.lru.Back()
|
||||
if now.Sub(old.Value.(*bucket).updated) < l.config.IdleTTL {
|
||||
break
|
||||
}
|
||||
delete(l.items, old.Value.(*bucket).key)
|
||||
l.lru.Remove(old)
|
||||
}
|
||||
if len(l.items) >= l.config.MaxKeys {
|
||||
return 0, true
|
||||
}
|
||||
e = l.lru.PushFront(&bucket{key: key, tokens: float64(l.config.Burst), updated: now})
|
||||
l.items[key] = e
|
||||
}
|
||||
b := e.Value.(*bucket)
|
||||
b.tokens = min(float64(l.config.Burst), b.tokens+max(0, now.Sub(b.updated).Seconds())*l.config.Rate)
|
||||
b.updated = now
|
||||
l.lru.MoveToFront(e)
|
||||
if b.tokens >= 1 {
|
||||
b.tokens--
|
||||
return 0, false
|
||||
}
|
||||
return time.Duration(math.Ceil((1 - b.tokens) / l.config.Rate * float64(time.Second))), false
|
||||
}
|
||||
178
middleware/redis/store.go
Normal file
178
middleware/redis/store.go
Normal file
@@ -0,0 +1,178 @@
|
||||
// Package redis supplies atomic shared middleware state in one Redis hash slot.
|
||||
// Use a dedicated namespace and a noeviction Redis deployment for correctness.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type IdempotencyConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxRecordBytes int `json:"maxRecordBytes"`
|
||||
}
|
||||
type IdempotencyStore struct {
|
||||
client goredis.Scripter
|
||||
keys []string
|
||||
cfg IdempotencyConfig
|
||||
signature string
|
||||
}
|
||||
|
||||
func namespace(prefix, kind string) (string, error) {
|
||||
if prefix == "" || len(prefix) > 256 || strings.ContainsAny(prefix, "{}\r\n") {
|
||||
return "", errors.New("middleware/redis: nonempty bounded namespace without braces required")
|
||||
}
|
||||
sum := sha256.Sum256([]byte(prefix))
|
||||
return "goOmni:{" + hex.EncodeToString(sum[:]) + "}:" + kind, nil
|
||||
}
|
||||
func NewIdempotencyStore(client goredis.Scripter, prefix string, cfg IdempotencyConfig) (*IdempotencyStore, error) {
|
||||
if client == nil || cfg.MaxEntries < 0 || cfg.MaxRecordBytes < 0 {
|
||||
return nil, errors.New("middleware/redis: invalid configuration")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 128
|
||||
}
|
||||
if cfg.MaxRecordBytes == 0 {
|
||||
cfg.MaxRecordBytes = 128 << 10
|
||||
}
|
||||
if cfg.MaxEntries > 100000 || cfg.MaxRecordBytes < 512 || cfg.MaxRecordBytes > 1<<20 {
|
||||
return nil, errors.New("middleware/redis: capacity outside supported bounds")
|
||||
}
|
||||
base, err := namespace(prefix, "idempotency")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &IdempotencyStore{client: client, keys: []string{base + ":records", base + ":expiry", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%d:%d", cfg.MaxEntries, cfg.MaxRecordBytes)}, nil
|
||||
}
|
||||
|
||||
var claim = goredis.NewScript(`
|
||||
local config=redis.call('GET',KEYS[3])
|
||||
if config and config~=ARGV[5] then return redis.error_reply('configuration mismatch') end
|
||||
if not config then redis.call('SET',KEYS[3],ARGV[5]) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now,'LIMIT',0,128)
|
||||
for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
|
||||
local expiry=redis.call('ZSCORE',KEYS[2],ARGV[1]);if expiry and tonumber(expiry)<=now then redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]) end
|
||||
local value=redis.call('HGET',KEYS[1],ARGV[1]);if value then return {0,value} end
|
||||
if redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return redis.error_reply('idempotency capacity reached') end
|
||||
value=cjson.encode({fingerprint=ARGV[2],owner=ARGV[3],state='running'})
|
||||
redis.call('HSET',KEYS[1],ARGV[1],value);return {1,value}
|
||||
`)
|
||||
|
||||
func (s *IdempotencyStore) Claim(ctx context.Context, key, fp, owner string) (middleware.IdempotencyRecord, bool, error) {
|
||||
if len(key) != 64 || len(fp) != 64 || len(owner) != 32 {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim")
|
||||
}
|
||||
result, err := claim.Run(ctx, s.client, s.keys, key, fp, owner, s.cfg.MaxEntries, s.signature).Slice()
|
||||
if err != nil {
|
||||
return middleware.IdempotencyRecord{}, false, err
|
||||
}
|
||||
if len(result) != 2 {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim result")
|
||||
}
|
||||
value, ok := result[1].(string)
|
||||
if !ok || len(value) > s.cfg.MaxRecordBytes {
|
||||
return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid record")
|
||||
}
|
||||
var record middleware.IdempotencyRecord
|
||||
err = json.Unmarshal([]byte(value), &record)
|
||||
return record, result[0] == int64(1), err
|
||||
}
|
||||
|
||||
var complete = goredis.NewScript(`
|
||||
local current=redis.call('HGET',KEYS[1],ARGV[1]);if not current then return redis.error_reply('claim missing') end
|
||||
local value=cjson.decode(current)
|
||||
if value.owner~=ARGV[2] or value.state~='running' then return redis.error_reply('claim owner/state changed') end
|
||||
local response=ARGV[3]
|
||||
if response~='' then value.state='completed';value.response=cjson.decode(response) else value.state='failed' end
|
||||
local encoded=cjson.encode(value)
|
||||
if string.len(encoded)>tonumber(ARGV[5]) then value.state='failed';value.response=nil;encoded=cjson.encode(value) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
redis.call('HSET',KEYS[1],ARGV[1],encoded);redis.call('ZADD',KEYS[2],now+tonumber(ARGV[4]),ARGV[1]);return 1
|
||||
`)
|
||||
|
||||
func (s *IdempotencyStore) Complete(ctx context.Context, key, owner string, response *middleware.StoredResponse, ttl time.Duration) error {
|
||||
if ttl <= 0 || ttl > 30*24*time.Hour {
|
||||
return errors.New("middleware/redis: TTL must be positive and <=30d")
|
||||
}
|
||||
var encoded []byte
|
||||
var err error
|
||||
if response != nil {
|
||||
encoded, err = json.Marshal(response)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(encoded) > s.cfg.MaxRecordBytes {
|
||||
encoded = nil
|
||||
}
|
||||
}
|
||||
return complete.Run(ctx, s.client, s.keys, key, owner, string(encoded), max(1, ttl.Milliseconds()), s.cfg.MaxRecordBytes).Err()
|
||||
}
|
||||
|
||||
// Resolve removes only a matching non-replayable claim after the operator has
|
||||
// reconciled the business outcome. Never invoke automatically on a timeout.
|
||||
func (s *IdempotencyStore) Resolve(ctx context.Context, key, owner string) error {
|
||||
return goredis.NewScript(`local v=redis.call('HGET',KEYS[1],ARGV[1]);if not v then return 0 end;local r=cjson.decode(v);if r.owner~=ARGV[2] or r.state=='completed' then return redis.error_reply('claim cannot be resolved') end;redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]);return 1`).Run(ctx, s.client, s.keys, key, owner).Err()
|
||||
}
|
||||
|
||||
type limiter struct {
|
||||
client goredis.Scripter
|
||||
keys []string
|
||||
cfg middleware.RateLimitConfig
|
||||
signature string
|
||||
}
|
||||
|
||||
func NewRateLimiter(client goredis.Scripter, prefix string, cfg middleware.RateLimitConfig) (*middleware.RateLimiter, error) {
|
||||
if client == nil {
|
||||
return nil, errors.New("middleware/redis: client required")
|
||||
}
|
||||
if cfg.MaxKeys == 0 {
|
||||
cfg.MaxKeys = 10000
|
||||
}
|
||||
if cfg.IdleTTL == 0 {
|
||||
cfg.IdleTTL = 24 * time.Hour
|
||||
}
|
||||
if _, err := middleware.NewRateLimiter(cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
base, err := namespace(prefix, "rate")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
backend := &limiter{client: client, keys: []string{base + ":buckets", base + ":seen", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%g:%d:%d:%d", cfg.Rate, cfg.Burst, cfg.MaxKeys, cfg.IdleTTL.Milliseconds())}
|
||||
cfg.Backend = backend
|
||||
return middleware.NewRateLimiter(cfg)
|
||||
}
|
||||
|
||||
var take = goredis.NewScript(`
|
||||
local config=redis.call('GET',KEYS[3]);if config and config~=ARGV[6] then return redis.error_reply('configuration mismatch') end
|
||||
if not config then redis.call('SET',KEYS[3],ARGV[6]) end
|
||||
local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
|
||||
local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now-tonumber(ARGV[5]),'LIMIT',0,128)
|
||||
for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
|
||||
local value=redis.call('HGET',KEYS[1],ARGV[1]);local tokens=tonumber(ARGV[3])
|
||||
if value then local b=cjson.decode(value);tokens=math.min(tokens,b.tokens+math.max(0,now-b.time)*tonumber(ARGV[2])/1000)
|
||||
elseif redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return -1 end
|
||||
local delay=0;if tokens>=1 then tokens=tokens-1 else delay=math.ceil((1-tokens)*1000/tonumber(ARGV[2])) end
|
||||
redis.call('HSET',KEYS[1],ARGV[1],cjson.encode({tokens=tokens,time=now}));redis.call('ZADD',KEYS[2],now,ARGV[1]);return delay
|
||||
`)
|
||||
|
||||
func (l *limiter) Take(ctx context.Context, key string) (time.Duration, error) {
|
||||
sum := sha256.Sum256([]byte(key))
|
||||
delay, err := take.Run(ctx, l.client, l.keys, hex.EncodeToString(sum[:]), l.cfg.Rate, l.cfg.Burst, l.cfg.MaxKeys, l.cfg.IdleTTL.Milliseconds(), l.signature).Int64()
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if delay < 0 {
|
||||
return 0, middleware.ErrLimitCapacity
|
||||
}
|
||||
return time.Duration(delay) * time.Millisecond, nil
|
||||
}
|
||||
145
middleware/redis/store_test.go
Normal file
145
middleware/redis/store_test.go
Normal file
@@ -0,0 +1,145 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSharedRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("requires isolated Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 8, MaxRetries: -1, ContextTimeoutEnabled: true, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
prefix := fmt.Sprint("shared:", time.Now().UnixNano())
|
||||
s, err := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer client.Del(context.Background(), s.keys...)
|
||||
key, fp := strings.Repeat("a", 64), strings.Repeat("b", 64)
|
||||
var claims atomic.Int32
|
||||
var wg sync.WaitGroup
|
||||
owners := make(chan string, 32)
|
||||
for i := 0; i < 32; i++ {
|
||||
wg.Add(1)
|
||||
go func(i int) {
|
||||
defer wg.Done()
|
||||
owner := fmt.Sprintf("%032x", i)
|
||||
_, ok, err := s.Claim(ctx, key, fp, owner)
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
if ok {
|
||||
claims.Add(1)
|
||||
owners <- owner
|
||||
}
|
||||
}(i)
|
||||
}
|
||||
wg.Wait()
|
||||
if claims.Load() != 1 {
|
||||
t.Fatal(claims.Load())
|
||||
}
|
||||
owner := <-owners
|
||||
if err = s.Complete(ctx, key, "wrong", nil, time.Minute); err == nil {
|
||||
t.Fatal("foreign completion accepted")
|
||||
}
|
||||
if err = s.Complete(ctx, key, owner, &middleware.StoredResponse{Status: 201, Headers: http.Header{}, Body: []byte{0, 1, 255}}, time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
record, ok, err := s.Claim(ctx, key, fp, strings.Repeat("c", 32))
|
||||
if err != nil || ok || record.State != "completed" || len(record.Response.Body) != 3 {
|
||||
t.Fatal(record, ok, err)
|
||||
}
|
||||
// Two independently constructed apps use the same backend, so only one runs.
|
||||
var executions atomic.Int32
|
||||
makeApp := func() *omni.App {
|
||||
backend, _ := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
|
||||
idem, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Store: backend, Scope: func(*httpx.Context) (string, error) { return "verifiedTenant", nil }})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a := omni.New(omni.WithLogger(false))
|
||||
a.POST("/submit", func(c *httpx.Context) error { executions.Add(1); return c.OK("saved") }, idem.Middleware())
|
||||
return a
|
||||
}
|
||||
a, b := makeApp(), makeApp()
|
||||
request := func(a *omni.App, body string) *httptest.ResponseRecorder {
|
||||
r := httptest.NewRequest("POST", "/submit", strings.NewReader(body))
|
||||
r.Header.Set("Idempotency-Key", "stableKey")
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, r)
|
||||
return w
|
||||
}
|
||||
first, second := request(a, `{}`), request(b, `{}`)
|
||||
if first.Code != 200 || second.Code != 200 || first.Body.String() != second.Body.String() || second.Header().Get("Idempotency-Replayed") != "true" || executions.Load() != 1 {
|
||||
t.Fatal(first.Code, second.Code, second.Body.String(), executions.Load())
|
||||
}
|
||||
if request(b, `{"changed":true}`).Code != 409 {
|
||||
t.Fatal("conflict accepted")
|
||||
}
|
||||
// Shared limits do not multiply when a second replica is added.
|
||||
limitPrefix := prefix + "limit"
|
||||
cfg := middleware.RateLimitConfig{Rate: .01, Burst: 2, Key: func(*httpx.Context) string { return "tenant" }}
|
||||
r1, _ := NewRateLimiter(client, limitPrefix, cfg)
|
||||
r2, _ := NewRateLimiter(client, limitPrefix, cfg)
|
||||
base, _ := namespace(limitPrefix, "rate")
|
||||
defer client.Del(context.Background(), base+":buckets", base+":seen", base+":config")
|
||||
limited := func(l *middleware.RateLimiter) int {
|
||||
a := omni.New(omni.WithLogger(false))
|
||||
a.GET("/", func(c *httpx.Context) error { return c.OK("ok") }, l.Middleware())
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
|
||||
return w.Code
|
||||
}
|
||||
if limited(r1) != 200 || limited(r2) != 200 || limited(r1) != 429 {
|
||||
t.Fatal("limits multiplied across instances")
|
||||
}
|
||||
// Redis failure must fail closed before executing a new request.
|
||||
client.Close()
|
||||
if status := limited(r2); status != 503 {
|
||||
t.Fatal(status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExpiredTargetBeyondCleanupBatch(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("requires isolated Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address})
|
||||
defer client.Close()
|
||||
s, _ := NewIdempotencyStore(client, fmt.Sprint("expiry:", time.Now().UnixNano()), IdempotencyConfig{MaxEntries: 256})
|
||||
defer client.Del(context.Background(), s.keys...)
|
||||
ctx := context.Background()
|
||||
owner := strings.Repeat("1", 32)
|
||||
fp := strings.Repeat("2", 64)
|
||||
for i := 0; i < 140; i++ {
|
||||
key := fmt.Sprintf("%064x", i)
|
||||
if _, ok, err := s.Claim(ctx, key, fp, owner); err != nil || !ok {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := s.Complete(ctx, key, owner, nil, time.Minute); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
client.ZAdd(ctx, s.keys[1], goredis.Z{Score: 0, Member: key})
|
||||
}
|
||||
if _, ok, err := s.Claim(ctx, fmt.Sprintf("%064x", 139), fp, owner); err != nil || !ok {
|
||||
t.Fatal("expired target retained beyond TTL", ok, err)
|
||||
}
|
||||
}
|
||||
102
middleware/sharedIdempotency.go
Normal file
102
middleware/sharedIdempotency.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
type StoredResponse struct {
|
||||
Status int `json:"status"`
|
||||
Headers http.Header `json:"headers"`
|
||||
Body []byte `json:"body"`
|
||||
}
|
||||
type IdempotencyRecord struct {
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
Owner string `json:"owner"`
|
||||
State string `json:"state"`
|
||||
Response *StoredResponse `json:"response,omitempty"`
|
||||
}
|
||||
|
||||
// IdempotencyStore must atomically claim absent keys, compare owners when
|
||||
// finishing and retain in-flight claims until explicit resolution. Only completed
|
||||
// or failed records expire; an uncertain outcome must never become a new claim.
|
||||
type IdempotencyStore interface {
|
||||
Claim(context.Context, string, string, string) (IdempotencyRecord, bool, error)
|
||||
Complete(context.Context, string, string, *StoredResponse, time.Duration) error
|
||||
}
|
||||
|
||||
func (i *Idempotency) shared(c *httpx.Context, next httpx.Handler, id, fingerprint [32]byte) error {
|
||||
key, fp := hex.EncodeToString(id[:]), hex.EncodeToString(fingerprint[:])
|
||||
token := make([]byte, 16)
|
||||
if _, err := rand.Read(token); err != nil {
|
||||
return err
|
||||
}
|
||||
owner := hex.EncodeToString(token)
|
||||
record, claimed, err := i.config.Store.Claim(c.Context(), key, fp, owner)
|
||||
if err != nil {
|
||||
return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "idempotency store unavailable").WithCause(err)
|
||||
}
|
||||
if !claimed {
|
||||
if record.Fingerprint != fp {
|
||||
return apierrors.New(409, "IDEMPOTENCY_CONFLICT", "key was used with a different request")
|
||||
}
|
||||
if record.State != "completed" || record.Response == nil {
|
||||
return apierrors.New(409, "IDEMPOTENCY_UNAVAILABLE", "request is in progress or outcome requires reconciliation")
|
||||
}
|
||||
response := record.Response
|
||||
headers := response.Headers.Clone()
|
||||
headers.Set("Idempotency-Replayed", "true")
|
||||
return c.WriteResponse(response.Status, headers, response.Body)
|
||||
}
|
||||
completed := false
|
||||
finish := func(response *StoredResponse) error {
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(c.Context()), time.Second)
|
||||
defer cancel()
|
||||
return i.config.Store.Complete(ctx, key, owner, response, i.config.TTL)
|
||||
}
|
||||
defer func() {
|
||||
if !completed {
|
||||
if err := finish(nil); err != nil {
|
||||
logx.ErrorContext(c.Context(), "idempotency.persistFailed", "error", err)
|
||||
}
|
||||
}
|
||||
}()
|
||||
buffer := &boundedResponse{headers: c.Raw().Header().Clone(), limit: i.config.MaxResponseBytes}
|
||||
child := c.Clone(c.Context(), buffer)
|
||||
err = next(child)
|
||||
if ctxErr := child.Context().Err(); ctxErr != nil {
|
||||
return ctxErr
|
||||
}
|
||||
if err != nil {
|
||||
child.WriteError(err)
|
||||
}
|
||||
if buffer.err != nil {
|
||||
return apierrors.Internal(buffer.err)
|
||||
}
|
||||
if buffer.status == 0 {
|
||||
buffer.status = 200
|
||||
}
|
||||
var response *StoredResponse
|
||||
if buffer.status < 500 && buffer.headers.Get("Set-Cookie") == "" && !strings.Contains(strings.ToLower(buffer.headers.Get("Cache-Control")), "no-store") {
|
||||
response = &StoredResponse{Status: buffer.status, Headers: buffer.headers.Clone(), Body: append([]byte(nil), buffer.body.Bytes()...)}
|
||||
}
|
||||
if err = finish(response); err != nil {
|
||||
return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "outcome requires reconciliation").WithCause(err)
|
||||
}
|
||||
completed = true
|
||||
return c.WriteResponse(buffer.status, buffer.headers, buffer.body.Bytes())
|
||||
}
|
||||
|
||||
var ErrLimitCapacity = errors.New("middleware: shared rate limiter capacity")
|
||||
|
||||
type RateLimitBackend interface {
|
||||
Take(context.Context, string) (time.Duration, error)
|
||||
}
|
||||
35
middleware/timeout.go
Normal file
35
middleware/timeout.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
// Timeout applies a cooperative deadline to handlers and downstream calls.
|
||||
// It does not spawn a goroutine or race a handler's response. Business code must
|
||||
// observe Context().Done(); an already committed response cannot be replaced.
|
||||
func Timeout(duration time.Duration) httpx.Middleware {
|
||||
if duration <= 0 {
|
||||
panic("middleware: timeout must be positive")
|
||||
}
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
parent := c.Context()
|
||||
ctx, cancel := context.WithTimeout(parent, duration)
|
||||
defer cancel()
|
||||
c.WithContext(ctx)
|
||||
defer c.WithContext(parent)
|
||||
if ctx.Err() != nil {
|
||||
return apierrors.New(504, "REQUEST_TIMEOUT", "request deadline exceeded")
|
||||
}
|
||||
err := next(c)
|
||||
if ctx.Err() == context.DeadlineExceeded && !c.Responded() {
|
||||
return apierrors.New(504, "REQUEST_TIMEOUT", "request deadline exceeded").WithCause(err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
20
omni.go
20
omni.go
@@ -9,10 +9,12 @@ import (
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/metrics"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
type App = app.App
|
||||
type Runner = app.Runner
|
||||
type RouterGroup = app.RouterGroup
|
||||
type Principal = auth.Principal
|
||||
type Authenticator = auth.Authenticator
|
||||
@@ -23,10 +25,18 @@ type Router = httpx.Router
|
||||
type HTTPRouterGroup = httpx.RouterGroup
|
||||
type APIError = apierrors.APIError
|
||||
type Config = config.Config
|
||||
type TLSConfig = config.TLSConfig
|
||||
type Option = config.Option
|
||||
type Response = response.Response
|
||||
type LogConfig = logx.Config
|
||||
type Logger = logx.Logger
|
||||
type Metrics = metrics.Metrics
|
||||
|
||||
const (
|
||||
TransportNetHTTP = config.TransportNetHTTP
|
||||
TransportNative = config.TransportNative
|
||||
TransportHTTP2 = config.TransportHTTP2
|
||||
)
|
||||
|
||||
func New(options ...Option) *App { return app.New(options...) }
|
||||
|
||||
@@ -34,14 +44,18 @@ func NewWithRouter(router Router, options ...Option) *App {
|
||||
return app.NewWithRouter(router, options...)
|
||||
}
|
||||
|
||||
func WithMode(mode string) Option { return config.WithMode(mode) }
|
||||
|
||||
func WithRequestIDHeader(header string) Option { return config.WithRequestIDHeader(header) }
|
||||
|
||||
func WithLogger(enabled bool) Option { return config.WithLogger(enabled) }
|
||||
|
||||
func WithTrustedProxies(proxies ...string) Option { return config.WithTrustedProxies(proxies...) }
|
||||
|
||||
func WithConfig(settings Config) Option { return config.WithConfig(settings) }
|
||||
|
||||
func WithTransport(name string) Option { return config.WithTransport(name) }
|
||||
|
||||
func WithTLS(certFile, keyFile string) Option { return config.WithTLS(certFile, keyFile) }
|
||||
|
||||
func WithLogConfig(settings LogConfig) Option { return config.WithLogConfig(settings) }
|
||||
|
||||
func WithShutdownTimeout(timeout time.Duration) Option { return config.WithShutdownTimeout(timeout) }
|
||||
@@ -50,6 +64,8 @@ func LoadConfig(prefix string) (Config, error) { return config.LoadFromEnv(prefi
|
||||
|
||||
func SetupLogger(cfg LogConfig) (func() error, error) { return logx.Setup(cfg) }
|
||||
|
||||
func NewMetrics() *Metrics { return metrics.New() }
|
||||
|
||||
func NewError(status int, code, message string) *APIError {
|
||||
return apierrors.New(status, code, message)
|
||||
}
|
||||
|
||||
267
openapi/openapi.go
Normal file
267
openapi/openapi.go
Normal file
@@ -0,0 +1,267 @@
|
||||
// Package openapi registers routes and their OpenAPI 3.0.3 contracts together.
|
||||
package openapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"net/http"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Schema struct {
|
||||
Type string `json:"type,omitempty"`
|
||||
Format string `json:"format,omitempty"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Properties map[string]*Schema `json:"properties,omitempty"`
|
||||
Required []string `json:"required,omitempty"`
|
||||
Items *Schema `json:"items,omitempty"`
|
||||
AdditionalProperties *Schema `json:"additionalProperties,omitempty"`
|
||||
Nullable bool `json:"nullable,omitempty"`
|
||||
}
|
||||
type Parameter struct {
|
||||
Name string `json:"name"`
|
||||
In string `json:"in"`
|
||||
Required bool `json:"required"`
|
||||
Description string `json:"description,omitempty"`
|
||||
Schema *Schema `json:"schema"`
|
||||
}
|
||||
type Response struct {
|
||||
Description string
|
||||
Schema *Schema
|
||||
}
|
||||
type Operation struct {
|
||||
Method, Path, OperationID, Summary string
|
||||
Parameters []Parameter
|
||||
Request *Schema
|
||||
Responses map[int]Response
|
||||
BearerAuth bool
|
||||
}
|
||||
type Registrar interface {
|
||||
Handle(string, string, httpx.Handler, ...httpx.Middleware)
|
||||
}
|
||||
type API struct {
|
||||
mu sync.RWMutex
|
||||
document map[string]any
|
||||
paths map[string]map[string]any
|
||||
ids map[string]bool
|
||||
}
|
||||
|
||||
var identifier = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
|
||||
|
||||
func New(title, version string) (*API, error) {
|
||||
if title == "" || version == "" {
|
||||
return nil, errors.New("openapi: title and version required")
|
||||
}
|
||||
paths := make(map[string]map[string]any)
|
||||
return &API{paths: paths, ids: make(map[string]bool), document: map[string]any{"openapi": "3.0.3", "info": map[string]string{"title": title, "version": version}, "paths": paths, "components": map[string]any{"securitySchemes": map[string]any{"bearerAuth": map[string]string{"type": "http", "scheme": "bearer"}}}}}, nil
|
||||
}
|
||||
|
||||
// Handle registers an absolute application route. To document a group, include
|
||||
// its full prefix here and pass middleware explicitly. Schemas describe wire
|
||||
// responses; Envelope wraps a data schema in GoOmni's standard JSON envelope.
|
||||
func (a *API) Handle(router Registrar, op Operation, handler httpx.Handler, middleware ...httpx.Middleware) error {
|
||||
if router == nil || handler == nil || !identifier.MatchString(op.OperationID) || !strings.HasPrefix(op.Path, "/") || strings.ContainsAny(op.Path, "*{}?# ") {
|
||||
return errors.New("openapi: invalid route")
|
||||
}
|
||||
method := strings.ToLower(op.Method)
|
||||
switch method {
|
||||
case "get", "post", "put", "patch", "delete", "head", "options":
|
||||
default:
|
||||
return errors.New("openapi: unsupported HTTP method")
|
||||
}
|
||||
parts := strings.Split(op.Path, "/")
|
||||
params := make(map[string]bool)
|
||||
for n, part := range parts {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
name := part[1:]
|
||||
if !identifier.MatchString(name) || params[name] {
|
||||
return errors.New("openapi: invalid path parameter")
|
||||
}
|
||||
parts[n] = "{" + name + "}"
|
||||
params[name] = true
|
||||
}
|
||||
}
|
||||
path := strings.Join(parts, "/")
|
||||
seen := make(map[string]bool)
|
||||
for _, p := range op.Parameters {
|
||||
if p.Name == "" || p.Schema == nil || seen[p.In+":"+p.Name] {
|
||||
return errors.New("openapi: invalid or duplicate parameter")
|
||||
}
|
||||
seen[p.In+":"+p.Name] = true
|
||||
switch p.In {
|
||||
case "path":
|
||||
if !params[p.Name] || !p.Required {
|
||||
return errors.New("openapi: path parameter must match and be required")
|
||||
}
|
||||
delete(params, p.Name)
|
||||
case "query", "header", "cookie":
|
||||
default:
|
||||
return errors.New("openapi: invalid parameter location")
|
||||
}
|
||||
}
|
||||
if len(params) != 0 || len(op.Responses) == 0 {
|
||||
return errors.New("openapi: declare all path parameters and at least one response")
|
||||
}
|
||||
responses := map[string]any{}
|
||||
for code, response := range op.Responses {
|
||||
if code < 100 || code > 599 || response.Description == "" {
|
||||
return errors.New("openapi: invalid response")
|
||||
}
|
||||
item := map[string]any{"description": response.Description}
|
||||
if response.Schema != nil {
|
||||
item["content"] = map[string]any{"application/json": map[string]any{"schema": response.Schema}}
|
||||
}
|
||||
responses[fmt.Sprint(code)] = item
|
||||
}
|
||||
operation := map[string]any{"operationId": op.OperationID, "summary": op.Summary, "responses": responses}
|
||||
if len(op.Parameters) > 0 {
|
||||
operation["parameters"] = op.Parameters
|
||||
}
|
||||
if op.Request != nil {
|
||||
operation["requestBody"] = map[string]any{"required": true, "content": map[string]any{"application/json": map[string]any{"schema": op.Request}}}
|
||||
}
|
||||
if op.BearerAuth {
|
||||
operation["security"] = []map[string][]string{{"bearerAuth": {}}}
|
||||
}
|
||||
// Detach caller-owned maps/slices before concurrent document reads.
|
||||
encoded, err := json.Marshal(operation)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var frozen any
|
||||
if err = json.Unmarshal(encoded, &frozen); err != nil {
|
||||
return err
|
||||
}
|
||||
a.mu.Lock()
|
||||
defer a.mu.Unlock()
|
||||
if a.ids[op.OperationID] || a.paths[path][method] != nil {
|
||||
return errors.New("openapi: duplicate operation ID or route")
|
||||
}
|
||||
router.Handle(strings.ToUpper(method), op.Path, handler, middleware...)
|
||||
if a.paths[path] == nil {
|
||||
a.paths[path] = make(map[string]any)
|
||||
}
|
||||
a.paths[path][method] = frozen
|
||||
a.ids[op.OperationID] = true
|
||||
return nil
|
||||
}
|
||||
func (a *API) JSON() ([]byte, error) {
|
||||
a.mu.RLock()
|
||||
defer a.mu.RUnlock()
|
||||
return json.MarshalIndent(a.document, "", " ")
|
||||
}
|
||||
func (a *API) Handler() httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
data, err := a.JSON()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.WriteResponse(200, http.Header{"Content-Type": {"application/json; charset=utf-8"}}, data)
|
||||
}
|
||||
}
|
||||
func Envelope(data *Schema) *Schema {
|
||||
return &Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}, "data": data}}
|
||||
}
|
||||
func SchemaOf[T any]() (*Schema, error) {
|
||||
return schema(reflect.TypeOf((*T)(nil)).Elem(), map[reflect.Type]bool{}, 0)
|
||||
}
|
||||
func schema(t reflect.Type, stack map[reflect.Type]bool, depth int) (*Schema, error) {
|
||||
if depth > 32 || stack[t] {
|
||||
return nil, errors.New("openapi: recursive DTO requires an explicit schema")
|
||||
}
|
||||
if t.Kind() == reflect.Pointer {
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
if err == nil {
|
||||
s.Nullable = true
|
||||
}
|
||||
return s, err
|
||||
}
|
||||
if t == reflect.TypeOf(time.Time{}) {
|
||||
return &Schema{Type: "string", Format: "date-time"}, nil
|
||||
}
|
||||
if t == reflect.TypeOf(json.RawMessage{}) {
|
||||
return &Schema{}, nil
|
||||
}
|
||||
if t == reflect.TypeOf(json.Number("")) {
|
||||
return &Schema{Type: "number"}, nil
|
||||
}
|
||||
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) {
|
||||
return nil, errors.New("openapi: custom JSON marshaler requires an explicit schema")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.String:
|
||||
return &Schema{Type: "string"}, nil
|
||||
case reflect.Bool:
|
||||
return &Schema{Type: "boolean"}, nil
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
return &Schema{Type: "integer"}, nil
|
||||
case reflect.Float32, reflect.Float64:
|
||||
return &Schema{Type: "number"}, nil
|
||||
case reflect.Slice, reflect.Array:
|
||||
if t.Elem().Kind() == reflect.Uint8 && t.Kind() == reflect.Slice {
|
||||
return &Schema{Type: "string", Format: "byte"}, nil
|
||||
}
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
return &Schema{Type: "array", Items: s}, err
|
||||
case reflect.Map:
|
||||
if t.Key().Kind() != reflect.String {
|
||||
return nil, errors.New("openapi: map keys must be strings")
|
||||
}
|
||||
s, err := schema(t.Elem(), stack, depth+1)
|
||||
return &Schema{Type: "object", AdditionalProperties: s}, err
|
||||
case reflect.Interface:
|
||||
return &Schema{}, nil
|
||||
case reflect.Struct:
|
||||
stack[t] = true
|
||||
defer delete(stack, t)
|
||||
s := &Schema{Type: "object", Properties: map[string]*Schema{}}
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
f := t.Field(n)
|
||||
if f.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
tag := strings.Split(f.Tag.Get("json"), ",")
|
||||
if tag[0] == "-" {
|
||||
continue
|
||||
}
|
||||
if f.Anonymous && tag[0] == "" {
|
||||
return nil, errors.New("openapi: embedded fields require an explicit JSON name")
|
||||
}
|
||||
name := tag[0]
|
||||
if name == "" {
|
||||
name = f.Name
|
||||
}
|
||||
child, err := schema(f.Type, stack, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if s.Properties[name] != nil {
|
||||
return nil, errors.New("openapi: duplicate JSON field")
|
||||
}
|
||||
s.Properties[name] = child
|
||||
optional := false
|
||||
for _, option := range tag[1:] {
|
||||
if option == "string" {
|
||||
child.Type = "string"
|
||||
child.Format = ""
|
||||
}
|
||||
if option == "omitempty" || option == "omitzero" {
|
||||
optional = true
|
||||
}
|
||||
}
|
||||
if !optional {
|
||||
s.Required = append(s.Required, name)
|
||||
}
|
||||
}
|
||||
return s, nil
|
||||
default:
|
||||
return nil, fmt.Errorf("openapi: unsupported DTO type %s", t)
|
||||
}
|
||||
}
|
||||
54
openapi/openapi_test.go
Normal file
54
openapi/openapi_test.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package openapi_test
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRouteAndContract(t *testing.T) {
|
||||
type item struct {
|
||||
ID int64 `json:"id,string"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
Payload json.RawMessage `json:"payload"`
|
||||
}
|
||||
schema, err := openapi.SchemaOf[item]()
|
||||
if err != nil || schema.Properties["id"].Type != "string" || schema.Properties["payload"].Type != "" {
|
||||
t.Fatal(schema, err)
|
||||
}
|
||||
api, _ := openapi.New("Test API", "1")
|
||||
app := omni.New(omni.WithLogger(false))
|
||||
op := openapi.Operation{Method: "GET", Path: "/items/:id", OperationID: "getItem", Parameters: []openapi.Parameter{{Name: "id", In: "path", Required: true, Schema: &openapi.Schema{Type: "string"}}}, Responses: map[int]openapi.Response{200: {Description: "OK", Schema: openapi.Envelope(schema)}}}
|
||||
if err = api.Handle(app, op, func(c *httpx.Context) error { return c.OK(item{ID: 1, Payload: json.RawMessage(`{}`)}) }); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Caller edits must not mutate the published contract.
|
||||
schema.Properties["id"].Type = "boolean"
|
||||
if err = api.Handle(app, op, func(*httpx.Context) error { return nil }); err == nil {
|
||||
t.Fatal("duplicate accepted")
|
||||
}
|
||||
app.GET("/openapi.json", api.Handler())
|
||||
r := httptest.NewRecorder()
|
||||
app.ServeHTTP(r, httptest.NewRequest("GET", "/openapi.json", nil))
|
||||
var doc struct {
|
||||
OpenAPI string `json:"openapi"`
|
||||
Paths map[string]json.RawMessage `json:"paths"`
|
||||
}
|
||||
if err = json.Unmarshal(r.Body.Bytes(), &doc); err != nil || doc.OpenAPI != "3.0.3" || doc.Paths["/items/{id}"] == nil {
|
||||
t.Fatal(r.Body.String(), err)
|
||||
}
|
||||
r = httptest.NewRecorder()
|
||||
app.ServeHTTP(r, httptest.NewRequest("GET", "/items/1", nil))
|
||||
if r.Code != 200 {
|
||||
t.Fatal(r.Code)
|
||||
}
|
||||
type recursive struct {
|
||||
Next *recursive `json:"next"`
|
||||
}
|
||||
if _, err = openapi.SchemaOf[recursive](); err == nil {
|
||||
t.Fatal("recursive DTO silently truncated")
|
||||
}
|
||||
}
|
||||
73
pagination/pagination.go
Normal file
73
pagination/pagination.go
Normal file
@@ -0,0 +1,73 @@
|
||||
// Package pagination defines bounded offset pagination with camelCase fields.
|
||||
package pagination
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/url"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
DefaultSize int `json:"defaultSize"`
|
||||
MaxSize int `json:"maxSize"`
|
||||
MaxOffset int64 `json:"maxOffset"`
|
||||
}
|
||||
type Request struct {
|
||||
Page int64 `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
|
||||
func (r Request) Offset() int64 { return (r.Page - 1) * int64(r.PageSize) }
|
||||
func Parse(query url.Values, cfg Config) (Request, error) {
|
||||
if cfg.DefaultSize == 0 {
|
||||
cfg.DefaultSize = 20
|
||||
}
|
||||
if cfg.MaxSize == 0 {
|
||||
cfg.MaxSize = 100
|
||||
}
|
||||
if cfg.MaxOffset == 0 {
|
||||
cfg.MaxOffset = 1000000
|
||||
}
|
||||
if cfg.DefaultSize < 1 || cfg.MaxSize < cfg.DefaultSize || cfg.MaxSize > 10000 || cfg.MaxOffset < 0 {
|
||||
return Request{}, errors.New("pagination: invalid configuration")
|
||||
}
|
||||
page, size := int64(1), int64(cfg.DefaultSize)
|
||||
for key, target := range map[string]*int64{"page": &page, "pageSize": &size} {
|
||||
if v, ok := query[key]; ok {
|
||||
if len(v) != 1 {
|
||||
return Request{}, errors.New("pagination: duplicate parameter")
|
||||
}
|
||||
n, err := strconv.ParseInt(v[0], 10, 64)
|
||||
if err != nil || n < 1 {
|
||||
return Request{}, errors.New("pagination: positive integer required")
|
||||
}
|
||||
*target = n
|
||||
}
|
||||
}
|
||||
if size > int64(cfg.MaxSize) || (page-1) > cfg.MaxOffset/size {
|
||||
return Request{}, errors.New("pagination: size or offset exceeds limit")
|
||||
}
|
||||
return Request{Page: page, PageSize: int(size)}, nil
|
||||
}
|
||||
|
||||
type Result[T any] struct {
|
||||
Items []T `json:"items"`
|
||||
Page int64 `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
Total int64 `json:"total"`
|
||||
TotalPages int64 `json:"totalPages"`
|
||||
}
|
||||
|
||||
func NewResult[T any](r Request, items []T, total int64) (Result[T], error) {
|
||||
if r.Page < 1 || r.PageSize < 1 || total < 0 || len(items) > r.PageSize || int64(len(items)) > total {
|
||||
return Result[T]{}, errors.New("pagination: invalid result")
|
||||
}
|
||||
if items == nil {
|
||||
items = []T{}
|
||||
}
|
||||
pages := total / int64(r.PageSize)
|
||||
if total%int64(r.PageSize) != 0 {
|
||||
pages++
|
||||
}
|
||||
return Result[T]{Items: items, Page: r.Page, PageSize: r.PageSize, Total: total, TotalPages: pages}, nil
|
||||
}
|
||||
30
pagination/pagination_test.go
Normal file
30
pagination/pagination_test.go
Normal file
@@ -0,0 +1,30 @@
|
||||
package pagination
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"math"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestBoundsAndEnvelope(t *testing.T) {
|
||||
for _, raw := range []string{"page=0", "page=1&page=2", "pageSize=101", "page=9223372036854775807&pageSize=100", "pageSize=-1"} {
|
||||
q, _ := url.ParseQuery(raw)
|
||||
if _, err := Parse(q, Config{}); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
r, err := Parse(url.Values{"page": {"2"}, "pageSize": {"30"}}, Config{})
|
||||
if err != nil || r.Offset() != 30 {
|
||||
t.Fatal(r, err)
|
||||
}
|
||||
result, err := NewResult(r, []string(nil), math.MaxInt64)
|
||||
if err != nil || result.TotalPages != math.MaxInt64/30+1 {
|
||||
t.Fatal(result, err)
|
||||
}
|
||||
raw, _ := json.Marshal(result)
|
||||
if !strings.Contains(string(raw), `"items":[]`) || !strings.Contains(string(raw), `"pageSize":30`) {
|
||||
t.Fatal(string(raw))
|
||||
}
|
||||
}
|
||||
44
performance/advancedAcceptance.md
Normal file
44
performance/advancedAcceptance.md
Normal file
@@ -0,0 +1,44 @@
|
||||
# 扩展模块验收
|
||||
|
||||
代码提交:`208a9c3fbdb7ccf2ed409ba380050842e89829d8`。本报告与随后文档提交没有改变被测运行代码。
|
||||
|
||||
2026-09-23 在独立 Linux amd64 验证目录运行,开始 06:51:31 UTC,完成 06:55:56 UTC。工作树在启动时干净。原始 [JSON](advancedLinux.json) 记录 98 个源码/模板/脚本摘要、镜像 ID、命令和容器清理;[TXT](advancedLinux.txt) 保存实际输出。回传本机后已逐个核对源码摘要及日志 SHA-256。
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyModules.py --advanced \
|
||||
--goproxy https://goproxy.cn,direct \
|
||||
--output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
## 环境与隔离
|
||||
|
||||
- Go 1.25.14,Linux amd64。实际依赖版本以提交内 go.mod/go.sum 为准;新增依赖后已覆盖原有网络适配器回归。
|
||||
- Redis 7.4.11,MinIO RELEASE.2025-09-07T16-13-09Z,MySQL 8.4.11。
|
||||
- 编译、Redis、MinIO、MySQL 顺序运行,最多一个测试容器;每个硬限制 2 CPU / 2 GiB 内存与内存加 swap / 256 pids。
|
||||
- 数据服务阶段 `--network none`,仅容器回环/Unix socket,无发布端口,数据位于本次临时目录/tmpfs。没有连接现有业务数据库。
|
||||
- 四个测试容器已清理,原有 7 个业务容器均仍运行。
|
||||
|
||||
## 通过范围
|
||||
|
||||
| 项目 | 实际证据 |
|
||||
| --- | --- |
|
||||
| 全仓库检查 | `go test -race -count=1 -timeout=240s ./...`、`go vet ./...`、`go build ./...`;脚手架集成开关已开启,包含独立生成项目 |
|
||||
| 认证与配置 | JWT 过期/缺 exp/错误发行者或受众/算法混用/轮换、Session 撤销与状态隔离、文件环境覆盖、非法/静态配置拒绝、子日志级别更新 |
|
||||
| 契约与观测 | OpenAPI 路径参数与 DTO、重复注册拒绝、W3C 入站/出站关联、外层访问日志 traceId、HTTP 指标、Redis miss 错误保持、SQL spans |
|
||||
| 熔断与通用模块 | 冷却探针、迟到结果、响应体期间槽位、panic 释放、分页溢出与重复参数、审计白名单与写入失败 |
|
||||
| 组合示例 | nethttp/native/实验 HTTP2 的真实 TLS 401/403/201、对象内容、非法路径、OpenAPI 和关闭;原有 modules/reference 回归同批通过 |
|
||||
| 共享 Redis | 并发 claim 唯一赢家、错误 owner 拒绝、二进制响应回放、两个应用实例仅执行一次、冲突 409、共享限流 429、存储失败 503、过期目标不受批次清理顺序影响 |
|
||||
| Redis 队列 | 重报回执、内容冲突、容量拒绝、旧 owner 无法 ACK、保留未确认消息后自动接管、两个 Consumer 竞争、有限重试/永久拒绝、失败消息分页/受控删除、完成身份 TTL |
|
||||
| 对象存储 | Local 契约及失败上传保留原对象、拒绝符号链接、签名篡改;OSS SDK 协议 fixture;S3 SDK fixture 和 R2 endpoint/region/签名参数;真实 MinIO 上传/覆盖/读取/删除与错误签名 403 |
|
||||
| MySQL 迁移 | 全新测试库 up、重复 up 不重执行、版本读取、SQL 查询/更新观测、失败迁移保留 dirty、后续执行被阻断 |
|
||||
|
||||
真实 Redis/MinIO/MySQL 阶段使用 Go 编译的静态测试程序;该阶段没有启用 race。全仓库 race 阶段不连接这些外部数据服务。两类验证各自的范围不混为一项。
|
||||
|
||||
## 保留边界
|
||||
|
||||
- 本批按用户决定不集成 Orbit,未修改 Orbit 或业务项目,也没有用通用队列模拟其流程编排协议。
|
||||
- OSS/R2 没有真实云账号、bucket 权限、临时凭据轮换、地域/公网故障验收;MinIO 通过不能替代这些验证。
|
||||
- 队列接管通过真实 Redis 中的未确认 delivery 验证,没有本批 Worker 进程强杀、Redis AOF 重启/复制切换、长期负载或 exactly-once 证明。测试 Redis 关闭持久化以隔离协议检查,生产需另行配置持久化与故障验收。
|
||||
- 没有在生产业务上自动执行迁移;没有证明任意 DDL 可事务回滚。MySQL 失败保留 dirty,由部署流程核查。
|
||||
- OpenTelemetry 的采集/传播/exporter 接口通过本批测试,未连接实际 OTLP collector 或告警系统。
|
||||
- 这是扩展模块的首版功能验收,不是生产规模容量或完整运维验收。
|
||||
303
performance/advancedLinux.json
Normal file
303
performance/advancedLinux.json
Normal file
@@ -0,0 +1,303 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T06:51:31.468977+00:00",
|
||||
"testedCommit": "208a9c3fbdb7ccf2ed409ba380050842e89829d8",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"audit/audit.go": "48da5d6ecd498230d22db04413d379650b9810b3faee5fb5d52415ab6f681eea",
|
||||
"audit/audit_test.go": "7ecb96dcc8294c905465bb026030a2b0d2d5064359a9cb6b95199d5d15491c49",
|
||||
"auth/adapters_test.go": "a45dd935fd91b91d16435c97c3b6b6b82615a55bc16463eb94448704a2433fe1",
|
||||
"auth/auth.go": "067b75edf2d73338ceeff3b70698e6f5426606c5d510a7d0249bcf6fd7796eb8",
|
||||
"auth/auth_test.go": "158626ac86c504a337e37faad65b2bad77d68f54ce6b6ac2cda9d287aeefa963",
|
||||
"auth/jwt.go": "cfd4ead3712e921cb08d65b9ea81a06d1b6de420bc94cf240540a95959a2f241",
|
||||
"auth/policy.go": "f02cf32593033908921172190a65debaf9f8c0af616ceb95982ea94649170b62",
|
||||
"auth/session.go": "544c71e00c6af913722c1cca510e1fb2938925234c246e0ec52de22284a81b18",
|
||||
"cache/cache.go": "4aad0a644c0c9ce6353e61e70059715bea5402f0d7132b50003ee126c3836cad",
|
||||
"cache/cache_test.go": "c850cd8a90c1e6a8d701a415e283d4147fbd66d12d054002014cbf348863aea2",
|
||||
"cache/memory.go": "3ac2a710a0d4677ad58dd6817005033ef7cc939be7eb3e40476212ce7f18ad42",
|
||||
"cache/redis/redis.go": "3f522258da8a0fc6627645573221e9f5ccbdfea0fe8a1450985cfd084e2c18ce",
|
||||
"cache/redis/redis_test.go": "4948647b2d31803252af1ee4e4b96372b53f9fac46f03ab064b40abe2d5a0aa1",
|
||||
"cmd/go-omni/main.go": "811cd23a5288cd0a7efc132782d9509307f736afbcbbb2d001c59aad9d8633a9",
|
||||
"cmd/go-omni/main_test.go": "312e0dea1d46cbbd436acdb4fcac0704d4781746d9e8433f1203f67ef9601065",
|
||||
"cmd/go-omni/migrate.go": "f1ce89fbe419205eb82172da2b90e8888905d2b85d1a9db2a72b568bcf849a70",
|
||||
"cmd/go-omni/migrate_test.go": "15d9a4aaebf9f899e523e908f1fa3c2282eb1e6c8cadbefae6b68f1ae4e00e6e",
|
||||
"cmd/go-omni/scaffold/addModule.go": "8f43bd68c2b3820cf1d036c4f2c5f9d06d1c7c8e5114f5081597d905f8d8cdcc",
|
||||
"cmd/go-omni/scaffold/addModule_test.go": "667c45312f628f0fb5b8206ddef7ccc02a75f33b72f73fd6a17b03da0cb62e91",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl": "9d3bfc46cac5618bcff7d8a6b2cc9f123e818aeb3dee9bcdfe0f91db584994bd",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl": "729cd0f6faba53d5d2c1492935e12ac2adcefde4490748e25c18be7ccb3f7139",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl": "a005611f6b455d4d8469e316bc26fa571043b2da1834db2b3dd14a96d66221d7",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl": "fb0be7fd571ba2c0edb01d70d68cdbbfe53561caab98346bf1ac7cd8df07205a",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/repository-repository.go.tmpl": "902da3d9f12395fb226178c4428ff547e1c2fce3261882dba98d1400d8c90747",
|
||||
"cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl": "49d26044a69a43e983aa5ba4b5b2399b26b77898b23e230751ad1372c2af3178",
|
||||
"cmd/go-omni/scaffold/scaffold.go": "15baca1b5c4607c8f96fb6f6d4b96a661a86899012f93a6c380bc6d894b18803",
|
||||
"cmd/go-omni/scaffold/scaffold_test.go": "58aab9251b01e394bbc90aeb538ff035dd32c73c0d4815132ac6cbc086527cda",
|
||||
"cmd/go-omni/scaffold/templates/README.md.tmpl": "dc5ce58c49b4c0b50a87fdca794513cad38f428975bdfe772e89944e39614bd4",
|
||||
"cmd/go-omni/scaffold/templates/cmd-server-main.go.tmpl": "07c4f81109ece43fcf15bb807ada5ae6dc7e732babc7293269514a21cd389955",
|
||||
"cmd/go-omni/scaffold/templates/databaseIntegration_test.go.tmpl": "81c4f7fbc919d74b738d3a7338248cbd83cfa0803b2058e8ee184e3ce51e7e9b",
|
||||
"cmd/go-omni/scaffold/templates/go.mod.tmpl": "7d3bb15d2292e36939cc1905258d2800aeaf0e37d7d041b2ff8cfdb6fc84c57a",
|
||||
"cmd/go-omni/scaffold/templates/internal-handler-user.go.tmpl": "7f8dbcd2f9b0129a90160d6059e47d19d4758274060a1a3e5160055d0592d34f",
|
||||
"cmd/go-omni/scaffold/templates/internal-model-errors.go.tmpl": "f123eace52952321b0665a92a2923514c52c8c3bf13df1065710e9675750f837",
|
||||
"cmd/go-omni/scaffold/templates/internal-model-user.go.tmpl": "b391143e7386e169b53e7fed25de1146e8c52f3fa287b12728e8d22b33f386de",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-memory.go.tmpl": "148abc273c1fab3f28536706792762ae41e8687dfe9986c374372f162600feac",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-mysql.go.tmpl": "34bd333e29211a0025ff6ebaea18c7f142f18f1cc451d692f9d360e8c8f6f635",
|
||||
"cmd/go-omni/scaffold/templates/internal-repository-users.go.tmpl": "6d7dd80f34a9c215daeb4a86bfbe11e845e7ff7e9864c8af43e1c2ec3a8b28ec",
|
||||
"cmd/go-omni/scaffold/templates/internal-service-user.go.tmpl": "3fde559ac634a6c85d322dbc3487691a192644268e85aa3911dd2561a4116196",
|
||||
"cmd/go-omni/scaffold/templates/internal-servicecontext-context.go.tmpl": "2504d9750194a543121c9e93328f5f2a535f1db9f8dbd361edf3681cdfa21c39",
|
||||
"cmd/go-omni/scaffold/templates/schema.sql.tmpl": "7c5108f18fc7d8ff75c97afd96fcaa9f8ed026c79c71c5612f55572d73354abb",
|
||||
"cmd/go-omni/scaffold/version.go": "4d2d1b0d6d0ea716ff4cb7ff08cc25e6f91e06b1a4585176e3128a22e2f2f4ad",
|
||||
"config/config.go": "5cbdf774afa62001910bac3d6523399dc5c9dfdafc6111ab86bd5ce1502d7811",
|
||||
"config/config_test.go": "53b72d5bbe2b5a38c0e3c17bd426f1bb5b43beb2014e7391aef9c438d9e521ac",
|
||||
"config/env.go": "2c05cc2d97ae778158a52d390bcec4226d4196833e1e051a76a2eeb425997c0b",
|
||||
"config/file.go": "2e9f0f9ae30f995994ba0841c27bd27ede7f0652b2c822d71140f439e023d95e",
|
||||
"config/file_test.go": "1479d625aad26a1f773bc4a08c146347f163bbf7b4f11385a32f687119df5817",
|
||||
"database/migrate/migrate.go": "fd6cd27d2dce5b35ffe80eb5442c47c657b93ffc6545bbcc1ee4651a70a4f80b",
|
||||
"database/migrate/migrate_test.go": "d7ba91eed8cfd97e4ef0741d582f493dd411b65ad3192f05b167f75d9b9dcf90",
|
||||
"examples/modules/main.go": "6b44b981c0f654e241664c94612cd95ba8a394155b589add0814d81397156a26",
|
||||
"examples/modules/main_test.go": "4ceb869a7e1ab3fb037af5411ec3dec47af66738adb66c4db01498d446aab7fb",
|
||||
"examples/platform/main.go": "f4fea7a6cdb37413ccab847de68fcbcf8608142858005e1c75754692e2c676d9",
|
||||
"examples/platform/main_test.go": "fd120cfcda2dcaeef5038bdee0ffaaff88bec4450e285ecb4a4de458230d7a12",
|
||||
"go.mod": "f025c81671bfe1399028ce52b934c3b9c264a788c563dfe63a8eb3bc1c458093",
|
||||
"go.sum": "8db5712ded87123d25b155ae67c5c5b168e8dc39d84e00a256766c5ef199cb76",
|
||||
"httpc/client.go": "bbec35b6234a554a4aefd0fef97ace3850288642faab2e382dcbe19a6cb95cd8",
|
||||
"httpc/client_test.go": "972ebd2efab618629e394484c288afe4e675fcfe0c50121d9fa9071680d6e763",
|
||||
"httpx/context.go": "104434499f074eedf2c54e4d0e90db587feb230ca8bb76803b87f2e42d9a8118",
|
||||
"internal/storagefixture/server.go": "defacc905246a643faf5371e437f9ce458f05461e0a922766034d1f9ffdaf003",
|
||||
"logx/logx.go": "1307adf3758967651dd802b814fc0f5804280163a4dc463478bf3667af641740",
|
||||
"logx/logx_test.go": "0f99eff445decdd81a55b3d5374c22d2e038d2a4748adb2c9367d6d03e481671",
|
||||
"middleware/idempotency.go": "a67e0040cbb15cd0140379154017e8f1549b06fc1f307c2549dad515d977e05f",
|
||||
"middleware/middleware.go": "a1e6cfae40d3e1701238a680009d66121f1bbaed786e81b3f6133eb1198a1cf6",
|
||||
"middleware/middleware_test.go": "9249aac16f251858bfe7937f768dc628bf4cc392af75f90223ccacd0f13d2cb9",
|
||||
"middleware/protection_test.go": "4347bb6074078c4c7bd9f65f5fa13a0ac2bc9a8615b297588cc3358aede69a7e",
|
||||
"middleware/rateLimit.go": "fa78a39495a432b3c3349915421a60bed8a76b0921d1f11d7962171fcb1e25ae",
|
||||
"middleware/redis/store.go": "168cac5588ed56e6ef33f2a5de6f8e91599ddf0583f509dd46f5210940fb148a",
|
||||
"middleware/redis/store_test.go": "6659dd4f486da08f4bffb27c698f18c67c3f8c3b59d9f3ebc68e7cd79b9dae15",
|
||||
"middleware/request_id.go": "d52bf17bf80245bafe878026b044857442f0af8df689bcaed4d967d006cbd947",
|
||||
"middleware/sharedIdempotency.go": "2fcc5dec3564680ee8039a8d43451948165553524c9c85248ec7218a5fa68ced",
|
||||
"middleware/timeout.go": "e2f446d992c9f0c986deb75fa71297f19c78f8e43593f174cd19fe93e385e440",
|
||||
"openapi/openapi.go": "13cc2a4fcaa4a50e6f38a38a3d168b6240dcdb49b2ebb5a2782f744f2635857d",
|
||||
"openapi/openapi_test.go": "c92f4a2f548f0304708ac157402fb8641273473e04c524226f2429c2b64b99ab",
|
||||
"pagination/pagination.go": "5c3ca00d27769dcef18d234ac14d5ccc635759ab5be77ef0c765aeba66d39532",
|
||||
"pagination/pagination_test.go": "f78fa86928d4d34a763e96423b29148bc74403aaf58c04d58fe29e225f6598eb",
|
||||
"queue/queue.go": "99a0a8d577dc401d7e69d01602c1889d93d370514bc333111706602044de158a",
|
||||
"queue/redis/consumer.go": "fdb9163db4917f27e4d716e5b42c64dc9694809c460e53b0bff034ebe2859be6",
|
||||
"queue/redis/consumer_test.go": "ea10f5e4804a3350bcee8bdc4bd0debe42820bd3451e28e36f6afc6aeadc0a46",
|
||||
"queue/redis/store.go": "ab5320bcdc7e80c7f3fdc77d583d895055d8970e0a22dbd78bac9b2801cfc634",
|
||||
"resilience/breaker.go": "ca1052185b7dd92e0c289c50aab7f37fb0e6aec6c4c214ae02dff7b8eb91b137",
|
||||
"resilience/breaker_test.go": "6f5abc9534d2bf2506afa3f79fcbb2efb18fa9d488cb010e870c42aa1c8c20d1",
|
||||
"resilience/http.go": "b84a64cef10943af51035c4fc19569b06cde0872046adf1bd1569b6b38170df4",
|
||||
"scripts/verifyModules.py": "ce68eb1f095bbaf721f924ec67c6fa48e0669303f3cdd83be56df06f97e18277",
|
||||
"storage/local/local.go": "ce5fac2e54680b96db3e7dfd9a8d065888f246c5607882607d79c302ee3d231b",
|
||||
"storage/local/local_test.go": "3a2241840968df8e37fd67169c3ef708228449e81f65aff908c8b60d49434dc0",
|
||||
"storage/oss/oss.go": "4b189138172ff42fb8c7549750eea59286c52bcc336f2e4da9d3831cdf3b41b9",
|
||||
"storage/oss/oss_test.go": "555a954010b1218804dc7951fc75e42eae24eba9a99c0871f5ba787f3fc87d45",
|
||||
"storage/s3/config.go": "626ec6a2c0b06c95ee7eb7aa0832b190cfcdeb8a127b6b89e99204040bc3b041",
|
||||
"storage/s3/s3.go": "a9f92c4c59930dd6bf9ec630bcb128ec46bef335f97704ca47647a9b73e15a58",
|
||||
"storage/s3/s3_test.go": "da670fb98a31ff1498d02682cdcd41e3adbab22a44b37f88907b1ea93ab2977f",
|
||||
"storage/signed.go": "72f0255c6ddd275298ba1b9fd50eb48c1858cc8102c43f6764e46f209fffb634",
|
||||
"storage/signed_test.go": "eaffd40894018814e976477f96cf0fbaecfa1987caad78830d9fc8e96995882f",
|
||||
"storage/storage.go": "cb670872f01cf1d7b4ebf21cec9fc6f52ad93bfdf523b2f28bbb8d9c7fe39a6c",
|
||||
"storage/storagetest/contract.go": "e024c5cd3c8a6c2ed656bad370a707f6960f4e717bc5d7dfd5624f6076c00605",
|
||||
"task/task.go": "ef671a9b7d84e7f53cdb2dcb787881e47b00ef0afb29fb6e4698fa8765a7bfe7",
|
||||
"task/task_test.go": "6e33ecd81345ece6102f48e60050a0d990e1bf75c3840e262d5da2962c743681",
|
||||
"telemetry/adapters.go": "a892ef0c2b268caf266a3ea0f833ecf7e9228ff59aeaeecae6fe4a2170ba51e2",
|
||||
"telemetry/telemetry.go": "5567b9d68f52ca54e34568c5f0c9a5b985d90c0c4b5224e57622d6f60adebe69",
|
||||
"telemetry/telemetry_test.go": "7040df811a5cbf9108af3d5e0a98639d1e31efdc8ce4eee6ac0332f3a58691de"
|
||||
},
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 900,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"redis:7.4-alpine": "sha256:858f009f9709ce576febc734aa78b8f6d624b82571f9ddb6bda4377c833b3499",
|
||||
"quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z": "sha256:14cea493d9a34af32f524e538b8346cf79f3321eff8e708c1e2960462bd8936e",
|
||||
"m.daocloud.io/docker.io/library/mysql:8.4": "sha256:b3b90af2a6552ae30c266fdb7d5dd55f3afb72404bb78d37fe8a23eb857fd3fb"
|
||||
},
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GO_OMNI_SCAFFOLD_INTEGRATION=1",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go test -race -count=1 -timeout=240s ./... && go vet ./... && go build ./... && CGO_ENABLED=0 go test -c -o /output/redis.test ./cache/redis && CGO_ENABLED=0 go test -c -o /output/shared.test ./middleware/redis && CGO_ENABLED=0 go test -c -o /output/queue.test ./queue/redis && CGO_ENABLED=0 go test -c -o /output/s3.test ./storage/s3 && CGO_ENABLED=0 go test -c -o /output/mysql.test ./database/migrate"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-redis",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"--user",
|
||||
"999:999",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=64m,mode=1777",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GO_OMNI_REDIS_TEST_ADDR=127.0.0.1:6379",
|
||||
"redis:7.4-alpine",
|
||||
"-c",
|
||||
"set -eu\nredis-server --version\nredis-server --bind 127.0.0.1 --port 6379 --save '' --appendonly no --maxmemory 32mb --daemonize yes --pidfile /tmp/redis.pid --logfile /tmp/redis.log --dir /tmp\ntrap 'redis-cli -h 127.0.0.1 shutdown nosave >/dev/null 2>&1 || true' EXIT\nattempt=0\nuntil redis-cli -h 127.0.0.1 ping >/dev/null 2>&1; do\n attempt=$((attempt + 1))\n if [ \"$attempt\" -gt 50 ]; then cat /tmp/redis.log; exit 1; fi\n sleep 0.1\ndone\n/test/redis.test -test.v -test.run '^TestRedisIntegration$' -test.count=1 -test.timeout=30s\n/test/shared.test -test.v -test.count=1 -test.timeout=45s\n/test/queue.test -test.v -test.count=1 -test.timeout=45s\n"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-s3",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"--user",
|
||||
"1000:1000",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=64m,mode=1777",
|
||||
"--tmpfs",
|
||||
"/data:rw,nosuid,noexec,size=128m,mode=1777",
|
||||
"-e",
|
||||
"MINIO_ROOT_USER=goOmniTest",
|
||||
"-e",
|
||||
"MINIO_ROOT_PASSWORD=goOmniTestSecret",
|
||||
"-e",
|
||||
"GO_OMNI_S3_TEST_ADDR=127.0.0.1:9000",
|
||||
"quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z",
|
||||
"-c",
|
||||
"set -eu\nminio --version\nminio server /data --address 127.0.0.1:9000 --console-address 127.0.0.1:9001 >/tmp/minio.log 2>&1 &\nserver=$!\ntrap 'kill \"$server\" 2>/dev/null || true; wait \"$server\" 2>/dev/null || true' EXIT\n/test/s3.test -test.v -test.run '^TestS3Integration$' -test.count=1 -test.timeout=45s || { cat /tmp/minio.log; exit 1; }\n"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-modules-7387653de963-mysql",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--cap-drop",
|
||||
"ALL",
|
||||
"--security-opt",
|
||||
"no-new-privileges",
|
||||
"-v",
|
||||
"/tmp/goOmniModulesc4suqkr6:/test:ro",
|
||||
"--entrypoint",
|
||||
"sh",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"--user",
|
||||
"999:999",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=512m,mode=1777",
|
||||
"--tmpfs",
|
||||
"/var/lib/mysql:rw,nosuid,noexec,size=1m,mode=1777",
|
||||
"-e",
|
||||
"GO_OMNI_MIGRATE_TEST_DSN=root@unix(/tmp/mysql.sock)/omni_test?parseTime=true&timeout=2s&readTimeout=5s&writeTimeout=5s",
|
||||
"m.daocloud.io/docker.io/library/mysql:8.4",
|
||||
"-c",
|
||||
"set -eu\nmysqld --version\nmkdir /tmp/mysql\nmysqld --no-defaults --initialize-insecure --datadir=/tmp/mysql --innodb-redo-log-capacity=67108864 >/tmp/init.log 2>&1 || { cat /tmp/init.log; exit 1; }\nmysqld --no-defaults --datadir=/tmp/mysql --socket=/tmp/mysql.sock --pid-file=/tmp/mysql.pid --bind-address=127.0.0.1 --port=3306 --mysqlx=OFF --skip-log-bin --skip-name-resolve --innodb-buffer-pool-size=134217728 --innodb-redo-log-capacity=67108864 --max-connections=8 --log-error=/tmp/mysql.log &\nserver=$!\ntrap 'kill \"$server\" 2>/dev/null || true; wait \"$server\" 2>/dev/null || true' EXIT\nattempt=0\nuntil mysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'SELECT 1' >/dev/null 2>&1; do\n attempt=$((attempt + 1)); if [ \"$attempt\" -gt 200 ]; then cat /tmp/mysql.log; exit 1; fi; sleep 0.1\ndone\nmysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'CREATE DATABASE omni_test'\n/test/mysql.test -test.v -test.run '^TestMySQLIntegration$' -test.count=1 -test.timeout=60s\n"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-modules-7387653de963-build",
|
||||
"go-omni-modules-7387653de963-redis",
|
||||
"go-omni-modules-7387653de963-s3",
|
||||
"go-omni-modules-7387653de963-mysql"
|
||||
],
|
||||
"notes": [
|
||||
"Full repository race/vet/build plus independent generated-module CRUD in the compiler container.",
|
||||
"Real Redis uses a statically linked non-race test binary, loopback only, network=none and no published ports.",
|
||||
"At most one validation container; no pre-existing service or database is accessed."
|
||||
],
|
||||
"logSha256": "e76d05ff859f96330f19e4cb2a96241cf5d63cf118abd80db16fe67ff3a1c51f",
|
||||
"finishedAt": "2026-09-23T06:55:56.890295+00:00",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
119
performance/advancedLinux.txt
Normal file
119
performance/advancedLinux.txt
Normal file
@@ -0,0 +1,119 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading github.com/golang-jwt/jwt/v5 v5.3.1
|
||||
go: downloading github.com/redis/go-redis/v9 v9.17.3
|
||||
go: downloading golang.org/x/mod v0.21.0
|
||||
go: downloading github.com/go-sql-driver/mysql v1.9.3
|
||||
go: downloading github.com/golang-migrate/migrate/v4 v4.18.1
|
||||
go: downloading gopkg.in/yaml.v3 v3.0.1
|
||||
go: downloading go.opentelemetry.io/otel/sdk v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel v1.31.0
|
||||
go: downloading github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0
|
||||
go: downloading github.com/minio/minio-go/v7 v7.0.83
|
||||
go: downloading golang.org/x/net v0.33.0
|
||||
go: downloading github.com/go-playground/validator/v10 v10.20.0
|
||||
go: downloading go.opentelemetry.io/otel/metric v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel/sdk/metric v1.31.0
|
||||
go: downloading go.opentelemetry.io/otel/trace v1.31.0
|
||||
go: downloading filippo.io/edwards25519 v1.1.0
|
||||
go: downloading github.com/cespare/xxhash/v2 v2.3.0
|
||||
go: downloading github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f
|
||||
go: downloading github.com/hashicorp/go-multierror v1.1.1
|
||||
go: downloading go.uber.org/atomic v1.7.0
|
||||
go: downloading golang.org/x/time v0.5.0
|
||||
go: downloading github.com/goccy/go-json v0.10.4
|
||||
go: downloading github.com/google/uuid v1.6.0
|
||||
go: downloading github.com/klauspost/compress v1.17.11
|
||||
go: downloading github.com/minio/md5-simd v1.1.2
|
||||
go: downloading github.com/go-ini/ini v1.67.0
|
||||
go: downloading github.com/gabriel-vasile/mimetype v1.4.3
|
||||
go: downloading github.com/go-playground/universal-translator v0.18.1
|
||||
go: downloading github.com/leodido/go-urn v1.4.0
|
||||
go: downloading golang.org/x/crypto v0.31.0
|
||||
go: downloading golang.org/x/text v0.21.0
|
||||
go: downloading golang.org/x/sys v0.28.0
|
||||
go: downloading github.com/hashicorp/errwrap v1.1.0
|
||||
go: downloading github.com/klauspost/cpuid/v2 v2.2.9
|
||||
go: downloading github.com/dustin/go-humanize v1.0.1
|
||||
go: downloading github.com/rs/xid v1.6.0
|
||||
go: downloading github.com/go-playground/locales v0.14.1
|
||||
go: downloading github.com/go-logr/logr v1.4.2
|
||||
go: downloading github.com/go-logr/stdr v1.2.2
|
||||
ok git.yuntuo.hk/k2safe/go-omni 1.098s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/app 1.652s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/audit 1.016s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/auth 1.048s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cache 1.034s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cache/redis 1.014s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni 1.194s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold 4.092s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/config 1.042s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/database 1.013s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/database/migrate 1.053s
|
||||
? git.yuntuo.hk/k2safe/go-omni/errors [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/basic [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/modules 1.072s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/platform 1.105s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/reference 1.156s
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/handler [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/model [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/examples/reference/repository 1.016s
|
||||
? git.yuntuo.hk/k2safe/go-omni/examples/reference/service [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/health 1.040s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/httpc 1.102s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/httpx 1.033s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/internal/perfcheck 1.050s
|
||||
? git.yuntuo.hk/k2safe/go-omni/internal/storagefixture [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/internal/testutil [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/logx 1.013s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/metrics 1.054s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/middleware 1.059s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/middleware/redis 1.036s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/openapi 1.036s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/pagination 1.013s
|
||||
? git.yuntuo.hk/k2safe/go-omni/queue [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/queue/redis 1.014s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/resilience 1.014s
|
||||
? git.yuntuo.hk/k2safe/go-omni/response [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage 1.020s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/local 1.021s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/oss 1.029s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/storage/s3 1.035s
|
||||
? git.yuntuo.hk/k2safe/go-omni/storage/storagetest [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/task 1.092s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/telemetry 1.038s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport 1.053s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 10.647s
|
||||
? git.yuntuo.hk/k2safe/go-omni/transport/internal/routing [no test files]
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/native 2.513s
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/nethttp 1.071s
|
||||
? git.yuntuo.hk/k2safe/go-omni/transport/transporttest [no test files]
|
||||
? git.yuntuo.hk/k2safe/go-omni/validate [no test files]
|
||||
Redis server v=7.4.11 sha=00000000:0 malloc=jemalloc-5.3.0 bits=64 build=40ff01a501d8e4b6
|
||||
=== RUN TestRedisIntegration
|
||||
--- PASS: TestRedisIntegration (0.09s)
|
||||
PASS
|
||||
=== RUN TestSharedRedisIntegration
|
||||
--- PASS: TestSharedRedisIntegration (0.01s)
|
||||
=== RUN TestExpiredTargetBeyondCleanupBatch
|
||||
--- PASS: TestExpiredTargetBeyondCleanupBatch (0.03s)
|
||||
PASS
|
||||
=== RUN TestQueueRedisIntegration
|
||||
=== RUN TestQueueRedisIntegration/publicationAndCapacity
|
||||
=== RUN TestQueueRedisIntegration/staleAcknowledgement
|
||||
=== RUN TestQueueRedisIntegration/crashReclaimRetryAndDeadLetters
|
||||
--- PASS: TestQueueRedisIntegration (1.11s)
|
||||
--- PASS: TestQueueRedisIntegration/publicationAndCapacity (0.00s)
|
||||
--- PASS: TestQueueRedisIntegration/staleAcknowledgement (0.00s)
|
||||
--- PASS: TestQueueRedisIntegration/crashReclaimRetryAndDeadLetters (1.11s)
|
||||
PASS
|
||||
minio version RELEASE.2025-09-07T16-13-09Z (commit-id=07c3a429bfed433e49018cb0f78a52145d4bedeb)
|
||||
Runtime: go1.24.6 linux/amd64
|
||||
License: GNU AGPLv3 - https://www.gnu.org/licenses/agpl-3.0.html
|
||||
Copyright: 2015-2025 MinIO, Inc.
|
||||
=== RUN TestS3Integration
|
||||
--- PASS: TestS3Integration (0.33s)
|
||||
PASS
|
||||
/usr/sbin/mysqld Ver 8.4.11 for Linux on x86_64 (MySQL Community Server - GPL)
|
||||
=== RUN TestMySQLIntegration
|
||||
--- PASS: TestMySQLIntegration (0.03s)
|
||||
PASS
|
||||
80
performance/http2FlowControlLinux.json
Normal file
80
performance/http2FlowControlLinux.json
Normal file
@@ -0,0 +1,80 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"testedCommit": "fa2be5e0166f1fa581a2b0c7ee2fc29c44ac2676",
|
||||
"startedAt": "2026-09-23T02:50:32.502669+00:00",
|
||||
"imageId": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"command": [
|
||||
"docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-boundary-363c306dd46d",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go test -race -count=1 -timeout 2m -v ./transport/http2 && go vet ./transport/http2 && go build ./transport/http2"
|
||||
],
|
||||
"wallTimeoutSeconds": 300,
|
||||
"finishedAt": "2026-09-23T02:51:28.854252+00:00",
|
||||
"exitCode": 0,
|
||||
"logSha256": "52c96133d72ac15fa13b5784ba045a732d5ed0cf728f111481f0046407efff81",
|
||||
"platform": "linux/amd64",
|
||||
"goVersion": "go1.25.14",
|
||||
"package": "./transport/http2",
|
||||
"checks": [
|
||||
"race",
|
||||
"vet",
|
||||
"build"
|
||||
],
|
||||
"passedTests": [
|
||||
"TestTLSConnectionLimitRecoversAfterHandshakeTimeout",
|
||||
"TestSlowResponseStreamIsolationAndRecovery",
|
||||
"TestConnectionWindowIsolationAndRecovery",
|
||||
"TestShutdownReleasesFlowControlledWriter",
|
||||
"TestShutdownSendsGoAwayAndDrains",
|
||||
"TestRSTStreamCancelsOnlyOneRequest",
|
||||
"TestCanceledServeContextWaitsForShutdownDeadline",
|
||||
"TestTLSMixedSoak",
|
||||
"TestServerHandlesH2CRequest",
|
||||
"TestServerEnforcesGlobalRequestLimitAndRecovers",
|
||||
"TestServerRejectsBodyAboveLimitAndRecovers",
|
||||
"TestServerRejectsHeadersAboveLimitAndRecovers",
|
||||
"TestServerMultiplexes128Streams",
|
||||
"TestServerStreamsLargeBodyWithFlowControl",
|
||||
"TestServerTLSUsesH2ALPN",
|
||||
"TestTLSSoak"
|
||||
],
|
||||
"notes": [
|
||||
"Sequentially run after the 30-minute mixed soak; limited to 2 CPUs, 2048 MiB RAM with no swap, and 256 tasks.",
|
||||
"The working tree was clean at testedCommit and mounted read-only. The supervisor enforced a 300-second wall timeout and removed only its own container.",
|
||||
"Malformed-frame and header-limit tests deliberately trigger HTTP/2 error logs; each corresponding test passed.",
|
||||
"Race-detector memory in this regression log is not a production memory baseline. Ordinary-mode mixed-soak measurements are recorded separately."
|
||||
],
|
||||
"postRunCheck": {
|
||||
"checkedAt": "2026-09-23T02:53:33.308077+00:00",
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingHTTP2ValidationContainers": []
|
||||
}
|
||||
}
|
||||
53
performance/http2FlowControlLinux.txt
Normal file
53
performance/http2FlowControlLinux.txt
Normal file
@@ -0,0 +1,53 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestTLSConnectionLimitRecoversAfterHandshakeTimeout
|
||||
--- PASS: TestTLSConnectionLimitRecoversAfterHandshakeTimeout (1.12s)
|
||||
=== RUN TestSlowResponseStreamIsolationAndRecovery
|
||||
--- PASS: TestSlowResponseStreamIsolationAndRecovery (0.00s)
|
||||
=== RUN TestConnectionWindowIsolationAndRecovery
|
||||
--- PASS: TestConnectionWindowIsolationAndRecovery (0.00s)
|
||||
=== RUN TestShutdownReleasesFlowControlledWriter
|
||||
--- PASS: TestShutdownReleasesFlowControlledWriter (0.10s)
|
||||
=== RUN TestShutdownSendsGoAwayAndDrains
|
||||
--- PASS: TestShutdownSendsGoAwayAndDrains (0.00s)
|
||||
=== RUN TestRSTStreamCancelsOnlyOneRequest
|
||||
--- PASS: TestRSTStreamCancelsOnlyOneRequest (0.00s)
|
||||
=== RUN TestCanceledServeContextWaitsForShutdownDeadline
|
||||
--- PASS: TestCanceledServeContextWaitsForShutdownDeadline (0.10s)
|
||||
=== RUN TestTLSMixedSoak
|
||||
mixedSoak_test.go:37: phase=before elapsed=1ms requests=0 failures=0 heapBytes=2409792 rssBytes=37199872 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:38: scenario=mixed connections=8 workersPerConnection=16 uploadBytes=262144 uploadChunkBytes=4096 uploadDelay=2ms downloadBytes=8388608 downloadChunkBytes=65536 downloadDelay=20ms
|
||||
mixedSoak_test.go:190: phase=after elapsed=5.655s requests=10489 failures=0 heapBytes=62134360 rssBytes=470847488 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:4 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:124: mixedTotals phase=after health=10356 slowUploads=117 slowDownloads=16 uploadedBytes=30670848 downloadedBytes=134217728 serverFailures=0
|
||||
mixedSoak_test.go:196: connection=0 health=1318 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=1 health=1228 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=2 health=1194 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=3 health=1322 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=4 health=1331 slowUploads=14 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=5 health=1342 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=6 health=1321 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:196: connection=7 health=1300 slowUploads=15 slowDownloads=2
|
||||
mixedSoak_test.go:209: phase=afterReclaim elapsed=5.661s requests=10489 failures=0 heapBytes=440880 rssBytes=468951040 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:4 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSMixedSoak (5.66s)
|
||||
=== RUN TestServerHandlesH2CRequest
|
||||
--- PASS: TestServerHandlesH2CRequest (0.00s)
|
||||
=== RUN TestServerEnforcesGlobalRequestLimitAndRecovers
|
||||
--- PASS: TestServerEnforcesGlobalRequestLimitAndRecovers (0.00s)
|
||||
=== RUN TestServerRejectsBodyAboveLimitAndRecovers
|
||||
--- PASS: TestServerRejectsBodyAboveLimitAndRecovers (0.00s)
|
||||
=== RUN TestServerRejectsHeadersAboveLimitAndRecovers
|
||||
2026/09/23 02:51:04 http2: server connection error from 127.0.0.1:36276: connection error: COMPRESSION_ERROR
|
||||
--- PASS: TestServerRejectsHeadersAboveLimitAndRecovers (1.00s)
|
||||
=== RUN TestServerMultiplexes128Streams
|
||||
--- PASS: TestServerMultiplexes128Streams (0.02s)
|
||||
=== RUN TestServerStreamsLargeBodyWithFlowControl
|
||||
--- PASS: TestServerStreamsLargeBodyWithFlowControl (0.16s)
|
||||
=== RUN TestServerTLSUsesH2ALPN
|
||||
--- PASS: TestServerTLSUsesH2ALPN (0.14s)
|
||||
=== RUN TestTLSSoak
|
||||
soak_test.go:26: phase=before elapsed=0s requests=0 failures=0 heapBytes=1497944 rssBytes=391569408 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
soak_test.go:159: phase=after elapsed=3.122s requests=18182 failures=0 heapBytes=4739552 rssBytes=238292992 goroutines=3 fileDescriptors=6 stats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSSoak (3.12s)
|
||||
PASS
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 12.478s
|
||||
3681
performance/http2MixedSoakLinux.json
Normal file
3681
performance/http2MixedSoakLinux.json
Normal file
File diff suppressed because it is too large
Load Diff
265
performance/http2MixedSoakLinux.txt
Normal file
265
performance/http2MixedSoakLinux.txt
Normal file
@@ -0,0 +1,265 @@
|
||||
HTTP/2 Linux soak image: golang:1.25
|
||||
HTTP/2 TLS soak duration: 30m
|
||||
HTTP/2 TLS soak mode: regular
|
||||
HTTP/2 TLS soak scenario: mixed
|
||||
HTTP/2 soak container: go-omni-http2-soak-fa422c57dfa1; limits: 2 CPUs, 2048 MiB, 256 tasks
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestTLSMixedSoak
|
||||
mixedSoak_test.go:37: phase=before elapsed=0s requests=0 failures=0 heapBytes=170136 rssBytes=8613888 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:0 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:38: scenario=mixed connections=8 workersPerConnection=16 uploadBytes=262144 uploadChunkBytes=4096 uploadDelay=2ms downloadBytes=8388608 downloadChunkBytes=65536 downloadDelay=20ms
|
||||
mixedSoak_test.go:135: phase=running elapsed=15.143s requests=444111 failures=0 heapBytes=41226328 rssBytes=100597760 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=443187 slowUploads=884 slowDownloads=40 uploadedBytes=231735296 downloadedBytes=335544320 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=30.143s requests=886889 failures=0 heapBytes=76939344 rssBytes=119885824 goroutines=351 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=885025 slowUploads=1776 slowDownloads=88 uploadedBytes=465567744 downloadedBytes=738197504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=45.143s requests=1327236 failures=0 heapBytes=98442464 rssBytes=119926784 goroutines=348 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=1324437 slowUploads=2663 slowDownloads=136 uploadedBytes=698089472 downloadedBytes=1140850688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m0.143s requests=1768857 failures=0 heapBytes=99328992 rssBytes=119402496 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=1765116 slowUploads=3557 slowDownloads=184 uploadedBytes=932446208 downloadedBytes=1543503872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m15.143s requests=2211456 failures=0 heapBytes=60732216 rssBytes=90337280 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=2206781 slowUploads=4445 slowDownloads=232 uploadedBytes=1165230080 downloadedBytes=1946157056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m30.143s requests=2652108 failures=0 heapBytes=53063064 rssBytes=85688320 goroutines=350 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=2646501 slowUploads=5335 slowDownloads=272 uploadedBytes=1398538240 downloadedBytes=2281701376 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=1m45.143s requests=3094082 failures=0 heapBytes=81458456 rssBytes=117563392 goroutines=343 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3087534 slowUploads=6228 slowDownloads=320 uploadedBytes=1632632832 downloadedBytes=2684354560 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m0.143s requests=3532884 failures=0 heapBytes=86878776 rssBytes=119468032 goroutines=292 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3525398 slowUploads=7118 slowDownloads=368 uploadedBytes=1865940992 downloadedBytes=3087007744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m15.143s requests=3974010 failures=0 heapBytes=98205016 rssBytes=119029760 goroutines=313 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=3965593 slowUploads=8008 slowDownloads=416 uploadedBytes=2099249152 downloadedBytes=3489660928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m30.143s requests=4415820 failures=0 heapBytes=90842632 rssBytes=109375488 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=4406456 slowUploads=8900 slowDownloads=464 uploadedBytes=2333081600 downloadedBytes=3892314112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=2m45.143s requests=4859795 failures=0 heapBytes=46212528 rssBytes=75120640 goroutines=382 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=4849501 slowUploads=9790 slowDownloads=504 uploadedBytes=2566389760 downloadedBytes=4227858432 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m0.143s requests=5302557 failures=0 heapBytes=59378560 rssBytes=103960576 goroutines=323 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=5291324 slowUploads=10681 slowDownloads=552 uploadedBytes=2799960064 downloadedBytes=4630511616 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m15.143s requests=5745355 failures=0 heapBytes=65581312 rssBytes=120012800 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=5733185 slowUploads=11571 slowDownloads=600 uploadedBytes=3033268224 downloadedBytes=5033164800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m30.143s requests=6184725 failures=0 heapBytes=83765136 rssBytes=119152640 goroutines=365 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=6171616 slowUploads=12461 slowDownloads=648 uploadedBytes=3266576384 downloadedBytes=5435817984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=3m45.143s requests=6627613 failures=0 heapBytes=83719272 rssBytes=119508992 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=6613566 slowUploads=13352 slowDownloads=696 uploadedBytes=3500146688 downloadedBytes=5838471168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m0.143s requests=7072007 failures=0 heapBytes=40007456 rssBytes=83099648 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7057021 slowUploads=14246 slowDownloads=740 uploadedBytes=3734503424 downloadedBytes=6207569920 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m15.143s requests=7515397 failures=0 heapBytes=52621088 rssBytes=88940544 goroutines=320 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7499478 slowUploads=15136 slowDownloads=784 uploadedBytes=3967811584 downloadedBytes=6576668672 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m30.143s requests=7957026 failures=0 heapBytes=49206800 rssBytes=115929088 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=7940168 slowUploads=16026 slowDownloads=832 uploadedBytes=4201119744 downloadedBytes=6979321856 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=4m45.143s requests=8395015 failures=0 heapBytes=57830768 rssBytes=120049664 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=8377223 slowUploads=16918 slowDownloads=880 uploadedBytes=4434952192 downloadedBytes=7381975040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m0.143s requests=8838797 failures=0 heapBytes=66531296 rssBytes=120057856 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=8820061 slowUploads=17808 slowDownloads=928 uploadedBytes=4668260352 downloadedBytes=7784628224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m15.143s requests=9279778 failures=0 heapBytes=60693440 rssBytes=103329792 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=9260102 slowUploads=18700 slowDownloads=976 uploadedBytes=4902092800 downloadedBytes=8187281408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m30.143s requests=9726070 failures=0 heapBytes=29946072 rssBytes=75563008 goroutines=288 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=9705463 slowUploads=19591 slowDownloads=1016 uploadedBytes=5135663104 downloadedBytes=8522825728 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=5m45.144s requests=10170774 failures=0 heapBytes=41152368 rssBytes=106692608 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=10149227 slowUploads=20483 slowDownloads=1064 uploadedBytes=5369495552 downloadedBytes=8925478912 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m0.143s requests=10614632 failures=0 heapBytes=80360752 rssBytes=120602624 goroutines=325 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=10592151 slowUploads=21373 slowDownloads=1112 uploadedBytes=5602803712 downloadedBytes=9328132096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m15.143s requests=11059452 failures=0 heapBytes=87305144 rssBytes=120602624 goroutines=324 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11036026 slowUploads=22266 slowDownloads=1160 uploadedBytes=5836898304 downloadedBytes=9730785280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m30.143s requests=11499123 failures=0 heapBytes=95978912 rssBytes=120606720 goroutines=365 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11474760 slowUploads=23155 slowDownloads=1208 uploadedBytes=6069944320 downloadedBytes=10133438464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=6m45.143s requests=11944358 failures=0 heapBytes=56042736 rssBytes=75292672 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=11919058 slowUploads=24048 slowDownloads=1252 uploadedBytes=6304038912 downloadedBytes=10502537216 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m0.143s requests=12386374 failures=0 heapBytes=40072440 rssBytes=94666752 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=12360140 slowUploads=24938 slowDownloads=1296 uploadedBytes=6537347072 downloadedBytes=10871635968 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m15.143s requests=12830700 failures=0 heapBytes=62440144 rssBytes=120365056 goroutines=328 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=12803533 slowUploads=25829 slowDownloads=1344 uploadedBytes=6770917376 downloadedBytes=11274289152 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m30.143s requests=13273492 failures=0 heapBytes=67101096 rssBytes=121159680 goroutines=334 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=13245381 slowUploads=26719 slowDownloads=1392 uploadedBytes=7004225536 downloadedBytes=11676942336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=7m45.143s requests=13715931 failures=0 heapBytes=78929632 rssBytes=120643584 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=13686880 slowUploads=27611 slowDownloads=1440 uploadedBytes=7238057984 downloadedBytes=12079595520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m0.143s requests=14158538 failures=0 heapBytes=83472520 rssBytes=102039552 goroutines=342 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=14128556 slowUploads=28500 slowDownloads=1488 uploadedBytes=7471104000 downloadedBytes=12482248704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m15.143s requests=14606106 failures=0 heapBytes=36808416 rssBytes=77897728 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=14575184 slowUploads=29394 slowDownloads=1528 uploadedBytes=7705460736 downloadedBytes=12817793024 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m30.143s requests=15050580 failures=0 heapBytes=51099992 rssBytes=106217472 goroutines=315 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15018720 slowUploads=30284 slowDownloads=1576 uploadedBytes=7938768896 downloadedBytes=13220446208 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=8m45.143s requests=15492752 failures=0 heapBytes=99120736 rssBytes=121679872 goroutines=326 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15459953 slowUploads=31175 slowDownloads=1624 uploadedBytes=8172339200 downloadedBytes=13623099392 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m0.143s requests=15937860 failures=0 heapBytes=95959552 rssBytes=121716736 goroutines=344 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=15904122 slowUploads=32066 slowDownloads=1672 uploadedBytes=8405909504 downloadedBytes=14025752576 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m15.143s requests=16379592 failures=0 heapBytes=54420112 rssBytes=121249792 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=16344914 slowUploads=32958 slowDownloads=1720 uploadedBytes=8639741952 downloadedBytes=14428405760 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m30.143s requests=16825611 failures=0 heapBytes=62855872 rssBytes=81920000 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=16790004 slowUploads=33848 slowDownloads=1764 uploadedBytes=8873050112 downloadedBytes=14797504512 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=9m45.144s requests=17273037 failures=0 heapBytes=56223560 rssBytes=98967552 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=17236495 slowUploads=34739 slowDownloads=1808 uploadedBytes=9106620416 downloadedBytes=15166603264 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m0.143s requests=17718289 failures=0 heapBytes=90363368 rssBytes=121196544 goroutines=312 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=17680804 slowUploads=35632 slowDownloads=1856 uploadedBytes=9340715008 downloadedBytes=15569256448 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m15.143s requests=18164393 failures=0 heapBytes=91620912 rssBytes=121708544 goroutines=328 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=18125968 slowUploads=36521 slowDownloads=1904 uploadedBytes=9573761024 downloadedBytes=15971909632 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m30.143s requests=18604920 failures=0 heapBytes=85189216 rssBytes=119672832 goroutines=397 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=18565557 slowUploads=37412 slowDownloads=1952 uploadedBytes=9807331328 downloadedBytes=16374562816 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=10m45.143s requests=19049168 failures=0 heapBytes=71446040 rssBytes=97730560 goroutines=371 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19008872 slowUploads=38304 slowDownloads=2000 uploadedBytes=10041163776 downloadedBytes=16777216000 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m0.143s requests=19496908 failures=0 heapBytes=35314648 rssBytes=81412096 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:6}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19455674 slowUploads=39195 slowDownloads=2040 uploadedBytes=10274734080 downloadedBytes=17112760320 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m15.143s requests=19941947 failures=0 heapBytes=52063200 rssBytes=112910336 goroutines=353 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=19899771 slowUploads=40088 slowDownloads=2088 uploadedBytes=10508828672 downloadedBytes=17515413504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m30.143s requests=20387835 failures=0 heapBytes=91179064 rssBytes=121262080 goroutines=337 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=20344721 slowUploads=40978 slowDownloads=2136 uploadedBytes=10742136832 downloadedBytes=17918066688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=11m45.143s requests=20834104 failures=0 heapBytes=96727232 rssBytes=121204736 goroutines=373 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=20790049 slowUploads=41871 slowDownloads=2184 uploadedBytes=10976231424 downloadedBytes=18320719872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m0.143s requests=21274032 failures=0 heapBytes=73363000 rssBytes=121729024 goroutines=395 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=21229039 slowUploads=42761 slowDownloads=2232 uploadedBytes=11209539584 downloadedBytes=18723373056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m15.143s requests=21717175 failures=0 heapBytes=55163016 rssBytes=80625664 goroutines=345 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=21671245 slowUploads=43654 slowDownloads=2276 uploadedBytes=11443634176 downloadedBytes=19092471808 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m30.143s requests=22165567 failures=0 heapBytes=58932064 rssBytes=100704256 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22118701 slowUploads=44546 slowDownloads=2320 uploadedBytes=11677466624 downloadedBytes=19461570560 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=12m45.143s requests=22605128 failures=0 heapBytes=65821152 rssBytes=121774080 goroutines=295 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22557331 slowUploads=45436 slowDownloads=2368 uploadedBytes=11910774784 downloadedBytes=19864223744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m0.143s requests=23048491 failures=0 heapBytes=71758864 rssBytes=121741312 goroutines=346 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=22999748 slowUploads=46327 slowDownloads=2416 uploadedBytes=12144345088 downloadedBytes=20266876928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m15.143s requests=23490751 failures=0 heapBytes=93734424 rssBytes=121221120 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=23441069 slowUploads=47219 slowDownloads=2464 uploadedBytes=12378177536 downloadedBytes=20669530112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m30.143s requests=23937415 failures=0 heapBytes=67599608 rssBytes=86278144 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=23886795 slowUploads=48109 slowDownloads=2511 uploadedBytes=12611485696 downloadedBytes=21063794688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=13m45.143s requests=24380375 failures=0 heapBytes=39698288 rssBytes=80867328 goroutines=299 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=24328828 slowUploads=49001 slowDownloads=2553 uploadedBytes=12845318144 downloadedBytes=21416116224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m0.143s requests=24821991 failures=0 heapBytes=56792672 rssBytes=110620672 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=24769501 slowUploads=49890 slowDownloads=2600 uploadedBytes=13078364160 downloadedBytes=21810380800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m15.143s requests=25267435 failures=0 heapBytes=59737600 rssBytes=121171968 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=25214005 slowUploads=50784 slowDownloads=2648 uploadedBytes=13312720896 downloadedBytes=22213033984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m30.143s requests=25712957 failures=0 heapBytes=91392208 rssBytes=119664640 goroutines=341 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=25658586 slowUploads=51675 slowDownloads=2696 uploadedBytes=13546291200 downloadedBytes=22615687168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=14m45.143s requests=26155555 failures=0 heapBytes=57260256 rssBytes=109522944 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26100248 slowUploads=52568 slowDownloads=2744 uploadedBytes=13780385792 downloadedBytes=23018340352 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m0.143s requests=26597733 failures=0 heapBytes=40190080 rssBytes=81768448 goroutines=359 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26541487 slowUploads=53458 slowDownloads=2788 uploadedBytes=14013693952 downloadedBytes=23387439104 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m15.143s requests=27037735 failures=0 heapBytes=66516128 rssBytes=100077568 goroutines=362 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=26980556 slowUploads=54350 slowDownloads=2832 uploadedBytes=14247526400 downloadedBytes=23756537856 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m30.143s requests=27476076 failures=0 heapBytes=60893368 rssBytes=121782272 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=27417961 slowUploads=55241 slowDownloads=2880 uploadedBytes=14481096704 downloadedBytes=24159191040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=15m45.143s requests=27919500 failures=0 heapBytes=88112872 rssBytes=121765888 goroutines=349 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:14}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=27860441 slowUploads=56131 slowDownloads=2928 uploadedBytes=14714404864 downloadedBytes=24561844224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m0.143s requests=28363283 failures=0 heapBytes=76293576 rssBytes=121774080 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=28303284 slowUploads=57023 slowDownloads=2976 uploadedBytes=14948237312 downloadedBytes=24964497408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m15.143s requests=28803143 failures=0 heapBytes=50553176 rssBytes=94076928 goroutines=335 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=28742207 slowUploads=57913 slowDownloads=3023 uploadedBytes=15181545472 downloadedBytes=25358761984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m30.143s requests=29248721 failures=0 heapBytes=50037296 rssBytes=88997888 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=29186854 slowUploads=58806 slowDownloads=3064 uploadedBytes=15415640064 downloadedBytes=25702694912 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=16m45.143s requests=29691598 failures=0 heapBytes=56490576 rssBytes=112680960 goroutines=339 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=29628794 slowUploads=59696 slowDownloads=3112 uploadedBytes=15648948224 downloadedBytes=26105348096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m0.143s requests=30135271 failures=0 heapBytes=101057680 rssBytes=121397248 goroutines=350 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30071522 slowUploads=60589 slowDownloads=3160 uploadedBytes=15883042816 downloadedBytes=26508001280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m15.143s requests=30578566 failures=0 heapBytes=102154656 rssBytes=121921536 goroutines=361 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:27}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30513878 slowUploads=61480 slowDownloads=3208 uploadedBytes=16116613120 downloadedBytes=26910654464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m30.143s requests=31020857 failures=0 heapBytes=86815776 rssBytes=105476096 goroutines=323 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=30955229 slowUploads=62372 slowDownloads=3256 uploadedBytes=16350445568 downloadedBytes=27313307648 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=17m45.143s requests=31459944 failures=0 heapBytes=51699872 rssBytes=79634432 goroutines=310 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=31393379 slowUploads=63265 slowDownloads=3300 uploadedBytes=16584540160 downloadedBytes=27682406400 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m0.143s requests=31903673 failures=0 heapBytes=77125768 rssBytes=103821312 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=31836174 slowUploads=64155 slowDownloads=3344 uploadedBytes=16817848320 downloadedBytes=28051505152 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m15.143s requests=32347088 failures=0 heapBytes=87810928 rssBytes=121810944 goroutines=354 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=32278650 slowUploads=65047 slowDownloads=3392 uploadedBytes=17051680768 downloadedBytes=28454158336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m30.143s requests=32791958 failures=0 heapBytes=83378912 rssBytes=121843712 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=32722582 slowUploads=65940 slowDownloads=3440 uploadedBytes=17285775360 downloadedBytes=28856811520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=18m45.143s requests=33238456 failures=0 heapBytes=73365344 rssBytes=121778176 goroutines=310 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=33168137 slowUploads=66831 slowDownloads=3488 uploadedBytes=17519345664 downloadedBytes=29259464704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m0.143s requests=33678524 failures=0 heapBytes=73470384 rssBytes=96964608 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:13}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=33607268 slowUploads=67723 slowDownloads=3533 uploadedBytes=17753178112 downloadedBytes=29636952064 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m15.143s requests=34124154 failures=0 heapBytes=44371688 rssBytes=86138880 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34051968 slowUploads=68615 slowDownloads=3576 uploadedBytes=17987010560 downloadedBytes=29997662208 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m30.143s requests=34567776 failures=0 heapBytes=69339968 rssBytes=116219904 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34494646 slowUploads=69506 slowDownloads=3624 uploadedBytes=18220580864 downloadedBytes=30400315392 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=19m45.143s requests=35012899 failures=0 heapBytes=77088312 rssBytes=122314752 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=34938829 slowUploads=70399 slowDownloads=3672 uploadedBytes=18454675456 downloadedBytes=30802968576 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m0.143s requests=35459295 failures=0 heapBytes=71491968 rssBytes=121794560 goroutines=346 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=35384287 slowUploads=71292 slowDownloads=3720 uploadedBytes=18688770048 downloadedBytes=31205621760 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m15.143s requests=35904737 failures=0 heapBytes=71675952 rssBytes=106598400 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=35828787 slowUploads=72182 slowDownloads=3768 uploadedBytes=18922078208 downloadedBytes=31608274944 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m30.143s requests=36350214 failures=0 heapBytes=50574208 rssBytes=84537344 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:10}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=36273329 slowUploads=73074 slowDownloads=3811 uploadedBytes=19155910656 downloadedBytes=31968985088 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=20m45.143s requests=36794194 failures=0 heapBytes=52755624 rssBytes=104304640 goroutines=347 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=36716374 slowUploads=73964 slowDownloads=3856 uploadedBytes=19389218816 downloadedBytes=32346472448 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m0.143s requests=37237267 failures=0 heapBytes=90759440 rssBytes=121802752 goroutines=285 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=37158506 slowUploads=74857 slowDownloads=3904 uploadedBytes=19623313408 downloadedBytes=32749125632 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m15.143s requests=37682135 failures=0 heapBytes=75071160 rssBytes=121802752 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=37602436 slowUploads=75747 slowDownloads=3952 uploadedBytes=19856621568 downloadedBytes=33151778816 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m30.143s requests=38125279 failures=0 heapBytes=84651016 rssBytes=117342208 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38044640 slowUploads=76639 slowDownloads=4000 uploadedBytes=20090454016 downloadedBytes=33554432000 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=21m45.143s requests=38569220 failures=0 heapBytes=70593904 rssBytes=97050624 goroutines=373 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38487645 slowUploads=77531 slowDownloads=4044 uploadedBytes=20324286464 downloadedBytes=33923530752 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m0.143s requests=39016925 failures=0 heapBytes=59654016 rssBytes=93556736 goroutines=325 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=38934413 slowUploads=78424 slowDownloads=4088 uploadedBytes=20558381056 downloadedBytes=34292629504 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m15.143s requests=39463910 failures=0 heapBytes=53261856 rssBytes=114139136 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=39380459 slowUploads=79315 slowDownloads=4136 uploadedBytes=20791951360 downloadedBytes=34695282688 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m30.143s requests=39910237 failures=0 heapBytes=62860672 rssBytes=121815040 goroutines=366 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=39825849 slowUploads=80209 slowDownloads=4184 uploadedBytes=21026308096 downloadedBytes=35097935872 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=22m45.143s requests=40360417 failures=0 heapBytes=70322976 rssBytes=121286656 goroutines=368 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=40275086 slowUploads=81099 slowDownloads=4232 uploadedBytes=21259616256 downloadedBytes=35500589056 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m0.143s requests=40801788 failures=0 heapBytes=76912600 rssBytes=97243136 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=40715518 slowUploads=81990 slowDownloads=4280 uploadedBytes=21493186560 downloadedBytes=35903242240 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m15.143s requests=41246645 failures=0 heapBytes=54950248 rssBytes=84979712 goroutines=337 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=41159438 slowUploads=82883 slowDownloads=4324 uploadedBytes=21727281152 downloadedBytes=36272340992 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m30.143s requests=41687236 failures=0 heapBytes=43818576 rssBytes=101183488 goroutines=321 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=41599097 slowUploads=83774 slowDownloads=4368 uploadedBytes=21960851456 downloadedBytes=36641439744 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=23m45.143s requests=42128546 failures=0 heapBytes=54495128 rssBytes=121827328 goroutines=288 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42039470 slowUploads=84667 slowDownloads=4416 uploadedBytes=22194946048 downloadedBytes=37044092928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m0.143s requests=42569443 failures=0 heapBytes=98238024 rssBytes=119263232 goroutines=362 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42479426 slowUploads=85556 slowDownloads=4464 uploadedBytes=22427992064 downloadedBytes=37446746112 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m15.143s requests=43006434 failures=0 heapBytes=61451176 rssBytes=115818496 goroutines=330 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=42915474 slowUploads=86448 slowDownloads=4512 uploadedBytes=22661824512 downloadedBytes=37849399296 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m30.143s requests=43446211 failures=0 heapBytes=53221152 rssBytes=96935936 goroutines=294 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=43354315 slowUploads=87339 slowDownloads=4557 uploadedBytes=22895394816 downloadedBytes=38226886656 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=24m45.143s requests=43888201 failures=0 heapBytes=41094952 rssBytes=88412160 goroutines=306 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=43795377 slowUploads=88230 slowDownloads=4600 uploadedBytes=23128965120 downloadedBytes=38587596800 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m0.143s requests=44331683 failures=0 heapBytes=74520696 rssBytes=117968896 goroutines=360 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=44237912 slowUploads=89123 slowDownloads=4648 uploadedBytes=23363059712 downloadedBytes=38990249984 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m15.143s requests=44771023 failures=0 heapBytes=61522184 rssBytes=121311232 goroutines=314 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=44676311 slowUploads=90016 slowDownloads=4696 uploadedBytes=23597154304 downloadedBytes=39392903168 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m30.143s requests=45210979 failures=0 heapBytes=71823520 rssBytes=121315328 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=45115328 slowUploads=90908 slowDownloads=4744 uploadedBytes=23830986752 downloadedBytes=39795556352 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=25m45.143s requests=45655488 failures=0 heapBytes=73294312 rssBytes=98951168 goroutines=352 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=45558899 slowUploads=91798 slowDownloads=4791 uploadedBytes=24064294912 downloadedBytes=40189820928 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m0.143s requests=46098111 failures=0 heapBytes=61891384 rssBytes=85794816 goroutines=331 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:11}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46000586 slowUploads=92690 slowDownloads=4835 uploadedBytes=24298127360 downloadedBytes=40558919680 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m15.143s requests=46542135 failures=0 heapBytes=60185208 rssBytes=103088128 goroutines=303 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:7}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46443672 slowUploads=93583 slowDownloads=4880 uploadedBytes=24532221952 downloadedBytes=40936407040 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m30.143s requests=46985865 failures=0 heapBytes=82859896 rssBytes=121839616 goroutines=364 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=46886464 slowUploads=94473 slowDownloads=4928 uploadedBytes=24765530112 downloadedBytes=41339060224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=26m45.143s requests=47426140 failures=0 heapBytes=76512768 rssBytes=121827328 goroutines=319 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=47325797 slowUploads=95367 slowDownloads=4976 uploadedBytes=24999886848 downloadedBytes=41741713408 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m0.143s requests=47864835 failures=0 heapBytes=96303144 rssBytes=115740672 goroutines=380 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=47763553 slowUploads=96258 slowDownloads=5024 uploadedBytes=25233457152 downloadedBytes=42144366592 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m15.143s requests=48307600 failures=0 heapBytes=74108016 rssBytes=93282304 goroutines=358 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=48205383 slowUploads=97149 slowDownloads=5068 uploadedBytes=25467027456 downloadedBytes=42513465344 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m30.143s requests=48750920 failures=0 heapBytes=48575472 rssBytes=90746880 goroutines=376 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=48647765 slowUploads=98043 slowDownloads=5112 uploadedBytes=25701384192 downloadedBytes=42882564096 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=27m45.143s requests=49191794 failures=0 heapBytes=89907488 rssBytes=120475648 goroutines=357 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49087700 slowUploads=98934 slowDownloads=5160 uploadedBytes=25934954496 downloadedBytes=43285217280 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m0.143s requests=49637025 failures=0 heapBytes=71241912 rssBytes=120827904 goroutines=338 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49531996 slowUploads=99826 slowDownloads=5208 uploadedBytes=26168786944 downloadedBytes=43687870464 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m15.143s requests=50077201 failures=0 heapBytes=71823760 rssBytes=121327616 goroutines=342 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=49971226 slowUploads=100719 slowDownloads=5256 uploadedBytes=26402881536 downloadedBytes=44090523648 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m30.143s requests=50524644 failures=0 heapBytes=88321680 rssBytes=107315200 goroutines=315 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=50417733 slowUploads=101610 slowDownloads=5303 uploadedBytes=26636451840 downloadedBytes=44484788224 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=28m45.143s requests=50971603 failures=0 heapBytes=49721352 rssBytes=87646208 goroutines=335 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:9}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=50863761 slowUploads=102501 slowDownloads=5346 uploadedBytes=26870022144 downloadedBytes=44845498368 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m0.143s requests=51417802 failures=0 heapBytes=55859872 rssBytes=104423424 goroutines=356 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:8}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=51309017 slowUploads=103394 slowDownloads=5392 uploadedBytes=27104116736 downloadedBytes=45231374336 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m15.143s requests=51862535 failures=0 heapBytes=63482984 rssBytes=121856000 goroutines=349 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:10}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=51752810 slowUploads=104285 slowDownloads=5440 uploadedBytes=27337687040 downloadedBytes=45634027520 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m30.143s requests=52305456 failures=0 heapBytes=85512792 rssBytes=118308864 goroutines=332 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:15}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=52194795 slowUploads=105177 slowDownloads=5488 uploadedBytes=27571519488 downloadedBytes=46036680704 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=29m45.143s requests=52747376 failures=0 heapBytes=74143584 rssBytes=114286592 goroutines=333 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:16}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=52635773 slowUploads=106069 slowDownloads=5536 uploadedBytes=27805351936 downloadedBytes=46439333888 serverFailures=0
|
||||
mixedSoak_test.go:135: phase=running elapsed=30m0.143s requests=53191576 failures=0 heapBytes=43650296 rssBytes=97910784 goroutines=272 fileDescriptors=23 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:8 ActiveRequests:12}
|
||||
mixedSoak_test.go:124: mixedTotals phase=running health=53079033 slowUploads=106963 slowDownloads=5580 uploadedBytes=28039708672 downloadedBytes=46808432640 serverFailures=0
|
||||
mixedSoak_test.go:190: phase=after elapsed=30m2.635s requests=53191687 failures=0 heapBytes=44586352 rssBytes=97927168 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:5 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
mixedSoak_test.go:124: mixedTotals phase=after health=53079128 slowUploads=106971 slowDownloads=5588 uploadedBytes=28041805824 downloadedBytes=46875541504 serverFailures=0
|
||||
mixedSoak_test.go:196: connection=0 health=6632549 slowUploads=13372 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=1 health=6629530 slowUploads=13371 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=2 health=6636544 slowUploads=13376 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=3 health=6634170 slowUploads=13369 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=4 health=6636236 slowUploads=13368 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=5 health=6640096 slowUploads=13371 slowDownloads=698
|
||||
mixedSoak_test.go:196: connection=6 health=6634043 slowUploads=13372 slowDownloads=699
|
||||
mixedSoak_test.go:196: connection=7 health=6635960 slowUploads=13372 slowDownloads=699
|
||||
mixedSoak_test.go:209: phase=afterReclaim elapsed=30m2.639s requests=53191687 failures=0 heapBytes=461176 rssBytes=95506432 goroutines=2 fileDescriptors=6 stats={AcceptedConnections:8 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:5 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestTLSMixedSoak (1802.64s)
|
||||
PASS
|
||||
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 1802.651s
|
||||
PASS HTTP/2 TLS soak
|
||||
42
performance/http2ProxyCleanupLinux.json
Normal file
42
performance/http2ProxyCleanupLinux.json
Normal file
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"runnerSha256": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"startedAt": "2026-09-23T03:13:01.977797+00:00",
|
||||
"cases": [
|
||||
{
|
||||
"case": "wallTimeout",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "Command '['/usr/bin/docker', 'run', '--rm', '--name', 'go-omni-http2-proxy-ba2aaeecdd00-build', '--cpus', '2', '--memory', '2048m', '--memory-swap', '2048m', '--pids-limit', '256', '--log-opt', 'max-size=10m', '--log-opt', 'max-file=1', '-v', '/root/go-omni-validation:/workspace:ro', '-v', '/tmp/goOmniHTTP2Proxyi0b0o9y6:/output', '-w', '/workspace', '-e', 'GOWORK=off', '-e', 'GOMAXPROCS=2', '-e', 'CGO_ENABLED=0', '-e', 'GOPROXY=https://goproxy.cn,direct', 'golang:1.25', 'sh', '-c', 'go version && go vet ./transport/http2 && go test -c -o /output/http2.test ./transport/http2']' timed out after 1.8251966070383787 seconds",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-ba2aaeecdd00-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
},
|
||||
{
|
||||
"case": "sigterm",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "KeyboardInterrupt",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-6307142e05ee-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
},
|
||||
{
|
||||
"case": "sighup",
|
||||
"status": "passed",
|
||||
"expectedFailureExitCode": 1,
|
||||
"runnerError": "KeyboardInterrupt",
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-dcf3bd54fbe4-build"
|
||||
],
|
||||
"preExistingContainersStillRunning": 7,
|
||||
"remainingValidationContainers": []
|
||||
}
|
||||
],
|
||||
"status": "passed",
|
||||
"finishedAt": "2026-09-23T03:13:06.934079+00:00"
|
||||
}
|
||||
142
performance/http2ProxyLinux.json
Normal file
142
performance/http2ProxyLinux.json
Normal file
@@ -0,0 +1,142 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T03:10:31.408631+00:00",
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"go.mod": "28d68eb2c0a019cceea884e88ab2335f44ad930e1890e28871638f4b073b4f86",
|
||||
"go.sum": "d9c2a5d0de94f08bd18a14a9626f9dda559c245d89bf8a689e32acbd2136fb05",
|
||||
"scripts/verifyHTTP2Proxy.py": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"transport/http2/admission_test.go": "342a7d02bba508100a207b77246ca92e0fa04dd5f8e4a27091a01866b6750d39",
|
||||
"transport/http2/flowControl_test.go": "595b2e937cef2142ff6b6386116e338d874d0fe49e63d80f6de2e1268378dba2",
|
||||
"transport/http2/frameObserver.go": "fcdf431f8269ad9cfd65dd14102ce5c42ac8c4b8b833ed42c4ae15f2a7c7f14c",
|
||||
"transport/http2/lifecycle_test.go": "370a16de8b20248bb78021ab338b9f62ec51f733f270c0fa92f48b1a3ccb9eae",
|
||||
"transport/http2/mixedSoak_test.go": "2111737fe0405bc809bf06325573f7356160ae7fcaebcce06dcd94b2d9e7dccd",
|
||||
"transport/http2/proxy_test.go": "808d0fa7e6c3b0237b1adf4a71a4f30ae2dbc7d1a9d53fab164cda33b15c9d69",
|
||||
"transport/http2/server.go": "aeecdc3aec42219862d70ec788bef5df332189895555bc5d390849a51bd0fbfb",
|
||||
"transport/http2/server_test.go": "c4613e77fa885b6362c06c393937c7dcf9a7ffb4a9a32f85e21fc2f741de81a6",
|
||||
"transport/http2/soak_test.go": "52baa37e37dba497c579b6b55d723bda866ac57676254dc878c511345acc5af2"
|
||||
},
|
||||
"repetitions": 3,
|
||||
"race": false,
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 600,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"nginx:1.30.5-alpine": "sha256:bf3201ab56f23e5954646379c775d511fc466e9f11376d9725361064ad07ed35"
|
||||
},
|
||||
"notes": [
|
||||
"Compiler and integration stages run sequentially; at most one 2-CPU/2-GiB validation container is active.",
|
||||
"The integration container has network=none, no published ports and a read-only root filesystem. Client, Nginx and origins communicate over loopback inside that container.",
|
||||
"Both TLS legs validate certificates; invalid upstream CA and hostname must fail before reaching the handler.",
|
||||
"HTTP/2 is verified independently on both legs. This is a functional integration test, not a throughput benchmark or a live Cloudflare deployment.",
|
||||
"Single-process Nginx fixture with generated test certificates. Rolling drain switches between old and backup origins; this does not test Nginx master/worker configuration reloads or live Cloudflare routing."
|
||||
],
|
||||
"proxyVersions": [
|
||||
"nginx version: nginx/1.30.5"
|
||||
],
|
||||
"results": [
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}"
|
||||
],
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/amd64",
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-52da1679e6df-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxyk3cc3h1_:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"CGO_ENABLED=0",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go vet ./transport/http2 && go test -c -o /output/http2.test ./transport/http2"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-52da1679e6df-check",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=128m",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxyk3cc3h1_:/test:ro",
|
||||
"--entrypoint",
|
||||
"/test/http2.test",
|
||||
"-e",
|
||||
"GO_OMNI_HTTP2_PROXY_TEST=1",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"nginx:1.30.5-alpine",
|
||||
"-test.v",
|
||||
"-test.run",
|
||||
"^TestNginxHTTP2Proxy$",
|
||||
"-test.timeout",
|
||||
"120s",
|
||||
"-test.count",
|
||||
"3"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-52da1679e6df-build",
|
||||
"go-omni-http2-proxy-52da1679e6df-check"
|
||||
],
|
||||
"finishedAt": "2026-09-23T03:11:00.363119+00:00",
|
||||
"logSha256": "1263ceb8ce7b65f542898e17031ba56127672d363defd0ee6520199fa1927472",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
40
performance/http2ProxyLinux.txt
Normal file
40
performance/http2ProxyLinux.txt
Normal file
@@ -0,0 +1,40 @@
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (1.03s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.39s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.42s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.22s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.81s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.17s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.45s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.20s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.89s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.21s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.44s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.23s)
|
||||
PASS
|
||||
147
performance/http2ProxyRaceLinux.json
Normal file
147
performance/http2ProxyRaceLinux.json
Normal file
@@ -0,0 +1,147 @@
|
||||
{
|
||||
"status": "passed",
|
||||
"startedAt": "2026-09-23T03:11:00.470197+00:00",
|
||||
"testedCommit": "b54a303ccdc034cd3062e846c6c6b2c24e55ffaa",
|
||||
"workingTreeStatus": "",
|
||||
"sourceSha256": {
|
||||
"go.mod": "28d68eb2c0a019cceea884e88ab2335f44ad930e1890e28871638f4b073b4f86",
|
||||
"go.sum": "d9c2a5d0de94f08bd18a14a9626f9dda559c245d89bf8a689e32acbd2136fb05",
|
||||
"scripts/verifyHTTP2Proxy.py": "9ef2774b1d19c05f1ae8d9527665dba541561a7c677f9984a24e72d5bddb6ad5",
|
||||
"transport/http2/admission_test.go": "342a7d02bba508100a207b77246ca92e0fa04dd5f8e4a27091a01866b6750d39",
|
||||
"transport/http2/flowControl_test.go": "595b2e937cef2142ff6b6386116e338d874d0fe49e63d80f6de2e1268378dba2",
|
||||
"transport/http2/frameObserver.go": "fcdf431f8269ad9cfd65dd14102ce5c42ac8c4b8b833ed42c4ae15f2a7c7f14c",
|
||||
"transport/http2/lifecycle_test.go": "370a16de8b20248bb78021ab338b9f62ec51f733f270c0fa92f48b1a3ccb9eae",
|
||||
"transport/http2/mixedSoak_test.go": "2111737fe0405bc809bf06325573f7356160ae7fcaebcce06dcd94b2d9e7dccd",
|
||||
"transport/http2/proxy_test.go": "808d0fa7e6c3b0237b1adf4a71a4f30ae2dbc7d1a9d53fab164cda33b15c9d69",
|
||||
"transport/http2/server.go": "aeecdc3aec42219862d70ec788bef5df332189895555bc5d390849a51bd0fbfb",
|
||||
"transport/http2/server_test.go": "c4613e77fa885b6362c06c393937c7dcf9a7ffb4a9a32f85e21fc2f741de81a6",
|
||||
"transport/http2/soak_test.go": "52baa37e37dba497c579b6b55d723bda866ac57676254dc878c511345acc5af2"
|
||||
},
|
||||
"repetitions": 3,
|
||||
"race": true,
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pidsLimit": 256
|
||||
},
|
||||
"wallTimeoutSeconds": 600,
|
||||
"imageIds": {
|
||||
"golang:1.25": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"nginx:1.30.5": "sha256:0aa2d81d65bc0cac0407e738b8f07d312c8685a84225fcb4db7bcbdd8c9bdf11"
|
||||
},
|
||||
"notes": [
|
||||
"Compiler and integration stages run sequentially; at most one 2-CPU/2-GiB validation container is active.",
|
||||
"The integration container has network=none, no published ports and a read-only root filesystem. Client, Nginx and origins communicate over loopback inside that container.",
|
||||
"Both TLS legs validate certificates; invalid upstream CA and hostname must fail before reaching the handler.",
|
||||
"HTTP/2 is verified independently on both legs. This is a functional integration test, not a throughput benchmark or a live Cloudflare deployment.",
|
||||
"Single-process Nginx fixture with generated test certificates. Rolling drain switches between old and backup origins; this does not test Nginx master/worker configuration reloads or live Cloudflare routing."
|
||||
],
|
||||
"proxyVersions": [
|
||||
"nginx version: nginx/1.30.5"
|
||||
],
|
||||
"results": [
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}",
|
||||
"tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true",
|
||||
"timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}",
|
||||
"goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}"
|
||||
],
|
||||
"goVersion": "go1.25.14",
|
||||
"platform": "linux/amd64",
|
||||
"commands": [
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"pull",
|
||||
"nginx:1.30.5"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-ca37da979ea6-build",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"-v",
|
||||
"/root/go-omni-validation:/workspace:ro",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxy8ll9pvvs:/output",
|
||||
"-w",
|
||||
"/workspace",
|
||||
"-e",
|
||||
"GOWORK=off",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"-e",
|
||||
"CGO_ENABLED=1",
|
||||
"-e",
|
||||
"GOPROXY=https://goproxy.cn,direct",
|
||||
"golang:1.25",
|
||||
"sh",
|
||||
"-c",
|
||||
"go version && go vet ./transport/http2 && go test -race -c -o /output/http2.test ./transport/http2"
|
||||
],
|
||||
[
|
||||
"/usr/bin/docker",
|
||||
"run",
|
||||
"--rm",
|
||||
"--name",
|
||||
"go-omni-http2-proxy-ca37da979ea6-check",
|
||||
"--cpus",
|
||||
"2",
|
||||
"--memory",
|
||||
"2048m",
|
||||
"--memory-swap",
|
||||
"2048m",
|
||||
"--pids-limit",
|
||||
"256",
|
||||
"--log-opt",
|
||||
"max-size=10m",
|
||||
"--log-opt",
|
||||
"max-file=1",
|
||||
"--network",
|
||||
"none",
|
||||
"--read-only",
|
||||
"--tmpfs",
|
||||
"/tmp:rw,nosuid,noexec,size=128m",
|
||||
"-v",
|
||||
"/tmp/goOmniHTTP2Proxy8ll9pvvs:/test:ro",
|
||||
"--entrypoint",
|
||||
"/test/http2.test",
|
||||
"-e",
|
||||
"GO_OMNI_HTTP2_PROXY_TEST=1",
|
||||
"-e",
|
||||
"GOMAXPROCS=2",
|
||||
"nginx:1.30.5",
|
||||
"-test.v",
|
||||
"-test.run",
|
||||
"^TestNginxHTTP2Proxy$",
|
||||
"-test.timeout",
|
||||
"120s",
|
||||
"-test.count",
|
||||
"3"
|
||||
]
|
||||
],
|
||||
"cleanedContainers": [
|
||||
"go-omni-http2-proxy-ca37da979ea6-build",
|
||||
"go-omni-http2-proxy-ca37da979ea6-check"
|
||||
],
|
||||
"finishedAt": "2026-09-23T03:12:51.684028+00:00",
|
||||
"logSha256": "aaae7678dd4f882c51c008eaeb78afd897ccf2f7ba31c926a40839de755bb595",
|
||||
"preExistingContainersStillRunning": 7
|
||||
}
|
||||
67
performance/http2ProxyRaceLinux.txt
Normal file
67
performance/http2ProxyRaceLinux.txt
Normal file
@@ -0,0 +1,67 @@
|
||||
1.30.5: Pulling from library/nginx
|
||||
757049f14b94: Pulling fs layer
|
||||
af529266d394: Pulling fs layer
|
||||
3a55ec62cf16: Pulling fs layer
|
||||
2aa35770407f: Pulling fs layer
|
||||
f3cb4f951034: Pulling fs layer
|
||||
6b37362b3da7: Pulling fs layer
|
||||
b9614c6dbd86: Pulling fs layer
|
||||
2aa35770407f: Download complete
|
||||
d6363fd2ac35: Download complete
|
||||
3a55ec62cf16: Download complete
|
||||
02ac47b2a7a6: Download complete
|
||||
af529266d394: Download complete
|
||||
757049f14b94: Download complete
|
||||
f3cb4f951034: Download complete
|
||||
6b37362b3da7: Download complete
|
||||
6b37362b3da7: Pull complete
|
||||
b9614c6dbd86: Download complete
|
||||
3a55ec62cf16: Pull complete
|
||||
f3cb4f951034: Pull complete
|
||||
b9614c6dbd86: Pull complete
|
||||
2aa35770407f: Pull complete
|
||||
af529266d394: Pull complete
|
||||
757049f14b94: Pull complete
|
||||
Digest: sha256:0aa2d81d65bc0cac0407e738b8f07d312c8685a84225fcb4db7bcbdd8c9bdf11
|
||||
Status: Downloaded newer image for nginx:1.30.5
|
||||
docker.io/library/nginx:1.30.5
|
||||
go version go1.25.14 linux/amd64
|
||||
go: downloading golang.org/x/net v0.25.0
|
||||
go: downloading golang.org/x/text v0.15.0
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.92s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.31s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.41s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.19s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (1.14s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.28s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.56s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.30s)
|
||||
=== RUN TestNginxHTTP2Proxy
|
||||
proxy_test.go:37: proxyVersion=nginx version: nginx/1.30.5
|
||||
=== RUN TestNginxHTTP2Proxy/TLSAndConnectionReuse
|
||||
proxy_test.go:163: proxyResult=tlsAndReuse reusedRequests=32 upstreamConnections=1 invalidTrustStatus=502 invalidNameStatus=502 recovered=true
|
||||
=== RUN TestNginxHTTP2Proxy/TimeoutAndCancellation
|
||||
proxy_test.go:206: proxyResult=timeoutAndCancellation timeoutStatus=504 canceledHandlers=2 recovered=true stats={AcceptedConnections:3 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:0 RSTStreams:0 GoAwayFrames:0 ActiveConnections:1 ActiveRequests:0}
|
||||
=== RUN TestNginxHTTP2Proxy/GoAwayAndRollingDrain
|
||||
proxy_test.go:281: proxyResult=goAwayAndRollingDrain oldRequests=2 newRequests=20 heldPostCompleted=true duplicateHandlers=0 oldStats={AcceptedConnections:1 RejectedConnections:0 RejectedRequests:0 ProtocolErrors:0 GoAwaySent:1 RSTStreams:0 GoAwayFrames:1 ActiveConnections:0 ActiveRequests:0}
|
||||
--- PASS: TestNginxHTTP2Proxy (0.92s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TLSAndConnectionReuse (0.31s)
|
||||
--- PASS: TestNginxHTTP2Proxy/TimeoutAndCancellation (0.44s)
|
||||
--- PASS: TestNginxHTTP2Proxy/GoAwayAndRollingDrain (0.16s)
|
||||
PASS
|
||||
84
performance/http2SmokeLinux.json
Normal file
84
performance/http2SmokeLinux.json
Normal file
@@ -0,0 +1,84 @@
|
||||
{
|
||||
"createdAt": "2026-09-22T17:22:44.674797+00:00",
|
||||
"platform": "linux/amd64",
|
||||
"image": "golang:1.25",
|
||||
"imageId": "sha256:699337d620559a59b4a2bb298ad59611e535d2ee755a34cf2d2a98f37578dc80",
|
||||
"sourceSha256": {
|
||||
"scripts/verifyHTTP2.py": "2c9f4ca095e16e9b30b80b5d812b2644577222274e5a8041e0fb178e305197a4",
|
||||
"transport/http2/soak_test.go": "9964a1670c8ffe30b27e96375e0ae72d00996b59beede748756013e8fd52eacf"
|
||||
},
|
||||
"containerLimits": {
|
||||
"cpus": 2,
|
||||
"memoryBytes": 2147483648,
|
||||
"memoryAndSwapBytes": 2147483648,
|
||||
"pids": 256
|
||||
},
|
||||
"gomaxprocs": 2,
|
||||
"workers": 128,
|
||||
"notes": [
|
||||
"Bounded smoke validation, not completion of the 30-minute HTTP/2 acceptance gate.",
|
||||
"TLS HTTP/2 over container loopback, with the client and server in one process; memory is not server-only.",
|
||||
"One TCP connection multiplexes the 128 workers; each response checks HTTP/2, status, body and a unique request ID.",
|
||||
"MiB uses 1024*1024 bytes. RSS is /proc/self/statm; heap is runtime.ReadMemStats. Samples every 15 seconds.",
|
||||
"The after sample includes the one monitoring goroutine; it is stopped and joined before the test returns.",
|
||||
"All seven pre-existing containers remained running, and no soak containers remained after cleanup.",
|
||||
"The prior unbounded 30-minute race run was stopped after RSS growth; its result is not counted as a pass.",
|
||||
"The short A/B comparison does not establish the complete cause of the old server incident."
|
||||
],
|
||||
"results": [
|
||||
{
|
||||
"mode": "regular",
|
||||
"duration": "2m",
|
||||
"elapsedSeconds": 120.04,
|
||||
"requests": 3885273,
|
||||
"failures": 0,
|
||||
"protocolErrors": 0,
|
||||
"sampledPeakRssBytes": 22437888,
|
||||
"before": {
|
||||
"heapBytes": 168040,
|
||||
"rssBytes": 8491008,
|
||||
"goroutines": 2,
|
||||
"fileDescriptors": 6
|
||||
},
|
||||
"after": {
|
||||
"heapBytes": 3925848,
|
||||
"rssBytes": 21364736,
|
||||
"goroutines": 3,
|
||||
"fileDescriptors": 6,
|
||||
"activeConnections": 0,
|
||||
"activeRequests": 0
|
||||
},
|
||||
"exitCode": 0
|
||||
},
|
||||
{
|
||||
"mode": "race",
|
||||
"duration": "60s",
|
||||
"elapsedSeconds": 60.127,
|
||||
"requests": 377788,
|
||||
"failures": 0,
|
||||
"protocolErrors": 0,
|
||||
"sampledPeakRssBytes": 492707840,
|
||||
"before": {
|
||||
"heapBytes": 169256,
|
||||
"rssBytes": 23732224,
|
||||
"goroutines": 2,
|
||||
"fileDescriptors": 6
|
||||
},
|
||||
"after": {
|
||||
"heapBytes": 2722632,
|
||||
"rssBytes": 487239680,
|
||||
"goroutines": 3,
|
||||
"fileDescriptors": 6,
|
||||
"activeConnections": 0,
|
||||
"activeRequests": 0
|
||||
},
|
||||
"exitCode": 0
|
||||
}
|
||||
],
|
||||
"cleanupValidation": {
|
||||
"sigterm": "passed",
|
||||
"sighup": "passed",
|
||||
"wallTimeout": "passed"
|
||||
},
|
||||
"thirtyMinuteAcceptance": "pending"
|
||||
}
|
||||
2624
performance/http2SoakLinux.json
Normal file
2624
performance/http2SoakLinux.json
Normal file
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user