feat: add independent platform modules and multi-provider storage

ADVANCED.md

@@ -0,0 +1,147 @@
+# 扩展业务模块
+
+本批提供八类扩展，以及分页、审计和显式数据库迁移。模块按需导入；`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase；Go 导出标识符和 `_test.go` 保持语言约定。
+
+按本次范围决定，**不集成 Orbit，不引入 Orbit SDK，也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit，可独立依赖它的 SDK；当前没有承诺只换队列适配器就能迁移其全部语义。
+
+## 能力与边界
+
+| 模块 | 本批实现 | 明确边界 |
+| --- | --- | --- |
+| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应；不自动推断任意 Handler 行为，不附带 Swagger UI |
+| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供；不自动获取 token 指定的密钥 |
+| `telemetry` | HTTP 入站/出站、Redis、SQL spans，HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发；SQL Query span 不覆盖后续遍历；不接管全局 provider |
+| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别 | 热更新只开放日志级别；连接池、监听、认证等静态变更要求重启 |
+| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖；默认内存状态，不做跨进程全局熔断 |
+| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace；TTL 窗口内保护不等于永久业务去重 |
+| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付，处理器必须幂等；持久性取决于 Redis 部署 |
+| `storage` | Local、S3 兼容（含 R2 配置）、OSS 官方 SDK 适配、临时下载链接 | 云平台专属功能单独扩展；本批没有云账号实网验收 |
+| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表；无自动 down/force；已应用文件摘要需部署方留存校对 |
+| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储；与业务同事务的审计要实现事务型 Sink |
+| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页，不承诺跨页快照，不实现游标分页 |
+
+## 对象存储
+
+业务只持有 `storage.Store`，调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处：
+
+```go
+type FileService struct { Files storage.Store }
+
+// Local：私有目录，流式临时文件 + 原子替换。
+files, err := local.New("/srv/app/objects", 32<<20)
+
+// S3：AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
+files, err := s3.Open(s3.Config{
+    Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
+    Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
+    Credentials: credentialProvider, Transport: transport,
+})
+
+// Cloudflare R2：复用 S3 适配器，签名区域为 auto。
+cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
+cfg.Transport = transport
+files, err := s3.Open(cfg)
+
+// OSS：单独适配官方 Go SDK v2，保留平台签名与凭据提供器。
+client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
+    WithRegion("cn-hangzhou").
+    WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
+files, err := oss.New(client, bucket, 32<<20)
+```
+
+上方是互斥初始化片段，实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`（完整 module 前缀 `git.yuntuo.hk/k2safe/`）；`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
+
+统一约定：key 是对象标识，不是磁盘路径；拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度，调用方保证与内容一致。默认单对象 32 MiB，流式读写；OSS 本适配器只做 simple upload，最大配置 5 GiB。Get 返回的 `Body` 由调用方关闭，读取过程也应有 context 截止。Delete 缺失对象成功；Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理，不会在单次操作后关闭。
+
+业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`；Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时，私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权；对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。首版未封装前端直传、分片续传、生命周期、ACL、CDN 自定义域名和版本管理。
+
+R2 实现 S3 的子集，配置入口使用标准账户 endpoint；法域专属 endpoint 可显式配置。兼容性依据：[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/)，没有假定全部 S3 行为与 OSS 等价。
+
+验证区分：Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试；Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限，不只是切配置。
+
+## 可靠队列
+
+```go
+backend, err := queueRedis.New(redisClient, queueRedis.Config{
+    Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
+    MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
+})
+consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
+    Workers: 4, MaxAttempts: 3,
+    TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
+})
+
+// ID 与内容在业务端持久化；超时后复用原 ID 和原内容。
+receipt, err := backend.Publish(ctx, queue.Message{
+    ID: operationID, Name: "generateReport", Payload: payload,
+})
+
+// 在服务生命周期中运行；返回错误须让监控/进程管理感知。
+err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
+    return generateReportIdempotently(ctx, item.ID, item.Payload)
+})
+```
+
+业务依赖 `queue.Publisher` / `queue.Consumer`，Redis 类型留在组合处。receipt 表示受理，不代表业务成功。普通 error/panic 按配置重试，`queue.Reject(code)` 保留为失败消息；超过次数同样保留。`DeadLetters(after, limit)` 是有界分页，`DiscardDead` 仅在人工核对后删除匹配的失败消息，并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略，框架不自动无限重投。
+
+一个 namespace 使用一个受管 consumer group；多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner，旧执行不能确认新执行；但不能阻止旧处理器已发出的外部副作用，因此仍必须业务幂等。处理器需响应 context；Go 无法强杀不配合的函数，超过 shutdownTimeout 返回错误时，最多 Workers 个处理器可能尚未退出。
+
+容量包含待处理、失败及未过期的已完成身份，满额拒绝新 ID，保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期；未完成/失败消息不自动清除，不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction；本批验证的是投递/接管协议，未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理，不能用 DELCONSUMER 丢掉尚未接管的 pending。
+
+`task.Runner` 仍适用于本进程短任务和定时轮询，重启会丢失；需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
+
+## 认证与授权
+
+`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表；只开放 RS256 / EdDSA，必须有 exp/sub，检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表，轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
+
+`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token；存储仅使用 token 的 SHA-256 键，可 `Revoke`。Redis 缓存可共享 Session，内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
+
+`RequireAuth(authenticator)` 在前，`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal，业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
+
+## 配置与日志热更新
+
+`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase，duration 可以写 `"5s"`；文件上限 1 MiB，拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线，例如 `BILLING_`。空环境变量沿用文件值，不清空已配置字段。
+
+```yaml
+address: '127.0.0.1:8080'
+transport: native
+shutdownTimeout: 10s
+log:
+  level: info
+  format: json
+```
+
+先按配置创建 `logx.New(cfg.Log)`，再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`；后者阻塞，生命周期由应用负责。只有 `log.level` 允许变化，校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
+
+## 熔断与观测
+
+为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`；通用调用用 `Do`，HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取，必须关闭 body；网络错误/5xx 计失败，冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
+
+`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`；可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter，基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站，`telemetry.Transport` 采集出站并传播 W3C trace context；Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`，SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
+
+创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类；自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定，避免用用户 ID 或对象 key。
+
+## 分布式限流与幂等
+
+`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶；满额/存储失败返回 503，额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`；Scope 必须来自认证后的租户/用户，先认证再幂等。不同请求内容复用同一身份返回 409，已完成请求回放原响应和 requestId。
+
+claim 在开始业务前持久保存，完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期；返回 409 等待核查，`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理，TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace，数据丢失时不存在框架层面的重复执行保证。
+
+## 迁移、审计与分页
+
+迁移命名：`0001_createUsers.up.sql`，可附 `.down.sql`，但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单，不连接数据库；部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁；MySQL DDL 可能部分提交，失败保留 dirty，禁止自动 force 或回滚。当前基于 golang-migrate，数据库版本表不保存每个历史文件校验和，部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止，重试需新建 Runner。
+
+审计：`audit.New(sink, "approvedField")` 建白名单，再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`；补入服务端时间、requestId/traceId，只保留明确允许的属性，Sink 失败原样返回。JSONSink 不保证与业务提交原子一致，需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
+
+分页：`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20，上限 100，最大 offset=1,000,000；拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages，空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
+
+## 验证入口
+
+```sh
+go test -race ./...
+go vet ./...
+go build ./...
+python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
+```
+
+Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器，各限 2 CPU、2 GiB、256 pids；数据阶段无外网、无发布端口，不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准，未提供云凭据的 OSS/R2 不标为实网通过。

MODULES.md

@@ -1,5 +1,7 @@
 # 业务基础模块
 
+第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
+
 这批能力位于 main，尚未包含在 `v0.6.0-preview` 标签中。统一示例见
 [examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`，中间件依赖
 `httpx`，不依赖具体服务端网络实现。各模块按需创建、注入 Service，再注册到应用关闭流程。

README.md

@@ -46,6 +46,13 @@ go-omni/
 ├── cache/        # 有界内存缓存、Redis 适配与合并回源
 ├── httpc/        # 出站 HTTP 连接池、超时和重试
 ├── task/         # 有界后台任务与间隔调度
+├── queue/        # 持久队列接口与 Redis Streams 适配
+├── storage/      # Local、S3/R2、OSS 与临时下载链接
+├── openapi/      # 路由和 OpenAPI 3.0.3 契约
+├── telemetry/    # 可注入 OpenTelemetry 观测
+├── resilience/   # 熔断与上游并发隔离
+├── audit/        # 显式业务审计与可注入 Sink
+├── pagination/   # 有界分页和统一结果
 ├── transport/nethttp/ # 标准库网络适配器
 ├── transport/native/  # 自研 HTTP/1.1 服务端预览
 ├── transport/http2/   # opt-in HTTP/2 实验适配器
@@ -57,6 +64,9 @@ main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台
 使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md)，
 可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
 
+扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
+使用与边界见 [扩展模块](ADVANCED.md)，可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit；需要流程编排的业务项目独立接入。
+
 ## 快速开始
 
 ```go

ROADMAP.md

@@ -81,4 +81,11 @@
 - 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试；测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
 - 当前幂等、限流、任务调度和合并回源均为单进程能力；Redis 仅提供缓存存储，不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
 
+## 7. 扩展业务模块（本批）
+
+- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储；补充迁移、审计和分页。
+- 业务依赖独立接口，云平台 SDK 和 Redis 留在适配器中；按本次范围决定不集成 Orbit，不实现工作流编排。
+- 已完成本地模块测试及 race 检查，隔离 Redis 的共享状态、重复提交、失败消息、超时接管和旧确认拒绝已通过第一轮验证。完整 Linux、真实 MinIO/MySQL 验收记录待本批验证完成后更新。
+- 契约、使用示例与限制见 [扩展模块](ADVANCED.md)；验证入口为 `scripts/verifyModules.py --advanced`，继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
+
 以上阶段以验收结果判断完成。当前仍是基础框架试用阶段；MySQL、适配层契约和一次业务模块试用已完成，自研网络层已有可运行预览，稳定性与生产规模验证尚未完成。

audit/audit.go

@@ -0,0 +1,106 @@
+// Package audit records explicitly selected business events without request bodies.
+package audit
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"io"
+	"sync"
+	"time"
+)
+
+type Event struct {
+	Time      time.Time         `json:"time"`
+	ActorID   string            `json:"actorId"`
+	TenantID  string            `json:"tenantId,omitempty"`
+	Action    string            `json:"action"`
+	Resource  string            `json:"resource"`
+	Outcome   string            `json:"outcome"`
+	RequestID string            `json:"requestId,omitempty"`
+	TraceID   string            `json:"traceId,omitempty"`
+	Fields    map[string]string `json:"fields,omitempty"`
+}
+type Sink interface {
+	Write(context.Context, Event) error
+}
+type Recorder struct {
+	sink    Sink
+	allowed map[string]bool
+}
+
+func New(sink Sink, allowedFields ...string) (*Recorder, error) {
+	if sink == nil || len(allowedFields) > 32 {
+		return nil, errors.New("audit: sink and at most 32 fields required")
+	}
+	r := &Recorder{sink: sink, allowed: map[string]bool{}}
+	for _, name := range allowedFields {
+		if name == "" || len(name) > 64 {
+			return nil, errors.New("audit: invalid field")
+		}
+		r.allowed[name] = true
+	}
+	return r, nil
+}
+func (r *Recorder) Record(ctx context.Context, event Event) error {
+	if ctx == nil {
+		return errors.New("audit: nil context")
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
+		return errors.New("audit: actor/action/resource and valid outcome required")
+	}
+	for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
+		if len(value) > 512 {
+			return errors.New("audit: event field too large")
+		}
+	}
+	event.Time = time.Now().UTC()
+	event.RequestID = logx.RequestID(ctx)
+	event.TraceID = logx.TraceID(ctx)
+	filtered := map[string]string{}
+	for name, value := range event.Fields {
+		if r.allowed[name] {
+			if len(value) > 1024 {
+				return errors.New("audit: attribute too large")
+			}
+			filtered[name] = value
+		}
+	}
+	event.Fields = filtered
+	return r.sink.Write(ctx, event)
+}
+
+// JSONSink borrows its writer and serializes complete JSON lines. Write errors
+// are returned; callers choose whether a failed audit must reject an operation.
+type JSONSink struct {
+	mu     sync.Mutex
+	writer io.Writer
+}
+
+func NewJSONSink(writer io.Writer) (*JSONSink, error) {
+	if writer == nil {
+		return nil, errors.New("audit: writer required")
+	}
+	return &JSONSink{writer: writer}, nil
+}
+func (s *JSONSink) Write(ctx context.Context, event Event) error {
+	data, err := json.Marshal(event)
+	if err != nil {
+		return err
+	}
+	data = append(data, '\n')
+	s.mu.Lock()
+	defer s.mu.Unlock()
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	n, err := s.writer.Write(data)
+	if err == nil && n != len(data) {
+		return io.ErrShortWrite
+	}
+	return err
+}

audit/audit_test.go

@@ -0,0 +1,50 @@
+package audit
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"io"
+	"sync"
+	"testing"
+)
+
+type failedWriter struct{}
+
+func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
+func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
+	var buf bytes.Buffer
+	sink, _ := NewJSONSink(&buf)
+	recorder, _ := New(sink, "status")
+	ctx := logx.WithRequestID(context.Background(), "r1")
+	ctx = logx.WithTraceID(ctx, "t1")
+	event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
+	var wg sync.WaitGroup
+	for i := 0; i < 20; i++ {
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			if err := recorder.Record(ctx, event); err != nil {
+				t.Error(err)
+			}
+		}()
+	}
+	wg.Wait()
+	decoder := json.NewDecoder(&buf)
+	for i := 0; i < 20; i++ {
+		var got Event
+		if err := decoder.Decode(&got); err != nil {
+			t.Fatal(err)
+		}
+		if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
+			t.Fatal(got)
+		}
+	}
+	bad, _ := NewJSONSink(failedWriter{})
+	recorder, _ = New(bad)
+	if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
+		t.Fatal(err)
+	}
+}

auth/adapters_test.go

@@ -0,0 +1,105 @@
+package auth
+
+import (
+	"context"
+	"crypto/ed25519"
+	"crypto/rand"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/cache"
+	"github.com/golang-jwt/jwt/v5"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestJWTValidationAndRotation(t *testing.T) {
+	pub, private, err := ed25519.GenerateKey(rand.Reader)
+	if err != nil {
+		t.Fatal(err)
+	}
+	j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
+	sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
+		t.Helper()
+		token := jwt.NewWithClaims(method, c)
+		token.Header["kid"] = kid
+		raw, err := token.SignedString(key)
+		if err != nil {
+			t.Fatal(err)
+		}
+		return raw
+	}
+	raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
+	// Constructor clones caller-owned key bytes.
+	pub[0] ^= 1
+	p, err := j.Authenticate(context.Background(), raw)
+	if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
+		t.Fatalf("%+v %v", p, err)
+	}
+	for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
+		t.Run(name, func(t *testing.T) {
+			bad := claims
+			change(&bad)
+			if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
+				t.Fatal(err)
+			}
+		})
+	}
+	for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
+		if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
+			t.Fatal(err)
+		}
+	}
+	if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
+		t.Fatal(err)
+	}
+	if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
+		t.Fatal(err)
+	}
+	if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
+		t.Fatal(err)
+	}
+	ctx, cancel := context.WithCancel(context.Background())
+	cancel()
+	if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
+		t.Fatal(err)
+	}
+}
+func TestSessionsRevokeAndIsolation(t *testing.T) {
+	mem, _ := cache.NewMemory(cache.MemoryConfig{})
+	defer mem.Close()
+	s, err := NewSessions(mem, "session:", time.Minute)
+	if err != nil {
+		t.Fatal(err)
+	}
+	ctx := context.Background()
+	token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	key, _ := s.key(token)
+	if strings.Contains(key, token) {
+		t.Fatal("plaintext session secret stored")
+	}
+	p, err := s.Authenticate(ctx, token)
+	if err != nil || p.UserID != "u1" {
+		t.Fatal(p, err)
+	}
+	p.Roles[0] = "admin"
+	p, _ = s.Authenticate(ctx, token)
+	if p.HasRole("admin") {
+		t.Fatal("principal alias")
+	}
+	if err = s.Revoke(ctx, token); err != nil {
+		t.Fatal(err)
+	}
+	if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
+		t.Fatal(err)
+	}
+	if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
+		t.Fatal(err)
+	}
+}

auth/auth.go

@@ -21,9 +21,10 @@ var (
 // Principal is the authenticated identity made available to business handlers.
 // Claims are optional provider-specific values and should not be required by core business code.
 type Principal struct {
-	UserID string
+	UserID   string         `json:"userId"`
-	Roles  []string
+	TenantID string         `json:"tenantId,omitempty"`
-	Claims map[string]any
+	Roles    []string       `json:"roles,omitempty"`
+	Claims   map[string]any `json:"claims,omitempty"`
 }
 
 func (p Principal) HasRole(role string) bool {

auth/jwt.go

@@ -0,0 +1,127 @@
+package auth
+
+import (
+	"context"
+	"crypto/ed25519"
+	"crypto/rsa"
+	"crypto/x509"
+	"errors"
+	"sync"
+	"time"
+
+	"github.com/golang-jwt/jwt/v5"
+)
+
+// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
+// Keys come from server configuration, never a token's jku/x5u URL.
+type JWTKey struct {
+	Algorithm string
+	PublicKey any
+}
+type JWTConfig struct {
+	Issuer   string            `json:"issuer"`
+	Audience string            `json:"audience"`
+	Leeway   time.Duration     `json:"leeway"`
+	Keys     map[string]JWTKey `json:"-"`
+}
+type JWT struct {
+	config JWTConfig
+	mu     sync.RWMutex
+	keys   map[string]JWTKey
+}
+type JWTClaims struct {
+	jwt.RegisteredClaims
+	TenantID string   `json:"tenantId,omitempty"`
+	Roles    []string `json:"roles,omitempty"`
+}
+
+func NewJWT(cfg JWTConfig) (*JWT, error) {
+	if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
+		return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
+	}
+	j := &JWT{config: cfg}
+	j.config.Keys = nil
+	if err := j.ReplaceKeys(cfg.Keys); err != nil {
+		return nil, err
+	}
+	return j, nil
+}
+
+// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
+// their tokens expire if a rotation should preserve existing sessions.
+func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
+	if len(keys) == 0 || len(keys) > 64 {
+		return errors.New("auth: require 1..64 JWT keys")
+	}
+	copyKeys := make(map[string]JWTKey, len(keys))
+	for id, key := range keys {
+		if id == "" || len(id) > 128 {
+			return errors.New("auth: invalid key ID")
+		}
+		switch key.Algorithm {
+		case "RS256":
+			k, ok := key.PublicKey.(*rsa.PublicKey)
+			if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
+				return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
+			}
+		case "EdDSA":
+			k, ok := key.PublicKey.(ed25519.PublicKey)
+			if !ok || len(k) != ed25519.PublicKeySize {
+				return errors.New("auth: EdDSA requires an Ed25519 public key")
+			}
+		default:
+			return errors.New("auth: supported algorithms are RS256 and EdDSA")
+		}
+		encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
+		if err != nil {
+			return err
+		}
+		key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
+		if err != nil {
+			return err
+		}
+		copyKeys[id] = key
+	}
+	j.mu.Lock()
+	j.keys = copyKeys
+	j.mu.Unlock()
+	return nil
+}
+
+func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
+	if ctx == nil {
+		return Principal{}, ErrInvalidToken
+	}
+	if err := ctx.Err(); err != nil {
+		return Principal{}, err
+	}
+	if len(raw) == 0 || len(raw) > 8192 {
+		return Principal{}, ErrInvalidToken
+	}
+	claims := new(JWTClaims)
+	token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
+		id, ok := t.Header["kid"].(string)
+		if !ok {
+			return nil, ErrInvalidToken
+		}
+		j.mu.RLock()
+		key, exists := j.keys[id]
+		j.mu.RUnlock()
+		if !exists || t.Method.Alg() != key.Algorithm {
+			return nil, ErrInvalidToken
+		}
+		return key.PublicKey, nil
+	}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
+	if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
+		return Principal{}, ErrInvalidToken
+	}
+	for _, role := range claims.Roles {
+		if role == "" || len(role) > 128 {
+			return Principal{}, ErrInvalidToken
+		}
+	}
+	if err := ctx.Err(); err != nil {
+		return Principal{}, err
+	}
+	return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
+}

auth/policy.go

@@ -0,0 +1,34 @@
+package auth
+
+import (
+	"context"
+	"errors"
+	apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+)
+
+type Policy interface {
+	Authorize(context.Context, Principal, string, string) error
+}
+type PolicyFunc func(context.Context, Principal, string, string) error
+
+func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
+	return f(ctx, p, action, resource)
+}
+func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
+	return func(next httpx.Handler) httpx.Handler {
+		return func(c *httpx.Context) error {
+			p, ok := FromContext(c)
+			if !ok {
+				return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
+			}
+			if policy == nil || action == "" || resource == nil {
+				return apierrors.Internal(errors.New("auth: incomplete permission policy"))
+			}
+			if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
+				return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
+			}
+			return next(c)
+		}
+	}
+}

auth/session.go

@@ -0,0 +1,87 @@
+package auth
+
+import (
+	"context"
+	"crypto/rand"
+	"crypto/sha256"
+	"encoding/base64"
+	"encoding/hex"
+	"encoding/json"
+	"errors"
+	"time"
+
+	"git.yuntuo.hk/k2safe/go-omni/cache"
+)
+
+// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
+// The caller verifies login credentials before Issue; Store owns persistence.
+type Sessions struct {
+	store  cache.Store
+	prefix string
+	ttl    time.Duration
+}
+
+func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
+	if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
+		return nil, errors.New("auth: invalid session configuration")
+	}
+	return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
+}
+func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
+	if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
+		return "", ErrInvalidToken
+	}
+	data, err := json.Marshal(p)
+	if err != nil {
+		return "", err
+	}
+	if len(data) > 16384 {
+		return "", errors.New("auth: session principal too large")
+	}
+	secret := make([]byte, 32)
+	if _, err := rand.Read(secret); err != nil {
+		return "", err
+	}
+	token := base64.RawURLEncoding.EncodeToString(secret)
+	key, _ := s.key(token)
+	if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
+		return "", err
+	}
+	return token, nil
+}
+func (s *Sessions) key(token string) (string, error) {
+	if len(token) != 43 {
+		return "", ErrInvalidToken
+	}
+	b, err := base64.RawURLEncoding.DecodeString(token)
+	if err != nil || len(b) != 32 {
+		return "", ErrInvalidToken
+	}
+	sum := sha256.Sum256(b)
+	return s.prefix + hex.EncodeToString(sum[:]), nil
+}
+func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
+	key, err := s.key(token)
+	if err != nil {
+		return Principal{}, err
+	}
+	data, err := s.store.Get(ctx, key)
+	if errors.Is(err, cache.ErrMiss) {
+		return Principal{}, ErrInvalidToken
+	}
+	if err != nil {
+		return Principal{}, err
+	}
+	var p Principal
+	if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
+		return Principal{}, ErrInvalidToken
+	}
+	return p, nil
+}
+func (s *Sessions) Revoke(ctx context.Context, token string) error {
+	key, err := s.key(token)
+	if err != nil {
+		return err
+	}
+	return s.store.Delete(ctx, key)
+}

cmd/go-omni/main.go

@@ -17,6 +17,11 @@ func main() {
 	}
 
 	switch os.Args[1] {
+	case "migrate":
+		if err := runMigrate(os.Args[2:]); err != nil {
+			fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
+			os.Exit(1)
+		}
 	case "add":
 		if err := runAdd(os.Args[2:]); err != nil {
 			fmt.Fprintln(os.Stderr, "go-omni add:", err)
@@ -108,6 +113,7 @@ func usage() {
 	fmt.Fprintln(os.Stderr, "GoOmni project tool")
 	fmt.Fprintln(os.Stderr, "")
 	fmt.Fprintln(os.Stderr, "Usage:")
+	fmt.Fprintln(os.Stderr, "  go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
 	fmt.Fprintln(os.Stderr, "  go-omni new [--module <module>] [--force] <directory>")
 	fmt.Fprintln(os.Stderr, "  go-omni add <name> [--dir <project>]  Generate an isolated business module")
 	fmt.Fprintln(os.Stderr, "  --framework-version <version>  Pin a remote framework version (defaults to CLI version)")

cmd/go-omni/migrate.go

@@ -0,0 +1,60 @@
+package main
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"flag"
+	"fmt"
+	migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
+	"io"
+	"os"
+	"os/signal"
+	"syscall"
+)
+
+func runMigrate(args []string) error {
+	flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
+	flags.SetOutput(io.Discard)
+	directory := flags.String("dir", "migrations", "SQL directory")
+	dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
+	apply := flags.Bool("apply", false, "apply pending up migrations")
+	if err := flags.Parse(args); err != nil {
+		return err
+	}
+	if flags.NArg() != 0 {
+		return errors.New("migrate: unexpected argument")
+	}
+	_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
+	if err != nil {
+		return err
+	}
+	if !*apply {
+		data, err := json.MarshalIndent(manifest, "", "  ")
+		if err != nil {
+			return err
+		}
+		fmt.Println(string(data))
+		return nil
+	}
+	dsn := os.Getenv(*dsnEnv)
+	if dsn == "" {
+		return errors.New("migrate: DSN environment variable is empty")
+	}
+	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
+	defer stop()
+	runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
+	if err != nil {
+		return err
+	}
+	defer runner.Close()
+	if err = runner.Up(ctx); err != nil {
+		return err
+	}
+	version, dirty, err := runner.Version()
+	if err != nil {
+		return err
+	}
+	fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
+	return nil
+}

cmd/go-omni/migrate_test.go

@@ -0,0 +1,25 @@
+package main
+
+import (
+	"os"
+	"path/filepath"
+	"testing"
+)
+
+func TestMigrationPreviewDoesNotConnect(t *testing.T) {
+	dir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
+		t.Fatal(err)
+	}
+	t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
+	if err := runMigrate([]string{"--dir", dir}); err != nil {
+		t.Fatal("preview tried to connect", err)
+	}
+	t.Setenv("GO_OMNI_DB_DSN", "")
+	if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
+		t.Fatal("missing DSN accepted")
+	}
+	if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
+		t.Fatal("unsafe unknown flag accepted")
+	}
+}

config/env.go

@@ -13,15 +13,18 @@ const defaultEnvPrefix = "GO_OMNI_"
 // LoadFromEnv loads application settings from GO_OMNI_* variables. The
 // database settings remain owned by database.ConfigFromEnv.
 func LoadFromEnv(prefix string) (Config, error) {
+	return applyEnv(Default(), prefix)
+}
+
+func applyEnv(cfg Config, prefix string) (Config, error) {
 	prefix = strings.TrimSpace(prefix)
 	if prefix == "" {
 		prefix = defaultEnvPrefix
 	}
-	cfg := Default()
 	cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
 	cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
-	cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", "")
+	cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
-	cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", "")
+	cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
 	cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
 	if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
 		cfg.TrustedProxies = strings.Split(value, ",")

config/file.go

@@ -0,0 +1,203 @@
+package config
+
+import (
+	"bytes"
+	"context"
+	"encoding/json"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"gopkg.in/yaml.v3"
+	"io"
+	"os"
+	"path/filepath"
+	"reflect"
+	"strings"
+	"sync"
+	"time"
+)
+
+// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
+// field names for both formats, rejects unknown fields and accepts durations
+// such as "3s". Application-specific validation remains with the caller.
+func DecodeFile[T any](path string, defaults T) (T, error) {
+	var zero T
+	f, err := os.Open(path)
+	if err != nil {
+		return zero, err
+	}
+	defer f.Close()
+	data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
+	if err != nil {
+		return zero, err
+	}
+	if len(data) > 1<<20 {
+		return zero, errors.New("config: file exceeds 1 MiB")
+	}
+	var value any
+	switch strings.ToLower(filepath.Ext(path)) {
+	case ".yaml", ".yml":
+		dec := yaml.NewDecoder(bytes.NewReader(data))
+		if err := dec.Decode(&value); err != nil {
+			return zero, err
+		}
+		var extra any
+		if err := dec.Decode(&extra); err != io.EOF {
+			return zero, errors.New("config: one YAML document required")
+		}
+	case ".json":
+		dec := json.NewDecoder(bytes.NewReader(data))
+		dec.UseNumber()
+		if err := dec.Decode(&value); err != nil {
+			return zero, err
+		}
+		var extra any
+		if err := dec.Decode(&extra); err != io.EOF {
+			return zero, errors.New("config: one JSON document required")
+		}
+	default:
+		return zero, errors.New("config: expected .json, .yaml or .yml")
+	}
+	if _, ok := value.(map[string]any); !ok {
+		return zero, errors.New("config: root must be an object")
+	}
+	value, err = durations(value, reflect.TypeOf(defaults))
+	if err != nil {
+		return zero, err
+	}
+	encoded, err := json.Marshal(value)
+	if err != nil {
+		return zero, err
+	}
+	// Deep copy defaults so errors cannot partially mutate caller-owned maps.
+	original, err := json.Marshal(defaults)
+	if err != nil {
+		return zero, err
+	}
+	var result T
+	if err = json.Unmarshal(original, &result); err != nil {
+		return zero, err
+	}
+	dec := json.NewDecoder(bytes.NewReader(encoded))
+	dec.DisallowUnknownFields()
+	if err = dec.Decode(&result); err != nil {
+		return zero, err
+	}
+	return result, nil
+}
+func durations(v any, t reflect.Type) (any, error) {
+	if t == nil {
+		return v, nil
+	}
+	for t.Kind() == reflect.Pointer {
+		t = t.Elem()
+	}
+	if t == reflect.TypeOf(time.Duration(0)) {
+		if s, ok := v.(string); ok {
+			d, err := time.ParseDuration(s)
+			return int64(d), err
+		}
+		return v, nil
+	}
+	if t.Kind() == reflect.Struct {
+		if m, ok := v.(map[string]any); ok {
+			for n := 0; n < t.NumField(); n++ {
+				field := t.Field(n)
+				name := strings.Split(field.Tag.Get("json"), ",")[0]
+				if name == "-" {
+					continue
+				}
+				if name == "" {
+					name = field.Name
+				}
+				if item, exists := m[name]; exists {
+					converted, err := durations(item, field.Type)
+					if err != nil {
+						return nil, err
+					}
+					m[name] = converted
+				}
+			}
+		}
+	}
+	if t.Kind() == reflect.Slice {
+		if items, ok := v.([]any); ok {
+			for n, item := range items {
+				converted, err := durations(item, t.Elem())
+				if err != nil {
+					return nil, err
+				}
+				items[n] = converted
+			}
+		}
+	}
+	return v, nil
+}
+func LoadFile(path, prefix string) (Config, error) {
+	cfg, err := DecodeFile(path, Default())
+	if err != nil {
+		return Config{}, err
+	}
+	return applyEnv(cfg, prefix)
+}
+
+// Reloader only accepts log.level changes. Static changes and invalid files
+// leave the current snapshot and logger untouched. Watch does not own a goroutine.
+type Reloader struct {
+	mu           sync.Mutex
+	path, prefix string
+	current      Config
+	logger       *logx.Logger
+}
+
+func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
+	cfg, err := LoadFile(path, prefix)
+	if err != nil {
+		return nil, err
+	}
+	if err = logger.SetLevel(cfg.Log.Level); err != nil {
+		return nil, err
+	}
+	return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
+}
+func (r *Reloader) Reload() error {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	next, err := LoadFile(r.path, r.prefix)
+	if err != nil {
+		return err
+	}
+	comparable := next
+	comparable.Log.Level = r.current.Log.Level
+	if !reflect.DeepEqual(comparable, r.current) {
+		return errors.New("config: static fields changed; restart required")
+	}
+	if err = r.logger.SetLevel(next.Log.Level); err != nil {
+		return err
+	}
+	r.current = next
+	return nil
+}
+func (r *Reloader) Snapshot() Config {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	cfg := r.current
+	cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
+	return cfg
+}
+func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
+	if ctx == nil || interval < 100*time.Millisecond {
+		return errors.New("config: context and interval >=100ms required")
+	}
+	ticker := time.NewTicker(interval)
+	defer ticker.Stop()
+	for {
+		select {
+		case <-ctx.Done():
+			return ctx.Err()
+		case <-ticker.C:
+			if err := r.Reload(); err != nil && onError != nil {
+				onError(err)
+			}
+		}
+	}
+}

config/file_test.go

@@ -0,0 +1,59 @@
+package config
+
+import (
+	"bytes"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"os"
+	"path/filepath"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestFileMergeAndSafeReload(t *testing.T) {
+	path := filepath.Join(t.TempDir(), "service.yaml")
+	write := func(raw string) {
+		t.Helper()
+		if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
+			t.Fatal(err)
+		}
+	}
+	write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n  level: info\n")
+	t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
+	cfg, err := LoadFile(path, "TEST_MODULE_")
+	if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
+		t.Fatal(cfg, err)
+	}
+	var output bytes.Buffer
+	lc := cfg.Log
+	lc.Writer = &output
+	l, err := logx.New(lc)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer l.Close()
+	child := l.With("scope", "test")
+	r, err := NewReloader(path, "TEST_MODULE_", l)
+	if err != nil {
+		t.Fatal(err)
+	}
+	child.Debug("hidden")
+	write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n  level: debug\n")
+	if err = r.Reload(); err != nil {
+		t.Fatal(err)
+	}
+	child.Debug("visible")
+	if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
+		t.Fatal(output.String())
+	}
+	write("shutdownTimeout: 8s\nlog:\n  level: error\n")
+	if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
+		t.Fatal("static reload changed live config", err)
+	}
+	for _, raw := range []string{"unknownOption: true", "log:\n  level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
+		write(raw)
+		if err = r.Reload(); err == nil {
+			t.Fatal(raw)
+		}
+	}
+}

database/migrate/migrate.go

@@ -0,0 +1,245 @@
+// Package migrate provides explicit MySQL schema migration jobs. Application
+// startup never runs migrations automatically; use a separate deployment step.
+package migrate
+
+import (
+	"archive/zip"
+	"bytes"
+	"context"
+	"crypto/sha256"
+	"database/sql"
+	"encoding/hex"
+	"errors"
+	"fmt"
+	mysqlDriver "github.com/go-sql-driver/mysql"
+	migration "github.com/golang-migrate/migrate/v4"
+	migrateMySQL "github.com/golang-migrate/migrate/v4/database/mysql"
+	"github.com/golang-migrate/migrate/v4/source/iofs"
+	"io"
+	"io/fs"
+	"path"
+	"regexp"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"time"
+)
+
+type File struct {
+	Version   uint64 `json:"version"`
+	Name      string `json:"name"`
+	Direction string `json:"direction"`
+	SHA256    string `json:"sha256"`
+}
+
+var filename = regexp.MustCompile(`^([0-9]+)_([A-Za-z][A-Za-z0-9]*)\.(up|down)\.sql$`)
+
+// Snapshot returns immutable migration bytes and a reviewable digest manifest.
+// Follow lowerCamelCase names, e.g. 0001_createUsers.up.sql.
+func Snapshot(source fs.FS, directory string) (fs.FS, []File, error) {
+	if source == nil || !fs.ValidPath(directory) {
+		return nil, nil, errors.New("migrate: invalid source directory")
+	}
+	entries, err := fs.ReadDir(source, directory)
+	if err != nil {
+		return nil, nil, err
+	}
+	var archive bytes.Buffer
+	writer := zip.NewWriter(&archive)
+	manifest := []File{}
+	seen := map[string]bool{}
+	total := 0
+	for _, entry := range entries {
+		if entry.IsDir() {
+			continue
+		}
+		parts := filename.FindStringSubmatch(entry.Name())
+		if parts == nil {
+			if strings.HasSuffix(entry.Name(), ".sql") {
+				return nil, nil, fmt.Errorf("migrate: invalid migration filename %s", entry.Name())
+			}
+			continue
+		}
+		if len(manifest) >= 1000 {
+			return nil, nil, errors.New("migrate: too many files")
+		}
+		version, err := strconv.ParseUint(parts[1], 10, 32)
+		if err != nil || version == 0 {
+			return nil, nil, errors.New("migrate: version must be 1..4294967295")
+		}
+		identity := fmt.Sprint(version) + parts[3]
+		if seen[identity] {
+			return nil, nil, errors.New("migrate: duplicate migration version/direction")
+		}
+		seen[identity] = true
+		file, err := source.Open(path.Join(directory, entry.Name()))
+		if err != nil {
+			return nil, nil, err
+		}
+		data, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
+		closeErr := file.Close()
+		if err == nil {
+			err = closeErr
+		}
+		if err != nil {
+			return nil, nil, err
+		}
+		total += len(data)
+		if len(data) == 0 || len(data) > 1<<20 || total > 16<<20 {
+			return nil, nil, errors.New("migrate: files must be 1..1MiB and <=16MiB total")
+		}
+		sum := sha256.Sum256(data)
+		manifest = append(manifest, File{Version: version, Name: parts[2], Direction: parts[3], SHA256: hex.EncodeToString(sum[:])})
+		fileWriter, err := writer.CreateHeader(&zip.FileHeader{Name: entry.Name(), Method: zip.Store})
+		if err != nil {
+			return nil, nil, err
+		}
+		if _, err = fileWriter.Write(data); err != nil {
+			return nil, nil, err
+		}
+	}
+	if len(manifest) == 0 {
+		return nil, nil, errors.New("migrate: no SQL migrations")
+	}
+	for _, file := range manifest {
+		if file.Direction == "down" && !seen[fmt.Sprint(file.Version)+"up"] {
+			return nil, nil, errors.New("migrate: down file lacks up migration")
+		}
+	}
+	sort.Slice(manifest, func(i, j int) bool {
+		if manifest[i].Version == manifest[j].Version {
+			return manifest[i].Direction < manifest[j].Direction
+		}
+		return manifest[i].Version < manifest[j].Version
+	})
+	if err = writer.Close(); err != nil {
+		return nil, nil, err
+	}
+	files, err := zip.NewReader(bytes.NewReader(archive.Bytes()), int64(archive.Len()))
+	return files, manifest, err
+}
+
+type Runner struct {
+	mu      sync.Mutex
+	engine  *migration.Migrate
+	db      *sql.DB
+	closed  bool
+	stopped bool
+}
+
+func NewMySQL(ctx context.Context, dsn string, source fs.FS, directory string) (runner *Runner, err error) {
+	if ctx == nil {
+		return nil, errors.New("migrate: context required")
+	}
+	snapshot, _, err := Snapshot(source, directory)
+	if err != nil {
+		return nil, err
+	}
+	cfg, err := mysqlDriver.ParseDSN(dsn)
+	if err != nil || cfg.DBName == "" {
+		return nil, errors.New("migrate: valid MySQL DSN with database required")
+	}
+	cfg.MultiStatements = true
+	if cfg.Timeout == 0 || cfg.Timeout > 5*time.Second {
+		cfg.Timeout = 5 * time.Second
+	}
+	if cfg.ReadTimeout == 0 || cfg.ReadTimeout > 30*time.Second {
+		cfg.ReadTimeout = 30 * time.Second
+	}
+	if cfg.WriteTimeout == 0 || cfg.WriteTimeout > 30*time.Second {
+		cfg.WriteTimeout = 30 * time.Second
+	}
+	connector, err := mysqlDriver.NewConnector(cfg)
+	if err != nil {
+		return nil, err
+	}
+	db := sql.OpenDB(connector)
+	db.SetMaxOpenConns(1)
+	db.SetMaxIdleConns(1)
+	defer func() {
+		if err != nil {
+			db.Close()
+		}
+	}()
+	conn, err := db.Conn(ctx)
+	if err != nil {
+		return nil, err
+	}
+	driver, err := migrateMySQL.WithConnection(ctx, conn, &migrateMySQL.Config{DatabaseName: cfg.DBName, StatementTimeout: 30 * time.Second})
+	if err != nil {
+		conn.Close()
+		return nil, err
+	}
+	src, err := iofs.New(snapshot, ".")
+	if err != nil {
+		driver.Close()
+		return nil, err
+	}
+	engine, err := migration.NewWithInstance("iofs", src, cfg.DBName, driver)
+	if err != nil {
+		src.Close()
+		driver.Close()
+		return nil, err
+	}
+	engine.LockTimeout = 10 * time.Second
+	return &Runner{engine: engine, db: db}, nil
+}
+
+// Up applies pending up scripts. MySQL DDL can partially commit; a failed
+// migration remains dirty and requires operator investigation. No automatic
+// down/force/dirty reset is provided. Cancellation stops after the current SQL
+// operation; statements/read I/O are bounded to 30 seconds.
+func (r *Runner) Up(ctx context.Context) error {
+	if ctx == nil {
+		return errors.New("migrate: context required")
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	if r.closed || r.stopped {
+		return errors.New("migrate: closed")
+	}
+	done := make(chan error, 1)
+	go func() { done <- r.engine.Up() }()
+	var err error
+	select {
+	case err = <-done:
+	case <-ctx.Done():
+		r.stopped = true // golang-migrate retains its graceful-stop state; create a fresh runner to retry.
+		select {
+		case r.engine.GracefulStop <- true:
+		default:
+		}
+		err = <-done
+		return errors.Join(ctx.Err(), err)
+	}
+	if errors.Is(err, migration.ErrNoChange) {
+		return nil
+	}
+	return err
+}
+func (r *Runner) Version() (uint, bool, error) {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	if r.closed {
+		return 0, false, errors.New("migrate: closed")
+	}
+	version, dirty, err := r.engine.Version()
+	if errors.Is(err, migration.ErrNilVersion) {
+		return 0, false, nil
+	}
+	return version, dirty, err
+}
+func (r *Runner) Close() error {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	if r.closed {
+		return nil
+	}
+	r.closed = true
+	a, b := r.engine.Close()
+	return errors.Join(a, b, r.db.Close())
+}

database/migrate/migrate_test.go

@@ -0,0 +1,99 @@
+package migrate
+
+import (
+	"context"
+	"database/sql"
+	"git.yuntuo.hk/k2safe/go-omni/telemetry"
+	"go.opentelemetry.io/otel/sdk/trace/tracetest"
+	"io/fs"
+	"os"
+	"testing"
+	"testing/fstest"
+	"time"
+)
+
+func TestSnapshotBoundsAndIdentity(t *testing.T) {
+	source := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("SELECT 1;")}}
+	f, manifest, err := Snapshot(source, ".")
+	if err != nil || len(manifest) != 1 {
+		t.Fatal(manifest, err)
+	}
+	source["0001_createItems.up.sql"].Data[0] = 'x'
+	raw, err := fs.ReadFile(f, "0001_createItems.up.sql")
+	if err != nil || string(raw) != "SELECT 1;" {
+		t.Fatal(string(raw), err)
+	}
+	for _, files := range []fstest.MapFS{{"0001_bad_name.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: []byte("x")}, "1_b.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.down.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: make([]byte, (1<<20)+1)}}} {
+		if _, _, err = Snapshot(files, "."); err == nil {
+			t.Fatal("invalid migration accepted")
+		}
+	}
+}
+func TestMySQLIntegration(t *testing.T) {
+	dsn := os.Getenv("GO_OMNI_MIGRATE_TEST_DSN")
+	if dsn == "" {
+		t.Skip("requires disposable MySQL database")
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
+	defer cancel()
+	files := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("CREATE TABLE testItems (id BIGINT NOT NULL PRIMARY KEY, value VARCHAR(64) NOT NULL);")}, "0002_seedItems.up.sql": &fstest.MapFile{Data: []byte("INSERT INTO testItems (id,value) VALUES (1,'original');")}}
+	runner, err := NewMySQL(ctx, dsn, files, ".")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if err = runner.Up(ctx); err != nil {
+		t.Fatal(err)
+	}
+	if err = runner.Up(ctx); err != nil {
+		t.Fatal("repeat", err)
+	}
+	v, dirty, err := runner.Version()
+	if err != nil || v != 2 || dirty {
+		t.Fatal(v, dirty, err)
+	}
+	runner.Close()
+	db, err := sql.Open("mysql", dsn)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer db.Close()
+	exporter := tracetest.NewInMemoryExporter()
+	tr, _ := telemetry.New(telemetry.Config{Service: "dbTest", SampleRatio: 1, Exporter: exporter})
+	defer tr.Close()
+	instrumented := telemetry.SQL{DB: db, Runtime: tr}
+	if _, err = instrumented.ExecContext(ctx, "UPDATE testItems SET value=? WHERE id=1", "secret parameter"); err != nil {
+		t.Fatal(err)
+	}
+	rows, err := instrumented.QueryContext(ctx, "SELECT value FROM testItems WHERE id=1")
+	if err != nil {
+		t.Fatal(err)
+	}
+	if !rows.Next() {
+		t.Fatal("missing row")
+	}
+	var value string
+	if err = rows.Scan(&value); err != nil || value != "secret parameter" {
+		t.Fatal(value, err)
+	}
+	rows.Close()
+	tr.Flush(ctx)
+	if len(exporter.GetSpans()) != 2 {
+		t.Fatal("SQL spans missing")
+	}
+	files["0003_failMigration.up.sql"] = &fstest.MapFile{Data: []byte("INSERT INTO missingTable VALUES (1);")}
+	runner, err = NewMySQL(ctx, dsn, files, ".")
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer runner.Close()
+	if err = runner.Up(ctx); err == nil {
+		t.Fatal("bad migration passed")
+	}
+	v, dirty, err = runner.Version()
+	if err != nil || v != 3 || !dirty {
+		t.Fatal(v, dirty, err)
+	}
+	if err = runner.Up(ctx); err == nil {
+		t.Fatal("dirty migration auto-cleared")
+	}
+}

examples/platform/README.md

@@ -0,0 +1,14 @@
+# 扩展模块组合示例
+
+业务通过 `storage.Store` 写文件，身份接口和 Policy 控制权限，OpenAPI 与路由一起注册，链路追踪关联 HTTP 与文件操作。`newApp` 不依赖 Local/S3/OSS 类型；启动处本例选择 Local。可替换成 S3/R2/OSS 构造结果，业务路由不变。
+
+```sh
+export GO_OMNI_DEMO_TOKEN='use-a-private-development-token'
+go run ./examples/platform --root /tmp/goOmniDemoFiles
+```
+
+接口：`POST /api/files`，JSON 为 `{"name":"hello.txt","content":"hello"}`，请求加 `Authorization: Bearer <GO_OMNI_DEMO_TOKEN>`；返回 201 和存储信息。`GET /openapi.json` 同样要求身份。
+
+也可传 `--config service.yaml`，配置规则见 [扩展模块](../../ADVANCED.md)。本例的静态身份只用于演示，实际应用注入 `auth.JWT` 或 `auth.Sessions`；trace provider 默认不外发，接入 collector 时注入 exporter/reader。
+
+`TestPlatformTLSAdapters` 通过三个 Runner 的真实 TLS 请求验证 401/403/201、文件内容、非法 key、OpenAPI 和关闭。文件根目录必须是应用私有目录。示例不执行自动数据迁移，也不启动 Orbit。

examples/platform/main.go

@@ -0,0 +1,132 @@
+// platform composes storage, authentication, permission policy, tracing and
+// OpenAPI using provider-neutral business dependencies.
+package main
+
+import (
+	"context"
+	"errors"
+	"flag"
+	"log"
+	"os"
+	"os/signal"
+	"strings"
+	"syscall"
+
+	omni "git.yuntuo.hk/k2safe/go-omni"
+	"git.yuntuo.hk/k2safe/go-omni/auth"
+	"git.yuntuo.hk/k2safe/go-omni/config"
+	"git.yuntuo.hk/k2safe/go-omni/openapi"
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	"git.yuntuo.hk/k2safe/go-omni/storage/local"
+	"git.yuntuo.hk/k2safe/go-omni/telemetry"
+)
+
+type dependencies struct {
+	Files   storage.Store
+	Auth    auth.Authenticator
+	Tracing *telemetry.Runtime
+}
+type upload struct {
+	Name    string `json:"name" binding:"required,max=128"`
+	Content string `json:"content" binding:"max=4096"`
+}
+
+// Dependencies are borrowed; the composition root owns their shutdown.
+func newApp(cfg omni.Config, d dependencies) (*omni.App, error) {
+	if d.Files == nil || d.Auth == nil || d.Tracing == nil {
+		return nil, errors.New("platform: dependencies required")
+	}
+	app := omni.New(omni.WithConfig(cfg))
+	app.Use(d.Tracing.Middleware())
+	api, err := openapi.New("File service", "1")
+	if err != nil {
+		return nil, err
+	}
+	input, err := openapi.SchemaOf[upload]()
+	if err != nil {
+		return nil, err
+	}
+	output, err := openapi.SchemaOf[storage.Info]()
+	if err != nil {
+		return nil, err
+	}
+	policy := auth.PolicyFunc(func(_ context.Context, p auth.Principal, action, resource string) error {
+		if p.TenantID != "" && p.HasRole("writer") && action == "create" && resource == "files" {
+			return nil
+		}
+		return errors.New("denied")
+	})
+	err = api.Handle(app, openapi.Operation{Method: "POST", Path: "/api/files", OperationID: "createFile", Summary: "Store a small text file", Request: input, BearerAuth: true, Responses: map[int]openapi.Response{201: {Description: "Created", Schema: openapi.Envelope(output)}}}, func(c *omni.Context) error {
+		var in upload
+		if err := c.BindJSON(&in); err != nil {
+			return err
+		}
+		p, _ := auth.FromContext(c)
+		if strings.ContainsAny(in.Name, "/\\") {
+			return omni.BadRequest("INVALID_NAME", "name must be one path segment")
+		}
+		key := p.TenantID + "/" + in.Name
+		if err := storage.Check(c.Context(), key); err != nil {
+			return omni.BadRequest("INVALID_KEY", "invalid storage key")
+		}
+		var info storage.Info
+		err := d.Tracing.Do(c.Context(), "files.put", func(ctx context.Context) error {
+			var err error
+			info, err = d.Files.Put(ctx, key, strings.NewReader(in.Content), int64(len(in.Content)), "text/plain; charset=utf-8")
+			return err
+		})
+		if err != nil {
+			return err
+		}
+		return c.Created(info)
+	}, auth.RequireAuth(d.Auth), auth.RequirePermission(policy, "create", func(*omni.Context) string { return "files" }))
+	if err != nil {
+		app.Close()
+		return nil, err
+	}
+	app.GET("/openapi.json", api.Handler(), auth.RequireAuth(d.Auth))
+	return app, nil
+}
+func run(ctx context.Context, configFile, root string) error {
+	var cfg config.Config
+	var err error
+	if configFile != "" {
+		cfg, err = config.LoadFile(configFile, "")
+	} else {
+		cfg, err = config.LoadFromEnv("")
+		if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
+			cfg.Address = "127.0.0.1:8080"
+		}
+	}
+	if err != nil {
+		return err
+	}
+	token := os.Getenv("GO_OMNI_DEMO_TOKEN")
+	if token == "" {
+		return errors.New("set GO_OMNI_DEMO_TOKEN for the demonstration credential")
+	}
+	files, err := local.New(root, 64<<10)
+	if err != nil {
+		return err
+	}
+	tracing, err := telemetry.New(telemetry.Config{Service: "fileService", SampleRatio: 1})
+	if err != nil {
+		return err
+	}
+	defer tracing.Close()
+	app, err := newApp(cfg, dependencies{Files: files, Auth: auth.StaticAuthenticator{token: {UserID: "demoUser", TenantID: "demoTenant", Roles: []string{"writer"}}}, Tracing: tracing})
+	if err != nil {
+		return err
+	}
+	return app.RunContext(ctx)
+}
+func main() {
+	file := flag.String("config", "", "optional JSON/YAML config")
+	root := flag.String("root", "./fileData", "private local object directory")
+	flag.Parse()
+	ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
+	defer stop()
+	if err := run(ctx, *file, *root); err != nil {
+		log.Fatal(err)
+	}
+}

examples/platform/main_test.go

@@ -0,0 +1,102 @@
+package main
+
+import (
+	"context"
+	omni "git.yuntuo.hk/k2safe/go-omni"
+	"git.yuntuo.hk/k2safe/go-omni/auth"
+	"git.yuntuo.hk/k2safe/go-omni/config"
+	"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
+	"git.yuntuo.hk/k2safe/go-omni/storage/local"
+	"git.yuntuo.hk/k2safe/go-omni/telemetry"
+	"io"
+	"net"
+	"net/http"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestPlatformTLSAdapters(t *testing.T) {
+	for _, name := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
+		t.Run(name, func(t *testing.T) {
+			cfg := config.Default()
+			cfg.Transport = name
+			cert, key, tlsClient := testutil.CertificateFiles(t)
+			cfg.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
+			store, _ := local.New(t.TempDir(), 64<<10)
+			tr, _ := telemetry.New(telemetry.Config{Service: "platformTest", SampleRatio: 1})
+			defer tr.Close()
+			app, err := newApp(cfg, dependencies{Files: store, Auth: auth.StaticAuthenticator{"writer": {UserID: "u", TenantID: "t", Roles: []string{"writer"}}, "reader": {UserID: "r", TenantID: "t"}}, Tracing: tr})
+			if err != nil {
+				t.Fatal(err)
+			}
+			listener, err := net.Listen("tcp", "127.0.0.1:0")
+			if err != nil {
+				t.Fatal(err)
+			}
+			ctx, cancel := context.WithCancel(context.Background())
+			done := make(chan error, 1)
+			go func() { done <- app.ServeContext(ctx, listener) }()
+			defer func() {
+				cancel()
+				select {
+				case err := <-done:
+					if err != nil {
+						t.Error(err)
+					}
+				case <-time.After(5 * time.Second):
+					listener.Close()
+					t.Error("shutdown timeout")
+				}
+			}()
+			client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
+			defer client.CloseIdleConnections()
+			request := func(method, path, body, token string) (int, string) {
+				t.Helper()
+				r, _ := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
+				r.Header.Set("Authorization", "Bearer "+token)
+				r.Header.Set("Content-Type", "application/json")
+				response, err := client.Do(r)
+				if err != nil {
+					t.Fatal(err)
+				}
+				raw, err := io.ReadAll(response.Body)
+				response.Body.Close()
+				if err != nil {
+					t.Fatal(err)
+				}
+				if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
+					t.Fatal("TLS unverified")
+				}
+				if name == omni.TransportHTTP2 && response.ProtoMajor != 2 {
+					t.Fatal("protocol")
+				}
+				return response.StatusCode, string(raw)
+			}
+			for _, test := range []struct {
+				token  string
+				status int
+			}{{"", 401}, {"reader", 403}, {"writer", 201}} {
+				status, body := request("POST", "/api/files", `{"name":"hello.txt","content":"hello"}`, test.token)
+				if status != test.status {
+					t.Fatal(status, body)
+				}
+			}
+			object, err := store.Get(context.Background(), "t/hello.txt")
+			if err != nil {
+				t.Fatal(err)
+			}
+			raw, _ := io.ReadAll(object.Body)
+			object.Body.Close()
+			if string(raw) != "hello" {
+				t.Fatal(string(raw))
+			}
+			if status, _ := request("POST", "/api/files", `{"name":"../escape","content":"x"}`, "writer"); status != 400 {
+				t.Fatal(status)
+			}
+			if status, body := request("GET", "/openapi.json", "", "writer"); status != 200 || !strings.Contains(body, `"/api/files"`) {
+				t.Fatal(status, body)
+			}
+		})
+	}
+}

go.mod

@@ -1,25 +1,48 @@
 module git.yuntuo.hk/k2safe/go-omni
 
-go 1.22
+go 1.22.0
 
 require (
+	github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0
 	github.com/go-playground/validator/v10 v10.20.0
 	github.com/go-sql-driver/mysql v1.9.3
+	github.com/golang-jwt/jwt/v5 v5.3.1
+	github.com/golang-migrate/migrate/v4 v4.18.1
+	github.com/minio/minio-go/v7 v7.0.83
 	github.com/redis/go-redis/v9 v9.17.3
-	golang.org/x/mod v0.17.0
+	go.opentelemetry.io/otel v1.31.0
-	golang.org/x/net v0.25.0
+	go.opentelemetry.io/otel/metric v1.31.0
+	go.opentelemetry.io/otel/sdk v1.31.0
+	go.opentelemetry.io/otel/sdk/metric v1.31.0
+	go.opentelemetry.io/otel/trace v1.31.0
+	golang.org/x/mod v0.21.0
+	golang.org/x/net v0.33.0
+	gopkg.in/yaml.v3 v3.0.1
 )
 
 require (
 	filippo.io/edwards25519 v1.1.0 // indirect
 	github.com/cespare/xxhash/v2 v2.3.0 // indirect
 	github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
+	github.com/dustin/go-humanize v1.0.1 // indirect
 	github.com/gabriel-vasile/mimetype v1.4.3 // indirect
+	github.com/go-ini/ini v1.67.0 // indirect
+	github.com/go-logr/logr v1.4.2 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-playground/locales v0.14.1 // indirect
 	github.com/go-playground/universal-translator v0.18.1 // indirect
+	github.com/goccy/go-json v0.10.4 // indirect
+	github.com/google/uuid v1.6.0 // indirect
+	github.com/hashicorp/errwrap v1.1.0 // indirect
+	github.com/hashicorp/go-multierror v1.1.1 // indirect
+	github.com/klauspost/compress v1.17.11 // indirect
+	github.com/klauspost/cpuid/v2 v2.2.9 // indirect
 	github.com/leodido/go-urn v1.4.0 // indirect
-	github.com/stretchr/testify v1.9.0 // indirect
+	github.com/minio/md5-simd v1.1.2 // indirect
-	golang.org/x/crypto v0.23.0 // indirect
+	github.com/rs/xid v1.6.0 // indirect
-	golang.org/x/sys v0.20.0 // indirect
+	go.uber.org/atomic v1.7.0 // indirect
-	golang.org/x/text v0.15.0 // indirect
+	golang.org/x/crypto v0.31.0 // indirect
+	golang.org/x/sys v0.28.0 // indirect
+	golang.org/x/text v0.21.0 // indirect
+	golang.org/x/time v0.5.0 // indirect
 )

go.sum

@@ -1,17 +1,45 @@
 filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
 filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
+github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161 h1:L/gRVlceqvL25UVaW/CKtUDjefjrs0SPonmDGUVOYP0=
+github.com/Azure/go-ansiterm v0.0.0-20230124172434-306776ec8161/go.mod h1:xomTg63KZ2rFqZQzSB4Vz2SUXa1BpHTVz9L5PTmPC4E=
+github.com/Microsoft/go-winio v0.6.2 h1:F2VQgta7ecxGYO8k3ZZz3RS8fVIXVxONVUPlNERoyfY=
+github.com/Microsoft/go-winio v0.6.2/go.mod h1:yd8OoFMLzJbo9gZq8j5qaps8bJ9aShtEA8Ipt1oGCvU=
+github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0 h1:uWzn3io54f9L9mvwsQQSv1KpkkFA06hBxI++RvIyvpI=
+github.com/aliyun/alibabacloud-oss-go-sdk-v2 v1.6.0/go.mod h1:FTzydeQVmR24FI0D6XWUOMKckjXehM/jgMn1xC+DA9M=
 github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
 github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
 github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
 github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
 github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
 github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
 github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f h1:lO4WD4F/rVNCu3HqELle0jiPLLBs70cWOduZpkS1E78=
 github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cuUVRXasLTGF7a8hSLbxyZXjz+1KgoB3wDUb6vlszIc=
+github.com/dhui/dktest v0.4.3 h1:wquqUxAFdcUgabAVLvSCOKOlag5cIZuaOjYIBOWdsR0=
+github.com/dhui/dktest v0.4.3/go.mod h1:zNK8IwktWzQRm6I/l2Wjp7MakiyaFWv4G1hjmodmMTs=
+github.com/distribution/reference v0.6.0 h1:0IXCQ5g4/QMHHkarYzh5l+u8T3t73zM5QvfrDyIgxBk=
+github.com/distribution/reference v0.6.0/go.mod h1:BbU0aIcezP1/5jX/8MP0YiH4SdvB5Y4f/wlDRiLyi3E=
+github.com/docker/docker v27.2.0+incompatible h1:Rk9nIVdfH3+Vz4cyI/uhbINhEZ/oLmc+CBXmH6fbNk4=
+github.com/docker/docker v27.2.0+incompatible/go.mod h1:eEKB0N0r5NX/I1kEveEz05bcu8tLC/8azJZsviup8Sk=
+github.com/docker/go-connections v0.5.0 h1:USnMq7hx7gwdVZq1L49hLXaFtUdTADjXGp+uj1Br63c=
+github.com/docker/go-connections v0.5.0/go.mod h1:ov60Kzw0kKElRwhNs9UlUHAE/F9Fe6GLaXnqyDdmEXc=
+github.com/docker/go-units v0.5.0 h1:69rxXcBk27SvSaaxTtLh/8llcHD8vYHT7WSdRZ/jvr4=
+github.com/docker/go-units v0.5.0/go.mod h1:fgPhTUdO+D/Jk86RDLlptpiXQzgHJF7gydDDbaIK4Dk=
+github.com/dustin/go-humanize v1.0.1 h1:GzkhY7T5VNhEkwH0PVJgjz+fX1rhBrR7pRT3mDkpeCY=
+github.com/dustin/go-humanize v1.0.1/go.mod h1:Mu1zIs6XwVuF/gI1OepvI0qD18qycQx+mFykh5fBlto=
+github.com/felixge/httpsnoop v1.0.4 h1:NFTV2Zj1bL4mc9sqWACXbQFVBBg2W3GPvqp8/ESS2Wg=
+github.com/felixge/httpsnoop v1.0.4/go.mod h1:m8KPJKqk1gH5J9DgRY2ASl2lWCfGKXixSwevea8zH2U=
 github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
 github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
+github.com/go-ini/ini v1.67.0 h1:z6ZrTEZqSWOTyH2FlglNbNgARyHG8oLW9gMELqKr06A=
+github.com/go-ini/ini v1.67.0/go.mod h1:ByCAeIL28uOIIG0E3PJtZPDL8WnHpFKFOtgjp+3Ies8=
+github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
+github.com/go-logr/logr v1.4.2 h1:6pFjapn8bFcIbiKo3XT4j/BhANplGihG6tvd+8rYgrY=
+github.com/go-logr/logr v1.4.2/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
+github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
+github.com/go-logr/stdr v1.2.2/go.mod h1:mMo/vtBO5dYbehREoey6XUKy/eSumjCCveDpRre4VKE=
 github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
 github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
 github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
@@ -22,23 +50,85 @@ github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBEx
 github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
 github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
 github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
+github.com/goccy/go-json v0.10.4 h1:JSwxQzIqKfmFX1swYPpUThQZp/Ka4wzJdK0LWVytLPM=
+github.com/goccy/go-json v0.10.4/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
+github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
+github.com/gogo/protobuf v1.3.2/go.mod h1:P1XiOD3dCwIKUDQYPy72D8LYyHL2YPYrpS2s69NZV8Q=
+github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
+github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
+github.com/golang-migrate/migrate/v4 v4.18.1 h1:JML/k+t4tpHCpQTCAD62Nu43NUFzHY4CV3uAuvHGC+Y=
+github.com/golang-migrate/migrate/v4 v4.18.1/go.mod h1:HAX6m3sQgcdO81tdjn5exv20+3Kb13cmGli1hrD6hks=
+github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
+github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
+github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
+github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
+github.com/hashicorp/errwrap v1.0.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
+github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY2I=
+github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
+github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
+github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
+github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
+github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
+github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
+github.com/klauspost/cpuid/v2 v2.2.9 h1:66ze0taIn2H33fBvCkXuv9BmCwDfafmiIVpKV9kKGuY=
+github.com/klauspost/cpuid/v2 v2.2.9/go.mod h1:rqkxqrZ1EhYM9G+hXH7YdowN5R5RGN6NK4QwQ3WMXF8=
 github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
 github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
+github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
+github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
+github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
+github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
+github.com/minio/minio-go/v7 v7.0.83 h1:W4Kokksvlz3OKf3OqIlzDNKd4MERlC2oN8YptwJ0+GA=
+github.com/minio/minio-go/v7 v7.0.83/go.mod h1:57YXpvc5l3rjPdhqNrDsvVlY0qPI6UTk1bflAe+9doY=
+github.com/moby/docker-image-spec v1.3.1 h1:jMKff3w6PgbfSa69GfNg+zN/XLhfXJGnEx3Nl2EsFP0=
+github.com/moby/docker-image-spec v1.3.1/go.mod h1:eKmb5VW8vQEh/BAr2yvVNvuiJuY6UIocYsFu/DxxRpo=
+github.com/moby/term v0.5.0 h1:xt8Q1nalod/v7BqbG21f8mQPqH+xAaC9C3N3wfWbVP0=
+github.com/moby/term v0.5.0/go.mod h1:8FzsFHVUBGZdbDsJw/ot+X+d5HLUbvklYLJ9uGfcI3Y=
+github.com/morikuni/aec v1.0.0 h1:nP9CBfwrvYnBRgY6qfDQkygYDmYwOilePFkwzv4dU8A=
+github.com/morikuni/aec v1.0.0/go.mod h1:BbKIizmSmc5MMPqRYbxO4ZU0S0+P200+tUnFx7PXmsc=
+github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
+github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
+github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
+github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
+github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
+github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/redis/go-redis/v9 v9.17.3 h1:fN29NdNrE17KttK5Ndf20buqfDZwGNgoUr9qjl1DQx4=
 github.com/redis/go-redis/v9 v9.17.3/go.mod h1:u410H11HMLoB+TP67dz8rL9s6QW2j76l0//kSOd3370=
+github.com/rs/xid v1.6.0 h1:fV591PaemRlL6JfRxGDEPl69wICngIQ3shQtzfy2gxU=
+github.com/rs/xid v1.6.0/go.mod h1:7XoLgs4eV+QndskICGsho+ADou8ySMSjJKDIan90Nz0=
+github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
+github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
 github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
 github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
-golang.org/x/crypto v0.23.0 h1:dIJU/v2J8Mdglj/8rJ6UUOM3Zc9zLZxVZwwxMooUSAI=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0 h1:TT4fX+nBOA/+LUkobKGW1ydGcn+G3vRw9+g5HwCphpk=
-golang.org/x/crypto v0.23.0/go.mod h1:CKFgDieR+mRhux2Lsu27y0fO304Db0wZe70UKqHu0v8=
+go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.54.0/go.mod h1:L7UH0GbB0p47T4Rri3uHjbpCFYrVrwc1I25QhNPiGK8=
-golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
+go.opentelemetry.io/otel v1.31.0 h1:NsJcKPIW0D0H3NgzPDHmo0WW6SptzPdqg/L1zsIm2hY=
-golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
+go.opentelemetry.io/otel v1.31.0/go.mod h1:O0C14Yl9FgkjqcCZAsE053C13OaddMYr/hz6clDkEJE=
-golang.org/x/net v0.25.0 h1:d/OCCoBEUq33pjydKrGQhw7IlUPI2Oylr+8qLx49kac=
+go.opentelemetry.io/otel/metric v1.31.0 h1:FSErL0ATQAmYHUIzSezZibnyVlft1ybhy4ozRPcF2fE=
-golang.org/x/net v0.25.0/go.mod h1:JkAGAh7GEvH74S6FOH42FLoXpXbE/aqXSrIQjXgsiwM=
+go.opentelemetry.io/otel/metric v1.31.0/go.mod h1:C3dEloVbLuYoX41KpmAhOqNriGbA+qqH6PQ5E5mUfnY=
-golang.org/x/sys v0.20.0 h1:Od9JTbYCk261bKm4M/mw7AklTlFYIa0bIp9BgSm1S8Y=
+go.opentelemetry.io/otel/sdk v1.31.0 h1:xLY3abVHYZ5HSfOg3l2E5LUj2Cwva5Y7yGxnSW9H5Gk=
-golang.org/x/sys v0.20.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+go.opentelemetry.io/otel/sdk v1.31.0/go.mod h1:TfRbMdhvxIIr/B2N2LQW2S5v9m3gOQ/08KsbbO5BPT0=
-golang.org/x/text v0.15.0 h1:h1V/4gjBv8v9cjcR6+AR5+/cIYK5N/WAgiv4xlsEtAk=
+go.opentelemetry.io/otel/sdk/metric v1.31.0 h1:i9hxxLJF/9kkvfHppyLL55aW7iIJz4JjxTeYusH7zMc=
-golang.org/x/text v0.15.0/go.mod h1:18ZOQIKpY8NJVqYksKHtTdi31H5itFRjB5/qKTNYzSU=
+go.opentelemetry.io/otel/sdk/metric v1.31.0/go.mod h1:CRInTMVvNhUKgSAMbKyTMxqOBC0zgyxzW55lZzX43Y8=
+go.opentelemetry.io/otel/trace v1.31.0 h1:ffjsj1aRouKewfr85U2aGagJ46+MvodynlQ1HYdmJys=
+go.opentelemetry.io/otel/trace v1.31.0/go.mod h1:TXZkRk7SM2ZQLtR6eoAWQFIHPvzQ06FJAsO1tJg480A=
+go.uber.org/atomic v1.7.0 h1:ADUqmZGgLDDfbSL9ZmPxKTybcoEYHgpYfELNoN+7hsw=
+go.uber.org/atomic v1.7.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
+golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
+golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
+golang.org/x/mod v0.21.0 h1:vvrHzRwRfVKSiLrG+d4FMl/Qi4ukBCE6kZlTUkDYRT0=
+golang.org/x/mod v0.21.0/go.mod h1:6SkKJ3Xj0I0BrPOZoBy3bdMptDDU9oJrpohJ3eWZ1fY=
+golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
+golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
+golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
+golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
+golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
+golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
+golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
+golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

internal/storagefixture/server.go

@@ -0,0 +1,63 @@
+// Package storagefixture supplies a bounded protocol fixture, not a cloud service.
+package storagefixture
+
+import (
+	"crypto/md5"
+	"fmt"
+	"io"
+	"net/http"
+	"sync"
+	"time"
+)
+
+type value struct {
+	data        []byte
+	contentType string
+	etag        string
+}
+
+func Handler() http.Handler {
+	var mu sync.Mutex
+	values := map[string]value{}
+	return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		mu.Lock()
+		defer mu.Unlock()
+		if r.Header.Get("Authorization") == "" && r.URL.Query().Get("X-Amz-Signature") == "" && r.URL.Query().Get("x-oss-signature") == "" {
+			http.Error(w, "signature required", 403)
+			return
+		}
+		key := r.URL.Path
+		switch r.Method {
+		case "PUT":
+			data, err := io.ReadAll(io.LimitReader(r.Body, 1025))
+			if err != nil || len(data) > 1024 {
+				http.Error(w, "too large", 413)
+				return
+			}
+			v := value{data: data, contentType: r.Header.Get("Content-Type"), etag: fmt.Sprintf("%x", md5.Sum(data))}
+			values[key] = v
+			w.Header().Set("ETag", `"`+v.etag+`"`)
+			w.WriteHeader(200)
+		case "DELETE":
+			delete(values, key)
+			w.WriteHeader(204)
+		case "GET", "HEAD":
+			v, ok := values[key]
+			if !ok {
+				w.Header().Set("Content-Type", "application/xml")
+				w.WriteHeader(404)
+				fmt.Fprint(w, `<Error><Code>NoSuchKey</Code><Message>not found</Message></Error>`)
+				return
+			}
+			w.Header().Set("Content-Length", fmt.Sprint(len(v.data)))
+			w.Header().Set("Content-Type", v.contentType)
+			w.Header().Set("ETag", `"`+v.etag+`"`)
+			w.Header().Set("Last-Modified", time.Now().UTC().Format(http.TimeFormat))
+			if r.Method == "GET" {
+				w.Write(v.data)
+			}
+		default:
+			http.Error(w, "unsupported", 405)
+		}
+	})
+}

logx/logx.go

@@ -21,14 +21,14 @@ import (
 // standard stream. File takes precedence over Output and appends to the named
 // file.
 type Config struct {
-	Level       string
+	Level       string    `json:"level"`
-	Format      string
+	Format      string    `json:"format"`
-	Output      string
+	Output      string    `json:"output"`
-	File        string
+	File        string    `json:"file"`
-	Service     string
+	Service     string    `json:"service"`
-	Environment string
+	Environment string    `json:"environment"`
-	AddSource   bool
+	AddSource   bool      `json:"addSource"`
-	Writer      io.Writer
+	Writer      io.Writer `json:"-"`
 }
 
 // Default returns the default production-friendly logger configuration.
@@ -45,6 +45,7 @@ func Default() Config {
 type Logger struct {
 	logger *slog.Logger
 	closer *closeState
+	level  *slog.LevelVar
 }
 
 type closeState struct {
@@ -78,7 +79,9 @@ func New(cfg Config) (*Logger, error) {
 		return nil, err
 	}
 
-	opts := &slog.HandlerOptions{Level: level, AddSource: cfg.AddSource}
+	dynamicLevel := new(slog.LevelVar)
+	dynamicLevel.Set(level)
+	opts := &slog.HandlerOptions{Level: dynamicLevel, AddSource: cfg.AddSource}
 	var handler slog.Handler
 	if cfg.Format == "text" {
 		handler = slog.NewTextHandler(writer, opts)
@@ -97,7 +100,7 @@ func New(cfg Config) (*Logger, error) {
 	if closeWriter != nil {
 		closer = &closeState{fn: closeWriter}
 	}
-	return &Logger{logger: logger, closer: closer}, nil
+	return &Logger{logger: logger, closer: closer, level: dynamicLevel}, nil
 }
 
 // Setup replaces the process-wide logger used by the package-level helpers.
@@ -152,14 +155,27 @@ func (l *Logger) With(args ...any) *Logger {
 	if l == nil || l.logger == nil {
 		return defaultLogger.Load().With(args...)
 	}
-	return &Logger{logger: l.logger.With(args...), closer: l.closer}
+	return &Logger{logger: l.logger.With(args...), closer: l.closer, level: l.level}
 }
 
 func (l *Logger) WithGroup(name string) *Logger {
 	if l == nil || l.logger == nil {
 		return defaultLogger.Load().WithGroup(name)
 	}
-	return &Logger{logger: l.logger.WithGroup(name), closer: l.closer}
+	return &Logger{logger: l.logger.WithGroup(name), closer: l.closer, level: l.level}
+}
+
+// SetLevel atomically changes the threshold shared by this logger and children.
+func (l *Logger) SetLevel(value string) error {
+	if l == nil || l.level == nil {
+		return errors.New("logx: logger does not support dynamic levels")
+	}
+	level, err := parseLevel(value)
+	if err != nil {
+		return err
+	}
+	l.level.Set(level)
+	return nil
 }
 
 // Close releases resources owned by the root logger. It is safe to call more

middleware/idempotency.go

@@ -17,6 +17,7 @@ import (
 )
 
 type IdempotencyConfig struct {
+	Store            IdempotencyStore `json:"-"` // Optional shared store; nil keeps local behavior.
 	TTL              time.Duration    `json:"ttl"`
 	MaxEntries       int              `json:"maxEntries"`
 	MaxBytes         int              `json:"maxBytes"`
@@ -53,6 +54,9 @@ func NewIdempotency(cfg IdempotencyConfig) (*Idempotency, error) {
 	if cfg.TTL == 0 {
 		cfg.TTL = 10 * time.Minute
 	}
+	if cfg.Store != nil && cfg.TTL > 30*24*time.Hour {
+		return nil, errors.New("middleware: shared idempotency TTL exceeds 30d")
+	}
 	if cfg.MaxEntries == 0 {
 		cfg.MaxEntries = 256
 	}
@@ -125,6 +129,9 @@ func (i *Idempotency) Middleware() httpx.Middleware {
 			copy(fingerprint[:], hash.Sum(nil))
 			namespace, _ := json.Marshal([]string{scope, key})
 			id := sha256.Sum256(namespace)
+			if i.config.Store != nil {
+				return i.shared(c, next, id, fingerprint)
+			}
 			i.mu.Lock()
 			now := i.now()
 			for k, v := range i.items {

middleware/rateLimit.go

@@ -13,6 +13,7 @@ import (
 )
 
 type RateLimitConfig struct {
+	Backend RateLimitBackend            `json:"-"`
 	Rate    float64                     `json:"rate"` // Tokens per second, per key.
 	Burst   int                         `json:"burst"`
 	MaxKeys int                         `json:"maxKeys"`
@@ -64,7 +65,17 @@ func (l *RateLimiter) Middleware() httpx.Middleware {
 			if key == "" || len(key) > 1024 {
 				return apierrors.BadRequest("INVALID_RATE_KEY", "invalid rate limit key")
 			}
-			delay, full := l.allow(key)
+			var delay time.Duration
+			var full bool
+			if l.config.Backend != nil {
+				var err error
+				delay, err = l.config.Backend.Take(c.Context(), key)
+				if err != nil {
+					return apierrors.New(503, "RATE_LIMIT_UNAVAILABLE", "rate limiter unavailable").WithCause(err)
+				}
+			} else {
+				delay, full = l.allow(key)
+			}
 			if full {
 				return apierrors.New(503, "RATE_LIMIT_CAPACITY", "rate limiter capacity reached")
 			}

middleware/redis/store.go

@@ -0,0 +1,178 @@
+// Package redis supplies atomic shared middleware state in one Redis hash slot.
+// Use a dedicated namespace and a noeviction Redis deployment for correctness.
+package redis
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"git.yuntuo.hk/k2safe/go-omni/middleware"
+	goredis "github.com/redis/go-redis/v9"
+	"strings"
+	"time"
+)
+
+type IdempotencyConfig struct {
+	MaxEntries     int `json:"maxEntries"`
+	MaxRecordBytes int `json:"maxRecordBytes"`
+}
+type IdempotencyStore struct {
+	client    goredis.Scripter
+	keys      []string
+	cfg       IdempotencyConfig
+	signature string
+}
+
+func namespace(prefix, kind string) (string, error) {
+	if prefix == "" || len(prefix) > 256 || strings.ContainsAny(prefix, "{}\r\n") {
+		return "", errors.New("middleware/redis: nonempty bounded namespace without braces required")
+	}
+	sum := sha256.Sum256([]byte(prefix))
+	return "goOmni:{" + hex.EncodeToString(sum[:]) + "}:" + kind, nil
+}
+func NewIdempotencyStore(client goredis.Scripter, prefix string, cfg IdempotencyConfig) (*IdempotencyStore, error) {
+	if client == nil || cfg.MaxEntries < 0 || cfg.MaxRecordBytes < 0 {
+		return nil, errors.New("middleware/redis: invalid configuration")
+	}
+	if cfg.MaxEntries == 0 {
+		cfg.MaxEntries = 128
+	}
+	if cfg.MaxRecordBytes == 0 {
+		cfg.MaxRecordBytes = 128 << 10
+	}
+	if cfg.MaxEntries > 100000 || cfg.MaxRecordBytes < 512 || cfg.MaxRecordBytes > 1<<20 {
+		return nil, errors.New("middleware/redis: capacity outside supported bounds")
+	}
+	base, err := namespace(prefix, "idempotency")
+	if err != nil {
+		return nil, err
+	}
+	return &IdempotencyStore{client: client, keys: []string{base + ":records", base + ":expiry", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%d:%d", cfg.MaxEntries, cfg.MaxRecordBytes)}, nil
+}
+
+var claim = goredis.NewScript(`
+local config=redis.call('GET',KEYS[3])
+if config and config~=ARGV[5] then return redis.error_reply('configuration mismatch') end
+if not config then redis.call('SET',KEYS[3],ARGV[5]) end
+local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
+local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now,'LIMIT',0,128)
+for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
+local expiry=redis.call('ZSCORE',KEYS[2],ARGV[1]);if expiry and tonumber(expiry)<=now then redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]) end
+local value=redis.call('HGET',KEYS[1],ARGV[1]);if value then return {0,value} end
+if redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return redis.error_reply('idempotency capacity reached') end
+value=cjson.encode({fingerprint=ARGV[2],owner=ARGV[3],state='running'})
+redis.call('HSET',KEYS[1],ARGV[1],value);return {1,value}
+`)
+
+func (s *IdempotencyStore) Claim(ctx context.Context, key, fp, owner string) (middleware.IdempotencyRecord, bool, error) {
+	if len(key) != 64 || len(fp) != 64 || len(owner) != 32 {
+		return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim")
+	}
+	result, err := claim.Run(ctx, s.client, s.keys, key, fp, owner, s.cfg.MaxEntries, s.signature).Slice()
+	if err != nil {
+		return middleware.IdempotencyRecord{}, false, err
+	}
+	if len(result) != 2 {
+		return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid claim result")
+	}
+	value, ok := result[1].(string)
+	if !ok || len(value) > s.cfg.MaxRecordBytes {
+		return middleware.IdempotencyRecord{}, false, errors.New("middleware/redis: invalid record")
+	}
+	var record middleware.IdempotencyRecord
+	err = json.Unmarshal([]byte(value), &record)
+	return record, result[0] == int64(1), err
+}
+
+var complete = goredis.NewScript(`
+local current=redis.call('HGET',KEYS[1],ARGV[1]);if not current then return redis.error_reply('claim missing') end
+local value=cjson.decode(current)
+if value.owner~=ARGV[2] or value.state~='running' then return redis.error_reply('claim owner/state changed') end
+local response=ARGV[3]
+if response~='' then value.state='completed';value.response=cjson.decode(response) else value.state='failed' end
+local encoded=cjson.encode(value)
+if string.len(encoded)>tonumber(ARGV[5]) then value.state='failed';value.response=nil;encoded=cjson.encode(value) end
+local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
+redis.call('HSET',KEYS[1],ARGV[1],encoded);redis.call('ZADD',KEYS[2],now+tonumber(ARGV[4]),ARGV[1]);return 1
+`)
+
+func (s *IdempotencyStore) Complete(ctx context.Context, key, owner string, response *middleware.StoredResponse, ttl time.Duration) error {
+	if ttl <= 0 || ttl > 30*24*time.Hour {
+		return errors.New("middleware/redis: TTL must be positive and <=30d")
+	}
+	var encoded []byte
+	var err error
+	if response != nil {
+		encoded, err = json.Marshal(response)
+		if err != nil {
+			return err
+		}
+		if len(encoded) > s.cfg.MaxRecordBytes {
+			encoded = nil
+		}
+	}
+	return complete.Run(ctx, s.client, s.keys, key, owner, string(encoded), max(1, ttl.Milliseconds()), s.cfg.MaxRecordBytes).Err()
+}
+
+// Resolve removes only a matching non-replayable claim after the operator has
+// reconciled the business outcome. Never invoke automatically on a timeout.
+func (s *IdempotencyStore) Resolve(ctx context.Context, key, owner string) error {
+	return goredis.NewScript(`local v=redis.call('HGET',KEYS[1],ARGV[1]);if not v then return 0 end;local r=cjson.decode(v);if r.owner~=ARGV[2] or r.state=='completed' then return redis.error_reply('claim cannot be resolved') end;redis.call('HDEL',KEYS[1],ARGV[1]);redis.call('ZREM',KEYS[2],ARGV[1]);return 1`).Run(ctx, s.client, s.keys, key, owner).Err()
+}
+
+type limiter struct {
+	client    goredis.Scripter
+	keys      []string
+	cfg       middleware.RateLimitConfig
+	signature string
+}
+
+func NewRateLimiter(client goredis.Scripter, prefix string, cfg middleware.RateLimitConfig) (*middleware.RateLimiter, error) {
+	if client == nil {
+		return nil, errors.New("middleware/redis: client required")
+	}
+	if cfg.MaxKeys == 0 {
+		cfg.MaxKeys = 10000
+	}
+	if cfg.IdleTTL == 0 {
+		cfg.IdleTTL = 24 * time.Hour
+	}
+	if _, err := middleware.NewRateLimiter(cfg); err != nil {
+		return nil, err
+	}
+	base, err := namespace(prefix, "rate")
+	if err != nil {
+		return nil, err
+	}
+	backend := &limiter{client: client, keys: []string{base + ":buckets", base + ":seen", base + ":config"}, cfg: cfg, signature: fmt.Sprintf("%g:%d:%d:%d", cfg.Rate, cfg.Burst, cfg.MaxKeys, cfg.IdleTTL.Milliseconds())}
+	cfg.Backend = backend
+	return middleware.NewRateLimiter(cfg)
+}
+
+var take = goredis.NewScript(`
+local config=redis.call('GET',KEYS[3]);if config and config~=ARGV[6] then return redis.error_reply('configuration mismatch') end
+if not config then redis.call('SET',KEYS[3],ARGV[6]) end
+local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
+local expired=redis.call('ZRANGEBYSCORE',KEYS[2],'-inf',now-tonumber(ARGV[5]),'LIMIT',0,128)
+for _,key in ipairs(expired) do redis.call('HDEL',KEYS[1],key);redis.call('ZREM',KEYS[2],key) end
+local value=redis.call('HGET',KEYS[1],ARGV[1]);local tokens=tonumber(ARGV[3])
+if value then local b=cjson.decode(value);tokens=math.min(tokens,b.tokens+math.max(0,now-b.time)*tonumber(ARGV[2])/1000)
+elseif redis.call('HLEN',KEYS[1])>=tonumber(ARGV[4]) then return -1 end
+local delay=0;if tokens>=1 then tokens=tokens-1 else delay=math.ceil((1-tokens)*1000/tonumber(ARGV[2])) end
+redis.call('HSET',KEYS[1],ARGV[1],cjson.encode({tokens=tokens,time=now}));redis.call('ZADD',KEYS[2],now,ARGV[1]);return delay
+`)
+
+func (l *limiter) Take(ctx context.Context, key string) (time.Duration, error) {
+	sum := sha256.Sum256([]byte(key))
+	delay, err := take.Run(ctx, l.client, l.keys, hex.EncodeToString(sum[:]), l.cfg.Rate, l.cfg.Burst, l.cfg.MaxKeys, l.cfg.IdleTTL.Milliseconds(), l.signature).Int64()
+	if err != nil {
+		return 0, err
+	}
+	if delay < 0 {
+		return 0, middleware.ErrLimitCapacity
+	}
+	return time.Duration(delay) * time.Millisecond, nil
+}

middleware/redis/store_test.go

@@ -0,0 +1,145 @@
+package redis
+
+import (
+	"context"
+	"fmt"
+	omni "git.yuntuo.hk/k2safe/go-omni"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"git.yuntuo.hk/k2safe/go-omni/middleware"
+	goredis "github.com/redis/go-redis/v9"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+func TestSharedRedisIntegration(t *testing.T) {
+	address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
+	if address == "" {
+		t.Skip("requires isolated Redis")
+	}
+	client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 8, MaxRetries: -1, ContextTimeoutEnabled: true, ReadTimeout: time.Second, WriteTimeout: time.Second})
+	defer client.Close()
+	ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
+	defer cancel()
+	prefix := fmt.Sprint("shared:", time.Now().UnixNano())
+	s, err := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer client.Del(context.Background(), s.keys...)
+	key, fp := strings.Repeat("a", 64), strings.Repeat("b", 64)
+	var claims atomic.Int32
+	var wg sync.WaitGroup
+	owners := make(chan string, 32)
+	for i := 0; i < 32; i++ {
+		wg.Add(1)
+		go func(i int) {
+			defer wg.Done()
+			owner := fmt.Sprintf("%032x", i)
+			_, ok, err := s.Claim(ctx, key, fp, owner)
+			if err != nil {
+				t.Error(err)
+			}
+			if ok {
+				claims.Add(1)
+				owners <- owner
+			}
+		}(i)
+	}
+	wg.Wait()
+	if claims.Load() != 1 {
+		t.Fatal(claims.Load())
+	}
+	owner := <-owners
+	if err = s.Complete(ctx, key, "wrong", nil, time.Minute); err == nil {
+		t.Fatal("foreign completion accepted")
+	}
+	if err = s.Complete(ctx, key, owner, &middleware.StoredResponse{Status: 201, Headers: http.Header{}, Body: []byte{0, 1, 255}}, time.Minute); err != nil {
+		t.Fatal(err)
+	}
+	record, ok, err := s.Claim(ctx, key, fp, strings.Repeat("c", 32))
+	if err != nil || ok || record.State != "completed" || len(record.Response.Body) != 3 {
+		t.Fatal(record, ok, err)
+	}
+	// Two independently constructed apps use the same backend, so only one runs.
+	var executions atomic.Int32
+	makeApp := func() *omni.App {
+		backend, _ := NewIdempotencyStore(client, prefix, IdempotencyConfig{MaxEntries: 256})
+		idem, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Store: backend, Scope: func(*httpx.Context) (string, error) { return "verifiedTenant", nil }})
+		if err != nil {
+			t.Fatal(err)
+		}
+		a := omni.New(omni.WithLogger(false))
+		a.POST("/submit", func(c *httpx.Context) error { executions.Add(1); return c.OK("saved") }, idem.Middleware())
+		return a
+	}
+	a, b := makeApp(), makeApp()
+	request := func(a *omni.App, body string) *httptest.ResponseRecorder {
+		r := httptest.NewRequest("POST", "/submit", strings.NewReader(body))
+		r.Header.Set("Idempotency-Key", "stableKey")
+		w := httptest.NewRecorder()
+		a.ServeHTTP(w, r)
+		return w
+	}
+	first, second := request(a, `{}`), request(b, `{}`)
+	if first.Code != 200 || second.Code != 200 || first.Body.String() != second.Body.String() || second.Header().Get("Idempotency-Replayed") != "true" || executions.Load() != 1 {
+		t.Fatal(first.Code, second.Code, second.Body.String(), executions.Load())
+	}
+	if request(b, `{"changed":true}`).Code != 409 {
+		t.Fatal("conflict accepted")
+	}
+	// Shared limits do not multiply when a second replica is added.
+	limitPrefix := prefix + "limit"
+	cfg := middleware.RateLimitConfig{Rate: .01, Burst: 2, Key: func(*httpx.Context) string { return "tenant" }}
+	r1, _ := NewRateLimiter(client, limitPrefix, cfg)
+	r2, _ := NewRateLimiter(client, limitPrefix, cfg)
+	base, _ := namespace(limitPrefix, "rate")
+	defer client.Del(context.Background(), base+":buckets", base+":seen", base+":config")
+	limited := func(l *middleware.RateLimiter) int {
+		a := omni.New(omni.WithLogger(false))
+		a.GET("/", func(c *httpx.Context) error { return c.OK("ok") }, l.Middleware())
+		w := httptest.NewRecorder()
+		a.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
+		return w.Code
+	}
+	if limited(r1) != 200 || limited(r2) != 200 || limited(r1) != 429 {
+		t.Fatal("limits multiplied across instances")
+	}
+	// Redis failure must fail closed before executing a new request.
+	client.Close()
+	if status := limited(r2); status != 503 {
+		t.Fatal(status)
+	}
+}
+
+func TestExpiredTargetBeyondCleanupBatch(t *testing.T) {
+	address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
+	if address == "" {
+		t.Skip("requires isolated Redis")
+	}
+	client := goredis.NewClient(&goredis.Options{Addr: address})
+	defer client.Close()
+	s, _ := NewIdempotencyStore(client, fmt.Sprint("expiry:", time.Now().UnixNano()), IdempotencyConfig{MaxEntries: 256})
+	defer client.Del(context.Background(), s.keys...)
+	ctx := context.Background()
+	owner := strings.Repeat("1", 32)
+	fp := strings.Repeat("2", 64)
+	for i := 0; i < 140; i++ {
+		key := fmt.Sprintf("%064x", i)
+		if _, ok, err := s.Claim(ctx, key, fp, owner); err != nil || !ok {
+			t.Fatal(err)
+		}
+		if err := s.Complete(ctx, key, owner, nil, time.Minute); err != nil {
+			t.Fatal(err)
+		}
+		client.ZAdd(ctx, s.keys[1], goredis.Z{Score: 0, Member: key})
+	}
+	if _, ok, err := s.Claim(ctx, fmt.Sprintf("%064x", 139), fp, owner); err != nil || !ok {
+		t.Fatal("expired target retained beyond TTL", ok, err)
+	}
+}

middleware/sharedIdempotency.go

@@ -0,0 +1,102 @@
+package middleware
+
+import (
+	"context"
+	"crypto/rand"
+	"encoding/hex"
+	"errors"
+	apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"net/http"
+	"strings"
+	"time"
+)
+
+type StoredResponse struct {
+	Status  int         `json:"status"`
+	Headers http.Header `json:"headers"`
+	Body    []byte      `json:"body"`
+}
+type IdempotencyRecord struct {
+	Fingerprint string          `json:"fingerprint"`
+	Owner       string          `json:"owner"`
+	State       string          `json:"state"`
+	Response    *StoredResponse `json:"response,omitempty"`
+}
+
+// IdempotencyStore must atomically claim absent keys, compare owners when
+// finishing and retain in-flight claims until explicit resolution. Only completed
+// or failed records expire; an uncertain outcome must never become a new claim.
+type IdempotencyStore interface {
+	Claim(context.Context, string, string, string) (IdempotencyRecord, bool, error)
+	Complete(context.Context, string, string, *StoredResponse, time.Duration) error
+}
+
+func (i *Idempotency) shared(c *httpx.Context, next httpx.Handler, id, fingerprint [32]byte) error {
+	key, fp := hex.EncodeToString(id[:]), hex.EncodeToString(fingerprint[:])
+	token := make([]byte, 16)
+	if _, err := rand.Read(token); err != nil {
+		return err
+	}
+	owner := hex.EncodeToString(token)
+	record, claimed, err := i.config.Store.Claim(c.Context(), key, fp, owner)
+	if err != nil {
+		return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "idempotency store unavailable").WithCause(err)
+	}
+	if !claimed {
+		if record.Fingerprint != fp {
+			return apierrors.New(409, "IDEMPOTENCY_CONFLICT", "key was used with a different request")
+		}
+		if record.State != "completed" || record.Response == nil {
+			return apierrors.New(409, "IDEMPOTENCY_UNAVAILABLE", "request is in progress or outcome requires reconciliation")
+		}
+		response := record.Response
+		headers := response.Headers.Clone()
+		headers.Set("Idempotency-Replayed", "true")
+		return c.WriteResponse(response.Status, headers, response.Body)
+	}
+	completed := false
+	finish := func(response *StoredResponse) error {
+		ctx, cancel := context.WithTimeout(context.WithoutCancel(c.Context()), time.Second)
+		defer cancel()
+		return i.config.Store.Complete(ctx, key, owner, response, i.config.TTL)
+	}
+	defer func() {
+		if !completed {
+			if err := finish(nil); err != nil {
+				logx.ErrorContext(c.Context(), "idempotency.persistFailed", "error", err)
+			}
+		}
+	}()
+	buffer := &boundedResponse{headers: c.Raw().Header().Clone(), limit: i.config.MaxResponseBytes}
+	child := c.Clone(c.Context(), buffer)
+	err = next(child)
+	if ctxErr := child.Context().Err(); ctxErr != nil {
+		return ctxErr
+	}
+	if err != nil {
+		child.WriteError(err)
+	}
+	if buffer.err != nil {
+		return apierrors.Internal(buffer.err)
+	}
+	if buffer.status == 0 {
+		buffer.status = 200
+	}
+	var response *StoredResponse
+	if buffer.status < 500 && buffer.headers.Get("Set-Cookie") == "" && !strings.Contains(strings.ToLower(buffer.headers.Get("Cache-Control")), "no-store") {
+		response = &StoredResponse{Status: buffer.status, Headers: buffer.headers.Clone(), Body: append([]byte(nil), buffer.body.Bytes()...)}
+	}
+	if err = finish(response); err != nil {
+		return apierrors.New(503, "IDEMPOTENCY_UNAVAILABLE", "outcome requires reconciliation").WithCause(err)
+	}
+	completed = true
+	return c.WriteResponse(buffer.status, buffer.headers, buffer.body.Bytes())
+}
+
+var ErrLimitCapacity = errors.New("middleware: shared rate limiter capacity")
+
+type RateLimitBackend interface {
+	Take(context.Context, string) (time.Duration, error)
+}

openapi/openapi.go

@@ -0,0 +1,267 @@
+// Package openapi registers routes and their OpenAPI 3.0.3 contracts together.
+package openapi
+
+import (
+	"encoding/json"
+	"errors"
+	"fmt"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"net/http"
+	"reflect"
+	"regexp"
+	"strings"
+	"sync"
+	"time"
+)
+
+type Schema struct {
+	Type                 string             `json:"type,omitempty"`
+	Format               string             `json:"format,omitempty"`
+	Description          string             `json:"description,omitempty"`
+	Properties           map[string]*Schema `json:"properties,omitempty"`
+	Required             []string           `json:"required,omitempty"`
+	Items                *Schema            `json:"items,omitempty"`
+	AdditionalProperties *Schema            `json:"additionalProperties,omitempty"`
+	Nullable             bool               `json:"nullable,omitempty"`
+}
+type Parameter struct {
+	Name        string  `json:"name"`
+	In          string  `json:"in"`
+	Required    bool    `json:"required"`
+	Description string  `json:"description,omitempty"`
+	Schema      *Schema `json:"schema"`
+}
+type Response struct {
+	Description string
+	Schema      *Schema
+}
+type Operation struct {
+	Method, Path, OperationID, Summary string
+	Parameters                         []Parameter
+	Request                            *Schema
+	Responses                          map[int]Response
+	BearerAuth                         bool
+}
+type Registrar interface {
+	Handle(string, string, httpx.Handler, ...httpx.Middleware)
+}
+type API struct {
+	mu       sync.RWMutex
+	document map[string]any
+	paths    map[string]map[string]any
+	ids      map[string]bool
+}
+
+var identifier = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9_]*$`)
+
+func New(title, version string) (*API, error) {
+	if title == "" || version == "" {
+		return nil, errors.New("openapi: title and version required")
+	}
+	paths := make(map[string]map[string]any)
+	return &API{paths: paths, ids: make(map[string]bool), document: map[string]any{"openapi": "3.0.3", "info": map[string]string{"title": title, "version": version}, "paths": paths, "components": map[string]any{"securitySchemes": map[string]any{"bearerAuth": map[string]string{"type": "http", "scheme": "bearer"}}}}}, nil
+}
+
+// Handle registers an absolute application route. To document a group, include
+// its full prefix here and pass middleware explicitly. Schemas describe wire
+// responses; Envelope wraps a data schema in GoOmni's standard JSON envelope.
+func (a *API) Handle(router Registrar, op Operation, handler httpx.Handler, middleware ...httpx.Middleware) error {
+	if router == nil || handler == nil || !identifier.MatchString(op.OperationID) || !strings.HasPrefix(op.Path, "/") || strings.ContainsAny(op.Path, "*{}?# ") {
+		return errors.New("openapi: invalid route")
+	}
+	method := strings.ToLower(op.Method)
+	switch method {
+	case "get", "post", "put", "patch", "delete", "head", "options":
+	default:
+		return errors.New("openapi: unsupported HTTP method")
+	}
+	parts := strings.Split(op.Path, "/")
+	params := make(map[string]bool)
+	for n, part := range parts {
+		if strings.HasPrefix(part, ":") {
+			name := part[1:]
+			if !identifier.MatchString(name) || params[name] {
+				return errors.New("openapi: invalid path parameter")
+			}
+			parts[n] = "{" + name + "}"
+			params[name] = true
+		}
+	}
+	path := strings.Join(parts, "/")
+	seen := make(map[string]bool)
+	for _, p := range op.Parameters {
+		if p.Name == "" || p.Schema == nil || seen[p.In+":"+p.Name] {
+			return errors.New("openapi: invalid or duplicate parameter")
+		}
+		seen[p.In+":"+p.Name] = true
+		switch p.In {
+		case "path":
+			if !params[p.Name] || !p.Required {
+				return errors.New("openapi: path parameter must match and be required")
+			}
+			delete(params, p.Name)
+		case "query", "header", "cookie":
+		default:
+			return errors.New("openapi: invalid parameter location")
+		}
+	}
+	if len(params) != 0 || len(op.Responses) == 0 {
+		return errors.New("openapi: declare all path parameters and at least one response")
+	}
+	responses := map[string]any{}
+	for code, response := range op.Responses {
+		if code < 100 || code > 599 || response.Description == "" {
+			return errors.New("openapi: invalid response")
+		}
+		item := map[string]any{"description": response.Description}
+		if response.Schema != nil {
+			item["content"] = map[string]any{"application/json": map[string]any{"schema": response.Schema}}
+		}
+		responses[fmt.Sprint(code)] = item
+	}
+	operation := map[string]any{"operationId": op.OperationID, "summary": op.Summary, "responses": responses}
+	if len(op.Parameters) > 0 {
+		operation["parameters"] = op.Parameters
+	}
+	if op.Request != nil {
+		operation["requestBody"] = map[string]any{"required": true, "content": map[string]any{"application/json": map[string]any{"schema": op.Request}}}
+	}
+	if op.BearerAuth {
+		operation["security"] = []map[string][]string{{"bearerAuth": {}}}
+	}
+	// Detach caller-owned maps/slices before concurrent document reads.
+	encoded, err := json.Marshal(operation)
+	if err != nil {
+		return err
+	}
+	var frozen any
+	if err = json.Unmarshal(encoded, &frozen); err != nil {
+		return err
+	}
+	a.mu.Lock()
+	defer a.mu.Unlock()
+	if a.ids[op.OperationID] || a.paths[path][method] != nil {
+		return errors.New("openapi: duplicate operation ID or route")
+	}
+	router.Handle(strings.ToUpper(method), op.Path, handler, middleware...)
+	if a.paths[path] == nil {
+		a.paths[path] = make(map[string]any)
+	}
+	a.paths[path][method] = frozen
+	a.ids[op.OperationID] = true
+	return nil
+}
+func (a *API) JSON() ([]byte, error) {
+	a.mu.RLock()
+	defer a.mu.RUnlock()
+	return json.MarshalIndent(a.document, "", "  ")
+}
+func (a *API) Handler() httpx.Handler {
+	return func(c *httpx.Context) error {
+		data, err := a.JSON()
+		if err != nil {
+			return err
+		}
+		return c.WriteResponse(200, http.Header{"Content-Type": {"application/json; charset=utf-8"}}, data)
+	}
+}
+func Envelope(data *Schema) *Schema {
+	return &Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}, "data": data}}
+}
+func SchemaOf[T any]() (*Schema, error) {
+	return schema(reflect.TypeOf((*T)(nil)).Elem(), map[reflect.Type]bool{}, 0)
+}
+func schema(t reflect.Type, stack map[reflect.Type]bool, depth int) (*Schema, error) {
+	if depth > 32 || stack[t] {
+		return nil, errors.New("openapi: recursive DTO requires an explicit schema")
+	}
+	if t.Kind() == reflect.Pointer {
+		s, err := schema(t.Elem(), stack, depth+1)
+		if err == nil {
+			s.Nullable = true
+		}
+		return s, err
+	}
+	if t == reflect.TypeOf(time.Time{}) {
+		return &Schema{Type: "string", Format: "date-time"}, nil
+	}
+	if t == reflect.TypeOf(json.RawMessage{}) {
+		return &Schema{}, nil
+	}
+	if t == reflect.TypeOf(json.Number("")) {
+		return &Schema{Type: "number"}, nil
+	}
+	marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
+	if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) {
+		return nil, errors.New("openapi: custom JSON marshaler requires an explicit schema")
+	}
+	switch t.Kind() {
+	case reflect.String:
+		return &Schema{Type: "string"}, nil
+	case reflect.Bool:
+		return &Schema{Type: "boolean"}, nil
+	case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
+		return &Schema{Type: "integer"}, nil
+	case reflect.Float32, reflect.Float64:
+		return &Schema{Type: "number"}, nil
+	case reflect.Slice, reflect.Array:
+		if t.Elem().Kind() == reflect.Uint8 && t.Kind() == reflect.Slice {
+			return &Schema{Type: "string", Format: "byte"}, nil
+		}
+		s, err := schema(t.Elem(), stack, depth+1)
+		return &Schema{Type: "array", Items: s}, err
+	case reflect.Map:
+		if t.Key().Kind() != reflect.String {
+			return nil, errors.New("openapi: map keys must be strings")
+		}
+		s, err := schema(t.Elem(), stack, depth+1)
+		return &Schema{Type: "object", AdditionalProperties: s}, err
+	case reflect.Interface:
+		return &Schema{}, nil
+	case reflect.Struct:
+		stack[t] = true
+		defer delete(stack, t)
+		s := &Schema{Type: "object", Properties: map[string]*Schema{}}
+		for n := 0; n < t.NumField(); n++ {
+			f := t.Field(n)
+			if f.PkgPath != "" {
+				continue
+			}
+			tag := strings.Split(f.Tag.Get("json"), ",")
+			if tag[0] == "-" {
+				continue
+			}
+			if f.Anonymous && tag[0] == "" {
+				return nil, errors.New("openapi: embedded fields require an explicit JSON name")
+			}
+			name := tag[0]
+			if name == "" {
+				name = f.Name
+			}
+			child, err := schema(f.Type, stack, depth+1)
+			if err != nil {
+				return nil, err
+			}
+			if s.Properties[name] != nil {
+				return nil, errors.New("openapi: duplicate JSON field")
+			}
+			s.Properties[name] = child
+			optional := false
+			for _, option := range tag[1:] {
+				if option == "string" {
+					child.Type = "string"
+					child.Format = ""
+				}
+				if option == "omitempty" || option == "omitzero" {
+					optional = true
+				}
+			}
+			if !optional {
+				s.Required = append(s.Required, name)
+			}
+		}
+		return s, nil
+	default:
+		return nil, fmt.Errorf("openapi: unsupported DTO type %s", t)
+	}
+}

openapi/openapi_test.go

@@ -0,0 +1,54 @@
+package openapi_test
+
+import (
+	"encoding/json"
+	omni "git.yuntuo.hk/k2safe/go-omni"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"git.yuntuo.hk/k2safe/go-omni/openapi"
+	"net/http/httptest"
+	"testing"
+)
+
+func TestRouteAndContract(t *testing.T) {
+	type item struct {
+		ID        int64           `json:"id,string"`
+		RequestID string          `json:"requestId,omitempty"`
+		Payload   json.RawMessage `json:"payload"`
+	}
+	schema, err := openapi.SchemaOf[item]()
+	if err != nil || schema.Properties["id"].Type != "string" || schema.Properties["payload"].Type != "" {
+		t.Fatal(schema, err)
+	}
+	api, _ := openapi.New("Test API", "1")
+	app := omni.New(omni.WithLogger(false))
+	op := openapi.Operation{Method: "GET", Path: "/items/:id", OperationID: "getItem", Parameters: []openapi.Parameter{{Name: "id", In: "path", Required: true, Schema: &openapi.Schema{Type: "string"}}}, Responses: map[int]openapi.Response{200: {Description: "OK", Schema: openapi.Envelope(schema)}}}
+	if err = api.Handle(app, op, func(c *httpx.Context) error { return c.OK(item{ID: 1, Payload: json.RawMessage(`{}`)}) }); err != nil {
+		t.Fatal(err)
+	}
+	// Caller edits must not mutate the published contract.
+	schema.Properties["id"].Type = "boolean"
+	if err = api.Handle(app, op, func(*httpx.Context) error { return nil }); err == nil {
+		t.Fatal("duplicate accepted")
+	}
+	app.GET("/openapi.json", api.Handler())
+	r := httptest.NewRecorder()
+	app.ServeHTTP(r, httptest.NewRequest("GET", "/openapi.json", nil))
+	var doc struct {
+		OpenAPI string                     `json:"openapi"`
+		Paths   map[string]json.RawMessage `json:"paths"`
+	}
+	if err = json.Unmarshal(r.Body.Bytes(), &doc); err != nil || doc.OpenAPI != "3.0.3" || doc.Paths["/items/{id}"] == nil {
+		t.Fatal(r.Body.String(), err)
+	}
+	r = httptest.NewRecorder()
+	app.ServeHTTP(r, httptest.NewRequest("GET", "/items/1", nil))
+	if r.Code != 200 {
+		t.Fatal(r.Code)
+	}
+	type recursive struct {
+		Next *recursive `json:"next"`
+	}
+	if _, err = openapi.SchemaOf[recursive](); err == nil {
+		t.Fatal("recursive DTO silently truncated")
+	}
+}

pagination/pagination.go

@@ -0,0 +1,73 @@
+// Package pagination defines bounded offset pagination with camelCase fields.
+package pagination
+
+import (
+	"errors"
+	"net/url"
+	"strconv"
+)
+
+type Config struct {
+	DefaultSize int   `json:"defaultSize"`
+	MaxSize     int   `json:"maxSize"`
+	MaxOffset   int64 `json:"maxOffset"`
+}
+type Request struct {
+	Page     int64 `json:"page"`
+	PageSize int   `json:"pageSize"`
+}
+
+func (r Request) Offset() int64 { return (r.Page - 1) * int64(r.PageSize) }
+func Parse(query url.Values, cfg Config) (Request, error) {
+	if cfg.DefaultSize == 0 {
+		cfg.DefaultSize = 20
+	}
+	if cfg.MaxSize == 0 {
+		cfg.MaxSize = 100
+	}
+	if cfg.MaxOffset == 0 {
+		cfg.MaxOffset = 1000000
+	}
+	if cfg.DefaultSize < 1 || cfg.MaxSize < cfg.DefaultSize || cfg.MaxSize > 10000 || cfg.MaxOffset < 0 {
+		return Request{}, errors.New("pagination: invalid configuration")
+	}
+	page, size := int64(1), int64(cfg.DefaultSize)
+	for key, target := range map[string]*int64{"page": &page, "pageSize": &size} {
+		if v, ok := query[key]; ok {
+			if len(v) != 1 {
+				return Request{}, errors.New("pagination: duplicate parameter")
+			}
+			n, err := strconv.ParseInt(v[0], 10, 64)
+			if err != nil || n < 1 {
+				return Request{}, errors.New("pagination: positive integer required")
+			}
+			*target = n
+		}
+	}
+	if size > int64(cfg.MaxSize) || (page-1) > cfg.MaxOffset/size {
+		return Request{}, errors.New("pagination: size or offset exceeds limit")
+	}
+	return Request{Page: page, PageSize: int(size)}, nil
+}
+
+type Result[T any] struct {
+	Items      []T   `json:"items"`
+	Page       int64 `json:"page"`
+	PageSize   int   `json:"pageSize"`
+	Total      int64 `json:"total"`
+	TotalPages int64 `json:"totalPages"`
+}
+
+func NewResult[T any](r Request, items []T, total int64) (Result[T], error) {
+	if r.Page < 1 || r.PageSize < 1 || total < 0 || len(items) > r.PageSize || int64(len(items)) > total {
+		return Result[T]{}, errors.New("pagination: invalid result")
+	}
+	if items == nil {
+		items = []T{}
+	}
+	pages := total / int64(r.PageSize)
+	if total%int64(r.PageSize) != 0 {
+		pages++
+	}
+	return Result[T]{Items: items, Page: r.Page, PageSize: r.PageSize, Total: total, TotalPages: pages}, nil
+}

pagination/pagination_test.go

@@ -0,0 +1,30 @@
+package pagination
+
+import (
+	"encoding/json"
+	"math"
+	"net/url"
+	"strings"
+	"testing"
+)
+
+func TestBoundsAndEnvelope(t *testing.T) {
+	for _, raw := range []string{"page=0", "page=1&page=2", "pageSize=101", "page=9223372036854775807&pageSize=100", "pageSize=-1"} {
+		q, _ := url.ParseQuery(raw)
+		if _, err := Parse(q, Config{}); err == nil {
+			t.Fatal(raw)
+		}
+	}
+	r, err := Parse(url.Values{"page": {"2"}, "pageSize": {"30"}}, Config{})
+	if err != nil || r.Offset() != 30 {
+		t.Fatal(r, err)
+	}
+	result, err := NewResult(r, []string(nil), math.MaxInt64)
+	if err != nil || result.TotalPages != math.MaxInt64/30+1 {
+		t.Fatal(result, err)
+	}
+	raw, _ := json.Marshal(result)
+	if !strings.Contains(string(raw), `"items":[]`) || !strings.Contains(string(raw), `"pageSize":30`) {
+		t.Fatal(string(raw))
+	}
+}

queue/queue.go

@@ -0,0 +1,62 @@
+// Package queue defines durable, at-least-once work delivery. Handlers must be
+// idempotent: a crash after a business commit but before ACK can cause redelivery.
+// This is a message queue, not a workflow engine or a distributed transaction.
+package queue
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strings"
+)
+
+var ErrFull = errors.New("queue: capacity reached")
+var ErrConflict = errors.New("queue: message ID already has different content")
+var ErrLeaseLost = errors.New("queue: delivery ownership changed")
+
+type Message struct {
+	ID      string          `json:"id"` // Stable producer identity, saved before submitting.
+	Name    string          `json:"name"`
+	Payload json.RawMessage `json:"payload"`
+}
+type Receipt struct {
+	ID        string `json:"id"`
+	BackendID string `json:"backendId"`
+}
+type Delivery struct {
+	Message
+	Attempt int64 `json:"attempt"`
+}
+type Handler func(context.Context, Delivery) error
+type Publisher interface {
+	Publish(context.Context, Message) (Receipt, error)
+}
+type Consumer interface {
+	Run(context.Context, Handler) error
+}
+
+// Reject marks a known permanent failure; it is retained for operator review.
+// Ordinary errors are retried up to the consumer's configured attempt limit.
+type Rejection struct{ Code string }
+
+func (e *Rejection) Error() string { return "queue: rejected: " + e.Code }
+func Reject(code string) error     { return &Rejection{Code: code} }
+
+func Validate(ctx context.Context, m Message, maxBytes int) error {
+	if ctx == nil {
+		return errors.New("queue: context required")
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	for _, v := range []string{m.ID, m.Name} {
+		if v == "" || len(v) > 128 || strings.IndexFunc(v, func(r rune) bool { return r < 33 || r > 126 }) >= 0 {
+			return errors.New("queue: ID and name must be 1..128 printable ASCII characters")
+		}
+	}
+	if len(m.Payload) > maxBytes || !json.Valid(m.Payload) {
+		return fmt.Errorf("queue: payload must be valid JSON within %d bytes", maxBytes)
+	}
+	return nil
+}

queue/redis/consumer.go

@@ -0,0 +1,213 @@
+package redis
+
+import (
+	"context"
+	"crypto/rand"
+	"encoding/hex"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"time"
+
+	"git.yuntuo.hk/k2safe/go-omni/queue"
+	goredis "github.com/redis/go-redis/v9"
+)
+
+type ConsumerConfig struct {
+	Workers           int           `json:"workers"`
+	MaxAttempts       int64         `json:"maxAttempts"`
+	TaskTimeout       time.Duration `json:"taskTimeout"`
+	VisibilityTimeout time.Duration `json:"visibilityTimeout"`
+	ShutdownTimeout   time.Duration `json:"shutdownTimeout"`
+}
+type Consumer struct {
+	store *Store
+	cfg   ConsumerConfig
+	used  atomic.Bool
+}
+
+func NewConsumer(store *Store, cfg ConsumerConfig) (*Consumer, error) {
+	if store == nil || cfg.Workers < 0 || cfg.MaxAttempts < 0 || cfg.TaskTimeout < 0 || cfg.VisibilityTimeout < 0 || cfg.ShutdownTimeout < 0 {
+		return nil, errors.New("queue/redis: invalid consumer configuration")
+	}
+	if cfg.Workers == 0 {
+		cfg.Workers = 4
+	}
+	if cfg.MaxAttempts == 0 {
+		cfg.MaxAttempts = 3
+	}
+	if cfg.TaskTimeout == 0 {
+		cfg.TaskTimeout = 30 * time.Second
+	}
+	if cfg.VisibilityTimeout == 0 {
+		cfg.VisibilityTimeout = 2 * cfg.TaskTimeout
+	}
+	if cfg.ShutdownTimeout == 0 {
+		cfg.ShutdownTimeout = 5 * time.Second
+	}
+	if cfg.Workers > 128 || cfg.MaxAttempts > 100 || cfg.TaskTimeout < 10*time.Millisecond || cfg.TaskTimeout > time.Hour || cfg.VisibilityTimeout < 2*cfg.TaskTimeout || cfg.VisibilityTimeout > 24*time.Hour || cfg.ShutdownTimeout > time.Minute {
+		return nil, errors.New("queue/redis: consumer limits exceeded")
+	}
+	return &Consumer{store: store, cfg: cfg}, nil
+}
+
+// Run is called once. Cancellation stops fetching and cancels handlers; ACK is
+// deferred to a later delivery if cancellation makes the outcome uncertain.
+// A handler must honor context cancellation. Go cannot terminate an arbitrary
+// function; an ignoring handler can outlive ShutdownTimeout (at most Workers).
+func (c *Consumer) Run(ctx context.Context, handler queue.Handler) error {
+	if ctx == nil || handler == nil {
+		return errors.New("queue/redis: context and handler required")
+	}
+	if !c.used.CompareAndSwap(false, true) {
+		return errors.New("queue/redis: consumer already run")
+	}
+	signature := fmt.Sprintf("%d:%d:%d", c.cfg.MaxAttempts, c.cfg.TaskTimeout.Milliseconds(), c.cfg.VisibilityTimeout.Milliseconds())
+	if err := goredis.NewScript(`local v=redis.call('GET',KEYS[1]);if v and v~=ARGV[1] then return redis.error_reply('consumer configuration mismatch') end;redis.call('SET',KEYS[1],ARGV[1]);return 1`).Run(ctx, c.store.client, []string{c.store.keys[5]}, signature).Err(); err != nil {
+		return err
+	}
+	if err := c.store.client.XGroupCreateMkStream(ctx, c.store.keys[0], "workers", "0").Err(); err != nil && !strings.HasPrefix(err.Error(), "BUSYGROUP ") {
+		return err
+	}
+	var token [16]byte
+	if _, err := rand.Read(token[:]); err != nil {
+		return err
+	}
+	runCtx, cancel := context.WithCancel(ctx)
+	defer cancel()
+	failures := make(chan error, c.cfg.Workers)
+	var wg sync.WaitGroup
+	for n := 0; n < c.cfg.Workers; n++ {
+		wg.Add(1)
+		go func(n int) {
+			defer wg.Done()
+			if err := c.loop(runCtx, hex.EncodeToString(token[:])+fmt.Sprint(n), handler); err != nil {
+				failures <- err
+				cancel()
+			}
+		}(n)
+	}
+	done := make(chan struct{})
+	go func() { wg.Wait(); close(done) }()
+	select {
+	case <-done:
+	case <-runCtx.Done():
+		select {
+		case <-done:
+		case <-time.After(c.cfg.ShutdownTimeout):
+			return errors.New("queue/redis: handler shutdown timeout")
+		}
+	}
+	select {
+	case err := <-failures:
+		return err
+	default:
+		return ctx.Err()
+	}
+}
+func (c *Consumer) loop(ctx context.Context, owner string, handler queue.Handler) error {
+	s := c.store
+	defer func() {
+		cleanup, cancel := context.WithTimeout(context.Background(), time.Second)
+		defer cancel()
+		_ = goredis.NewScript(`local p=redis.call('XPENDING',KEYS[1],'workers','-','+',1,ARGV[1]);if #p==0 then return redis.call('XGROUP','DELCONSUMER',KEYS[1],'workers',ARGV[1]) end;return 0`).Run(cleanup, s.client, []string{s.keys[0]}, owner).Err()
+	}()
+	cursor := "0-0"
+	for ctx.Err() == nil {
+		messages, next, err := s.client.XAutoClaim(ctx, &goredis.XAutoClaimArgs{Stream: s.keys[0], Group: "workers", Consumer: owner, MinIdle: c.cfg.VisibilityTimeout, Start: cursor, Count: 1}).Result()
+		if err != nil && err != goredis.Nil {
+			if ctx.Err() != nil {
+				return nil
+			}
+			return err
+		}
+		cursor = next
+		if cursor == "" {
+			cursor = "0-0"
+		}
+		if len(messages) == 0 {
+			streams, err := s.client.XReadGroup(ctx, &goredis.XReadGroupArgs{Group: "workers", Consumer: owner, Streams: []string{s.keys[0], ">"}, Count: 1, Block: -1}).Result()
+			if err != nil && err != goredis.Nil {
+				if ctx.Err() != nil {
+					return nil
+				}
+				return err
+			}
+			if len(streams) > 0 {
+				messages = streams[0].Messages
+			}
+		}
+		if len(messages) == 0 {
+			select {
+			case <-ctx.Done():
+				return nil
+			case <-time.After(25 * time.Millisecond):
+			}
+			continue
+		}
+		message := messages[0]
+		raw, ok := message.Values["message"].(string)
+		var m queue.Message
+		if !ok || len(raw) > s.cfg.MaxPayloadBytes+1024 || json.Unmarshal([]byte(raw), &m) != nil {
+			return errors.New("queue/redis: corrupt message; operator review required")
+		}
+		pending, err := s.client.XPendingExt(ctx, &goredis.XPendingExtArgs{Stream: s.keys[0], Group: "workers", Start: message.ID, End: message.ID, Count: 1}).Result()
+		if err != nil {
+			if ctx.Err() != nil {
+				return nil
+			}
+			return err
+		}
+		if len(pending) != 1 || pending[0].Consumer != owner {
+			continue
+		}
+		delivery := queue.Delivery{Message: m, Attempt: pending[0].RetryCount}
+		reason := ""
+		if delivery.Attempt > c.cfg.MaxAttempts {
+			reason = "attemptsExhausted"
+		} else {
+			workCtx, cancel := context.WithTimeout(ctx, c.cfg.TaskTimeout)
+			err = invoke(workCtx, handler, delivery)
+			deadlineErr := workCtx.Err()
+			cancel()
+			if ctx.Err() != nil {
+				return nil
+			}
+			var rejection *queue.Rejection
+			if errors.As(err, &rejection) {
+				reason = "rejected"
+			} else if err != nil || deadlineErr != nil {
+				if delivery.Attempt < c.cfg.MaxAttempts {
+					continue
+				}
+				reason = "attemptsExhausted"
+			}
+		}
+		ackCtx, ackCancel := context.WithTimeout(ctx, 3*time.Second)
+		err = s.finish(ackCtx, message.ID, owner, delivery, reason)
+		ackCancel()
+		if err != nil && !errors.Is(err, queue.ErrLeaseLost) {
+			if ctx.Err() != nil {
+				return nil
+			}
+			return err
+		}
+	}
+	return nil
+}
+func invoke(ctx context.Context, handler queue.Handler, delivery queue.Delivery) (err error) {
+	defer func() {
+		if recover() != nil {
+			err = errors.New("queue: handler panic")
+		}
+	}()
+	if err = ctx.Err(); err != nil {
+		return err
+	}
+	return handler(ctx, delivery)
+}
+
+var _ queue.Consumer = (*Consumer)(nil)

queue/redis/consumer_test.go

@@ -0,0 +1,154 @@
+package redis
+
+import (
+	"context"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"git.yuntuo.hk/k2safe/go-omni/queue"
+	goredis "github.com/redis/go-redis/v9"
+	"os"
+	"sync/atomic"
+	"testing"
+	"time"
+)
+
+func testStore(t *testing.T, max int) *Store {
+	t.Helper()
+	address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
+	if address == "" {
+		t.Skip("requires isolated Redis")
+	}
+	client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 4, MaxRetries: -1, ContextTimeoutEnabled: true, ReadTimeout: time.Second, WriteTimeout: time.Second})
+	s, err := New(client, Config{Namespace: fmt.Sprint(t.Name(), time.Now().UnixNano()), MaxRecords: max, DedupTTL: time.Second})
+	if err != nil {
+		t.Fatal(err)
+	}
+	t.Cleanup(func() { client.Del(context.Background(), s.keys...); client.Close() })
+	return s
+}
+func message(id string) queue.Message {
+	return queue.Message{ID: id, Name: "test", Payload: json.RawMessage(`{"value":1}`)}
+}
+func eventually(t *testing.T, f func() bool) {
+	t.Helper()
+	deadline := time.Now().Add(3 * time.Second)
+	for !f() {
+		if time.Now().After(deadline) {
+			t.Fatal("condition did not recover")
+		}
+		time.Sleep(10 * time.Millisecond)
+	}
+}
+func TestQueueRedisIntegration(t *testing.T) {
+	t.Run("publicationAndCapacity", func(t *testing.T) {
+		s := testStore(t, 1)
+		ctx := context.Background()
+		receipt, err := s.Publish(ctx, message("first"))
+		if err != nil {
+			t.Fatal(err)
+		}
+		again, err := s.Publish(ctx, message("first"))
+		if err != nil || again != receipt {
+			t.Fatal(again, err)
+		}
+		changed := message("first")
+		changed.Payload = json.RawMessage(`2`)
+		if _, err = s.Publish(ctx, changed); !errors.Is(err, queue.ErrConflict) {
+			t.Fatal(err)
+		}
+		if _, err = s.Publish(ctx, message("second")); !errors.Is(err, queue.ErrFull) {
+			t.Fatal(err)
+		}
+	})
+	t.Run("staleAcknowledgement", func(t *testing.T) {
+		s := testStore(t, 10)
+		ctx := context.Background()
+		receipt, _ := s.Publish(ctx, message("first"))
+		s.client.XGroupCreateMkStream(ctx, s.keys[0], "workers", "0")
+		s.client.XReadGroup(ctx, &goredis.XReadGroupArgs{Group: "workers", Consumer: "old", Streams: []string{s.keys[0], ">"}, Count: 1, Block: -1})
+		if err := s.client.XClaim(ctx, &goredis.XClaimArgs{Stream: s.keys[0], Group: "workers", Consumer: "new", MinIdle: 0, Messages: []string{receipt.BackendID}}).Err(); err != nil {
+			t.Fatal(err)
+		}
+		delivery := queue.Delivery{Message: message("first"), Attempt: 2}
+		if err := s.finish(ctx, receipt.BackendID, "old", delivery, ""); !errors.Is(err, queue.ErrLeaseLost) {
+			t.Fatal(err)
+		}
+		if err := s.finish(ctx, receipt.BackendID, "new", delivery, ""); err != nil {
+			t.Fatal(err)
+		}
+		if n := s.client.XLen(ctx, s.keys[0]).Val(); n != 0 {
+			t.Fatal(n)
+		}
+	})
+	t.Run("crashReclaimRetryAndDeadLetters", func(t *testing.T) {
+		s := testStore(t, 10)
+		ctx := context.Background()
+		s.Publish(ctx, message("crashed"))
+		s.Publish(ctx, message("retry"))
+		s.Publish(ctx, message("reject"))
+		s.client.XGroupCreateMkStream(ctx, s.keys[0], "workers", "0")
+		s.client.XReadGroup(ctx, &goredis.XReadGroupArgs{Group: "workers", Consumer: "crashedProcess", Streams: []string{s.keys[0], ">"}, Count: 1, Block: -1})
+		cfg := ConsumerConfig{Workers: 1, MaxAttempts: 3, TaskTimeout: 50 * time.Millisecond, VisibilityTimeout: 100 * time.Millisecond, ShutdownTimeout: time.Second}
+		consumer, _ := NewConsumer(s, cfg)
+		second, _ := NewConsumer(s, cfg)
+		runCtx, cancel := context.WithCancel(ctx)
+		var retries, reclaimed, rejected atomic.Int32
+		handler := func(ctx context.Context, d queue.Delivery) error {
+			switch d.ID {
+			case "crashed":
+				if d.Attempt != 2 {
+					t.Error("reclaim count", d.Attempt)
+				}
+				reclaimed.Add(1)
+				return nil
+			case "retry":
+				retries.Add(1)
+				return errors.New("known retryable failure")
+			default:
+				rejected.Add(1)
+				return queue.Reject("invalidInput")
+			}
+		}
+		done := make(chan error, 2)
+		go func() { done <- consumer.Run(runCtx, handler) }()
+		go func() { done <- second.Run(runCtx, handler) }()
+		defer cancel()
+		eventually(t, func() bool {
+			dead, err := s.DeadLetters(ctx, "", 100)
+			return err == nil && len(dead) == 2 && reclaimed.Load() == 1
+		})
+		cancel()
+		for i := 0; i < 2; i++ {
+			if err := <-done; !errors.Is(err, context.Canceled) {
+				t.Fatal(err)
+			}
+		}
+		if retries.Load() != 3 || rejected.Load() != 1 {
+			t.Fatal(retries.Load(), rejected.Load())
+		}
+		dead, err := s.DeadLetters(ctx, "", 1)
+		if err != nil || len(dead) != 1 {
+			t.Fatal(dead, err)
+		}
+		rest, err := s.DeadLetters(ctx, dead[0].StreamID, 1)
+		if err != nil || len(rest) != 1 || rest[0].StreamID == dead[0].StreamID {
+			t.Fatal(rest, err)
+		}
+		if err = s.DiscardDead(ctx, dead[0].Message.ID, dead[0].StreamID); err != nil {
+			t.Fatal(err)
+		}
+		remaining, _ := s.DeadLetters(ctx, "", 100)
+		if len(remaining) != 1 {
+			t.Fatal(remaining)
+		}
+		// Retained completed IDs expire and release publication capacity.
+		eventually(t, func() bool {
+			_, err := s.Publish(ctx, message("crashed"))
+			if err != nil {
+				return false
+			}
+			return s.client.XLen(ctx, s.keys[0]).Val() == 2
+		})
+	})
+}

queue/redis/store.go

@@ -0,0 +1,187 @@
+// Package redis implements one managed Redis Streams consumer group per queue.
+// Use Redis 6.2+, persistence, noeviction and a private namespace. The application
+// owns the client. Delivery durability depends on the Redis deployment.
+package redis
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/hex"
+	"encoding/json"
+	"errors"
+	"fmt"
+	"time"
+
+	"git.yuntuo.hk/k2safe/go-omni/queue"
+	goredis "github.com/redis/go-redis/v9"
+)
+
+type Config struct {
+	Namespace       string        `json:"namespace"`
+	MaxRecords      int           `json:"maxRecords"` // Includes pending, dead and retained completed identities.
+	MaxPayloadBytes int           `json:"maxPayloadBytes"`
+	DedupTTL        time.Duration `json:"dedupTtl"` // Starts at ACK; pending/dead records never expire automatically.
+}
+type Store struct {
+	client    goredis.UniversalClient
+	cfg       Config
+	keys      []string
+	signature string
+}
+
+func New(client goredis.UniversalClient, cfg Config) (*Store, error) {
+	if client == nil || cfg.Namespace == "" || len(cfg.Namespace) > 256 || cfg.MaxRecords < 0 || cfg.MaxPayloadBytes < 0 || cfg.DedupTTL < 0 {
+		return nil, errors.New("queue/redis: invalid configuration")
+	}
+	if cfg.MaxRecords == 0 {
+		cfg.MaxRecords = 1000
+	}
+	if cfg.MaxPayloadBytes == 0 {
+		cfg.MaxPayloadBytes = 64 << 10
+	}
+	if cfg.DedupTTL == 0 {
+		cfg.DedupTTL = 24 * time.Hour
+	}
+	if cfg.MaxRecords > 100000 || cfg.MaxPayloadBytes > 1<<20 || cfg.DedupTTL < time.Second || cfg.DedupTTL > 30*24*time.Hour {
+		return nil, errors.New("queue/redis: configuration exceeds supported bounds")
+	}
+	sum := sha256.Sum256([]byte(cfg.Namespace))
+	base := "goOmni:{" + hex.EncodeToString(sum[:]) + "}:queue"
+	return &Store{client: client, cfg: cfg, keys: []string{base + ":stream", base + ":records", base + ":expiry", base + ":dead", base + ":config", base + ":consumerConfig"}, signature: fmt.Sprintf("%d:%d:%d", cfg.MaxRecords, cfg.MaxPayloadBytes, cfg.DedupTTL.Milliseconds())}, nil
+}
+
+var publish = goredis.NewScript(`
+local config=redis.call('GET',KEYS[5]);if config and config~=ARGV[5] then return redis.error_reply('configuration mismatch') end
+if not config then redis.call('SET',KEYS[5],ARGV[5]) end
+local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000)
+local expired=redis.call('ZRANGEBYSCORE',KEYS[3],'-inf',now,'LIMIT',0,128)
+for _,id in ipairs(expired) do redis.call('HDEL',KEYS[2],id);redis.call('ZREM',KEYS[3],id) end
+local expiry=redis.call('ZSCORE',KEYS[3],ARGV[1]);if expiry and tonumber(expiry)<=now then redis.call('HDEL',KEYS[2],ARGV[1]);redis.call('ZREM',KEYS[3],ARGV[1]) end
+local old=redis.call('HGET',KEYS[2],ARGV[1]);if old then local r=cjson.decode(old);if r.fingerprint~=ARGV[2] then return {-1,''} end;return {0,r.streamId} end
+if redis.call('HLEN',KEYS[2])>=tonumber(ARGV[4]) then return {-2,''} end
+local id=redis.call('XADD',KEYS[1],'*','message',ARGV[3])
+redis.call('HSET',KEYS[2],ARGV[1],cjson.encode({fingerprint=ARGV[2],streamId=id,state='pending'}));return {1,id}
+`)
+
+func (s *Store) Publish(ctx context.Context, m queue.Message) (queue.Receipt, error) {
+	if err := queue.Validate(ctx, m, s.cfg.MaxPayloadBytes); err != nil {
+		return queue.Receipt{}, err
+	}
+	raw, _ := json.Marshal(m)
+	if len(raw) > s.cfg.MaxPayloadBytes+1024 {
+		return queue.Receipt{}, errors.New("queue/redis: encoded message exceeds limit")
+	}
+	sum := sha256.Sum256(raw)
+	r, err := publish.Run(ctx, s.client, s.keys, m.ID, hex.EncodeToString(sum[:]), string(raw), s.cfg.MaxRecords, s.signature).Slice()
+	if err != nil {
+		return queue.Receipt{}, err
+	}
+	if len(r) != 2 {
+		return queue.Receipt{}, errors.New("queue/redis: invalid publish result")
+	}
+	switch r[0] {
+	case int64(-1):
+		return queue.Receipt{}, queue.ErrConflict
+	case int64(-2):
+		return queue.Receipt{}, queue.ErrFull
+	}
+	id, ok := r[1].(string)
+	if !ok {
+		return queue.Receipt{}, errors.New("queue/redis: invalid receipt")
+	}
+	return queue.Receipt{ID: m.ID, BackendID: id}, nil
+}
+
+var finish = goredis.NewScript(`
+local p=redis.call('XPENDING',KEYS[1],'workers',ARGV[1],ARGV[1],1)
+if #p~=1 or p[1][2]~=ARGV[2] then return 0 end
+local raw=redis.call('HGET',KEYS[2],ARGV[3]);if not raw then return redis.error_reply('message record missing') end
+local record=cjson.decode(raw);if record.streamId~=ARGV[1] or record.state~='pending' then return redis.error_reply('message identity mismatch') end
+if ARGV[4]~='' then
+ record.state='dead';redis.call('ZADD',KEYS[4],0,ARGV[1]);record.dead=cjson.decode(ARGV[4])
+else
+ record.state='completed';local clock=redis.call('TIME');local now=clock[1]*1000+math.floor(clock[2]/1000);redis.call('ZADD',KEYS[3],now+tonumber(ARGV[5]),ARGV[3])
+end
+redis.call('HSET',KEYS[2],ARGV[3],cjson.encode(record));redis.call('XACK',KEYS[1],'workers',ARGV[1])
+if ARGV[4]=='' then redis.call('XDEL',KEYS[1],ARGV[1]) end
+return 1
+`)
+
+func (s *Store) finish(ctx context.Context, id, owner string, m queue.Delivery, reason string) error {
+	var dead []byte
+	if reason != "" {
+		dead, _ = json.Marshal(map[string]any{"attempt": m.Attempt, "reason": reason})
+	}
+	n, err := finish.Run(ctx, s.client, s.keys, id, owner, m.ID, string(dead), s.cfg.DedupTTL.Milliseconds()).Int()
+	if err != nil {
+		return err
+	}
+	if n != 1 {
+		return queue.ErrLeaseLost
+	}
+	return nil
+}
+
+type DeadLetter struct {
+	StreamID string        `json:"streamId"`
+	Message  queue.Message `json:"message"`
+	Attempt  int64         `json:"attempt"`
+	Reason   string        `json:"reason"`
+}
+
+// DeadLetters returns a bounded page, ordered by opaque Redis stream ID. Pass
+// the last StreamID as after. Payloads remain retained until explicit removal.
+func (s *Store) DeadLetters(ctx context.Context, after string, limit int64) ([]DeadLetter, error) {
+	if ctx == nil || limit < 1 || limit > 100 {
+		return nil, errors.New("queue/redis: context and limit 1..100 required")
+	}
+	min := "-"
+	if after != "" {
+		min = "(" + after
+	}
+	ids, err := s.client.ZRangeByLex(ctx, s.keys[3], &goredis.ZRangeBy{Min: min, Max: "+", Count: limit}).Result()
+	if err != nil {
+		return nil, err
+	}
+	out := make([]DeadLetter, 0, len(ids))
+	for _, id := range ids {
+		messages, err := s.client.XRangeN(ctx, s.keys[0], id, id, 1).Result()
+		if err != nil {
+			return nil, err
+		}
+		if len(messages) == 0 {
+			continue
+		}
+		var m queue.Message
+		raw, _ := messages[0].Values["message"].(string)
+		if err = json.Unmarshal([]byte(raw), &m); err != nil {
+			return nil, err
+		}
+		record, err := s.client.HGet(ctx, s.keys[1], m.ID).Result()
+		if err == goredis.Nil {
+			continue
+		}
+		if err != nil {
+			return nil, err
+		}
+		var saved struct {
+			Dead struct {
+				Attempt int64  `json:"attempt"`
+				Reason  string `json:"reason"`
+			} `json:"dead"`
+		}
+		if err = json.Unmarshal([]byte(record), &saved); err != nil {
+			return nil, err
+		}
+		out = append(out, DeadLetter{StreamID: id, Message: m, Attempt: saved.Dead.Attempt, Reason: saved.Dead.Reason})
+	}
+	return out, nil
+}
+
+// DiscardDead deletes only a matching dead delivery after operator review.
+// It retains the producer identity for DedupTTL, preventing immediate resubmit.
+func (s *Store) DiscardDead(ctx context.Context, messageID, streamID string) error {
+	return goredis.NewScript(`local raw=redis.call('HGET',KEYS[2],ARGV[1]);if not raw then return 0 end;local r=cjson.decode(raw);if r.state~='dead' or r.streamId~=ARGV[2] then return redis.error_reply('dead identity mismatch') end;r.state='discarded';r.dead=nil;redis.call('HSET',KEYS[2],ARGV[1],cjson.encode(r));redis.call('ZREM',KEYS[4],ARGV[2]);redis.call('XDEL',KEYS[1],ARGV[2]);local t=redis.call('TIME');redis.call('ZADD',KEYS[3],t[1]*1000+math.floor(t[2]/1000)+tonumber(ARGV[3]),ARGV[1]);return 1`).Run(ctx, s.client, s.keys, messageID, streamID, s.cfg.DedupTTL.Milliseconds()).Err()
+}
+
+var _ queue.Publisher = (*Store)(nil)

resilience/breaker.go

@@ -0,0 +1,129 @@
+// Package resilience provides per-dependency circuit breaking and concurrency isolation.
+package resilience
+
+import (
+	"context"
+	"errors"
+	"sync"
+	"time"
+)
+
+var ErrOpen = errors.New("resilience: circuit open")
+var ErrBusy = errors.New("resilience: concurrency limit reached")
+
+type Config struct {
+	FailureThreshold int              `json:"failureThreshold"`
+	MaxConcurrent    int              `json:"maxConcurrent"`
+	OpenTimeout      time.Duration    `json:"openTimeout"`
+	IsFailure        func(error) bool `json:"-"`
+}
+type Breaker struct {
+	mu               sync.Mutex
+	config           Config
+	active, failures int
+	state            string
+	until            time.Time
+	generation       uint64
+	now              func() time.Time
+}
+type Snapshot struct {
+	State    string `json:"state"`
+	Active   int    `json:"active"`
+	Failures int    `json:"failures"`
+}
+
+func New(cfg Config) (*Breaker, error) {
+	if cfg.FailureThreshold < 0 || cfg.MaxConcurrent < 0 || cfg.OpenTimeout < 0 {
+		return nil, errors.New("resilience: invalid configuration")
+	}
+	if cfg.FailureThreshold == 0 {
+		cfg.FailureThreshold = 5
+	}
+	if cfg.MaxConcurrent == 0 {
+		cfg.MaxConcurrent = 32
+	}
+	if cfg.OpenTimeout == 0 {
+		cfg.OpenTimeout = 5 * time.Second
+	}
+	if cfg.IsFailure == nil {
+		cfg.IsFailure = func(err error) bool { return err != nil && !errors.Is(err, context.Canceled) }
+	}
+	return &Breaker{config: cfg, state: "closed", now: time.Now}, nil
+}
+
+// Acquire returns a completion callback; it must be called exactly once.
+// A half-open circuit admits one probe. Old-generation results cannot close it.
+func (b *Breaker) Acquire(ctx context.Context) (func(bool), error) {
+	if ctx == nil {
+		return nil, errors.New("resilience: nil context")
+	}
+	if err := ctx.Err(); err != nil {
+		return nil, err
+	}
+	b.mu.Lock()
+	defer b.mu.Unlock()
+	if b.state == "open" {
+		if b.now().Before(b.until) {
+			return nil, ErrOpen
+		}
+		b.state = "halfOpen"
+		b.generation++
+	}
+	if b.state == "halfOpen" && b.active > 0 {
+		return nil, ErrOpen
+	}
+	if b.active >= b.config.MaxConcurrent {
+		return nil, ErrBusy
+	}
+	b.active++
+	generation := b.generation
+	var once sync.Once
+	return func(success bool) {
+		once.Do(func() {
+			b.mu.Lock()
+			defer b.mu.Unlock()
+			b.active--
+			if generation != b.generation {
+				return
+			}
+			if success {
+				b.failures = 0
+				if b.state == "halfOpen" {
+					b.state = "closed"
+					b.generation++
+				}
+				return
+			}
+			b.failures++
+			if b.state == "halfOpen" || b.failures >= b.config.FailureThreshold {
+				b.state = "open"
+				b.until = b.now().Add(b.config.OpenTimeout)
+				b.generation++
+			}
+		})
+	}, nil
+}
+func (b *Breaker) Do(ctx context.Context, work func(context.Context) error) (err error) {
+	if work == nil {
+		return errors.New("resilience: nil work")
+	}
+	done, err := b.Acquire(ctx)
+	if err != nil {
+		return err
+	}
+	complete := false
+	defer func() {
+		if !complete {
+			done(false)
+		}
+	}()
+	err = work(ctx)
+	done(!b.config.IsFailure(err))
+	complete = true
+	return err
+}
+func (b *Breaker) Snapshot() Snapshot {
+	b.mu.Lock()
+	defer b.mu.Unlock()
+	return Snapshot{State: b.state, Active: b.active, Failures: b.failures}
+}

resilience/breaker_test.go

@@ -0,0 +1,75 @@
+package resilience
+
+import (
+	"context"
+	"errors"
+	"io"
+	"net/http"
+	"strings"
+	"testing"
+	"time"
+)
+
+type roundTripFunc func(*http.Request) (*http.Response, error)
+
+func (f roundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
+func TestBreakerProbeAndLateResult(t *testing.T) {
+	b, _ := New(Config{FailureThreshold: 1, MaxConcurrent: 2, OpenTimeout: time.Second})
+	now := time.Now()
+	b.now = func() time.Time { return now }
+	ctx := context.Background()
+	first, _ := b.Acquire(ctx)
+	late, _ := b.Acquire(ctx)
+	first(false)
+	late(true)
+	if b.Snapshot().State != "open" {
+		t.Fatal(b.Snapshot())
+	}
+	if _, err := b.Acquire(ctx); !errors.Is(err, ErrOpen) {
+		t.Fatal(err)
+	}
+	now = now.Add(time.Second)
+	probe, err := b.Acquire(ctx)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err = b.Acquire(ctx); !errors.Is(err, ErrOpen) {
+		t.Fatal(err)
+	}
+	probe(true)
+	probe(false)
+	if s := b.Snapshot(); s.State != "closed" || s.Active != 0 {
+		t.Fatal(s)
+	}
+}
+func TestTransportHoldsSlotAndReleasesPanic(t *testing.T) {
+	b, _ := New(Config{MaxConcurrent: 1})
+	tr := Transport{Breaker: b, Base: roundTripFunc(func(*http.Request) (*http.Response, error) {
+		return &http.Response{StatusCode: 200, Body: io.NopCloser(strings.NewReader("hello"))}, nil
+	})}
+	req, _ := http.NewRequest("GET", "http://example.test", nil)
+	r, err := tr.RoundTrip(req)
+	if err != nil {
+		t.Fatal(err)
+	}
+	if _, err = tr.RoundTrip(req); !errors.Is(err, ErrBusy) {
+		t.Fatal(err)
+	}
+	io.ReadAll(r.Body)
+	r.Body.Close()
+	if b.Snapshot().Active != 0 {
+		t.Fatal(b.Snapshot())
+	}
+	tr.Base = roundTripFunc(func(*http.Request) (*http.Response, error) { panic("fixture") })
+	func() {
+		defer func() {
+			if recover() == nil {
+				t.Error("panic lost")
+			}
+		}()
+		tr.RoundTrip(req)
+	}()
+	if b.Snapshot().Active != 0 {
+		t.Fatal(b.Snapshot())
+	}
+}

resilience/http.go

@@ -0,0 +1,67 @@
+package resilience
+
+import (
+	"errors"
+	"io"
+	"net/http"
+)
+
+// Transport isolates one logical upstream. Create separate Breakers for
+// different dependencies. Slots remain occupied until body Close/EOF.
+type Transport struct {
+	Base    http.RoundTripper
+	Breaker *Breaker
+}
+
+func (t Transport) RoundTrip(r *http.Request) (*http.Response, error) {
+	if t.Breaker == nil {
+		return nil, errors.New("resilience: breaker required")
+	}
+	done, err := t.Breaker.Acquire(r.Context())
+	if err != nil {
+		return nil, err
+	}
+	defer func() {
+		if v := recover(); v != nil {
+			done(false)
+			panic(v)
+		}
+	}()
+	base := t.Base
+	if base == nil {
+		base = http.DefaultTransport
+	}
+	response, err := base.RoundTrip(r)
+	if err != nil {
+		done(!t.Breaker.config.IsFailure(err))
+		if response != nil && response.Body != nil {
+			response.Body.Close()
+		}
+		return nil, err
+	}
+	if response == nil || response.Body == nil {
+		done(false)
+		return nil, errors.New("resilience: invalid transport response")
+	}
+	response.Body = &breakerBody{ReadCloser: response.Body, done: done, success: response.StatusCode < 500}
+	return response, nil
+}
+
+type breakerBody struct {
+	io.ReadCloser
+	done    func(bool)
+	success bool
+}
+
+func (b *breakerBody) Read(p []byte) (int, error) {
+	n, err := b.ReadCloser.Read(p)
+	if err != nil {
+		b.done(b.success && err == io.EOF)
+	}
+	return n, err
+}
+func (b *breakerBody) Close() error {
+	err := b.ReadCloser.Close()
+	b.done(b.success && err == nil)
+	return err
+}

scripts/verifyModules.py

@@ -22,6 +22,9 @@ def main():
     parser = argparse.ArgumentParser(description=__doc__)
     parser.add_argument("--image", default="golang:1.25")
     parser.add_argument("--redis-image", default="redis:7.4-alpine")
+    parser.add_argument("--advanced", action="store_true", help="also verify shared state, durable queue, S3 and MySQL migrations")
+    parser.add_argument("--s3-image", default="quay.io/minio/minio:RELEASE.2025-09-07T16-13-09Z")
+    parser.add_argument("--mysql-image", default="m.daocloud.io/docker.io/library/mysql:8.4")
     parser.add_argument("--goproxy", default="")
     parser.add_argument("--wall-timeout", type=int, default=900)
     parser.add_argument("--output", type=Path, default=Path("performance/modulesLinux.json"))
@@ -40,9 +43,13 @@ def main():
     output_path.parent.mkdir(parents=True, exist_ok=True)
     log_path = output_path.with_suffix(".txt")
     prefix = "go-omni-modules-" + uuid.uuid4().hex[:12]
-    names = [prefix + "-" + stage for stage in ("build", "redis")]
+    stages = ("build", "redis", "s3", "mysql") if args.advanced else ("build", "redis")
+    names = [prefix + "-" + stage for stage in stages]
     paths = [root / "go.mod", root / "go.sum", root / "httpx/context.go", Path(__file__).resolve()]
-    for directory in ("cache", "httpc", "task", "middleware", "cmd/go-omni", "examples/modules"):
+    directories = ["cache", "httpc", "task", "middleware", "cmd/go-omni", "examples/modules"]
+    if args.advanced:
+        directories += ["auth", "config", "logx", "openapi", "telemetry", "resilience", "storage", "queue", "audit", "pagination", "database/migrate", "internal/storagefixture", "examples/platform"]
+    for directory in directories:
         paths.extend(p for p in (root / directory).rglob("*") if p.is_file() and p.suffix in (".go", ".tmpl"))
     report = {
         "status": "running", "startedAt": datetime.now(timezone.utc).isoformat(),
@@ -86,7 +93,10 @@ def main():
                 report["cleanedContainers"].append(name)
 
         try:
-            for image in (args.image, args.redis_image):
+            images = [args.image, args.redis_image]
+            if args.advanced:
+                images += [args.s3_image, args.mysql_image]
+            for image in images:
                 result = subprocess.run([docker, "image", "inspect", image], capture_output=True, timeout=15)
                 if result.returncode:
                     print("Pulling " + image, flush=True)
@@ -101,9 +111,11 @@ def main():
                              "-e", "GOWORK=off", "-e", "GOMAXPROCS=2", "-e", "GO_OMNI_SCAFFOLD_INTEGRATION=1"]
                 if args.goproxy:
                     arguments += ["-e", "GOPROXY=" + args.goproxy]
-                arguments += [args.image, "sh", "-c",
+                checks = "go version && go test -race -count=1 -timeout=240s ./... && go vet ./... && go build ./... && CGO_ENABLED=0 go test -c -o /output/redis.test ./cache/redis"
-                              "go version && go test -race -count=1 -timeout=240s ./... && go vet ./... && go build ./... && "
+                if args.advanced:
-                              "CGO_ENABLED=0 go test -c -o /output/redis.test ./cache/redis"]
+                    for binary, package in (("shared", "middleware/redis"), ("queue", "queue/redis"), ("s3", "storage/s3"), ("mysql", "database/migrate")):
+                        checks += f" && CGO_ENABLED=0 go test -c -o /output/{binary}.test ./{package}"
+                arguments += [args.image, "sh", "-c", checks]
                 container(names[0], arguments)
                 print("Real Redis TTL/read/write/delete/failure checks; external network disabled", flush=True)
                 redis_check = '''set -eu
@@ -118,15 +130,54 @@ until redis-cli -h 127.0.0.1 ping >/dev/null 2>&1; do
 done
 /test/redis.test -test.v -test.run '^TestRedisIntegration$' -test.count=1 -test.timeout=30s
 '''
+                if args.advanced:
+                    redis_check += "/test/shared.test -test.v -test.count=1 -test.timeout=45s\n/test/queue.test -test.v -test.count=1 -test.timeout=45s\n"
                 container(names[1], ["--network", "none", "--read-only", "--cap-drop", "ALL", "--security-opt", "no-new-privileges",
                                      "--user", "999:999", "--tmpfs", "/tmp:rw,nosuid,noexec,size=64m,mode=1777",
                                      "-v", f"{temporary}:/test:ro", "--entrypoint", "sh",
                                      "-e", "GOMAXPROCS=2", "-e", "GO_OMNI_REDIS_TEST_ADDR=127.0.0.1:6379",
                                      args.redis_image, "-c", redis_check])
+                if args.advanced:
+                    isolated = ["--network", "none", "--read-only", "--cap-drop", "ALL", "--security-opt", "no-new-privileges",
+                                "-v", f"{temporary}:/test:ro", "--entrypoint", "sh", "-e", "GOMAXPROCS=2"]
+                    print("Real S3 object operations and signature verification in isolated MinIO", flush=True)
+                    s3_check = '''set -eu
+minio --version
+minio server /data --address 127.0.0.1:9000 --console-address 127.0.0.1:9001 >/tmp/minio.log 2>&1 &
+server=$!
+trap 'kill "$server" 2>/dev/null || true; wait "$server" 2>/dev/null || true' EXIT
+/test/s3.test -test.v -test.run '^TestS3Integration$' -test.count=1 -test.timeout=45s || { cat /tmp/minio.log; exit 1; }
+'''
+                    container(names[2], isolated + ["--user", "1000:1000", "--tmpfs", "/tmp:rw,nosuid,noexec,size=64m,mode=1777",
+                              "--tmpfs", "/data:rw,nosuid,noexec,size=128m,mode=1777", "-e", "MINIO_ROOT_USER=goOmniTest", "-e", "MINIO_ROOT_PASSWORD=goOmniTestSecret",
+                              "-e", "GO_OMNI_S3_TEST_ADDR=127.0.0.1:9000", args.s3_image, "-c", s3_check])
+                    print("Real MySQL migrations, repeat apply, dirty-state blocking and SQL spans", flush=True)
+                    mysql_check = '''set -eu
+mysqld --version
+mkdir /tmp/mysql
+mysqld --no-defaults --initialize-insecure --datadir=/tmp/mysql --innodb-redo-log-capacity=67108864 >/tmp/init.log 2>&1 || { cat /tmp/init.log; exit 1; }
+mysqld --no-defaults --datadir=/tmp/mysql --socket=/tmp/mysql.sock --pid-file=/tmp/mysql.pid --bind-address=127.0.0.1 --port=3306 --mysqlx=OFF --skip-log-bin --skip-name-resolve --innodb-buffer-pool-size=134217728 --innodb-redo-log-capacity=67108864 --max-connections=8 --log-error=/tmp/mysql.log &
+server=$!
+trap 'kill "$server" 2>/dev/null || true; wait "$server" 2>/dev/null || true' EXIT
+attempt=0
+until mysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'SELECT 1' >/dev/null 2>&1; do
+    attempt=$((attempt + 1)); if [ "$attempt" -gt 200 ]; then cat /tmp/mysql.log; exit 1; fi; sleep 0.1
+done
+mysql --no-defaults --socket=/tmp/mysql.sock -uroot -e 'CREATE DATABASE omni_test'
+/test/mysql.test -test.v -test.run '^TestMySQLIntegration$' -test.count=1 -test.timeout=60s
+'''
+                    container(names[3], isolated + ["--user", "999:999", "--tmpfs", "/tmp:rw,nosuid,noexec,size=512m,mode=1777",
+                              "--tmpfs", "/var/lib/mysql:rw,nosuid,noexec,size=1m,mode=1777",
+                              "-e", "GO_OMNI_MIGRATE_TEST_DSN=root@unix(/tmp/mysql.sock)/omni_test?parseTime=true&timeout=2s&readTimeout=5s&writeTimeout=5s",
+                              args.mysql_image, "-c", mysql_check])
             log.flush()
             content = log_path.read_text()
             if "--- PASS: TestRedisIntegration" not in content:
                 raise RuntimeError("missing real Redis pass evidence")
+            if args.advanced:
+                for test in ("TestSharedRedisIntegration", "TestExpiredTargetBeyondCleanupBatch", "TestQueueRedisIntegration", "TestS3Integration", "TestMySQLIntegration"):
+                    if "--- PASS: " + test not in content:
+                        raise RuntimeError("missing integration pass evidence: " + test)
             report["status"] = "passed"
         except (Exception, KeyboardInterrupt) as error:
             failure = str(error) or type(error).__name__

storage/local/local.go

@@ -0,0 +1,171 @@
+// Package local stores objects atomically in a private, flat managed directory.
+// This directory must not be writable by untrusted local users/processes.
+package local
+
+import (
+	"context"
+	"crypto/sha256"
+	"encoding/binary"
+	"encoding/hex"
+	"encoding/json"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	"io"
+	"os"
+	"path/filepath"
+	"strings"
+)
+
+type Store struct {
+	root     string
+	maxBytes int64
+}
+
+func New(root string, maxBytes int64) (*Store, error) {
+	if root == "" || maxBytes < 0 || maxBytes > 1<<40 {
+		return nil, errors.New("storage/local: invalid configuration")
+	}
+	if maxBytes == 0 {
+		maxBytes = 32 << 20
+	}
+	if err := os.MkdirAll(root, 0700); err != nil {
+		return nil, err
+	}
+	root, err := filepath.EvalSymlinks(root)
+	if err != nil {
+		return nil, err
+	}
+	root, err = filepath.Abs(root)
+	if err != nil {
+		return nil, err
+	}
+	info, err := os.Stat(root)
+	if err != nil {
+		return nil, err
+	}
+	if !info.IsDir() || info.Mode().Perm()&0022 != 0 {
+		return nil, errors.New("storage/local: root must be a private writable directory")
+	}
+	return &Store{root: root, maxBytes: maxBytes}, nil
+}
+func (s *Store) path(key string) string {
+	sum := sha256.Sum256([]byte(key))
+	return filepath.Join(s.root, hex.EncodeToString(sum[:])+".object")
+}
+func (s *Store) Put(ctx context.Context, key string, reader io.Reader, size int64, contentType string) (storage.Info, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Info{}, err
+	}
+	if reader == nil || size < 0 || len(contentType) > 256 || strings.ContainsAny(contentType, "\r\n") {
+		return storage.Info{}, errors.New("storage/local: invalid upload")
+	}
+	if size > s.maxBytes {
+		return storage.Info{}, storage.ErrTooLarge
+	}
+	info := storage.Info{Key: key, Size: size, ContentType: contentType}
+	metadata, _ := json.Marshal(info)
+	f, err := os.CreateTemp(s.root, ".upload-")
+	if err != nil {
+		return storage.Info{}, err
+	}
+	defer os.Remove(f.Name())
+	defer f.Close()
+	if err = binary.Write(f, binary.BigEndian, uint32(len(metadata))); err != nil {
+		return storage.Info{}, err
+	}
+	if _, err = f.Write(metadata); err != nil {
+		return storage.Info{}, err
+	}
+	n, err := io.Copy(f, io.LimitReader(storage.ContextReader{Context: ctx, Reader: reader}, size+1))
+	if err != nil {
+		return storage.Info{}, err
+	}
+	if n != size {
+		return storage.Info{}, errors.New("storage/local: declared size mismatch")
+	}
+	if err = ctx.Err(); err != nil {
+		return storage.Info{}, err
+	}
+	if err = f.Sync(); err != nil {
+		return storage.Info{}, err
+	}
+	stat, err := f.Stat()
+	if err != nil {
+		return storage.Info{}, err
+	}
+	if err = f.Close(); err != nil {
+		return storage.Info{}, err
+	}
+	if err = os.Rename(f.Name(), s.path(key)); err != nil {
+		return storage.Info{}, err
+	}
+	info.ModifiedAt = stat.ModTime()
+	return info, nil
+}
+func (s *Store) Get(ctx context.Context, key string) (storage.Object, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Object{}, err
+	}
+	path := s.path(key)
+	stat, err := os.Lstat(path)
+	if os.IsNotExist(err) {
+		return storage.Object{}, storage.ErrNotFound
+	}
+	if err != nil {
+		return storage.Object{}, err
+	}
+	if !stat.Mode().IsRegular() {
+		return storage.Object{}, errors.New("storage/local: not a regular managed object")
+	}
+	f, err := os.Open(path)
+	if err != nil {
+		return storage.Object{}, err
+	}
+	// A concurrent Put can atomically replace the path between Lstat/Open.
+	// Validate size against the descriptor whose bytes we actually read.
+	stat, err = f.Stat()
+	if err != nil {
+		f.Close()
+		return storage.Object{}, err
+	}
+	ok := false
+	defer func() {
+		if !ok {
+			f.Close()
+		}
+	}()
+	var length uint32
+	if err = binary.Read(f, binary.BigEndian, &length); err != nil {
+		return storage.Object{}, err
+	}
+	if length > 4096 {
+		return storage.Object{}, errors.New("storage/local: invalid object header")
+	}
+	data := make([]byte, length)
+	if _, err = io.ReadFull(f, data); err != nil {
+		return storage.Object{}, err
+	}
+	var info storage.Info
+	if json.Unmarshal(data, &info) != nil || info.Key != key || info.Size < 0 || info.Size > s.maxBytes || int64(length)+4+info.Size != stat.Size() {
+		return storage.Object{}, errors.New("storage/local: corrupt object")
+	}
+	info.ModifiedAt = stat.ModTime()
+	ok = true
+	return storage.Object{Info: info, Body: &body{Reader: storage.ContextReader{Context: ctx, Reader: io.LimitReader(f, info.Size)}, Closer: f}}, nil
+}
+
+type body struct {
+	io.Reader
+	io.Closer
+}
+
+func (s *Store) Delete(ctx context.Context, key string) error {
+	if err := storage.Check(ctx, key); err != nil {
+		return err
+	}
+	err := os.Remove(s.path(key))
+	if os.IsNotExist(err) {
+		return nil
+	}
+	return err
+}

storage/local/local_test.go

@@ -0,0 +1,42 @@
+package local
+
+import (
+	"context"
+	"git.yuntuo.hk/k2safe/go-omni/storage/storagetest"
+	"io"
+	"os"
+	"strings"
+	"testing"
+)
+
+func TestContractAndAtomicFailure(t *testing.T) {
+	root := t.TempDir()
+	s, err := New(root, 64)
+	if err != nil {
+		t.Fatal(err)
+	}
+	storagetest.Run(t, s)
+	ctx := context.Background()
+	if _, err = s.Put(ctx, "saved", strings.NewReader("old"), 3, "text/plain"); err != nil {
+		t.Fatal(err)
+	}
+	if _, err = s.Put(ctx, "saved", strings.NewReader("short"), 20, "text/plain"); err == nil {
+		t.Fatal("short upload committed")
+	}
+	again, _ := New(root, 64)
+	object, err := again.Get(ctx, "saved")
+	if err != nil {
+		t.Fatal(err)
+	}
+	raw, _ := io.ReadAll(object.Body)
+	object.Body.Close()
+	if string(raw) != "old" {
+		t.Fatal(string(raw))
+	}
+	if err = os.Symlink(s.path("saved"), s.path("linked")); err != nil {
+		t.Fatal(err)
+	}
+	if _, err = s.Get(ctx, "linked"); err == nil {
+		t.Fatal("symlink followed")
+	}
+}

storage/oss/oss.go

@@ -0,0 +1,101 @@
+// Package oss adapts the caller-owned official Alibaba Cloud OSS v2 client.
+package oss
+
+import (
+	"context"
+	"errors"
+	"io"
+	"net/url"
+	"strings"
+	"time"
+
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	sdk "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
+)
+
+type Store struct {
+	client   *sdk.Client
+	bucket   string
+	maxBytes int64
+}
+
+// New borrows a configured SDK client. This adapter uses simple uploads; large
+// multipart uploads are a separate provider capability, not buffered in memory.
+func New(client *sdk.Client, bucket string, maxBytes int64) (*Store, error) {
+	if client == nil || bucket == "" || maxBytes < 0 || maxBytes > 5<<30 {
+		return nil, errors.New("storage/oss: invalid client, bucket or size limit")
+	}
+	if maxBytes == 0 {
+		maxBytes = 32 << 20
+	}
+	return &Store{client: client, bucket: bucket, maxBytes: maxBytes}, nil
+}
+
+func (s *Store) Put(ctx context.Context, key string, reader io.Reader, size int64, contentType string) (storage.Info, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Info{}, err
+	}
+	if reader == nil || size < 0 || len(contentType) > 256 || strings.ContainsAny(contentType, "\r\n") {
+		return storage.Info{}, errors.New("storage/oss: invalid upload")
+	}
+	if size > s.maxBytes {
+		return storage.Info{}, storage.ErrTooLarge
+	}
+	_, err := s.client.PutObject(ctx, &sdk.PutObjectRequest{Bucket: sdk.Ptr(s.bucket), Key: sdk.Ptr(key), ContentLength: sdk.Ptr(size), ContentType: sdk.Ptr(contentType), Body: storage.ContextReader{Context: ctx, Reader: reader}})
+	if err != nil {
+		return storage.Info{}, mapError(err)
+	}
+	return storage.Info{Key: key, Size: size, ContentType: contentType, ModifiedAt: time.Now().UTC()}, nil
+}
+func (s *Store) Get(ctx context.Context, key string) (storage.Object, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Object{}, err
+	}
+	r, err := s.client.GetObject(ctx, &sdk.GetObjectRequest{Bucket: sdk.Ptr(s.bucket), Key: sdk.Ptr(key)})
+	if err != nil {
+		return storage.Object{}, mapError(err)
+	}
+	if r.ContentLength < 0 || r.ContentLength > s.maxBytes {
+		r.Body.Close()
+		return storage.Object{}, storage.ErrTooLarge
+	}
+	info := storage.Info{Key: key, Size: r.ContentLength, ContentType: sdk.ToString(r.ContentType)}
+	if r.LastModified != nil {
+		info.ModifiedAt = *r.LastModified
+	}
+	return storage.Object{Info: info, Body: r.Body}, nil
+}
+func (s *Store) Delete(ctx context.Context, key string) error {
+	if err := storage.Check(ctx, key); err != nil {
+		return err
+	}
+	_, err := s.client.DeleteObject(ctx, &sdk.DeleteObjectRequest{Bucket: sdk.Ptr(s.bucket), Key: sdk.Ptr(key)})
+	err = mapError(err)
+	if errors.Is(err, storage.ErrNotFound) {
+		return nil
+	}
+	return err
+}
+func (s *Store) PresignGet(ctx context.Context, key string, ttl time.Duration) (*url.URL, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return nil, err
+	}
+	if ttl < time.Second || ttl > time.Hour {
+		return nil, errors.New("storage/oss: TTL must be 1s..1h")
+	}
+	r, err := s.client.Presign(ctx, &sdk.GetObjectRequest{Bucket: sdk.Ptr(s.bucket), Key: sdk.Ptr(key), ResponseContentDisposition: sdk.Ptr("attachment")}, sdk.PresignExpires(ttl))
+	if err != nil {
+		return nil, err
+	}
+	return url.Parse(r.URL)
+}
+func mapError(err error) error {
+	var serviceError *sdk.ServiceError
+	if errors.As(err, &serviceError) && serviceError.Code == "NoSuchKey" {
+		return storage.ErrNotFound
+	}
+	return err
+}
+
+var _ storage.Store = (*Store)(nil)
+var _ storage.Presigner = (*Store)(nil)

storage/oss/oss_test.go

@@ -0,0 +1,31 @@
+package oss
+
+import (
+	"context"
+	"git.yuntuo.hk/k2safe/go-omni/internal/storagefixture"
+	"git.yuntuo.hk/k2safe/go-omni/storage/storagetest"
+	sdk "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
+	"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestSDKProtocolContract(t *testing.T) {
+	server := httptest.NewTLSServer(storagefixture.Handler())
+	defer server.Close()
+	cfg := sdk.LoadDefaultConfig().WithRegion("cn-hangzhou").WithEndpoint(server.URL).WithUsePathStyle(true).WithCredentialsProvider(credentials.NewStaticCredentialsProvider("fixtureAccess", "fixtureSecret")).WithHttpClient(server.Client())
+	store, err := New(sdk.NewClient(cfg), "testbucket", 64)
+	if err != nil {
+		t.Fatal(err)
+	}
+	storagetest.Run(t, store)
+	u, err := store.PresignGet(context.Background(), "a/file.txt", time.Minute)
+	if err != nil || u.Query().Get("x-oss-signature") == "" || !strings.Contains(u.Path, "a/file.txt") {
+		t.Fatal("invalid signed URL", err)
+	}
+	if _, err = store.PresignGet(context.Background(), "a", 2*time.Hour); err == nil {
+		t.Fatal("TTL accepted")
+	}
+}

storage/s3/config.go

@@ -0,0 +1,42 @@
+package s3
+
+import (
+	"errors"
+	"net/http"
+	"regexp"
+
+	"github.com/minio/minio-go/v7"
+	"github.com/minio/minio-go/v7/pkg/credentials"
+)
+
+// Open creates a store using an injected credential provider and transport.
+// The application owns transport shutdown; no global HTTP defaults are changed.
+type Config struct {
+	Endpoint    string                   `json:"endpoint"` // Host[:port], without scheme or path.
+	Region      string                   `json:"region"`
+	Bucket      string                   `json:"bucket"`
+	Secure      bool                     `json:"secure"`
+	MaxBytes    int64                    `json:"maxBytes"`
+	Credentials *credentials.Credentials `json:"-"`
+	Transport   http.RoundTripper        `json:"-"`
+}
+
+func Open(cfg Config) (*Store, error) {
+	if cfg.Credentials == nil {
+		return nil, errors.New("storage/s3: credentials provider required")
+	}
+	client, err := minio.New(cfg.Endpoint, &minio.Options{Creds: cfg.Credentials, Secure: cfg.Secure, Region: cfg.Region, Transport: cfg.Transport, BucketLookup: minio.BucketLookupPath})
+	if err != nil {
+		return nil, err
+	}
+	return New(client, cfg.Bucket, cfg.MaxBytes)
+}
+
+// R2Config selects Cloudflare's standard S3 API endpoint. Jurisdiction-specific
+// endpoints can be set explicitly with Config. R2 implements an S3 subset.
+func R2Config(accountID, bucket string, creds *credentials.Credentials) (Config, error) {
+	if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(accountID) {
+		return Config{}, errors.New("storage/s3: invalid R2 account ID")
+	}
+	return Config{Endpoint: accountID + ".r2.cloudflarestorage.com", Region: "auto", Bucket: bucket, Secure: true, Credentials: creds}, nil
+}

storage/s3/s3.go

@@ -0,0 +1,93 @@
+// Package s3 adapts a caller-owned MinIO client to S3-compatible object storage.
+package s3
+
+import (
+	"context"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	"github.com/minio/minio-go/v7"
+	"io"
+	"net/url"
+	"strings"
+	"time"
+)
+
+type Store struct {
+	client   *minio.Client
+	bucket   string
+	maxBytes int64
+}
+
+func New(client *minio.Client, bucket string, maxBytes int64) (*Store, error) {
+	if client == nil || bucket == "" || maxBytes < 0 {
+		return nil, errors.New("storage/s3: client, bucket and nonnegative limit required")
+	}
+	if maxBytes == 0 {
+		maxBytes = 32 << 20
+	}
+	return &Store{client: client, bucket: bucket, maxBytes: maxBytes}, nil
+}
+func (s *Store) Put(ctx context.Context, key string, reader io.Reader, size int64, contentType string) (storage.Info, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Info{}, err
+	}
+	if reader == nil || size < 0 || len(contentType) > 256 || strings.ContainsAny(contentType, "\r\n") {
+		return storage.Info{}, errors.New("storage/s3: invalid upload")
+	}
+	if size > s.maxBytes {
+		return storage.Info{}, storage.ErrTooLarge
+	}
+	result, err := s.client.PutObject(ctx, s.bucket, key, storage.ContextReader{Context: ctx, Reader: reader}, size, minio.PutObjectOptions{ContentType: contentType})
+	if err != nil {
+		return storage.Info{}, err
+	}
+	return storage.Info{Key: key, Size: result.Size, ContentType: contentType, ModifiedAt: time.Now().UTC()}, nil
+}
+func (s *Store) Get(ctx context.Context, key string) (storage.Object, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return storage.Object{}, err
+	}
+	object, err := s.client.GetObject(ctx, s.bucket, key, minio.GetObjectOptions{})
+	if err != nil {
+		return storage.Object{}, mapError(err)
+	}
+	stat, err := object.Stat()
+	if err != nil {
+		object.Close()
+		return storage.Object{}, mapError(err)
+	}
+	if stat.Size > s.maxBytes {
+		object.Close()
+		return storage.Object{}, storage.ErrTooLarge
+	}
+	return storage.Object{Info: storage.Info{Key: key, Size: stat.Size, ContentType: stat.ContentType, ModifiedAt: stat.LastModified}, Body: object}, nil
+}
+func (s *Store) Delete(ctx context.Context, key string) error {
+	if err := storage.Check(ctx, key); err != nil {
+		return err
+	}
+	err := s.client.RemoveObject(ctx, s.bucket, key, minio.RemoveObjectOptions{})
+	if errors.Is(mapError(err), storage.ErrNotFound) {
+		return nil
+	}
+	return err
+}
+func (s *Store) PresignGet(ctx context.Context, key string, ttl time.Duration) (*url.URL, error) {
+	if err := storage.Check(ctx, key); err != nil {
+		return nil, err
+	}
+	if ttl < time.Second || ttl > time.Hour {
+		return nil, errors.New("storage/s3: TTL must be 1s..1h")
+	}
+	return s.client.PresignedGetObject(ctx, s.bucket, key, ttl, url.Values{"response-content-disposition": {"attachment"}})
+}
+func mapError(err error) error {
+	if err == nil {
+		return nil
+	}
+	switch minio.ToErrorResponse(err).Code {
+	case "NoSuchKey", "NoSuchObject":
+		return storage.ErrNotFound
+	}
+	return err
+}

storage/s3/s3_test.go

@@ -0,0 +1,104 @@
+package s3
+
+import (
+	"context"
+	"fmt"
+	"git.yuntuo.hk/k2safe/go-omni/internal/storagefixture"
+	"git.yuntuo.hk/k2safe/go-omni/storage/storagetest"
+	"github.com/minio/minio-go/v7"
+	"github.com/minio/minio-go/v7/pkg/credentials"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestS3ProtocolAndR2Config(t *testing.T) {
+	server := httptest.NewTLSServer(storagefixture.Handler())
+	defer server.Close()
+	creds := credentials.NewStaticV4("fixtureAccess", "fixtureSecret", "")
+	s, err := Open(Config{Endpoint: strings.TrimPrefix(server.URL, "https://"), Region: "us-east-1", Bucket: "testbucket", Secure: true, MaxBytes: 64, Credentials: creds, Transport: server.Client().Transport})
+	if err != nil {
+		t.Fatal(err)
+	}
+	storagetest.Run(t, s)
+	cfg, err := R2Config(strings.Repeat("a", 32), "bucket", creds)
+	if err != nil {
+		t.Fatal(err)
+	}
+	r2, err := Open(cfg)
+	if err != nil {
+		t.Fatal(err)
+	}
+	u, err := r2.PresignGet(context.Background(), "path/file", time.Minute)
+	if err != nil || u.Host != strings.Repeat("a", 32)+".r2.cloudflarestorage.com" || u.Query().Get("X-Amz-Signature") == "" || !strings.Contains(u.Query().Get("X-Amz-Credential"), "/auto/s3/") {
+		t.Fatal("R2 presign settings", err)
+	}
+}
+func TestS3Integration(t *testing.T) {
+	address := os.Getenv("GO_OMNI_S3_TEST_ADDR")
+	if address == "" {
+		t.Skip("requires isolated MinIO")
+	}
+	ctx, cancel := context.WithTimeout(context.Background(), 25*time.Second)
+	defer cancel()
+	ready := false
+	for n := 0; n < 100; n++ {
+		req, _ := http.NewRequestWithContext(ctx, "GET", "http://"+address+"/minio/health/ready", nil)
+		r, err := http.DefaultClient.Do(req)
+		if err == nil {
+			r.Body.Close()
+			if r.StatusCode == 200 {
+				ready = true
+				break
+			}
+		}
+		if ctx.Err() != nil {
+			break
+		}
+		time.Sleep(100 * time.Millisecond)
+	}
+	if !ready {
+		t.Fatal("isolated S3 server did not start")
+	}
+	client, err := minio.New(address, &minio.Options{Creds: credentials.NewStaticV4("goOmniTest", "goOmniTestSecret", ""), Region: "us-east-1"})
+	if err != nil {
+		t.Fatal(err)
+	}
+	bucket := fmt.Sprintf("test-%d", time.Now().UnixNano())
+	if err = client.MakeBucket(ctx, bucket, minio.MakeBucketOptions{}); err != nil {
+		t.Fatal(err)
+	}
+	defer client.RemoveBucket(context.Background(), bucket)
+	s, _ := New(client, bucket, 64)
+	storagetest.Run(t, s)
+	s.Put(ctx, "signed", strings.NewReader("hello"), 5, "text/plain")
+	defer s.Delete(context.Background(), "signed")
+	u, err := s.PresignGet(ctx, "signed", time.Minute)
+	if err != nil {
+		t.Fatal(err)
+	}
+	response, err := http.Get(u.String())
+	if err != nil {
+		t.Fatal(err)
+	}
+	raw, _ := io.ReadAll(response.Body)
+	response.Body.Close()
+	if response.StatusCode != 200 || string(raw) != "hello" {
+		t.Fatal(response.StatusCode, string(raw))
+	}
+	q := u.Query()
+	q.Set("X-Amz-Signature", strings.Repeat("0", 64))
+	u.RawQuery = q.Encode()
+	response, err = http.Get(u.String())
+	if err != nil {
+		t.Fatal(err)
+	}
+	response.Body.Close()
+	if response.StatusCode != 403 {
+		t.Fatal("tampered signature", response.StatusCode)
+	}
+}

storage/signed.go

@@ -0,0 +1,93 @@
+package storage
+
+import (
+	"context"
+	"crypto/hmac"
+	"crypto/sha256"
+	"encoding/base64"
+	"errors"
+	"io"
+	"net/http"
+	"net/url"
+	"strconv"
+	"time"
+)
+
+type Signer struct {
+	store  Store
+	base   *url.URL
+	secret []byte
+	now    func() time.Time
+}
+
+func NewSigner(store Store, baseURL string, secret []byte) (*Signer, error) {
+	base, err := url.Parse(baseURL)
+	if err != nil {
+		return nil, err
+	}
+	if store == nil || len(secret) < 32 || base.Host == "" || (base.Scheme != "https" && base.Scheme != "http") || base.RawQuery != "" || base.Fragment != "" || base.User != nil {
+		return nil, errors.New("storage: store, HTTP base URL and >=32-byte secret required")
+	}
+	return &Signer{store: store, base: base, secret: append([]byte(nil), secret...), now: time.Now}, nil
+}
+func (s *Signer) signature(key, expiry string) string {
+	mac := hmac.New(sha256.New, s.secret)
+	io.WriteString(mac, "GET\n"+key+"\n"+expiry)
+	return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
+}
+func (s *Signer) PresignGet(ctx context.Context, key string, ttl time.Duration) (*url.URL, error) {
+	if err := Check(ctx, key); err != nil {
+		return nil, err
+	}
+	if ttl < time.Second || ttl > time.Hour {
+		return nil, errors.New("storage: TTL must be 1s..1h")
+	}
+	expiry := strconv.FormatInt(s.now().Add(ttl).Unix(), 10)
+	u := *s.base
+	u.RawQuery = url.Values{"key": {key}, "expires": {expiry}, "signature": {s.signature(key, expiry)}}.Encode()
+	return &u, nil
+}
+
+// ServeHTTP serves signed downloads as attachments. Issue links only after
+// authorization; possession of an unexpired URL grants access to its object.
+func (s *Signer) ServeHTTP(w http.ResponseWriter, r *http.Request) {
+	if r.Method != "GET" && r.Method != "HEAD" {
+		w.Header().Set("Allow", "GET, HEAD")
+		http.Error(w, "method not allowed", 405)
+		return
+	}
+	q, err := url.ParseQuery(r.URL.RawQuery)
+	if err != nil || len(q) != 3 || len(q["key"]) != 1 || len(q["expires"]) != 1 || len(q["signature"]) != 1 {
+		http.Error(w, "invalid link", 403)
+		return
+	}
+	key, expiry := q.Get("key"), q.Get("expires")
+	expires, err := strconv.ParseInt(expiry, 10, 64)
+	now := s.now().Unix()
+	if err != nil || expires <= now || expires > now+3600 || Check(r.Context(), key) != nil || !hmac.Equal([]byte(q.Get("signature")), []byte(s.signature(key, expiry))) {
+		http.Error(w, "invalid or expired link", 403)
+		return
+	}
+	object, err := s.store.Get(r.Context(), key)
+	if err != nil {
+		status := 500
+		if errors.Is(err, ErrNotFound) {
+			status = 404
+		}
+		http.Error(w, "object unavailable", status)
+		return
+	}
+	defer object.Body.Close()
+	contentType := object.ContentType
+	if contentType == "" {
+		contentType = "application/octet-stream"
+	}
+	w.Header().Set("Content-Type", contentType)
+	w.Header().Set("Content-Disposition", "attachment")
+	w.Header().Set("X-Content-Type-Options", "nosniff")
+	w.Header().Set("Cache-Control", "private, no-store")
+	w.Header().Set("Content-Length", strconv.FormatInt(object.Size, 10))
+	if r.Method == "GET" {
+		_, _ = io.Copy(w, object.Body)
+	}
+}

storage/signed_test.go

@@ -0,0 +1,35 @@
+package storage_test
+
+import (
+	"context"
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	"git.yuntuo.hk/k2safe/go-omni/storage/local"
+	"net/http/httptest"
+	"strings"
+	"testing"
+	"time"
+)
+
+func TestLocalSignedDownload(t *testing.T) {
+	s, _ := local.New(t.TempDir(), 64)
+	s.Put(context.Background(), "file", strings.NewReader("hello"), 5, "text/plain")
+	signer, _ := storage.NewSigner(s, "https://example.test/download", []byte(strings.Repeat("x", 32)))
+	u, err := signer.PresignGet(context.Background(), "file", time.Minute)
+	if err != nil {
+		t.Fatal(err)
+	}
+	request := httptest.NewRequest("GET", u.String(), nil)
+	response := httptest.NewRecorder()
+	signer.ServeHTTP(response, request)
+	if response.Code != 200 || response.Body.String() != "hello" || response.Header().Get("Content-Disposition") != "attachment" {
+		t.Fatal(response)
+	}
+	q := u.Query()
+	q.Set("key", "another")
+	u.RawQuery = q.Encode()
+	response = httptest.NewRecorder()
+	signer.ServeHTTP(response, httptest.NewRequest("GET", u.String(), nil))
+	if response.Code != 403 {
+		t.Fatal(response.Code)
+	}
+}

storage/storage.go

@@ -0,0 +1,68 @@
+// Package storage defines streaming object storage; callers own returned bodies.
+package storage
+
+import (
+	"context"
+	"errors"
+	"io"
+	"net/url"
+	"strings"
+	"time"
+	"unicode"
+	"unicode/utf8"
+)
+
+var ErrNotFound = errors.New("storage: object not found")
+var ErrTooLarge = errors.New("storage: object exceeds limit")
+
+type Info struct {
+	Key         string    `json:"key"`
+	Size        int64     `json:"size"`
+	ContentType string    `json:"contentType"`
+	ModifiedAt  time.Time `json:"modifiedAt"`
+}
+type Object struct {
+	Info
+	Body io.ReadCloser
+}
+type Store interface {
+	Put(context.Context, string, io.Reader, int64, string) (Info, error)
+	Get(context.Context, string) (Object, error)
+	Delete(context.Context, string) error
+}
+
+// Presigner is an optional capability. URLs grant temporary read access and
+// must not be logged. Providers may expire them earlier when credentials expire.
+type Presigner interface {
+	PresignGet(context.Context, string, time.Duration) (*url.URL, error)
+}
+
+func Check(ctx context.Context, key string) error {
+	if ctx == nil {
+		return errors.New("storage: nil context")
+	}
+	if err := ctx.Err(); err != nil {
+		return err
+	}
+	if key == "" || len(key) > 1024 || !utf8.ValidString(key) || strings.Contains(key, "\\") || strings.IndexFunc(key, unicode.IsControl) >= 0 {
+		return errors.New("storage: invalid key")
+	}
+	for _, part := range strings.Split(key, "/") {
+		if part == "" || part == "." || part == ".." {
+			return errors.New("storage: invalid key path")
+		}
+	}
+	return nil
+}
+
+type ContextReader struct {
+	Context context.Context
+	Reader  io.Reader
+}
+
+func (r ContextReader) Read(p []byte) (int, error) {
+	if err := r.Context.Err(); err != nil {
+		return 0, err
+	}
+	return r.Reader.Read(p)
+}

storage/storagetest/contract.go

@@ -0,0 +1,59 @@
+// Package storagetest provides shared adapter checks. Use a dedicated test
+// bucket/root and a Store configured with a 64-byte object limit.
+package storagetest
+
+import (
+	"context"
+	"errors"
+	"git.yuntuo.hk/k2safe/go-omni/storage"
+	"io"
+	"strings"
+	"testing"
+)
+
+func Run(t *testing.T, store storage.Store) {
+	t.Helper()
+	ctx := context.Background()
+	key := "contract/文件 +%.txt"
+	defer store.Delete(ctx, key)
+	if _, err := store.Get(ctx, key); !errors.Is(err, storage.ErrNotFound) {
+		t.Fatal("missing object", err)
+	}
+	for _, body := range []string{"original", "replacement"} {
+		info, err := store.Put(ctx, key, strings.NewReader(body), int64(len(body)), "text/plain")
+		if err != nil || info.Size != int64(len(body)) {
+			t.Fatal(info, err)
+		}
+		object, err := store.Get(ctx, key)
+		if err != nil {
+			t.Fatal(err)
+		}
+		data, err := io.ReadAll(object.Body)
+		object.Body.Close()
+		if err != nil || string(data) != body || object.ContentType != "text/plain" {
+			t.Fatal(string(data), object.Info, err)
+		}
+	}
+	if _, err := store.Put(ctx, "tooBig", strings.NewReader(strings.Repeat("a", 65)), 65, "text/plain"); !errors.Is(err, storage.ErrTooLarge) {
+		t.Fatal(err)
+	}
+	for _, key := range []string{"../bad", "a/../b", "/absolute", "a//b", "a\\b"} {
+		if _, err := store.Put(ctx, key, strings.NewReader("a"), 1, "text/plain"); err == nil {
+			t.Fatal(key)
+		}
+	}
+	canceled, cancel := context.WithCancel(ctx)
+	cancel()
+	if _, err := store.Get(canceled, key); !errors.Is(err, context.Canceled) {
+		t.Fatal(err)
+	}
+	if err := store.Delete(ctx, key); err != nil {
+		t.Fatal(err)
+	}
+	if err := store.Delete(ctx, key); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := store.Get(ctx, key); !errors.Is(err, storage.ErrNotFound) {
+		t.Fatal(err)
+	}
+}

telemetry/adapters.go

@@ -0,0 +1,78 @@
+package telemetry
+
+import (
+	"context"
+	"database/sql"
+	"github.com/redis/go-redis/v9"
+	"go.opentelemetry.io/otel/codes"
+	"go.opentelemetry.io/otel/trace"
+	"io"
+	"net"
+	"sync"
+)
+
+type spanBody struct {
+	io.ReadCloser
+	span trace.Span
+	once sync.Once
+}
+
+func (b *spanBody) finish(err error) {
+	b.once.Do(func() {
+		if err != nil && err != io.EOF {
+			b.span.SetStatus(codes.Error, "body read failed")
+		}
+		b.span.End()
+	})
+}
+func (b *spanBody) Read(p []byte) (int, error) {
+	n, err := b.ReadCloser.Read(p)
+	if err != nil {
+		b.finish(err)
+	}
+	return n, err
+}
+func (b *spanBody) Close() error { err := b.ReadCloser.Close(); b.finish(err); return err }
+
+type RedisHook struct{ Runtime *Runtime }
+
+func (h RedisHook) DialHook(next redis.DialHook) redis.DialHook {
+	return func(ctx context.Context, network, addr string) (conn net.Conn, err error) {
+		err = h.Runtime.Do(ctx, "redis.connect", func(ctx context.Context) error { conn, err = next(ctx, network, addr); return err })
+		return
+	}
+}
+func (h RedisHook) ProcessHook(next redis.ProcessHook) redis.ProcessHook {
+	return func(ctx context.Context, cmd redis.Cmder) error {
+		var result error
+		_ = h.Runtime.Do(ctx, "redis.command", func(ctx context.Context) error {
+			result = next(ctx, cmd)
+			if result == redis.Nil {
+				return nil
+			}
+			return result
+		})
+		return result
+	}
+}
+func (h RedisHook) ProcessPipelineHook(next redis.ProcessPipelineHook) redis.ProcessPipelineHook {
+	return func(ctx context.Context, cmds []redis.Cmder) error {
+		return h.Runtime.Do(ctx, "redis.pipeline", func(ctx context.Context) error { return next(ctx, cmds) })
+	}
+}
+
+// SQL records execution spans without SQL text or arguments. Query spans cover
+// obtaining rows; iteration/Scan remain caller-owned. Use Do around transactions.
+type SQL struct {
+	DB      *sql.DB
+	Runtime *Runtime
+}
+
+func (d SQL) ExecContext(ctx context.Context, query string, args ...any) (result sql.Result, err error) {
+	err = d.Runtime.Do(ctx, "db.exec", func(ctx context.Context) error { result, err = d.DB.ExecContext(ctx, query, args...); return err })
+	return
+}
+func (d SQL) QueryContext(ctx context.Context, query string, args ...any) (rows *sql.Rows, err error) {
+	err = d.Runtime.Do(ctx, "db.query", func(ctx context.Context) error { rows, err = d.DB.QueryContext(ctx, query, args...); return err })
+	return
+}

telemetry/telemetry.go

@@ -0,0 +1,187 @@
+// Package telemetry adds opt-in OpenTelemetry without process-global providers.
+package telemetry
+
+import (
+	"context"
+	"errors"
+	apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"go.opentelemetry.io/otel/attribute"
+	"go.opentelemetry.io/otel/codes"
+	"go.opentelemetry.io/otel/metric"
+	"go.opentelemetry.io/otel/propagation"
+	sdkmetric "go.opentelemetry.io/otel/sdk/metric"
+	"go.opentelemetry.io/otel/sdk/resource"
+	sdktrace "go.opentelemetry.io/otel/sdk/trace"
+	"go.opentelemetry.io/otel/trace"
+	"math"
+	"net/http"
+	"time"
+)
+
+type Config struct {
+	Service     string                `json:"service"`
+	SampleRatio float64               `json:"sampleRatio"`
+	Exporter    sdktrace.SpanExporter `json:"-"`
+	Reader      sdkmetric.Reader      `json:"-"`
+}
+type Runtime struct {
+	traces     *sdktrace.TracerProvider
+	meters     *sdkmetric.MeterProvider
+	tracer     trace.Tracer
+	propagator propagation.TextMapPropagator
+	duration   metric.Float64Histogram
+	requests   metric.Int64Counter
+}
+
+func New(cfg Config) (*Runtime, error) {
+	if cfg.Service == "" || math.IsNaN(cfg.SampleRatio) || cfg.SampleRatio < 0 || cfg.SampleRatio > 1 {
+		return nil, errors.New("telemetry: service and sampleRatio in 0..1 required")
+	}
+	res := resource.NewSchemaless(attribute.String("service.name", cfg.Service))
+	traceOptions := []sdktrace.TracerProviderOption{sdktrace.WithResource(res), sdktrace.WithSampler(sdktrace.ParentBased(sdktrace.TraceIDRatioBased(cfg.SampleRatio)))}
+	if cfg.Exporter != nil {
+		traceOptions = append(traceOptions, sdktrace.WithBatcher(cfg.Exporter, sdktrace.WithMaxQueueSize(512), sdktrace.WithMaxExportBatchSize(128), sdktrace.WithExportTimeout(3*time.Second)))
+	}
+	metricOptions := []sdkmetric.Option{sdkmetric.WithResource(res)}
+	if cfg.Reader != nil {
+		metricOptions = append(metricOptions, sdkmetric.WithReader(cfg.Reader))
+	}
+	r := &Runtime{traces: sdktrace.NewTracerProvider(traceOptions...), meters: sdkmetric.NewMeterProvider(metricOptions...), propagator: propagation.TraceContext{}}
+	r.tracer = r.traces.Tracer("git.yuntuo.hk/k2safe/go-omni")
+	meter := r.meters.Meter("git.yuntuo.hk/k2safe/go-omni")
+	r.duration, _ = meter.Float64Histogram("http.server.request.duration", metric.WithUnit("s"), metric.WithExplicitBucketBoundaries(.005, .01, .025, .05, .1, .25, .5, 1, 2.5, 5, 10))
+	r.requests, _ = meter.Int64Counter("http.server.requests")
+	return r, nil
+}
+func (r *Runtime) Start(ctx context.Context, name string, options ...trace.SpanStartOption) (context.Context, trace.Span) {
+	ctx, span := r.tracer.Start(ctx, name, options...)
+	ctx = logx.WithTraceID(ctx, span.SpanContext().TraceID().String())
+	return ctx, span
+}
+func (r *Runtime) Do(ctx context.Context, name string, work func(context.Context) error) (err error) {
+	if ctx == nil || work == nil || name == "" {
+		return errors.New("telemetry: context, operation name and work required")
+	}
+	ctx, span := r.Start(ctx, name)
+	defer span.End()
+	defer func() {
+		if p := recover(); p != nil {
+			span.SetStatus(codes.Error, "panic")
+			panic(p)
+		}
+	}()
+	err = work(ctx)
+	if err != nil {
+		span.SetStatus(codes.Error, "operation failed")
+	}
+	return err
+}
+func (r *Runtime) Middleware() httpx.Middleware {
+	return func(next httpx.Handler) httpx.Handler {
+		return func(c *httpx.Context) (err error) {
+			parent := c.Context()
+			ctx := r.propagator.Extract(parent, propagation.HeaderCarrier(c.Request().Header))
+			route := httpx.RoutePattern(parent)
+			if route == "" {
+				route = "unmatched"
+			}
+			method := c.Request().Method
+			switch method {
+			case "GET", "POST", "PUT", "PATCH", "DELETE", "HEAD", "OPTIONS", "CONNECT", "TRACE":
+			default:
+				method = "OTHER"
+			}
+			ctx, span := r.Start(ctx, method+" "+route, trace.WithSpanKind(trace.SpanKindServer))
+			c.WithContext(ctx)
+			started := time.Now()
+			defer func() {
+				status := c.Status()
+				if err != nil && !c.Responded() {
+					status = apierrors.From(err).Status
+				}
+				p := recover()
+				if p != nil {
+					status = 500
+				}
+				attrs := []attribute.KeyValue{attribute.String("http.request.method", method), attribute.String("http.route", route), attribute.Int("http.response.status_code", status)}
+				span.SetAttributes(attrs...)
+				if status >= 500 {
+					span.SetStatus(codes.Error, "request failed")
+				}
+				r.requests.Add(ctx, 1, metric.WithAttributes(attrs...))
+				r.duration.Record(ctx, time.Since(started).Seconds(), metric.WithAttributes(attrs...))
+				span.End()
+				// Preserve correlation for outer access-log middleware without
+				// exposing an ended span as its active span.
+				c.WithContext(logx.WithTraceID(parent, span.SpanContext().TraceID().String()))
+				if p != nil {
+					panic(p)
+				}
+			}()
+			return next(c)
+		}
+	}
+}
+func (r *Runtime) Shutdown(ctx context.Context) error {
+	return errors.Join(r.traces.Shutdown(ctx), r.meters.Shutdown(ctx))
+}
+func (r *Runtime) Close() error {
+	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
+	defer cancel()
+	return r.Shutdown(ctx)
+}
+func (r *Runtime) Flush(ctx context.Context) error {
+	return errors.Join(r.traces.ForceFlush(ctx), r.meters.ForceFlush(ctx))
+}
+
+// Transport borrows Base. Headers are injected on a clone, and the span lives
+// until EOF/Close. URL paths, query strings and credentials are never recorded.
+type Transport struct {
+	Runtime *Runtime
+	Base    http.RoundTripper
+}
+
+func (t Transport) RoundTrip(request *http.Request) (*http.Response, error) {
+	if t.Runtime == nil {
+		return nil, errors.New("telemetry: runtime required")
+	}
+	ctx, span := t.Runtime.Start(request.Context(), "http.client", trace.WithSpanKind(trace.SpanKindClient))
+	defer func() {
+		if p := recover(); p != nil {
+			span.SetStatus(codes.Error, "panic")
+			span.End()
+			panic(p)
+		}
+	}()
+	req := request.Clone(ctx)
+	if req.Header == nil {
+		req.Header = make(http.Header)
+	}
+	t.Runtime.propagator.Inject(ctx, propagation.HeaderCarrier(req.Header))
+	base := t.Base
+	if base == nil {
+		base = http.DefaultTransport
+	}
+	response, err := base.RoundTrip(req)
+	if err != nil {
+		span.SetStatus(codes.Error, "transport failed")
+		span.End()
+		if response != nil && response.Body != nil {
+			response.Body.Close()
+		}
+		return nil, err
+	}
+	if response == nil || response.Body == nil {
+		span.SetStatus(codes.Error, "invalid response")
+		span.End()
+		return nil, errors.New("telemetry: invalid transport response")
+	}
+	span.SetAttributes(attribute.Int("http.response.status_code", response.StatusCode))
+	if response.StatusCode >= 500 {
+		span.SetStatus(codes.Error, "upstream failed")
+	}
+	response.Body = &spanBody{ReadCloser: response.Body, span: span}
+	return response, nil
+}

telemetry/telemetry_test.go

@@ -0,0 +1,117 @@
+package telemetry_test
+
+import (
+	"context"
+	omni "git.yuntuo.hk/k2safe/go-omni"
+	"git.yuntuo.hk/k2safe/go-omni/httpx"
+	"git.yuntuo.hk/k2safe/go-omni/logx"
+	"git.yuntuo.hk/k2safe/go-omni/telemetry"
+	"github.com/redis/go-redis/v9"
+	sdkmetric "go.opentelemetry.io/otel/sdk/metric"
+	"go.opentelemetry.io/otel/sdk/metric/metricdata"
+	"go.opentelemetry.io/otel/sdk/trace/tracetest"
+	"go.opentelemetry.io/otel/trace"
+	"io"
+	"net/http"
+	"net/http/httptest"
+	"strings"
+	"testing"
+)
+
+func TestHTTPTracePropagationMetricsAndRedisNil(t *testing.T) {
+	exporter := tracetest.NewInMemoryExporter()
+	reader := sdkmetric.NewManualReader()
+	runtime, err := telemetry.New(telemetry.Config{Service: "test", SampleRatio: 1, Exporter: exporter, Reader: reader})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer runtime.Close()
+	ctx := context.Background()
+	upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		if !strings.HasPrefix(r.Header.Get("Traceparent"), "00-11111111111111111111111111111111-") {
+			t.Error("missing propagated trace")
+		}
+		io.WriteString(w, "ok")
+	}))
+	defer upstream.Close()
+	app := omni.New(omni.WithLogger(false))
+	app.Use(func(next httpx.Handler) httpx.Handler {
+		return func(c *httpx.Context) error {
+			err := next(c)
+			if logx.TraceID(c.Context()) != "11111111111111111111111111111111" {
+				t.Error("outer access log lost trace correlation")
+			}
+			return err
+		}
+	})
+	app.Use(runtime.Middleware())
+	app.GET("/users/:id", func(c *httpx.Context) error {
+		if logx.TraceID(c.Context()) != "11111111111111111111111111111111" {
+			t.Error("log correlation")
+		}
+		req, _ := http.NewRequestWithContext(c.Context(), "GET", upstream.URL+"?secret=hidden", nil)
+		resp, err := (telemetry.Transport{Runtime: runtime}).RoundTrip(req)
+		if err != nil {
+			return err
+		}
+		io.ReadAll(resp.Body)
+		resp.Body.Close()
+		if req.Header.Get("Traceparent") != "" {
+			t.Error("caller headers mutated")
+		}
+		return c.OK("ok")
+	})
+	req := httptest.NewRequest("GET", "/users/privateUser?secret=hidden", nil)
+	req.Header.Set("Traceparent", "00-11111111111111111111111111111111-2222222222222222-01")
+	rec := httptest.NewRecorder()
+	app.ServeHTTP(rec, req)
+	if rec.Code != 200 {
+		t.Fatal(rec.Code)
+	}
+	hook := telemetry.RedisHook{Runtime: runtime}
+	if err = hook.ProcessHook(func(context.Context, redis.Cmder) error { return redis.Nil })(ctx, redis.NewStringCmd(ctx, "GET", "secret")); err != redis.Nil {
+		t.Fatal("cache miss was swallowed", err)
+	}
+	if err = runtime.Flush(ctx); err != nil {
+		t.Fatal(err)
+	}
+	spans := exporter.GetSpans()
+	if len(spans) != 3 {
+		t.Fatalf("got %d spans", len(spans))
+	}
+	var client, server bool
+	for _, s := range spans {
+		if strings.Contains(s.Name, "privateUser") || strings.Contains(s.Name, "secret") {
+			t.Fatal(s.Name)
+		}
+		if s.SpanKind == trace.SpanKindServer {
+			server = true
+			if s.Name != "GET /users/:id" || s.Parent.SpanID().String() != "2222222222222222" {
+				t.Fatal(s.Name, s.Parent)
+			}
+		}
+		if s.SpanKind == trace.SpanKindClient {
+			client = true
+		}
+	}
+	if !client || !server {
+		t.Fatal("missing client/server span")
+	}
+	var metrics metricdata.ResourceMetrics
+	if err = reader.Collect(ctx, &metrics); err != nil {
+		t.Fatal(err)
+	}
+	var count int64
+	for _, scope := range metrics.ScopeMetrics {
+		for _, m := range scope.Metrics {
+			if sum, ok := m.Data.(metricdata.Sum[int64]); ok {
+				for _, p := range sum.DataPoints {
+					count += p.Value
+				}
+			}
+		}
+	}
+	if count != 1 {
+		t.Fatal("request metric", count)
+	}
+}