Compare commits

..

5 Commits

62 changed files with 4517 additions and 396 deletions

16
API.md
View File

@@ -1,5 +1,7 @@
# API 契约与项目生成
首次接入请先看 [README](README.md):包含固定远程版本安装、可运行项目、完整 `.api` 示例和已有服务接入步骤。本页用于查阅语法与生成规则;下方带 `--framework-dir` 的命令用于框架源码开发。
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
## 与 go-zero 的对应关系
@@ -103,7 +105,7 @@ service user-api {
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
暂不支持匿名嵌入、别名、递归类型、外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供,CLI 与业务项目中的框架依赖需一起升级。
@@ -210,11 +212,21 @@ go test ./...
路由总览的 `(undeclared)` 仅表示没有声明 auth;旧式中间件仍可能做认证。
外发文档可用 export;项目内受生成器管理的 `docs/openapi.json` 请通过 generate 更新,以同步文件哈希。
## 迁移扩展(v0.9.0-preview)
- 动态 JSON 字段 `any` / `interface{}`,包含切片和 map;外围契约仍严格。
- `scalar Name (...)` 显式引用项目类型,声明 JSON 输入/输出类型及文本绑定;支持已有 JSON/Text 转换方法。
- `@body (mode: json/raw/multipart; maxBytes: ...)`:按路由限制请求体。
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,raw/multipart 使用这个入口。
- manual 可用 `@produces "text/plain"` 等声明无 DTO 成功输出。默认状态 200,初始手写 stub 返回 501。
完整语法、边界和示例见 [迁移扩展指南](MIGRATION.md)。新增生成代码要求 CLI 与框架依赖一起升级到 `v0.9.0-preview`;不兼容全部 goctl 语法。
## 参数语义
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;非指针拒绝 null;`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;普通非指针字段拒绝 null(动态 `any` 允许);`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。

View File

@@ -1,5 +1,22 @@
# Changelog
## v0.9.1-preview
- 新增标准 `database.OpenConnector` 和独立 `PoolConfig`;与 DSN 入口共享池参数、限时启动 Ping、失败清理、Health、事务,不注册全局驱动配置。
- 新增 `omni.WithHTTPConfig` / `Config.HTTP`:读头、读请求、写响应、空闲超时和头大小,贯通代码、JSON/YAML、环境变量、Run/Serve 与 NewServer;nethttp 支持 Go-only ErrorLog。
- 保持已有默认行为;负值和实验 http2 不支持的超时项在监听前报错;记录各网络层的具体语义。
- 增加真实 MySQL TLS/驱动日志隔离、连接池归属/取消/事务、nethttp/native 慢连接与写背压验收,以及无本地 replace 的远程消费者脚本。
- 兼容 v0.9.0-preview 生成代码;不修改业务项目的数据库策略或生成代码最低版本。
## v0.9.0-preview
- 标准 HTTP Handler/Middleware 桥接:跟踪 Raw 状态与提交,保留取消、流式能力和中间件跨请求状态;已提交响应不追加 JSON。
- `.api` 支持动态 JSON `any` / `interface{}`,保持重复键、大小、外围未知字段校验和数字精度。
- 路由级请求体限额、自定义绑定和 manual Handler;上传、原始 body、下载可保留手写实现,重生成受保护。
- 自定义 scalar 显式对接项目 JSON/Text 方法;支持可空和集合,分别生成请求/响应 OpenAPI;不放宽其他字段。
- 新增 [迁移扩展指南](MIGRATION.md)、可运行示例和真实生成项目回归。CLI 与框架依赖一起升级。
## v0.8.0-preview
- 公共响应 `codeType` 支持 `int32` / `int64` / `string`,校验默认码及映射范围,OpenAPI 标注整数位宽;JSON/YAML 和生成代码保留 int64 精度,旧 `integer` 兼容。

182
MIGRATION.md Normal file
View File

@@ -0,0 +1,182 @@
# 已有 HTTP 项目逐步接入
从 `v0.9.0-preview` 开始,以下能力贯通运行时、`.api`、代码生成和 OpenAPI。CLI 和项目框架依赖一起升级;入口见 [README](README.md)。这些都是通用扩展点,框架不内置任何项目的 ID、支付回调、鉴权或数据库规则。
## 1. 保留标准 Handler 与中间件
```go
app := omni.New(omni.WithLogger(true))
app.Use(httpx.WrapMiddleware(existingMiddleware)) // func(http.Handler) http.Handler
// go-zero 风格的 func(http.HandlerFunc) http.HandlerFunc:
app.Use(httpx.WrapFuncMiddleware(existingFuncMiddleware))
app.POST("/legacy", httpx.WrapHandler(http.HandlerFunc(existingHandler)))
```
实际使用时先配置所有 `Use`,再注册路由。`WrapHandler` 也接受 `http.FileServer` 等标准 `http.Handler`。
- Handler 经过 App 的日志、requestId、恢复和应用中间件链,原来的 URL、状态码、Header、响应内容由它自己控制。
- `c.Raw()` 的写入也会更新 `c.Status()` / `c.Responded()`;404 会记为 404。提交响应后返回错误或 panic,不会再追加错误 JSON。103 等临时响应不算最终提交。
- 桥接器不缓存完整响应。下载继续流式复制;底层支持时保留 Flusher、Hijacker、Pusher、ReaderFrom、CloseNotifier。`http.ResponseController` 可沿 `Unwrap` 访问底层控制能力。能力仍取决于网络适配器和其他中间件。
- 请求 context、取消信号和已有值继续传递,Omni 的路径参数可用 Go 1.22 的 `r.PathValue("id")` 读取。其他框架私有的参数读取器仍需由项目适配。标准中间件应从原 `r.Context()` 派生上下文;不能换成无关的 Background,也不能在 ServeHTTP 返回后调用 next。异步 timeout middleware 不在此桥接器的契约内。
- 标准中间件在每条注册链构造一次,状态会跨请求保留。下游返回的错误在标准中间件退出前编码,因此压缩、审计等中间件能处理错误响应。它们自己的缓冲或改写行为仍由项目负责。
调用流使用 `App.Handle/GET/POST` 等入口即可,不要绕到原始 Router 注册。暂不使用 `.api` 的旧接口也能这样接入。
## 2. 动态 JSON:允许渐进补全 DTO
```go
type LegacyReply {
Data any `json:"data"`
}
```
`interface{}` 是 `any` 的等价写法,支持 `[]any`、`map[string]any` 和对应的 `interface{}` 写法。带方法的 interface 不支持,会提示明确的类型错误。
动态字段允许 object、array、string、number、bool、null;OpenAPI 字段 schema 为 `{}`,不假装知道内部结构。它放开的是**该字段的内容**,外围 DTO 未声明的字段仍被拒绝,JSON 重复键、深度、总大小限制仍生效。动态数字解析为 `json.Number`,避免大整数先转 float64 丢精度。原始 JSON 的空白、转义和字节布局不会保留;验签请走原始 body 入口。
这不会自动补齐缺少的业务路由,也不代表兼容全部 goctl 语法。能确定结构的新接口优先使用命名 DTO。
## 3. 按路由限制 body 与保留手写入口
普通 JSON 路由只声明需要的大小,不改变其他路由默认的 1 MiB:
```text
@body (mode: json
maxBytes: 2097152
)
@handler createItem
post /items (CreateItemReq) returns (ItemResp)
```
`maxBytes` 单位为字节,允许 1 至 1 TiB;不写时为 1 MiB。已知超长或读取未知长度/chunked body 超限都按 413 处理。内置 App/RouterGroup 在全局中间件读取前安装 reader 限额;同时保持接口响应策略。网络层若有更小限额,以更小值为准。手写 Handler 吞掉读取错误或已经提交响应后,框架不能重新改写 HTTP 状态,务必检查读取错误。
### 只替换绑定函数
```text
@binding custom
@body (mode: json
maxBytes: 65536
)
@handler createItem
post /items (CreateItemReq) returns (ItemResp)
```
首次生成 `internal/handler/<group>/createItemBind.go`:
```go
func (h *Handler) bindCreateItem(c *httpx.Context, req *types.CreateItemReq) error {
// 调用项目自己的解码器,填充 req;失败返回 error。
}
```
Handler 继续调用 Service 并使用统一响应;只把绑定过程交给这个函数。初始 hook 返回 501,不能忽略实现。自定义绑定需自己实现 Content-Type、未知字段、尾部数据等校验,并遵守 DTO 文档;框架负责路由总大小限制,不隐式套第二次 JSON 绑定。
### 上传、原文验签、下载
```text
@handlerMode manual
@body (mode: multipart
maxBytes: 10485760
)
@handler upload
post /files
@handlerMode manual
@body (mode: raw
maxBytes: 65536
)
@produces "text/plain"
@handler callback
post /callback
@handlerMode manual
@produces "application/octet-stream"
@handler download
get /download
```
首次生成 `<operation>Manual.go`,由项目实现整个 `func(h *Handler) Callback(c *httpx.Context) error`;依赖通过 `h.deps` 访问。生成器保留路由、中间件与文档管理,不会预读、JSON 解码或重新包装手写输出,也不生成这个路由的 Service 方法。需要调用 Service 时自行组合。
- manual 无返回类型默认成功状态 200;也可 `returns (UploadResp)` 描述 JSON DTO,手写实现按声明调用 `c.OK` 等。`@produces` 用于无 DTO 的手写输出,描述成功响应媒体类型,不会自动设置运行时 Header。
- raw 的文档为二进制请求体;multipart 为开放对象,具体文件字段与逐部分校验由项目实现。raw/multipart 请求 DTO 仅可含 path/form/header 字段,不能同时声明 JSON body 字段。`form` 始终指 URL query。
- 原文验签直接读取 `c.Request().Body`,在声明的小限额内 `io.ReadAll`;大文件用 `MultipartReader` 配合 `io.Copy` 流式处理,不能整个读入内存。
- 使用 `ParseMultipartForm` 时,其内存参数只是内存阈值,不是总上传限额。App/桥接器在请求结束(含错误和 panic)后删除临时文件;业务仍须关闭 `FormFile` 得到的文件。`MultipartReader` 不自动建临时文件,业务自行创建的文件由业务清理。
- 手写 Handler 的成功输出由它控制;返回的 error、鉴权拒绝仍走接口的错误策略。三方若要求错误也返回指定文本,使用 [自定义 Encoder](RESPONSE.md#手写路由配置文件与三方回调) 并维护对应错误文档;`@produces` 不会改变错误协议。
直接注册 Go 路由:
```go
app.HandleWithBodyLimit("POST", "/upload", 10<<20, nil, uploadHandler)
// nil encoder 沿用 App 策略,也可传项目自己的 response.Encoder。
```
通用注册器使用 `httpx.HandleWithBodyLimit(...)`;实现 `httpx.BodyLimitRegistrar` 可像 App 一样在全局中间件前限流量,未实现时只能在路由中间件入口限制。单独 `httpx.LimitBody` 同样受中间件顺序影响。
### 重新生成规则
`handlersGen.go`、`routesGen.go`、DTO、OpenAPI 继续由生成器管理;`*Bind.go`、`*Manual.go` 只首次创建,后续不会覆盖或删除。切换 manual 回自动模式时,先迁移实现并自行移除对应 Manual 文件,否则生成器明确拒绝,避免重复方法。修改 DTO 导致手写函数签名不匹配时,需要一起调整项目代码;`go test ./...` 是必要检查。
## 4. 自定义标量:项目定义行为,框架连接调用
例如输入 ID 兼容数字/字符串、输出统一为字符串:
```text
scalar BusinessID (
goImport: "example.com/billing/internal/ids"
goType: ID
jsonInput: string | int64
jsonOutput: string
text: true
)
type ItemReq {
ID BusinessID `path:"id"`
}
type ItemResp {
ID BusinessID `json:"id"`
}
```
`goImport`/`goType` 指向项目已有的具体 Go 类型,不是框架内置 ID。生成器创建类型别名和 `types.ContractScalars()`,然后把注册表显式传给每个 Binder。运行时使用标准接口:
| 项目类型的方法 | 用途 |
| --- | --- |
| 值接收者 `MarshalJSON() ([]byte, error)` | JSON 输出 |
| 指针接收者 `UnmarshalJSON([]byte) error` | JSON 输入 |
| 指针接收者 `UnmarshalText([]byte) error` | `text: true` 时的 path/query/header |
完整参考 [示例 ID 类型](examples/compatibility/ids/id.go)。CLI 校验声明并生成代码,不执行项目 Go 代码;启动时校验方法是否齐全,缺失会明确失败。
输入可选 `string`、`bool`、`int32`、`int64`、`float64`,用 `|` 声明多个允许的 JSON 类型;输出只声明一个。框架先校验 JSON 类型和数字范围,再调用项目转换方法。字符串是否是合法 ID、是否允许空串或负数由项目方法决定,不能把数字溢出静默截断。path/query/header 实际传输文本,OpenAPI 对这些位置描述 string;query 切片通过重复键传递。
普通字段不受注册影响,普通 `int64` 仍拒绝 JSON 字符串。没有全局注册表,也不会自动给所有 ID 放宽规则。标量可嵌套、放入切片/map,`*BusinessID` 允许 null;缺失默认必填,`optional`/`omitempty` 可选。标量的额外约束放在转换器中,目前不叠加 `binding`、default/options/range 等标签。
请求 OpenAPI 按 `string | int64` 生成 `anyOf`,响应按 `string` 生成,避免把输入兼容误描述为输出也有两种类型。输出方法必须遵守声明,框架不会每次重新验证响应;项目需为自定义类型测试边界行为。
不使用 `.api` 也可显式注册:
```go
binder := contract.MustNew[CreateReq](
contract.ScalarOf[ids.ID]("string|int64", "string", true),
)
// binder.Bind(c.Request(), httpx.RouteParams(c.Context()), &req)
```
## 可运行示例与验收
```sh
go run ./examples/compatibility
curl -i http://127.0.0.1:8080/legacy
curl -i -X POST http://127.0.0.1:8080/echo \
-H 'Content-Type: application/json' -d '{"id":9223372036854775807,"data":{"hello":true}}'
curl -i -F file=@README.md http://127.0.0.1:8080/upload
curl -OJ http://127.0.0.1:8080/download
```
完整契约:[compatibility.api](examples/contracts/compatibility.api)。生成项目起初的手写 hook 仍为 501;真实实现与原文 HMAC 验签回归见 [消费者 fixture](cmd/go-omni/apigen/testdata/compatibility)。
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/apigen -run Compatibility
go test -race ./httpx ./contract
```
这些是框架验收,不代表任何业务项目已经迁移完毕;业务的路由覆盖、响应兼容、权限、资金事务和数据库行为需要业务自己的回归。

552
README.md
View File

@@ -1,316 +1,400 @@
# GoOmni
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用只依赖标准库的网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Service 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
**从这份 README 开始即可接入。** 当前版本为 **`v0.9.1-preview`**,Go **1.22+**;CLI 和项目依赖使用同一固定版本。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
## 选择接入方式
| 你的项目 | 从哪里开始 |
| --- | --- |
| 新建服务,先定义接口再开发 | [新项目:生成并运行](#新项目生成并运行) |
| 已有服务,逐步接入 HTTP 或基础模块 | [已有项目:按模块接入](#已有项目按模块接入) |
| 已有标准 Handler、动态 JSON、上传或自定义 ID 类型 | [迁移扩展指南](MIGRATION.md) |
| 想统一返回体、业务错误或三方回调 | [配置公共响应与独立协议](#配置公共响应与独立协议) |
| 要接日志、数据库、鉴权和存储 | [业务开发与依赖装配](#业务开发与依赖装配) |
| 要切换网络层、配置 TLS 和退出清理 | [配置与运行](#配置与运行) |
| 已接入,需要升级和检查生成代码 | [日常开发与升级](#日常开发与升级) |
## 准备环境与安装
module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可用,并具有仓库读取权限;使用自己的 Git 凭据管理器,不把账号或 Token 写进 module 地址。
以下命令在同一个终端执行。已有 `GOPRIVATE` 时,把 `git.yuntuo.hk` 追加到原来的逗号分隔列表。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.8.0-preview
export GOPRIVATE=git.yuntuo.hk
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
如果配置了自定义 `GOBIN`,将那个目录加入 PATH。只引用框架包、不生成项目时可以跳过 CLI 安装,直接使用下文的 `go get`。
创建新业务项目:
## 新项目:生成并运行
在准备存放业务项目的目录执行,`./hello` 必须是新目录或空目录;把 `example.com/hello` 换成自己的 Go module 名称。
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.8.0-preview
go-omni new --module example.com/billing ./billing
cd billing
go-omni api new --framework-version v0.9.1-preview \
--module example.com/hello ./hello
cd hello
go mod tidy
go run ./cmd/server
```
远程安装的 CLI 默认引用自身版本,生成的项目不需要本地 `replace`。
具体版本和本地开发选项见 [CLI 文档](cmd/go-omni/README.md)。
需要数据库持久化时,使用 `go-omni new --database mysql --module example.com/billing ./billing`。
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
## API 定义与项目生成
当前源码新增 **API 定义驱动项目生成**:写 `.api` 定义请求、响应和路由,自动生成项目目录、DTO、Handler、Service 骨架及 OpenAPI。重复生成保留业务代码,支持只读 CI 检查,完整语法和示例见 [API 开发指南](API.md)。
另开终端验证:
```sh
go run ./cmd/go-omni api new --framework-dir . \
--module example.com/omniDemo ../omniDemo
curl -i http://127.0.0.1:8080/api/greet/Alice
curl -i http://127.0.0.1:8080/health/ready
curl -sS http://127.0.0.1:8080/openapi.json
```
API 生成支持 `--database mysql --storage local --log-output file` 组合初始化。上面的命令使用本地源码;远程安装见 [发布说明](RELEASE.md),完整业务案例见 [userService](examples/userService/README.md)。
第一个接口返回 HTTP 200,业务数据为 `{"message":"Hello Alice"}`。初始模板未配置响应格式,使用默认公共体:
## 目录结构
```json
{"code":"OK","message":"success","data":{"message":"Hello Alice"},"requestId":"本次请求编号"}
```
按 Ctrl+C 停止服务。项目依赖已固定为远程版本,不需要克隆框架源码或添加本地 `replace`。
### 生成的项目放什么
```text
go-omni/
├── app/ # 应用启动和生命周期
├── httpx/ # 与网络框架无关的 HTTP 抽象
├── response/ # 可配置公共体和响应编码器
├── errors/ # 统一错误
├── health/ # 存活和就绪检查
├── middleware/ # 请求 ID 等基础中间件
├── logx/ # 业务层结构化日志
├── metrics/ # 请求指标与状态统计
├── config/ # 配置管理
├── validate/ # 参数校验
├── contract/ # API 字段契约、严格多源绑定与文档约束
├── auth/ # 认证和权限接口
├── database/ # database/sql 生命周期和事务
├── cache/ # 有界内存缓存、Redis 适配与合并回源
├── httpc/ # 出站 HTTP 连接池、超时和重试
├── task/ # 有界后台任务与间隔调度
├── queue/ # 持久队列接口与 Redis Streams 适配
├── storage/ # Local、S3/R2、OSS 与临时下载链接
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
├── telemetry/ # 可注入 OpenTelemetry 观测
├── resilience/ # 熔断与上游并发隔离
├── audit/ # 显式业务审计与可注入 Sink
├── pagination/ # 有界分页和统一结果
├── transport/nethttp/ # 标准库网络适配器
├── transport/native/ # 自研 HTTP/1.1 服务端预览
├── transport/http2/ # opt-in HTTP/2 实验适配器
├── examples/ # 示例项目
└── cmd/ # go-omni 项目脚手架
hello/
├── api/service.api # 维护接口定义,也可 import 其他 .api
├── cmd/server/main.go # 启动、路由注册、优雅退出
├── etc/config.yaml # 项目配置
├── internal/
│ ├── settings/ # 配置读取
│ ├── servicecontext/ # 数据库、日志、外部客户端等依赖
│ ├── types/typesGen.go # 生成的请求与业务返回类型
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
│ ├── service/greet/ # 在这里实现业务逻辑
│ ├── repository/greet/ # 在这里实现数据访问
│ └── middleware/ # 声明鉴权等中间件后生成对应 hook
├── docs/openapi.json # 生成的 OpenAPI 3.0.3
└── .goOmni/apiManifest.json # 生成文件所有权与校验信息,一起提交 Git
```
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
可以直接运行 [组合示例](examples/modules/README.md)。这些能力从 `v0.7.0-preview` 提供。
修改 `api/service.api` 后重新生成,业务写在 `internal/service`,数据访问写在 `internal/repository`,依赖在 `internal/servicecontext` 装配。声明业务错误后还会生成 `internal/apierrors/errorsGen.go`。
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
`api new` 的问候接口可以直接运行;自定义接口新生成的 Service 和鉴权 hook 返回 **501**,需要实现后才可用。生成器不会根据 HTTP 字段猜数据库表或业务规则。
## 快速开始
### 从自己的接口定义生成
保存下面的完整示例为 `catalog.api`,再执行后面的生成命令。这里的响应格式、错误码和提示都由这个业务项目定义。
```text
syntax = "v1"
response CommonResponse (
body: {"code":"${code}", "msg":"${message}", "data":"${data}"}
codeType: int32
successCode: 200
errorCode: 9999
successMessage: "操作成功"
dataMode: always
codes: {"ITEM_NOT_FOUND":404001}
)
error ItemNotFound 404 "ITEM_NOT_FOUND" "记录不存在"
type GetItemReq {
ItemID string `path:"itemId" binding:"required"`
}
type ItemResp {
ItemID string `json:"itemId"`
Name string `json:"name"`
}
@server (
group: item
prefix: /api/v1
response: CommonResponse
)
service catalog-api {
@doc "查询商品"
@errors ItemNotFound
@handler getItem
get /items/:itemId (GetItemReq) returns (ItemResp)
}
```
```sh
go-omni api validate --api catalog.api
go-omni new --api catalog.api --framework-version v0.9.1-preview \
--module example.com/catalog ./catalog
cd catalog
go mod tidy
```
将生成的 `internal/service/item/getItem.go` 替换为下面的演示实现。实际业务中换成 Repository 查询,Service 返回业务数据或错误,不再手工包装公共体:
```go
package item
import (
"context"
"example.com/catalog/internal/apierrors"
"example.com/catalog/internal/types"
)
func (s *Service) GetItem(ctx context.Context, req *types.GetItemReq) (*types.ItemResp, error) {
if req.ItemID != "1001" {
return nil, apierrors.ItemNotFound()
}
return &types.ItemResp{ItemID: req.ItemID, Name: "示例商品"}, nil
}
```
示例 import 对应上面的 `example.com/catalog`,使用其他 module 名称时同步替换。启动后,在另一个终端验证:
```sh
# catalog 项目目录中启动;先停掉前面的 hello 服务,释放 8080 端口。
go run ./cmd/server
# 另一个终端:
curl -i http://127.0.0.1:8080/api/v1/items/1001
curl -i http://127.0.0.1:8080/api/v1/items/missing
```
成功时得到 `{"code":200,"msg":"操作成功","data":{"itemId":"1001","name":"示例商品"}}`;不存在时为 HTTP 404、`{"code":404001,"msg":"记录不存在","data":null}`。
需要 MySQL、文件存储或文件日志时,**首次生成**可以使用:
```sh
go-omni new --api catalog.api --framework-version v0.9.1-preview \
--module example.com/catalogFull --database mysql --storage local \
--log-output file --log-format json ./catalogFull
```
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL DSN 通过 `GO_OMNI_DB_DSN` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整装配例子见 [userService](examples/userService/README.md)。
不采用 `.api` 时,也可以用 `go-omni new --framework-version v0.9.1-preview --module example.com/billing ./billing` 创建手写接口的 CRUD 示例项目;两种生成方式选择一种即可。
## 已有项目:按模块接入
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.9.1-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。
下面是可独立运行的 `main.go` 示例;迁入现有项目时,把路由调用接到自己的业务服务即可。
```go
package main
import omni "git.yuntuo.hk/k2safe/go-omni"
func main() {
app := omni.New()
app.GET("/health", func(ctx *omni.Context) error {
return ctx.OK(map[string]string{"status": "ok"})
})
app.Run(":8080")
}
```
成功响应默认使用统一结构:
```json
{
"code": "OK",
"message": "success",
"data": {"status": "ok"},
"requestId": "..."
}
```
业务错误可以直接返回:
```go
return omni.NotFound("USER_NOT_FOUND", "user was not found")
```
默认网络适配器位于 `transport/nethttp`。业务层不需要导入具体网络库;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
```go
application := omni.NewWithRouter(myRouter)
```
仓库还提供了使用标准库路由的 `transport/nethttp` 参考实现。两种适配器已通过同一套路由、业务处理链和取消传播测试,完整用户 CRUD 复用同一份 Handler / Service。
自研适配器可以直接运行 `transport/transporttest.Run` 验收。具体规则、v0.4.0 的斜杠 / Any / 405 行为调整,以及自研网络内核与当前 HTTP 接口的边界,见 [适配层契约](transport/README.md)。
`app.Engine()` 返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,业务 API 不锁定具体网络实现。
需要试用第一版自研网络层时,业务注册代码无需改变,只替换启动器:
```go
import (
"context"
"log"
"os"
"os/signal"
"syscall"
"log"
"net/http"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/transport/native"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel()
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
app := omni.New()
app.GET("/health", func(ctx *omni.Context) error {
return ctx.OK(map[string]string{"status": "ok"})
})
if err := app.RunWith(ctx, native.New(nil), ":8080"); err != nil {
log.Fatal(err)
func run() error {
closeLog, err := logx.Setup(logx.Config{
Service: "catalog", Level: "info", Format: "json", Output: "stdout",
})
if err != nil {
return err
}
defer closeLog()
app := omni.New(
omni.WithLogger(true),
omni.WithResponse(response.Config{
Body: map[string]any{
"code": "${code}", "msg": "${message}", "data": "${data}",
},
CodeType: "int32", SuccessCode: 200, ErrorCode: 9999,
SuccessMessage: "操作成功", DataMode: "always",
Codes: map[string]any{"ITEM_NOT_FOUND": 404001},
}),
)
app.GET("/api/v1/items/:itemId", func(c *omni.Context) error {
itemID := c.Param("itemId")
logx.InfoContext(c.Context(), "item.get", "itemId", itemID)
if itemID != "1001" {
return omni.NotFound("ITEM_NOT_FOUND", "记录不存在")
}
return c.OK(map[string]string{"itemId": itemID, "name": "示例商品"})
})
// 第三方协议可明确输出文本,不经过公共 JSON 包装。
app.GET("/ping", func(c *omni.Context) error {
return c.Text(http.StatusOK, "pong")
})
return app.Run("127.0.0.1:8080")
}
```
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 和 chunked 请求体、`http.Flusher` 流式响应、可选 TLS、读写超时和优雅关闭;不支持的传输编码会被拒绝。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
运行 `go run .`,访问 `/api/v1/items/1001`、`/api/v1/items/missing` 和 `/ping`,可分别验证成功、业务错误及纯文本输出。`App` 实现标准 `http.Handler`,也可挂进现有 `http.Server` 或父路由;这时监听、TLS、信号和资源关闭由现有入口负责,结束时调用 `app.Close()`。
native TLS 当前只支持并宣告 ALPN `http/1.1`;配置 `h2` 会在启动时快速失败。需要 HTTP/2 时,可由 Nginx/Cloudflare 终止后回源 HTTP/1.1,或使用默认的 `transport/nethttp` 适配器。
现有 `http.HandlerFunc` 可直接通过 `app.POST("/legacy", httpx.WrapHandler(http.HandlerFunc(oldHandler)))` 接入;状态码与访问日志一致,下载和流式输出保持原协议。标准中间件、动态 JSON、上传/原文 body、手写绑定和自定义类型的完整用法见 [迁移扩展指南](MIGRATION.md)。
`transport/http2` 是独立的 opt-in 实验适配器,使用 `x/net/http2` 处理帧和流控,同时保留 GoOmni 的连接上限、全局请求并发闸门和生命周期边界。它不会改变 native 默认路径;实验入口、限制和验收结果见 [HTTP/2 进入条件](transport/native/HTTP2.md)。
接入现有业务时,逐项确认:
自研网络层还提供 `server.Stats()`,可读取 accepted、rejected、active connections 和 active requests,与上面的业务请求 metrics 组合后能区分连接堆积、过载拒绝和 Handler 慢请求。`rejectedConnections` 表示连接闸门拒绝数,`rejectedRequests` 表示并发请求闸门返回 503 的次数。
- **协议**:保留既有 URL、HTTP 状态、字段、分页和空值语义。已有业务错误类型需要在项目内转换为 Omni 错误,或使用 `WithResponseEncoder` 适配原错误;设置字段模板不会自动识别其他库的错误。
- **鉴权**:把现有登录态和权限校验接入中间件,继续使用原来的授权范围和业务规则。原来依赖 `gin.Context` 等类型的 HTTP 代码需要适配;标准 `context.Context` 的业务服务可以继续复用。
- **数据类型**:ID、金额、日期等按项目协议转换。用 `.api` 的 `scalar` 显式关联项目自己的类型与 JSON/Text 转换方法,分别声明输入和输出协议;普通字段保持严格绑定,见 [自定义标量](MIGRATION.md#4-自定义标量项目定义行为框架连接调用)。
- **生成器**:它不会自动接管任意已有 Go 项目。希望引入 `.api` 时,先在独立新目录生成对照,再规划路由与生成目录的接入,不要用脚手架覆盖现有业务目录。
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖具体网络库:
先通过一个接口的成功、参数错误、未登录、无权限及业务错误回归,再按模块迁移。框架不要求所有项目使用同一套业务目录或身份模型。
```go
func (s *UserService) Create(ctx *httpx.Context) error {
logx.InfoContext(ctx.Context(), "user.create", "email", input.Email)
return nil
}
```
## 配置公共响应与独立协议
需要基础运行指标时,可以注册 transport-neutral metrics middleware 和接口:
```go
requestMetrics := omni.NewMetrics()
app.Use(requestMetrics.Middleware())
app.GET("/metrics", requestMetrics.Handler())
```
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。文件轮转、敏感字段脱敏、异常堆栈与慢请求配置见 [logx 文档](logx/README.md);文件直传与云端分片上传见 [storage 文档](storage/README.md)。 创建新业务项目时可通过 `go-omni new --log-output file --storage local` 按需生成配置、装配代码与可运行示例,当前源码使用方式见 [CLI 文档](cmd/go-omni/README.md)。
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
```go
settings, err := omni.LoadConfig("")
if err != nil {
log.Fatal(err)
}
logger, err := logx.New(settings.Log)
if err != nil {
log.Fatal(err)
}
application := omni.New(omni.WithConfig(settings))
application.OnShutdown(logger.Close)
```
常用变量包括 `GO_OMNI_ENABLE_LOGGER`、`GO_OMNI_LOG_LEVEL`、
`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_OUTPUT`、`GO_OMNI_SERVICE`、
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR`、`GO_OMNI_SHUTDOWN_TIMEOUT` 和
`GO_OMNI_TRUSTED_PROXIES`。可信代理变量使用逗号分隔的 IP/CIDR,例如
`GO_OMNI_TRUSTED_PROXIES=127.0.0.1,10.0.0.0/8`;只有来自这些地址的连接才会解析
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
| 配置 | 默认值 / 用途 |
| 接口来源或需求 | 配置位置 |
| --- | --- |
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
| `.api` 生成的接口 | `response Name (...)` 定义,`@server (response: Name)` 引用 |
| 多个 `.api` 共用返回体 | 将声明放到 `common.api`,使用 `import "common.api"` |
| 某个生成接口用另一种 JSON 公共体 | 路由声明 `@response OtherResponse` |
| 只要业务 JSON、无成功包装 | 路由声明 `@response raw`;失败仍沿用组级格式 |
| 手写路由和应用级 404/405 | `omni.WithResponse(response.Config{...})`,或读取配置后传入 `WithConfig` |
| 成功和失败都要遵循特殊回调协议 | `HandleWithResponse` 绑定项目自己的 `response.Encoder` |
| 单次直接输出 JSON、文本、XML、字节或空响应 | `c.JSON` / `c.Text` / `c.XML` / `c.Bytes` / `c.NoContent` |
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
`${data}` 是业务返回值,`${message}` 是公开提示,`${code}` 是业务码;占位符保留原值类型,不是字符串拼接。还支持 `${requestId}`、`${ok}`、`${status}`,字段名和嵌套结构由项目决定。
```sh
GO_OMNI_TRANSPORT=native go run ./examples/reference
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
go run ./examples/reference
```
- `codeType` 可以是 `string`、`int32`、`int64`;整数类型需要配置 `successCode` 和 `errorCode`。
- `codes` 把逻辑错误码映射成对外业务码;未单独映射的失败使用 `errorCode`。JSON 业务码不会改变 HTTP 状态,例如 `404001` 仍对应 HTTP 404。
- `successMessage` 设置成功提示;未设置 `errorMessage` 时沿用错误的公开提示。`messages` 可按逻辑错误码单独覆盖,优先级最高。全局 `errorMessage` 会替换未单独映射的所有失败提示,只有需要统一失败文案时才设置。
- `dataMode: always` 在无数据时输出 null;默认 `omitNil` 省略模板里相应对象字段。没有 `returns` 的 `.api` 路由默认 HTTP 204、无响应体,这与 `200 + data:null` 是不同的接口契约。
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
**生成接口的格式以 `.api` 为准**,修改后重新生成,运行时与 OpenAPI 一起更新;应用 YAML 不会覆盖它。404/405 没有匹配到某个 `.api` 接口,使用应用默认配置,需要统一时也配置应用默认值。
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
文本/XML 回调用 Go 编码器或直接输出,`.api` 可用 `@handlerMode manual` 与 `@produces` 声明成功媒体类型;特殊错误协议仍需项目维护。`@response raw` 仅表示原始 JSON。前面示例的 `c.Text` 控制这次正常返回;需要连鉴权失败、业务错误和 panic 都按回调协议响应时使用 `HandleWithResponse`。完整配置和回调示例见 [响应参考](RESPONSE.md)。
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
## 业务开发与依赖装配
应用资源可以直接挂到 `omni.New` 返回的生命周期上:
### 日志
业务包直接使用 `logx`,不需要 HTTP 框架对象:
```go
application := omni.New()
application.OnShutdown(db.Close)
// ctx 是 Service 接收的标准 context.Context。
logx.InfoContext(ctx, "order.created", "orderId", orderID)
logx.ErrorContext(ctx, "order.failed", "error", err)
```
清理函数会按注册顺序的逆序执行,`application.Close()` 和 `RunContext` 都会触发,且重复调用安全。
入口用 `logx.Setup` 初始化,退出时关闭;生成的 API 项目已经完成日志初始化和退出清理。Service、Repository 和外部请求持续传递请求的 `context.Context`,业务日志可关联 `requestId`,同时保留取消信号。框架不规定或自动注入 `userId`、`tenantId`。
服务可以把数据库、缓存等依赖注册为就绪探针:
`WithLogger(true)` 开启 HTTP 访问日志。格式、级别、文件轮转、脱敏键和慢请求阈值均可配置,见 [日志](logx/README.md)。
```go
checker, err := health.New(health.Probe{
Name: "database",
Check: db.PingContext,
})
if err != nil {
log.Fatal(err)
}
application.GET("/health/live", checker.LivenessHandler())
application.GET("/health/ready", checker.ReadinessHandler())
### 鉴权与业务错误
`.api` 可声明认证方式和关联中间件,例如把下面的声明放入契约,并在相应 `@server` 块中加上 `auth: UserAuth`:
```text
auth UserAuth (
type: bearer
middleware: Auth
)
```
存活检查只确认进程在线;就绪检查会执行全部探针,任意依赖失败时返回 HTTP 503。
生成器会创建 `internal/middleware/auth.go`,由项目实现凭据校验和身份注入。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的 hook 返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
新项目脚手架会生成项目自己的 `internal/servicecontext`。它集中保存 Logger、数据库、缓存和外部客户端,Service 只依赖这个项目上下文,不把基础设施初始化散落到各个 Handler:
业务错误在 `.api` 中用 `error` 声明,通过 `@errors` 关联路由后,生成可调用的错误函数和 OpenAPI 中对应的 401、403、409 等响应。手写路由可以直接返回 `omni.NotFound`、`omni.Forbidden` 等错误。已有项目在自己的适配层转换原错误,内部原因和公开提示分开处理。
```go
services, err := servicecontext.New(ctx, logger)
if err != nil {
return err
}
defer services.Close()
userService := service.NewUsers(services)
```
### 数据库、存储及其他模块
## 完整业务参考项目
依赖在启动入口或项目自己的 `internal/servicecontext` 中创建,通过构造函数传给 Service/Repository,避免每个请求都新建连接池。只注册自己拥有的资源,例如 `app.OnShutdown(db.Close)`;关闭按注册的逆序执行。
[`examples/reference`](examples/reference/README.md) 提供按 Handler / Service / Repository / Model 分层的用户 CRUD 示例。默认使用并发安全的内存存储,支持分页、邮箱唯一校验、结构化请求日志和统一错误,无需配置数据库。
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 等驱动的独立 TLS、Logger、时区与拨号参数由项目构造 Connector;框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
从远程版本生成并升级一个独立业务模块,可运行:
| 能力 | 包与接入说明 |
| --- | --- |
| 数据库连接池、事务、显式迁移 | `database`,见 [数据库](database/README.md);表结构与驱动由项目选择 |
| 本地文件、S3、R2、OSS、直传与临时下载 | `storage`,见 [文件存储](storage/README.md);业务依赖接口,装配时选择适配器 |
| 缓存、出站 HTTP、限流、幂等、后台任务 | `cache`、`httpc`、`middleware`、`task`,见 [基础模块](MODULES.md) |
| JWT/Session、策略、Redis 队列、审计、分页、观测 | 见 [扩展模块](ADVANCED.md);按需注入实现 |
| 存活与就绪检查 | `health`,见 [健康检查](health/README.md);数据库等依赖作为 readiness 探针 |
API 项目的数据库迁移需要单独维护并显式执行;启动不会自动改表。完整示例 [userService](examples/userService/README.md) 展示 MySQL、事务、鉴权、日志和文件导出,可参考依赖装配方式后接入自己的业务规则。
## 配置与运行
生成项目从 `etc/config.yaml` 读取配置,优先级为 **默认值 → 配置文件 → 环境变量**。密钥通过环境变量提供。手写入口需显式调用配置加载函数并传给 `omni.New(omni.WithConfig(settings))`;单独调用 `omni.New()` 不会自动读取环境变量。
| 环境变量 | 用途 |
| --- | --- |
| `GO_OMNI_HTTP_ADDR` | 监听地址;生成项目默认 `127.0.0.1:8080` |
| `GO_OMNI_TRANSPORT` | `nethttp`(默认)、`native`、`http2` |
| `GO_OMNI_TLS_CERT_FILE` / `GO_OMNI_TLS_KEY_FILE` | 成对设置 TLS 证书和私钥路径 |
| `GO_OMNI_HTTP_READ_HEADER_TIMEOUT` / `GO_OMNI_HTTP_READ_TIMEOUT` | 请求头 / 请求读取超时,如 `5s` / `30s` |
| `GO_OMNI_HTTP_WRITE_TIMEOUT` / `GO_OMNI_HTTP_IDLE_TIMEOUT` | 响应写入 / Keep-Alive 空闲超时 |
| `GO_OMNI_HTTP_MAX_HEADER_BYTES` | 请求头大小限制,字节数 |
| `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间,默认 `10s` |
| `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 |
| `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR,逗号分隔 |
| `GO_OMNI_DB_DSN` | 选择 MySQL 初始化时的数据库连接配置 |
例如在生成项目中切换 native:
```sh
python3 scripts/verifyBusiness.py --version v0.6.0-preview
GO_OMNI_TRANSPORT=native go run ./cmd/server
```
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.6.0-preview`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
手写入口可用 `omni.WithTransport(omni.TransportNative)`;TLS 配置为 `omni.WithTLS(certFile, keyFile)`。native 提供 HTTP/1.1、Keep-Alive、TLS、超时、连接/请求并发上限和对象复用;HTTP/2 是单独的实验适配器,必须配置 TLS。默认 nethttp 的 TLS 支持 HTTP/1.1 与 HTTP/2。
标准库与 native 的并发、延迟和长连接内存对照见 [PERFORMANCE.md](PERFORMANCE.md)。`python3 scripts/verifyPerformance.py` 运行检查与 benchmark;增加 `--load --output /tmp/goOmniPerf.json` 可测量独立服务进程并保存报告。
读写超时也可以在生成项目的 `etc/config.yaml` 中声明(放在 `server.http` 下):
HTTP/2 仍是显式 opt-in 实验适配器,不会改变 native 的 HTTP/1.1 默认路径。Linux TLS 长测使用 `python3 scripts/verifyHTTP2.py --duration 30m`,默认使用普通 Go 运行模式;`--race` 默认改为 60 秒,用于并发检查,其额外内存开销不能当成生产运行基线。
```yaml
server:
http:
readHeaderTimeout: 5s
readTimeout: 30s
writeTimeout: 30s
idleTimeout: 60s
maxHeaderBytes: 1048576
```
多连接混合场景使用 `python3 scripts/verifyHTTP2.py --scenario mixed --duration 30m`:8 条独立 TLS 连接,每条连接 14 个普通请求 worker、1 个慢上传 worker、1 个慢下载 worker,共 128 个 worker。上传为 256 KiB、每 4 KiB 等待 2 ms;下载为 8 MiB、每次最多读取 64 KiB 后等待 20 ms。日志分别记录三类成功数、传输字节数、每条连接的覆盖情况,以及停止并释放测试数据后两次 GC 的采样。
代码配置使用 `omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: 30*time.Second, WriteTimeout: 30*time.Second})`。`Run`、`RunContext`、`ServeContext` 会应用它;`NewServer` 会配置返回的标准 `http.Server`,包含可选的 Go 字段 `ErrorLog`。**正值覆盖、零值保留所选网络层的原有默认行为、负值报错**;不配置时升级行为不变。超时含义与支持范围见 [HTTP 配置](transport/README.md#http-配置v091-preview),不要把网络 I/O 超时当成业务执行时限。
该脚本默认把测试容器限制为 2 核、2 GiB 内存(禁用 swap)、256 个进程/线程,整个命令最多运行 40 分钟,中断和超时会清理本次容器。这些限制可用 `--cpus`、`--memory-mb`、`--pids-limit`、`--wall-timeout` 调整;共享服务器应先用 `--duration 60s` 短测,再逐步延长。每 15 秒输出的堆、RSS 和 FD 统计包含同一进程内的客户端与服务端。
自研网络层通过 `RunWith(ctx, runner)` 接入监听与连接生命周期,通过 `NewWithRouter(router)` 替换路由;业务 Handler/Service 不需要导入具体适配器。细节与验收契约见 [网络适配层](transport/README.md)。
Linux amd64 已完成 30 分钟、单 TLS 连接、128 个 worker 的普通模式长测:58,102,476 次请求,0 失败、0 协议错误,采样 RSS 峰值约 22.6 MiB,连接/活动请求归零且 FD 回到基线。完整采样和原始日志见 [HTTP/2 长测报告](performance/http2SoakLinux.json) 与 [原始日志](performance/http2SoakLinux.txt),此前的 race 对照见 [短测记录](performance/http2SmokeLinux.json)。这项结果覆盖单连接复用。
`Run` 处理 SIGINT/SIGTERM;`RunContext` 接收调用方控制的退出信号。服务先停止接收新请求,等待活跃请求完成,再清理资源;超时会取消请求并关闭连接,业务代码仍需响应 context 取消。
多连接慢客户端场景也已完成 Linux amd64 的 30 分钟长测:8 条 TLS 连接、128 个 worker,53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,共 53,191,687 次,0 失败与协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端、服务端和下载缓冲);退出后连接/请求归零,goroutine 回到 2、FD 回到 6;释放测试数据并两次 GC 后存活堆约 450 KiB,RSS 不要求立即归还操作系统。完整 120 份采样见 [混合长测报告](performance/http2MixedSoakLinux.json) 和 [原始日志](performance/http2MixedSoakLinux.txt)。连接级窗口耗尽、WINDOW_UPDATE 恢复及零窗口写阻塞的关闭回收也已有独立测试;Nginx 的 HTTP/2 回源验收结果见下方,适配器保持实验状态。
## 日常开发与升级
`python3 scripts/verifyHTTP2Proxy.py --count 3` 验证真实 Nginx 双段 TLS/h2、回源连接复用、证书拒绝、超时/取消恢复与新旧实例的 GOAWAY 排空;增加 `--race` 可运行并发检查。Linux amd64、Nginx 1.30.5 的普通与 race 模式各 3 轮均已通过。编译和测试容器顺序运行,均限制为 2 核、2 GiB;测试容器禁用外部网络,全部请求在容器内完成。使用方法、报告及当前边界见 [HTTP/2 代理回源验收](transport/http2/PROXY.md)。
在框架根目录运行:
在已生成的 API 项目目录执行:
```sh
go run ./examples/reference
go-omni api validate
go-omni api inspect
go-omni api generate --dry-run
go-omni api generate
go-omni api generate --check
go test ./...
```
默认地址为 `http://127.0.0.1:8080`。示例文档提供完整的 curl 操作流程。数据只保存在内存中,重启后清空。
`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Service、Repository、中间件实现、main、配置和依赖装配属于项目,重复生成保留手写内容;契约改变后仍需自行调整业务签名并通过编译。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 可运行 `generate --check` 检查是否同步。
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入网络库。
升级时先安装目标版本的 CLI,再在业务 module 中 `go get` **同一版本**;API 项目随后执行上述生成与测试步骤,手写项目只需依赖升级及业务回归。`v0.9.1-preview` 的变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 当前边界
## 按需深入
缓存、Redis 消息队列和进程内定时任务已作为可选模块提供。认证层提供 `Authenticator`、JWT/Session 适配器和 Policy 接口,登录流程与业务授权规则由项目实现。核心数据库层只依赖 `database/sql`,具体驱动、迁移和表结构由项目选择。
| 需要了解 | 文档 |
| --- | --- |
| `.api` 类型、参数绑定、错误、鉴权和生成规则 | [API 开发指南](API.md) |
| 公共响应体、提示优先级、独立回调协议 | [响应参考](RESPONSE.md) |
| CLI 完整参数、本地框架开发和模块脚手架 | [CLI](cmd/go-omni/README.md) |
| 完整 API 业务项目 | [userService](examples/userService/README.md) |
| 手写接口分层示例 | [reference](examples/reference/README.md) |
| 框架成熟度、历史变更 | [路线图](ROADMAP.md)、[更新记录](CHANGELOG.md) |
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
当前成熟度与后续验收标准见 [ROADMAP.md](ROADMAP.md),版本变更见 [CHANGELOG.md](CHANGELOG.md)。
`v0.8.0-preview` 新增 [公共响应模板](RESPONSE.md):在 `.api` 中定义一次返回公共体,分组与接口引用,运行时和 OpenAPI 共用。支持数字错误码、嵌套结构、原始 JSON,以及独立的文本/XML回调编码器。源码开发可通过 `--framework-dir` 显式引用本地框架。
当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。

View File

@@ -1,41 +1,31 @@
# v0.8.0-preview
# v0.9.1-preview
这一版让业务自行定义公共返回体:在 `.api` 中配置字段、结构、成功/失败提示和错误码类型,按分组复用或由单接口覆盖,运行时与 OpenAPI 共用定义。需要特殊协议的回调可独立控制状态、Header 和内容。
本版继续作为预览版供固定版本接入;HTTP/2 仍是独立实验适配器。
本版补齐已有项目接入的两个框架缺口:标准数据库 Connector 注入,以及应用级 HTTP 读写超时配置。TLS、数据库驱动日志、字符集/时区、迁移与业务规则继续由项目决定。
## 安装与升级
```sh
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.8.0-preview
# 已有业务项目中执行:
GOPRIVATE=git.yuntuo.hk go get git.yuntuo.hk/k2safe/go-omni@v0.8.0-preview
go-omni api generate
go test ./...
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
# 在各业务 module 中:
go get git.yuntuo.hk/k2safe/go-omni@v0.9.1-preview
```
新项目可以直接 `go-omni api new --module example.com/demo ./demo`,或用 `go-omni new --api service.api --module example.com/demo ./demo`。远程安装的 CLI 会固定生成项目的框架版本,无需本地 replace。需要仓库权限时使用自己的 Git 凭据管理器,不把凭据写入 module path 或 go.mod。
私有仓库使用本机 Git 凭据管理器,并将 `git.yuntuo.hk` 追加到已有 `GOPRIVATE`。CLI 和项目依赖使用同一固定版本。v0.9.0-preview 生成代码继续兼容,本次无需为了这两个入口重新生成接口。
## 本版功能
## 接入变化
- `response Name (...)` 定义公共 JSON 体,支持嵌套结构、独立 errorBody、常量及类型保持的占位符;通过 import 共享配置。
- `@server (response: Name)` 为整个 service 块选择模板,`@response Name` 单接口覆盖;`@response raw` 成功返回业务 JSON,错误保留组级格式。
- `codeType: int32 / int64 / string` 使用与业务字段一致的名称;默认码和映射都校验,int64 配置及生成代码保留整数精度。旧 `integer` 写法仍兼容。
- `successMessage` / `errorMessage` 配置默认提示,`codes` / `messages` 按逻辑错误码覆盖。业务只返回数据或错误,公共体由框架组装。
- `omni.WithResponse` 设置手写路由的应用默认格式,`WithResponseEncoder` 与 `HandleWithResponse` 支持特殊协议;Text / XML / Bytes / JSON / NoContent 可直接输出。
- 生成路由在全局中间件之前选定策略,提前鉴权拒绝、限流、panic 和 Context 克隆沿用正确格式。OpenAPI 同步模板结构与业务错误示例。
- 将项目已有 `sql.OpenDB(connector)` 的池创建替换为 `database.OpenConnector(ctx, connector, poolConfig)`;不要同时保留两个池。返回对象可用于 Health/InTx;需要标准库指针时取 `.DB`。
- 项目原有 `mysql.NewConnector`、独立 TLS、driver Logger、驱动读写时限、MySQL 版本/时区/排序规则校验继续保留,配置不进入核心框架。
- 将原来逐项修改 `http.Server` 的读写/空闲/头大小配置放入 `omni.HTTPConfig`,通过 `WithHTTPConfig` 或 `WithConfig` 传入。Go-only `ErrorLog` 支持 nethttp。
- 原有 `database.Open(ctx, Config)`、手动组装 `http.Server`、自定义 Runner 均可继续使用。零 HTTP 配置保留所选网络层已有默认行为;正值覆盖,负值报错。
详见 [公共响应体](RESPONSE.md)、[完整定义示例](examples/contracts/responses.api) 和 [更新记录](CHANGELOG.md)。
完整可复制示例:[数据库 Connector](database/README.md#connector-接入v091-preview)、[HTTP 配置及语义](transport/README.md#http-配置v091-preview)。其他迁移能力见 [迁移扩展指南](MIGRATION.md),从项目接入开始见 [README](README.md)。
## 升级行为
## 边界
1. CLI 与框架依赖一起升级到本版。新生成路由使用新的响应策略入口,旧运行时不能编译。旧 `.api` 可继续使用;生成器不会覆盖已有 Service、Repository、middleware、配置和 main。
2. 生成接口以 `.api` 为响应契约来源,即使没有声明模板也固定使用默认格式;应用 YAML 的 response 用于手写路由和应用兜底。需要变更生成接口的格式,请修改 `.api` 后重新生成。
3. `response.Response` DTO 保留为兼容类型,运行时不再依赖它固定结构。未配置的正常业务响应继续使用默认 code/message/data/requestId。
4. 内置 App 的 404/405 现在经过应用响应策略和中间件,默认输出 JSON;原先依赖纯文本兜底的客户端需调整。直接使用 nethttp.Router 的兜底行为不变。
5. `int32` / `int64` 按有符号整数范围校验并输出 JSON 数值;消费端需支持相应精度。需要文本码时使用 string。旧 integer 继续保留原有 ±(2^53−1) 范围。
6. `.api` 的模板和 raw 当前描述 JSON。文本/XML回调用 Go 编码器并维护对应文档;TLS/HTTP解析错误、连接级过载等发生在进入应用前的错误仍按网络协议处理。
独立实验性 `TransportHTTP2` 本次支持 IdleTimeout/MaxHeaderBytes 的配置透传,但不支持读头、读请求和写响应超时,设置这些正值会在监听前明确报错。native 与标准库服务器的超时含义不完全相同,详见上表;网络超时不等于 Handler 执行时间限制。`RunWith` 自定义 Runner 的配置由项目管理。
Connector 必须遵守驱动 context 协议,资源所有权按标准 `database/sql` 管理。有资源的 Connector 不跨池共享;启动 Ping 失败关闭池,成功后应用负责关闭。没有把 eTax/ePay 的数据库策略写进框架,也未修改这些业务项目或宣称它们已完成迁移。
## 验证入口
@@ -43,13 +33,13 @@ go test ./...
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...
go vet ./...
go build ./...
python3 scripts/verifyAPIWorkflow.py --report performance/responsePoliciesWorkflowLocal.json
python3 scripts/verifyResponseRelease.py --version v0.8.0-preview \
--report performance/responsePoliciesRemote.json
python3 scripts/verifyAPIWorkflow.py --version v0.8.0-preview \
--report performance/responsePoliciesWorkflowRemote.json
python3 scripts/verifyConnector.py --local --report performance/connectorLocal.json
python3 scripts/verifyConnector.py --version v0.9.1-preview --report performance/connectorRemote.json
python3 scripts/verifyCompatibilityRelease.py --version v0.9.1-preview --report performance/connectorCompatibilityRemote.json
```
远程验收分别从全新 module cache 安装 CLI,验证生成项目固定依赖且没有本地 replace,以及完整用户示例的真实 MySQL、nethttp/native、事务、日志和生命周期。数据库容器只使用本机临时资源(1 CPU / 768 MiB),结束清理,不连接业务服务器。
Connector 脚本只使用本地独立 MySQL 8.4 容器,限制 1 CPU、768 MiB、128 PID,生成一次性 CA/证书及凭据,结束清理自己的容器/卷/临时文件。测试验证正确 TLS、错误 CA/主机名拒绝、每池时区/日志配置隔离、事务、关闭与失败回收。HTTP 测试使用回环地址验证慢头、慢 body、空闲连接、流式写背压及请求头上限。常规单测不依赖 Docker,显式脚本要求真实 TLS 测试通过,不能跳过。
验收记录见 [响应模板验收](performance/responsePoliciesAcceptance.md)。OSS/R2 云账号实网验收与长时间压力测试不属于本次发布范围。
历史发布及验收记录保留在 [CHANGELOG](CHANGELOG.md)、[v0.9.0 接入验收](performance/compatibilityAcceptance.md)。
全量 race(40 个测试包)、静态检查、构建及本地隔离验收已通过,详见 [验收记录](performance/connectorAcceptance.md)。

View File

@@ -125,7 +125,7 @@ app.HandleWithResponse("POST", "/hooks/payment", ack, paymentHandler)
编码器也可以设置 HTTP 状态、Header、XML 或任意字节。`omni.WithResponseEncoder` 设置应用默认编码器;`httpx.WithResponse` 是普通路由中间件,按中间件顺序生效。需要覆盖全局中间件提前拒绝时,应使用 `HandleWithResponse`。自定义 Registrar 可以实现 `httpx.ResponseRegistrar` 提供相同的早期挂载能力;未实现时辅助函数退化为普通路由中间件。
`.api` 当前声明的是 JSON 协议,`@response raw` 也是原始 JSON,不代表任意文本/XML协议。自定义文本/XML回调使用上述 Go 入口,并由应用维护相应文档;生成器不会猜测自定义 Go 编码器的输出。如果在命名中间件里替换编码器,业务也必须同步维护文档。
`@response raw` 表示原始 JSON,不代表任意文本/XML协议。从 v0.9.0-preview 开始,`.api` 可用 `@handlerMode manual` 配合 `@produces` 描述文本/文件等成功输出,详见 [迁移扩展](MIGRATION.md)。自定义回调错误协议仍使用上述 Go 入口,并由应用维护相应文档;生成器不会猜测自定义 Go 编码器的输出。如果在命名中间件里替换编码器,业务也必须同步维护文档。
框架内置路由的 404/405 使用应用默认格式;它们不属于某个 `.api` 分组。TLS 握手、畸形 HTTP、连接级过载等发生在进入应用前,仍按网络层协议处理。自定义编码器失败、panic 或返回非法状态/204 body 时,最终返回空 500,避免递归编码或泄露另一种响应体。

View File

@@ -156,16 +156,24 @@ func (a *App) Run(address ...string) error {
return a.RunContext(ctx, address...)
}
// NewServer creates a plain standard HTTP server around the application.
// NewServer creates a plain standard HTTP server with the configured HTTP options.
// Invalid HTTP options panic; RunContext/ServeContext return validation errors.
// Use RunContext or ServeContext to apply the configured transport and TLS.
func (a *App) NewServer(address string) *http.Server {
if err := a.config.HTTP.Validate(); err != nil {
panic(err)
}
if address == "" {
address = a.config.Address
if address == "" {
address = ":8080"
}
}
return &http.Server{Addr: address, Handler: a.router}
cfg := a.config.HTTP
return &http.Server{Addr: address, Handler: a.router,
ReadHeaderTimeout: cfg.ReadHeaderTimeout, ReadTimeout: cfg.ReadTimeout,
WriteTimeout: cfg.WriteTimeout, IdleTimeout: cfg.IdleTimeout,
MaxHeaderBytes: cfg.MaxHeaderBytes, ErrorLog: cfg.ErrorLog}
}
// RunContext serves until the context is canceled, then performs a graceful shutdown.

32
app/body.go Normal file
View File

@@ -0,0 +1,32 @@
package app
import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/response"
"net/http"
)
func (a *App) bodyHandler(maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
checked := httpx.LimitBody(maxBytes)(handler)
if encoder == nil {
encoder = a.responseEncoder
}
next := a.adaptResponse(encoder, checked, middleware...)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
r = r.WithContext(r.Context())
r.Body = http.MaxBytesReader(w, r.Body, maxBytes)
}
next.ServeHTTP(w, r)
})
}
// HandleWithBodyLimit caps the reader before application middleware and preserves
// the response policy, including auth failures and parameter errors.
func (a *App) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
a.router.Handle(method, path, a.bodyHandler(maxBytes, encoder, handler, middleware...))
}
func (g *RouterGroup) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
combined := append(append([]httpx.Middleware{}, g.middlewares...), middleware...)
g.group.Handle(method, path, g.app.bodyHandler(maxBytes, encoder, handler, combined...))
}

199
app/httpConfig_test.go Normal file
View File

@@ -0,0 +1,199 @@
package app_test
import (
"bufio"
"context"
"errors"
"io"
"log"
"net"
"net/http"
"strings"
"sync/atomic"
"testing"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
)
func TestHTTPConfiguredStandardServer(t *testing.T) {
cfg := omni.HTTPConfig{ReadHeaderTimeout: time.Second, ReadTimeout: 2 * time.Second, WriteTimeout: 3 * time.Second, IdleTimeout: 4 * time.Second, MaxHeaderBytes: 8192, ErrorLog: log.New(io.Discard, "", 0)}
a := omni.New(omni.WithHTTPConfig(cfg))
s := a.NewServer(":9000")
if s.ReadHeaderTimeout != cfg.ReadHeaderTimeout || s.ReadTimeout != cfg.ReadTimeout || s.WriteTimeout != cfg.WriteTimeout || s.IdleTimeout != cfg.IdleTimeout || s.MaxHeaderBytes != cfg.MaxHeaderBytes || s.ErrorLog != cfg.ErrorLog || s.Addr != ":9000" {
t.Fatalf("server options: %+v", s)
}
s = omni.New().NewServer("")
if s.ReadHeaderTimeout != 0 || s.ReadTimeout != 0 || s.WriteTimeout != 0 || s.IdleTimeout != 0 || s.MaxHeaderBytes != 0 || s.ErrorLog != nil {
t.Fatal("default behavior changed")
}
}
func TestHTTPInvalidStartupReleasesResources(t *testing.T) {
a := omni.New(omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: -time.Second}))
called := 0
cleanupErr := errors.New("cleanup")
a.OnShutdown(func() error { called++; return cleanupErr })
err := a.RunContext(context.Background(), "bad address")
if err == nil || !strings.Contains(err.Error(), "http.readTimeout") || !errors.Is(err, cleanupErr) || called != 1 {
t.Fatalf("validation/cleanup: %v, %d", err, called)
}
defer func() {
if recover() == nil {
t.Error("NewServer accepted invalid config")
}
}()
a.NewServer("")
}
type httpConfigListener struct{ net.Listener }
func (l httpConfigListener) Accept() (net.Conn, error) {
c, err := l.Listener.Accept()
if err == nil {
if tcp, ok := c.(*net.TCPConn); ok {
_ = tcp.SetWriteBuffer(32 << 10)
}
}
return c, err
}
func httpConfigConnection(t *testing.T, transport string, cfg omni.HTTPConfig, handler omni.Handler) net.Conn {
t.Helper()
a := omni.New(omni.WithTransport(transport), omni.WithHTTPConfig(cfg), omni.WithShutdownTimeout(time.Second))
a.GET("/", handler)
a.POST("/", handler)
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- a.ServeContext(ctx, httpConfigListener{l}) }()
t.Cleanup(func() {
cancel()
select {
case err := <-done:
if err != nil {
t.Errorf("server shutdown: %v", err)
}
case <-time.After(3 * time.Second):
t.Error("server did not stop")
}
})
c, err := net.DialTimeout("tcp", l.Addr().String(), time.Second)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { c.Close() })
_ = c.SetDeadline(time.Now().Add(3 * time.Second))
return c
}
func TestHTTPConfiguredNetworkLimits(t *testing.T) {
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative} {
t.Run(transport, func(t *testing.T) {
empty := func(c *omni.Context) error { w := c.Raw(); w.WriteHeader(204); return nil }
t.Run("slowHeaders", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{ReadHeaderTimeout: 60 * time.Millisecond}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\nX-Incomplete:"); err != nil {
t.Fatal(err)
}
_, err := io.ReadAll(c)
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("server read-header timeout not applied")
}
})
t.Run("slowBody", func(t *testing.T) {
var accepted atomic.Bool
c := httpConfigConnection(t, transport, omni.HTTPConfig{ReadTimeout: 60 * time.Millisecond}, func(c *omni.Context) error {
_, err := io.ReadAll(c.Request().Body)
accepted.Store(err == nil)
return nil
})
if _, err := io.WriteString(c, "POST / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\nContent-Length: 10\r\n\r\nx"); err != nil {
t.Fatal(err)
}
_, err := io.ReadAll(c)
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("read timeout not applied")
}
if accepted.Load() {
t.Fatal("incomplete body accepted")
}
})
t.Run("idle", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{IdleTimeout: 60 * time.Millisecond}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"); err != nil {
t.Fatal(err)
}
r, err := http.ReadResponse(bufio.NewReader(c), nil)
if err != nil {
t.Fatal(err)
}
r.Body.Close()
if r.StatusCode != 204 {
t.Fatal(r.StatusCode)
}
_, err = c.Read(make([]byte, 1))
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("idle timeout not applied")
}
if err == nil {
t.Fatal("idle connection not closed")
}
})
t.Run("writeBackpressure", func(t *testing.T) {
done := make(chan error, 1)
c := httpConfigConnection(t, transport, omni.HTTPConfig{WriteTimeout: 60 * time.Millisecond}, func(c *omni.Context) error {
w := c.Raw()
w.WriteHeader(200)
if err := http.NewResponseController(w).Flush(); err != nil {
done <- err
return nil
}
_, err := io.Copy(w, io.LimitReader(repeatingHTTPReader{}, 64<<20))
done <- err
return nil
})
if tcp, ok := c.(*net.TCPConn); ok {
_ = tcp.SetReadBuffer(1024)
}
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"); err != nil {
t.Fatal(err)
}
select {
case err := <-done:
if err == nil {
t.Fatal("write unexpectedly completed")
}
case <-time.After(2 * time.Second):
t.Fatal("write timeout not applied")
}
})
t.Run("headerLimit", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{MaxHeaderBytes: 1024}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\nX-Large: "+strings.Repeat("a", 8192)+"\r\n\r\n"); err != nil {
t.Fatal(err)
}
r, err := http.ReadResponse(bufio.NewReader(c), nil)
if err != nil {
t.Fatal(err)
}
r.Body.Close()
if r.StatusCode != 431 {
t.Fatalf("oversized header status: %d", r.StatusCode)
}
})
})
}
}
type repeatingHTTPReader struct{}
func (repeatingHTTPReader) Read(b []byte) (int, error) {
for i := range b {
b[i] = 'x'
}
return len(b), nil
}

View File

@@ -31,6 +31,7 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
}
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
requestContext.SetResponseEncoder(encoder)
defer func() { httpx.CleanupMultipart(requestContext.Request()) }()
if err := wrapped(requestContext); err != nil {
a.writeError(requestContext, err)
}

View File

@@ -57,6 +57,22 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
case config.TransportNative:
server := native.New(a)
server.ShutdownTimeout = shutdownTimeout
cfg := a.config.HTTP
if cfg.ReadHeaderTimeout > 0 {
server.ReadHeaderTimeout = cfg.ReadHeaderTimeout
}
if cfg.ReadTimeout > 0 {
server.ReadTimeout = cfg.ReadTimeout
}
if cfg.WriteTimeout > 0 {
server.WriteTimeout = cfg.WriteTimeout
}
if cfg.IdleTimeout > 0 {
server.IdleTimeout = cfg.IdleTimeout
}
if cfg.MaxHeaderBytes > 0 {
server.MaxHeaderBytes = cfg.MaxHeaderBytes
}
if tlsConfig != nil {
return func(ctx context.Context, listener net.Listener) error {
return server.ServeTLS(ctx, listener, tlsConfig)
@@ -66,6 +82,12 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
case config.TransportHTTP2:
server := h2transport.New(a)
server.ShutdownTimeout = shutdownTimeout
if a.config.HTTP.IdleTimeout > 0 {
server.IdleTimeout = a.config.HTTP.IdleTimeout
}
if a.config.HTTP.MaxHeaderBytes > 0 {
server.MaxHeaderBytes = a.config.HTTP.MaxHeaderBytes
}
return func(ctx context.Context, listener net.Listener) error {
return server.ServeTLS(ctx, listener, tlsConfig)
}, nil

View File

@@ -1,5 +1,7 @@
# go-omni CLI
首次接入见 [README](../../README.md),无需克隆框架源码。本页说明命令参数;下方 `go run ./cmd/go-omni` 示例在框架仓库中执行,并显式引用本地源码。
## API 契约驱动项目
```sh
@@ -73,8 +75,8 @@ Handler / Service / Repository / Model 与模块注册入口。模块名为 lowe
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview.1
go-omni new --module example.com/billing ./billing
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
go-omni new --framework-version v0.9.1-preview --module example.com/billing ./billing
cd billing
go mod tidy
go run ./cmd/server

View File

@@ -0,0 +1,161 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestCompatibilityContract(t *testing.T) {
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
if err != nil {
t.Fatal(err)
}
c, err := Compile(s)
if err != nil {
t.Fatal(err)
}
if len(s.Scalars) != 1 || len(s.Routes) != 6 {
t.Fatal("incomplete contract")
}
if !strings.Contains(string(c.Document), `"anyOf"`) || !strings.Contains(string(c.Document), `"multipart/form-data"`) {
t.Fatal("missing schemas")
}
for _, change := range [][2]string{{"jsonInput: string | int64", "jsonInput: mystery"}, {"text: true", "text: false"}, {"mode: multipart", "mode: invalid"}, {"maxBytes: 128", "maxBytes: 0"}, {"goType: ID", "goType: id"}, {"Data any", "Data interface{ Method() }"}} {
source := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
spec, err := parseText(t, strings.Replace(source, change[0], change[1], 1))
if err == nil {
_, err = Compile(spec)
}
if err == nil {
t.Fatalf("accepted %s", change[1])
}
}
}
func TestCompatibilityConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
t.Skip("consumer integration")
}
root := frameworkRoot(t)
dir := filepath.Join(t.TempDir(), "consumer")
o := Options{API: filepath.Join(root, "examples/contracts/compatibility.api"), Dir: dir, Module: "example.com/compatibility", FrameworkDir: root}
releaseCLI := os.Getenv("GO_OMNI_RELEASE_CLI")
generate := func(check bool) {
t.Helper()
if releaseCLI == "" {
options := o
options.Check = check
if _, err := Generate(options); err != nil {
t.Fatal(err)
}
return
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
args := []string{"api", "generate", "--api", o.API, "--dir", dir, "--module", o.Module}
if check {
args = append(args, "--check")
}
cmd := exec.CommandContext(ctx, releaseCLI, args...)
cmd.Dir = filepath.Dir(dir)
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("released CLI: %v\n%s", err, output)
}
}
generate(false)
files := map[string]string{
"internal/service/demo/echo.go": "echo.go",
"internal/handler/demo/customBind.go": "hooks.go",
"internal/handler/files/downloadManual.go": "download.go",
"consumer_test.go": "consumer_test.go",
}
// The fixture implements related methods together; remove only its generated
// one-time stubs, then fill every stub so regeneration can preserve all files.
for _, name := range []string{"internal/service/demo/item.go", "internal/service/demo/custom.go"} {
writeFile(t, filepath.Join(dir, name), "package demo\n")
}
for _, name := range []string{"internal/handler/demo/uploadManual.go", "internal/handler/demo/callbackManual.go"} {
writeFile(t, filepath.Join(dir, name), "package demo\n")
}
for target, source := range files {
writeFile(t, filepath.Join(dir, target), readFile(t, filepath.Join("testdata/compatibility", source)))
}
before := map[string]string{}
for target := range files {
before[target] = readFile(t, filepath.Join(dir, target))
}
generate(false)
for target, content := range before {
if readFile(t, filepath.Join(dir, target)) != content {
t.Fatalf("overwrote %s", target)
}
}
generate(true)
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("consumer: %v\n%s", err, output)
}
if releaseCLI != "" {
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("module: %v\n%s", err, output)
}
var module struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(output, &module); err != nil {
t.Fatal(err)
}
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
t.Fatalf("not a fixed remote dependency: %+v", module)
}
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
}
}
func TestNestedDynamicAliases(t *testing.T) {
source := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
for _, typ := range []string{"[]interface{}", "map[string]interface{}", "map[string][]interface{}"} {
s, err := parseText(t, strings.Replace(source, "Data any", "Data "+typ, 1))
if err == nil {
_, err = Compile(s)
}
if err != nil {
t.Fatalf("%s: %v", typ, err)
}
}
}
func TestSwitchFromManualRequiresExplicitMigration(t *testing.T) {
root := frameworkRoot(t)
o := Options{API: filepath.Join(root, "examples/contracts/compatibility.api"), Dir: filepath.Join(t.TempDir(), "consumer"), Module: "example.com/compatibility", FrameworkDir: root}
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
// The hand-written source may contain business code. Never delete it to make
// the newly automatic handler compile; fail before touching generated files.
o.API = filepath.Join(o.Dir, "api/compatibility.api")
source := readFile(t, o.API)
source = strings.Replace(source, "@handlerMode manual\n @produces \"application/octet-stream\"", "", 1)
writeFile(t, o.API, source)
before := snapshot(t, o.Dir)
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "migrate and remove") {
t.Fatalf("unsafe mode switch: %v", err)
}
if !sameFiles(before, snapshot(t, o.Dir)) {
t.Fatal("failed mode switch mutated project")
}
}

View File

@@ -42,6 +42,9 @@ func Compile(spec *Spec) (*Compiled, error) {
}
definitions := map[string]Type{}
for _, t := range spec.Types {
if t.Name == "ContractScalars" && len(spec.Scalars) > 0 {
return nil, errors.New("ContractScalars is reserved when declaring scalars")
}
if !exportedName.MatchString(t.Name) {
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
}
@@ -50,17 +53,24 @@ func Compile(spec *Spec) (*Compiled, error) {
}
definitions[t.Name] = t
}
scalarTypes, scalarDefs, err := compileScalars(spec, definitions)
if err != nil {
return nil, err
}
resolved := map[string]reflect.Type{}
resolving := map[string]bool{}
var resolve func(string, int) (reflect.Type, error)
var expression func(ast.Expr, int) (reflect.Type, error)
primitives := map[string]reflect.Type{"string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
primitives := map[string]reflect.Type{"any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
if depth > 32 {
return nil, errors.New("type nesting exceeds 32")
}
switch e := e.(type) {
case *ast.Ident:
if t, ok := scalarTypes[e.Name]; ok {
return t, nil
}
if t, ok := primitives[e.Name]; ok {
return t, nil
}
@@ -140,7 +150,7 @@ func Compile(spec *Spec) (*Compiled, error) {
if err != nil {
return nil, err
}
b, err := contract.ForType(rt)
b, err := contract.Describe(rt, scalarDefs...)
if err != nil {
return nil, fmt.Errorf("%s: %w", t.Position, err)
}
@@ -217,12 +227,15 @@ func Compile(spec *Spec) (*Compiled, error) {
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
return nil, errors.New("@status supports 200, 201, 202 or 204")
}
if (r.Status == 204) != (r.Response == "") {
if !r.Manual && (r.Status == 204) != (r.Response == "") {
return nil, errors.New("204 requires no response; other success statuses require a response type")
}
if r.Method == "head" && r.Response != "" {
return nil, errors.New("HEAD must not declare a response body")
}
if err := validateRouteExtensions(r); err != nil {
return nil, fmt.Errorf("%s: %w", r.Position, err)
}
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
if r.Request != "" {
b, ok := c.Bindings[r.Request]
@@ -231,6 +244,9 @@ func Compile(spec *Spec) (*Compiled, error) {
}
op.Parameters = b.Parameters()
op.Request = b.BodySchema()
if op.Request != nil && (r.BodyMode == "raw" || r.BodyMode == "multipart") {
return nil, errors.New("raw/multipart request DTOs may declare only path/form/header fields; body binding is manual")
}
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
}

View File

@@ -140,6 +140,7 @@ func decorateDocument(doc map[string]any, spec *Spec) {
}
}
op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any)
decorateRequestOptions(op, r)
if r.ResponseFormat != "" {
op["x-omni-response"] = r.ResponseFormat
}

View File

@@ -19,7 +19,7 @@ import (
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.8.0-preview"
const minimumVersion = "v0.9.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
@@ -94,6 +94,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
return Output{}, err
}
o.Dir = directory
for _, r := range c.Spec.Routes {
if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
if _, err := os.Lstat(path); err == nil {
return Output{}, fmt.Errorf("manual handler %s exists; migrate and remove it explicitly before switching to automatic generation", path)
} else if !errors.Is(err, os.ErrNotExist) {
return Output{}, err
}
}
}
if err := resolveProfile(&o); err != nil {
return Output{}, err
}
@@ -130,13 +140,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if !filepath.IsAbs(path) {
path = filepath.Join(directory, path)
}
if _, err := os.Stat(filepath.Join(path, "httpx", "response.go")); err == nil {
if _, err := os.Stat(filepath.Join(path, "contract", "scalar.go")); err == nil {
localCurrent = true
}
}
}
if !localCurrent {
return Output{}, fmt.Errorf("existing framework dependency predates configurable responses; use %s or a current local replacement", minimumVersion)
return Output{}, fmt.Errorf("existing framework dependency predates migration extensions; use %s or a current local replacement", minimumVersion)
}
}
if o.FrameworkDir != "" {
@@ -309,6 +319,20 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
put(generated, "internal/apierrors/errorsGen.go", generatedHeader+content)
}
types := "package types\n\n"
if len(c.Spec.Scalars) > 0 {
types += "import("
for i, d := range c.Spec.Scalars {
types += fmt.Sprintf("scalar%d %q;", i, d.GoImport)
}
types += ")\n"
registry := fmt.Sprintf("package types\nimport %q\nfunc ContractScalars()[]contract.Scalar{return []contract.Scalar{", frameworkModule+"/contract")
for i, d := range c.Spec.Scalars {
types += fmt.Sprintf("type %s = scalar%d.%s\n", d.Name, i, d.GoType)
registry += fmt.Sprintf("contract.ScalarOf[%s](%q,%q,%t),", d.Name, d.JSONInput, d.JSONOutput, d.Text)
}
registry += "}}\n"
put(generated, "internal/types/scalarsGen.go", generatedHeader+registry)
}
for _, t := range c.Spec.Types {
types += "type " + t.Name + " struct {\n"
for _, f := range t.Fields {
@@ -331,6 +355,10 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
put(generated, "internal/handler/responseGen.go", generatedHeader+policySource)
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
if len(c.Spec.Scalars) > 0 {
imports += fmt.Sprintf("%q;%q;", frameworkModule+"/contract", o.Module+"/internal/types")
routes += "if err:=contract.ValidateScalars(types.ContractScalars()...);err!=nil{return err}\n"
}
middlewareSet := map[string]bool{}
for _, r := range c.Spec.Routes {
for _, m := range r.Middleware {
@@ -347,7 +375,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
for i, group := range c.Groups {
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps))\n", i, i, i)
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps),deps)\n", i, i, i)
groupRoutes := []Route{}
for _, r := range c.Spec.Routes {
if r.Group == group {
@@ -356,12 +384,16 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
packageName := filepath.Base(group)
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
needsTypes, needsBinder := false, false
needsTypes, needsBinder, hasAutomatic := false, false, false
for _, r := range groupRoutes {
if r.Manual {
continue
}
hasAutomatic = true
needsTypes = needsTypes || r.Request != "" || r.Response != ""
needsBinder = needsBinder || r.Request != ""
needsBinder = needsBinder || (r.Request != "" && !r.CustomBinding)
}
text += fmt.Sprintf("%q;", frameworkModule+"/httpx")
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/servicecontext")
if needsTypes {
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
}
@@ -371,18 +403,47 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
text += ")\n"
text += "type Service interface {\n"
for _, r := range groupRoutes {
text += signature(r) + "\n"
if !r.Manual {
text += signature(r) + "\n"
}
}
text += "}\ntype Handler struct {service Service}\nfunc New(s Service)*Handler {return &Handler{service:s}}\n"
text += "}\ntype Handler struct {service Service;deps *servicecontext.Context}\nfunc New(s Service,deps ...*servicecontext.Context)*Handler {h:=&Handler{service:s};if len(deps)>0{h.deps=deps[0]};return h}\n"
for _, r := range groupRoutes {
name := upper(r.Handler)
if r.Request != "" {
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s]()\n", lower(r.Handler), r.Request)
// The response policy and body reader cap are installed before App middleware.
if r.MaxBodyBytes > 0 {
routes += fmt.Sprintf("httpx.HandleWithBodyLimit(router,%q,%q,%d,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, r.MaxBodyBytes, responseName(r), r.ResponseFormat == "raw", i, name)
} else {
routes += fmt.Sprintf("httpx.HandleWithResponse(router,%q,%q,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, responseName(r), r.ResponseFormat == "raw", i, name)
}
for _, m := range r.Middleware {
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
}
routes += ")\n"
if r.Manual {
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Manual.go", fmt.Sprintf("package %s\nimport(%q;apierrors %q)\n// %s owns raw/multipart binding and response output. h.deps holds project dependencies.\n// This file is preserved on regeneration.\nfunc(h *Handler)%s(c *httpx.Context)error{return apierrors.New(501,\"NOT_IMPLEMENTED\",\"manual handler is not implemented\")}\n", packageName, frameworkModule+"/httpx", frameworkModule+"/errors", name, name))
continue
}
if r.Request != "" && !r.CustomBinding {
scalarArgs := ""
if len(c.Spec.Scalars) > 0 {
scalarArgs = "types.ContractScalars()..."
}
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s](%s)\n", lower(r.Handler), r.Request, scalarArgs)
}
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
call := "c.Context()"
if r.Request != "" {
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req);err!=nil{return err}\n", r.Request, lower(r.Handler))
if r.CustomBinding {
text += fmt.Sprintf("var req types.%s\nif err:=h.bind%s(c,&req);err!=nil{return err}\n", r.Request, name)
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Bind.go", fmt.Sprintf("package %s\nimport(%q;%q;apierrors %q)\n// bind%s is project-owned; repeated generation preserves it.\nfunc(h *Handler)bind%s(c *httpx.Context,req *types.%s)error{return apierrors.New(501,\"BINDING_NOT_IMPLEMENTED\",\"custom binding is not implemented\")}\n", packageName, frameworkModule+"/httpx", o.Module+"/internal/types", frameworkModule+"/errors", name, name, r.Request))
} else {
limit := ""
if r.MaxBodyBytes > 0 {
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
}
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req%s);err!=nil{return err}\n", r.Request, lower(r.Handler), limit)
}
call += ", &req"
}
if r.Response != "" {
@@ -392,11 +453,6 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
} else {
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
}
routes += fmt.Sprintf("httpx.HandleWithResponse(router,%q,%q,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, responseName(r), r.ResponseFormat == "raw", i, name)
for _, m := range r.Middleware {
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
}
routes += ")\n"
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
if r.Request != "" || r.Response != "" {
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
@@ -417,6 +473,10 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
}
if !hasAutomatic {
text = strings.Replace(text, strconv.Quote(frameworkModule+"/httpx")+";", "", 1)
text = strings.Replace(text, "\"context\";", "", 1)
}
if !hasResponse(groupRoutes) {
text = strings.Replace(text, "\"fmt\";", "", 1)
}
@@ -464,7 +524,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
func hasResponse(routes []Route) bool {
for _, r := range routes {
if r.Response != "" {
if r.Response != "" && !r.Manual {
return true
}
}

View File

@@ -18,6 +18,7 @@ type Spec struct {
Errors []ErrorDefinition
Auth []AuthDefinition
Responses []ResponseDefinition
Scalars []ScalarDefinition
Types []Type
Routes []Route
Sources map[string][]byte // relative to the entry API's directory
@@ -30,6 +31,9 @@ type Type struct {
}
type Field struct{ Name, Type, Tag string }
type Route struct {
Manual, CustomBinding bool
BodyMode, Produces string
MaxBodyBytes int64
Method, Path, Request, Response, Handler, Group, Summary string
Middleware []string
Auth string
@@ -199,6 +203,10 @@ func (p *parser) parse() error {
syntaxSeen := false
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
switch p.peek() {
case "scalar":
if err := p.parseScalar(); err != nil {
return err
}
case "syntax":
if syntaxSeen {
return p.err("duplicate syntax")
@@ -393,7 +401,15 @@ func (p *parser) parseType() error {
}
expression := ""
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
expression += p.next().text
if p.peek() == "interface" {
p.next()
if p.expect("{") != nil || p.expect("}") != nil {
return p.err("only empty interface{} is supported; use any for dynamic JSON")
}
expression += "any"
} else {
expression += p.next().text
}
}
if expression == "" {
return p.err("field type required; embedded types are not supported")
@@ -453,6 +469,28 @@ func (p *parser) parseService(server map[string]string) error {
}
seen[annotation] = true
switch annotation {
case "handlerMode":
mode, err := p.ident()
if err != nil {
return err
}
if mode != "manual" {
return p.err("handlerMode must be manual")
}
route.Manual = true
case "binding":
mode, err := p.ident()
if err != nil {
return err
}
if mode != "custom" {
return p.err("binding must be custom")
}
route.CustomBinding = true
case "produces":
route.Produces, e = p.quoted()
case "body":
e = p.parseBody(&route)
case "handler":
route.Handler, e = p.ident()
case "doc":
@@ -512,7 +550,7 @@ func (p *parser) parseService(server map[string]string) error {
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
return p.err("unexpected route suffix")
}
if route.Response == "" && !seen["status"] {
if route.Response == "" && !seen["status"] && !route.Manual {
route.Status = 204
}
p.spec.Routes = append(p.spec.Routes, route)

View File

@@ -0,0 +1,95 @@
package apigen
import (
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"mime"
"strconv"
"strings"
)
func (p *parser) parseBody(r *Route) error {
props, err := p.metadata()
if err != nil {
return err
}
for key, value := range props {
switch key {
case "mode":
r.BodyMode = value
case "maxBytes":
r.MaxBodyBytes, err = strconv.ParseInt(value, 10, 64)
if err != nil || r.MaxBodyBytes <= 0 || r.MaxBodyBytes > 1<<40 {
return p.err("maxBytes must be 1..1099511627776 bytes")
}
default:
return fmt.Errorf("unknown body property %s", key)
}
}
if r.MaxBodyBytes == 0 {
r.MaxBodyBytes = contract.DefaultMaxBodyBytes
}
return nil
}
func validateRouteExtensions(r Route) error {
if r.BodyMode != "" && r.BodyMode != "json" && r.BodyMode != "raw" && r.BodyMode != "multipart" {
return errors.New("body mode must be json, raw or multipart")
}
if (r.BodyMode == "raw" || r.BodyMode == "multipart") && !r.Manual {
return errors.New("raw/multipart body requires @handlerMode manual")
}
if r.CustomBinding && (r.Manual || r.Request == "") {
return errors.New("@binding custom requires an automatic handler with a request DTO")
}
if r.MaxBodyBytes > 0 && (r.Method == "get" || r.Method == "head") {
return errors.New("GET/HEAD cannot declare a body limit")
}
if r.Produces != "" {
if !r.Manual {
return errors.New("@produces requires @handlerMode manual")
}
media, params, err := mime.ParseMediaType(r.Produces)
if err != nil || len(params) != 0 || media != r.Produces || strings.Contains(media, "*") {
return errors.New("produces requires a concrete media type without parameters")
}
if r.Response != "" || r.Status == 204 || r.Method == "head" {
return errors.New("@produces uses a manual response without a DTO, 204 or HEAD")
}
}
if r.Manual && r.Status == 204 && r.Response != "" {
return errors.New("204 cannot declare a response")
}
return nil
}
func decorateRequestOptions(op map[string]any, r Route) {
if r.Manual {
op["x-omni-handler-mode"] = "manual"
}
if r.CustomBinding {
op["x-omni-binding"] = "custom"
}
if r.MaxBodyBytes > 0 {
op["x-omni-max-body-bytes"] = r.MaxBodyBytes
}
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
media := "application/octet-stream"
schema := map[string]any{"type": "string", "format": "binary"}
if r.BodyMode == "multipart" {
media = "multipart/form-data"
schema = map[string]any{"type": "object", "additionalProperties": true}
}
op["requestBody"] = map[string]any{"required": true, "content": map[string]any{media: map[string]any{"schema": schema}}}
}
if r.Produces != "" {
schema := map[string]any{"type": "string"}
if r.Produces == "application/octet-stream" {
schema["format"] = "binary"
}
if r.Produces == "application/json" {
schema = map[string]any{}
}
op["responses"].(map[string]any)[strconv.Itoa(r.Status)] = map[string]any{"description": "Success", "content": map[string]any{r.Produces: map[string]any{"schema": schema}}}
}
}

View File

@@ -234,6 +234,9 @@ func TestResponseTemplateConsumer(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, releaseCLI, "api", "generate", "--api", api, "--dir", dir, "--module", "example.com/responseConsumer")
// The framework checkout intentionally enables local development mode.
// Run the installed CLI outside it to exercise remote-only consumption.
cmd.Dir = filepath.Dir(dir)
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("released CLI: %v\n%s", err, output)
}

View File

@@ -0,0 +1,74 @@
package apigen
import (
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"golang.org/x/mod/module"
"reflect"
"strings"
)
type ScalarDefinition struct {
Name, GoImport, GoType, JSONInput, JSONOutput, Position string
Text bool
}
func (p *parser) parseScalar() error {
pos := p.next().position
name, err := p.ident()
if err != nil {
return err
}
p.skip()
props, err := p.metadata()
if err != nil {
return err
}
d := ScalarDefinition{Name: name, Position: pos}
for key, value := range props {
switch key {
case "goImport":
d.GoImport = value
case "goType":
d.GoType = value
case "jsonInput":
d.JSONInput = value
case "jsonOutput":
d.JSONOutput = value
case "text":
if value != "true" && value != "false" {
return p.err("text must be true or false")
}
d.Text = value == "true"
default:
return fmt.Errorf("%s: unknown scalar property %s", pos, key)
}
}
p.spec.Scalars = append(p.spec.Scalars, d)
return nil
}
func compileScalars(spec *Spec, types map[string]Type) (map[string]reflect.Type, []contract.Scalar, error) {
out := map[string]reflect.Type{}
var defs []contract.Scalar
goTypes := map[string]bool{}
for _, d := range spec.Scalars {
key := d.GoImport + "." + d.GoType
if d.Name == "ContractScalars" || !exportedName.MatchString(d.Name) || !exportedName.MatchString(d.GoType) || out[d.Name] != nil || types[d.Name].Name != "" || goTypes[key] {
return nil, nil, fmt.Errorf("%s: scalar names and Go types must be unique; names must be UpperCamelCase", d.Position)
}
if err := module.CheckImportPath(d.GoImport); err != nil || strings.Contains(d.GoImport, "\\") {
return nil, nil, fmt.Errorf("%s: invalid scalar goImport", d.Position)
}
// Unique stand-in types allow CLI schema checks without executing project code.
t := reflect.StructOf([]reflect.StructField{{Name: d.Name, Type: reflect.TypeOf(false)}})
def := contract.Scalar{Type: t, JSONInput: d.JSONInput, JSONOutput: d.JSONOutput, Text: d.Text}
probe := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: t, Tag: `json:"value"`}})
if _, err := contract.Describe(probe, def); err != nil {
return nil, nil, fmt.Errorf("%s: %w", d.Position, err)
}
out[d.Name] = t
defs = append(defs, def)
goTypes[key] = true
}
return out, defs, nil
}

View File

@@ -0,0 +1,117 @@
package compatibility_test
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"example.com/compatibility/internal/handler"
"example.com/compatibility/internal/servicecontext"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"io"
"mime/multipart"
"net/http/httptest"
"os"
"strings"
"testing"
)
func TestGeneratedCompatibility(t *testing.T) {
logger, err := logx.New(logx.Config{Writer: io.Discard})
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := servicecontext.New(context.Background(), logger)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
app := omni.New()
defer app.Close()
if err = handler.RegisterRoutes(app, deps); err != nil {
t.Fatal(err)
}
send := func(method, path, body, contentType string) *httptest.ResponseRecorder {
r := httptest.NewRequest(method, path, strings.NewReader(body))
r.Header.Set("Content-Type", contentType)
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
return w
}
for _, id := range []string{`9223372036854775807`, `"9223372036854775807"`} {
w := send("POST", "/echo", `{"id":`+id+`,"data":{"n":9223372036854775807}}`, "application/json")
if w.Code != 200 || !strings.Contains(w.Body.String(), `"id":"9223372036854775807"`) || !strings.Contains(w.Body.String(), `"n":9223372036854775807`) {
t.Fatalf("scalar/dynamic: %d %s", w.Code, w.Body)
}
}
for _, body := range []string{`{"id":true,"data":null}`, `{"id":9223372036854775808,"data":null}`, `{"id":1,"data":{"x":1,"x":2}}`, `{"id":1,"data":null,"extra":1}`} {
if w := send("POST", "/echo", body, "application/json"); w.Code != 400 {
t.Fatalf("accepted %s: %d", body, w.Code)
}
}
payload := `{"id":1,"data":"` + strings.Repeat("x", (1<<20)+1) + `"}`
if w := send("POST", "/echo", payload, "application/json"); w.Code != 200 {
t.Fatalf("route limit not passed to Binder: %d", w.Code)
}
if w := send("POST", "/echo", `{"id":1,"data":"`+strings.Repeat("x", 2<<20)+`"}`, "application/json"); w.Code != 413 {
t.Fatalf("large JSON: %d", w.Code)
}
if w := send("GET", "/items/9223372036854775807", "", ""); w.Code != 200 || !strings.Contains(w.Body.String(), `"id":"9223372036854775807"`) {
t.Fatalf("text scalar: %d %s", w.Code, w.Body)
}
if w := send("POST", "/custom", `{"value":"hello"}`, "application/json"); w.Code != 200 || !strings.Contains(w.Body.String(), "HELLO") {
t.Fatalf("custom binder: %d %s", w.Code, w.Body)
}
raw := `{ "event": 1, "id": "original bytes" }`
mac := hmac.New(sha256.New, []byte("fixture-only-key"))
mac.Write([]byte(raw))
r := httptest.NewRequest("POST", "/callback", strings.NewReader(raw))
r.Header.Set("X-Signature", hex.EncodeToString(mac.Sum(nil)))
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 200 || w.Body.String() != "accepted" {
t.Fatalf("raw callback: %d %s", w.Code, w.Body)
}
r = httptest.NewRequest("POST", "/callback", strings.NewReader(strings.Repeat("x", 129)))
r.ContentLength = -1
w = httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 413 {
t.Fatalf("chunked body limit: %d %s", w.Code, w.Body)
}
temp := t.TempDir()
t.Setenv("TMPDIR", temp)
var form bytes.Buffer
writer := multipart.NewWriter(&form)
part, err := writer.CreateFormFile("file", "sample.bin")
if err != nil {
t.Fatal(err)
}
part.Write(bytes.Repeat([]byte("x"), 8192))
writer.Close()
w = send("POST", "/upload", form.String(), writer.FormDataContentType())
if w.Code != 200 || !strings.Contains(w.Body.String(), `"bytes":8192`) {
t.Fatalf("upload: %d %s", w.Code, w.Body)
}
entries, err := os.ReadDir(temp)
if err != nil || len(entries) != 0 {
t.Fatalf("multipart temp files leaked: %v %v", entries, err)
}
w = send("GET", "/download", "", "")
if w.Code != 200 || w.Body.Len() != 128<<10 || w.Header().Get("Content-Disposition") == "" {
t.Fatalf("download wrapped: %d %d", w.Code, w.Body.Len())
}
var doc map[string]any
if err = json.Unmarshal([]byte(handler.OpenAPIDocument), &doc); err != nil {
t.Fatal(err)
}
paths := doc["paths"].(map[string]any)
media := paths["/download"].(map[string]any)["get"].(map[string]any)["responses"].(map[string]any)["200"].(map[string]any)["content"].(map[string]any)
if media["application/octet-stream"] == nil || media["application/json"] != nil {
t.Fatal("download schema is JSON")
}
}

View File

@@ -0,0 +1,16 @@
package files
import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"io"
"net/http"
"strings"
)
func (h *Handler) Download(c *httpx.Context) error {
return httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", `attachment; filename="demo.bin"`)
_, _ = io.Copy(w, strings.NewReader(strings.Repeat("x", 128<<10)))
}))(c)
}

View File

@@ -0,0 +1,16 @@
package demo
import (
"context"
"example.com/compatibility/internal/types"
)
func (s *Service) Echo(ctx context.Context, req *types.EchoReq) (*types.EchoResp, error) {
return &types.EchoResp{ID: req.ID, Data: req.Data}, nil
}
func (s *Service) Item(ctx context.Context, req *types.ItemReq) (*types.EchoResp, error) {
return &types.EchoResp{ID: req.ID, Data: map[string]any{"found": true}}, nil
}
func (s *Service) Custom(ctx context.Context, req *types.CustomReq) (*types.CustomResp, error) {
return &types.CustomResp{Value: req.Value}, nil
}

View File

@@ -0,0 +1,58 @@
package demo
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"example.com/compatibility/internal/types"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"io"
"net/http"
"strings"
)
func (h *Handler) bindCustom(c *httpx.Context, req *types.CustomReq) error {
decoder := json.NewDecoder(c.Request().Body)
decoder.DisallowUnknownFields()
if err := decoder.Decode(req); err != nil {
return apierrors.BadRequest("INVALID_REQUEST", "invalid request").WithCause(err)
}
var extra any
if err := decoder.Decode(&extra); err != io.EOF {
return apierrors.BadRequest("INVALID_REQUEST", "one JSON value required").WithCause(err)
}
req.Value = strings.ToUpper(req.Value)
return nil
}
func (h *Handler) Upload(c *httpx.Context) error {
// The small memory threshold deliberately exercises disk spill and cleanup.
if err := c.Request().ParseMultipartForm(8); err != nil {
return err
}
file, _, err := c.Request().FormFile("file")
if err != nil {
return err
}
defer file.Close()
count, err := io.Copy(io.Discard, file)
if err != nil {
return err
}
return c.OK(map[string]any{"bytes": count})
}
func (h *Handler) Callback(c *httpx.Context) error {
raw, err := io.ReadAll(c.Request().Body)
if err != nil {
return err
}
// This fixed key belongs only to the test fixture. Applications inject secrets.
mac := hmac.New(sha256.New, []byte("fixture-only-key"))
mac.Write(raw)
signature, err := hex.DecodeString(c.Header("X-Signature"))
if err != nil || !hmac.Equal(signature, mac.Sum(nil)) {
return c.Text(http.StatusUnauthorized, "invalid signature")
}
return c.Text(http.StatusOK, "accepted")
}

View File

@@ -252,7 +252,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "internal/handler/responseGen.go" {
if name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
return true
}
if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.8.0-preview"
const defaultFrameworkVersion = "v0.9.1-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.

View File

@@ -28,6 +28,7 @@ type Config struct {
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
@@ -74,11 +75,20 @@ func (c Config) Validate() error {
// ValidateTransport checks built-in transport selection without opening files
// or sockets. Empty Transport preserves the standard net/http default.
func (c Config) ValidateTransport() error {
if err := c.HTTP.Validate(); err != nil {
return err
}
switch c.Transport {
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
default:
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
}
if c.Transport == TransportHTTP2 && (c.HTTP.ReadHeaderTimeout > 0 || c.HTTP.ReadTimeout > 0 || c.HTTP.WriteTimeout > 0) {
return &ConfigError{Field: "http", Message: "readHeaderTimeout, readTimeout and writeTimeout are not supported by the experimental http2 transport"}
}
if c.HTTP.ErrorLog != nil && c.Transport != "" && c.Transport != TransportNetHTTP {
return &ConfigError{Field: "http.errorLog", Message: "is supported only by the nethttp transport"}
}
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
if (cert == "") != (key == "") {
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}

View File

@@ -48,6 +48,28 @@ func applyEnv(cfg Config, prefix string) (Config, error) {
cfg.Log.Environment = envString(prefix+"ENVIRONMENT", cfg.Log.Environment)
var err error
for _, field := range []struct {
name string
value *time.Duration
}{
{"HTTP_READ_HEADER_TIMEOUT", &cfg.HTTP.ReadHeaderTimeout},
{"HTTP_READ_TIMEOUT", &cfg.HTTP.ReadTimeout},
{"HTTP_WRITE_TIMEOUT", &cfg.HTTP.WriteTimeout},
{"HTTP_IDLE_TIMEOUT", &cfg.HTTP.IdleTimeout},
} {
if value := strings.TrimSpace(os.Getenv(prefix + field.name)); value != "" {
*field.value, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%s%s must be a duration: %w", prefix, field.name, err)
}
}
}
if value := strings.TrimSpace(os.Getenv(prefix + "HTTP_MAX_HEADER_BYTES")); value != "" {
cfg.HTTP.MaxHeaderBytes, err = strconv.Atoi(value)
if err != nil {
return Config{}, fmt.Errorf("%sHTTP_MAX_HEADER_BYTES must be an integer: %w", prefix, err)
}
}
if cfg.EnableLogger, err = envBool(prefix+"ENABLE_LOGGER", cfg.EnableLogger); err != nil {
return Config{}, err
}

45
config/http.go Normal file
View File

@@ -0,0 +1,45 @@
package config
import (
"log"
"time"
)
// HTTPConfig controls built-in HTTP servers. Positive values override the
// selected transport's defaults; zero keeps its existing behavior. Timeouts
// follow that transport's semantics, not a handler execution deadline.
type HTTPConfig struct {
ReadHeaderTimeout time.Duration `json:"readHeaderTimeout"`
ReadTimeout time.Duration `json:"readTimeout"`
WriteTimeout time.Duration `json:"writeTimeout"`
IdleTimeout time.Duration `json:"idleTimeout"`
MaxHeaderBytes int `json:"maxHeaderBytes"`
// ErrorLog is a Go-only hook for the nethttp server's internal diagnostics.
ErrorLog *log.Logger `json:"-"`
}
func (c HTTPConfig) Validate() error {
for _, field := range []struct {
name string
value time.Duration
}{
{"readHeaderTimeout", c.ReadHeaderTimeout},
{"readTimeout", c.ReadTimeout},
{"writeTimeout", c.WriteTimeout},
{"idleTimeout", c.IdleTimeout},
} {
if field.value < 0 {
return &ConfigError{Field: "http." + field.name, Message: "must be nonnegative"}
}
}
if c.MaxHeaderBytes < 0 {
return &ConfigError{Field: "http.maxHeaderBytes", Message: "must be nonnegative"}
}
return nil
}
// WithHTTPConfig sets the built-in server's HTTP options. RunWith uses a
// caller-owned Runner and does not apply these options to that runner.
func WithHTTPConfig(settings HTTPConfig) Option {
return func(cfg *Config) { cfg.HTTP = settings }
}

87
config/http_test.go Normal file
View File

@@ -0,0 +1,87 @@
package config_test
import (
"io"
"log"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
func TestHTTPFileAndEnvironment(t *testing.T) {
for _, file := range []struct{ name, body string }{
{"server.json", `{"http":{"readHeaderTimeout":"2s","readTimeout":"3s","writeTimeout":"4s","idleTimeout":"5s","maxHeaderBytes":8192}}`},
{"server.yaml", "http:\n readHeaderTimeout: 2s\n readTimeout: 3s\n writeTimeout: 4s\n idleTimeout: 5s\n maxHeaderBytes: 8192\n"},
} {
t.Run(file.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), file.name)
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadFile(path, "HTTP_TEST_")
if err != nil {
t.Fatal(err)
}
want := config.HTTPConfig{ReadHeaderTimeout: 2 * time.Second, ReadTimeout: 3 * time.Second, WriteTimeout: 4 * time.Second, IdleTimeout: 5 * time.Second, MaxHeaderBytes: 8192}
if cfg.HTTP != want {
t.Fatalf("HTTP: %+v", cfg.HTTP)
}
t.Setenv("HTTP_TEST_HTTP_READ_HEADER_TIMEOUT", "6s")
t.Setenv("HTTP_TEST_HTTP_READ_TIMEOUT", "7s")
t.Setenv("HTTP_TEST_HTTP_WRITE_TIMEOUT", "8s")
t.Setenv("HTTP_TEST_HTTP_IDLE_TIMEOUT", "9s")
t.Setenv("HTTP_TEST_HTTP_MAX_HEADER_BYTES", "16384")
got, err := config.ApplyEnv(cfg, "HTTP_TEST_")
if err != nil {
t.Fatal(err)
}
want = config.HTTPConfig{ReadHeaderTimeout: 6 * time.Second, ReadTimeout: 7 * time.Second, WriteTimeout: 8 * time.Second, IdleTimeout: 9 * time.Second, MaxHeaderBytes: 16384}
if got.HTTP != want {
t.Fatalf("overlay: %+v", got.HTTP)
}
})
}
}
func TestHTTPInvalidAndUnsupportedSettings(t *testing.T) {
for _, key := range []string{"READ_HEADER_TIMEOUT", "READ_TIMEOUT", "WRITE_TIMEOUT", "IDLE_TIMEOUT", "MAX_HEADER_BYTES"} {
for _, value := range []string{"invalid", "-1"} {
t.Run(key+value, func(t *testing.T) {
if value == "-1" && key != "MAX_HEADER_BYTES" {
value = "-1s"
}
t.Setenv("HTTP_INVALID_HTTP_"+key, value)
if _, err := config.LoadFromEnv("HTTP_INVALID_"); err == nil {
t.Fatal("invalid HTTP value accepted")
}
})
}
}
for _, http := range []config.HTTPConfig{{ReadHeaderTimeout: time.Second}, {ReadTimeout: time.Second}, {WriteTimeout: time.Second}} {
cfg := config.Default()
cfg.Transport = config.TransportHTTP2
cfg.TLS = config.TLSConfig{CertFile: "c", KeyFile: "k"}
cfg.HTTP = http
if err := cfg.ValidateTransport(); err == nil || !strings.Contains(err.Error(), "not supported") {
t.Fatalf("HTTP/2: %v", err)
}
}
cfg := config.Default()
cfg.Transport = config.TransportHTTP2
cfg.TLS = config.TLSConfig{CertFile: "c", KeyFile: "k"}
cfg.HTTP = config.HTTPConfig{IdleTimeout: time.Minute, MaxHeaderBytes: 8192}
if err := cfg.ValidateTransport(); err != nil {
t.Fatal(err)
}
for _, transport := range []string{config.TransportNative, config.TransportHTTP2} {
cfg.Transport = transport
cfg.HTTP = config.HTTPConfig{ErrorLog: log.New(io.Discard, "", 0)}
if err := cfg.ValidateTransport(); err == nil || !strings.Contains(err.Error(), "http.errorLog") {
t.Fatalf("ErrorLog silently ignored: %v", err)
}
}
}

View File

@@ -19,7 +19,7 @@ import (
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
// parameters never overwrite path/body/header values. form means URL query only.
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
if b == nil || r == nil {
if b == nil || r == nil || b.schemaOnly {
return apierrors.Internal(errors.New("contract: nil binder or request"))
}
v := reflect.ValueOf(target)
@@ -65,6 +65,10 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max
return tooLarge()
}
if err != io.EOF {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(errors.New("exactly one JSON value required"))
}
var ok bool
@@ -126,10 +130,16 @@ func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request bod
func fromStrings(n *node, values []string) (any, error) {
n = base(n)
if n.scalar != nil {
if len(values) != 1 {
return nil, errors.New("duplicate scalar parameter")
}
return scalarText(values[0]), nil
}
if n.typ.Kind() == reflect.Slice {
out := make([]any, 0, len(values))
for _, s := range values {
v, err := parseScalar(s, base(n.elem).typ)
v, err := fromStrings(n.elem, []string{s})
if err != nil {
return nil, err
}
@@ -158,6 +168,17 @@ func assignField(f field, dst reflect.Value, value any, present bool) error {
return checkField(f, dst)
}
func setValue(n *node, dst reflect.Value, value any) error {
if n.scalar != nil {
return n.scalar.decode(dst, value)
}
if n.typ.Kind() == reflect.Interface && n.typ.NumMethod() == 0 {
if value == nil {
dst.SetZero()
} else {
dst.Set(reflect.ValueOf(value))
}
return nil
}
if value == nil {
if n.typ.Kind() == reflect.Pointer {
dst.SetZero()

View File

@@ -23,10 +23,12 @@ var valuesValidator = validator.New()
// Binder is immutable after compilation and safe for concurrent requests.
type Binder struct {
root *node
typ reflect.Type
root *node
typ reflect.Type
schemaOnly bool
}
type node struct {
scalar *Scalar
typ reflect.Type
schema *openapi.Schema
fields []field
@@ -42,9 +44,11 @@ type field struct {
binding string
}
func New[T any]() (*Binder, error) { return ForType(reflect.TypeOf((*T)(nil)).Elem()) }
func MustNew[T any]() *Binder {
b, err := New[T]()
func New[T any](scalars ...Scalar) (*Binder, error) {
return ForType(reflect.TypeOf((*T)(nil)).Elem(), scalars...)
}
func MustNew[T any](scalars ...Scalar) *Binder {
b, err := New[T](scalars...)
if err != nil {
panic(err)
}
@@ -52,20 +56,36 @@ func MustNew[T any]() *Binder {
}
// ForType also lets generators check the exact runtime contract before writing files.
// The root must be a struct; recursive and custom-marshaling types are rejected.
func ForType(t reflect.Type) (*Binder, error) {
// The root must be a struct; custom marshalers require explicit scalar registration.
func ForType(t reflect.Type, scalars ...Scalar) (*Binder, error) { return build(t, false, scalars) }
// Describe compiles schemas without loading or executing application Go types.
// The returned Binder cannot bind requests; it is for generators only.
func Describe(t reflect.Type, scalars ...Scalar) (*Binder, error) { return build(t, true, scalars) }
func build(t reflect.Type, schemaOnly bool, scalars []Scalar) (*Binder, error) {
if t == nil || t.Kind() != reflect.Struct {
return nil, errors.New("contract: root must be a struct")
}
c := compiler{scalars: map[reflect.Type]Scalar{}}
for _, scalar := range scalars {
if err := scalar.validate(schemaOnly); err != nil {
return nil, err
}
if _, exists := c.scalars[scalar.Type]; exists {
return nil, fmt.Errorf("contract: duplicate scalar type %s", scalar.Type)
}
c.scalars[scalar.Type] = scalar
}
budget := 10000
n, err := compile(t, true, map[reflect.Type]bool{}, 0, &budget)
n, err := c.compile(t, true, map[reflect.Type]bool{}, 0, &budget)
if err != nil {
return nil, err
}
return &Binder{root: n, typ: t}, nil
return &Binder{root: n, typ: t, schemaOnly: schemaOnly}, nil
}
func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
func (c compiler) compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
*budget--
if *budget < 0 {
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
@@ -79,21 +99,32 @@ func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int,
stack[t] = true
defer delete(stack, t)
n := &node{typ: t, schema: &openapi.Schema{}}
if scalar, ok := c.scalars[t]; ok {
if root {
return nil, errors.New("contract: root cannot be a scalar")
}
n.scalar = &scalar
n.schema = scalar.inputSchema()
return n, nil
}
// Wire behavior must be described completely by these tags.
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler) {
return nil, errors.New("contract: custom JSON marshalers are unsupported")
if t.Kind() != reflect.Pointer && (t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler)) {
return nil, errors.New("contract: custom JSON marshalers require explicit Scalar registration")
}
switch t.Kind() {
case reflect.Interface:
if t.NumMethod() != 0 {
return nil, errors.New("contract: only any / interface{} is supported")
}
case reflect.Pointer:
e, err := compile(t.Elem(), false, stack, depth+1, budget)
e, err := c.compile(t.Elem(), false, stack, depth+1, budget)
if err != nil {
return nil, err
}
n.elem = e
*n.schema = *e.schema
n.schema.Nullable = true
n.schema = nullableSchema(e.schema)
case reflect.String:
n.schema.Type = "string"
case reflect.Bool:
@@ -132,10 +163,10 @@ func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int,
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
return nil, errors.New("contract: map keys must be strings")
}
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 {
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 && c.scalars[t.Elem()].Type == nil {
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
}
e, err := compile(t.Elem(), false, stack, depth+1, budget)
e, err := c.compile(t.Elem(), false, stack, depth+1, budget)
if err != nil {
return nil, err
}
@@ -158,7 +189,7 @@ func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int,
if sf.PkgPath != "" || sf.Anonymous {
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
}
f, err := compileField(sf, root, stack, depth+1, budget)
f, err := c.compileField(sf, root, stack, depth+1, budget)
if err != nil {
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
}
@@ -182,7 +213,7 @@ func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int,
return n, nil
}
func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
func (c compiler) compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
f := field{}
// Reject typos and unsupported tags instead of silently losing validation.
tags, err := ParseTags(string(sf.Tag))
@@ -218,11 +249,37 @@ func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
}
n, err := compile(sf.Type, false, stack, depth, budget)
n, err := c.compile(sf.Type, false, stack, depth, budget)
if err != nil {
return f, err
}
f.value = n
opaque := base(n).scalar
isList := base(n).typ.Kind() == reflect.Slice && opaque == nil
if isList {
opaque = base(base(n).elem).scalar
}
if opaque != nil {
if isList && (f.source == "path" || f.source == "header") {
return f, errors.New("path/header values must be scalars")
}
if f.source != "json" && !opaque.Text {
return f, errors.New("custom scalar needs text: true and UnmarshalText for path/query/header")
}
if f.binding != "" {
return f, errors.New("custom scalar constraints belong in its decoder; use optional/pointer for presence and null")
}
for _, option := range parts[1:] {
if option != "optional" && option != "omitempty" {
return f, errors.New("custom scalar supports only optional/omitempty field options")
}
f.optional = true
}
if f.source == "path" && (f.optional || sf.Type.Kind() == reflect.Pointer) {
return f, errors.New("path fields must be required non-pointer values")
}
return f, nil
}
scalar := base(n).typ.Kind()
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
return f, errors.New("path/query/header values must be scalars or scalar slices")
@@ -370,7 +427,7 @@ func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
return nil, errors.New("contract: response fields must use json tags")
}
}
return copySchema(b.root.schema), nil
return responseSchema(b.root), nil
}
func (b *Binder) Parameters() []openapi.Parameter {
var out []openapi.Parameter
@@ -382,7 +439,13 @@ func (b *Binder) Parameters() []openapi.Parameter {
if in == "form" {
in = "query"
}
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: copySchema(f.value.schema)})
schema := copySchema(f.value.schema)
if base(f.value).scalar != nil {
schema = &openapi.Schema{Type: "string"}
} else if base(f.value).typ.Kind() == reflect.Slice && base(base(f.value).elem).scalar != nil {
schema.Items = &openapi.Schema{Type: "string"}
}
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema})
}
return out
}

199
contract/scalar.go Normal file
View File

@@ -0,0 +1,199 @@
package contract
import (
"encoding"
"encoding/json"
"errors"
"fmt"
"reflect"
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
// Scalar explicitly opts one application Go type into custom wire conversion.
// Input is a | separated union of scalar wire types; Output is one wire type.
// The value must implement json.Marshaler, its pointer json.Unmarshaler, and
// Text=true additionally requires encoding.TextUnmarshaler on the pointer.
// No process-global registry or mutation of other binders is involved.
type Scalar struct {
Type reflect.Type
JSONInput, JSONOutput string
Text bool
}
func ScalarOf[T any](input, output string, text bool) Scalar {
return Scalar{Type: reflect.TypeOf((*T)(nil)).Elem(), JSONInput: input, JSONOutput: output, Text: text}
}
type compiler struct {
scalars map[reflect.Type]Scalar
}
type scalarText string
func (s Scalar) validate(schemaOnly bool) error {
if s.Type == nil || s.Type.Kind() == reflect.Pointer || s.Type.Kind() == reflect.Interface {
return errors.New("contract: scalar requires a concrete non-pointer type")
}
seen := map[string]bool{}
for _, kind := range strings.Split(s.JSONInput, "|") {
kind = strings.TrimSpace(kind)
if wireSchema(kind) == nil || seen[kind] {
return fmt.Errorf("contract: invalid/duplicate scalar input %q", kind)
}
seen[kind] = true
}
if wireSchema(s.JSONOutput) == nil {
return fmt.Errorf("contract: invalid scalar output %q", s.JSONOutput)
}
if !schemaOnly {
if !s.Type.Implements(reflect.TypeOf((*json.Marshaler)(nil)).Elem()) || !reflect.PointerTo(s.Type).Implements(reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()) {
return fmt.Errorf("contract: %s needs value MarshalJSON and pointer UnmarshalJSON", s.Type)
}
if s.Text && !reflect.PointerTo(s.Type).Implements(reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()) {
return fmt.Errorf("contract: %s needs UnmarshalText", s.Type)
}
}
return nil
}
func wireSchema(kind string) *openapi.Schema {
switch kind {
case "string":
return &openapi.Schema{Type: "string"}
case "bool":
return &openapi.Schema{Type: "boolean"}
case "int32", "int64":
return &openapi.Schema{Type: "integer", Format: kind}
case "float64":
return &openapi.Schema{Type: "number", Format: "double"}
}
return nil
}
func (s Scalar) inputSchema() *openapi.Schema {
var alternatives []*openapi.Schema
for _, kind := range strings.Split(s.JSONInput, "|") {
alternatives = append(alternatives, wireSchema(strings.TrimSpace(kind)))
}
if len(alternatives) == 1 {
return alternatives[0]
}
return &openapi.Schema{AnyOf: alternatives}
}
func (s Scalar) decode(dst reflect.Value, value any) (err error) {
// User decoders may fail; do not allow their panic to escape a binder.
defer func() {
if r := recover(); r != nil {
err = errors.New("custom scalar decoder panicked")
}
}()
if text, ok := value.(scalarText); ok {
decoder, ok := dst.Addr().Interface().(encoding.TextUnmarshaler)
if !s.Text || !ok {
return errors.New("text decoder not available")
}
return decoder.UnmarshalText([]byte(text))
}
allowed := false
for _, kind := range strings.Split(s.JSONInput, "|") {
if scalarAccepts(strings.TrimSpace(kind), value) {
allowed = true
break
}
}
if !allowed {
return errors.New("value does not match declared scalar input")
}
decoder, ok := dst.Addr().Interface().(json.Unmarshaler)
if !ok {
return errors.New("JSON decoder not available")
}
raw, err := json.Marshal(value) // json.Number preserves exact integer tokens.
if err != nil {
return err
}
return decoder.UnmarshalJSON(raw)
}
func scalarAccepts(kind string, value any) bool {
switch kind {
case "string":
_, ok := value.(string)
return ok
case "bool":
_, ok := value.(bool)
return ok
case "int32", "int64":
n, ok := value.(json.Number)
if !ok {
return false
}
bits := 64
if kind == "int32" {
bits = 32
}
_, err := strconv.ParseInt(string(n), 10, bits)
return err == nil
case "float64":
n, ok := value.(json.Number)
if !ok {
return false
}
_, err := strconv.ParseFloat(string(n), 64)
return err == nil
}
return false
}
func responseSchema(n *node) *openapi.Schema {
b := base(n)
if b.scalar != nil {
s := wireSchema(b.scalar.JSONOutput)
if n.typ.Kind() == reflect.Pointer {
s = nullableSchema(s)
}
return s
}
// Preserve the field's own constraints (including pointer range/enum and
// collection length) while changing only nested custom scalar wire types.
s := copySchema(n.schema)
if b.typ.Kind() == reflect.Slice {
s.Items = responseSchema(b.elem)
}
if b.typ.Kind() == reflect.Map {
s.AdditionalProperties = responseSchema(b.elem)
}
for _, f := range b.fields {
if f.source == "json" {
s.Properties[f.name] = responseSchema(f.value)
}
}
return s
}
// ValidateScalars checks the application's explicitly registered method contracts.
func ValidateScalars(scalars ...Scalar) error {
seen := map[reflect.Type]bool{}
for _, s := range scalars {
if err := s.validate(false); err != nil {
return err
}
if seen[s.Type] {
return fmt.Errorf("contract: duplicate scalar type %s", s.Type)
}
seen[s.Type] = true
}
return nil
}
// OpenAPI 3.0 nullable only applies alongside an explicit type. A union needs
// an explicit null alternative; an unconstrained dynamic schema already accepts null.
func nullableSchema(schema *openapi.Schema) *openapi.Schema {
s := copySchema(schema)
if s.Type != "" {
s.Nullable = true
return s
}
if len(s.AnyOf) > 0 {
s.AnyOf = append(s.AnyOf, &openapi.Schema{Type: "string", Nullable: true, Enum: []any{nil}})
}
return s
}

222
contract/scalar_test.go Normal file
View File

@@ -0,0 +1,222 @@
package contract
import (
"encoding/json"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"net/http"
"net/http/httptest"
"reflect"
"strconv"
"strings"
"testing"
)
type flexibleID int64
func (v flexibleID) MarshalJSON() ([]byte, error) {
return json.Marshal(strconv.FormatInt(int64(v), 10))
}
func (v *flexibleID) UnmarshalJSON(b []byte) error {
var s string
if len(b) > 0 && b[0] == '"' {
if err := json.Unmarshal(b, &s); err != nil {
return err
}
} else {
s = string(b)
}
return v.UnmarshalText([]byte(s))
}
func (v *flexibleID) UnmarshalText(b []byte) error {
n, err := strconv.ParseInt(string(b), 10, 64)
if err == nil {
*v = flexibleID(n)
}
return err
}
func TestDynamicJSONAndCustomScalars(t *testing.T) {
type request struct {
ID flexibleID `json:"id"`
Dynamic any `json:"dynamic"`
IDs []flexibleID `json:"ids,optional"`
}
scalar := ScalarOf[flexibleID]("string|int64", "string", true)
b, err := New[request](scalar)
if err != nil {
t.Fatal(err)
}
for _, id := range []string{`"9223372036854775807"`, `9223372036854775807`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":`+id+`,"dynamic":{"n":9223372036854775807},"ids":["1",2]}`))
r.Header.Set("Content-Type", "application/json")
var req request
if err = b.Bind(r, nil, &req); err != nil {
t.Fatal(err)
}
if req.ID != flexibleID(9223372036854775807) || req.Dynamic.(map[string]any)["n"] != json.Number("9223372036854775807") {
t.Fatalf("precision: %+v", req)
}
encoded, _ := json.Marshal(req)
if !strings.Contains(string(encoded), `"id":"9223372036854775807"`) {
t.Fatal(string(encoded))
}
}
for _, body := range []string{`{"id":true,"dynamic":null}`, `{"id":1.1,"dynamic":null}`, `{"id":9223372036854775808,"dynamic":null}`, `{"id":1,"dynamic":{"x":1,"x":2}}`, `{"id":1,"dynamic":null,"unexpected":1}`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
req := request{ID: 7}
if b.Bind(r, nil, &req) == nil || req.ID != 7 {
t.Fatalf("accepted or mutated %s", body)
}
}
for _, value := range []string{`null`, `[1,"x"]`, `false`, `"text"`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":1,"dynamic":`+value+`}`))
r.Header.Set("Content-Type", "application/json")
if err := b.Bind(r, nil, &request{}); err != nil {
t.Fatal(err)
}
}
if _, err := New[request](); err == nil {
t.Fatal("implicit custom type registration")
}
response, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if response.Properties["id"].Type != "string" || response.Properties["ids"].Items.Type != "string" || len(b.BodySchema().Properties["id"].AnyOf) != 2 {
t.Fatal("input/output schemas conflated")
}
type path struct {
ID flexibleID `path:"id"`
}
p, err := New[path](scalar)
if err != nil {
t.Fatal(err)
}
var req path
if err := p.Bind(httptest.NewRequest("GET", "/", nil), map[string]string{"id": "9223372036854775807"}, &req); err != nil {
t.Fatal(err)
}
if p.Parameters()[0].Schema.Type != "string" {
t.Fatal("text parameters need text schema")
}
if _, err := New[path](ScalarOf[flexibleID]("string|int64", "string", false)); err == nil {
t.Fatal("text support not explicit")
}
schemaOnly, err := Describe(reflect.TypeOf(request{}), scalar)
if err != nil {
t.Fatal(err)
}
if schemaOnly.Bind(httptest.NewRequest("POST", "/", nil), nil, &request{}) == nil {
t.Fatal("schema-only binder executed")
}
}
func TestScalarPointersCollectionsAndRegistration(t *testing.T) {
scalar := ScalarOf[flexibleID]("string|int64", "string", true)
type nullable struct {
ID *flexibleID `json:"id,optional"`
Values map[string]*flexibleID `json:"values,optional"`
}
b, err := New[nullable](scalar)
if err != nil {
t.Fatal(err)
}
for _, body := range []string{`{}`, `{"id":null}`, `{"id":1,"values":{"a":"2","b":null}}`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
if err := b.Bind(r, nil, &nullable{}); err != nil {
t.Fatal(err)
}
}
input := b.BodySchema().Properties["id"]
if len(input.AnyOf) != 3 || !input.AnyOf[2].Nullable || len(input.AnyOf[2].Enum) != 1 || input.AnyOf[2].Enum[0] != nil {
t.Fatal("nullable input union does not describe null")
}
out, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if out.Properties["id"].Type != "string" || !out.Properties["id"].Nullable {
t.Fatal("nullable output type")
}
type query struct {
IDs []flexibleID `form:"ids"`
}
q, err := New[query](scalar)
if err != nil {
t.Fatal(err)
}
var req query
if err := q.Bind(httptest.NewRequest("GET", "/?ids=1&ids=2", nil), nil, &req); err != nil || !reflect.DeepEqual(req.IDs, []flexibleID{1, 2}) {
t.Fatalf("query scalar collection: %+v %v", req, err)
}
if q.Parameters()[0].Schema.Items.Type != "string" {
t.Fatal("text array schema")
}
if _, err := New[query](ScalarOf[flexibleID]("string|int64", "string", false)); err == nil {
t.Fatal("array bypassed text declaration")
}
if err := ValidateScalars(ScalarOf[int64]("int64", "string", true)); err == nil {
t.Fatal("missing methods accepted")
}
if err := ValidateScalars(scalar, scalar); err == nil {
t.Fatal("duplicate registration")
}
type strict struct {
ID int64 `json:"id"`
}
strictBinder := MustNew[strict]()
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":"1"}`))
r.Header.Set("Content-Type", "application/json")
if strictBinder.Bind(r, nil, &strict{}) == nil {
t.Fatal("other binder globally loosened")
}
}
type panicScalar string
func (v panicScalar) MarshalJSON() ([]byte, error) { return json.Marshal(string(v)) }
func (v *panicScalar) UnmarshalJSON([]byte) error { panic("decoder failure") }
func TestScalarPanicIsBindingError(t *testing.T) {
type req struct {
Value panicScalar `json:"value"`
}
b := MustNew[req](ScalarOf[panicScalar]("string", "string", false))
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"value":"bad"}`))
r.Header.Set("Content-Type", "application/json")
dst := req{Value: "original"}
if err := b.Bind(r, nil, &dst); err == nil || dst.Value != "original" {
t.Fatalf("decoder panic: %+v %v", dst, err)
}
}
func TestLimitedJSONTrailingWhitespace(t *testing.T) {
type req struct {
Value any `json:"value"`
}
b := MustNew[req]()
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"value":null}`+strings.Repeat(" ", 65)))
r.Header.Set("Content-Type", "application/json")
r.ContentLength = -1
r.Body = http.MaxBytesReader(httptest.NewRecorder(), r.Body, 64)
err := b.Bind(r, nil, &req{}, 64)
if apierrors.From(err).Status != 413 {
t.Fatalf("tail limit error: %v", err)
}
}
func TestResponseSchemaPreservesOrdinaryPointerConstraints(t *testing.T) {
type dto struct {
Count *int `json:"count,optional,range=[1:10]"`
Choice *string `json:"choice,options=a|b"`
Values *[]int `json:"values,optional" binding:"max=3"`
}
b := MustNew[dto]()
response, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(response, b.BodySchema()) {
t.Fatalf("changed ordinary response constraints: %#v", response)
}
}

View File

@@ -3,7 +3,7 @@
`database` 包提供基于标准库 `database/sql` 的连接池生命周期,不绑定 ORM 或驱动:
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。
- `Open`:创建连接池、设置连接上限并在返回前执行 `PingContext`。
- `Open` / `OpenConnector`:通过 DSN 或标准 Connector 创建连接池、设置连接上限并在返回前执行 `PingContext`。
- `Health`:用于就绪检查,等待连接和 Ping 均受 `PingTimeout` 限制。
- `InTx`:统一事务提交和回滚;工作函数返回错误或 panic 时回滚。
- `Close`:由嵌入的 `*sql.DB` 提供,应用退出时显式调用。
@@ -40,7 +40,7 @@ GO_OMNI_DB_CONN_MAX_IDLE_TIME=5m
GO_OMNI_DB_PING_TIMEOUT=5s
```
`DSN` 必填,驱动默认是 `mysql`。数据库建库、迁移、表结构和具体 Repository 属于业务项目或可选模块,不会被核心包自动执行。
使用 `ConfigFromEnv` / `Open` 时 `DSN` 必填,环境变量入口的驱动默认是 `mysql`。数据库建库、迁移、表结构和具体 Repository 属于业务项目或可选模块,不会被核心包自动执行。
`MaxIdleConns=0` 会禁用空闲连接。初始化 Ping 失败会关闭已创建的连接池。
@@ -64,3 +64,61 @@ python3 scripts/verifyDatabase.py --version v0.2.0
`Config` 的 JSON/YAML 字段使用 lowerCamelCase,`DSN` 使用 `json:"-"`,连接密钥从环境变量加载。
API 脚手架 `--database mysql` 使用此流程,并要求正的 `maxOpenConns`,提供数据库 readiness 与幂等清理。
具体表结构、迁移和 Repository 留在业务项目,参考 `examples/userService`。
## Connector 接入(v0.9.1-preview)
项目已有 `database/sql/driver.Connector` 时使用 `OpenConnector`。这是标准驱动接口,核心包不增加具体驱动依赖。项目可以在同一进程创建多个使用不同证书、日志、拨号策略的连接池。
下面是 MySQL 的完整组装函数;连接地址、凭据、TLS 和驱动 Logger 都从调用方传入,不在框架中约定:
```go
package connection
import (
"context"
"crypto/tls"
"github.com/go-sql-driver/mysql"
"git.yuntuo.hk/k2safe/go-omni/database"
)
func Open(ctx context.Context, address, user, password, name string,
tlsConfig *tls.Config, logger mysql.Logger,
pool database.PoolConfig) (*database.DB, error) {
cfg := mysql.NewConfig()
cfg.Net, cfg.Addr = "tcp", address
cfg.User, cfg.Passwd, cfg.DBName = user, password, name
cfg.TLS = tlsConfig // 项目设置 RootCAs、ServerName、MinVersion 等
if logger != nil {
cfg.Logger = logger // 项目现有的 Print(...any) 日志适配器
}
connector, err := mysql.NewConnector(cfg)
if err != nil {
return nil, err
}
return database.OpenConnector(ctx, connector, pool)
}
```
调用时以 `database.DefaultPoolConfig()` 为起点,按项目需要调整。已有 `database.Config` 可调用 `.PoolConfig()` 提取池设置;`ConfigFromEnv` / `ApplyEnv` 仍是 DSN 入口,不用为 Connector 填假 DSN。`PoolConfig` 支持通过 `config.DecodeFile` 加载项目的 JSON/YAML 嵌套配置,环境覆盖由项目按自己的驱动配置组织。
| PoolConfig 字段 | 默认值 / 含义 |
| --- | --- |
| `MaxOpenConns` | `10`;`0` 表示不限制 |
| `MaxIdleConns` | `5`;`0` 禁止保留空闲连接 |
| `ConnMaxLifetime` | `30m`;`0` 不因连接年龄回收 |
| `ConnMaxIdleTime` | `5m`;`0` 不因空闲时长回收 |
| `PingTimeout` | `5s`;必须为正,约束启动 Ping 和 Health 等待 |
返回的 `*database.DB` 嵌入 `*sql.DB`,可使用 `QueryContext`、`ExecContext`、`Stats`,以及框架的 `Health`、`InTx`。需要标准数据库指针的 ORM/迁移器可取 `db.DB`;不要重复建立池。项目负责驱动支持的 context 取消与读写超时设置,框架不能强制中断不遵守 context 的自定义驱动。
**生命周期:** nil/带类型的 nil Connector、非法池参数、启动前已取消的 context 会在创建池前返回,Connector 仍归调用方。进入 `sql.OpenDB` 后由池持有;启动 Ping 失败自动关闭,成功后由调用方负责 `db.Close`,通常注册 `app.OnShutdown(db.Close)`。若 Connector 实现 `io.Closer`,遵循标准 `database/sql` 规则随池关闭;有资源的 Connector 不应跨多个池共享。资源初始化过程中、注册退出回调之前若业务校验失败,项目也须关闭池。
`db.PoolConfig()` 返回池配置快照。Connector 入口的 `db.Config()` 中 `Driver` / `DSN` 为空,不能用它重建 Connector。TLS、Logger、驱动版本检查、字符集、时区以及迁移规则保持由项目负责;不写入框架配置快照。
验收脚本使用独立本地 MySQL 8.4 容器(1 CPU / 768 MiB),自动生成测试 CA/证书与随机凭据,检查 TLS 信任/主机名、两池配置隔离、驱动日志、事务和 HTTP 超时,结束后清理本次资源:
```sh
python3 scripts/verifyConnector.py --local --report performance/connectorLocal.json
# 发布后通过全新 module cache 验证远程依赖,无本地 replace:
python3 scripts/verifyConnector.py --version v0.9.1-preview --report performance/connectorRemote.json
```

82
database/connector.go Normal file
View File

@@ -0,0 +1,82 @@
package database
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"reflect"
"time"
)
// PoolConfig contains only driver-independent settings. Connection credentials,
// TLS, logging and dial behavior belong to the application's Connector.
// Start from DefaultPoolConfig; zero PingTimeout is invalid.
type PoolConfig struct {
MaxOpenConns int `json:"maxOpenConns"`
MaxIdleConns int `json:"maxIdleConns"`
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
PingTimeout time.Duration `json:"pingTimeout"`
}
func DefaultPoolConfig() PoolConfig {
return PoolConfig{MaxOpenConns: 10, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute, ConnMaxIdleTime: 5 * time.Minute, PingTimeout: 5 * time.Second}
}
func (c PoolConfig) Validate() error {
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
return errors.New("database pool limits cannot be negative")
}
if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns {
return errors.New("max idle connections cannot exceed max open connections")
}
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
return errors.New("database timeouts are invalid")
}
return nil
}
func (c PoolConfig) config() Config {
return Config{MaxOpenConns: c.MaxOpenConns, MaxIdleConns: c.MaxIdleConns, ConnMaxLifetime: c.ConnMaxLifetime, ConnMaxIdleTime: c.ConnMaxIdleTime, PingTimeout: c.PingTimeout}
}
// OpenConnector opens an independent pool using the supplied standard Connector.
// No registered driver name or DSN is required. It applies cfg and completes a
// startup Ping bounded by ctx and PingTimeout before returning.
//
// Validation failures and an already-canceled ctx leave the Connector untouched.
// Once sql.OpenDB is called, the pool owns its lifecycle: a failed startup closes
// the pool; on success the caller must Close the returned DB. As in database/sql,
// this also closes a Connector that implements io.Closer. Use a separate Connector
// for each pool if it owns resources. Nil ctx means context.Background.
func OpenConnector(ctx context.Context, connector driver.Connector, cfg PoolConfig) (*DB, error) {
if nilConnector(connector) {
return nil, errors.New("database connector is required")
}
if err := cfg.Validate(); err != nil {
return nil, err
}
if ctx == nil {
ctx = context.Background()
}
if err := ctx.Err(); err != nil {
return nil, err
}
return initializePool(ctx, sql.OpenDB(connector), cfg.config())
}
func nilConnector(connector driver.Connector) bool {
if connector == nil {
return true
}
v := reflect.ValueOf(connector)
switch v.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
}
// PoolConfig returns a copy of the settings used to create this pool.
func (db *DB) PoolConfig() PoolConfig { return db.config.PoolConfig() }

336
database/connector_test.go Normal file
View File

@@ -0,0 +1,336 @@
package database_test
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"io"
"sync"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
)
type fixtureConnector struct {
identity string
connects, closedConnections, closedConnector, commits, rollbacks atomic.Int32
connect func(context.Context) error
ping func(context.Context) error
query func(context.Context) error
closeErr error
}
func (c *fixtureConnector) Connect(ctx context.Context) (driver.Conn, error) {
c.connects.Add(1)
if c.connect != nil {
if err := c.connect(ctx); err != nil {
return nil, err
}
}
return &fixtureConn{owner: c}, nil
}
func (c *fixtureConnector) Driver() driver.Driver { return fixtureDriver{} }
func (c *fixtureConnector) Close() error { c.closedConnector.Add(1); return c.closeErr }
type fixtureDriver struct{}
func (fixtureDriver) Open(string) (driver.Conn, error) {
return nil, errors.New("unexpected driver name/DSN path")
}
type fixtureConn struct{ owner *fixtureConnector }
func (c *fixtureConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("not used") }
func (c *fixtureConn) Close() error { c.owner.closedConnections.Add(1); return nil }
func (c *fixtureConn) Begin() (driver.Tx, error) { return fixtureTx{c.owner}, nil }
func (c *fixtureConn) Ping(ctx context.Context) error {
if c.owner.ping != nil {
return c.owner.ping(ctx)
}
return nil
}
func (c *fixtureConn) QueryContext(ctx context.Context, _ string, _ []driver.NamedValue) (driver.Rows, error) {
if c.owner.query != nil {
if err := c.owner.query(ctx); err != nil {
return nil, err
}
}
return &fixtureRows{identity: c.owner.identity}, nil
}
type fixtureTx struct{ owner *fixtureConnector }
func (tx fixtureTx) Commit() error { tx.owner.commits.Add(1); return nil }
func (tx fixtureTx) Rollback() error { tx.owner.rollbacks.Add(1); return nil }
type fixtureRows struct {
identity string
sent bool
}
func (r *fixtureRows) Columns() []string { return []string{"identity"} }
func (r *fixtureRows) Close() error { return nil }
func (r *fixtureRows) Next(values []driver.Value) error {
if r.sent {
return io.EOF
}
r.sent = true
values[0] = r.identity
return nil
}
func TestConnectorIndependentPoolsAndContext(t *testing.T) {
type key struct{}
ctx := context.WithValue(context.Background(), key{}, "startup")
cfg := database.DefaultPoolConfig()
cfg.MaxOpenConns = 2
cfg.MaxIdleConns = 2
var owners []*fixtureConnector
var pools []*database.DB
for _, identity := range []string{"alpha", "beta"} {
owner := &fixtureConnector{identity: identity}
// Startup Connect/Ping and each query preserve their caller context.
// database/sql may also open replacement connections in the background.
owner.connect = func(ctx context.Context) error {
if owner.connects.Load() == 1 && ctx.Value(key{}) != "startup" {
return errors.New("lost request context")
}
return nil
}
owner.ping = func(ctx context.Context) error {
if ctx.Value(key{}) != "startup" {
return errors.New("lost ping context")
}
return nil
}
owner.query = func(ctx context.Context) error {
if ctx.Value(key{}) != "startup" {
return errors.New("lost query context")
}
return nil
}
pool, err := database.OpenConnector(ctx, owner, cfg)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pool.Close() })
if pool.PoolConfig() != cfg || pool.Config().Driver != "" || pool.Config().DSN != "" || pool.Stats().MaxOpenConnections != 2 {
t.Fatal("pool settings or DSN boundary")
}
owners = append(owners, owner)
pools = append(pools, pool)
}
var wg sync.WaitGroup
for i, pool := range pools {
for n := 0; n < 16; n++ {
wg.Add(1)
go func(i int, pool *database.DB) {
defer wg.Done()
var identity string
if err := pool.QueryRowContext(ctx, "identity").Scan(&identity); err != nil {
t.Error(err)
} else if identity != owners[i].identity {
t.Error("crossed connectors")
}
}(i, pool)
}
}
wg.Wait()
for i, pool := range pools {
if pool.Stats().OpenConnections > cfg.MaxOpenConns {
t.Fatal("pool cap not applied")
}
if err := pool.Close(); err != nil {
t.Fatal(err)
}
if err := pool.Close(); err != nil {
t.Fatal(err)
}
if owners[i].closedConnector.Load() != 1 || owners[i].closedConnections.Load() != owners[i].connects.Load() {
t.Fatal("close did not release owned resources exactly once")
}
if pool.Health(ctx) == nil {
t.Fatal("closed pool is healthy")
}
}
}
func TestConnectorRejectsInvalidInputWithoutTakingOwnership(t *testing.T) {
var typedNil *fixtureConnector
for _, connector := range []driver.Connector{nil, typedNil} {
if db, err := database.OpenConnector(nil, connector, database.DefaultPoolConfig()); err == nil || db != nil {
t.Fatal("nil connector accepted")
}
}
for _, mutate := range []func(*database.PoolConfig){
func(c *database.PoolConfig) { c.MaxOpenConns = -1 }, func(c *database.PoolConfig) { c.MaxIdleConns = -1 },
func(c *database.PoolConfig) { c.MaxIdleConns = c.MaxOpenConns + 1 }, func(c *database.PoolConfig) { c.PingTimeout = 0 },
func(c *database.PoolConfig) { c.ConnMaxLifetime = -1 }, func(c *database.PoolConfig) { c.ConnMaxIdleTime = -1 },
} {
cfg := database.DefaultPoolConfig()
mutate(&cfg)
owner := &fixtureConnector{}
if db, err := database.OpenConnector(nil, owner, cfg); err == nil || db != nil {
t.Fatal("invalid config accepted")
}
if owner.connects.Load() != 0 || owner.closedConnector.Load() != 0 {
t.Fatal("validation consumed connector")
}
}
owner := &fixtureConnector{}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if db, err := database.OpenConnector(ctx, owner, database.DefaultPoolConfig()); !errors.Is(err, context.Canceled) || db != nil {
t.Fatalf("pre-canceled context: %v", err)
}
if owner.connects.Load() != 0 || owner.closedConnector.Load() != 0 {
t.Fatal("pre-cancellation consumed connector")
}
if err := database.DefaultConfig().Validate(); err == nil {
t.Fatal("legacy DSN requirements changed")
}
}
func TestConnectorStartupFailuresReleaseResources(t *testing.T) {
failure := errors.New("fixture unavailable")
closeFailure := errors.New("fixture close failed")
for _, phase := range []string{"connect", "ping"} {
t.Run(phase, func(t *testing.T) {
owner := &fixtureConnector{closeErr: closeFailure}
if phase == "connect" {
owner.connect = func(context.Context) error { return failure }
} else {
owner.ping = func(context.Context) error { return failure }
}
pool, err := database.OpenConnector(nil, owner, database.DefaultPoolConfig())
if pool != nil || !errors.Is(err, failure) || !errors.Is(err, closeFailure) {
t.Fatalf("lost failure or cleanup error: %v", err)
}
expectedConnections := int32(0)
if phase == "ping" {
expectedConnections = 1
}
if owner.closedConnector.Load() != 1 || owner.closedConnections.Load() != expectedConnections {
t.Fatal("startup failure leaked resources")
}
})
}
}
func TestConnectorTimeoutBoundsConnectAndPing(t *testing.T) {
for _, phase := range []string{"connect", "ping"} {
t.Run(phase, func(t *testing.T) {
owner := &fixtureConnector{}
cfg := database.DefaultPoolConfig()
cfg.PingTimeout = 20 * time.Millisecond
wait := func(ctx context.Context) error {
if _, ok := ctx.Deadline(); !ok {
return errors.New("missing deadline")
}
<-ctx.Done()
return ctx.Err()
}
if phase == "connect" {
owner.connect = wait
} else {
owner.ping = wait
}
pool, err := database.OpenConnector(context.Background(), owner, cfg)
if pool != nil || !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("%s timeout: %v", phase, err)
}
if owner.closedConnector.Load() != 1 {
t.Fatal("timeout leaked connector")
}
})
}
}
func TestConnectorHealthBoundsPoolWaitAndRecovers(t *testing.T) {
cfg := database.DefaultPoolConfig()
cfg.MaxOpenConns = 1
cfg.MaxIdleConns = 1
cfg.PingTimeout = 20 * time.Millisecond
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, cfg)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
conn, err := pool.Conn(context.Background())
if err != nil {
t.Fatal(err)
}
defer conn.Close()
if err = pool.Health(nil); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("waiting for busy pool was not bounded: %v", err)
}
if err = conn.Close(); err != nil {
t.Fatal(err)
}
if err = pool.Health(nil); err != nil {
t.Fatal(err)
}
}
func TestConnectorIdleZeroAndLifetime(t *testing.T) {
for _, mode := range []string{"noIdle", "expired"} {
t.Run(mode, func(t *testing.T) {
cfg := database.DefaultPoolConfig()
if mode == "noIdle" {
cfg.MaxIdleConns = 0
} else {
cfg.ConnMaxLifetime = time.Nanosecond
}
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, cfg)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
if err = pool.Health(nil); err != nil {
t.Fatal(err)
}
if owner.connects.Load() < 2 || owner.closedConnections.Load() == 0 {
t.Fatal("pool did not retire connections")
}
if mode == "noIdle" && pool.Stats().Idle != 0 {
t.Fatal("zero idle not honored")
}
if mode == "expired" && pool.Stats().MaxLifetimeClosed == 0 {
t.Fatal("lifetime not honored")
}
})
}
}
func TestConnectorTransactionsKeepCommitAndRollbackSemantics(t *testing.T) {
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, database.DefaultPoolConfig())
if err != nil {
t.Fatal(err)
}
defer pool.Close()
ctx := context.Background()
if err = pool.InTx(ctx, nil, func(*sql.Tx) error { return nil }); err != nil {
t.Fatal(err)
}
failed := errors.New("business failed")
if err = pool.InTx(ctx, nil, func(*sql.Tx) error { return failed }); !errors.Is(err, failed) {
t.Fatal(err)
}
func() {
defer func() {
if recover() != failed {
t.Error("panic not preserved")
}
}()
_ = pool.InTx(ctx, nil, func(*sql.Tx) error { panic(failed) })
}()
if owner.commits.Load() != 1 || owner.rollbacks.Load() != 2 {
t.Fatal("transaction semantics changed")
}
}

View File

@@ -23,13 +23,14 @@ type Config struct {
PingTimeout time.Duration `json:"pingTimeout"`
}
func DefaultConfig() Config {
return Config{
MaxOpenConns: 10,
MaxIdleConns: 5,
ConnMaxLifetime: 30 * time.Minute,
ConnMaxIdleTime: 5 * time.Minute,
PingTimeout: 5 * time.Second,
func DefaultConfig() Config { return DefaultPoolConfig().config() }
// PoolConfig returns driver-independent connection pool settings.
func (c Config) PoolConfig() PoolConfig {
return PoolConfig{
MaxOpenConns: c.MaxOpenConns, MaxIdleConns: c.MaxIdleConns,
ConnMaxLifetime: c.ConnMaxLifetime, ConnMaxIdleTime: c.ConnMaxIdleTime,
PingTimeout: c.PingTimeout,
}
}
@@ -83,16 +84,7 @@ func (c Config) Validate() error {
if strings.TrimSpace(c.DSN) == "" {
return errors.New("database DSN is required")
}
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
return errors.New("database pool limits cannot be negative")
}
if c.MaxIdleConns > c.MaxOpenConns && c.MaxOpenConns > 0 {
return errors.New("max idle connections cannot exceed max open connections")
}
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
return errors.New("database timeouts are invalid")
}
return nil
return c.PoolConfig().Validate()
}
// DB owns a connection pool and exposes the embedded database/sql methods.
@@ -112,23 +104,30 @@ func Open(ctx context.Context, cfg Config) (*DB, error) {
if err != nil {
return nil, fmt.Errorf("open database: %w", err)
}
if cfg.MaxOpenConns > 0 {
db.SetMaxOpenConns(cfg.MaxOpenConns)
}
// database/sql uses zero to disable idle connections; do not leave its
// default pool size in place when the caller explicitly chooses zero.
db.SetMaxIdleConns(cfg.MaxIdleConns)
return initializePool(ctx, db, cfg)
}
// initializePool is the shared ownership boundary for both creation paths.
// All configuration is validated before creating the sql.DB.
func initializePool(ctx context.Context, db *sql.DB, cfg Config) (*DB, error) {
db.SetMaxOpenConns(cfg.MaxOpenConns)
db.SetMaxIdleConns(cfg.MaxIdleConns) // Zero explicitly disables idle connections.
db.SetConnMaxLifetime(cfg.ConnMaxLifetime)
db.SetConnMaxIdleTime(cfg.ConnMaxIdleTime)
checkCtx, cancel := context.WithTimeout(ctx, cfg.PingTimeout)
defer cancel()
if err := db.PingContext(checkCtx); err != nil {
_ = db.Close()
return nil, fmt.Errorf("ping database: %w", err)
pingErr := fmt.Errorf("ping database: %w", err)
if closeErr := db.Close(); closeErr != nil {
return nil, errors.Join(pingErr, fmt.Errorf("close database after failed ping: %w", closeErr))
}
return nil, pingErr
}
return &DB{DB: db, config: cfg}, nil
}
// Config returns the original DSN configuration. Pools opened with a Connector
// have empty Driver/DSN fields; their connector cannot be reconstructed from Config.
func (db *DB) Config() Config { return db.config }
func (db *DB) Health(ctx context.Context) error {

View File

@@ -0,0 +1,180 @@
package database_test
import (
"context"
"crypto/tls"
"crypto/x509"
"database/sql"
"errors"
"net"
"os"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"github.com/go-sql-driver/mysql"
)
type connectorDriverLog struct{ count atomic.Int32 }
func (l *connectorDriverLog) Print(...any) { l.count.Add(1) }
func TestMySQLConnectorDriverLoggerIsolation(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
stopped := make(chan struct{})
go func() {
defer close(stopped)
for {
c, err := listener.Accept()
if err != nil {
return
}
c.Close()
}
}()
defer func() { listener.Close(); <-stopped }()
logs := [2]connectorDriverLog{}
for i := range logs {
cfg := mysql.NewConfig()
cfg.Net = "tcp"
cfg.Addr = listener.Addr().String()
cfg.Timeout = time.Second
cfg.ReadTimeout = time.Second
cfg.Logger = &logs[i]
connector, err := mysql.NewConnector(cfg)
if err != nil {
t.Fatal(err)
}
before := [2]int32{logs[0].count.Load(), logs[1].count.Load()}
db, err := database.OpenConnector(context.Background(), connector, database.DefaultPoolConfig())
if err == nil {
db.Close()
t.Fatal("handshake to closed socket succeeded")
}
if logs[i].count.Load() <= before[i] || logs[1-i].count.Load() != before[1-i] {
t.Fatal("driver logger did not stay on its connector")
}
}
}
func TestMySQLConnectorTLSAndPoolIsolation(t *testing.T) {
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
t.Skip("use scripts/verifyConnector.py for isolated MySQL TLS verification")
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
ca, err := os.ReadFile(os.Getenv("GO_OMNI_CONNECTOR_CA_FILE"))
if err != nil {
t.Fatal(err)
}
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM(ca) {
t.Fatal("invalid test CA")
}
cfg := mysql.NewConfig()
cfg.Net = "tcp"
cfg.Addr = os.Getenv("GO_OMNI_CONNECTOR_ADDR")
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
cfg.Passwd = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
cfg.DBName = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
cfg.Timeout = time.Second
cfg.ReadTimeout = 2 * time.Second
cfg.WriteTimeout = 2 * time.Second
cfg.TLS = &tls.Config{MinVersion: tls.VersionTLS12, ServerName: "localhost", RootCAs: roots}
pool := database.DefaultPoolConfig()
pool.MaxOpenConns = 2
pool.MaxIdleConns = 2
pool.PingTimeout = 2 * time.Second
open := func(cfg *mysql.Config) (*database.DB, error) {
connector, err := mysql.NewConnector(cfg)
if err != nil {
return nil, err
}
return database.OpenConnector(ctx, connector, pool)
}
cfg.Params = map[string]string{"time_zone": "'+00:00'"}
first, err := open(cfg)
if err != nil {
t.Fatal(err)
}
defer first.Close()
other := cfg.Clone()
other.Params["time_zone"] = "'+08:00'"
second, err := open(other)
if err != nil {
t.Fatal(err)
}
defer second.Close()
for i, db := range []*database.DB{first, second} {
var name, cipher, zone string
if err := db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&name, &cipher); err != nil {
t.Fatal(err)
}
if cipher == "" {
t.Fatal("connection is not encrypted")
}
if err := db.QueryRowContext(ctx, "SELECT @@session.time_zone").Scan(&zone); err != nil {
t.Fatal(err)
}
if zone != []string{"+00:00", "+08:00"}[i] {
t.Fatalf("session config leaked: %s", zone)
}
if db.Stats().MaxOpenConnections != 2 {
t.Fatal("pool configuration ignored")
}
}
for _, bad := range []string{"untrustedCA", "wrongHostname"} {
t.Run(bad, func(t *testing.T) {
invalid := cfg.Clone()
if bad == "untrustedCA" {
invalid.TLS.RootCAs = x509.NewCertPool()
} else {
invalid.TLS.ServerName = "wrong.invalid"
}
db, err := open(invalid)
if err == nil {
db.Close()
t.Fatal("invalid server certificate accepted")
}
var verification *tls.CertificateVerificationError
if !errors.As(err, &verification) {
t.Fatalf("not a TLS verification failure: %v", err)
}
if err := first.Health(ctx); err != nil {
t.Fatalf("failed connector affected existing pool: %v", err)
}
})
}
if _, err := first.ExecContext(ctx, "CREATE TABLE connector_check (id INT PRIMARY KEY)"); err != nil {
t.Fatal(err)
}
if err := first.InTx(ctx, nil, func(tx *sql.Tx) error {
_, err := tx.ExecContext(ctx, "INSERT INTO connector_check VALUES (1)")
return err
}); err != nil {
t.Fatal(err)
}
rollbackErr := errors.New("rollback requested")
if err := first.InTx(ctx, nil, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, "INSERT INTO connector_check VALUES (2)"); err != nil {
return err
}
return rollbackErr
}); !errors.Is(err, rollbackErr) {
t.Fatal(err)
}
var count int
if err := second.QueryRowContext(ctx, "SELECT COUNT(*) FROM connector_check").Scan(&count); err != nil || count != 1 {
t.Fatalf("transaction count=%d err=%v", count, err)
}
if err := first.Close(); err != nil {
t.Fatal(err)
}
if err := second.Health(ctx); err != nil {
t.Fatal("closing first pool affected second:", err)
}
}

View File

@@ -0,0 +1,29 @@
// Package ids demonstrates an application-owned scalar. The framework does not
// impose this ID policy on any other type or project.
package ids
import (
"encoding/json"
"strconv"
)
type ID int64
func (id ID) MarshalJSON() ([]byte, error) { return json.Marshal(strconv.FormatInt(int64(id), 10)) }
func (id *ID) UnmarshalJSON(raw []byte) error {
value := string(raw)
if len(raw) > 0 && raw[0] == '"' {
if err := json.Unmarshal(raw, &value); err != nil {
return err
}
}
return id.UnmarshalText([]byte(value))
}
func (id *ID) UnmarshalText(raw []byte) error {
value, err := strconv.ParseInt(string(raw), 10, 64)
if err != nil {
return err
}
*id = ID(value)
return nil
}

View File

@@ -0,0 +1,75 @@
// This example only uses local HTTP and bounded streaming; it has no database.
package main
import (
"io"
"log"
"net/http"
"strings"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
type echoRequest struct {
ID ids.ID `json:"id"`
Data any `json:"data"`
}
func main() {
app := omni.New(omni.WithLogger(true))
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Example", "compatibility")
next.ServeHTTP(w, r)
})
}))
app.GET("/legacy", httpx.WrapHandler(http.HandlerFunc(http.NotFound)))
binder := contract.MustNew[echoRequest](contract.ScalarOf[ids.ID]("string|int64", "string", true))
app.POST("/echo", func(c *httpx.Context) error {
var req echoRequest
if err := binder.Bind(c.Request(), nil, &req); err != nil {
return err
}
return c.OK(req)
})
app.HandleWithBodyLimit("POST", "/upload", 10<<20, nil, func(c *httpx.Context) error {
reader, err := c.Request().MultipartReader()
if err != nil {
return omni.BadRequest("INVALID_MULTIPART", "multipart body required")
}
var total int64
for {
part, err := reader.NextPart()
if err == io.EOF {
break
}
if err != nil {
return err
}
// Replace io.Discard with a streaming storage writer. Do not trust file names
// as filesystem paths. io.Copy never buffers the complete uploaded file.
n, copyErr := io.Copy(io.Discard, part)
closeErr := part.Close()
if copyErr != nil {
return copyErr
}
if closeErr != nil {
return closeErr
}
total += n
}
return c.OK(map[string]int64{"bytes": total})
})
app.GET("/download", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", `attachment; filename="example.txt"`)
_, _ = io.Copy(w, strings.NewReader("A standard Handler keeps its original download response.\n"))
})))
if err := app.Run("127.0.0.1:8080"); err != nil {
log.Fatal(err)
}
}

View File

@@ -0,0 +1,64 @@
syntax = "v1"
scalar BusinessID (
goImport: "git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids"
goType: ID
jsonInput: string | int64
jsonOutput: string
text: true
)
type EchoReq {
ID BusinessID `json:"id"`
Data any `json:"data"`
Legacy interface{} `json:"legacy,optional"`
}
type EchoResp {
ID BusinessID `json:"id"`
Data any `json:"data"`
}
type ItemReq {
ID BusinessID `path:"id"`
}
type CustomReq {
Value string `json:"value"`
}
type CustomResp {
Value string `json:"value"`
}
@server (group: demo)
service compatibility-api {
@body (maxBytes: 2097152)
@handler echo
post /echo (EchoReq) returns (EchoResp)
@handler item
get /items/:id (ItemReq) returns (EchoResp)
@binding custom
@body (maxBytes: 512)
@handler custom
post /custom (CustomReq) returns (CustomResp)
@handlerMode manual
@body (mode: multipart
maxBytes: 65536)
@handler upload
post /upload
@handlerMode manual
@body (mode: raw
maxBytes: 128)
@produces "text/plain"
@handler callback
post /callback
}
@server (group: files)
service compatibility-api {
@handlerMode manual
@produces "application/octet-stream"
@handler download
get /download
}

View File

@@ -13,8 +13,8 @@
"internal/apierrors/errorsGen.go": "20e94bbc90ae9f4330de4c25b9a550026c4c352502719a643ede3e8912a6566e",
"internal/handler/openapiGen.go": "e4130f5c7494f7db4830bfef203fa8154a9cd087c95954f64c29d74693ead433",
"internal/handler/responseGen.go": "d742efc2a3d1d168fae7849e905297bd440c3d6b3802b6150f494c07e6097f4f",
"internal/handler/routesGen.go": "0ff429a8aa7f8643e0e844f7c43a63a1f82cca5ce6bfe917f06edd860ebe923f",
"internal/handler/user/handlersGen.go": "06f3c2798ff323812ff3eb2d8b933a569af3ce1b86a67b66036374add98c0d7e",
"internal/handler/routesGen.go": "4e0c6879cea40e9c0155439fa1fce2cd19b5f83ea6e304523c41f8dc9c3a28e0",
"internal/handler/user/handlersGen.go": "1808afc8e4e8ca9785a8552eff64fad39bd5fe4e6e92971f35afc9006e23d76e",
"internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
"internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876"
}

View File

@@ -17,7 +17,7 @@ func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) erro
if deps.Middlewares["Auth"] == nil {
return fmt.Errorf("missing middleware Auth")
}
h0 := h0.New(s0.New(deps))
h0 := h0.New(s0.New(deps), deps)
httpx.HandleWithResponse(router, "POST", "/api/v1/users", responseEncoder("", false), h0.CreateUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "GET", "/api/v1/users/:id", responseEncoder("", false), h0.GetUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "GET", "/api/v1/users", responseEncoder("", false), h0.ListUsers, deps.Middlewares["Auth"])

View File

@@ -3,6 +3,7 @@ package user
import (
"context"
"example.com/userService/internal/servicecontext"
"example.com/userService/internal/types"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
@@ -17,9 +18,18 @@ type Service interface {
DeleteUser(ctx context.Context, req *types.UserIDReq) error
ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error)
}
type Handler struct{ service Service }
type Handler struct {
service Service
deps *servicecontext.Context
}
func New(s Service) *Handler { return &Handler{service: s} }
func New(s Service, deps ...*servicecontext.Context) *Handler {
h := &Handler{service: s}
if len(deps) > 0 {
h.deps = deps[0]
}
return h
}
var createUserRequest = contract.MustNew[types.CreateUserReq]()

60
httpx/body.go Normal file
View File

@@ -0,0 +1,60 @@
package httpx
import (
"errors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/response"
"net/http"
)
// LimitBody is useful for handwritten routes. Built-in App.HandleWithBodyLimit
// additionally installs the reader cap before global middleware can read it.
func LimitBody(maxBytes int64) Middleware {
if maxBytes <= 0 || maxBytes > 1<<40 {
panic("httpx: body limit must be 1..1 TiB")
}
return func(next Handler) Handler {
return func(c *Context) error {
r := c.Request()
if r.ContentLength > maxBytes {
return bodyTooLarge()
}
if r.Body != nil {
r.Body = http.MaxBytesReader(c.Raw(), r.Body, maxBytes)
}
defer CleanupMultipart(r)
err := next(c)
var limitErr *http.MaxBytesError
if errors.As(err, &limitErr) {
return bodyTooLarge()
}
return err
}
}
}
func bodyTooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
// CleanupMultipart removes disk spill files created by ParseMultipartForm.
// Callers still close individual multipart files. Streaming MultipartReader
// handlers own any files they create themselves.
func CleanupMultipart(r *http.Request) {
if r != nil && r.MultipartForm != nil {
_ = r.MultipartForm.RemoveAll()
}
}
type BodyLimitRegistrar interface {
HandleWithBodyLimit(string, string, int64, response.Encoder, Handler, ...Middleware)
}
func HandleWithBodyLimit(router interface {
Handle(string, string, Handler, ...Middleware)
}, method, path string, maxBytes int64, encoder response.Encoder, handler Handler, middleware ...Middleware) {
if r, ok := router.(BodyLimitRegistrar); ok {
r.HandleWithBodyLimit(method, path, maxBytes, encoder, handler, middleware...)
return
}
// Custom registrars without the early hook cap at route middleware entry.
all := append([]Middleware{LimitBody(maxBytes)}, middleware...)
HandleWithResponse(router, method, path, encoder, handler, all...)
}

80
httpx/bridge.go Normal file
View File

@@ -0,0 +1,80 @@
package httpx
import (
"context"
"net/http"
)
type bridgeState struct{ parent *Context }
// WrapHandler runs an existing standard handler inside the App middleware chain.
// It preserves the original response, including downloads and streaming output.
func WrapHandler(handler http.Handler) Handler {
if handler == nil {
panic("httpx: nil HTTP handler")
}
return func(c *Context) error {
r := bridgeRequest(c)
defer CleanupMultipart(r)
handler.ServeHTTP(c.Raw(), r)
return nil
}
}
// WrapMiddleware adapts synchronous standard HTTP middleware. The middleware is
// constructed once per chain, preserving its shared state across requests.
// Errors from next are encoded before unwinding so response-transforming
// middleware sees them. Preserve the incoming request context (derive with
// WithContext), and never call next after ServeHTTP returns, as with net/http.
func WrapMiddleware(middleware func(http.Handler) http.Handler) Middleware {
if middleware == nil {
panic("httpx: nil HTTP middleware")
}
return func(next Handler) Handler {
// Each nested bridge has its own key, including repeated use of one adapter.
key := &bridgeState{}
wrapped := middleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
state, ok := r.Context().Value(key).(*bridgeState)
if !ok {
panic("httpx: standard middleware must preserve the incoming request context")
}
c := state.parent
child := c.Clone(r.Context(), w)
child.request = r
if c.Responded() {
child.writer.(interface{ inheritCommit(int) }).inheritCommit(c.Status())
}
defer CleanupMultipart(r)
defer func() { c.values = child.values; c.encoder = child.encoder }()
if err := next(child); err != nil && !c.Responded() {
child.WriteError(err)
}
}))
if wrapped == nil {
panic("httpx: HTTP middleware returned nil handler")
}
return func(c *Context) error {
r := bridgeRequest(c).WithContext(context.WithValue(c.Context(), key, &bridgeState{parent: c}))
defer CleanupMultipart(r)
wrapped.ServeHTTP(c.Raw(), r)
return nil
}
}
}
// WrapFuncMiddleware supports middleware shaped as func(http.HandlerFunc) http.HandlerFunc.
func WrapFuncMiddleware(middleware func(http.HandlerFunc) http.HandlerFunc) Middleware {
if middleware == nil {
panic("httpx: nil HTTP middleware")
}
return WrapMiddleware(func(next http.Handler) http.Handler { return middleware(next.ServeHTTP) })
}
// Populate Go 1.22 path values without mutating a parent router's request maps.
func bridgeRequest(c *Context) *http.Request {
r := c.Request().Clone(c.Context())
for name, value := range c.params {
r.SetPathValue(name, value)
}
return r
}

363
httpx/bridge_test.go Normal file
View File

@@ -0,0 +1,363 @@
package httpx_test
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"mime/multipart"
"net"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
func TestLegacyResponseTrackingAndMiddleware(t *testing.T) {
var logs bytes.Buffer
closeLog, err := logx.Setup(logx.Config{Writer: &logs, Format: "json"})
if err != nil {
t.Fatal(err)
}
defer closeLog()
app := omni.New(omni.WithLogger(true))
app.GET("/legacy", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { http.NotFound(w, r) })))
app.GET("/late", func(c *httpx.Context) error {
c.Raw().WriteHeader(404)
_, _ = c.Raw().Write([]byte("original"))
return errors.New("late error")
})
app.GET("/panic", func(c *httpx.Context) error { _, _ = c.Raw().Write([]byte("partial")); panic("late panic") })
for _, tc := range []struct {
path string
status int
body string
}{{"/legacy", 404, "404 page not found\n"}, {"/late", 404, "original"}, {"/panic", 200, "partial"}} {
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", tc.path, nil))
if w.Code != tc.status || w.Body.String() != tc.body {
t.Fatalf("%s: %d %q", tc.path, w.Code, w.Body.String())
}
}
count := 0
for _, line := range strings.Split(logs.String(), "\n") {
var e map[string]any
if json.Unmarshal([]byte(line), &e) != nil || e["msg"] != "http.request" {
continue
}
count++
if e["path"] != "/panic" && e["status"] != float64(404) {
t.Fatalf("wrong log: %s", line)
}
}
if count != 3 {
t.Fatalf("access logs: %d", count)
}
type key struct{}
app = omni.New()
app.Use(httpx.WrapFuncMiddleware(func(next http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Before", "yes")
next(w, r.WithContext(context.WithValue(r.Context(), key{}, "identity")))
}
}))
app.GET("/context", func(c *httpx.Context) error {
if c.Context().Value(key{}) != "identity" || c.RequestID() == "" {
t.Error("lost context")
}
return omni.NotFound("MISSING", "missing")
})
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", "/context", nil))
if w.Code != 404 || w.Header().Get("X-Before") != "yes" || strings.Count(w.Body.String(), "MISSING") != 1 {
t.Fatalf("middleware response: %d %s", w.Code, w.Body)
}
}
type plainWriter struct {
header http.Header
status int
}
func (w *plainWriter) Header() http.Header { return w.header }
func (w *plainWriter) WriteHeader(status int) { w.status = status }
func (w *plainWriter) Write(p []byte) (int, error) { return len(p), nil }
func TestWriterCapabilitiesAndInformationalStatus(t *testing.T) {
w := &plainWriter{header: http.Header{}}
c := httpx.NewContext(httptest.NewRequest("GET", "/", nil), w, nil)
if _, ok := c.Raw().(http.Flusher); ok {
t.Fatal("invented Flusher")
}
if _, ok := c.Raw().(http.Hijacker); ok {
t.Fatal("invented Hijacker")
}
if err := http.NewResponseController(c.Raw()).Flush(); !errors.Is(err, http.ErrNotSupported) {
t.Fatalf("Flush: %v", err)
}
c.Raw().WriteHeader(103)
if c.Responded() {
t.Fatal("103 committed final response")
}
c.Raw().WriteHeader(201)
c.Raw().WriteHeader(500)
if !c.Responded() || c.Status() != 201 || w.status != 201 {
t.Fatal("first final status not preserved")
}
recorder := httptest.NewRecorder()
c = httpx.NewContext(httptest.NewRequest("GET", "/", nil), recorder, nil)
if err := http.NewResponseController(c.Raw()).Flush(); err != nil {
t.Fatal(err)
}
if !c.Responded() || c.Status() != 200 || !recorder.Flushed {
t.Fatal("flush not tracked")
}
}
func TestLegacyStreamingAcrossTransports(t *testing.T) {
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative} {
t.Run(transport, func(t *testing.T) {
release := make(chan struct{})
defer close(release)
app := omni.New(omni.WithTransport(transport))
app.GET("/stream", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/plain")
_, _ = io.WriteString(w, "first\n")
if err := http.NewResponseController(w).Flush(); err != nil {
t.Error(err)
return
}
select {
case <-release:
case <-r.Context().Done():
}
})))
app.GET("/file", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Disposition", `attachment; filename="data.bin"`)
_, _ = io.Copy(w, strings.NewReader(strings.Repeat("x", 2<<20)))
})))
app.HandleWithBodyLimit("POST", "/bounded", 32, nil, func(c *httpx.Context) error {
_, err := io.Copy(io.Discard, c.Request().Body)
if err != nil {
return err
}
return c.NoContent()
})
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- app.ServeContext(ctx, listener) }()
defer func() {
cancel()
select {
case <-done:
case <-time.After(15 * time.Second):
t.Error("shutdown timed out")
}
}()
client := &http.Client{Timeout: 3 * time.Second}
defer client.CloseIdleConnections()
for _, body := range []io.Reader{strings.NewReader(strings.Repeat("x", 33)), io.NopCloser(strings.NewReader(strings.Repeat("x", 33)))} {
res, err := client.Post("http://"+listener.Addr().String()+"/bounded", "application/octet-stream", body)
if err != nil {
t.Fatal(err)
}
_, _ = io.Copy(io.Discard, res.Body)
res.Body.Close()
if res.StatusCode != 413 {
t.Fatalf("real transport body limit: %d", res.StatusCode)
}
}
res, err := client.Get("http://" + listener.Addr().String() + "/stream")
if err != nil {
t.Fatal(err)
}
first := make([]byte, 6)
_, err = io.ReadFull(res.Body, first)
res.Body.Close()
if err != nil || string(first) != "first\n" {
t.Fatalf("stream was buffered: %s %v", first, err)
}
res, err = client.Get("http://" + listener.Addr().String() + "/file")
if err != nil {
t.Fatal(err)
}
size, err := io.Copy(io.Discard, res.Body)
res.Body.Close()
if err != nil || size != 2<<20 || res.Header.Get("Content-Disposition") == "" {
t.Fatalf("file: %d %v", size, err)
}
})
}
}
func TestStandardMiddlewareStateAndShortCircuit(t *testing.T) {
constructions, calls := 0, 0
app := omni.New()
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
constructions++
count := 0
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
count++
if count == 2 {
http.Error(w, "denied", 401)
return
}
next.ServeHTTP(w, r)
})
}))
app.GET("/", func(c *httpx.Context) error { calls++; return c.NoContent() })
for _, status := range []int{204, 401, 204} {
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
if w.Code != status {
t.Fatalf("got %d want %d", w.Code, status)
}
}
if constructions != 1 || calls != 2 {
t.Fatalf("middleware lifecycle: %d constructions, %d calls", constructions, calls)
}
}
func TestStandardMiddlewareTransformsErrorAndPreservesCancellation(t *testing.T) {
app := omni.New()
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// A legacy middleware may buffer its own small response. The bridge must
// report its eventual physical status, and encode errors before it unwinds.
temp := httptest.NewRecorder()
next.ServeHTTP(temp, r)
if temp.Code != 404 {
t.Errorf("inner error not encoded: %d", temp.Code)
}
w.WriteHeader(409)
_, _ = io.WriteString(w, strings.ToUpper(temp.Body.String()))
})
}))
app.GET("/", func(c *httpx.Context) error {
if !errors.Is(c.Context().Err(), context.Canceled) {
t.Error("cancellation lost")
}
return omni.NotFound("missing", "missing")
})
ctx, cancel := context.WithCancel(context.Background())
cancel()
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", "/", nil).WithContext(ctx))
if w.Code != 409 || !strings.Contains(w.Body.String(), "MISSING") {
t.Fatalf("%d %s", w.Code, w.Body)
}
}
func TestRouteBodyLimitBeforeMiddleware(t *testing.T) {
for _, length := range []int64{-1, 33} {
t.Run(fmt.Sprint(length), func(t *testing.T) {
app := omni.New()
defer app.Close()
read := 0
app.Use(func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
body, err := io.ReadAll(c.Request().Body)
read = len(body)
if err != nil {
return err
}
return next(c)
}
})
app.Group("/api").HandleWithBodyLimit("POST", "/upload", 32, nil, func(c *httpx.Context) error { t.Error("called oversized handler"); return nil })
r := httptest.NewRequest("POST", "/api/upload", strings.NewReader(strings.Repeat("x", 33)))
r.ContentLength = length
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 413 || read > 32 {
t.Fatalf("limit: status=%d read=%d", w.Code, read)
}
})
}
}
func TestMultipartCleanupOnPanic(t *testing.T) {
temp := t.TempDir()
t.Setenv("TMPDIR", temp)
var body bytes.Buffer
form := multipart.NewWriter(&body)
part, err := form.CreateFormFile("file", "x.bin")
if err != nil {
t.Fatal(err)
}
_, _ = part.Write(bytes.Repeat([]byte("x"), 4096))
_ = form.Close()
app := omni.New()
defer app.Close()
app.HandleWithBodyLimit("POST", "/", 8192, nil, httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if err := r.ParseMultipartForm(1); err != nil {
t.Error(err)
return
}
panic("after disk spill")
})))
r := httptest.NewRequest("POST", "/", &body)
r.Header.Set("Content-Type", form.FormDataContentType())
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
entries, err := os.ReadDir(temp)
if w.Code != 500 || err != nil || len(entries) != 0 {
t.Fatalf("cleanup: %d %v %v", w.Code, entries, err)
}
}
func TestStandardMiddlewareCommitsBeforeCallingNext(t *testing.T) {
app := omni.New()
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(202)
_, _ = io.WriteString(w, "accepted")
next.ServeHTTP(w, r)
})
}))
app.GET("/", func(c *httpx.Context) error {
if !c.Responded() || c.Status() != 202 {
t.Error("lost middleware commit")
}
return errors.New("late failure")
})
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
if w.Code != 202 || w.Body.String() != "accepted" {
t.Fatalf("late append: %d %q", w.Code, w.Body.String())
}
}
func TestBridgeStandardPathValues(t *testing.T) {
app := omni.New()
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.PathValue("itemId") != "abc" {
t.Error("middleware path value lost")
}
next.ServeHTTP(w, r)
})
}))
app.GET("/items/:itemId", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _, _ = io.WriteString(w, r.PathValue("itemId")) })))
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", "/items/abc", nil))
if w.Body.String() != "abc" {
t.Fatal("handler path value lost")
}
}

View File

@@ -3,6 +3,7 @@ package httpx
import (
"context"
"encoding/json"
"errors"
"net/http"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
@@ -101,12 +102,15 @@ func NewContext(request *http.Request, writer http.ResponseWriter, params map[st
if request == nil {
request = &http.Request{}
}
return &Context{
c := &Context{
request: request,
writer: writer,
params: params,
values: make(map[string]any),
}
if writer != nil {
c.writer = trackWriter(writer, c)
}
return c
}
func (c *Context) Request() *http.Request { return c.request }
@@ -120,8 +124,8 @@ func (c *Context) Context() context.Context {
return c.request.Context()
}
// Raw returns the response writer supplied by the active network adapter.
// Prefer the transport-neutral methods on Context for portable handlers.
// Raw returns a streaming response writer that tracks commits and status.
// Optional HTTP capabilities are preserved only when the underlying writer supports them.
func (c *Context) Raw() http.ResponseWriter { return c.writer }
// Clone gives a buffering middleware its own response state and values map.
@@ -146,7 +150,6 @@ func (c *Context) WriteResponse(status int, headers http.Header, body []byte) er
for key, values := range headers {
c.writer.Header()[key] = append([]string(nil), values...)
}
c.responded, c.status = true, status
c.writer.WriteHeader(status)
if len(body) > 0 {
if _, err := c.writer.Write(body); err != nil {
@@ -216,8 +219,6 @@ func (c *Context) JSON(status int, value any) error {
if err != nil {
return apierrors.Internal(err)
}
c.responded = true
c.status = status
c.writer.Header().Set("Content-Type", "application/json; charset=utf-8")
c.writer.WriteHeader(status)
if _, err := c.writer.Write(append(data, '\n')); err != nil {
@@ -243,8 +244,6 @@ func (c *Context) NoContent() error {
if c.writer == nil {
return apierrors.Internal(nil)
}
c.responded = true
c.status = http.StatusNoContent
c.writer.WriteHeader(http.StatusNoContent)
return nil
}
@@ -271,12 +270,15 @@ func (c *Context) Status() int {
func (c *Context) Responded() bool { return c.responded }
// WriteError writes a stable public error response. Internal causes are not
// serialized. Responses committed through Context methods are preserved;
// direct Raw writes are outside this response-tracking contract.
// serialized. Responses committed through Context methods or Raw are preserved.
func (c *Context) WriteError(err error) {
if c.responded || err == nil {
return
}
var bodyErr *http.MaxBytesError
if errors.As(err, &bodyErr) {
err = bodyTooLarge()
}
apiErr := apierrors.From(err)
if writeErr := c.writeResult(response.Result{
Status: apiErr.Status,

315
httpx/trackingWriter.go Normal file
View File

@@ -0,0 +1,315 @@
package httpx
import (
"bufio"
"io"
"net"
"net/http"
)
// trackingWriter does not buffer response bodies. Each wrapper has its own
// commit flag, so a standard middleware may safely wrap an already tracked writer.
type trackingWriter struct {
http.ResponseWriter
context *Context
committed bool
}
func (w *trackingWriter) Unwrap() http.ResponseWriter { return w.ResponseWriter }
func (w *trackingWriter) inheritCommit(status int) { w.mark(status) }
func (w *trackingWriter) mark(status int) {
w.committed = true
w.context.responded = true
w.context.status = status
}
func (w *trackingWriter) WriteHeader(status int) {
if w.committed {
return
}
// Let the server validate the status before updating the observation.
w.ResponseWriter.WriteHeader(status)
if status == 101 || status >= 200 {
w.mark(status)
}
}
func (w *trackingWriter) Write(p []byte) (int, error) {
if !w.committed {
w.mark(http.StatusOK)
}
return w.ResponseWriter.Write(p)
}
type flushWriter struct{ *trackingWriter }
func (w flushWriter) Flush() { _ = w.FlushError() }
func (w flushWriter) FlushError() error {
if !w.committed {
w.WriteHeader(http.StatusOK)
}
return http.NewResponseController(w.ResponseWriter).Flush()
}
type hijackWriter struct{ *trackingWriter }
func (w hijackWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
conn, rw, err := w.ResponseWriter.(http.Hijacker).Hijack()
if err == nil {
w.context.responded = true
w.committed = true
}
return conn, rw, err
}
type pushWriter struct{ *trackingWriter }
func (w pushWriter) Push(target string, options *http.PushOptions) error {
return w.ResponseWriter.(http.Pusher).Push(target, options)
}
type readFromWriter struct{ *trackingWriter }
func (w readFromWriter) ReadFrom(r io.Reader) (int64, error) {
// A writer-only view makes every write observable, including partial failures,
// while io.Copy retains bounded streaming memory and source WriterTo support.
return io.Copy(struct{ io.Writer }{w.trackingWriter}, r)
}
type closeNotifyWriter struct{ *trackingWriter }
func (w closeNotifyWriter) CloseNotify() <-chan bool {
return w.ResponseWriter.(http.CloseNotifier).CloseNotify()
}
func trackWriter(writer http.ResponseWriter, c *Context) http.ResponseWriter {
w := &trackingWriter{ResponseWriter: writer, context: c}
mask := 0
if _, ok := writer.(http.Flusher); ok {
mask |= 1
} else if _, ok := writer.(interface{ FlushError() error }); ok {
mask |= 1
}
if _, ok := writer.(http.Hijacker); ok {
mask |= 2
}
if _, ok := writer.(http.Pusher); ok {
mask |= 4
}
if _, ok := writer.(io.ReaderFrom); ok {
mask |= 8
}
if _, ok := writer.(http.CloseNotifier); ok {
mask |= 16
}
switch mask {
case 1:
return struct {
*trackingWriter
flushCapability
}{w, flushWriter{w}}
case 2:
return struct {
*trackingWriter
http.Hijacker
}{w, hijackWriter{w}}
case 3:
return struct {
*trackingWriter
flushCapability
http.Hijacker
}{w, flushWriter{w}, hijackWriter{w}}
case 4:
return struct {
*trackingWriter
http.Pusher
}{w, pushWriter{w}}
case 5:
return struct {
*trackingWriter
flushCapability
http.Pusher
}{w, flushWriter{w}, pushWriter{w}}
case 6:
return struct {
*trackingWriter
http.Hijacker
http.Pusher
}{w, hijackWriter{w}, pushWriter{w}}
case 7:
return struct {
*trackingWriter
flushCapability
http.Hijacker
http.Pusher
}{w, flushWriter{w}, hijackWriter{w}, pushWriter{w}}
case 8:
return struct {
*trackingWriter
io.ReaderFrom
}{w, readFromWriter{w}}
case 9:
return struct {
*trackingWriter
flushCapability
io.ReaderFrom
}{w, flushWriter{w}, readFromWriter{w}}
case 10:
return struct {
*trackingWriter
http.Hijacker
io.ReaderFrom
}{w, hijackWriter{w}, readFromWriter{w}}
case 11:
return struct {
*trackingWriter
flushCapability
http.Hijacker
io.ReaderFrom
}{w, flushWriter{w}, hijackWriter{w}, readFromWriter{w}}
case 12:
return struct {
*trackingWriter
http.Pusher
io.ReaderFrom
}{w, pushWriter{w}, readFromWriter{w}}
case 13:
return struct {
*trackingWriter
flushCapability
http.Pusher
io.ReaderFrom
}{w, flushWriter{w}, pushWriter{w}, readFromWriter{w}}
case 14:
return struct {
*trackingWriter
http.Hijacker
http.Pusher
io.ReaderFrom
}{w, hijackWriter{w}, pushWriter{w}, readFromWriter{w}}
case 15:
return struct {
*trackingWriter
flushCapability
http.Hijacker
http.Pusher
io.ReaderFrom
}{w, flushWriter{w}, hijackWriter{w}, pushWriter{w}, readFromWriter{w}}
case 16:
return struct {
*trackingWriter
http.CloseNotifier
}{w, closeNotifyWriter{w}}
case 17:
return struct {
*trackingWriter
flushCapability
http.CloseNotifier
}{w, flushWriter{w}, closeNotifyWriter{w}}
case 18:
return struct {
*trackingWriter
http.Hijacker
http.CloseNotifier
}{w, hijackWriter{w}, closeNotifyWriter{w}}
case 19:
return struct {
*trackingWriter
flushCapability
http.Hijacker
http.CloseNotifier
}{w, flushWriter{w}, hijackWriter{w}, closeNotifyWriter{w}}
case 20:
return struct {
*trackingWriter
http.Pusher
http.CloseNotifier
}{w, pushWriter{w}, closeNotifyWriter{w}}
case 21:
return struct {
*trackingWriter
flushCapability
http.Pusher
http.CloseNotifier
}{w, flushWriter{w}, pushWriter{w}, closeNotifyWriter{w}}
case 22:
return struct {
*trackingWriter
http.Hijacker
http.Pusher
http.CloseNotifier
}{w, hijackWriter{w}, pushWriter{w}, closeNotifyWriter{w}}
case 23:
return struct {
*trackingWriter
flushCapability
http.Hijacker
http.Pusher
http.CloseNotifier
}{w, flushWriter{w}, hijackWriter{w}, pushWriter{w}, closeNotifyWriter{w}}
case 24:
return struct {
*trackingWriter
io.ReaderFrom
http.CloseNotifier
}{w, readFromWriter{w}, closeNotifyWriter{w}}
case 25:
return struct {
*trackingWriter
flushCapability
io.ReaderFrom
http.CloseNotifier
}{w, flushWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 26:
return struct {
*trackingWriter
http.Hijacker
io.ReaderFrom
http.CloseNotifier
}{w, hijackWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 27:
return struct {
*trackingWriter
flushCapability
http.Hijacker
io.ReaderFrom
http.CloseNotifier
}{w, flushWriter{w}, hijackWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 28:
return struct {
*trackingWriter
http.Pusher
io.ReaderFrom
http.CloseNotifier
}{w, pushWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 29:
return struct {
*trackingWriter
flushCapability
http.Pusher
io.ReaderFrom
http.CloseNotifier
}{w, flushWriter{w}, pushWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 30:
return struct {
*trackingWriter
http.Hijacker
http.Pusher
io.ReaderFrom
http.CloseNotifier
}{w, hijackWriter{w}, pushWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
case 31:
return struct {
*trackingWriter
flushCapability
http.Hijacker
http.Pusher
io.ReaderFrom
http.CloseNotifier
}{w, flushWriter{w}, hijackWriter{w}, pushWriter{w}, readFromWriter{w}, closeNotifyWriter{w}}
}
return w
}
type flushCapability interface {
Flush()
FlushError() error
}

View File

@@ -30,7 +30,7 @@ if err != nil { return err }
defer closeLog()
```
也可用 `omni.WithLogConfig(...)` 交给应用运行生命周期管理。子日志共用输出、级别和关闭状态。独立 `logx.New` 返回的实例由创建者负责关闭。
`omni.WithLogConfig(...)` 只保存日志配置,`omni.New()` 不会因此创建或关闭 Logger。手写入口需要像上例一样调用 `logx.Setup` 并负责关闭,或使用 `logx.New`、`logx.SetDefault` 和 `app.OnShutdown(logger.Close)` 显式装配。生成的 API 项目已在入口完成这些步骤。子日志共用输出、级别和关闭状态;独立 `logx.New` 返回的实例由创建者负责关闭。
轮转显式开启;三个数值为 0 时分别取 100 MiB、10 份、7 天。超过大小后轮转,旧文件按份数/年龄清理,可 gzip 压缩;保留天数不表示每天定时切割。压缩和历史清理由底层异步执行,不承诺 Close 等待压缩完成。轮转使用 [lumberjack v2](https://github.com/natefinch/lumberjack/tree/v2.2.1),**一个日志文件只能由一个进程拥有**。容器通常直接用 stdout/stderr,让运行平台管理日志文件。

View File

@@ -26,6 +26,7 @@ type HTTPRouterGroup = httpx.RouterGroup
type APIError = apierrors.APIError
type Config = config.Config
type TLSConfig = config.TLSConfig
type HTTPConfig = config.HTTPConfig
type Option = config.Option
type Response = response.Response
type LogConfig = logx.Config
@@ -62,6 +63,8 @@ func WithConfig(settings Config) Option { return config.WithConfig(settings) }
func WithTransport(name string) Option { return config.WithTransport(name) }
func WithHTTPConfig(settings HTTPConfig) Option { return config.WithHTTPConfig(settings) }
func WithTLS(certFile, keyFile string) Option { return config.WithTLS(certFile, keyFile) }
func WithLogConfig(settings LogConfig) Option { return config.WithLogConfig(settings) }

View File

@@ -0,0 +1,32 @@
# v0.9.0-preview 迁移扩展验收
本轮只使用本机临时服务与生成项目;没有运行业务服务器压测,没有修改 ePay/eTax 业务代码。
## 本地验收
- `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...`:40 个有测试的包通过;包含真实生成消费者、标准 Handler/Middleware、动态 JSON、路由限额和 scalar。
- `go vet ./...`、`go build ./...`、gofmt、`git diff --check` 通过。
- 独立 `examples/userService` module 的 race/vet/build 通过;生成文件 `--check` 通过,手写业务保持原样。
- nethttp/native 本机真实 TCP 测试:首块数据立即 Flush、2 MiB 文件保持原响应、已知长度和 chunked 超限返回 413,并能继续处理正常请求。
- 404 访问日志、提交后错误/panic 不追加 JSON、标准中间件提前写入/跨请求状态/取消/PathValue 通过。
- multipart 落盘临时文件在成功和 panic 后清理;原始请求体 HMAC 验签、定制绑定和生成代码保护通过。
- scalar 最大 int64、数字/字符串兼容、溢出/错误类型拒绝、指针 null、query 集合、输入/输出 schema 分离通过;普通字段仍严格。
- ePay 原始契约只读校验:26 types、43 routes、0 errors、0 auth schemes。后两项是声明数量,不代表无需鉴权或没有业务错误;不代表 144 条注册路由已全部迁移。
环境:`go version go1.27.1 darwin/arm64`。记录时间:2026-09-23T17:47:26+00:00。
## 远程验收
已执行 `scripts/verifyCompatibilityRelease.py --version v0.9.0-preview`,通过。详见 [远程验收记录](compatibilityRemote.json)。
- 从全新 module cache 安装发布版 CLI,消费者固定依赖 `v0.9.0-preview`,无本地 replace。
- 消费者实际生成、race 测试、自定义 ID/动态 JSON、请求体限额、上传清理、原文签名、下载和重生成保护通过。
- 发布源码:`a8c973d8a2c24f3cc572f32a976349d9a97427a1`。
- module 校验和:`h1:cYpW6HwZnLHsxq7thO1C2pR/H3vyoUQQhuBVuEHhtHw=`。
- 耗时 13.51 秒;临时目录和缓存已清理。脚本与 fixture 哈希保存在 JSON 中。
验收结果在 main 单独记录,发布标签保持不可变。
## 范围
本次不是完整 ePay 迁移验收;资金事务、原权限体系、业务响应细节仍需业务自己的回归。未重跑 Linux soak、Nginx/Cloudflare 代理或云存储账号实网验收。

View File

@@ -0,0 +1,31 @@
{
"passed": true,
"frameworkVersion": "v0.9.0-preview",
"frameworkSum": "h1:cYpW6HwZnLHsxq7thO1C2pR/H3vyoUQQhuBVuEHhtHw=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 13.51,
"checks": [
"remoteCLIInstall",
"scalarContractValidation",
"manualProjectGeneration",
"int64MaxValueOnWire",
"dynamicJSON",
"rawBodySignature",
"routeBodyLimits",
"multipartTempCleanup",
"unwrappedDownloads",
"handwrittenFilesPreserved",
"generatedConsumerRaceTests",
"fixedRemoteModuleChecksum"
],
"sourceHashes": {
"scripts/verifyCompatibilityRelease.py": "90e487d3203ed4eb4bc721da8086356413345b3e05193e473250a834aeb8d293",
"cmd/go-omni/apigen/compatibility_test.go": "98cf5fbe6cb6413c12f709806d385726f3dec1468575f76967786de4707667ba",
"examples/contracts/compatibility.api": "3ede909e7f60386012e55c916a8e4887dd08562e7f023665f663288c809873d4",
"cmd/go-omni/apigen/testdata/compatibility/consumer_test.go": "0a99ffd6a5cc16d47ae8208d9140c8d6afd9c19106fac8e5e61096cb90d8c463",
"cmd/go-omni/apigen/testdata/compatibility/hooks.go": "435baf37c7f8adffd18d4be0353a791124e0bc5dcb79abdcc345276b6f9867f4",
"cmd/go-omni/apigen/testdata/compatibility/echo.go": "9e13cb18a0951974e723a1fd915c0f4610d823c95eadb752c3d1cd75c44fa293",
"cmd/go-omni/apigen/testdata/compatibility/download.go": "e7f4bcdb34c90ec63fdab1f3c4e524964ac153944352d8a9053a9f19b74030fd"
}
}

View File

@@ -0,0 +1,24 @@
# v0.9.1-preview Connector 与 HTTP 配置验收
## 范围
新增标准 `driver.Connector` 注入和 HTTP 服务器读写配置;没有修改 eTax/ePay 业务代码。数据库驱动/TLS/Logger/时区/版本校验仍归项目。网络测试只监听本地回环地址,MySQL 使用独立本地容器(1 CPU / 768 MiB / 128 PID),没有使用业务服务器压测。
## 全量回归
- `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...`:40 个有测试的包全部通过,包含真实脚手架/契约生成的集成消费者。
- `go vet ./...`、`go build ./...`、gofmt、`git diff --check` 通过。
## 专项验收
- `go test -race -count=3 ./database ./app ./config` 通过。
- 两个 Connector 并发读写保持独立;启动 Connect/Ping 与查询 context 保留,后台补连接遵循 database/sql 行为;最大连接数、禁用 idle、连接生命周期、健康检查等待上限及恢复验证通过。
- nil/带类型的 nil、无效参数、启动前取消不取得 Connector 所有权;Connect/Ping 失败与超时关闭池,清理错误和原错误可同时检查;成功关闭幂等,事务提交/错误/panic 回滚通过。
- 标准库服务器字段、ErrorLog 透传、无效配置启动前返回并清理资源、JSON/YAML/环境变量覆盖通过;不支持的独立 http2 选项明确拒绝。
- nethttp/native 的实际 TCP 慢请求头、慢 body、Keep-Alive 空闲关闭、Flush 后客户端停止读取触发写超时、超限头 431 通过。测试不假定两种网络层有完全相同的超时响应格式。
- `scripts/verifyConnector.py --local` 的独立消费者通过:真实 MySQL 8.4 TLS、错误 CA/主机名拒绝、不同 session time_zone、每个 Connector 的驱动 Logger、两池独立关闭、事务提交/回滚。所有专项测试无跳过,详情见 [本地记录](connectorLocal.json)。
- MySQL 镜像固定记录为 `sha256:1afd25fe334a1be64a3866495dd43a6bdba3c55dd1d6190ea12933163fe2843c`。临时容器、卷、证书、凭据及消费者目录已清理。
环境:`go version go1.27.1 darwin/arm64`。记录时间:2026-09-23T18:17:16.897323+00:00。
独立 `TransportHTTP2` 的逐请求读写时限未实现,本次没有进行 Linux soak、代理长测或完整业务迁移验收。

View File

@@ -0,0 +1,70 @@
{
"passed": true,
"frameworkVersion": "v0.9.1-preview",
"localSource": true,
"frameworkSum": null,
"freshModuleCache": false,
"noLocalReplace": false,
"mysqlImage": "sha256:1afd25fe334a1be64a3866495dd43a6bdba3c55dd1d6190ea12933163fe2843c",
"mysqlLimits": {
"cpus": 1,
"memoryMiB": 768,
"pids": 128
},
"durationSeconds": 9.61,
"tests": [
"TestHTTPFileAndEnvironment/server.json",
"TestHTTPFileAndEnvironment/server.yaml",
"TestHTTPFileAndEnvironment",
"TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUTinvalid",
"TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUT-1",
"TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUTinvalid",
"TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUT-1",
"TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUTinvalid",
"TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUT-1",
"TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUTinvalid",
"TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUT-1",
"TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTESinvalid",
"TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTES-1",
"TestHTTPInvalidAndUnsupportedSettings",
"TestHTTPConfiguredStandardServer",
"TestHTTPInvalidStartupReleasesResources",
"TestHTTPConfiguredNetworkLimits/nethttp/slowHeaders",
"TestHTTPConfiguredNetworkLimits/nethttp/slowBody",
"TestHTTPConfiguredNetworkLimits/nethttp/idle",
"TestHTTPConfiguredNetworkLimits/nethttp/writeBackpressure",
"TestConnectorIndependentPoolsAndContext",
"TestConnectorRejectsInvalidInputWithoutTakingOwnership",
"TestConnectorStartupFailuresReleaseResources/connect",
"TestConnectorStartupFailuresReleaseResources/ping",
"TestConnectorStartupFailuresReleaseResources",
"TestConnectorTimeoutBoundsConnectAndPing/connect",
"TestConnectorTimeoutBoundsConnectAndPing/ping",
"TestConnectorTimeoutBoundsConnectAndPing",
"TestConnectorHealthBoundsPoolWaitAndRecovers",
"TestConnectorIdleZeroAndLifetime/noIdle",
"TestConnectorIdleZeroAndLifetime/expired",
"TestConnectorIdleZeroAndLifetime",
"TestConnectorTransactionsKeepCommitAndRollbackSemantics",
"TestMySQLConnectorDriverLoggerIsolation",
"TestMySQLConnectorTLSAndPoolIsolation/untrustedCA",
"TestMySQLConnectorTLSAndPoolIsolation/wrongHostname",
"TestMySQLConnectorTLSAndPoolIsolation",
"TestHTTPConfiguredNetworkLimits/nethttp/headerLimit",
"TestHTTPConfiguredNetworkLimits/nethttp",
"TestHTTPConfiguredNetworkLimits/native/slowHeaders",
"TestHTTPConfiguredNetworkLimits/native/slowBody",
"TestHTTPConfiguredNetworkLimits/native/idle",
"TestHTTPConfiguredNetworkLimits/native/writeBackpressure",
"TestHTTPConfiguredNetworkLimits/native/headerLimit",
"TestHTTPConfiguredNetworkLimits/native",
"TestHTTPConfiguredNetworkLimits"
],
"sourceHashes": {
"database/connector_test.go": "0663dfc9794ba24130225fa3b6418a6d5f75d57d0f01686151f70c7ea3f592c0",
"database/mysqlConnector_test.go": "e1cdd11ef76b09f1332ba92bfdde5b1917d84c85cc42fac48f65a8b23f5e9720",
"app/httpConfig_test.go": "b060dc0b76fb8e5feebae7e3b12cb7bca845693fcc097e2a0496a5ba937c837a",
"config/http_test.go": "0cae3dcf213369cfaff5487bf630c2bc3ec097739ca1d40794124e69c3880db8",
"scripts/verifyConnector.py": "a0ba3fa29a4d1efcbed5dad82c09afb97d8ddecd904f20fee1e3aa74ec0f9e06"
}
}

View File

@@ -1,41 +1,40 @@
# 公共响应模板验收
# v0.8.0-preview 公共响应模板验收
版本 `v0.8.0-preview`。本报告记录实现验收与发布验证,旧版本标签保持不变。
已发布 `v0.8.0-preview`,固定标签提交为 `7670b222acbad18d1fd19d54bbbac38d9cf5030b`。发布后只在 main 更新验收工具与记录,没有移动标签。
## 实现范围
## 完成范围
- `.api` 定义具名公共响应体,可跨文件导入、分组引用、接口覆盖;业务类型放入 `${data}`。支持嵌套 JSON、独立 errorBody、固定字段与类型保持的占位符。
- 数字/字符串错误码、默认码、按逻辑码映射、消息映射、null/省略行为可配置;OpenAPI 同步结构、类型和真实错误示例。
- `@response raw` 成功返回原始业务 JSON,错误继承组级模板。文本、XML及特殊状态的回调通过 Go 编码器实现,不冒充 `.api` 已支持任意媒体类型。
- 应用之间的配置独立;创建/快照深复制;空对象模板序列化后保持有效,非法占位符、配置冲突、循环和过深结构拒绝。
- 接口策略在全局中间件前设置;提前拒绝、panic、Context 克隆和错误边界使用正确策略。显式 JSON/Text/XML/Bytes/NoContent 绕过包装,已提交响应不追加内容。
- 内置路由 404/405 使用应用默认格式。网络层进入应用前的 TLS/HTTP协议错误仍由 transport 处理。
- `.api` 具名公共体支持导入、分组引用、单接口覆盖、嵌套 JSON、独立 errorBody、常量与类型保持的占位符。业务返回类型放入 `${data}`。
- `int32` / `int64` / `string` 业务码、默认成功/失败码与提示、按逻辑码映射和 null/省略行为可配置;OpenAPI 同步结构、整数位宽和错误示例,旧 integer 写法保持兼容。
- `@response raw` 成功返回原始业务 JSON,错误继承组级模板。文本/XML及特殊状态回调使用 Go 编码器,由应用维护对应文档。
- 配置在创建与快照时复制,应用间独立;空模板往返序列化有效,非法占位符、冲突、越界和过深模板拒绝。
- 接口策略在全局中间件前生效,提前拒绝、panic、Context 克隆与错误边界使用正确格式;已提交响应不追加内容。
- 内置 App 的 404/405 使用应用策略;TLS/HTTP解析与连接级过载发生在进入应用之前,仍由 transport 处理。
## 验证结果
## 发布前验证
1. `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...`:40 个有测试的包全部通过。
2. 首轮 `-p=2` 的原有 S3 fixture 测试遇到 10 秒 deadline;单包复跑通过,随后上述串行全量复跑通过,未修改 S3 实现或放宽断言。
3. 专项集成生成临时消费者项目,执行真实生成路由和业务 Service:具名公共体、数字 401、路由覆盖、raw JSON、错误文档,以及重复生成业务文件保护通过。
4. App 在 nethttp/native 的真实 TCP 请求中验证配置响应、callback panic 和超时;并发应用配置隔离、克隆、全局中间件提前拒绝及已提交响应保护通过。
5. 原有 API 实际 HTTP 回归 54 项通过:starter 两种 transport 各 6 项,contract 两种 transport 各 21 项;见 [回归报告](responsePoliciesAPI.json)。
6. `go vet ./...`、`go build ./...`、示例 `.api` validate/inspect、差异空白检查通过。既有用户示例重新生成及 `--check` 通过,28 份原有业务文件保持不变;go.mod 显式升级目标版本并继续使用本地 replace。
- 最终代码 `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...`:40 个有测试的包全部通过,见 [封版测试](responseReleaseTests.txt)。
- `go vet ./...`、`go build ./...`、格式/差异检查和 userService `api generate --check` 通过。
- 临时生成项目执行真实路由和业务 Service,验证数字 401、raw JSON、路由覆盖、完整 int64 输出、OpenAPI 和重复生成保护;nethttp/native 真实 TCP 检查配置响应、callback panic 和超时。
- 本机隔离 MySQL 下,nethttp/native 共 34 项 HTTP/业务/生命周期检查通过;包含事务提交、错误/panic/取消回滚、初始化清理、迁移、日志与优雅退出,见 [本地业务报告](responsePoliciesWorkflowLocal.json)。
- 开发阶段额外完成 54 项 API 实际 HTTP 回归与类型专项检查,见 [API 回归](responsePoliciesAPI.json)、[类型专项](responseCodeTypesTests.txt)。这些历史报告分别记录当时的源码哈希。
- 开发阶段原有 S3 fixture 曾出现一次 10 秒 deadline,单包及后续全量复跑通过;本次封版全量也通过,未修改 S3 逻辑或放宽断言。
完整框架记录见 [race 测试](responsePoliciesTests.txt)。本轮仅使用本机临时目录和回环监听,没有连接业务服务器,没有执行远程云存储/数据库验收或压力长测。远程模块下载验收需发布后另行完成。
## 远程发布验证
## 使用入口
两个验收分别使用全新的 module cache 安装 `v0.8.0-preview`,实际消费者都无本地 replace,模块校验和一致:
[公共响应说明](../RESPONSE.md) 与 [可生成的 .api 示例](../examples/contracts/responses.api)。远程使用请安装同版本 CLI 和框架;源码开发可带 `--framework-dir`。
`h1:lB4rU4FPsoFr3IedKx/K9xvvsGBO5SO/qdjnjKJ2mvQ=`
## 后续:类型名与业务字段统一
1. [公共响应远程报告](responsePoliciesRemote.json):实际下载 CLI,解析 int32 示例,生成并运行公共模板消费者;完整 int64 最大值、数字鉴权错误、raw JSON、路由覆盖、业务文件保护、race 检查和固定版本依赖校验全部通过。
2. [完整业务远程报告](responsePoliciesWorkflowRemote.json):远程 CLI 新建 MySQL + local 项目并编译;用户示例在 nethttp/native 共 34 项 HTTP/业务/生命周期检查通过,事务/初始化清理测试同时通过,重生成保留 29 份开发者文件。
`codeType` 新增 `int32` / `int64`,保留 `string` 与旧 `integer`;int32/int64 使用各自有符号整数范围,默认码和映射都校验。OpenAPI 为 integer + 对应 format。配置 JSON/YAML、生成项目的 int64 数值保留完整精度。
首次公共响应远程验收从框架源码目录启动 CLI,触发原有本地开发检测并加入 replace;验收正确拒绝了这种结果。main 的测试工具已改为从独立临时目录启动,随后上述纯远程校验通过。此修复只涉及验收测试的工作目录,不改变已发布 CLI 或框架;复现远程验收请使用 main 的验收脚本和测试。
本次增量运行根包、response、config、httpx、openapi、CLI/apigen/scaffold 共 8 个包的 race 测试及生成消费者集成,全部通过。覆盖边界/越界、小数拒绝、int64 最大值真实输出、配置精度、平铺/嵌套文档和旧 integer 兼容。vet、全模块 build、示例 validate 与差异检查通过;见 [类型测试记录](responseCodeTypesTests.txt)。上方全量及 HTTP 报告是此前公共响应体验收,此次未重复远程或数据库验收。
报告中保留各自实际测试源码哈希,已逐项核对;通过 module version、checksum 和无 Replace 三项确认消费者使用的是固定发布模块。
## v0.8.0-preview 封版前验证
## 资源与使用
- 最终代码全量 race 通过:40 个有测试的包;包括源码 CLI 的生成消费者、完整 int64 输出和模板配置回归,见 [封版测试记录](responseReleaseTests.txt)。
- `go vet ./...`、`go build ./...`、格式检查、差异检查和 userService `api generate --check` 通过。
- 本机隔离 MySQL 下,nethttp/native 共 34 项 HTTP、业务和生命周期检查通过;数据库事务提交/错误回滚/panic回滚/取消回滚、初始化清理和迁移通过。
- 重生成保留 29 份开发者文件,见 [封版本地业务报告](responsePoliciesWorkflowLocal.json)。临时容器已清理。
- 发布后将使用 `verifyResponseRelease.py` 和 `verifyAPIWorkflow.py --version v0.8.0-preview` 验证真实远程 CLI、无本地 replace 的模块、公共模板及完整业务流程。标签不随验收记录变动。
只使用本机回环监听、临时目录和受限 MySQL 容器(1 CPU / 768 MiB),结束均清理,没有连接业务服务器。未执行 OSS/R2 云账号实网验收或网络压力长测。
[发布说明](../RELEASE.md)、[公共响应说明](../RESPONSE.md) 与 [完整 .api 示例](../examples/contracts/responses.api)。CLI 与框架依赖一起固定到 `v0.8.0-preview`;源码开发可使用 `--framework-dir`。

View File

@@ -0,0 +1,25 @@
{
"passed": true,
"frameworkVersion": "v0.8.0-preview",
"frameworkSum": "h1:lB4rU4FPsoFr3IedKx/K9xvvsGBO5SO/qdjnjKJ2mvQ=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 13.23,
"checks": [
"remoteCLIInstall",
"int32ContractValidation",
"templateProjectGeneration",
"int64MaxValueOnWire",
"numericAuthErrors",
"rawJSON",
"routeOverride",
"handwrittenFilesPreserved",
"generatedConsumerRaceTests",
"fixedRemoteModuleChecksum"
],
"sourceHashes": {
"scripts/verifyResponseRelease.py": "0cd47ba265ee6a9f545e1be321742aa5807dd5040522653011ff91653fc89ac1",
"cmd/go-omni/apigen/response_test.go": "bdaea21a94297607172ab173f20640aa5c27e2d7e417376e9fe695d4c02ba524",
"examples/contracts/responses.api": "d0fcc8d417a2c00bbea4e8d586379b5fd2443518817960f271804b3813e96eb1"
}
}

View File

@@ -0,0 +1,164 @@
{
"passed": true,
"frameworkVersion": "v0.8.0-preview",
"noLocalReplace": true,
"durationSeconds": 28.28,
"regenerationPreservedHandwrittenFiles": 29,
"groups": [
{
"transport": "nethttp",
"checks": [
"authBeforeService",
"readOnlyPermission403",
"emptyPageAndQueryDefaults",
"contractValidation",
"createReadNormalize",
"uniqueEmailConflict",
"updateAndPagination",
"createUpdateDeleteRollbackOnAuditFailure",
"storageUploadAndSignedDownload",
"signedDownloadTamperRejected",
"openAPIMatchesContract",
"declaredAuthAndBusinessErrorDocumentation",
"readinessOutageAndRecovery",
"persistenceAfterServiceRestart",
"deleteAndTransactionalAudit",
"gracefulStopAndListenerRelease",
"businessFileLogsAndSecretExclusion"
]
},
{
"transport": "native",
"checks": [
"authBeforeService",
"readOnlyPermission403",
"emptyPageAndQueryDefaults",
"contractValidation",
"createReadNormalize",
"uniqueEmailConflict",
"updateAndPagination",
"createUpdateDeleteRollbackOnAuditFailure",
"storageUploadAndSignedDownload",
"signedDownloadTamperRejected",
"openAPIMatchesContract",
"declaredAuthAndBusinessErrorDocumentation",
"readinessOutageAndRecovery",
"persistenceAfterServiceRestart",
"deleteAndTransactionalAudit",
"gracefulStopAndListenerRelease",
"businessFileLogsAndSecretExclusion"
]
}
],
"transactionTests": [
"commit",
"errorRollback",
"panicRollback",
"cancellationRollback"
],
"startupTests": [
"storageFailureClosesDatabase",
"idempotentClose",
"closedContextCannotReopen"
],
"migrationCLI": true,
"frameworkSum": "h1:lB4rU4FPsoFr3IedKx/K9xvvsGBO5SO/qdjnjKJ2mvQ=",
"sourceHashes": {
"cmd/go-omni/api.go": "d313526cf9f67f6b1a36260748f2c21bf53c19a01c41f640386cad08bb5538b1",
"cmd/go-omni/api_test.go": "172e4e2ba4984820ba729f88273b1ae133415081344e51c3654f297ef33216d4",
"cmd/go-omni/apigen/apigen_test.go": "7cbe654e84c60f7b7f3a9631472bb66c1999ff4a9eaacaec224d26e78d08a38c",
"cmd/go-omni/apigen/compile.go": "cc801f744ed18ffd34a9cecc1b01d869ccb609448f7fcb705fbf5a7183637c25",
"cmd/go-omni/apigen/contractMetadata.go": "a7dcacde64d95eff32b0a93e7d87b60cdc7f9b4056587a77a1f20f5a9ef5231e",
"cmd/go-omni/apigen/declarations.go": "3f8073fb0b0bfa775f7712ed28e670f8dc98571636656726e18a6d592692bc81",
"cmd/go-omni/apigen/generate.go": "93c028e31bd59b50e57ab59826366a2adbc621cccda545bde065116a40d8d905",
"cmd/go-omni/apigen/metadata_test.go": "ab0c55e90e9b7a8bdaf260415e4c8564f6b2378de625fd9379a2b801cb4bc0e5",
"cmd/go-omni/apigen/parse.go": "73576fca857a94c21ffb3636ffb318297f10f6c41f67949ce37c0b9415bbb78d",
"cmd/go-omni/apigen/profile.go": "877939398384c44316e5e052d92c4a93fb3b74a6742df2414a0054913f3fca27",
"cmd/go-omni/apigen/profile_test.go": "5f3c091da241fe0f8b99ac19e9efd2c3177aea72e63a346f99fecc0f548aad1a",
"cmd/go-omni/apigen/projectEntry.go": "817936d391ec7dc76f7b5b9f7d134537947ea238d2cd281eb3df536511cb5486",
"cmd/go-omni/apigen/response.go": "2448a09472ebe87c57c2dd48ccae1691e94f8cd029fcae72a85490a0405907e3",
"cmd/go-omni/apigen/response_test.go": "bdaea21a94297607172ab173f20640aa5c27e2d7e417376e9fe695d4c02ba524",
"cmd/go-omni/apigen/templates.go": "f5ed976109c2a227329635644774c24cc8f34caede86dc0dabb7295845352803",
"cmd/go-omni/apigen/write.go": "a995ffd412a98620dd9db5244433abb3a44a20ae7f09513e710ab6d4c52a78eb",
"cmd/go-omni/main.go": "060b064973f2316b6f01c7b9a716d57c3dbc8ad33a5d6153f6cfddaf649cd5fc",
"cmd/go-omni/main_test.go": "b8040d2b19a1c86672fe1b1be7fa1bf7f885aece79ded6ffa0cc670887b7136f",
"cmd/go-omni/migrate.go": "f1ce89fbe419205eb82172da2b90e8888905d2b85d1a9db2a72b568bcf849a70",
"cmd/go-omni/migrate_test.go": "15d9a4aaebf9f899e523e908f1fa3c2282eb1e6c8cadbefae6b68f1ae4e00e6e",
"cmd/go-omni/scaffold/addModule.go": "8f43bd68c2b3820cf1d036c4f2c5f9d06d1c7c8e5114f5081597d905f8d8cdcc",
"cmd/go-omni/scaffold/addModule_test.go": "667c45312f628f0fb5b8206ddef7ccc02a75f33b72f73fd6a17b03da0cb62e91",
"cmd/go-omni/scaffold/apiInfrastructure.go": "f493364448254cae2d8e658bf5c69e96a3264c2bf3fa50fd48a5ebdb20edabd8",
"cmd/go-omni/scaffold/moduleProfiles_test.go": "ae9790e5d3549d32fc72739a10a31ea858e5a8f8189cf1f5c1840bf80b3f8ad6",
"cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl": "9d3bfc46cac5618bcff7d8a6b2cc9f123e818aeb3dee9bcdfe0f91db584994bd",
"cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl": "729cd0f6faba53d5d2c1492935e12ac2adcefde4490748e25c18be7ccb3f7139",
"cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl": "a005611f6b455d4d8469e316bc26fa571043b2da1834db2b3dd14a96d66221d7",
"cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl": "fb0be7fd571ba2c0edb01d70d68cdbbfe53561caab98346bf1ac7cd8df07205a",
"cmd/go-omni/scaffold/moduleTemplates/repository-repository.go.tmpl": "902da3d9f12395fb226178c4428ff547e1c2fce3261882dba98d1400d8c90747",
"cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl": "49d26044a69a43e983aa5ba4b5b2399b26b77898b23e230751ad1372c2af3178",
"cmd/go-omni/scaffold/scaffold.go": "71b3091b83c3e45ac1981164e8c819c997ad96607d535f1492333370f4872548",
"cmd/go-omni/scaffold/scaffold_test.go": "58aab9251b01e394bbc90aeb538ff035dd32c73c0d4815132ac6cbc086527cda",
"cmd/go-omni/scaffold/templates/README.md.tmpl": "8ec215775fcafd437480022307fffd2ffe43685bc2b5d328b0c1d58402986e0a",
"cmd/go-omni/scaffold/templates/api-context.go.tmpl": "d2d67c3d235c4b1c965364f40f49c441d92764ad7ab8fd6848f56b1028e93b1b",
"cmd/go-omni/scaffold/templates/cmd-server-main.go.tmpl": "07c4f81109ece43fcf15bb807ada5ae6dc7e732babc7293269514a21cd389955",
"cmd/go-omni/scaffold/templates/databaseIntegration_test.go.tmpl": "81c4f7fbc919d74b738d3a7338248cbd83cfa0803b2058e8ee184e3ce51e7e9b",
"cmd/go-omni/scaffold/templates/go.mod.tmpl": "7d3bb15d2292e36939cc1905258d2800aeaf0e37d7d041b2ff8cfdb6fc84c57a",
"cmd/go-omni/scaffold/templates/internal-handler-user.go.tmpl": "7f8dbcd2f9b0129a90160d6059e47d19d4758274060a1a3e5160055d0592d34f",
"cmd/go-omni/scaffold/templates/internal-model-errors.go.tmpl": "f123eace52952321b0665a92a2923514c52c8c3bf13df1065710e9675750f837",
"cmd/go-omni/scaffold/templates/internal-model-user.go.tmpl": "b391143e7386e169b53e7fed25de1146e8c52f3fa287b12728e8d22b33f386de",
"cmd/go-omni/scaffold/templates/internal-repository-memory.go.tmpl": "148abc273c1fab3f28536706792762ae41e8687dfe9986c374372f162600feac",
"cmd/go-omni/scaffold/templates/internal-repository-mysql.go.tmpl": "34bd333e29211a0025ff6ebaea18c7f142f18f1cc451d692f9d360e8c8f6f635",
"cmd/go-omni/scaffold/templates/internal-repository-users.go.tmpl": "6d7dd80f34a9c215daeb4a86bfbe11e845e7ff7e9864c8af43e1c2ec3a8b28ec",
"cmd/go-omni/scaffold/templates/internal-service-user.go.tmpl": "33fdee3a2c3d98e7ca6a323b5e888a73582f94e327b6c24a34c09bf180db7b3d",
"cmd/go-omni/scaffold/templates/internal-servicecontext-context.go.tmpl": "de70ecc4194db8a2da5d65297f452a40b453652cb814cd228ce03d465761c856",
"cmd/go-omni/scaffold/templates/modules-README.md.tmpl": "4b84fbc2310b856d86c6fcd9a47d97d2261019c74ea8e0cdbc41229086aa3527",
"cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl": "b8734080ae6d533a175bcad946ce2cb55d8bac1729918291753b9ca03064910f",
"cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl": "426867fbef5b99a85394fa4457dd38b9a194adf4bdfb293384b2fbbc799091da",
"cmd/go-omni/scaffold/templates/modules-filesHandler_test.go.tmpl": "9571e39bd0a330739eb1fd7e636cd58961b41b44772b2b49162e41deda98547f",
"cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl": "6460ee9a873af45ca6cec779bf53302acfce21499610425a4704c2f179eddb8f",
"cmd/go-omni/scaffold/templates/modules-gitignore.tmpl": "2579f2a63ff2962239b92dd841b0d0d26e783d85c9d0100011936c8bcbf5d0d3",
"cmd/go-omni/scaffold/templates/modules-main.go.tmpl": "cb8daddd4e12afd238540be7022187c74a548d9e409b3a7f1eaf84a9139218bf",
"cmd/go-omni/scaffold/templates/modules-settings.go.tmpl": "91084caaa1fd994dde714f52b242d23e41adb731383b12d74399247c87ac5597",
"cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl": "76e55f8954b7606d7146eb1e965065c521a2b81234a47742ed69a93aac300406",
"cmd/go-omni/scaffold/templates/modules-storage.go.tmpl": "a775acf2d2490288a8e5482bb0a8769be6b0a8c9cfcc9bb10b62aabbd05fa0c7",
"cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl": "dacca42d228bd9352948054eb2d071829a8a61d8c6fa24d95ff00fc39a74309e",
"cmd/go-omni/scaffold/templates/schema.sql.tmpl": "7c5108f18fc7d8ff75c97afd96fcaa9f8ed026c79c71c5612f55572d73354abb",
"cmd/go-omni/scaffold/version.go": "1d1bd1fc0be25d7ce479587d13a2ffc978e879eb36f1c23a6e58be831b776a2d",
"database/configOverlay_test.go": "01c9b44fded226494ca12f7d0f41a9ff417d8135d263a5833727c1c028b6b8e3",
"database/database.go": "3d8615e1bf58ecdae21558591e745c241e499ba6e85fc07acfa2344e04ce95db",
"database/database_test.go": "18eac1e4c6dbd6a09216168839ba3511aac8a38c2ddf6fdd5267f35435b09f67",
"database/lifecycle_test.go": "e141b52edfb9867184dedac0e79feed56a5b137acbe64280ad69647653bbea21",
"database/migrate/migrate.go": "fd6cd27d2dce5b35ffe80eb5442c47c657b93ffc6545bbcc1ee4651a70a4f80b",
"database/migrate/migrate_test.go": "d7ba91eed8cfd97e4ef0741d582f493dd411b65ad3192f05b167f75d9b9dcf90",
"examples/userService/api/user.api": "ed4d45cb796d9a017af9d341076553ca576918faef155aeea528718515582c95",
"examples/userService/cmd/server/main.go": "80ba4721f677947b7f2ece21d5f3e89ee0c4d2cc653412d83aef2b3a45bd3ccb",
"examples/userService/etc/config.yaml": "f50df3cddbc3c23f233405e7a9d8895b9d83d2a0607ccf001261fcd4b8978ee9",
"examples/userService/internal/apierrors/errorsGen.go": "20e94bbc90ae9f4330de4c25b9a550026c4c352502719a643ede3e8912a6566e",
"examples/userService/internal/handler/openapiGen.go": "e4130f5c7494f7db4830bfef203fa8154a9cd087c95954f64c29d74693ead433",
"examples/userService/internal/handler/responseGen.go": "d742efc2a3d1d168fae7849e905297bd440c3d6b3802b6150f494c07e6097f4f",
"examples/userService/internal/handler/routesGen.go": "0ff429a8aa7f8643e0e844f7c43a63a1f82cca5ce6bfe917f06edd860ebe923f",
"examples/userService/internal/handler/user/handlersGen.go": "06f3c2798ff323812ff3eb2d8b933a569af3ce1b86a67b66036374add98c0d7e",
"examples/userService/internal/middleware/auth.go": "fbfe8ff8b12f698218b9ef5f77fd85b612b3aed362a73dc26c3627f2d688c040",
"examples/userService/internal/middleware/auth_test.go": "63d6117dadf43b766e0aa485a63c4725a5109ab814c2b4a1fa7a59ee6d1c4cf6",
"examples/userService/internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
"examples/userService/internal/model/errors.go": "f123eace52952321b0665a92a2923514c52c8c3bf13df1065710e9675750f837",
"examples/userService/internal/model/user.go": "b391143e7386e169b53e7fed25de1146e8c52f3fa287b12728e8d22b33f386de",
"examples/userService/internal/repository/user/mysql.go": "924cae233a68e506e9b0b953529e0ccb13a0e71d24723309383c464e1852b833",
"examples/userService/internal/repository/user/repository.go": "2aca07788d8d59831b10ac665882ca8157b20f384115bb90742f235f8352a69d",
"examples/userService/internal/repository/user/store.go": "f776832e4f3b26fb889dad53b65742833ddb7f43b7dddc72915ad3fe47ec4d3e",
"examples/userService/internal/repository/user/transaction_test.go": "9af4cb3a03b71026b80aa5078ddda4e3c4f72a7ab14bed6c87603a4db2cc2a22",
"examples/userService/internal/service/user/createUser.go": "53d0ea7371e10a0bd3f32f57b9bff9b48b5f9a3315218f95a258fa45b81d9368",
"examples/userService/internal/service/user/deleteUser.go": "0cd0b038d5a11429de438311980ef19a8f6e28ea0d7cf5ebaaf78858daf3a124",
"examples/userService/internal/service/user/errors_test.go": "c0193bdc674cbe15c4871a91e288323a99193d19ba8fd874f9bf17c390917e5c",
"examples/userService/internal/service/user/exportUser.go": "dac2adb0c9ea43a4af15511ae488afaa6d9c9b351e68e9b8a54f4014d29de14d",
"examples/userService/internal/service/user/getUser.go": "a307fa716c70286a2a6c5d5c0a6c35ec6d258c3278e2a7fe86b74b324d7b7283",
"examples/userService/internal/service/user/listUsers.go": "f0bf8b9c3a4c4a699112e3493b7fe611e2dd4f6e8a4437106396c47d484f1383",
"examples/userService/internal/service/user/service.go": "cd6c203459fccbc030e88eccfa785d5db029e45784f8b242be68ea1c9394a736",
"examples/userService/internal/service/user/updateUser.go": "9b0e3e4296c0e0df98956961a928612d9203c0fa09aa71bb0683e430e1133ff0",
"examples/userService/internal/servicecontext/context.go": "5527ca07049f5c5f5deffa65e5759494b2e0a77804b629b3ecb1705e7557ee25",
"examples/userService/internal/servicecontext/context_test.go": "04d008b6cf5b7894752bc5222abb8f4120f1ebe060034ac55ec9663c1f7bc613",
"examples/userService/internal/servicecontext/storage.go": "9402e1ee7a258b2db4b6dbf38fcb0674ad8907fb2a948d47dd8374f78bec444d",
"examples/userService/internal/settings/config.go": "721c3744a61a2d2807edcdfa190336901368966a0fb952561d0e00da5730cd3a",
"examples/userService/internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876",
"examples/userService/migrations/000001_users.down.sql": "64b47d3d2644745a2ba1197b73760e63fe0f2c1d12c193888c4417b3d5abb75e",
"examples/userService/migrations/000001_users.up.sql": "c6ea5cb92ee53fe20e43373d084ff360d49921b785602145e55380e4adbee1dc"
}
}

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env python3
"""Install the released CLI in a fresh cache, then test its generated consumer.
The fixture tests custom scalars, dynamic JSON, uploads, raw callback signatures,
downloads, body limits and regeneration protection. Only local temporary resources
are used; the framework and CLI are fetched from the specified remote version.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import subprocess
import tempfile
import time
ROOT = Path(__file__).resolve().parents[1]
MODULE = "git.yuntuo.hk/k2safe/go-omni"
def run(args, cwd, env, timeout=300):
result = subprocess.run([str(arg) for arg in args], cwd=cwd, env=env,
capture_output=True, text=True, timeout=timeout)
output = re.sub(r"https://[^/@\s]+:[^/@\s]+@", "https://", result.stdout + result.stderr)
if result.returncode:
raise RuntimeError(f"{args[0]} failed ({result.returncode}):\n{output}")
return output
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument("--version", required=True)
parser.add_argument("--report", type=Path, default=ROOT / "performance/compatibilityRemote.json")
args = parser.parse_args()
if not re.fullmatch(r"v\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?", args.version):
parser.error("a fixed release version is required")
env = {k: v for k, v in os.environ.items() if not k.startswith("GO_OMNI_")}
env.update(GOWORK="off", GOFLAGS="", GOPRIVATE="git.yuntuo.hk", GONOSUMDB="git.yuntuo.hk")
start = time.monotonic()
with tempfile.TemporaryDirectory(prefix="goOmniCompatibilityRelease-") as directory:
temp = Path(directory)
(temp / "bin").mkdir()
env.update(GOBIN=str(temp / "bin"), GOMODCACHE=str(temp / "modcache"))
print(f"Installing CLI {args.version} in an empty module cache", flush=True)
run(["go", "install", MODULE + "/cmd/go-omni@" + args.version], temp, env)
cli = temp / "bin/go-omni"
run([cli, "api", "validate", "--api", ROOT / "examples/contracts/compatibility.api"], temp, env)
env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version)
print("Testing remote generation, scalar binding, uploads and callback signatures", flush=True)
output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=180s",
"./cmd/go-omni/apigen", "-run", "^TestCompatibilityConsumer$"], ROOT, env)
events = [json.loads(line) for line in output.splitlines() if line.startswith("{")]
passed = [e["Test"] for e in events if e.get("Action") == "pass" and "Test" in e]
assert passed == ["TestCompatibilityConsumer"], output
evidence = "".join(e.get("Output", "") for e in events)
checksum = re.search(r"checksum (h1:\S+); no local replace", evidence)
assert checksum, "consumer did not verify its remote dependency"
report = {
"passed": True, "frameworkVersion": args.version, "frameworkSum": checksum.group(1),
"freshModuleCache": True, "noLocalReplace": True,
"durationSeconds": round(time.monotonic() - start, 2),
"checks": ["remoteCLIInstall", "scalarContractValidation", "manualProjectGeneration",
"int64MaxValueOnWire", "dynamicJSON", "rawBodySignature", "routeBodyLimits",
"multipartTempCleanup", "unwrappedDownloads",
"handwrittenFilesPreserved", "generatedConsumerRaceTests", "fixedRemoteModuleChecksum"],
"sourceHashes": {name: hashlib.sha256((ROOT / name).read_bytes()).hexdigest() for name in
("scripts/verifyCompatibilityRelease.py", "cmd/go-omni/apigen/compatibility_test.go",
"examples/contracts/compatibility.api",
"cmd/go-omni/apigen/testdata/compatibility/consumer_test.go",
"cmd/go-omni/apigen/testdata/compatibility/hooks.go",
"cmd/go-omni/apigen/testdata/compatibility/echo.go",
"cmd/go-omni/apigen/testdata/compatibility/download.go")},
}
args.report.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
print(f"PASS remote compatibility contract {args.version}; temporary files removed", flush=True)
if __name__ == "__main__":
main()

133
scripts/verifyConnector.py Normal file
View File

@@ -0,0 +1,133 @@
#!/usr/bin/env python3
"""Verify Connector TLS/isolation and HTTP limits from a local or remote module.
Uses a disposable local MySQL container (1 CPU, 768 MiB), private test credentials,
and generated test certificates. Never connects to a project database.
"""
import argparse
import hashlib
import json
import os
from pathlib import Path
import re
import secrets
import shutil
import subprocess
import tempfile
import time
import uuid
ROOT = Path(__file__).resolve().parents[1]
MODULE = "git.yuntuo.hk/k2safe/go-omni"
SOURCES = ("database/connector_test.go", "database/mysqlConnector_test.go",
"app/httpConfig_test.go", "config/http_test.go")
def run(args, cwd, env, timeout=300):
result = subprocess.run([str(a) for a in args], cwd=cwd, env=env,
text=True, capture_output=True, timeout=timeout)
output = re.sub(r"https://[^/@\s]+:[^/@\s]+@", "https://", result.stdout + result.stderr)
# Never print generated database credentials, including from a failed command.
for key in ("MYSQL_ROOT_PASSWORD", "MYSQL_PASSWORD", "GO_OMNI_CONNECTOR_PASSWORD"):
if env.get(key):
output = output.replace(env[key], "[redacted]")
if result.returncode:
raise RuntimeError(f"{args[0]} failed ({result.returncode}): {output}")
return output
def main():
parser = argparse.ArgumentParser(description=__doc__)
source = parser.add_mutually_exclusive_group(required=True)
source.add_argument("--local", action="store_true")
source.add_argument("--version")
parser.add_argument("--report", type=Path, required=True)
parser.add_argument("--image", default="mysql:8.4")
args = parser.parse_args()
if args.version and not re.fullmatch(r"v\d+\.\d+\.\d+(?:-[0-9A-Za-z.-]+)?", args.version):
parser.error("a fixed release version is required")
env = {k: v for k, v in os.environ.items() if not k.startswith("GO_OMNI_")}
env.update(GOWORK="off", GOFLAGS="")
for key in ("GOPRIVATE", "GONOSUMDB"):
env[key] = ",".join(filter(None, [env.get(key), "git.yuntuo.hk"]))
began = time.monotonic()
with tempfile.TemporaryDirectory(prefix="goOmniConnector-") as directory:
temp = Path(directory)
endpoint = env.get("DOCKER_HOST") or run(["docker", "context", "inspect", "--format", "{{.Endpoints.docker.Host}}"], temp, env).strip()
if not endpoint.startswith(("unix://", "npipe://")):
raise RuntimeError("requires local Docker; remote servers are not used")
certs = temp / "certs"
certs.mkdir(mode=0o755)
run(["openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-keyout", certs / "ca.key", "-out", certs / "ca.pem", "-days", "2", "-subj", "/CN=GoOmni Test CA"], temp, env)
run(["openssl", "req", "-newkey", "rsa:2048", "-nodes", "-keyout", certs / "server.key", "-out", temp / "server.csr", "-subj", "/CN=localhost"], temp, env)
(temp / "extensions.cnf").write_text("subjectAltName=DNS:localhost,IP:127.0.0.1\nbasicConstraints=CA:FALSE\nextendedKeyUsage=serverAuth\n")
run(["openssl", "x509", "-req", "-in", temp / "server.csr", "-CA", certs / "ca.pem", "-CAkey", certs / "ca.key", "-CAcreateserial", "-out", certs / "server.pem", "-days", "2", "-extfile", temp / "extensions.cnf"], temp, env)
(certs / "ca.key").unlink()
for path in certs.iterdir():
path.chmod(0o644) # Disposable fixtures, readable by the container's mysql uid.
project = temp / "consumer"
project.mkdir()
version = args.version or "v0.9.1-preview"
(project / "go.mod").write_text(f"module example.com/connectorAcceptance\n\ngo 1.22.0\n\nrequire {MODULE} {version}\n")
if args.local:
run(["go", "mod", "edit", f"-replace={MODULE}={ROOT}"], project, env)
else:
env["GOMODCACHE"] = str(temp / "modcache")
for name in SOURCES:
target = project / name
target.parent.mkdir(exist_ok=True)
shutil.copyfile(ROOT / name, target)
print("Resolving consumer dependency", flush=True)
run(["go", "mod", "tidy"], project, env)
module = json.loads(run(["go", "list", "-m", "-json", MODULE], project, env))
if not args.local:
assert module["Version"] == args.version and module.get("Sum") and not module.get("Replace"), module
env.update(MYSQL_ROOT_PASSWORD=secrets.token_hex(24), MYSQL_PASSWORD=secrets.token_hex(24))
name = "go-omni-connector-" + uuid.uuid4().hex[:12]
created = False
try:
print("Starting isolated MySQL TLS fixture (1 CPU, 768 MiB)", flush=True)
run(["docker", "create", "--name", name, "--label", "go-omni.test=connector", "--cpus", "1", "--memory", "768m", "--pids-limit", "128", "--publish", "127.0.0.1::3306", "--mount", f"type=bind,src={certs},dst=/certs,readonly", "--env", "MYSQL_ROOT_PASSWORD", "--env", "MYSQL_PASSWORD", "--env", "MYSQL_DATABASE=omni_test", "--env", "MYSQL_USER=omni_test", args.image, "--ssl-ca=/certs/ca.pem", "--ssl-cert=/certs/server.pem", "--ssl-key=/certs/server.key", "--require-secure-transport=ON", "--innodb-buffer-pool-size=64M", "--max-connections=20"], temp, env)
created = True
run(["docker", "start", name], temp, env)
ready = False
for _ in range(60):
try:
check_env = dict(env, MYSQL_PWD=env["MYSQL_PASSWORD"])
result = run(["docker", "exec", "--env", "MYSQL_PWD", name, "mysql", "--protocol=tcp", "-h127.0.0.1", "-uomni_test", "--batch", "--skip-column-names", "-e", "SELECT 1"], temp, check_env, timeout=5)
if result.strip() == "1":
ready = True
break
except (RuntimeError, subprocess.TimeoutExpired):
pass
time.sleep(1)
if not ready:
raise RuntimeError("isolated MySQL fixture did not become ready")
addr = run(["docker", "port", name, "3306/tcp"], temp, env).strip()
assert addr.startswith("127.0.0.1:"), addr
env.update(GO_OMNI_CONNECTOR_MYSQL_TEST="1", GO_OMNI_CONNECTOR_ADDR=addr,
GO_OMNI_CONNECTOR_USER="omni_test", GO_OMNI_CONNECTOR_PASSWORD=env["MYSQL_PASSWORD"],
GO_OMNI_CONNECTOR_DATABASE="omni_test", GO_OMNI_CONNECTOR_CA_FILE=str(certs / "ca.pem"))
print("Testing TLS trust, per-connector options, driver logging, lifecycle and HTTP limits", flush=True)
output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=60s", "./..."], project, env)
events = [json.loads(line) for line in output.splitlines() if line.startswith("{")]
passed = [e["Test"] for e in events if e.get("Action") == "pass" and "Test" in e]
assert "TestMySQLConnectorTLSAndPoolIsolation" in passed
assert not any(e.get("Action") in ("fail", "skip") for e in events), output
image = run(["docker", "inspect", "--format", "{{.Image}}", name], temp, env).strip()
finally:
if created:
run(["docker", "rm", "--force", "--volumes", name], temp, env)
report = {"passed": True, "frameworkVersion": version, "localSource": args.local,
"frameworkSum": module.get("Sum"), "freshModuleCache": not args.local,
"noLocalReplace": not bool(module.get("Replace")), "mysqlImage": image,
"mysqlLimits": {"cpus": 1, "memoryMiB": 768, "pids": 128},
"durationSeconds": round(time.monotonic() - began, 2), "tests": passed,
"sourceHashes": {name: hashlib.sha256((ROOT / name).read_bytes()).hexdigest() for name in (*SOURCES, "scripts/verifyConnector.py")}}
args.report.parent.mkdir(parents=True, exist_ok=True)
args.report.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n")
print("PASS; isolated container, credentials, certificates and consumer removed", flush=True)
if __name__ == "__main__":
main()

View File

@@ -14,7 +14,7 @@ registerRoutes(application) // 与 omni.New() 复用同一份业务注册代码
`transport/nethttp` 是当前内置的标准库路由实现。它提供一套可移植的路径、方法和错误回退规则;自研网络层可以直接实现同一个 `httpx.Router` 契约。
`transport/http2` 是独立的 opt-in 实验 Runner,负责连接接入、TLS/ALPN、连接上限和全局请求背压,底层帧解析与 stream 流控暂由 `golang.org/x/net/http2` 提供。它不替换 `transport/native`,也不改变默认启动器;只有通过 `App.RunWith(ctx, http2.New(nil), address)` 显式启用才会运行。
`transport/http2` 是独立的 opt-in 实验 Runner,负责连接接入、TLS/ALPN、连接上限和全局请求背压,底层帧解析与 stream 流控暂由 `golang.org/x/net/http2` 提供。它不替换 `transport/native`,也不改变默认启动器;通过 `WithTransport(TransportHTTP2)` 或 `App.RunWith(ctx, http2.New(nil), address)` 显式启用后运行。
`transport/native` 是第一版自研 HTTP/1.1 服务端:它自己管理 TCP listener、连接、Keep-Alive、请求解析、响应写回和优雅关闭,再把请求交给现有 `http.Handler`。请求体支持 `Content-Length` 和 chunked,并统一受 `MaxBodyBytes` 限制;其他传输编码会被拒绝。它仍使用标准 `http.Request` / `http.ResponseWriter` 作为业务桥接,因此不会影响现有 Handler / Service。
@@ -119,3 +119,37 @@ python3 scripts/verifyTransports.py --version v0.6.0-preview
第一条验证标准库适配器复用完整 Handler / Service / Repository。第二条在源码目录外创建无 replace 的消费模块,从远程版本导入同一套验收包;自研适配器也应使用同一个公共测试包。
当前替换边界分成两层:`transport/nethttp` 提供标准库路由,`transport/native` 进一步接管 TCP 监听、连接管理和 HTTP/1.1 协议 I/O。`App.Run` / `App.RunContext` 仍使用标准库 `net/http.Server`,需要显式调用 `App.RunWith(ctx, native.New(nil), address)` 才会启用 native 预览实现。native 当前通过标准 `http.Request` / `http.ResponseWriter` 桥接到业务层,后续可在不改 Handler / Service 的前提下逐步替换解析器和响应写回。
## HTTP 配置(v0.9.1-preview)
`omni.WithHTTPConfig` / `config.Config.HTTP` 提供服务端配置入口,作用于 `Run`、`RunContext`、`ServeContext`;也作用于 `NewServer` 创建的标准库服务器。与客户端 `httpc` 的出站请求超时无关。
```go
application := omni.New(omni.WithHTTPConfig(omni.HTTPConfig{
ReadHeaderTimeout: 5 * time.Second,
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
IdleTimeout: 60 * time.Second,
MaxHeaderBytes: 1 << 20,
// ErrorLog: existingLogger, // *log.Logger,仅 nethttp;不通过 YAML/JSON 序列化
}))
// 注册业务路由后:
return application.RunContext(ctx, ":8080")
```
| 字段 | nethttp | native | 独立实验 http2 |
| --- | --- | --- | --- |
| `ReadHeaderTimeout` | 标准 http.Server 语义 | 请求头读取时限 | 正值拒绝启动 |
| `ReadTimeout` | 整个请求读取时限,包括头与 body | 头解析完成后读取 body 的时限 | 正值拒绝启动 |
| `WriteTimeout` | 标准 http.Server 写截止时间 | 每次底层响应写入/Flush 的截止时间 | 正值拒绝启动 |
| `IdleTimeout` | Keep-Alive 等待下一请求 | 复用连接等待下一请求(含头读取);首个请求读头也取它与读头时限的较小值 | HTTP/2 空闲连接超时 |
| `MaxHeaderBytes` | 标准 http.Server 头限制语义 | native 请求头字节限制 | 解压后请求头限制 |
| `ErrorLog` | 标准库内部错误日志,不是访问日志 | 非 nil 拒绝启动 | 非 nil 拒绝启动 |
参数为正时覆盖适配器默认值;**零值保留默认行为,不统一表示禁用**。nethttp 未配置时遵循 Go 标准服务器零值,包括读头/空闲时限回退到 ReadTimeout 的规则;native 仍默认读头 5s、读 body 30s、写 30s、空闲 60s、头 1 MiB。负数在启动前报错;HTTP/2 不支持的字段也会明确报错,不会悄悄忽略。默认 nethttp 的 TLS/HTTP2 行为仍由标准库管理,表格最后一列专指 `TransportHTTP2`。
I/O 超时不会强制终止 CPU 计算或数据库调用;业务和下游调用继续通过 `context` 控制时限。native 需要流式输出时显式 `Flush()`,否则保持原有缓冲响应方式。不要依赖不同网络层在慢请求时返回完全相同的 HTTP 错误体;有些超时会直接断开连接。
YAML/JSON 字段与上述字段对应的 lowerCamelCase 一致,位于 `config.Config` 的 `http` 对象。脚手架将框架配置包在 `server` 下,因此生成项目填写 `server.http`;直接用 `config.LoadFile` 加载框架配置时填写根级 `http`。环境变量为 `GO_OMNI_HTTP_READ_HEADER_TIMEOUT`、`GO_OMNI_HTTP_READ_TIMEOUT`、`GO_OMNI_HTTP_WRITE_TIMEOUT`、`GO_OMNI_HTTP_IDLE_TIMEOUT`(如 `30s`),以及 `GO_OMNI_HTTP_MAX_HEADER_BYTES`(整数)。`omni.New` 不自动加载环境;先 `config.LoadFile` / `config.ApplyEnv`,再 `WithConfig`。Go-only 的 `ErrorLog` 在文件加载之后设置。
`NewServer` 始终返回普通 `*http.Server`,应用 HTTP 字段但不负责加载 TLS、监听、退出或资源清理;非法 HTTP 值会 panic。`RunContext` / `ServeContext` 的非法配置则作为 error 返回并关闭 App 资源。手动修改返回服务器的字段仍然有效。项目自己的 Runner 通过 `RunWith` 接入时,配置由该 Runner 持有,这组 HTTP 配置不会覆盖它。